Skip to content

Commit

Permalink
Code review
Browse files Browse the repository at this point in the history
  • Loading branch information
@sjberman
sjberman committed Feb 29, 2024
1 parent 8a9aaaa commit 85c01e4
Show file tree
Hide file tree
Showing 2 changed files with 277 additions and 16 deletions.
22 changes: 12 additions & 10 deletions site/content/how-to/monitoring/prometheus.md
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Prometheus Metrics and Grafana"
description: "Learn how to monitor your NGINX Gateway Fabric effectively. This guide provides easy steps for configuring and understanding key performance metrics using Prometheus and Grafana."
title: "Prometheus Metrics"
description: "This document describes how to monitor NGINX Gateway Fabric using Prometheus and Grafana. It explains installation and configuration, as well as what metrics are available."
weight: 100
toc: true
docs: "DOCS-1418"
Expand All @@ -11,14 +11,16 @@ docs: "DOCS-1418"
## Overview


NGINX Gateway Fabric metrics are displayed in [Prometheus](https://prometheus.io/) format, simplifying monitoring. These metrics are served via a metrics server orchestrated by the controller-runtime package on HTTP port `9113`. When installed, Prometheus automatically scrapes this port and collects metrics. [Grafana](https://grafana.com/) can be used for rich visualization of these metrics.
NGINX Gateway Fabric metrics are displayed in [Prometheus](https://prometheus.io/) format. These metrics are served through a metrics server orchestrated by the controller-runtime package on HTTP port `9113`. When installed, Prometheus automatically scrapes this port and collects metrics. [Grafana](https://grafana.com/) can be used for rich visualization of these metrics.

{{<call-out "important" "Security note for metrics">}}
Metrics are served over HTTP by default. Enabling HTTPS will secure the metrics endpoint with a self-signed certificate. When using HTTPS, adjust the Prometheus Pod scrape settings by adding the `insecure_skip_verify` flag to handle the self-signed certificate. For further details, refer to the [Prometheus documentation](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#tls_config).
{{</call-out>}}

## Installing Prometheus and Grafana

{{< note >}}These installations are for demonstration purposes and have not been tuned for a production environment.{{< /note >}}

### Prometheus

```shell
Expand Down Expand Up @@ -64,21 +66,21 @@ In the Grafana UI menu, go to `Connections` then `Data sources`. Add your Promet

Download the following sample dashboard and Import as a new Dashboard in the Grafana UI.

{{< download "static/grafana-dashboard.json" "ngf-grafana-dashboard.json" >}}


## Available metrics in NGINX Gateway Fabric

NGINX Gateway Fabric provides a variety of metrics to assist in monitoring and analyzing performance. These metrics are categorized as follows:
NGINX Gateway Fabric provides a variety of metrics for monitoring and analyzing performance. These metrics are categorized as follows:

### NGINX/NGINX Plus metrics

NGINX metrics, essential for monitoring specific NGINX operations, include details like the total number of accepted client connections. For a complete list of available NGINX/NGINX Plus metrics, refer to the [NGINX Prometheus Exporter developer docs](https://github.com/nginxinc/nginx-prometheus-exporter#exported-metrics).
NGINX metrics cover specific NGINX operations such as the total number of accepted client connections. For a complete list of available NGINX/NGINX Plus metrics, refer to the [NGINX Prometheus Exporter developer docs](https://github.com/nginxinc/nginx-prometheus-exporter#exported-metrics).

These metrics use the `nginx_gateway_fabric` namespace and include the `class` label, indicating the NGINX Gateway class. For example, `nginx_gateway_fabric_connections_accepted{class="nginx"}`.
These metrics use the `nginx_gateway_fabric` namespace and include the `class` label, indicating the NGINX Gateway class. For example, `nginx_gateway_fabric_connections_accepted{class="nginx"}`.

### NGINX Gateway Fabric metrics

Metrics specific to the NGINX Gateway Fabric include:
Metrics specific to NGINX Gateway Fabric include:

- `nginx_reloads_total`: Counts successful NGINX reloads.
- `nginx_reload_errors_total`: Counts NGINX reload failures.
Expand All @@ -90,15 +92,15 @@ All these metrics are under the `nginx_gateway_fabric` namespace and include a `

### Controller-runtime metrics

Provided by the [controller-runtime](https://github.com/kubernetes-sigs/controller-runtime) library, these metrics cover a range of aspects:
Provided by the [controller-runtime](https://github.com/kubernetes-sigs/controller-runtime) library, these metrics include:

- General resource usage like CPU and memory.
- Go runtime metrics such as the number of Go routines, garbage collection duration, and Go version.
- Controller-specific metrics, including reconciliation errors per controller, length of the reconcile queue, and reconciliation latency.

## How to change the default metrics configuration

Configuring NGINX Gateway Fabric for monitoring is straightforward. You can change metric settings using Helm or Kubernetes manifests, depending on your setup.
You can configure monitoring metrics for NGINX Gateway Fabric using Helm or Manifests.

### Using Helm

Expand Down
271 changes: 265 additions & 6 deletions site/static/grafana-dashboard.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 0,
"id": 5,
"id": 1,
"links": [],
"liveNow": false,
"panels": [
Expand Down Expand Up @@ -129,7 +129,7 @@
"type": "stat"
},
{
"collapsed": true,
"collapsed": false,
"gridPos": {
"h": 1,
"w": 24,
Expand Down Expand Up @@ -226,7 +226,7 @@
},
"disableTextWrap": false,
"editorMode": "code",
"expr": "irate(nginx_gateway_fabric_connections_accepted{instance=~\"$instance\"}[5m])",
"expr": "irate(nginx_gateway_fabric_connections_accepted{instance=~\"$instance\"}[1m])",
"fullMetaSearch": false,
"includeNullMetadata": false,
"instant": false,
Expand All @@ -242,7 +242,7 @@
"uid": "${DS_PROMETHEUS}"
},
"editorMode": "code",
"expr": "irate(nginx_gateway_fabric_connections_handled{instance=~\"$instance\"}[5m])",
"expr": "irate(nginx_gateway_fabric_connections_handled{instance=~\"$instance\"}[1m])",
"hide": false,
"instant": false,
"legendFormat": "{{instance}} handled",
Expand Down Expand Up @@ -469,8 +469,8 @@
"uid": "${DS_PROMETHEUS}"
},
"disableTextWrap": false,
"editorMode": "builder",
"expr": "irate(nginx_gateway_fabric_http_requests_total{instance=~\"$instance\"}[5m])",
"editorMode": "code",
"expr": "irate(nginx_gateway_fabric_http_requests_total{instance=~\"$instance\"}[1m])",
"fullMetaSearch": false,
"includeNullMetadata": false,
"instant": false,
Expand All @@ -482,6 +482,265 @@
],
"title": "Total Requests",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "cb451057-d630-4956-9a0e-f19cdc9c8097"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 1,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unitScale": true
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 24
},
"id": 8,
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "10.3.3",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "cb451057-d630-4956-9a0e-f19cdc9c8097"
},
"disableTextWrap": false,
"editorMode": "code",
"expr": "irate(nginx_gateway_fabric_nginx_reloads_total{instance=~\"$instance\"}[1m])",
"fullMetaSearch": false,
"includeNullMetadata": false,
"instant": false,
"legendFormat": "{{instance}}",
"range": true,
"refId": "A",
"useBackend": false
}
],
"title": "Total NGINX Reloads Rate",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "cb451057-d630-4956-9a0e-f19cdc9c8097"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 1
}
]
},
"unitScale": true
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 6,
"x": 12,
"y": 24
},
"id": 9,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showPercentChange": false,
"textMode": "auto",
"wideLayout": true
},
"pluginVersion": "10.3.3",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "cb451057-d630-4956-9a0e-f19cdc9c8097"
},
"disableTextWrap": false,
"editorMode": "builder",
"expr": "nginx_gateway_fabric_nginx_reload_errors_total{instance=~\"$instance\"}",
"fullMetaSearch": false,
"includeNullMetadata": true,
"instant": false,
"legendFormat": "{{instance}}",
"range": true,
"refId": "A",
"useBackend": false
}
],
"title": "Total NGINX Reload Errors",
"type": "stat"
},
{
"datasource": {
"type": "prometheus",
"uid": "cb451057-d630-4956-9a0e-f19cdc9c8097"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"mappings": [
{
"options": {
"0": {
"color": "semi-dark-green",
"index": 0,
"text": "Up to date"
},
"1": {
"color": "semi-dark-red",
"index": 1,
"text": "Stale"
}
},
"type": "value"
}
],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "semi-dark-red",
"value": 1
}
]
},
"unitScale": true
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 6,
"x": 18,
"y": 24
},
"id": 10,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showPercentChange": false,
"textMode": "auto",
"wideLayout": true
},
"pluginVersion": "10.3.3",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "cb451057-d630-4956-9a0e-f19cdc9c8097"
},
"disableTextWrap": false,
"editorMode": "builder",
"expr": "nginx_gateway_fabric_nginx_stale_config{instance=~\"$instance\"}",
"fullMetaSearch": false,
"includeNullMetadata": true,
"instant": false,
"legendFormat": "__auto",
"range": true,
"refId": "A",
"useBackend": false
}
],
"title": "NGINX Config State",
"type": "stat"
}
],
"refresh": "5s",
Expand Down

0 comments on commit 85c01e4

Please sign in to comment.