Integration gcp detectors firebase (#578)

* create detectors for firebase database * create detectors for firebase database * fix conf detector * update read me for detector firebase database * fix typo * Update modules/integration_gcp-firebase/conf/00-heatbeat.yaml Co-authored-by: Jean-Baptiste Simillon <[email protected]> * fix configuration of detectors * fix configuration of detectors * Update modules/integration_gcp-firebase/conf/01-database_load.yaml Co-authored-by: Jean-Baptiste Simillon <[email protected]> * Update modules/integration_gcp-firebase/conf/01-database_load.yaml Co-authored-by: Jean-Baptiste Simillon <[email protected]> * Update modules/integration_gcp-firebase/conf/02-io_utilization.yaml Co-authored-by: Jean-Baptiste Simillon <[email protected]> * Update modules/integration_gcp-firebase/conf/02-io_utilization.yaml Co-authored-by: Jean-Baptiste Simillon <[email protected]> * fix typo --------- Co-authored-by: Jean-Baptiste Simillon <[email protected]>
claranet · Dec 17, 2024 · ca6dcf5 · ca6dcf5
1 parent eca7084
commit ca6dcf5
Show file tree

Hide file tree

Showing 16 changed files with 650 additions and 0 deletions.
diff --git a/docs/severity.md b/docs/severity.md
@@ -73,6 +73,7 @@
 - [integration_gcp-cloud-sql-mysql](#integration_gcp-cloud-sql-mysql)
 - [integration_gcp-cloud-sql-postgresql](#integration_gcp-cloud-sql-postgresql)
 - [integration_gcp-compute-engine](#integration_gcp-compute-engine)
+- [integration_gcp-firebase](#integration_gcp-firebase)
 - [integration_gcp-load-balancing](#integration_gcp-load-balancing)
 - [integration_gcp-memorystore-redis](#integration_gcp-memorystore-redis)
 - [integration_gcp-pubsub-subscription](#integration_gcp-pubsub-subscription)
@@ -805,6 +806,15 @@
 |GCP GCE Instance disk throttled ops|X|X|-|-|-|
 
 
+## integration_gcp-firebase
+
+|Detector|Critical|Major|Minor|Warning|Info|
+|---|---|---|---|---|---|
+|GCP Firebase database heartbeat|X|-|-|-|-|
+|GCP Firebase database load|X|X|-|-|-|
+|GCP Firebase database io utilization|X|X|-|-|-|
+
+
 ## integration_gcp-load-balancing
 
 |Detector|Critical|Major|Minor|Warning|Info|

diff --git a/modules/integration_gcp-firebase/README.md b/modules/integration_gcp-firebase/README.md
@@ -0,0 +1,169 @@
+# GCP-FIREBASE SignalFx detectors
+
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
+:link: **Contents**
+
+- [How to use this module?](#how-to-use-this-module)
+- [What are the available detectors in this module?](#what-are-the-available-detectors-in-this-module)
+- [How to collect required metrics?](#how-to-collect-required-metrics)
+  - [Metrics](#metrics)
+- [Notes](#notes)
+  - [Metadata configuration for default filtering](#metadata-configuration-for-default-filtering)
+  - [Database load](#database-load)
+  - [Database IO](#database-io)
+- [Related documentation](#related-documentation)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+
+## How to use this module?
+
+This directory defines a [Terraform](https://www.terraform.io/)
+[module](https://www.terraform.io/language/modules/syntax) you can use in your
+existing [stack](https://github.com/claranet/terraform-signalfx-detectors/wiki/Getting-started#stack) by adding a
+`module` configuration and setting its `source` parameter to URL of this folder:
+
+```hcl
+module "signalfx-detectors-integration-gcp-firebase" {
+  source = "github.com/claranet/terraform-signalfx-detectors.git//modules/integration_gcp-firebase?ref={revision}"
+
+  environment    = var.environment
+  notifications  = local.notifications
+  gcp_project_id = "fillme"
+}
+```
+
+Note the following parameters:
+
+* `source`: Use this parameter to specify the URL of the module. The double slash (`//`) is intentional  and required.
+  Terraform uses it to specify subfolders within a Git repo (see [module
+  sources](https://www.terraform.io/language/modules/sources)). The `ref` parameter specifies a specific Git tag in
+  this repository. It is recommended to use the latest "pinned" version in place of `{revision}`. Avoid using a branch
+  like `master` except for testing purpose. Note that every modules in this repository are available on the Terraform
+  [registry](https://registry.terraform.io/modules/claranet/detectors/signalfx) and we recommend using it as source
+  instead of `git` which is more flexible but less future-proof.
+
+* `environment`: Use this parameter to specify the
+  [environment](https://github.com/claranet/terraform-signalfx-detectors/wiki/Getting-started#environment) used by this
+  instance of the module.
+  Its value will be added to the `prefixes` list at the start of the [detector
+  name](https://github.com/claranet/terraform-signalfx-detectors/wiki/Templating#example).
+  In general, it will also be used in the `filtering` internal sub-module to [apply
+  filters](https://github.com/claranet/terraform-signalfx-detectors/wiki/Guidance#filtering) based on our default
+  [tagging convention](https://github.com/claranet/terraform-signalfx-detectors/wiki/Tagging-convention) by default.
+
+* `notifications`: Use this parameter to define where alerts should be sent depending on their severity. It consists
+  of a Terraform [object](https://www.terraform.io/language/expressions/type-constraints#object) where each key represents an available
+  [detector rule severity](https://docs.splunk.com/observability/alerts-detectors-notifications/create-detectors-for-alerts.html#severity)
+  and its value is a list of recipients. Every recipients must respect the [detector notification
+  format](https://registry.terraform.io/providers/splunk-terraform/signalfx/latest/docs/resources/detector#notification-format).
+  Check the [notification binding](https://github.com/claranet/terraform-signalfx-detectors/wiki/Notifications-binding)
+  documentation to understand the recommended role of each severity.
+
+These 3 parameters along with all variables defined in [common-variables.tf](common-variables.tf) are common to all
+[modules](../) in this repository. Other variables, specific to this module, are available in
+[variables.tf](variables.tf) and [variables-gen.tf](variables-gen.tf).
+In general, the default configuration "works" but all of these Terraform
+[variables](https://www.terraform.io/language/values/variables) make it possible to
+customize the detectors behavior to better fit your needs.
+
+Most of them represent usual tips and rules detailed in the
+[guidance](https://github.com/claranet/terraform-signalfx-detectors/wiki/Guidance) documentation and listed in the
+common [variables](https://github.com/claranet/terraform-signalfx-detectors/wiki/Variables) dedicated documentation.
+
+Feel free to explore the [wiki](https://github.com/claranet/terraform-signalfx-detectors/wiki) for more information about
+general usage of this repository.
+
+## What are the available detectors in this module?
+
+This module creates the following SignalFx detectors which could contain one or multiple alerting rules:
+
+|Detector|Critical|Major|Minor|Warning|Info|
+|---|---|---|---|---|---|
+|GCP Firebase database heartbeat|X|-|-|-|-|
+|GCP Firebase database load|X|X|-|-|-|
+|GCP Firebase database io utilization|X|X|-|-|-|
+
+## How to collect required metrics?
+
+This module deploys detectors using metrics reported by the
+[GCP integration](https://docs.splunk.com/observability/en/gdi/get-data-in/connect/gcp/gcp-metrics.html) configurable
+with [this Terraform module](https://github.com/claranet/terraform-signalfx-integrations/tree/master/cloud/gcp).
+
+
+Check the [Related documentation](#related-documentation) section for more detailed and specific information about this module dependencies.
+
+
+
+### Metrics
+
+
+Here is the list of required metrics for detectors in this module.
+
+* `io/database_load`
+* `io/utilization`
+* `network/active_connections`
+
+
+## Notes
+
+
+### Metadata configuration for default filtering
+
+label to use : 
+
+sfx_env = true
+sfx_monitored = true
+
+### Database load
+
+Monitoring the CPU utilization helps in understanding the system's capability and efficiency.
+
+```hcl
+module "signalfx-detectors-integration_gcp-cloud-run" {
+  source = "github.com/claranet/terraform-signalfx-detectors.git//modules/integration_gcp-firebase"
+
+  environment    = var.environment
+  gcp_project_id = var.project_id
+  notifications  = local.notifications
+
+  # We keep default filtering policy here, we just want to append additional filter to it
+  filtering_append = true
+  # We define the additional filter
+  filtering_custom = "filter('service_name', '*service-name*')"
+  # We can configure the thresholds of the probes
+  firebase_database_load_threshold_critical = 5
+  firebase_database_load_threshold_major    = 3
+}
+```
+
+### Database IO
+
+Monitoring the IO of the database helps in understanding the system's capability and efficiency.
+
+```hcl                                                                                                                                                                                                                                                                                                              
+module "signalfx-detectors-integration_gcp-cloud-run" {                                                                                                                                                                                                                                                             
+  source = "github.com/claranet/terraform-signalfx-detectors.git//modules/integration_gcp-firebase"                                                                                                                                                                                                                 
+                                                                                                                                                                                                                                                                                                                    
+  environment    = var.environment                                                                                                                                                                                                                                                                                  
+  gcp_project_id = var.project_id                                                                                                                                                                                                                                                                                   
+  notifications  = local.notifications                                                                                                                                                                                                                                                                              
+                                                                                                                                                                                                                                                                                                                    
+  # We keep default filtering policy here, we just want to append additional filter to it                                                                                                                                                                                                                           
+  filtering_append = true                                                                                                                                                                                                                                                                                           
+  # We define the additional filter                                                                                                                                                                                                                                                                                 
+  filtering_custom = "filter('service_name', '*service-name*')"                                                                                                                                                                                                                                                     
+  # We can configure the thresholds of the probes                                                                                                                                                                                                                                                                   
+  firebase_database_io_utilization_threshold_critical = 5                                                                                                                                                                                                                                                                     
+  firebase_database_io_utilization_threshold_major    = 3                                                                                                                                                                                                                                                                     
+}                                                                                                                                                                                                                                                                                                                   
+```  
+
+
+## Related documentation
+
+* [Terraform SignalFx provider](https://registry.terraform.io/providers/splunk-terraform/signalfx/latest/docs)
+* [Terraform SignalFx detector](https://registry.terraform.io/providers/splunk-terraform/signalfx/latest/docs/resources/detector)
+* [Splunk Observability integrations](https://docs.splunk.com/Observability/gdi/get-data-in/integrations.html)
+* [Stackdriver metrics for Firebase](https://cloud.google.com/monitoring/api/metrics_gcp#gcp-firebasedatabase)
+* [Splunk Observability metrics](https://docs.splunk.com/observability/en/gdi/get-data-in/connect/gcp/gcp.html)
diff --git a/modules/integration_gcp-firebase/common-locals.tf b/modules/integration_gcp-firebase/common-locals.tf
@@ -0,0 +1 @@
+../../common/module/locals.tf
diff --git a/modules/integration_gcp-firebase/common-modules.tf b/modules/integration_gcp-firebase/common-modules.tf
@@ -0,0 +1 @@
+../../common/module/modules.tf
diff --git a/modules/integration_gcp-firebase/common-variables.tf b/modules/integration_gcp-firebase/common-variables.tf
@@ -0,0 +1 @@
+../../common/module/variables.tf
diff --git a/modules/integration_gcp-firebase/common-versions.tf b/modules/integration_gcp-firebase/common-versions.tf
@@ -0,0 +1 @@
+../../common/module/versions.tf
diff --git a/modules/integration_gcp-firebase/conf/00-heatbeat.yaml b/modules/integration_gcp-firebase/conf/00-heatbeat.yaml
@@ -0,0 +1,12 @@
+## Example
+module: GCP Firebase database
+name: heartbeat
+
+transformation: false
+aggregation: true
+
+signals:
+  signal:
+    metric: "network/active_connections"
+rules:
+  critical:
diff --git a/modules/integration_gcp-firebase/conf/01-database_load.yaml b/modules/integration_gcp-firebase/conf/01-database_load.yaml
@@ -0,0 +1,20 @@
+module: "GCP Firebase database"
+name: "Load"
+
+transformation: true
+
+signals:
+  signal:
+    metric: "io/database_load"
+
+rules:
+  critical:
+    threshold: 10
+    comparator: ">"
+    lasting_duration: '30m'
+
+  major:
+    threshold: 5
+    comparator: ">"
+    lasting_duration: '30m'
+    dependency: "critical"
diff --git a/modules/integration_gcp-firebase/conf/02-io_utilization.yaml b/modules/integration_gcp-firebase/conf/02-io_utilization.yaml
@@ -0,0 +1,20 @@
+module: "GCP Firebase database"
+name: "IO utilization"
+
+transformation: true 
+
+signals:
+  signal:
+    metric: "io/utilization"
+
+rules:
+  critical:
+    threshold: 10
+    comparator: ">"
+    lasting_duration: '30m'
+
+  major:
+    threshold: 5
+    comparator: ">"
+    lasting_duration: '30m'
+    dependency: "critical"