Adapt nginx log based metrics to GCP v2 environment

[data-sync-user]

The GCP v1 env have a couple Log Based Metrics setup for interpreting metrics from nginx from its logs.

The v2 move has moved nginx logs into BigQuery, so these old metrics are broken. We may need to read them from the BigQuery datasource directly instead of utilizing GCP’s Log Based Metrics.

┆Issue is synchronized with this Jira Task

[data-sync-user]

➤ Taddes Korris commented:

webservices-infra terraform to export to grafana here https://github.com/mozilla/webservices-infra/pull/6493 ( https://github.com/mozilla/webservices-infra/pull/6493|smart-link )

Can confirm that we get this data in BigQuery, queries below with example link to dashboard:

https://console.cloud.google.com/bigquery?inv=1&invt=Ab2gTw&project=moz-fx-sync-prod&ws=!1m10!1m4!1m3!1smoz-fx-sync-prod!2sbquxjob_4707ae13_197fbc4affb!3sUS!1m4!4m3!1smoz-fx-sync-prod!2sgke_sync_prod_log!3sstdout ( https://console.cloud.google.com/bigquery?inv=1&invt=Ab2gTw&project=moz-fx-sync-prod&ws=!1m10!1m4!1m3!1smoz-fx-sync-prod!2sbquxjob_4707ae13_197fbc4affb!3sUS!1m4!4m3!1smoz-fx-sync-prod!2sgke_sync_prod_log!3sstdout )

SELECT * FROM moz-fx-sync-prod.gke_sync_prod_log.stdout WHERE TIMESTAMP_TRUNC(_PARTITIONTIME, DAY) = TIMESTAMP("2025-07-11") and resource.labels.container_name="nginx" LIMIT 1000

SELECT jsonPayload.request_time FROM moz-fx-sync-prod.gke_sync_prod_log.stdout WHERE TIMESTAMP_TRUNC(_PARTITIONTIME, DAY) = TIMESTAMP("2025-07-11") and resource.labels.container_name="nginx" LIMIT 1000

SELECT jsonPayload.upstream_time FROM moz-fx-sync-prod.gke_sync_prod_log.stdout WHERE TIMESTAMP_TRUNC(_PARTITIONTIME, DAY) = TIMESTAMP("2025-07-11") and resource.labels.container_name="nginx" LIMIT 1000Verifying that export to Grafana is functional and stable.

[data-sync-user]

➤ Taddes Korris commented:

Noting this ticket here https://mozilla-hub.atlassian.net/browse/SREIN-204 ( https://mozilla-hub.atlassian.net/browse/SREIN-204|smart-link ) , as we don’t have edit access to the Log-Based Metrics panel and it may be that we need additional Terraform configuration like in Bouncer: Ex 1 ( https://github.com/mozilla/webservices-infra/blob/main/bouncer/tf/stage/metrics.tf )& Ex 2 ( https://github.com/search?q=repo%3Amozilla%2Fwebservices-infra%20google_logging_metric&type=code ). Given it’s not well documented and we don’t have edit access, getting some additional SRE assistance.