Skip to content

feat: Backend for SDK metrics #5623

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 20 commits into
base: main
Choose a base branch
from
Open

feat: Backend for SDK metrics #5623

wants to merge 20 commits into from

Conversation

khvn26
Copy link
Member

@khvn26 khvn26 commented Jun 19, 2025

Thanks for submitting a PR! Please check the boxes below:

  • I have added information to docs/ if required so people know about the feature!
  • I have filled in the "Changes" section below?
  • I have filled in the "How did you test this code" section below?
  • I have used a Conventional Commit title for this Pull Request

Changes

Closes #5581.

Contributes to #2479.

This PR adds support for optional header tracking in the backend. It adds extensible, fully typed mechanism to support other optional labels for analytics in the future. The usage data and feature evaluation data routes are updated to return labeled data buckets. Additionally, numerous typing and code structure improvements are introduced to the analytics-related code base.

SDK API

All tracked SDK APIs now expect the following optional headers, translated to labels written to PostgreSQL or InfluxDB usage and evaluation data buckets:

  • Flagsmith-Application-Name header: client_application_name label
  • Flagsmith-Application-Version header: client_application_version label
  • User-Agent header: user_agent label

Relevant caches and tasks are changed to properly compact the raw data, producing buckets with unique label sets.

Management API

The following API routes are updated to support labels:

  • /api/v1/organisations/{organisation_pk}/usage-data/
  • /api/v1/projects/{project_pk}/features/{id}/evaluation-data/

Previously, these routes were returning one bucket per day. Now, multiple buckets can be attributed to the same day, as long as they have different unique label sets. The label sets are returned under the label key for each bucket:

// GET /api/v1/projects/1/features/19/evaluation-data/

[
    {
        "day": "2025-07-09",
        "count": 674,
        "labels": {
            "client_application_name": null,
            "client_application_version": null,
            "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36"
        }
    },
    {
        "day": "2025-07-09",
        "count": 62,
        "labels": {
            "client_application_name": null,
            "client_application_version": null,
            "user_agent": "python-requests/2.32.3"
        }
    }
]

Both of them now accept new query parameters:

  • client_application_name
  • client_application_version
  • user_agent

to allow bucket filtering by label.

The feature is hidden behind the "sdk_metrics_labels" flag.

Task processing

Due to incompatible signature change, the track_feature_evaluation is decommissioned in favour of the new track_feature_evaluations_by_environment task, which now supports labels included as part of feature evaluation data.

How did you test this code?

Unit tests have been added and updated to reflect new behaviour. Manually with local PostgreSQL storage. Additional tests will be conducted in staging against InfluxDB Cloud to measure the performance.

Copy link

vercel bot commented Jun 19, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

3 Skipped Deployments
Name Status Preview Comments Updated (UTC)
docs ⬜️ Ignored (Inspect) Visit Preview Jul 10, 2025 10:27pm
flagsmith-frontend-preview ⬜️ Ignored (Inspect) Visit Preview Jul 10, 2025 10:27pm
flagsmith-frontend-staging ⬜️ Ignored (Inspect) Visit Preview Jul 10, 2025 10:27pm

@github-actions github-actions bot added api Issue related to the REST API feature New feature or request and removed feature New feature or request labels Jun 19, 2025
@khvn26 khvn26 force-pushed the feat/client-app-telemetry branch from a36fe70 to 2b706ee Compare June 19, 2025 10:02
@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jun 19, 2025
@khvn26 khvn26 force-pushed the feat/client-app-telemetry branch from 2b706ee to 88498ed Compare June 19, 2025 10:26
@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jun 19, 2025
@khvn26 khvn26 force-pushed the feat/client-app-telemetry branch from 88498ed to 421522b Compare June 19, 2025 11:01
@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jun 19, 2025
@khvn26 khvn26 force-pushed the feat/client-app-telemetry branch from 421522b to 8664928 Compare June 19, 2025 11:16
@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jun 19, 2025
Copy link

codecov bot commented Jun 19, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.82%. Comparing base (d688c09) to head (34de05e).
Report is 8 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5623      +/-   ##
==========================================
+ Coverage   97.77%   97.82%   +0.04%     
==========================================
  Files        1255     1257       +2     
  Lines       44351    44523     +172     
==========================================
+ Hits        43365    43553     +188     
+ Misses        986      970      -16     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@khvn26 khvn26 force-pushed the feat/client-app-telemetry branch from 8664928 to 2a67741 Compare June 19, 2025 14:41
@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jun 19, 2025
@khvn26 khvn26 force-pushed the feat/client-app-telemetry branch from 2a67741 to 02c72e2 Compare June 19, 2025 16:00
@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jun 19, 2025
@khvn26 khvn26 changed the title feat: Backend for client app telementry feat: Backend for client app telemetry Jun 19, 2025
@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jun 19, 2025
@khvn26 khvn26 marked this pull request as ready for review June 19, 2025 17:31
@khvn26 khvn26 force-pushed the feat/client-app-telemetry branch from 4e7697a to 0d8d43f Compare July 9, 2025 16:24
@emyller emyller dismissed their stale review July 9, 2025 16:24

Not enough context to validate approval.

@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jul 9, 2025
@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jul 9, 2025
@khvn26
Copy link
Member Author

khvn26 commented Jul 9, 2025

Added "sdk_metrics_labels" flag, so this can be merged after approval.

CC @matthewelwell @emyller

@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jul 9, 2025
matthewelwell
matthewelwell previously approved these changes Jul 10, 2025
@@ -79,6 +79,7 @@
"django.contrib.contenttypes",
"django.contrib.sessions",
"django.contrib.messages",
"django.contrib.postgres",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might cause an issue in EE repo for oracle?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have to have settings/common.py identical to the open source one?


from . import constants
from .types import PERIOD_TYPE
logger = getLogger(__name__)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to me like we should standardise on one approach, and we can always filter events out that we don't care about on the other end, right?

@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jul 10, 2025
Copy link
Member Author

@khvn26 khvn26 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to me like we should standardise on one approach, and we can always filter events out that we don't care about on the other end, right?

Switched to structlog in 34de05e.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api Issue related to the REST API feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement API for tracking SDK metrics
4 participants