Extend the statistics APIs to include package downloads

[tellison]

The Adoptium API currently provides statistics about the total downloads and daily download tracking.

The total downloads API response currently includes numbers from GitHub and Docker image pulls. This feature request is to extend the download numbers to also include download numbers from packages.adoptium.net.

The method for tracking the packages downloads made from packages.adoptium.net is best performed via counting successful retrievals. Historically this was difficult to determine since some packages are served from the JFrog backing store, some from the Fastly CDN cache, and some from the CloudFlare cache. We can now get the sum of these download numbers from CloudFlare since all requests come through the packages.adoptium.net DNS.

CloudFlare provides a GraphQL API to get the required information. An example is provided here. Note that the duration of statistics that can be requested is bounded by CloudFlare, and the CloudFlare query API is rate limited, so we will likely need some local storage to maintain the statistics over time.

The proposal is to enhance the API that provides the download stats to include packages, such as this mock-up:

{
    "date": "2024-11-17T20:09:53Z",
    "docker_pulls": {
        "eclipse-temurin": 166950108
    },
    "github_downloads": {
        "8": 80187095,
        "11": 88479693,
        "16": 7887657,
        "17": 100492055,
        "18": 15539729,
        "19": 11686259,
        "20": 6619229,
        "21": 25364811,
        "22": 5133776,
        "23": 2303376,
        "24": 402000
    },
    "package_downloads": {
        "8": 123456,
        "11": 123456,
        "21": 123456
    },
    "total_downloads": {
        "docker_pulls": 166950108,
        "github_downloads": 344095680,
        "package_downloads": 123456,
        "total": 511045788
    }
}

We can discuss whether the tracking API should also be enhanced to include package downloads in the daily totals it reports.