Test Telemetry Generator

[jaydeluca]

As part of #13468, we are currently running a bash script in order to execute a specific list of gradle tasks with the -PcollectMetadata=true flag. This results in the generation of all the .telemetry files that we then scrape and analyze when we run the :instrumentation-docs:runAnalysis task to populate the instrumentation-list.yaml.

The reasons we need this script:

• As we work out the mechanics of the telemetry generation, it is easier to start small and only enable and run the process for a small number of modules to make reviewing the results easier.
• Running the entire test suite for every instrumentation module takes a really long time, and can be flaky on a local machine. Therefore at least for now, it makes sense to just run the modules we care about/have enabled things for.
• Some test suites don't work on ARM architectures, and need to be run using colima instead of standard docker. This script separates those instrumentations and runs them differently in order to avoid failures when generating telemetry locally.

Once we have worked through and enabled and validated the telemetry emitted by all modules, the plan is to run this process on a schedule using github actions, and much of this complexity won't be needed.

In the interim, the bash script is getting a bit heavy. It would likely be better to orchestrate these gradle tasks another way. This issue is to investigate using a solution written in Python or using another approach.

Requirements:

• Runs test suites with the -PcollectMetadata=true flag enabled
• Supports running multiple suites per module (ex: test, testStableSemconv)
• When generating telemetry, it clears out previously generated data so that the data is fresh
• Ability to run some modules using a standard docker environment, but also be able to specify modules that should be run on colima x86_64
• It would be nice to be able to specify particular modules to run the process for, instead of the entire list, since that takes a long time. As of now I have just been commenting out all modules except the ones I want to run it for, and that is cumbersome.