app: vtgen tool

[jiekun]

Describe Your Changes

vtgen is a trace data generator. It reads the OTLP request body in testdata/testdata.bin, modifies the trace_id, start_time and end_time, and sends them to OTLP trace endpoint (/v1/traces).

vtgen can be used for:

1. Generating identical data for different targets, which is useful for query performance benchmarking of different storage backends against the same dataset.
2. Generating different data at the same rate (on a best-effort basis) for different targets, which is useful for data ingestion performance benchmarking of different storage backends.

To send identical data to different targets:

./vtgen -addrs=http://example-url1:port/v1/traces,http://example-url2:port/insert/opentelemetry/v1/traces

The performance of different targets will affect each other, as vtgen generates data and makes HTTP requests to them one by one.

To send (potentially) different data to different addresses at the same rate, simply run multiple vtgen with different HTTP listening port:

./vtgen -addrs=http://example-url1:port/v1/traces -httpListenAddr=0.0.0.0:8080
./vtgen -addrs=http://example-url2:port/insert/opentelemetry/v1/traces -httpListenAddr=0.0.0.0:8081

Checklist

The following checks are mandatory:

• My change adheres to VictoriaMetrics contributing guidelines.
• My change adheres to VictoriaMetrics development goals.

[Copilot]

Pull Request Overview

This PR introduces a new vtgen tool for generating and sending OTLP trace data by reading binary test data, adjusting trace IDs and timestamps, and exporting via HTTP.

• Adds a rate-limited, multi-worker trace generator application
• Includes a Makefile for building vtgen binaries
• Provides documentation in a new README

Reviewed Changes

Copilot reviewed 4 out of 11 changed files in this pull request and generated 3 comments.

File	Description
go.mod	Add golang.org/x/time dependency for rate limiting
app/vtgen/main.go	Implement vtgen tool: load test data, modify traces, and send
app/vtgen/README.md	Document tool usage, metrics, and flags
app/vtgen/Makefile	Add build and run targets for vtgen

Comments suppressed due to low confidence (1)

app/vtgen/main.go:173

• [nitpick] There are no unit tests for loadTestData or generateTraceID. Adding tests would help ensure binary decoding and ID generation behave as expected.

func loadTestData() [][]byte {

[Copilot]

Pull Request Overview

This PR introduces vtgen, a trace data generator tool for VictoriaTraces that reads OTLP request bodies from test data, modifies trace IDs and timestamps, and sends them to OTLP trace endpoints. The tool is designed for performance benchmarking of different storage backends by generating either identical or different trace data at configurable rates.

• Adds a new command-line tool vtgen with configurable endpoints, rates, workers, and authentication
• Implements trace ID generation and timestamp manipulation for realistic test data
• Provides HTTP metrics and pprof endpoints for monitoring performance during benchmarking

Reviewed Changes

Copilot reviewed 4 out of 11 changed files in this pull request and generated 5 comments.

File	Description
go.mod	Adds golang.org/x/time dependency for rate limiting functionality
app/vtgen/main.go	Core implementation with trace generation, HTTP client, rate limiting, and metrics
app/vtgen/README.md	Documentation covering usage, metrics, and command-line flags
app/vtgen/Makefile	Build configuration for multiple platforms and architectures

Comments suppressed due to low confidence (1)

app/vtgen/main.go:31

• Variable name 'httpListenAddrs' is misleading as it's singular but named as plural. It should be 'httpListenAddr' to match its usage and description.

	httpListenAddrs     = flag.String("httpListenAddr", "0.0.0.0:8080", "http listen address for pprof and metrics.")

[Copilot]

Using MD5 for trace ID generation is problematic due to its cryptographic weaknesses and potential for collisions. For trace IDs, consider using crypto/rand.Read() to generate random bytes, or use a UUID library for better uniqueness guarantees.

[Copilot]

The hardcoded file path should be configurable via a command-line flag or constant to improve flexibility and testability.

Suggested change

	data, err := os.ReadFile("./app/vtgen/testdata/testdata.bin")
	data, err := os.ReadFile(*testDataPath)