Skip to content

Commit 7b58af9

Browse files
SACHINsAchin-680
authored andcommitted
feat(mdatagen): add automatic feature gate documentation generation
Implements automatic documentation generation for component feature gates via mdatagen, addressing issue #14067. Changes: - Extended metadata-schema.yaml with feature_gates section - Added FeatureGate types and validation to metadata.go - Created documentation template for feature gates - Integrated into documentation generation pipeline - Added test data and test cases - Updated README with usage examples All tests passing (282 tests) and lint checks clean. Signed-off-by: SACHIN <[email protected]>
1 parent 09a2719 commit 7b58af9

File tree

14 files changed

+270
-32
lines changed

14 files changed

+270
-32
lines changed

cmd/mdatagen/README.md

Lines changed: 53 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,25 @@
11
# Metadata Generator
22

33
<!-- status autogenerated section -->
4-
| Status | |
5-
| ------------- |-----------|
6-
| Stability | [alpha]: metrics |
7-
| Issues | [![Open issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector?query=is%3Aissue%20is%3Aopen%20label%3Acmd%2Fmdatagen%20&label=open&color=orange&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector/issues?q=is%3Aopen+is%3Aissue+label%3Acmd%2Fmdatagen) [![Closed issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector?query=is%3Aissue%20is%3Aclosed%20label%3Acmd%2Fmdatagen%20&label=closed&color=blue&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector/issues?q=is%3Aclosed+is%3Aissue+label%3Acmd%2Fmdatagen) |
8-
| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner) | [@dmitryax](https://www.github.com/dmitryax) |
4+
5+
| Status | |
6+
| -------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
7+
| Stability | [alpha]: metrics |
8+
| Issues | [![Open issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector?query=is%3Aissue%20is%3Aopen%20label%3Acmd%2Fmdatagen%20&label=open&color=orange&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector/issues?q=is%3Aopen+is%3Aissue+label%3Acmd%2Fmdatagen) [![Closed issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector?query=is%3Aissue%20is%3Aclosed%20label%3Acmd%2Fmdatagen%20&label=closed&color=blue&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector/issues?q=is%3Aclosed+is%3Aissue+label%3Acmd%2Fmdatagen) |
9+
| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner) | [@dmitryax](https://www.github.com/dmitryax) |
910

1011
[alpha]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/component-stability.md#alpha
12+
1113
<!-- end autogenerated section -->
1214

1315
Every component's documentation should include a brief description of the component and guidance on how to use it.
1416
There is also some information about the component (or metadata) that should be included to help end-users understand the current state of the component and whether it is right for their use case.
1517
Examples of this metadata about a component are:
1618

17-
* its stability level
18-
* the distributions containing it
19-
* the types of pipelines it supports
20-
* metrics emitted in the case of a scraping receiver, a scraper, or a connector
19+
- its stability level
20+
- the distributions containing it
21+
- the types of pipelines it supports
22+
- metrics emitted in the case of a scraping receiver, a scraper, or a connector
2123

2224
The metadata generator defines a schema for specifying this information to ensure it is complete and well-formed.
2325
The metadata generator is then able to ingest the metadata, validate it against the schema and produce documentation in a standardized format.
@@ -26,10 +28,12 @@ An example of how this generated documentation looks can be found in [documentat
2628
## Using the Metadata Generator
2729

2830
In order for a component to benefit from the metadata generator (`mdatagen`) these requirements need to be met:
31+
2932
1. A yaml file containing the metadata that needs to be included in the component
3033
2. The component should declare a `go:generate mdatagen` directive which tells `mdatagen` what to generate
3134

3235
As an example, here is a minimal `metadata.yaml` for the [OTLP receiver](https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver/otlpreceiver):
36+
3337
```yaml
3438
type: otlp
3539
status:
@@ -42,6 +46,7 @@ status:
4246
Detailed information about the schema of `metadata.yaml` can be found in [metadata-schema.yaml](./metadata-schema.yaml).
4347

4448
The `go:generate mdatagen` directive is usually defined in a `doc.go` file in the same package as the component, for example:
49+
4550
```go
4651
//go:generate mdatagen metadata.yaml
4752
@@ -50,11 +55,48 @@ package main
5055

5156
Below are some more examples that can be used for reference:
5257

53-
* The ElasticSearch receiver has an extensive [metadata.yaml](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/elasticsearchreceiver/metadata.yaml)
54-
* The host metrics receiver has internal subcomponents, each with their own `metadata.yaml` and `doc.go`. See [cpuscraper](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/hostmetricsreceiver/internal/scraper/cpuscraper) for example.
58+
- The ElasticSearch receiver has an extensive [metadata.yaml](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/elasticsearchreceiver/metadata.yaml)
59+
- The host metrics receiver has internal subcomponents, each with their own `metadata.yaml` and `doc.go`. See [cpuscraper](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/hostmetricsreceiver/internal/scraper/cpuscraper) for example.
5560

5661
You can run `cd cmd/mdatagen && $(GOCMD) install .` to install the `mdatagen` tool in `GOBIN` and then run `mdatagen metadata.yaml` to generate documentation for a specific component or you can run `make generate` to generate documentation for all components.
5762

63+
### Feature Gates Documentation
64+
65+
The metadata generator supports automatic documentation generation for feature gates used by components. Feature gates are documented by adding a `feature_gates` section to your `metadata.yaml`:
66+
67+
```yaml
68+
type: mycomponent
69+
status:
70+
class: receiver
71+
stability:
72+
beta: [metrics, traces]
73+
74+
feature_gates:
75+
mycomponent.newFeature:
76+
description: 'Enables new feature functionality that improves performance'
77+
stage: alpha
78+
from_version: 'v0.100.0'
79+
reference_url: 'https://github.com/open-telemetry/opentelemetry-collector/issues/12345'
80+
81+
mycomponent.stableFeature:
82+
description: 'A feature that has reached stability'
83+
stage: stable
84+
from_version: 'v0.90.0'
85+
to_version: 'v0.95.0'
86+
reference_url: 'https://github.com/open-telemetry/opentelemetry-collector/issues/11111'
87+
```
88+
89+
This will generate a "Feature Gates" section in the component's `documentation.md` file with a table containing:
90+
91+
- **Feature Gate**: The gate identifier
92+
- **Stage**: The lifecycle stage (alpha, beta, stable, deprecated)
93+
- **Description**: Brief description of what the gate controls
94+
- **From Version**: Version when the gate was introduced
95+
- **To Version**: Version when stable/deprecated gates will be removed (if applicable)
96+
- **Reference**: Link to additional contextual information
97+
98+
The feature gate definitions should correspond to actual gates registered in your component code using the [Feature Gates API](../../featuregate/README.md).
99+
58100
### Generate multiple metadata packages
59101

60102
By default, `mdatagen` will generate a package called `metadata` in the `internal` directory. If you want to generate a package with a different name, you can use the `generated_package_name` configuration field to provide an alternate name.

cmd/mdatagen/internal/command.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -172,7 +172,7 @@ func run(ymlPath string) error {
172172
}
173173
}
174174

175-
if len(md.Metrics) != 0 || len(md.Telemetry.Metrics) != 0 || len(md.ResourceAttributes) != 0 || len(md.Events) != 0 { // if there's metrics or internal metrics or events, generate documentation for them
175+
if len(md.Metrics) != 0 || len(md.Telemetry.Metrics) != 0 || len(md.ResourceAttributes) != 0 || len(md.Events) != 0 || len(md.FeatureGates) != 0 { // if there's metrics or internal metrics or events or feature gates, generate documentation for them
176176
toGenerate[filepath.Join(tmplDir, "documentation.md.tmpl")] = filepath.Join(ymlDir, "documentation.md")
177177
}
178178

cmd/mdatagen/internal/command_test.go

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ func TestRunContents(t *testing.T) {
4747
wantGoleakSkip bool
4848
wantGoleakSetup bool
4949
wantGoleakTeardown bool
50+
wantDocumentationGenerated bool
5051
wantErr bool
5152
wantOrderErr bool
5253
wantAttributes []string
@@ -192,6 +193,13 @@ func TestRunContents(t *testing.T) {
192193
wantComponentTestGenerated: true,
193194
wantLogsGenerated: true,
194195
},
196+
{
197+
yml: "feature_gates.yaml",
198+
wantStatusGenerated: true,
199+
wantReadmeGenerated: true,
200+
wantComponentTestGenerated: true,
201+
wantDocumentationGenerated: true,
202+
},
195203
{
196204
yml: "with_conditional_attribute.yaml",
197205
wantStatusGenerated: true,
@@ -301,7 +309,9 @@ foo
301309
require.NoFileExists(t, filepath.Join(tmpdir, generatedPackageDir, "generated_telemetry.go"))
302310
}
303311

304-
if !tt.wantMetricsGenerated && !tt.wantTelemetryGenerated && !tt.wantResourceAttributesGenerated && !tt.wantEventsGenerated {
312+
if tt.wantDocumentationGenerated {
313+
require.FileExists(t, filepath.Join(tmpdir, "documentation.md"))
314+
} else if !tt.wantMetricsGenerated && !tt.wantTelemetryGenerated && !tt.wantResourceAttributesGenerated && !tt.wantEventsGenerated {
305315
require.NoFileExists(t, filepath.Join(tmpdir, "documentation.md"))
306316
}
307317

cmd/mdatagen/internal/embedded_templates_test.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ func TestEnsureTemplatesLoaded(t *testing.T) {
4040
path.Join(rootDir, "telemetrytest.go.tmpl"): {},
4141
path.Join(rootDir, "telemetrytest_test.go.tmpl"): {},
4242
path.Join(rootDir, "helper.tmpl"): {},
43+
path.Join(rootDir, "feature_gates.md.tmpl"): {},
4344
}
4445
count = 0
4546
)

cmd/mdatagen/internal/metadata.go

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,8 @@ type Metadata struct {
4848
Tests Tests `mapstructure:"tests"`
4949
// PackageName is the name of the package where the component is defined.
5050
PackageName string `mapstructure:"package_name"`
51+
// FeatureGates that are managed by the component.
52+
FeatureGates map[FeatureGateID]FeatureGate `mapstructure:"feature_gates"`
5153
}
5254

5355
func (md Metadata) GetCodeCovComponentID() string {
@@ -89,6 +91,10 @@ func (md *Metadata) Validate() error {
8991
errs = errors.Join(errs, err)
9092
}
9193

94+
if err := md.validateFeatureGates(); err != nil {
95+
errs = errors.Join(errs, err)
96+
}
97+
9298
return errs
9399
}
94100

@@ -270,6 +276,47 @@ func validateEvents(events map[EventName]Event, attributes map[AttributeName]Att
270276
return errs
271277
}
272278

279+
func (md *Metadata) validateFeatureGates() error {
280+
var errs error
281+
for gateID, gate := range md.FeatureGates {
282+
// Validate gate ID is not empty
283+
if string(gateID) == "" {
284+
errs = errors.Join(errs, errors.New("feature gate ID cannot be empty"))
285+
continue
286+
}
287+
288+
// Validate gate has required fields
289+
if gate.Description == "" {
290+
errs = errors.Join(errs, fmt.Errorf(`feature gate "%v": description is required`, gateID))
291+
}
292+
293+
// Validate stage is one of the allowed values
294+
validStages := map[FeatureGateStage]bool{
295+
FeatureGateStageAlpha: true,
296+
FeatureGateStageBeta: true,
297+
FeatureGateStageStable: true,
298+
FeatureGateStageDeprecated: true,
299+
}
300+
if !validStages[gate.Stage] {
301+
errs = errors.Join(errs, fmt.Errorf(`feature gate "%v": invalid stage "%v", must be one of: alpha, beta, stable, deprecated`, gateID, gate.Stage))
302+
}
303+
304+
// Validate version formats if provided
305+
if gate.FromVersion != "" && !strings.HasPrefix(gate.FromVersion, "v") {
306+
errs = errors.Join(errs, fmt.Errorf(`feature gate "%v": from_version "%v" must start with 'v'`, gateID, gate.FromVersion))
307+
}
308+
if gate.ToVersion != "" && !strings.HasPrefix(gate.ToVersion, "v") {
309+
errs = errors.Join(errs, fmt.Errorf(`feature gate "%v": to_version "%v" must start with 'v'`, gateID, gate.ToVersion))
310+
}
311+
312+
// Validate that stable/deprecated gates should have to_version
313+
if (gate.Stage == FeatureGateStageStable || gate.Stage == FeatureGateStageDeprecated) && gate.ToVersion == "" {
314+
errs = errors.Join(errs, fmt.Errorf(`feature gate "%v": to_version is required for %v stage gates`, gateID, gate.Stage))
315+
}
316+
}
317+
return errs
318+
}
319+
273320
type AttributeName string
274321

275322
// AttributeRequirementLevel defines the requirement level of an attribute.
@@ -514,3 +561,30 @@ type EntityAttributeRef struct {
514561
// Ref is the reference to a resource attribute.
515562
Ref AttributeName `mapstructure:"ref"`
516563
}
564+
565+
// FeatureGateID represents the identifier for a feature gate.
566+
type FeatureGateID string
567+
568+
// FeatureGateStage represents the lifecycle stage of a feature gate.
569+
type FeatureGateStage string
570+
571+
const (
572+
FeatureGateStageAlpha FeatureGateStage = "alpha"
573+
FeatureGateStageBeta FeatureGateStage = "beta"
574+
FeatureGateStageStable FeatureGateStage = "stable"
575+
FeatureGateStageDeprecated FeatureGateStage = "deprecated"
576+
)
577+
578+
// FeatureGate represents a feature gate definition in metadata.
579+
type FeatureGate struct {
580+
// Description of the feature gate.
581+
Description string `mapstructure:"description"`
582+
// Stage is the lifecycle stage of the feature gate.
583+
Stage FeatureGateStage `mapstructure:"stage"`
584+
// FromVersion is the version when the feature gate was introduced.
585+
FromVersion string `mapstructure:"from_version"`
586+
// ToVersion is the version when the feature gate reached stable stage.
587+
ToVersion string `mapstructure:"to_version"`
588+
// ReferenceURL is the URL with contextual information about the feature gate.
589+
ReferenceURL string `mapstructure:"reference_url"`
590+
}

cmd/mdatagen/internal/templates/documentation.md.tmpl

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -264,3 +264,19 @@ The following telemetry is emitted by this component.
264264
{{- end }}
265265

266266
{{- end }}
267+
268+
{{- if .FeatureGates }}
269+
270+
## Feature Gates
271+
272+
This component has the following feature gates:
273+
274+
| Feature Gate | Stage | Description | From Version | To Version | Reference |
275+
| ------------ | ----- | ----------- | ------------ | ---------- | --------- |
276+
{{- range $gateID, $gate := .FeatureGates }}
277+
| `{{ $gateID }}` | {{ $gate.Stage }} | {{ $gate.Description }} | {{ if $gate.FromVersion }}{{ $gate.FromVersion }}{{ else }}N/A{{ end }} | {{ if $gate.ToVersion }}{{ $gate.ToVersion }}{{ else }}N/A{{ end }} | {{ if $gate.ReferenceURL }}[Link]({{ $gate.ReferenceURL }}){{ else }}N/A{{ end }} |
278+
{{- end }}
279+
280+
For more information about feature gates, see the [Feature Gates](https://github.com/open-telemetry/opentelemetry-collector/blob/main/featuregate/README.md) documentation.
281+
282+
{{- end }}
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{{- if len .FeatureGates }}
2+
3+
## Feature Gates
4+
5+
This component has the following feature gates:
6+
7+
| Feature Gate | Stage | Description | From Version | To Version | Reference |
8+
| ------------ | ----- | ----------- | ------------ | ---------- | --------- |
9+
{{- range $gateID, $gate := .FeatureGates }}
10+
| `{{ $gateID }}` | {{ $gate.Stage }} | {{ $gate.Description }} | {{ if $gate.FromVersion }}{{ $gate.FromVersion }}{{ else }}N/A{{ end }} | {{ if $gate.ToVersion }}{{ $gate.ToVersion }}{{ else }}N/A{{ end }} | {{ if $gate.ReferenceURL }}[Link]({{ $gate.ReferenceURL }}){{ else }}N/A{{ end }} |
11+
{{- end }}
12+
13+
For more information about feature gates, see the [Feature Gates](https://github.com/open-telemetry/opentelemetry-collector/blob/main/featuregate/README.md) documentation.
14+
15+
{{- end }}

cmd/mdatagen/internal/testdata/documentation.md

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,13 @@
22

33
# sample
44

5-
## Resource Attributes
5+
## Feature Gates
66

7-
| Name | Description | Values | Enabled |
8-
| ---- | ----------- | ------ | ------- |
9-
| host.id | The unique host identifier | Any Str | true |
10-
| host.name | The hostname | Any Str | true |
11-
| process.pid | The process identifier | Any Int | true |
7+
This component has the following feature gates:
8+
9+
| Feature Gate | Stage | Description | From Version | To Version | Reference |
10+
| ------------ | ----- | ----------- | ------------ | ---------- | --------- |
11+
| `sample.feature.gate` | alpha | This is a sample feature gate for testing purposes | v0.100.0 | N/A | [Link](https://github.com/open-telemetry/opentelemetry-collector/issues/12345) |
12+
| `stable.feature.gate` | stable | This is a stable feature gate | v0.90.0 | v0.95.0 | [Link](https://github.com/open-telemetry/opentelemetry-collector/issues/11111) |
13+
14+
For more information about feature gates, see the [Feature Gates](https://github.com/open-telemetry/opentelemetry-collector/blob/main/featuregate/README.md) documentation.
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
type: sample
2+
status:
3+
class: receiver
4+
stability:
5+
beta: [metrics]
6+
7+
feature_gates:
8+
sample.feature.gate:
9+
description: 'This is a sample feature gate for testing purposes'
10+
stage: alpha
11+
from_version: 'v0.100.0'
12+
reference_url: 'https://github.com/open-telemetry/opentelemetry-collector/issues/12345'
13+
14+
stable.feature.gate:
15+
description: 'This is a stable feature gate'
16+
stage: stable
17+
from_version: 'v0.90.0'
18+
to_version: 'v0.95.0'
19+
reference_url: 'https://github.com/open-telemetry/opentelemetry-collector/issues/11111'

cmd/mdatagen/internal/testdata/generated_component_test.go

Lines changed: 0 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)