Require all Settings are tested

[sheidkamp]

Description

Resolves #11556

Update the settings test to ensure that all settings are tested by moving the ENV out of the test case and into a separate data structure that can be validated against the Settings struct.

When a new field is added to the Settings struct, the "all settings are tested" test will fail unless an ENV var with a non-default is added to the tests.

Also ordered fields in the test to mirror the order in the Settings struct and removed the KGW_STS_CLUSTER_NAME and KGW_STS_URI env vars which do not correspond to any Settings fields

Some code has been borrowed and modified from the kelseyhightower/envconfig package to determine what ENV vars should be set for the test.

Change Type

/kind cleanup

Changelog

NONE

Additional Notes

[Copilot]

Pull Request Overview

Adds a safeguard to ensure every field in the Settings struct is covered by a corresponding environment‐variable test by extracting the hardcoded map into allEnvVarsSet() and comparing it against a reflection‐derived list.

• Extracted the full ENV map into allEnvVarsSet() for reuse
• Added expectedEnvVars() to reflectively generate expected env-var names and defaults
• Introduced two new subtests to verify (1) every setting is tested, and (2) expectedEnvVars() behaves as intended

Reviewed Changes

Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.

File	Description
pkg/settings/settings_test.go	Moved hardcoded env var map into allEnvVarsSet(), added expectedEnvVars() helper
pkg/settings/settings_test.go	Added “all settings are tested” and “expectedEnvVars” subtests

Comments suppressed due to low confidence (2)

pkg/settings/settings_test.go:214

• [nitpick] The subtest name expectedEnvVars matches the helper function name and could be confused. Consider renaming the subtest to validateExpectedEnvVars or similar.

	t.Run("expectedEnvVars", func(t *testing.T) {

pkg/settings/settings_test.go:24

• The STS-related fields (KGW_STS_CLUSTER_NAME and KGW_STS_URI) were removed from allEnvVarsSet(), but if Settings still defines these fields, the new all settings are tested subtest will fail. Add entries for those env vars or remove the fields from Settings.

		"KGW_ISTIO_NAMESPACE":               "my-istio-namespace",

[timflannagan]

Seems reasonable at a high-level - a couple of comments on the approach.

[timflannagan]

Yeah I think maintaining a canonical map is the right approach.

[timflannagan]

I think this can be simplified quite a bit. Using reflection still makes sense to me. Sketching this out inline:

	want := map[string]struct{}{}
	st := reflect.TypeOf(settings.Settings{})
	for i := 0; i < st.NumField(); i++ {
		f := st.Field(i)
		want["KGW_"+camelToSnake(f.Name)] = struct{}{}
	}
        ...

And we define a helper function to do the camel case conversion:

func camelToSnake(s string) string {
	var b strings.Builder
	for i, r := range s {
		if unicode.IsUpper(r) && i > 0 {
			b.WriteByte('_')
		}
		b.WriteRune(unicode.ToUpper(r))
	}
	return b.String()
}

Not sure if strings.Builder is correct here, but should do the job.

[sheidkamp]

I started down a similar path, but the reason I went the complicated route was to match the logic used by envconfig.

Your suggestion works with all the current fields, but doesn't work if a field is created with an "alt" name or uses an acronym, or doesn't use "split_words:true". (If we want to standardize that you can't use an "alt" and "split_words" must be true, we could validate/enforce that as well)

Also, by parsing out the default value we can check that the test is configured properly by validating the ENV var value used in the test is not the default. WDYT?

[timflannagan]

I think we move this into it's own standalone test focused on flagging settings w/o test coverage, duplicate settings, or handle the scenario where settings that live in the allEnvVarsSet have been removed.

func TestEnvVarCoverage(t *testing.T) {
	want := map[string]struct{}{}
	st := reflect.TypeOf(settings.Settings{})
	for i := 0; i < st.NumField(); i++ {
		f := st.Field(i)
		want["KGW_"+camelToSnake(f.Name)] = struct{}{}
	}

	have := allValuesSetEnvVars()
	var missing, extra []string
	for k := range want {
		if _, ok := have[k]; !ok {
			missing = append(missing, k)
		}
	}
	for k := range have {
		if _, ok := want[k]; !ok {
			extra = append(extra, k)
		}
	}
	if len(missing) > 0 || len(extra) > 0 {
		t.Fatalf("env var coverage out of date:\nmissing: %v\nextra: %v", missing, extra)
	}
}

And the settings_test.go is now responsible for checking the defaulting logic, providing negative test cases, etc. WYDT?

[sheidkamp]

Makes sense, I will setup a separate test (contents pending response to the other comment)