Issue 3138 - Conformance Tests for BackendTLSPolicy - normative #3212

candita · 2024-07-23T20:36:20Z

What type of PR is this?

/kind test
/area conformance

What this PR does / why we need it:

Add a normative test of Gateway API BackendTLSPolicy implementations.

Which issue(s) this PR fixes:
Fixes #3138

Does this PR introduce a user-facing change?:

NONE

k8s-ci-robot · 2024-07-23T20:36:22Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

conformance/echo-basic/echo-basic.go

candita · 2024-08-13T20:18:30Z

/test pull-gateway-api-verify

candita · 2024-08-14T20:02:18Z

/test pull-gateway-api-verify

candita · 2024-08-19T17:16:41Z

/test pull-gateway-api-verify

candita · 2024-08-19T18:51:48Z

/test pull-gateway-api-verify

candita · 2024-08-19T19:09:29Z

/test pull-gateway-api-verify

candita · 2024-08-19T19:56:40Z

/test pull-gateway-api-verify

candita · 2024-08-19T20:36:48Z

/test pull-gateway-api-verify

candita · 2024-08-19T22:33:00Z

/test pull-gateway-api-verify

candita · 2024-08-19T23:16:00Z

/test pull-gateway-api-test

kl52752 · 2025-06-05T17:54:55Z

conformance/base/manifests.yaml

-        # From https://github.com/kubernetes-sigs/ingress-controller-conformance/tree/master/images/echoserver
-        image: gcr.io/k8s-staging-gateway-api/echo-basic:v20240412-v1.0.0-394-g40c666fd
+        # Originally from https://github.com/kubernetes-sigs/ingress-controller-conformance/tree/master/images/echoserver
+        image: gcr.io/k8s-staging-gateway-api/echo-basic:v20241007-v1.2.0-6-g9f820af9


nit: there is newer version of this image
gcr.io/k8s-staging-gateway-api/echo-basic:v20250605-v1.3.0-25-g77baa438

Thanks. This version will need to be updated for all once the changes for echo-basic are accepted. Then, I will split this PR, merge the echo-basic changes, and it will give me a new version to use. Then I will change all the images to use the new version.

I will not be updating echo-basic after all. We've not made any changes to echo-basic that we need to pick up, since March 2024.

kl52752 · 2025-06-05T17:56:01Z

conformance/base/manifests.yaml

@@ -346,7 +346,7 @@ spec:
    spec:
      containers:
      - name: tls-backend
-        image: gcr.io/k8s-staging-gateway-api/echo-basic:v20240412-v1.0.0-394-g40c666fd
+        image: echo-basic:2.7


why different image for this container?

This is how to test changes to echo-basic locally. I build the echo-basic image, tag it with my own version number, and load that image to my kind cluster. Then when I run this test it will pick up the local version with my changes. The versions will be fixed later, as described in #3212 (comment).

conformance/echo-basic/echo-basic.go

kl52752 · 2025-06-05T18:13:09Z

conformance/echo-basic/echo-basic.go

+	var config tls.Config
+
+	if cacert == "" {
+		return &config, fmt.Errorf("empty CA cert specified")


I think it is good practice to return nil object on error. When you return error from the function it should be a strong signal not to use this variable

Here I think I don't want to return nil in a pointer like *tls.Config. Am I wrong about that?

If you return the error then the other object which was returned should not be used and this is recommended to return nil.
If you want to cover case where client haven't has certs then I suggest to split or refactor the func (the function cannot return default config and error you can have one or another).

This file will be removed.

conformance/utils/http/http.go

conformance/utils/tls/tls.go

snorwin · 2025-06-13T07:42:21Z

conformance/utils/suite/suite.go

 		suite.Applier.MustApplyObjectsWithCleanup(t, suite.Client, suite.TimeoutConfig, []client.Object{secret}, suite.Cleanup)
 		secret = kubernetes.MustCreateSelfSignedCertSecret(t, "gateway-conformance-app-backend", "tls-passthrough-checks-certificate", []string{"abc.example.com"})
 		suite.Applier.MustApplyObjectsWithCleanup(t, suite.Client, suite.TimeoutConfig, []client.Object{secret}, suite.Cleanup)
+		caConfigMap := kubernetes.MustCreateCASignedCertConfigMap(t, "gateway-conformance-infra", "backend-tls-checks-certificate", []string{"abc.example.com"})


I think this CA certificate should be used to create the server certificate of the backend otherwise the verification will never pass.

I saw that you mean the 'tls-checks-certificate' that needs to be created with the CA cert, and I tested it and that is the only way it works. But why can't tls-checks-certificate and backend-tls-checks-certificate be separately signed certs?

The CA certificate configured in the BackendTLSPolicy is used to validate the backend’s certificate during the TLS handshake. For this validation to succeed, the backend certificate must be signed by the specified CA, or by a CA derived from it (i.e., a CA in the same trust chain).

Documentation for caCertificateRefs in BackendTLSPolicy:

CACertificateRefs contains one or more references to Kubernetes objects that contain a PEM-encoded TLS CA certificate bundle, which is used to validate a TLS handshake between the Gateway and backend Pod.

snorwin · 2025-06-17T05:20:50Z

conformance/utils/kubernetes/certificate.go

+		},
+		Data: map[string]string{
+			"ca.crt":  certData.String(),
+			"key.crt": keyData.String(),


nit: I don’t think we should store the private key of the CA certificate in the ConfigMap, as this is against best practices and isn’t necessary for certificate validation.

snorwin · 2025-06-17T05:22:26Z

conformance/utils/kubernetes/certificate.go

+
+// MustCreateCASignedCertConfigMap will create a ConfigMap containing a CA Certificate, given a TLS Secret
+// for that CA certificate.
+func MustCreateCASignedCertConfigMap(t *testing.T, namespace, configMapName string, hosts []string) *corev1.ConfigMap {


Suggested change

func MustCreateCASignedCertConfigMap(t *testing.T, namespace, configMapName string, hosts []string) *corev1.ConfigMap {

func MustCreateCACertConfigMap(t *testing.T, namespace, configMapName string, hosts []string) *corev1.ConfigMap {

Despite the name, this function creates a CA certificate, not a CA-signed certificate.

Although the suggestion may be outdated, the comment remains valid.

snorwin · 2025-06-17T05:27:38Z

conformance/utils/suite/suite.go

 		suite.Applier.MustApplyObjectsWithCleanup(t, suite.Client, suite.TimeoutConfig, []client.Object{secret}, suite.Cleanup)
 		secret = kubernetes.MustCreateSelfSignedCertSecret(t, "gateway-conformance-app-backend", "tls-passthrough-checks-certificate", []string{"abc.example.com"})
 		suite.Applier.MustApplyObjectsWithCleanup(t, suite.Client, suite.TimeoutConfig, []client.Object{secret}, suite.Cleanup)
+		caConfigMap := kubernetes.MustCreateCASignedCertConfigMap(t, "gateway-conformance-infra", "backend-tls-checks-certificate", []string{"abc.example.com"})


The CA certificate configured in the BackendTLSPolicy is used to validate the backend’s certificate during the TLS handshake. For this validation to succeed, the backend certificate must be signed by the specified CA, or by a CA derived from it (i.e., a CA in the same trust chain).

Documentation for caCertificateRefs in BackendTLSPolicy:

CACertificateRefs contains one or more references to Kubernetes objects that contain a PEM-encoded TLS CA certificate bundle, which is used to validate a TLS handshake between the Gateway and backend Pod.

BackendTLSPolicy.

…e original as well. See https://github.com/kubernetes-sigs/gateway-api/pull/2745/files#diff-cf5b4a9b433acc91f8c7cc2dc802e29aa712c8d820887742f3bc5ae45b7a9d0fR24-R28

conformance/base/manifests.yaml - fix yaml conformance/tests/backendtlspolicy.yaml - fix yaml conformance/tests/tlsroute-simple-same-namespace.go - rename cert for sharing conformance/utils/suite/conformance.go - fix a bug in cleanup-base-resources flag application conformance/utils/suite/suite.go - rename cert for sharing

call, some debugging, and fix yaml

Fix certificate unit test.

# Conflicts: # conformance/utils/http/http.go

Add conformance profiles to logged information.

Remove echo-basic changes, fix cert building, and adjust the port used for gateways with multiple listeners Co-authored-by: Norwin Schnyder <[email protected]>

k8s-ci-robot · 2025-06-19T03:01:47Z

@candita: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-gateway-api-verify	`967e2b9`	link	true	`/test pull-gateway-api-verify`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

snorwin · 2025-06-19T05:15:06Z

conformance/tests/backendtlspolicy.go

+		kubernetes.NamespacesMustBeReady(t, suite.Client, suite.TimeoutConfig, []string{ns})
+		gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAcceptedMultipleListeners(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN)
+		kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN)
+


I think we’re also missing an assertion here to verify that the BackendTLSPolicy has been accepted, with the appropriate ancestor references and statusconditions set.

snorwin · 2025-06-19T05:18:49Z

conformance/tests/backendtlspolicy.yaml

+    port: 443
+    targetPort: 8443
+---
+# Deployment must not be applied until after the secret is generated.


Same here: I think we should reuse the tls-backend. We can still create a dedicated copy of its Service for this test to avoid interference with other test cases.

snorwin · 2025-06-19T05:20:58Z

conformance/utils/kubernetes/helpers.go


+	RouteTypeMustHaveParentsField(t, routeType)
+	// If the Gateway has multiple listeners, get a portless gwAddr.
+	if !usePort {


For better readability, it might be preferable to always return the address including the port, and strip the port only in the few cases where it isn’t needed.

k8s-ci-robot requested review from michaelbeaumont and youngnick July 23, 2024 20:36

whitneygriffith reviewed Aug 1, 2024

View reviewed changes

conformance/echo-basic/echo-basic.go Outdated Show resolved Hide resolved

candita force-pushed the issue3138-BackendTLSPolicy-echoserver branch from 76c8e10 to 6d9ab9e Compare August 13, 2024 00:38

candita force-pushed the issue3138-BackendTLSPolicy-echoserver branch from 0ec34a8 to 75551a0 Compare August 19, 2024 17:12

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 19, 2024

candita force-pushed the issue3138-BackendTLSPolicy-echoserver branch from 75551a0 to 7626aaa Compare August 19, 2024 18:48

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 19, 2024

candita force-pushed the issue3138-BackendTLSPolicy-echoserver branch from 7626aaa to 1bc71f0 Compare August 19, 2024 19:09

candita force-pushed the issue3138-BackendTLSPolicy-echoserver branch from 1bc71f0 to 99e7eac Compare August 19, 2024 19:55

candita force-pushed the issue3138-BackendTLSPolicy-echoserver branch from 99e7eac to b774245 Compare August 19, 2024 20:35

candita force-pushed the issue3138-BackendTLSPolicy-echoserver branch from b774245 to 91488aa Compare August 19, 2024 22:30

kl52752 reviewed Jun 5, 2025

View reviewed changes

snorwin reviewed Jun 13, 2025

View reviewed changes

snorwin reviewed Jun 17, 2025

View reviewed changes

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 19, 2025

candita marked this pull request as draft June 19, 2025 01:35

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 19, 2025

candita force-pushed the issue3138-BackendTLSPolicy-echoserver branch 2 times, most recently from 2970c69 to efe1338 Compare June 19, 2025 02:26

candita and others added 14 commits June 18, 2025 22:33

Issue 3138 - add normative conformance test for

e77cb2e

BackendTLSPolicy.

Fix where the new go module needed by echo-basic resides, but keep th…

2d8a2e4

…e original as well. See https://github.com/kubernetes-sigs/gateway-api/pull/2745/files#diff-cf5b4a9b433acc91f8c7cc2dc802e29aa712c8d820887742f3bc5ae45b7a9d0fR24-R28

Rebase and make requested updates.

09087a5

Incorporate Shane and Flynn's feedback

b8e7515

Add unit testing for generateCACert, new HTTPS

d827fc6

call, some debugging, and fix yaml

Update echo-basic images

946aa2c

Fix lint errors and condition evaluation in tests

1c55e6f

Fix yaml for httpRoute and backendTLSPolicy. Fix CA generation.

7de9fba

Fix certificate unit test.

Refactor test, fix yaml

2e60114

Fix the tests for normative BackendTLSPolicy

e3c4df5

# Conflicts: # conformance/utils/http/http.go

Make changes from review comments.

9cf61c7

Add conformance profiles to logged information.

Address further review comments

919185f

Address review comments:

967e2b9

Remove echo-basic changes, fix cert building, and adjust the port used for gateways with multiple listeners Co-authored-by: Norwin Schnyder <[email protected]>

candita force-pushed the issue3138-BackendTLSPolicy-echoserver branch from efe1338 to 967e2b9 Compare June 19, 2025 02:46

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 19, 2025

candita marked this pull request as ready for review June 19, 2025 02:47

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 19, 2025

snorwin reviewed Jun 19, 2025

View reviewed changes

	func MustCreateCASignedCertConfigMap(t testing.T, namespace, configMapName string, hosts []string) corev1.ConfigMap {
	func MustCreateCACertConfigMap(t testing.T, namespace, configMapName string, hosts []string) corev1.ConfigMap {

Issue 3138 - Conformance Tests for BackendTLSPolicy - normative #3212

Are you sure you want to change the base?

Issue 3138 - Conformance Tests for BackendTLSPolicy - normative #3212

Conversation

candita commented Jul 23, 2024

Uh oh!

k8s-ci-robot commented Jul 23, 2024

Uh oh!

Uh oh!

candita commented Aug 13, 2024

Uh oh!

candita commented Aug 14, 2024

Uh oh!

candita commented Aug 19, 2024

Uh oh!

candita commented Aug 19, 2024

Uh oh!

candita commented Aug 19, 2024

Uh oh!

candita commented Aug 19, 2024

Uh oh!

candita commented Aug 19, 2024

Uh oh!

candita commented Aug 19, 2024

Uh oh!

candita commented Aug 19, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

candita Jun 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

k8s-ci-robot commented Jun 19, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

candita Jun 19, 2025 •

edited

Loading