Consider integration tests confirming they are running against the revision expected

[janbrasna]

Description

There is currently no verification that a deployment runs the expected SHA ref.

Maybe if the integration test added a step reading the /revision.txt of the host it's run against, and notify it doesn't agree to what the trigger ref should have been?

---

Success Criteria

• Consider what overlap between pods can happen during the run of the test on subsequent deployments
• Check the ref both before start and after finish?
• Use some form of annotation in case the refs do not agree?

[stevejalim]

One thing about revision.txt is that I've noticed it's cached at the CDN (and browser) so we'll need to fix that first I think #16806

[janbrasna]

NB: For this reason CDN "public" hosts are mostly not used in CI (they are used in a few tests, e.g. CDN/TLS & download). Origin hosts show the fresh revision correctly (used in most of tests esp. functional after deploying, and places like whatsdeployed etc.)

The integration tests triggering after deployments I'm mostly interested in here re. the deployment checks for refs only hit the origins.

[stevejalim]

Yes, agreed - the integration tests deliberately run against the origins to avoid false results due to cached content, so revision.txt should be safe to check.

I like this addition as part of the tests. We should be sure, though, that whatever changes we add don't preclude running the tests locally - so I'm thinking putting that check in at the GHA level, not inside the functional-tests script itself, is the way to go.

Thoughts @janbrasna ?

[janbrasna]

I also want to mark the revision in the Slack notification (and/or put emphasis on it if it disagrees from the triggering payload hash value), so it'll definitely be in the workflow here, GHA specific.

I'm thinking job summaries or something similarly high level to communicate the refs, and potential differences.