Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support expansion in gator verify #3650

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

David-Jaeyoon-Lee
Copy link
Contributor

What this PR does / why we need it:
It allows for expansion in gator verify.

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):
Fixes #3432

Special notes for your reviewer:

@David-Jaeyoon-Lee David-Jaeyoon-Lee requested a review from a team as a code owner October 16, 2024 21:17
@David-Jaeyoon-Lee David-Jaeyoon-Lee marked this pull request as draft October 16, 2024 21:17
@codecov-commenter
Copy link

codecov-commenter commented Oct 16, 2024

Codecov Report

Attention: Patch coverage is 50.74627% with 33 lines in your changes missing coverage. Please review.

Project coverage is 47.65%. Comparing base (3350319) to head (6c1464a).
Report is 173 commits behind head on master.

Files with missing lines Patch % Lines
pkg/gator/verify/runner.go 59.64% 16 Missing and 7 partials ⚠️
pkg/gator/reader/read_resources.go 0.00% 10 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (3350319) and HEAD (6c1464a). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (3350319) HEAD (6c1464a)
unittests 2 1
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3650      +/-   ##
==========================================
- Coverage   54.49%   47.65%   -6.85%     
==========================================
  Files         134      236     +102     
  Lines       12329    19777    +7448     
==========================================
+ Hits         6719     9425    +2706     
- Misses       5116     9465    +4349     
- Partials      494      887     +393     
Flag Coverage Δ
unittests 47.65% <50.74%> (-6.85%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

pkg/gator/errors.go Outdated Show resolved Hide resolved
pkg/gator/verify/runner.go Outdated Show resolved Hide resolved
@JaydipGabani
Copy link
Contributor

Are we going to add expannsionTemplate to suite?

@David-Jaeyoon-Lee David-Jaeyoon-Lee marked this pull request as ready for review October 24, 2024 22:55
@maxsmythe maxsmythe requested a review from a team October 25, 2024 23:52
Copy link
Contributor

@maxsmythe maxsmythe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couple small nits

pkg/gator/errors.go Outdated Show resolved Hide resolved
pkg/gator/reader/read_constraints.go Outdated Show resolved Hide resolved
pkg/gator/verify/runner.go Show resolved Hide resolved
@ritazh ritazh added this to the v3.18.0 milestone Oct 30, 2024
Copy link
Contributor

@JaydipGabani JaydipGabani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@David-Jaeyoon-Lee LGTM after gator tests are modified for expansion template, this might help adding those tests David-Jaeyoon-Lee#2.

@David-Jaeyoon-Lee
Copy link
Contributor Author

David-Jaeyoon-Lee commented Oct 31, 2024

@David-Jaeyoon-Lee LGTM after gator tests are modified for expansion template, this might help adding those tests David-Jaeyoon-Lee#2.

I think I added similar tests in this pr already. Or am I misunderstanding something? What specifically do we want to modify?

Edit: Nevermind I see

@David-Jaeyoon-Lee David-Jaeyoon-Lee force-pushed the davjlee/feature/gator-verify-expansion branch from d3892b4 to 8339ab4 Compare November 4, 2024 21:25
pkg/gator/errors.go Outdated Show resolved Hide resolved
pkg/gator/errors.go Outdated Show resolved Hide resolved
@JaydipGabani
Copy link
Contributor

@David-Jaeyoon-Lee you will probably need to update docs as well to let users know how to use this. This is probably the best place to include that information - https://open-policy-agent.github.io/gatekeeper/website/docs/gator#the-gator-verify-subcommand.

I am ok with a follow up PR to update the docs. @maxsmythe @ritazh @sozercan wdyt?

@@ -128,7 +128,7 @@ gator test --filename=manifests-and-policies/ --output=json
`gator verify` organizes tests into three levels: Suites, Tests, and Cases:

- A Suite is a file which defines Tests.
- A Test declares a ConstraintTemplate, a Constraint, and Cases to test the
- A Test declares a ConstraintTemplate, a Constraint, an ExpansionTemplate (optional), and Cases to test the
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it necessary to add here or should it just be a separate section.

@@ -162,6 +162,8 @@ ConstraintTemplate. It is an error for the Constraint to have a different type
than that defined in the ConstraintTemplate spec.crd.spec.names.kind, or for the
ConstraintTemplate to not compile.

A Test can also optionally compile an ExpansionTemplate.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

@@ -264,6 +266,25 @@ the `run` flag:
gator verify path/to/suites/... --run "disallowed"
```

### Validating Generated Resources with ExpansionTemplates
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is too much or if its not enough (i.e. no example or a more complete example).

Copy link
Contributor

@JaydipGabani JaydipGabani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@David-Jaeyoon-Lee Thanks for the doc changes. LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

source:generated within constraint.yaml causes gator verify to fail
5 participants