Refinement of Output for Mequal and Automated Policy Bundle Documentatation Generation #9

mertbugrabicak · 2025-05-07T09:42:42Z

First off sorry for the gigantic PR!

The output we eventually wanted to get required doing some data processing before runtime, where we use OPA to generate objects containing the policy annotations in the source code, map them to a clean JSON, and then include them in the Mequal bundle for policies to make use of.

So the main changes:

Script to generate annotation objects using OPA inspect -a
Script to map these annotation object fields to one clean json file to show all metadata
Create and inject this json into the bundle upon container build
Add a "main" bundle to dynamically collect and show results for all bundles included in Mequal
eval.rego to get evaluations of all the policies for a given input
full.rego to get both the evaluations and a list of all policies ran on said input
Add possibility to define severity, level, and any other custom fields in policies and rules via metadata annotations
Edit policies to reflect/use the additional metadata fields
Edit policies to remove the leveling at the directory level, since no longer will be done this way.
Define external policy bundles included into Mequal using a json config file
Script to download all external policy bundles to put into Mequal
Script to generate AsciiDocs of the Mequal policies to be published in the documentation
Github action to automatically generate and publish Mequal policy bundle documentation to be put on the Mequal website

…ct annotations

…mage build

…ccessed by policies

…nd rule levels

…ed in mequal

…est json structure

jonathanchristison · 2025-05-12T13:12:40Z

container_files/Containerfile

    && mkdir ./bundle

 RUN adduser -u 1000 -U mequal -m

 USER root
 COPY ./policy/ ./policy/
+COPY ./hack/ ./hack/


I think of hack as scripts to support building and expect to run locally, these are ran as part of the container image build, we should perhaps re-purpose local-opa-run.sh to be ran at container build time, rather than maintain two seperate approaches to building the policy bundles.

Yes, you're absolutely right about this. I'll move the scripts that the image depends on to a scripts folder, and in hack I can create scripts to prep the development environment by running those scripts locally. It currently relies on building and running the image for development and testing, since inside the image it generates that metadata object to be used.

With something like dev-setup.sh, we generate the metadata object and put it into where it would be in the directory, so we can then use OPA locally to do our development. This script can also include downloading the helper functions to be used, so we can remove them from the code at least and let it live in another repo. (currently each policy repo has them copied over as a temporary solution, which is not good at all)

Both can then be marked by gitignore so they don't end up making it into the source code.

Separated them into separate scripts folder

jonathanchristison · 2025-05-12T13:23:58Z

hack/generate_bundle_info.py

@@ -0,0 +1,222 @@
+# Generated with assistance from a large language model trained by Google.


Add #!/bin/env python and set executable, I'm not sure about the current header, it doesn't give much information or attribution about what was used

Sure, I can do that

Done as #!/usr/bin/env python3

Forgot to address the LLM comment: Gemini by default isn't really attaching any info to the code it generates, and I just asked how to credit it. And it said to do this way since versioning may differ in the background if a specific model etc is specified. So I just kept it broad like this

jonathanchristison · 2025-05-12T16:47:31Z

policy/main/.manifest

@@ -0,0 +1,4 @@
+{
+    "revision": "v1.0.0",


This doesn't need fixing just yet as the container image contains the git sha but I created a PR for a temporary solution to having this also set in the policy manifest as I think we will forget to update this and it may catch us out if we distribute policies by alternative methods - mertbugrabicak#2 but feel free to ignore

Oh yes, this was something I wanted to do but forgot! We can definitely inject it during image build.

For the mequal bundle versioning, we can use our git sha, and for the external policies we define in our config json, I can create a revision field where we can concretely define what sha we are fetching the external policy from, and then inject those shas to their respective manifests.

Will take in your changes and work from there

Merged your changes, will now expand it to include external policies

Will leave this for now, to think of a better solution if possible for the external policy revisions

jonathanchristison · 2025-05-12T17:13:03Z

policy/main/eval.rego

+import rego.v1
+
+# Main evaluation rule to generate the desired "evaluation" output object
+evaluation := {


opa fmt changes this but as far as I can see there are no functional or syntax changes, only formatting, given we want to use regal in future I don't think we need to apply the changes but feel free

I think I should be able to do this

Merged fine

In this commit: * Added Gemini generated script to use current git revision as .manifest version * Call this script before building policy bundle in container

In this commit: * Allow overrides to the manifest revision * Update readme for building * Update github ci to override manifest revision

In this commit: * `opa fmt policy --v1-compatible -w`

…ve helpers and get externally

…tiple locations

mertbugrabicak · 2025-05-13T15:48:52Z

Moved the scripts used during image build to a ./scripts/ directory
Added dev-setup.sh and dev-cleanup.sh to automatically get local dev environment ready.
Moved helper functions to own directory where it is fetched from there now:
https://github.com/project-ncl/mequal-helpers
.gitignore for dev related stuff, external bundles pulled, etc
Added shebang to python script
Optimized the container for Mequal to reduce to half its size

mertbugrabicak · 2025-05-13T15:54:34Z

@jonathanchristison if you can have another look tomorrow, hopefully it should be mergeable now. it's it a lot better shape

mertbugrabicak · 2025-05-13T15:55:33Z

There is also a TODO to adapt the local opa run script to our latest bundle structure, i might also work on that tomorrow, but that can also wait

jonathanchristison · 2025-05-15T11:01:48Z

@mertbugrabicak looks good to me and some nice additions with local scripts and reduced runtime image size 👍

mertbugrabicak and others added 25 commits April 11, 2025 13:09

docs: update documents to reflect pivot to opa from conforma

eec046a

feat: add additional metadata to bundle manifest and package annotations

0b8259e

feat: add scripts to generate bundle metadata and docs from opa inspe…

080ced1

…ct annotations

feat: add main bundle in order to display cross bundle information

22a49ab

feat: run bundle metadata generation and inclusion into mequal upon i…

bd1bda1

…mage build

Merge branch 'project-ncl:main' into edit-docs

cf261b0

feat: added metadata json inside of the bundle on image build to be a…

2fd3466

…ccessed by policies

feat: edit mapper to include custom fields, rule severities, policy a…

46c53d0

…nd rule levels

feat: add main endpoint that controls output from all bundles at once

eb8ec7c

feat: add and move around policy information in policies

389b46f

fix: add if keywords to make v1 opa compatible

87b5a48

feat: add full endpoint to return both bundle info and eval results

93acee5

fix: change cli to point to full endpoint in image run

92638b4

feat: include prodsec bundle in evaluation results

0b85599

fix: adjust code comment to clarify

942cefe

feat: restructure mequal policies to exclude level from directory

f961a47

feat: dynamically fetch and show eval results from all bundles includ…

687ae13

…ed in mequal

feat: update the documentation generate script for bundles to use lat…

b2f31d6

…est json structure

feat: check if OPA binary defined as env before running script

1b7343f

docs: auto generate docs for policy bundles included

b9556d5

fix: make annotation script executable

4c84b8c

chore: credit gemini for doc generate script

ef1df54

feat: get and download external policy bundles from configuration file

814b8e1

docs: fetch external policies for auto generating documentation

259267e

feat: make download external bundles script executable

23aa99e

mertbugrabicak requested review from jonathanchristison and goldmann May 12, 2025 12:34

mertbugrabicak marked this pull request as ready for review May 12, 2025 12:35

jonathanchristison reviewed May 12, 2025

View reviewed changes

jonathanchristison and others added 7 commits May 13, 2025 10:12

feat(policy versioning): add script to version policy

738331b

In this commit: * Added Gemini generated script to use current git revision as .manifest version * Call this script before building policy bundle in container

feat(policy versioning): set manifest version for podman build

ec633ec

In this commit: * Allow overrides to the manifest revision * Update readme for building * Update github ci to override manifest revision

chore(policies): opa fmt

9ef8e17

In this commit: * `opa fmt policy --v1-compatible -w`

feat: separate image scripts, create actual dev scripts in hack, remo…

9dfebff

…ve helpers and get externally

feat: separate binary downloads into a separate script for use in mul…

2828bc4

…tiple locations

fix: dev setup script will also download binaries

3f4eb67

feat: multi stage build with ubi minimal final image to reduce size

e5b0dcb

jonathanchristison merged commit f3ba37a into project-ncl:main May 15, 2025
3 checks passed

		@@ -0,0 +1,222 @@
		# Generated with assistance from a large language model trained by Google.

Refinement of Output for Mequal and Automated Policy Bundle Documentatation Generation #9

Refinement of Output for Mequal and Automated Policy Bundle Documentatation Generation #9

Uh oh!

Conversation

mertbugrabicak commented May 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mertbugrabicak commented May 13, 2025

Uh oh!

mertbugrabicak commented May 13, 2025

Uh oh!

mertbugrabicak commented May 13, 2025

Uh oh!

jonathanchristison commented May 15, 2025

Uh oh!

Uh oh!

Uh oh!

mertbugrabicak commented May 7, 2025 •

edited

Loading