PolicyServer leader controller

[fabriziosestito]

The leader reconciler will be responsible for pulling policies from the registry, validating policy settings, precompiling the policy, and storing it in the shared cache.

Implement a leader election mechanism to ensure that only one Policy Server instance acts as a leader at a time.

References

• https://github.com/kubewarden/rfc/blob/main/rfc/0022-policy-lifecycle.md#policyserver-leader-reconciler
• https://github.com/kubewarden/rfc/blob/main/rfc/0022-policy-lifecycle.md#shared-policy-cache
• https://github.com/kubewarden/rfc/blob/main/rfc/0022-policy-lifecycle.md#policy-lifecycle

Acceptance criteria

• Implement leader election using Kubernetes Leases via the kubert crate
• Create leader reconciliation loop to process new/updated PolicyRevisions
• Implement policy pulling, validation, and precompilation in the leader
• Store the precompiled modules in a volume
• Handle Wasmtime version compatibility across different versions
• Update PolicyRevision status conditions (Scheduled, Initialized)
• Implement cleanup mechanism for old policies in the shared cache
• Ensure proper handover of leader responsibilities during failover

[fabriziosestito]

Example of a reconciler found in the POC: https://github.com/fabriziosestito/policy-server/blob/6e84438717f2f58e71cba685bb64b2a77a3209ec/src/controller/replica_reconciler.rs#L34

[fabriziosestito]

We could use this early-stage project ktest to create a mock k8s cluster for integration tests.