[Question] Is there a way to enforce only matching resources on a set of nodes?

[js-timbirkett]

OK, the question isn't the greatest 😬 - I'll try to explain a bit more:

We have a separate set of "build" nodes for CI/CD, but this could be applied to any scenario where you have a separate set of tainted nodes.

These nodes are typically short-lived and are used to allow docker-in-docker reducing the risk that a malicious app or user could run containers that potentially compromise or cause issues for other containers running on the host's Docker daemon.

We use nodeSelector, taints and tolerations to ensure that build agents run on build nodes and no other workloads get scheduled there.

It'd be nice if we could specifically deny (or allow) resources to run on these nodes with k-rail, and allow docker socket mounts on these nodes only based on label or taint. I'm not sure if this ability exists already or if it's a feature that others. would be interested in?

I can write a policy up for this and submit a PR?

[js-timbirkett]

I've thought about deploying OPA Gatekeeper and scribing up some rego, but I'd like to extend k-rail where it makes sense to do so and contribute back to the community that uses it :)

[dustin-decker]

This does not currently exist but would be a welcome feature addition.

Here are a few things to look at to enable it:

1. A new exemptions field we need to be added and checked in: https://github.com/cruise-automation/k-rail/blob/master/policies/exemption.go#L26
2. IsExempt would need to check if the given resource has any matching exemptions:

   k-rail/policies/exemption.go

   Line 118 in cafdc99

   // IsExempt returns whether a resource is exempt from a given policy

3. You'll probably need to use the Pod decoder to obtain the nodeSelector, taints, or tolerations https://github.com/cruise-automation/k-rail/blob/master/resource/pod.go#L30

[mark-adams]

👋 The k-rail project has been deprecated and is no longer under active development. We recommend taking a look at OPA Gatekeeper to see if it might meet your needs going forward.

Thanks for your contribution(s) to the project!