Excessive Kernel NET Capabilities

[Somaimo]

Version higher than v0.29.x assign net_cap_raw capabilities to the /usr/local/bin/synthetic-monitoring-agent binary. This is sadly not allowed in our environment and I presume also in many other enterprise settings.

This capability is assigned on Line 15 in the Dockerfile

Is there a possibility to remove this capability or have a "hardened" image, where this capability is not assigned?

[nadiamoe]

Hey @Somaimo, thanks for letting us know!

https://github.com/grafana/synthetic-monitoring-agent/releases/tag/v0.34.0 actually removed quite a lot of privileges from the SM agent and container. Before this version, the binary of the agent ran as root, meaning that it has (implicit) access to the privileged operations cap_net_raw grant, as well as many others.

As of v0.34.0, only cap_net_raw is added, which should be relatively harmless when running on a dedicated network namespace (such as within a container). The agent needs this capability to perform ping and traceroute checks, unfortunately.

We haven't talked about the possibility of offering an unprivileged flavor of the image, so I'll check on that with the team.

[Somaimo]

Hi @nadiamoe

Yeah, you are absolutely right, compared to before (full root privileges) this is much more manageable from a security perspective and I'm sorry for wasting your time. I see your point concerning cap_net_raw not being that big of a deal security wise in the container namespace and I totally understand why the container needs this ability for ping and traceroute.

I will check internally if we can get a security exception to allow that capability to be set on the Pod.

Thank you very much for your fast reply and I will close this issue.

[nadiamoe]

Hey @Somaimo, no worries at all.

Please let me know how that goes, if allowing cap_net_raw is not an option we can check to see if there's any viable alternative.

[Somaimo]

Hi @nadiamoe

I reopened the issue to keep you updated on the progress internally.

[Somaimo]

Hi again

Our internally discussion revealed, that our container platform team is very hesitant in giving us an policy exception for this workload to have cap_net_raw. There is a certain security risk in allowing a workload to have this capability (even in though this workload is properly namespaced). The reason being, that any binary in the container can forge network packets.

My takeaway from the feedback from our container platform team is, that it would be great to have a "hardened" Container Image that doesn't require cap_net_raw and therefore wouldn't support ping or traceroute checks.

Would this be something that you could discuss internally?

[Somaimo]

Just for the curious, technically speaking neither ping nor traceroute (depending on the implementation of traceroute) requires root or cap_net_raw privileges.

There is a sysctl flag net.ipv4.ping_group_range that allows non-root users to run ping commands. This is also already set by default in many Linux distros and contaienr runtimes.

There are also traceroute implementations like mtr or tracepath that do not use raw sockets and can therefore be used by non-root users.

[Somaimo]

Hi @nadiamoe, could you possibly evaluate this issue again, based on the updated info provided in my comments above? A "hardened" Image without ping/traceroute capability would be a great short term solution, if that would be something you could imagine managing.

[nadiamoe]

Hey @Somaimo,

Unfortunately, last time I checked, setting net.ipv4.ping_group_range was not enough for the agent to work properly, I haven't check on the details as of why. If you take a look at the icmp prober source code, there are quite some things going on in there that outscope "just ping", so I suspect the reason would be related to that.

I'll bring this to the team and see if we can help provide a less privileged image :)

[Somaimo]

Hey @nadiamoe

Awesome stuff, thanks for evaluating if a less privileged image could be an option. :-) ah yes, I see, there are a lot more things going on, besides a simple icmp echo request.

[peterschretlen]

@Somaimo we've had several discussion about this on the synthetics team, and some of the discussion is captured in this RFC.

The favoured option is to have a second 'unprivileged' binary and a non-default entrypoint in the docker image that uses the unprivileged binary. While that would work, it's unappealing from a product and maintenance perspective, and it doesn't actually remove the capability from the image. Alternatives options have tradeoffs were not willing to make right now. So I acknowledge this is an issue, but in the short term we've decided not to address it. We'll leave the issue open, but it will go in our backlog.

If this is blocking others in the community, we'll use upvotes as a signal to revisit/reconsider this.

[Somaimo]

Hi @peterschretlen

Thanks a lot for taking time to discuss this issue. I completely understand your decision to shelve this issue for now.