SELinux prevents reading /etc/ssl/certs/ca-certificates.crt when mounted as a volume

[jdoss]

Issue Description

I need to add my Root CA into some of my container workloads from the host. I run my own CA and I have trusted it on my server's trust store. When I try to mount /etc/ssl/certs/ca-certificates.crt into a container Traefik cannot read the file due to SELinux. Setting setenforce 1 allows the file to be read.

# journalctl --since today --no-host --no-pager|grep AVC
May 24 03:04:10 audit[2199830]: AVC avc:  denied  { read } for  pid=2199830 comm="traefik" name="tls-ca-bundle.pem" dev="md127" ino=540276337 scontext=system_u:system_r:container_t:s0:c90,c461 tcontext=unconfined_u:object_r:cert_t:s0 tclass=file permissive=0
May 24 03:04:10 audit[2199830]: AVC avc:  denied  { read } for  pid=2199830 comm="traefik" name="tls-ca-bundle.pem" dev="md127" ino=540276337 scontext=system_u:system_r:container_t:s0:c90,c461 tcontext=unconfined_u:object_r:cert_t:s0 tclass=file permissive=0
May 24 03:04:10 audit[2199830]: AVC avc:  denied  { read } for  pid=2199830 comm="traefik" name="tls-ca-bundle.pem" dev="md127" ino=540276337 scontext=system_u:system_r:container_t:s0:c90,c461 tcontext=unconfined_u:object_r:cert_t:s0 tclass=file permissive=0
May 24 03:09:06 audit[2200975]: AVC avc:  denied  { read } for  pid=2200975 comm="cat" name="tls-ca-bundle.pem" dev="md127" ino=540276337 scontext=system_u:system_r:container_t:s0:c90,c461 tcontext=unconfined_u:object_r:cert_t:s0 tclass=file permissive=0
May 24 03:10:58 audit[2201192]: AVC avc:  denied  { read } for  pid=2201192 comm="cat" name="tls-ca-bundle.pem" dev="md127" ino=540276337 scontext=system_u:system_r:container_t:s0:c90,c461 tcontext=unconfined_u:object_r:cert_t:s0 tclass=file permissive=1
May 24 03:10:58 audit[2201192]: AVC avc:  denied  { open } for  pid=2201192 comm="cat" path="/etc/ssl/certs/ca-certificates.crt" dev="md127" ino=540276337 scontext=system_u:system_r:container_t:s0:c90,c461 tcontext=unconfined_u:object_r:cert_t:s0 tclass=file permissive=1
May 24 03:13:11 audit[2201660]: AVC avc:  denied  { read } for  pid=2201660 comm="traefik" name="tls-ca-bundle.pem" dev="md127" ino=540276337 scontext=system_u:system_r:container_t:s0:c435,c826 tcontext=unconfined_u:object_r:cert_t:s0 tclass=file permissive=0
May 24 03:13:11 audit[2201660]: AVC avc:  denied  { read } for  pid=2201660 comm="traefik" name="tls-ca-bundle.pem" dev="md127" ino=540276337 scontext=system_u:system_r:container_t:s0:c435,c826 tcontext=unconfined_u:object_r:cert_t:s0 tclass=file permissive=0
May 24 03:13:11 audit[2201660]: AVC avc:  denied  { read } for  pid=2201660 comm="traefik" name="tls-ca-bundle.pem" dev="md127" ino=540276337 scontext=system_u:system_r:container_t:s0:c435,c826 tcontext=unconfined_u:object_r:cert_t:s0 tclass=file permissive=0
May 24 03:17:55 audit[2202534]: AVC avc:  denied  { read } for  pid=2202534 comm="traefik" name="tls-ca-bundle.pem" dev="md127" ino=540276337 scontext=system_u:system_r:container_t:s0:c590,c762 tcontext=unconfined_u:object_r:cert_t:s0 tclass=file permissive=0
May 24 03:17:55 audit[2202534]: AVC avc:  denied  { read } for  pid=2202534 comm="traefik" name="tls-ca-bundle.pem" dev="md127" ino=540276337 scontext=system_u:system_r:container_t:s0:c590,c762 tcontext=unconfined_u:object_r:cert_t:s0 tclass=file permissive=0
May 24 03:17:55 audit[2202534]: AVC avc:  denied  { read } for  pid=2202534 comm="traefik" name="tls-ca-bundle.pem" dev="md127" ino=540276337 scontext=system_u:system_r:container_t:s0:c590,c762 tcontext=unconfined_u:object_r:cert_t:s0 tclass=file permissive=0
May 24 03:20:34 audit[2203138]: AVC avc:  denied  { read } for  pid=2203138 comm="traefik" name="tls-ca-bundle.pem" dev="md127" ino=540276337 scontext=system_u:system_r:container_t:s0:c24,c316 tcontext=unconfined_u:object_r:cert_t:s0 tclass=file permissive=0
May 24 03:20:34 audit[2203138]: AVC avc:  denied  { read } for  pid=2203138 comm="traefik" name="tls-ca-bundle.pem" dev="md127" ino=540276337 scontext=system_u:system_r:container_t:s0:c24,c316 tcontext=unconfined_u:object_r:cert_t:s0 tclass=file permissive=0
May 24 03:20:34 audit[2203138]: AVC avc:  denied  { read } for  pid=2203138 comm="traefik" name="tls-ca-bundle.pem" dev="md127" ino=540276337 scontext=system_u:system_r:container_t:s0:c24,c316 tcontext=unconfined_u:object_r:cert_t:s0 tclass=file permissive=0
May 24 03:34:59 audit[2204736]: AVC avc:  denied  { read } for  pid=2204736 comm="cat" name="tls-ca-bundle.pem" dev="md127" ino=540276337 scontext=system_u:system_r:container_t:s0:c510,c752 tcontext=unconfined_u:object_r:cert_t:s0 tclass=file permissive=0

Steps to reproduce the issue

Steps to reproduce the issue

1. Run podman run -it -v /etc/ssl/certs/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt:ro,Z --rm fedora bash
2. Run cat /etc/ssl/certs/ca-certificates.crt which results in cat: /etc/ssl/certs/ca-certificates.crt: Permission denied
3. Run journalctl --since today --no-host --no-pager|grep AVC
4. Observe the SELinux denial

May 24 03:34:59 audit[2204736]: AVC avc:  denied  { read } for  pid=2204736 comm="cat" name="tls-ca-bundle.pem" dev="md127" ino=540276337 scontext=system_u:system_r:container_t:s0:c510,c752 tcontext=unconfined_u:object_r:cert_t:s0 tclass=file permissive=0

Describe the results you received

SELinux denial. Woohoo!

Describe the results you expected

I expect to be able tor read this file from inside a container when it is mounted as a volume.

podman info output

# podman info
host:
  arch: amd64
  buildahVersion: 1.29.0
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.6-3.fc37.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.6, commit: '
  cpuUtilization:
    idlePercent: 98.62
    systemPercent: 0.65
    userPercent: 0.73
  cpus: 20
  distribution:
    distribution: fedora
    variant: coreos
    version: "37"
  eventLogger: journald
  hostname: node000
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 6.1.14-200.fc37.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 22661918720
  memTotal: 33660600320
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun-1.8.1-1.fc37.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.8.1
      commit: f8a096be060b22ccd3d5f3ebe44108517fbf6c30
      rundir: /run/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.0-8.fc37.x86_64
    version: |-
      slirp4netns version 1.2.0
      commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.3
  swapFree: 4240961536
  swapTotal: 4294963200
  uptime: 1149h 43m 7.00s (Approximately 47.88 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /usr/share/containers/storage.conf
  containerStore:
    number: 14
    paused: 0
    running: 14
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 321830064128
  graphRootUsed: 15002718208
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 22
  runRoot: /run/containers/storage
  transientStore: false
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 4.4.1
  Built: 1676629579
  BuiltTime: Fri Feb 17 10:26:19 2023
  GitCommit: ""
  GoVersion: go1.19.5
  Os: linux
  OsArch: linux/amd64
  Version: 4.4.1

Podman in a container

No

Privileged Or Rootless

None

Upstream Latest Release

Yes

Additional environment details

This is on Fedora CoreOS 37.20230303.3.0

Additional information

No response

[jdoss]

I will note that running the container with --privileged allows it to read /etc/ssl/certs/ca-certificates.crt without issues. I'd rather not go that route if possible.

[Luap99]

@rhatdan PTAL

You do not need --privileged, --security-opt label=disable should be enough to only disable selinux for this container.

[jdoss]

@Luap99 thanks for getting back with me. This workload uses the https://github.com/hashicorp/nomad-driver-podman and we can't set --security-opt label=disable through the driver yet.

[rhatdan]

Can you pass --ipc=host? That has the side-effect of disabling SELinux.

We could add a boolean to allow all containers to read cert files, if this was a common enough case.

[djarbz]

I would vote for a Boolean.

I have an example of a Go binary that is added to a scratch image.
This binary needs the trusted CA certificates, I don't think it would be best to pre-package these during the image build, especially if new CAs are added or revoked.
I would hardly consider myself an expert in this regards however and it might not be as big of a deal as I think.

[yrro]

Looks like Dan added the container_read_certs boolean shortly after writing his comment. I'm now able to get rid of some custom policy modules as a result!

[jdoss]

I don't think I can pass --ipc=host either with the Nomad Podman Driver. TBH, I'd rather not disable SELinux either for this workload. A boolean sounds awesome tho.

For Traefik specifically it looks like I can set SSL_CERT_FILE=/etc/traefik/stepca.ca.pem and it will use that as well so I don't need to mount in the trust store, but I have other workloads that need to use this CA that don't respect that env so mounting in /etc/ssl/certs/ca-certificates.crt is the only option.

[rhatdan]

containers/container-selinux#255