Skip to content

Add docker config fragment #47

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

lool
Copy link
Contributor

@lool lool commented May 19, 2025

Add and use docker config fragment

  • kernel-configs: Add docker fragment
  • workflows: linux: use new docker config fragment
  • README: Use new docker config fragment

Fixes #43

@lool
Copy link
Contributor Author

lool commented May 20, 2025

I manually built this and ran check-config again:

info: reading kernel config from /proc/config.gz ...

Generally Necessary:
- cgroup hierarchy: cgroupv2
  Controllers:
  - cpu: available
  - cpuset: available
  - io: available
  - memory: available
  - pids: available
- CONFIG_NAMESPACES: enabled
- CONFIG_NET_NS: enabled
- CONFIG_PID_NS: enabled
- CONFIG_IPC_NS: enabled
- CONFIG_UTS_NS: enabled
- CONFIG_CGROUPS: enabled
- CONFIG_CGROUP_CPUACCT: enabled
- CONFIG_CGROUP_DEVICE: enabled
- CONFIG_CGROUP_FREEZER: enabled
- CONFIG_CGROUP_SCHED: enabled
- CONFIG_CPUSETS: enabled
- CONFIG_MEMCG: enabled
- CONFIG_KEYS: enabled
- CONFIG_VETH: enabled (as module)
- CONFIG_BRIDGE: enabled (as module)
- CONFIG_BRIDGE_NETFILTER: enabled (as module)
- CONFIG_IP_NF_FILTER: enabled (as module)
- CONFIG_IP_NF_MANGLE: enabled (as module)
- CONFIG_IP_NF_TARGET_MASQUERADE: enabled (as module)
- CONFIG_IP6_NF_FILTER: enabled (as module)
- CONFIG_IP6_NF_MANGLE: enabled (as module)
- CONFIG_IP6_NF_TARGET_MASQUERADE: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_CONNTRACK: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_IPVS: enabled (as module)
- CONFIG_NETFILTER_XT_MARK: enabled (as module)
- CONFIG_IP_NF_RAW: enabled (as module)
- CONFIG_IP_NF_NAT: enabled (as module)
- CONFIG_NF_NAT: enabled (as module)
- CONFIG_IP6_NF_RAW: enabled (as module)
- CONFIG_IP6_NF_NAT: enabled (as module)
- CONFIG_NF_NAT: enabled (as module)
- CONFIG_POSIX_MQUEUE: enabled
- CONFIG_CGROUP_BPF: enabled

Optional Features:
- CONFIG_USER_NS: enabled
- CONFIG_SECCOMP: enabled
- CONFIG_SECCOMP_FILTER: enabled
- CONFIG_CGROUP_PIDS: enabled
- CONFIG_MEMCG_SWAP: missing
    (cgroup swap accounting is currently enabled)
- CONFIG_BLK_CGROUP: enabled
- CONFIG_BLK_DEV_THROTTLING: enabled
- CONFIG_CGROUP_PERF: enabled
- CONFIG_CGROUP_HUGETLB: enabled
- CONFIG_NET_CLS_CGROUP: enabled (as module)
- CONFIG_CGROUP_NET_PRIO: enabled
- CONFIG_CFS_BANDWIDTH: enabled
- CONFIG_FAIR_GROUP_SCHED: enabled
- CONFIG_IP_NF_TARGET_REDIRECT: enabled (as module)
- CONFIG_IP_VS: enabled (as module)
- CONFIG_IP_VS_NFCT: enabled
- CONFIG_IP_VS_PROTO_TCP: enabled
- CONFIG_IP_VS_PROTO_UDP: enabled
- CONFIG_IP_VS_RR: enabled (as module)
- CONFIG_SECURITY_SELINUX: enabled
- CONFIG_SECURITY_APPARMOR: enabled
- CONFIG_EXT4_FS: enabled
- CONFIG_EXT4_FS_POSIX_ACL: enabled
- CONFIG_EXT4_FS_SECURITY: enabled
- Network Drivers:
  - "overlay":
    - CONFIG_VXLAN: enabled (as module)
    - CONFIG_BRIDGE_VLAN_FILTERING: enabled
      Optional (for encrypted networks):
      - CONFIG_CRYPTO: enabled
      - CONFIG_CRYPTO_AEAD: enabled
      - CONFIG_CRYPTO_GCM: enabled (as module)
      - CONFIG_CRYPTO_SEQIV: enabled (as module)
      - CONFIG_CRYPTO_GHASH: enabled (as module)
      - CONFIG_XFRM: enabled
      - CONFIG_XFRM_USER: enabled (as module)
      - CONFIG_XFRM_ALGO: enabled (as module)
      - CONFIG_INET_ESP: enabled (as module)
      - CONFIG_NETFILTER_XT_MATCH_BPF: enabled (as module)
  - "ipvlan":
    - CONFIG_IPVLAN: enabled (as module)
  - "macvlan":
    - CONFIG_MACVLAN: enabled (as module)
    - CONFIG_DUMMY: enabled (as module)
  - "ftp,tftp client in container":
    - CONFIG_NF_NAT_FTP: enabled (as module)
    - CONFIG_NF_CONNTRACK_FTP: enabled (as module)
    - CONFIG_NF_NAT_TFTP: enabled (as module)
    - CONFIG_NF_CONNTRACK_TFTP: enabled (as module)
- Storage Drivers:
  - "btrfs":
    - CONFIG_BTRFS_FS: enabled (as module)
    - CONFIG_BTRFS_FS_POSIX_ACL: enabled
  - "overlay":
    - CONFIG_OVERLAY_FS: enabled (as module)
  - "zfs":
    - /dev/zfs: missing
    - zfs command: missing
    - zpool command: missing

Limits:
- /proc/sys/kernel/keys/root_maxkeys: 1000000

Which looks good (CONFIG_MEMCG_SWAP being obsolete).

Unfortunately, docker doesn't run:

May 20 09:41:36 debian dockerd[726]: failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to register "bridge" driver: failed to create NAT chain DOCKER: iptables failed: iptables -t nat -N DOCKER: iptables: Failed to initialize nft: Protocol not supported

Indeed:

$ sudo iptables --wait -L -n
iptables: Failed to initialize nft: Protocol not supported

Which is due to netlink for netfilter being missing:

sudo strace -f iptables --wait -L -n
[...]
socket(AF_NETLINK, SOCK_RAW, NETLINK_NETFILTER) = -1 EPROTONOSUPPORT (Protocol not supported)
[...]

I believe this is provided by linux/net/netfilter/nfnetlink.c which is enabled by CONFIG_NETFILTER_NETLINK=m but that's typically not enabled directly, instead Debian kernels set:

# extended accounting via NFNETLINK
CONFIG_NETFILTER_NETLINK_ACCT=m
# queueing packets via NFNETLINK
CONFIG_NETFILTER_NETLINK_QUEUE=m
# logging packets via NFNETLINK
CONFIG_NETFILTER_NETLINK_LOG=m
# passive OS fingerprint via NFNETLINK
CONFIG_NETFILTER_NETLINK_OSF=m

I'm not sure which of these docker needs, perhaps only CONFIG_NETFILTER_NETLINK, perhaps CONFIG_NETFILTER_NETLINK_ACCT to have some stats on network traffic from containers.

I've turn them all as modules in the docker config fragment, but now I get:

$ sudo iptables -L -n
iptables v1.8.11 (nf_tables): Could not fetch rule set generation id: Invalid argument

This is because iptables in Debian is built for nftables, so CONFIG_NF_TABLES is needed (it's =m in the Debian kernel).

After this change, iptables -L works, but docker still fails to start with:

May 20 11:05:06 debian dockerd[699]: failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to register "bridge" driver: failed to create NAT chain DOCKER: iptables failed: iptables --wait -t nat -N DOCKER: iptables v1.8.11 (nf_tables):  TABLE_ADD failed (Operation not supported): table nat

and indeed, this fails:

sudo iptables --wait -t nat -N DOCKER
iptables v1.8.11 (nf_tables):  TABLE_ADD failed (Operation not supported): table nat

but I have yet to figure out what's missing in the kernel for this to work.

lool added 3 commits May 20, 2025 14:44
Ran Moby's check-config script against a mainline + defconfig kernel and
followed Debian's existing configs as much as possible. Will need to
maintain and update from time to time.

Signed-off-by: Loïc Minier <[email protected]>
@lool lool force-pushed the docker-kernel-configs branch from b3ea50a to a36ec34 Compare May 20, 2025 12:44
@lool
Copy link
Contributor Author

lool commented May 23, 2025

I checked configs from LmP starting with these:
https://github.com/foundriesio/lmp-kernel-cache/blob/c6f25306a02f3bfd7a4c140f0f347402f6a72ee6/features/docker/docker.cfg

all were already set either as =m or =y except for CONFIG_DM_THIN_PROVISIONING which seems unrelated to these issues and CONFIG_CFQ_GROUP_IOSCHED which seems to be an obsolete config (dropped in 4.20).

I'll check https://github.com/foundriesio/lmp-kernel-cache/blob/c6f25306a02f3bfd7a4c140f0f347402f6a72ee6/features/netfilter/netfilter.cfg next.

@lool
Copy link
Contributor Author

lool commented May 23, 2025

I checked https://raw.githubusercontent.com/foundriesio/lmp-kernel-cache/c6f25306a02f3bfd7a4c140f0f347402f6a72ee6/features/netfilter/netfilter.cfg

and these were the missing configs:

CONFIG_NF_CONNTRACK_MARK missing
CONFIG_NF_CT_PROTO_GRE missing
CONFIG_NF_CONNTRACK_AMANDA missing
CONFIG_NF_CONNTRACK_H323 missing
CONFIG_NF_CONNTRACK_IRC missing
CONFIG_NF_CONNTRACK_NETBIOS_NS missing
CONFIG_NF_CONNTRACK_PPTP missing
CONFIG_NF_CONNTRACK_SANE missing
CONFIG_NF_CONNTRACK_SIP missing
CONFIG_NF_CONNTRACK_TIMEOUT missing
CONFIG_NF_CONNTRACK_TIMESTAMP missing
CONFIG_NF_CT_NETLINK missing
CONFIG_NETFILTER_XTABLES_COMPAT missing
CONFIG_NETFILTER_XT_TARGET_CLASSIFY missing
CONFIG_NETFILTER_XT_TARGET_CONNMARK missing
CONFIG_NETFILTER_XT_TARGET_CT missing
CONFIG_NETFILTER_XT_TARGET_DSCP missing
CONFIG_NETFILTER_XT_TARGET_HL missing
CONFIG_NETFILTER_XT_TARGET_MARK missing
CONFIG_NETFILTER_XT_TARGET_NFLOG missing
CONFIG_NETFILTER_XT_TARGET_NFQUEUE missing
CONFIG_NETFILTER_XT_TARGET_TRACE missing
CONFIG_NETFILTER_XT_TARGET_TCPMSS missing
CONFIG_NETFILTER_XT_MATCH_COMMENT missing
CONFIG_NETFILTER_XT_MATCH_CONNBYTES missing
CONFIG_NETFILTER_XT_MATCH_CONNLIMIT missing
CONFIG_NETFILTER_XT_MATCH_CONNMARK missing
CONFIG_NETFILTER_XT_MATCH_DCCP missing
CONFIG_NETFILTER_XT_MATCH_DSCP missing
CONFIG_NETFILTER_XT_MATCH_ESP missing
CONFIG_NETFILTER_XT_MATCH_HASHLIMIT missing
CONFIG_NETFILTER_XT_MATCH_HELPER missing
CONFIG_NETFILTER_XT_MATCH_HL missing
CONFIG_NETFILTER_XT_MATCH_LENGTH missing
CONFIG_NETFILTER_XT_MATCH_LIMIT missing
CONFIG_NETFILTER_XT_MATCH_MAC missing
CONFIG_NETFILTER_XT_MATCH_MARK missing
CONFIG_NETFILTER_XT_MATCH_MULTIPORT missing
CONFIG_NETFILTER_XT_MATCH_POLICY missing
CONFIG_NETFILTER_XT_MATCH_PKTTYPE missing
CONFIG_NETFILTER_XT_MATCH_QUOTA missing
CONFIG_NETFILTER_XT_MATCH_REALM missing
CONFIG_NETFILTER_XT_MATCH_SCTP missing
CONFIG_NETFILTER_XT_MATCH_STATE missing
CONFIG_NETFILTER_XT_MATCH_STATISTIC missing
CONFIG_NETFILTER_XT_MATCH_STRING missing
CONFIG_NETFILTER_XT_MATCH_TCPMSS missing
CONFIG_NETFILTER_XT_MATCH_U32 missing
CONFIG_IP_NF_MATCH_AH missing
CONFIG_IP_NF_MATCH_ECN missing
CONFIG_IP_NF_MATCH_TTL missing
CONFIG_IP_NF_MATCH_RPFILTER missing
CONFIG_IP_NF_TARGET_SYNPROXY missing
CONFIG_IP_NF_TARGET_NETMAP missing
CONFIG_NF_NAT_SNMP_BASIC missing
CONFIG_NF_NAT_IRC missing
CONFIG_NF_NAT_AMANDA missing
CONFIG_NF_NAT_PPTP missing
CONFIG_NF_NAT_H323 missing
CONFIG_NF_NAT_SIP missing
CONFIG_IP_NF_TARGET_ECN missing
CONFIG_IP_NF_TARGET_TTL missing
CONFIG_IP_NF_ARPTABLES missing
CONFIG_IP_NF_ARPFILTER missing
CONFIG_IP_NF_ARP_MANGLE missing
CONFIG_IP_NF_SECURITY missing

I'll add a netfilter config fragment based on the Yocto one; I haven't figured proper copyright/licensing attribution for it yet though.

@lool
Copy link
Contributor Author

lool commented May 23, 2025

Even after these additions, I still get:

debian@debian:~$ sudo iptables --wait -t nat -N DOCKER
iptables v1.8.11 (nf_tables):  TABLE_ADD failed (Operation not supported): table nat

lool added 3 commits May 23, 2025 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RB1: iptables-ntables and hence docker not working with kernel 6.15
2 participants