Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attach tc programs using netlink API #105

Merged
merged 2 commits into from
Oct 21, 2024
Merged

Attach tc programs using netlink API #105

merged 2 commits into from
Oct 21, 2024

Conversation

iluxa
Copy link
Collaborator

@iluxa iluxa commented Oct 18, 2024

resolves https://github.com/kubeshark/worker/issues/263

Using new eBPF programs attach API doesn't break cilium network anymore.

Tested on EKS with eksctl on clusters version 1.29 and 1.31. In case of 1.29 ebpf capture doesn't work because of #108, but after the fix cluster itself continues to work without issues

Test scenario:

  • create cluster.yaml:
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: <cluster name>
  region: <region>
  version: <k8s version>

iam:
  withOIDC: true

addonsConfig:
  disableDefaultAddons: true
addons:
  - name: coredns
  • run eksctl create cluster -f cluster.yaml
  • check kubectl get pods -A -o wide :
NAMESPACE     NAME                       READY   STATUS    RESTARTS   AGE     IP       NODE     NOMINATED NODE   READINESS GATES
kube-system   coredns-844dbb9f6f-hl2w7   0/1     Pending   0          2m34s   <none>   <none>   <none>           <none>
kube-system   coredns-844dbb9f6f-wmv7m   0/1     Pending   0          2m34s   <none>   <none>   <none>           <none>
  • install cilium via helm:
helm install cilium cilium/cilium --version 1.16.3 \
  --namespace kube-system \
  --set kubeProxyReplacement=true \
  --set k8sServiceHost=$(aws eks describe-cluster --name <cluster name> --region <region> --query "cluster.endpoint" --output text | sed 's/https:\/\///') \
  --set k8sServicePort=443 \
  --set ipam.mode=cluster-pool \
  --set enableIPv4Masquerade=true \
  --set enableIPv6Masquerade=true \
  --set nodeinit.enabled=true \
  --set bpf.masquerade=true \
  --set enableXDP=true \
  --set enableHubble=true \
  --set hubble.relay.enabled=true \
  --set hubble.ui.enabled=true
  • create nodegroup.yaml:
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: <cluster name>
  region: <region>

managedNodeGroups:
  - name: byocni
    instanceType: m6i.xlarge
    desiredCapacity: 2
    privateNetworking: true
    ssh:
      allow: true

  • create nodegroup:
eksctl create nodegroup -f ./nodegroup.yaml
  • install kubeshark and test (without -disable-ebpf option in both sniffer and tracer)

links:
https://isovalent.com/blog/post/eks-byocni-cilium/
https://medium.com/@amitmavgupta/cilium-installing-cilium-in-eks-with-no-kube-proxy-86f54a56c360

@iluxa iluxa marked this pull request as draft October 18, 2024 17:18
@iluxa iluxa marked this pull request as ready for review October 21, 2024 08:54
@iluxa iluxa merged commit 0cf445f into master Oct 21, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant