Skip to content

Increased CPU usage (doubled) in aws-node container after Kubernetes v1.33 upgrade #3414

@schahal

Description

@schahal

What happened:

After upgrading EKS cluster from Kubernetes v1.32 to v1.33, the aws-node container (VPC CNI) exhibits approximately 2x higher CPU usage:

Image

Is there a technical reason for this?

The only thing I see that may account for this is the CNI release notes now supports multi-NICs on an instant (so probably more work), but nothing in the k8s release notes suggests it would trigger that after the upgrade (?)

The CNI driver was on v1.20.x before and after the upgrade

Attach logs

Will run this script on a node (my node is missing several dependencies to run this script) and share with [email protected].

The log output of the CNI did not change before and after.

What you expected to happen:

The CPU usage to remain relatively similar before and after the Kubernetes cluster upgrade

How to reproduce it (as minimally and precisely as possible):

  1. Run the CNI driver at the version listed in Environment below, but make sure you're on EKS' Kubernetes v1.32
  2. Upgrade the EKS control plane to Kubernetes v1.33
  3. Observe relatively higher aws-node CPU usage

Anything else we need to know?:

The following key environment variables are configured for
the aws-node container:

env:                                                             
  - name: AWS_VPC_CNI_NODE_PORT_SUPPORT                          
    value: "true"                                                
  - name: AWS_VPC_ENI_MTU                                        
    value: "9001"                                                
  - name: AWS_VPC_K8S_CNI_EXTERNALSNAT                           
    value: "false"                                               
  - name: AWS_VPC_K8S_CNI_LOGLEVEL                               
    value: "DEBUG"                                               
  - name: AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG                     
    value: "false"                                               
  - name: ENABLE_PREFIX_DELEGATION                               
    value: "true"                                                
  - name: WARM_ENI_TARGET                                        
    value: "1"                                                   
  - name: WARM_PREFIX_TARGET                                     
    value: "1"                                                   
  - name: DISABLE_TCP_EARLY_DEMUX                                
    value: "true"

Environment:

  • Kubernetes version (use kubectl version): v1.33.2-eks-931bdca

  • CNI Version: v1.20.1

  • OS (e.g: cat /etc/os-release):

      NAME=Bottlerocket
      ID=bottlerocket
      VERSION="1.44.0 (aws-k8s-1.33)"
      PRETTY_NAME="Bottlerocket OS 1.44.0 (aws-k8s-1.33)"
      VARIANT_ID=aws-k8s-1.33
      VERSION_ID=1.44.0
      BUILD_ID=244cd3a5
      VENDOR_NAME="Bottlerocket"
      HOME_URL="https://github.com/bottlerocket-os/bottlerocket"
      SUPPORT_URL="https://github.com/bottlerocket-os/bottlerocket/discussions"
      BUG_REPORT_URL="https://github.com/bottlerocket-os/bottlerocket/issues"
      DOCUMENTATION_URL="https://bottlerocket.dev"
    
  • Kernel (e.g. uname -a): Linux ip-100-64-63-20.dev1.internal 6.12.37 #1 SMP PREEMPT_DYNAMIC Thu Jul 24 23:19:42 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions