Open
Description
Cannot start the pod-cpu-hog experiment. Would you please help to have a look at the issue? The litmus version is 2.14.0.
C41K60Q0XJ:nginx$ k get pod -n litmus
NAME READY STATUS RESTARTS AGE
chaos-exporter-d767fcf5-hsbgq 1/1 Running 0 18h
chaos-operator-ce-7cf6cc79b4-xwh99 1/1 Running 0 18h
chaos-stress-cpu-1671764370-276557960 0/2 Completed 0 9m55s
chaos-stress-cpu-1671764370-670059491 0/2 Completed 0 10m
event-tracker-5ddb594676-9dhtj 1/1 Running 0 18h
litmusportal-auth-server-57899b796d-2tvvq 1/1 Running 0 18h
litmusportal-frontend-857dc974b7-n4jv4 1/1 Running 0 18h
litmusportal-server-59f79479db-92qmb 1/1 Running 0 78m
mongo-0 1/1 Running 0 18h
pod-cpu-hog-0hytghxz-runner 0/1 Completed 0 9m51s
pod-cpu-hog-helper-xhsnlr 0/1 Error 0 9m26s
pod-cpu-hog-mfbudy-skzzc 0/1 Completed 0 9m46s
subscriber-5cbcb4df94-22cg7 1/1 Running 0 18h
workflow-controller-8c548f686-zmk9c 1/1 Running 0 18h
C41K60Q0XJ:nginx I068888$ k logs pod-cpu-hog-helper-xhsnlr -n litmus
time="2022-12-23T03:01:21Z" level=info msg="Helper Name: stress-chaos"
time="2022-12-23T03:01:21Z" level=info msg="[PreReq]: Getting the ENV variables"
time="2022-12-23T03:01:21Z" level=info msg="container ID of nginx container, containerID: e15be7fdcc7442be032e1caf56d1e5bb41852bb07e395f2aaf88e5dd2588ff70"
time="2022-12-23T03:01:21Z" level=error msg="[docker]: **Failed to run docker inspect: []\nError: No such object: e15be7fdcc7442be032e1caf56d1e5bb41852bb07e395f2aaf88e5dd2588ff70\n"**
time="2022-12-23T03:01:21Z" level=fatal msg="helper pod failed, err: exit status 1"
C41K60Q0XJ:nginx I068888$ k logs pod-cpu-hog-0hytghxz-runner -n litmus
W1223 03:00:45.714442 1 client_config.go:615] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
time="2022-12-23T03:00:45Z" level=info msg="Experiments details are as follows" Experiments List="[pod-cpu-hog]" Engine Name=pod-cpu-hog-0hytghxz appLabels="app=nginx" appNs=nginx appKind=deployment Service Account Name=litmus-admin Engine Namespace=litmus
time="2022-12-23T03:00:45Z" level=info msg="Getting the ENV Variables"
time="2022-12-23T03:00:45Z" level=info msg="Preparing to run Chaos Experiment: pod-cpu-hog"
time="2022-12-23T03:00:46Z" level=info msg="Started Chaos Experiment Name: pod-cpu-hog, with Job Name: pod-cpu-hog-mfbudy"
time="2022-12-23T03:01:42Z" level=info msg="Chaos Pod Completed, Experiment Name: pod-cpu-hog, with Job Name: pod-cpu-hog-mfbudy"
time="2022-12-23T03:01:44Z" level=info msg="Chaos Engine has been updated with result, Experiment Name: pod-cpu-hog"
time="2022-12-23T03:01:44Z" level=info msg="[skip]: skipping the job deletion as jobCleanUpPolicy is set to {}"
C41K60Q0XJ:nginx I068888$ k logs pod-cpu-hog-0hytghxz-runner -n litmus
W1223 03:00:45.714442 1 client_config.go:615] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
time="2022-12-23T03:00:45Z" level=info msg="Experiments details are as follows" Experiments List="[pod-cpu-hog]" Engine Name=pod-cpu-hog-0hytghxz appLabels="app=nginx" appNs=nginx appKind=deployment Service Account Name=litmus-admin Engine Namespace=litmus
time="2022-12-23T03:00:45Z" level=info msg="Getting the ENV Variables"
time="2022-12-23T03:00:45Z" level=info msg="Preparing to run Chaos Experiment: pod-cpu-hog"
time="2022-12-23T03:00:46Z" level=info msg="Started Chaos Experiment Name: pod-cpu-hog, with Job Name: pod-cpu-hog-mfbudy"
time="2022-12-23T03:01:42Z" level=info msg="Chaos Pod Completed, Experiment Name: pod-cpu-hog, with Job Name: pod-cpu-hog-mfbudy"
time="2022-12-23T03:01:44Z" level=info msg="Chaos Engine has been updated with result, Experiment Name: pod-cpu-hog"
time="2022-12-23T03:01:44Z" level=info msg="[skip]: skipping the job deletion as jobCleanUpPolicy is set to {}"
C41K60Q0XJ:nginx I068888$ k logs pod-cpu-hog-mfbudy-skzzc -n litmus
time="2022-12-23T03:00:58Z" level=info msg="Experiment Name: pod-cpu-hog"
time="2022-12-23T03:00:58Z" level=info msg="[PreReq]: Getting the ENV for the pod-cpu-hog experiment"
time="2022-12-23T03:01:00Z" level=info msg="[PreReq]: Updating the chaos result of pod-cpu-hog experiment (SOT)"
time="2022-12-23T03:01:02Z" level=info msg="The application information is as follows" Namespace=nginx Label="app=nginx" App Kind=deployment
time="2022-12-23T03:01:02Z" level=info msg="[Status]: Verify that the AUT (Application Under Test) is running (pre-chaos)"
time="2022-12-23T03:01:02Z" level=info msg="[Status]: Checking whether application containers are in ready state"
time="2022-12-23T03:01:02Z" level=info msg="[Status]: The Container status are as follows" Pod=nginx-deployment1-5bcb875959-gnxnl Readiness=true container=nginx
time="2022-12-23T03:01:04Z" level=info msg="[Status]: Checking whether application pods are in running state"
time="2022-12-23T03:01:04Z" level=info msg="[Status]: The status of Pods are as follows" Status=Running Pod=nginx-deployment1-5bcb875959-gnxnl
time="2022-12-23T03:01:06Z" level=info msg="[Info]: The chaos tunables are:" Sequence=parallel PodsAffectedPerc=0 CPU Core=1 CPU Load Percentage=100
time="2022-12-23T03:01:06Z" level=info msg="[Chaos]:Number of pods targeted: 1"
time="2022-12-23T03:01:06Z" level=info msg="[Info]: Target pods list for chaos, [nginx-deployment1-5bcb875959-gnxnl]"
time="2022-12-23T03:01:06Z" level=info msg="[Info]: Details of application under chaos injection" PodName=nginx-deployment1-5bcb875959-gnxnl NodeName=ip-10-180-6-87.eu-central-1.compute.internal ContainerName=nginx
time="2022-12-23T03:01:06Z" level=info msg="[Status]: Checking the status of the helper pods"
time="2022-12-23T03:01:24Z" level=info msg="pod-cpu-hog-helper-xhsnlr helper pod is in Running state"
time="2022-12-23T03:01:26Z" level=info msg="[Wait]: waiting till the completion of the helper pod"
time="2022-12-23T03:01:26Z" level=info msg="helper pod status: Running"
time="2022-12-23T03:01:26Z" level=info msg="[Status]: The running status of Pods are as follows" Pod=pod-cpu-hog-helper-xhsnlr Status=Failed
time="2022-12-23T03:01:27Z" level=error msg="[Error]: CPU hog failed, err: helper pod failed"
Metadata
Metadata
Assignees
Labels
No labels