Skip to content
This repository was archived by the owner on Feb 21, 2020. It is now read-only.

Commit e51f7df

Browse files
danielleedaniellee
committed
adds dynamic node element to all pod metrics
1 parent dd8657d commit e51f7df

File tree

6 files changed

+75
-43
lines changed

6 files changed

+75
-43
lines changed

‎README.md‎

Lines changed: 14 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# snap collector plugin - kube state
1+
# snap collector plugin - kubestate
22

33
This plugin collects metrics from Kubernetes about the state of pods, nodes and deployments.
44

@@ -100,22 +100,22 @@ This plugin has the ability to gather the following metrics:
100100

101101
Namespace | Description (optional)
102102
----------|-----------------------
103-
/grafanalabs/kubestate/pod/[NAMESPACE]/[POD]/status/condition/ready | specifies if the pod is ready to serve requests
104-
/grafanalabs/kubestate/pod/[NAMESPACE]/[POD]/status/condition/scheduled | status of the scheduling process for the pod
105-
/grafanalabs/kubestate/pod/[NAMESPACE]/[POD]/status/phase/Pending | This includes time before being bound to a node, as well as time spent pulling images onto the host.
106-
/grafanalabs/kubestate/pod/[NAMESPACE]/[POD]/status/phase/Running | The pod has been bound to a node and all of the containers have been started.
107-
/grafanalabs/kubestate/pod/[NAMESPACE]/[POD]/status/phase/Succeeded | All containers in the pod have voluntarily terminated with a container exit code of 0, and the system is not going to restart any of these containers.
108-
/grafanalabs/kubestate/pod/[NAMESPACE]/[POD]/status/phase/Failed | All containers in the pod have terminated, and at least one container has terminated in a failure.
109-
/grafanalabs/kubestate/pod/[NAMESPACE]/[POD]/status/phase/Unknown | For some reason the state of the pod could not be obtained, typically due to an error in communicating with the host of the pod.
103+
/grafanalabs/kubestate/pod/[NODE]/[NAMESPACE]/[POD]/status/condition/ready | specifies if the pod is ready to serve requests
104+
/grafanalabs/kubestate/pod/[NODE]/[NAMESPACE]/[POD]/status/condition/scheduled | status of the scheduling process for the pod
105+
/grafanalabs/kubestate/pod/[NODE]/[NAMESPACE]/[POD]/status/phase/Pending | This includes time before being bound to a node, as well as time spent pulling images onto the host.
106+
/grafanalabs/kubestate/pod/[NODE]/[NAMESPACE]/[POD]/status/phase/Running | The pod has been bound to a node and all of the containers have been started.
107+
/grafanalabs/kubestate/pod/[NODE]/[NAMESPACE]/[POD]/status/phase/Succeeded | All containers in the pod have voluntarily terminated with a container exit code of 0, and the system is not going to restart any of these containers.
108+
/grafanalabs/kubestate/pod/[NODE]/[NAMESPACE]/[POD]/status/phase/Failed | All containers in the pod have terminated, and at least one container has terminated in a failure.
109+
/grafanalabs/kubestate/pod/[NODE]/[NAMESPACE]/[POD]/status/phase/Unknown | For some reason the state of the pod could not be obtained, typically due to an error in communicating with the host of the pod.
110110
/grafanalabs/kubestate/container/[NAMESPACE]/[NODE]/[POD]/[CONTAINER]/limits/cpu/cores | The limit on cpu cores to be used by a container.
111111
/grafanalabs/kubestate/container/[NAMESPACE]/[NODE]/[POD]/[CONTAINER]/limits/memory/bytes | The limit on memory to be used by a container in bytes.
112112
/grafanalabs/kubestate/container/[NAMESPACE]/[NODE]/[POD]/[CONTAINER]/requested/cpu/cores | The number of requested cpu cores by a container.
113113
/grafanalabs/kubestate/container/[NAMESPACE]/[NODE]/[POD]/[CONTAINER]/requested/memory/bytes | The number of requested memory bytes by a container.
114-
/grafanalabs/kubestate/container/[NAMESPACE]/[POD]/[CONTAINER]/status/ready | specifies whether the container has passed its readiness probe
115-
/grafanalabs/kubestate/container/[NAMESPACE]/[POD]/[CONTAINER]/status/restarts | number of times the container has been restarted
116-
/grafanalabs/kubestate/container/[NAMESPACE]/[POD]/[CONTAINER]/status/running | value 1 if container is running else value 0
117-
/grafanalabs/kubestate/container/[NAMESPACE]/[POD]/[CONTAINER]/status/terminated | value 1 if container is terminated else value 0
118-
/grafanalabs/kubestate/container/[NAMESPACE]/[POD]/[CONTAINER]/status/waiting | value 1 if container is waiting else value 0
114+
/grafanalabs/kubestate/container/[NAMESPACE]/[NODE]/[POD]/[CONTAINER]/status/ready | specifies whether the container has passed its readiness probe
115+
/grafanalabs/kubestate/container/[NAMESPACE]/[NODE]/[POD]/[CONTAINER]/status/restarts | number of times the container has been restarted
116+
/grafanalabs/kubestate/container/[NAMESPACE]/[NODE]/[POD]/[CONTAINER]/status/running | value 1 if container is running else value 0
117+
/grafanalabs/kubestate/container/[NAMESPACE]/[NODE]/[POD]/[CONTAINER]/status/terminated | value 1 if container is terminated else value 0
118+
/grafanalabs/kubestate/container/[NAMESPACE]/[NODE]/[POD]/[CONTAINER]/status/waiting | value 1 if container is waiting else value 0
119119

120120
#### Nodes
121121

@@ -139,6 +139,7 @@ Namespace | Description (optional)
139139
/grafanalabs/kubestate/deployment/[NAMESPACE]/[DEPLOYMENT]/status/availablereplicas | Total number of available pods (ready for at least minReadySeconds) targeted by this deployment.
140140
/grafanalabs/kubestate/deployment/[NAMESPACE]/[DEPLOYMENT]/status/unavailablereplicas | Total number of unavailable pods targeted by this deployment.
141141
/grafanalabs/kubestate/deployment/[NAMESPACE]/[DEPLOYMENT]/spec/desiredreplicas | Number of desired pods.
142+
/grafanalabs/kubestate/deployment/[NAMESPACE]/[DEPLOYMENT]/status/deploynotfinished | If desired and observed generation are not the same, then either an ongoing deploy or a failed deploy.
142143

143144
### Examples
144145

‎kubestate/deployment.go‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,10 @@ func (*deploymentCollector) Collect(mts []plugin.Metric, deployment v1beta1.Depl
5151
} else if ns[5] == "spec" && ns[6] == "paused" {
5252
metric := createDeploymentMetric(mt, ns, deployment, boolInt(deployment.Spec.Paused))
5353
metrics = append(metrics, metric)
54+
} else if ns[5] == "status" && ns[6] == "deploynotfinished" {
55+
notFinished := deployment.Generation-deployment.Status.ObservedGeneration > 0
56+
metric := createDeploymentMetric(mt, ns, deployment, boolInt(notFinished))
57+
metrics = append(metrics, metric)
5458
}
5559
}
5660

‎kubestate/deployment_test.go‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,7 @@ var deploymentCases = []struct {
7575
"grafanalabs.kubestate.deployment.default.BeingDeployed.status.updatedreplicas 2",
7676
"grafanalabs.kubestate.deployment.default.BeingDeployed.spec.desiredreplicas 16",
7777
"grafanalabs.kubestate.deployment.default.BeingDeployed.spec.paused 0",
78+
"grafanalabs.kubestate.deployment.default.BeingDeployed.status.deploynotfinished 1",
7879
},
7980
},
8081
{
@@ -88,6 +89,7 @@ var deploymentCases = []struct {
8889
"grafanalabs.kubestate.deployment.default.NoDesiredReplicas.status.unavailablereplicas 0",
8990
"grafanalabs.kubestate.deployment.default.NoDesiredReplicas.status.updatedreplicas 0",
9091
"grafanalabs.kubestate.deployment.default.NoDesiredReplicas.spec.paused 0",
92+
"grafanalabs.kubestate.deployment.default.NoDesiredReplicas.status.deploynotfinished 0",
9193
},
9294
},
9395
{
@@ -102,6 +104,7 @@ var deploymentCases = []struct {
102104
"grafanalabs.kubestate.deployment.default.PausedDeploy.status.updatedreplicas 2",
103105
"grafanalabs.kubestate.deployment.default.PausedDeploy.spec.desiredreplicas 16",
104106
"grafanalabs.kubestate.deployment.default.PausedDeploy.spec.paused 1",
107+
"grafanalabs.kubestate.deployment.default.PausedDeploy.status.deploynotfinished 1",
105108
},
106109
},
107110
{

‎kubestate/kubestate.go‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,7 @@ func getPodMetricTypes() []plugin.Metric {
114114
mts = append(mts, plugin.Metric{
115115
Namespace: plugin.NewNamespace("grafanalabs", "kubestate", "pod").
116116
AddDynamicElement("namespace", "kubernetes namespace").
117+
AddDynamicElement("node", "node name").
117118
AddDynamicElement("pod", "pod name").
118119
AddStaticElements("status", "phase", "Pending"),
119120
Version: 1,
@@ -122,6 +123,7 @@ func getPodMetricTypes() []plugin.Metric {
122123
mts = append(mts, plugin.Metric{
123124
Namespace: plugin.NewNamespace("grafanalabs", "kubestate", "pod").
124125
AddDynamicElement("namespace", "kubernetes namespace").
126+
AddDynamicElement("node", "node name").
125127
AddDynamicElement("pod", "pod name").
126128
AddStaticElements("status", "phase", "Running"),
127129
Version: 1,
@@ -130,6 +132,7 @@ func getPodMetricTypes() []plugin.Metric {
130132
mts = append(mts, plugin.Metric{
131133
Namespace: plugin.NewNamespace("grafanalabs", "kubestate", "pod").
132134
AddDynamicElement("namespace", "kubernetes namespace").
135+
AddDynamicElement("node", "node name").
133136
AddDynamicElement("pod", "pod name").
134137
AddStaticElements("status", "phase", "Succeeded"),
135138
Version: 1,
@@ -138,6 +141,7 @@ func getPodMetricTypes() []plugin.Metric {
138141
mts = append(mts, plugin.Metric{
139142
Namespace: plugin.NewNamespace("grafanalabs", "kubestate", "pod").
140143
AddDynamicElement("namespace", "kubernetes namespace").
144+
AddDynamicElement("node", "node name").
141145
AddDynamicElement("pod", "pod name").
142146
AddStaticElements("status", "phase", "Failed"),
143147
Version: 1,
@@ -146,6 +150,7 @@ func getPodMetricTypes() []plugin.Metric {
146150
mts = append(mts, plugin.Metric{
147151
Namespace: plugin.NewNamespace("grafanalabs", "kubestate", "pod").
148152
AddDynamicElement("namespace", "kubernetes namespace").
153+
AddDynamicElement("node", "node name").
149154
AddDynamicElement("pod", "pod name").
150155
AddStaticElements("status", "phase", "Unknown"),
151156
Version: 1,
@@ -154,6 +159,7 @@ func getPodMetricTypes() []plugin.Metric {
154159
mts = append(mts, plugin.Metric{
155160
Namespace: plugin.NewNamespace("grafanalabs", "kubestate", "pod").
156161
AddDynamicElement("namespace", "kubernetes namespace").
162+
AddDynamicElement("node", "node name").
157163
AddDynamicElement("pod", "pod name").
158164
AddStaticElement("status").
159165
AddStaticElements("condition", "ready"),
@@ -163,6 +169,7 @@ func getPodMetricTypes() []plugin.Metric {
163169
mts = append(mts, plugin.Metric{
164170
Namespace: plugin.NewNamespace("grafanalabs", "kubestate", "pod").
165171
AddDynamicElement("namespace", "kubernetes namespace").
172+
AddDynamicElement("node", "node name").
166173
AddDynamicElement("pod", "pod name").
167174
AddStaticElement("status").
168175
AddStaticElements("condition", "scheduled"),
@@ -178,6 +185,7 @@ func getPodContainerMetricTypes() []plugin.Metric {
178185
mts = append(mts, plugin.Metric{
179186
Namespace: plugin.NewNamespace("grafanalabs", "kubestate", "container").
180187
AddDynamicElement("namespace", "kubernetes namespace").
188+
AddDynamicElement("node", "node name").
181189
AddDynamicElement("pod", "pod name").
182190
AddDynamicElement("container", "container name").
183191
AddStaticElements("status", "restarts"),
@@ -187,6 +195,7 @@ func getPodContainerMetricTypes() []plugin.Metric {
187195
mts = append(mts, plugin.Metric{
188196
Namespace: plugin.NewNamespace("grafanalabs", "kubestate", "container").
189197
AddDynamicElement("namespace", "kubernetes namespace").
198+
AddDynamicElement("node", "node name").
190199
AddDynamicElement("pod", "pod name").
191200
AddDynamicElement("container", "container name").
192201
AddStaticElements("status", "ready"),
@@ -196,6 +205,7 @@ func getPodContainerMetricTypes() []plugin.Metric {
196205
mts = append(mts, plugin.Metric{
197206
Namespace: plugin.NewNamespace("grafanalabs", "kubestate", "container").
198207
AddDynamicElement("namespace", "kubernetes namespace").
208+
AddDynamicElement("node", "node name").
199209
AddDynamicElement("pod", "pod name").
200210
AddDynamicElement("container", "container name").
201211
AddStaticElements("status", "waiting"),
@@ -205,6 +215,7 @@ func getPodContainerMetricTypes() []plugin.Metric {
205215
mts = append(mts, plugin.Metric{
206216
Namespace: plugin.NewNamespace("grafanalabs", "kubestate", "container").
207217
AddDynamicElement("namespace", "kubernetes namespace").
218+
AddDynamicElement("node", "node name").
208219
AddDynamicElement("pod", "pod name").
209220
AddDynamicElement("container", "container name").
210221
AddStaticElements("status", "running"),
@@ -214,6 +225,7 @@ func getPodContainerMetricTypes() []plugin.Metric {
214225
mts = append(mts, plugin.Metric{
215226
Namespace: plugin.NewNamespace("grafanalabs", "kubestate", "container").
216227
AddDynamicElement("namespace", "kubernetes namespace").
228+
AddDynamicElement("node", "node name").
217229
AddDynamicElement("pod", "pod name").
218230
AddDynamicElement("container", "container name").
219231
AddStaticElements("status", "terminated"),
@@ -393,6 +405,14 @@ func getDeploymentMetricTypes() []plugin.Metric {
393405
Version: 1,
394406
})
395407

408+
mts = append(mts, plugin.Metric{
409+
Namespace: plugin.NewNamespace("grafanalabs", "kubestate", "deployment").
410+
AddDynamicElement("namespace", "Kubernetes namespace").
411+
AddDynamicElement("deployment", "deployment name").
412+
AddStaticElements("status", "deploynotfinished"),
413+
Version: 1,
414+
})
415+
396416
return mts
397417
}
398418

‎kubestate/pod.go‎

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -14,12 +14,13 @@ const (
1414
minPodNamespaceSize = 8
1515
metricTypeNsPart = 2
1616
namespaceNsPart = metricTypeNsPart + 1
17-
podNameNsPart = metricTypeNsPart + 2
18-
podStatusNsPart = metricTypeNsPart + 3
19-
podStatusTypeNsPart = metricTypeNsPart + 4
20-
podStatusValueNsPart = metricTypeNsPart + 5
21-
containerStatusNsPart = metricTypeNsPart + 4
22-
containerStatusValueNsPart = metricTypeNsPart + 5
17+
nodeNsPart = metricTypeNsPart + 2
18+
podNameNsPart = metricTypeNsPart + 3
19+
podStatusNsPart = metricTypeNsPart + 4
20+
podStatusTypeNsPart = metricTypeNsPart + 5
21+
podStatusValueNsPart = metricTypeNsPart + 6
22+
containerStatusNsPart = metricTypeNsPart + 5
23+
containerStatusValueNsPart = metricTypeNsPart + 6
2324
containerResourceNsPart = metricTypeNsPart + 5
2425
containerResourceValueNsPart = metricTypeNsPart + 6
2526
)
@@ -40,6 +41,7 @@ func (*podCollector) Collect(mts []plugin.Metric, pod v1.Pod) ([]plugin.Metric,
4041
if ns[metricTypeNsPart] == "pod" && ns[podStatusNsPart] == "status" {
4142
if ns[podStatusTypeNsPart] == "phase" {
4243
ns[namespaceNsPart] = pod.Namespace
44+
ns[nodeNsPart] = slugify(pod.Spec.NodeName)
4345
ns[podNameNsPart] = pod.Name
4446

4547
mt.Namespace = plugin.NewNamespace(ns...)
@@ -54,6 +56,7 @@ func (*podCollector) Collect(mts []plugin.Metric, pod v1.Pod) ([]plugin.Metric,
5456
metrics = append(metrics, mt)
5557
} else if ns[podStatusTypeNsPart] == "condition" {
5658
ns[namespaceNsPart] = pod.Namespace
59+
ns[nodeNsPart] = slugify(pod.Spec.NodeName)
5760
ns[podNameNsPart] = pod.Name
5861
mt.Namespace = plugin.NewNamespace(ns...)
5962

@@ -140,8 +143,9 @@ func (*podCollector) Collect(mts []plugin.Metric, pod v1.Pod) ([]plugin.Metric,
140143

141144
func createContainerStatusMetric(mt plugin.Metric, ns []string, pod v1.Pod, cs v1.ContainerStatus, value interface{}) plugin.Metric {
142145
ns[namespaceNsPart] = pod.Namespace
143-
ns[namespaceNsPart+1] = pod.Name
144-
ns[namespaceNsPart+2] = cs.Name
146+
ns[nodeNsPart] = slugify(pod.Spec.NodeName)
147+
ns[nodeNsPart+1] = pod.Name
148+
ns[nodeNsPart+2] = cs.Name
145149
mt.Namespace = plugin.NewNamespace(ns...)
146150

147151
mt.Data = value

‎kubestate/pod_test.go‎

Lines changed: 22 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -132,24 +132,24 @@ var cases = []struct {
132132
pod: mockPods[0],
133133
metrics: getPodMetricTypes(),
134134
expected: []string{
135-
"grafanalabs.kubestate.pod.default.pod1.status.phase.Pending 0",
136-
"grafanalabs.kubestate.pod.default.pod1.status.phase.Running 1",
137-
"grafanalabs.kubestate.pod.default.pod1.status.phase.Succeeded 0",
138-
"grafanalabs.kubestate.pod.default.pod1.status.phase.Failed 0",
139-
"grafanalabs.kubestate.pod.default.pod1.status.phase.Unknown 0",
140-
"grafanalabs.kubestate.pod.default.pod1.status.condition.ready 1",
141-
"grafanalabs.kubestate.pod.default.pod1.status.condition.scheduled 1",
135+
"grafanalabs.kubestate.pod.default.127_0_0_1.pod1.status.phase.Pending 0",
136+
"grafanalabs.kubestate.pod.default.127_0_0_1.pod1.status.phase.Running 1",
137+
"grafanalabs.kubestate.pod.default.127_0_0_1.pod1.status.phase.Succeeded 0",
138+
"grafanalabs.kubestate.pod.default.127_0_0_1.pod1.status.phase.Failed 0",
139+
"grafanalabs.kubestate.pod.default.127_0_0_1.pod1.status.phase.Unknown 0",
140+
"grafanalabs.kubestate.pod.default.127_0_0_1.pod1.status.condition.ready 1",
141+
"grafanalabs.kubestate.pod.default.127_0_0_1.pod1.status.condition.scheduled 1",
142142
},
143143
},
144144
{
145145
pod: mockPods[0],
146146
metrics: getPodContainerMetricTypes(),
147147
expected: []string{
148-
"grafanalabs.kubestate.container.default.pod1.container1.status.restarts 3",
149-
"grafanalabs.kubestate.container.default.pod1.container1.status.ready 1",
150-
"grafanalabs.kubestate.container.default.pod1.container1.status.waiting 0",
151-
"grafanalabs.kubestate.container.default.pod1.container1.status.running 1",
152-
"grafanalabs.kubestate.container.default.pod1.container1.status.terminated 0",
148+
"grafanalabs.kubestate.container.default.127_0_0_1.pod1.container1.status.restarts 3",
149+
"grafanalabs.kubestate.container.default.127_0_0_1.pod1.container1.status.ready 1",
150+
"grafanalabs.kubestate.container.default.127_0_0_1.pod1.container1.status.waiting 0",
151+
"grafanalabs.kubestate.container.default.127_0_0_1.pod1.container1.status.running 1",
152+
"grafanalabs.kubestate.container.default.127_0_0_1.pod1.container1.status.terminated 0",
153153
"grafanalabs.kubestate.container.default.127_0_0_1.pod1.container1.requested.cpu.cores 0.1",
154154
"grafanalabs.kubestate.container.default.127_0_0_1.pod1.container1.requested.memory.bytes 1e+08",
155155
"grafanalabs.kubestate.container.default.127_0_0_1.pod1.container1.limits.cpu.cores 0.2",
@@ -160,16 +160,16 @@ var cases = []struct {
160160
pod: mockPods[1],
161161
metrics: getPodContainerMetricTypes(),
162162
expected: []string{
163-
"grafanalabs.kubestate.container.kube-system.pod2.container1.status.restarts 3",
164-
"grafanalabs.kubestate.container.kube-system.pod2.container2.status.restarts 5",
165-
"grafanalabs.kubestate.container.kube-system.pod2.container1.status.ready 1",
166-
"grafanalabs.kubestate.container.kube-system.pod2.container2.status.ready 0",
167-
"grafanalabs.kubestate.container.kube-system.pod2.container1.status.waiting 1",
168-
"grafanalabs.kubestate.container.kube-system.pod2.container2.status.waiting 0",
169-
"grafanalabs.kubestate.container.kube-system.pod2.container1.status.running 0",
170-
"grafanalabs.kubestate.container.kube-system.pod2.container2.status.running 0",
171-
"grafanalabs.kubestate.container.kube-system.pod2.container1.status.terminated 0",
172-
"grafanalabs.kubestate.container.kube-system.pod2.container2.status.terminated 1",
163+
"grafanalabs.kubestate.container.kube-system.node1.pod2.container1.status.restarts 3",
164+
"grafanalabs.kubestate.container.kube-system.node1.pod2.container2.status.restarts 5",
165+
"grafanalabs.kubestate.container.kube-system.node1.pod2.container1.status.ready 1",
166+
"grafanalabs.kubestate.container.kube-system.node1.pod2.container2.status.ready 0",
167+
"grafanalabs.kubestate.container.kube-system.node1.pod2.container1.status.waiting 1",
168+
"grafanalabs.kubestate.container.kube-system.node1.pod2.container2.status.waiting 0",
169+
"grafanalabs.kubestate.container.kube-system.node1.pod2.container1.status.running 0",
170+
"grafanalabs.kubestate.container.kube-system.node1.pod2.container2.status.running 0",
171+
"grafanalabs.kubestate.container.kube-system.node1.pod2.container1.status.terminated 0",
172+
"grafanalabs.kubestate.container.kube-system.node1.pod2.container2.status.terminated 1",
173173
"grafanalabs.kubestate.container.kube-system.node1.pod2.container1.limits.cpu.cores 0.2",
174174
"grafanalabs.kubestate.container.kube-system.node1.pod2.container2.limits.cpu.cores 0.2",
175175
"grafanalabs.kubestate.container.kube-system.node1.pod2.container1.limits.memory.bytes 2e+08",

0 commit comments

Comments
 (0)