Skip to content

Commit a81e82b

Browse files
kpattaswamykpattaswamy
authored
[nr-k8s-otel-collector] Add support for additional configs and LDM support (#1719)
<!-- Thank you for contributing to New Relic's Helm charts. Before you submit this PR we'd like to make sure you are aware of our technical requirements: * https://github.com/newrelic-experimental/helm-charts/blob/master/CONTRIBUTING.md#technical-requirements For a quick overview across what we will look at reviewing your PR, please read our review guidelines: * https://github.com/newrelic-experimental/helm-charts/blob/master/REVIEW_GUIDELINES.md Following our best practices right from the start will accelerate the review process and help get your PR merged quicker. When updates to your PR are requested, please add new commits and do not squash the history. This will make it easier to identify new changes. The PR will be squashed anyways when it is merged. Thanks. For fast feedback, please @-mention maintainers that are listed in the Chart.yaml file. Please make sure you test your changes before you push them. Once pushed, a Github Action will run across your changes and do some initial checks and linting. These checks run very quickly. Please check the results. We would like these checks to pass before we even continue reviewing your changes. --> #### Is this a new chart No #### What this PR does / why we need it: TSIA #### Which issue this PR fixes *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)* - fixes # #### Special notes for your reviewer: #### Checklist [Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.] - [x] Chart Version bumped - [x] Variables are documented in the README.md - [x] Title of the PR starts with chart name (e.g. `[mychartname]`) --------- Signed-off-by: kpattaswamy <[email protected]>
1 parent 2bd1709 commit a81e82b

File tree

7 files changed

+332
-79
lines changed

7 files changed

+332
-79
lines changed

‎charts/nr-k8s-otel-collector/Chart.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ type: application
1717
# This is the chart version. This version number should be incremented each time you make changes
1818
# to the chart and its templates, including the app version.
1919
# Versions are expected to follow Semantic Versioning (https://semver.org/)
20-
version: 0.8.16
20+
version: 0.8.17
2121

2222
dependencies:
2323
- name: common-library

‎charts/nr-k8s-otel-collector/README.md

Lines changed: 67 additions & 58 deletions
Large diffs are not rendered by default.

‎charts/nr-k8s-otel-collector/templates/_config_map.tpl

Lines changed: 126 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,142 @@
11
{{- /* Defines if the deployment config map has to be created or not */ -}}
2-
{{- define "nrKubernetesOtel.deployment.configMap.config" -}}
2+
{{- define "nrKubernetesOtel.deployment.configMap.overrideConfig" -}}
33

44
{{- /* Look for a local creation of a deployment config map */ -}}
55
{{- if get .Values.deployment "configMap" | kindIs "map" -}}
6-
{{- if .Values.deployment.configMap.config -}}
7-
{{- toYaml .Values.deployment.configMap.config -}}
6+
{{- if .Values.deployment.configMap.overrideConfig -}}
7+
{{- toYaml .Values.deployment.configMap.overrideConfig -}}
8+
{{- end -}}
9+
{{- end -}}
10+
{{- end -}}
11+
12+
{{- /* Defines if the deployment receivers have to be added to the config */ -}}
13+
{{- define "nrKubernetesOtel.deployment.configMap.extraConfig.receivers" -}}
14+
15+
{{- if get .Values.deployment "configMap" | kindIs "map" -}}
16+
{{- if .Values.deployment.configMap.extraConfig -}}
17+
{{- if .Values.deployment.configMap.extraConfig.receivers -}}
18+
{{- toYaml .Values.deployment.configMap.extraConfig.receivers -}}
19+
{{- end -}}
20+
{{- end -}}
21+
{{- end -}}
22+
{{- end -}}
23+
24+
{{- /* Defines if the deployment processors have to be added to the config */ -}}
25+
{{- define "nrKubernetesOtel.deployment.configMap.extraConfig.processors" -}}
26+
27+
{{- if get .Values.deployment "configMap" | kindIs "map" -}}
28+
{{- if .Values.deployment.configMap.extraConfig -}}
29+
{{- if .Values.deployment.configMap.extraConfig.processors -}}
30+
{{- toYaml .Values.deployment.configMap.extraConfig.processors -}}
31+
{{- end -}}
32+
{{- end -}}
33+
{{- end -}}
34+
{{- end -}}
35+
36+
{{- /* Defines if the deployment exporters have to be added to the config */ -}}
37+
{{- define "nrKubernetesOtel.deployment.configMap.extraConfig.exporters" -}}
38+
39+
{{- if get .Values.deployment "configMap" | kindIs "map" -}}
40+
{{- if .Values.deployment.configMap.extraConfig -}}
41+
{{- if .Values.deployment.configMap.extraConfig.exporters -}}
42+
{{- toYaml .Values.deployment.configMap.extraConfig.exporters -}}
43+
{{- end -}}
44+
{{- end -}}
45+
{{- end -}}
46+
{{- end -}}
47+
48+
{{- /* Defines if the deployment connectors have to be added to the config */ -}}
49+
{{- define "nrKubernetesOtel.deployment.configMap.extraConfig.connectors" -}}
50+
51+
{{- if get .Values.deployment "configMap" | kindIs "map" -}}
52+
{{- if .Values.deployment.configMap.extraConfig -}}
53+
{{- if .Values.deployment.configMap.extraConfig.connectors -}}
54+
{{- toYaml .Values.deployment.configMap.extraConfig.connectors -}}
55+
{{- end -}}
56+
{{- end -}}
57+
{{- end -}}
58+
{{- end -}}
59+
60+
{{- /* Defines if the deployment pipelines have to be added to the config */ -}}
61+
{{- define "nrKubernetesOtel.deployment.configMap.extraConfig.pipelines" -}}
62+
63+
{{- if get .Values.deployment "configMap" | kindIs "map" -}}
64+
{{- if .Values.deployment.configMap.extraConfig -}}
65+
{{- if .Values.deployment.configMap.extraConfig.pipelines -}}
66+
{{- toYaml .Values.deployment.configMap.extraConfig.pipelines -}}
67+
{{- end -}}
868
{{- end -}}
969
{{- end -}}
1070
{{- end -}}
1171

1272

1373
{{- /* Defines if the daemonset config map has to be created or not */ -}}
14-
{{- define "nrKubernetesOtel.daemonset.configMap.config" -}}
74+
{{- define "nrKubernetesOtel.daemonset.configMap.overrideConfig" -}}
1575

1676
{{- /* Look for a local creation of a daemonset config map */ -}}
1777
{{- if get .Values.daemonset "configMap" | kindIs "map" -}}
18-
{{- if .Values.daemonset.configMap.config -}}
19-
{{- toYaml .Values.daemonset.configMap.config -}}
78+
{{- if .Values.daemonset.configMap.overrideConfig -}}
79+
{{- toYaml .Values.daemonset.configMap.overrideConfig -}}
80+
{{- end -}}
81+
{{- end -}}
82+
{{- end -}}
83+
84+
{{- /* Defines if the daemonset receivers have to be added to the config */ -}}
85+
{{- define "nrKubernetesOtel.daemonset.configMap.extraConfig.receivers" -}}
86+
87+
{{- if get .Values.daemonset "configMap" | kindIs "map" -}}
88+
{{- if .Values.daemonset.configMap.extraConfig -}}
89+
{{- if .Values.daemonset.configMap.extraConfig.receivers -}}
90+
{{- toYaml .Values.daemonset.configMap.extraConfig.receivers -}}
91+
{{- end -}}
92+
{{- end -}}
93+
{{- end -}}
94+
{{- end -}}
95+
96+
{{- /* Defines if the daemonset processors have to be added to the config */ -}}
97+
{{- define "nrKubernetesOtel.daemonset.configMap.extraConfig.processors" -}}
98+
99+
{{- if get .Values.daemonset "configMap" | kindIs "map" -}}
100+
{{- if .Values.daemonset.configMap.extraConfig -}}
101+
{{- if .Values.daemonset.configMap.extraConfig.processors -}}
102+
{{- toYaml .Values.daemonset.configMap.extraConfig.processors -}}
103+
{{- end -}}
104+
{{- end -}}
105+
{{- end -}}
106+
{{- end -}}
107+
108+
{{- /* Defines if the daemonset exporters have to be added to the config */ -}}
109+
{{- define "nrKubernetesOtel.daemonset.configMap.extraConfig.exporters" -}}
110+
111+
{{- if get .Values.daemonset "configMap" | kindIs "map" -}}
112+
{{- if .Values.daemonset.configMap.extraConfig -}}
113+
{{- if .Values.daemonset.configMap.extraConfig.exporters -}}
114+
{{- toYaml .Values.daemonset.configMap.extraConfig.exporters -}}
115+
{{- end -}}
116+
{{- end -}}
117+
{{- end -}}
118+
{{- end -}}
119+
120+
{{- /* Defines if the daemonset connectors have to be added to the config */ -}}
121+
{{- define "nrKubernetesOtel.daemonset.configMap.extraConfig.connectors" -}}
122+
123+
{{- if get .Values.daemonset "configMap" | kindIs "map" -}}
124+
{{- if .Values.daemonset.configMap.extraConfig -}}
125+
{{- if .Values.daemonset.configMap.extraConfig.connectors -}}
126+
{{- toYaml .Values.daemonset.configMap.extraConfig.connectors -}}
127+
{{- end -}}
128+
{{- end -}}
129+
{{- end -}}
130+
{{- end -}}
131+
132+
{{- /* Defines if the daemonset pipelines have to be added to the config */ -}}
133+
{{- define "nrKubernetesOtel.daemonset.configMap.extraConfig.pipelines" -}}
134+
135+
{{- if get .Values.daemonset "configMap" | kindIs "map" -}}
136+
{{- if .Values.daemonset.configMap.extraConfig -}}
137+
{{- if .Values.daemonset.configMap.extraConfig.pipelines -}}
138+
{{- toYaml .Values.daemonset.configMap.extraConfig.pipelines -}}
139+
{{- end -}}
20140
{{- end -}}
21141
{{- end -}}
22142
{{- end -}}

‎charts/nr-k8s-otel-collector/templates/daemonset-configmap.yaml

Lines changed: 53 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,11 @@ metadata:
77
{{- include "newrelic.common.labels" . | nindent 4 }}
88
data:
99
daemonset-config.yaml: |
10-
{{- with include "nrKubernetesOtel.daemonset.configMap.config" . }}
10+
{{- with include "nrKubernetesOtel.daemonset.configMap.overrideConfig" . }}
1111
{{- . | nindent 4 }}
1212
{{- else }}
1313
receivers:
14+
{{- include "nrKubernetesOtel.daemonset.configMap.extraConfig.receivers" . | nindent 6 }}
1415
hostmetrics:
1516
# TODO (chris): this is a linux specific configuration
1617
{{- if not .Values.gkeAutopilot }}
@@ -215,6 +216,8 @@ data:
215216
to: resource["k8s.pod.uid"]
216217
217218
processors:
219+
{{- include "nrKubernetesOtel.daemonset.configMap.extraConfig.processors" . | nindent 6 }}
220+
218221
metricstransform/ldm:
219222
transforms:
220223
- include: .*
@@ -611,6 +614,8 @@ data:
611614
send_batch_size : 800
612615
613616
exporters:
617+
{{- include "nrKubernetesOtel.daemonset.configMap.extraConfig.exporters" . | nindent 6 }}
618+
614619
otlphttp/newrelic:
615620
endpoint: {{ include "nrKubernetesOtel.endpoint" . }}
616621
headers:
@@ -619,6 +624,37 @@ data:
619624
verbosity: detailed
620625
sampling_initial: 5
621626
sampling_thereafter: 200
627+
628+
connectors:
629+
{{- include "nrKubernetesOtel.daemonset.configMap.extraConfig.connectors" . | nindent 6 }}
630+
routing/nr_pipelines:
631+
default_pipelines: [metrics/default]
632+
error_mode: propagate
633+
table:
634+
- context: metric
635+
condition: instrumentation_scope.name == "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/networkscraper"
636+
pipelines: [metrics/nr]
637+
- context: metric
638+
condition: instrumentation_scope.name == "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/loadscraper"
639+
pipelines: [metrics/nr]
640+
- context: metric
641+
condition: instrumentation_scope.name == "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/diskscraper"
642+
pipelines: [metrics/nr]
643+
- context: metric
644+
condition: instrumentation_scope.name == "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/memoryscraper"
645+
pipelines: [metrics/nr]
646+
- context: metric
647+
condition: instrumentation_scope.name == "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/cpuscraper"
648+
pipelines: [metrics/nr]
649+
- context: metric
650+
condition: instrumentation_scope.name == "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/filesystemscraper"
651+
pipelines: [metrics/nr]
652+
- context: metric
653+
condition: instrumentation_scope.name == "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver"
654+
pipelines: [metrics/nr]
655+
- context: metric
656+
condition: instrumentation_scope.name == "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/kubeletstatsreceiver"
657+
pipelines: [metrics/nr]
622658
623659
service:
624660
{{- if include "newrelic.common.verboseLog" . }}
@@ -627,8 +663,10 @@ data:
627663
level: "debug"
628664
{{- end }}
629665
pipelines:
666+
{{- include "nrKubernetesOtel.daemonset.configMap.extraConfig.pipelines" . | nindent 8 }}
667+
630668
{{- if or .Values.receivers.hostmetrics.enabled (or .Values.receivers.kubeletstats.enabled .Values.receivers.prometheus.enabled) }}
631-
metrics:
669+
metrics/all_receivers:
632670
receivers:
633671
{{- if .Values.receivers.hostmetrics.enabled }}
634672
- hostmetrics
@@ -639,6 +677,11 @@ data:
639677
{{- if .Values.receivers.prometheus.enabled }}
640678
- prometheus
641679
{{- end }}
680+
exporters:
681+
- routing/nr_pipelines
682+
metrics/nr:
683+
receivers:
684+
- routing/nr_pipelines
642685
processors:
643686
- memory_limiter
644687
{{- if include "nrKubernetesOtel.lowDataMode" . }}
@@ -674,6 +717,14 @@ data:
674717
exporters:
675718
- otlphttp/newrelic
676719
{{- end }}
720+
metrics/default:
721+
receivers:
722+
- routing/nr_pipelines
723+
processors:
724+
- memory_limiter
725+
- batch
726+
exporters:
727+
- otlphttp/newrelic
677728
{{- if and .Values.receivers.filelog.enabled (not .Values.gkeAutopilot) }}
678729
logs:
679730
receivers:

‎charts/nr-k8s-otel-collector/templates/deployment-configmap.yaml

Lines changed: 38 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,11 @@ metadata:
77
{{- include "newrelic.common.labels" . | nindent 4 }}
88
data:
99
deployment-config.yaml: |
10-
{{- with include "nrKubernetesOtel.deployment.configMap.config" . }}
10+
{{- with include "nrKubernetesOtel.deployment.configMap.overrideConfig" . }}
1111
{{- . | nindent 4 }}
1212
{{- else }}
1313
receivers:
14+
{{- include "nrKubernetesOtel.deployment.configMap.extraConfig.receivers" . | nindent 6 }}
1415
otlp:
1516
protocols:
1617
http:
@@ -67,6 +68,8 @@ data:
6768
replacement: apiserver
6869
6970
processors:
71+
{{- include "nrKubernetesOtel.deployment.configMap.extraConfig.processors" . | nindent 6 }}
72+
7073
groupbyattrs:
7174
keys:
7275
- pod
@@ -557,6 +560,7 @@ data:
557560
send_batch_size : 800
558561
559562
exporters:
563+
{{- include "nrKubernetesOtel.deployment.configMap.extraConfig.exporters" . | nindent 6 }}
560564
otlphttp/newrelic:
561565
endpoint: {{ include "nrKubernetesOtel.endpoint" . }}
562566
headers:
@@ -565,6 +569,20 @@ data:
565569
verbosity: detailed
566570
sampling_initial: 5
567571
sampling_thereafter: 200
572+
573+
connectors:
574+
{{- include "nrKubernetesOtel.deployment.configMap.extraConfig.connectors" . | nindent 6 }}
575+
576+
routing/nr_pipelines:
577+
default_pipelines: [metrics/default]
578+
error_mode: propagate
579+
table:
580+
- context: datapoint
581+
condition: attributes["job_label"] == "kube-state-metrics"
582+
pipelines: [metrics/nr_ksm]
583+
- context: datapoint
584+
condition: attributes["job_label"] == "apiserver"
585+
pipelines: [metrics/nr_controlplane]
568586
569587
service:
570588
{{- if include "newrelic.common.verboseLog" . }}
@@ -573,10 +591,18 @@ data:
573591
level: "debug"
574592
{{- end }}
575593
pipelines:
594+
{{- include "nrKubernetesOtel.deployment.configMap.extraConfig.pipelines" . | nindent 8 }}
595+
576596
{{- if .Values.receivers.prometheus.enabled }}
577-
metrics/ksm:
597+
metrics/all_receivers:
578598
receivers:
579599
- prometheus/ksm
600+
- prometheus/controlplane
601+
exporters:
602+
- routing/nr_pipelines
603+
metrics/nr_ksm:
604+
receivers:
605+
- routing/nr_pipelines
580606
processors:
581607
- memory_limiter
582608
- metricstransform/kube_pod_status_phase
@@ -599,11 +625,9 @@ data:
599625
- batch
600626
exporters:
601627
- otlphttp/newrelic
602-
{{- end }}
603-
{{- if .Values.receivers.prometheus.enabled }}
604-
metrics:
628+
metrics/nr_controlplane:
605629
receivers:
606-
- prometheus/controlplane
630+
- routing/nr_pipelines
607631
processors:
608632
- memory_limiter
609633
- metricstransform/k8s_cluster_info
@@ -622,6 +646,14 @@ data:
622646
exporters:
623647
- otlphttp/newrelic
624648
{{- end }}
649+
metrics/default:
650+
receivers:
651+
- routing/nr_pipelines
652+
processors:
653+
- memory_limiter
654+
- batch
655+
exporters:
656+
- otlphttp/newrelic
625657
{{- if .Values.receivers.k8sEvents.enabled }}
626658
logs/events:
627659
receivers:

0 commit comments

Comments
 (0)