Skip to content

Ingest non-cluster flows through fluentd #3954

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jun 13, 2025
Merged

Ingest non-cluster flows through fluentd #3954

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
8801f70
Ingest non-cluster flows through fluentd
pasanw May 22, 2025
0420584
Update API for store-specific forwarding options
pasanw Jun 11, 2025
e2e693b
Update CRDs
pasanw Jun 11, 2025
722de08
Make host scope properly optional
pasanw Jun 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion api/v1/logcollector_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Copyright (c) 2020-2024 Tigera, Inc. All rights reserved.
// Copyright (c) 2020-2025 Tigera, Inc. All rights reserved.
/*

Licensed under the Apache License, Version 2.0 (the "License");
Expand Down Expand Up @@ -77,6 +77,9 @@ type AdditionalLogStoreSpec struct {
// If specified, enables exporting of flow, audit, and DNS logs to splunk.
// +optional
Splunk *SplunkStoreSpec `json:"splunk,omitempty"`
// If true, only logs from NonClusterHost instances will be forwarded to additional log stores defined in this spec.
// +optional
NonClusterLogsOnly bool `json:"nonClusterLogsOnly,omitempty"`
}
Copy link
Member

@rene-dekker rene-dekker Jun 3, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think twice about bool fields. Many ideas start as boolean but eventually trend towards a small set of mutually exclusive options. Plan for future expansions by describing the policy options explicitly as a string type alias (e.g. TerminationMessagePolicy). source: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md

Today, the only situation where we want to omit part of the data is nch vs standard, but in the future we may have other such scenarios. For this reason it may be better to think of a different field name with string type enum values that describe the modes that we want to support.


type AdditionalLogSourceSpec struct {
Expand Down
21 changes: 20 additions & 1 deletion pkg/controller/logcollector/logcollector_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import (
"fmt"
"strings"

"github.com/tigera/operator/pkg/dns"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/runtime"
Expand Down Expand Up @@ -157,6 +158,9 @@ func add(mgr manager.Manager, c ctrlruntime.Controller) error {
return fmt.Errorf("logcollector-controller failed to watch log-collector Tigerastatus: %w", err)
}

if err = c.WatchObject(&operatorv1.NonClusterHost{}, &handler.EnqueueRequestForObject{}); err != nil {
return fmt.Errorf("logcollector-controller failed to watch resource: %w", err)
}
return nil
}

Expand Down Expand Up @@ -374,7 +378,8 @@ func (r *ReconcileLogCollector) Reconcile(ctx context.Context, request reconcile
}

// fluentdKeyPair is the key pair fluentd presents to identify itself
fluentdKeyPair, err := certificateManager.GetOrCreateKeyPair(r.client, render.FluentdPrometheusTLSSecretName, common.OperatorNamespace(), []string{render.FluentdPrometheusTLSSecretName})
httpInputServiceNames := dns.GetServiceDNSNames(render.FluentdInputService, render.LogCollectorNamespace, r.clusterDomain)
fluentdKeyPair, err := certificateManager.GetOrCreateKeyPair(r.client, render.FluentdPrometheusTLSSecretName, common.OperatorNamespace(), append([]string{render.FluentdPrometheusTLSSecretName}, httpInputServiceNames...))
if err != nil {
r.status.SetDegraded(operatorv1.ResourceCreateError, "Error creating TLS certificate", err, reqLogger)
return reconcile.Result{}, err
Expand Down Expand Up @@ -563,6 +568,19 @@ func (r *ReconcileLogCollector) Reconcile(ctx context.Context, request reconcile
return reconcile.Result{}, err
}

// Check if non-cluster host feature is enabled.
nonclusterhost, err := utils.GetNonClusterHost(ctx, r.client)
if err != nil {
r.status.SetDegraded(operatorv1.ResourceReadError, "Failed to query NonClusterHost resource", err, reqLogger)
return reconcile.Result{}, err
}
if nonclusterhost != nil {
if _, _, _, err := url.ParseEndpoint(nonclusterhost.Spec.Endpoint); err != nil {
r.status.SetDegraded(operatorv1.ResourceReadError, "Failed to read parse endpoint from NonClusterHost resource", err, reqLogger)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be helpful to add nonclusterhost.Spec.Endpoint to the message for the user's convenience.

return reconcile.Result{}, err
}
}

// Create a component handler to manage the rendered component.
handler := utils.NewComponentHandler(log, r.client, r.scheme, instance)

Expand All @@ -585,6 +603,7 @@ func (r *ReconcileLogCollector) Reconcile(ctx context.Context, request reconcile
ExternalElastic: r.externalElastic,
EKSLogForwarderKeyPair: eksLogForwarderKeyPair,
PacketCapture: packetcaptureapi,
NonClusterHost: nonclusterhost,
}
// Render the fluentd component for Linux
comp := render.Fluentd(fluentdCfg)
Expand Down
33 changes: 30 additions & 3 deletions pkg/crds/enterprise/crd.projectcalico.org_felixconfigurations.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1013,6 +1013,14 @@ spec:
FlowLogGoldmaneServer is the flow server endpoint to
which flow data should be published.
type: string
flowLogsLocalReporter:
description:
"FlowLogsLocalReporter configures local unix socket for
reporting flow data from each node. [Default: Disabled]"
enum:
- Disabled
- Enabled
type: string
flowLogsMaxOriginalIPsIncluded:
description:
FlowLogsMaxOriginalIPsIncluded specifies the number of
Expand Down Expand Up @@ -1428,19 +1436,19 @@ spec:
description:
"LogSeverityFile is the log severity above which logs
are sent to the log file. [Default: Info]"
pattern: ^(?i)(Debug|Info|Warning|Error|Fatal)?$
pattern: ^(?i)(Trace|Debug|Info|Warning|Error|Fatal)?$
type: string
logSeverityScreen:
description:
"LogSeverityScreen is the log severity above which logs
are sent to the stdout. [Default: Info]"
pattern: ^(?i)(Debug|Info|Warning|Error|Fatal)?$
pattern: ^(?i)(Trace|Debug|Info|Warning|Error|Fatal)?$
type: string
logSeveritySys:
description: |-
LogSeveritySys is the log severity above which logs are sent to the syslog. Set to None for no logging to syslog.
[Default: Info]
pattern: ^(?i)(Debug|Info|Warning|Error|Fatal)?$
pattern: ^(?i)(Trace|Debug|Info|Warning|Error|Fatal)?$
type: string
maxIpsetSize:
description: |-
Expand Down Expand Up @@ -1472,6 +1480,17 @@ spec:
is leaving the network. By default the address used is an address on the interface the traffic is leaving on
(i.e. it uses the iptables MASQUERADE target).
type: string
natOutgoingExclusions:
description: |-
When a IP pool setting `natOutgoing` is true, packets sent from Calico networked containers in this IP pool to destinations will be masqueraded.
Configure which type of destinations is excluded from being masqueraded.
- IPPoolsOnly: destinations outside of this IP pool will be masqueraded.
- IPPoolsAndHostIPs: destinations outside of this IP pool and all hosts will be masqueraded.
[Default: IPPoolsOnly]
enum:
- IPPoolsOnly
- IPPoolsAndHostIPs
type: string
natPortRange:
anyOf:
- type: integer
Expand Down Expand Up @@ -1564,6 +1583,14 @@ spec:
PolicySyncPathPrefix is used to by Felix to communicate policy changes to external services,
like Application layer policy. [Default: Empty]
type: string
programRoutes:
description: |-
ProgramRoutes specifies whether Felix should program IPIP or unencapsulated routes instead of BIRD.
Felix always programs VXLAN routes. [Default: Disabled]
enum:
- Enabled
- Disabled
type: string
prometheusGoMetricsEnabled:
description: |-
PrometheusGoMetricsEnabled disables Go runtime metrics collection, which the Prometheus client does by default, when
Expand Down
34 changes: 34 additions & 0 deletions pkg/crds/enterprise/crd.projectcalico.org_globalnetworkpolicies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,23 @@ spec:
type: object
http:
properties:
headers:
items:
properties:
header:
type: string
operator:
type: string
values:
items:
type: string
type: array
required:
- header
- operator
- values
type: object
type: array
methods:
items:
type: string
Expand Down Expand Up @@ -256,6 +273,23 @@ spec:
type: object
http:
properties:
headers:
items:
properties:
header:
type: string
operator:
type: string
values:
items:
type: string
type: array
required:
- header
- operator
- values
type: object
type: array
methods:
items:
type: string
Expand Down
2 changes: 2 additions & 0 deletions pkg/crds/enterprise/crd.projectcalico.org_kubecontrollersconfigurations.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ spec:
items:
properties:
generateName:
maxLength: 253
type: string
interfaceCIDRs:
items:
Expand Down Expand Up @@ -143,6 +144,7 @@ spec:
items:
properties:
generateName:
maxLength: 253
type: string
interfaceCIDRs:
items:
Expand Down
34 changes: 34 additions & 0 deletions pkg/crds/enterprise/crd.projectcalico.org_networkpolicies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,23 @@ spec:
type: object
http:
properties:
headers:
items:
properties:
header:
type: string
operator:
type: string
values:
items:
type: string
type: array
required:
- header
- operator
- values
type: object
type: array
methods:
items:
type: string
Expand Down Expand Up @@ -252,6 +269,23 @@ spec:
type: object
http:
properties:
headers:
items:
properties:
header:
type: string
operator:
type: string
values:
items:
type: string
type: array
required:
- header
- operator
- values
type: object
type: array
methods:
items:
type: string
Expand Down
34 changes: 34 additions & 0 deletions pkg/crds/enterprise/crd.projectcalico.org_stagedglobalnetworkpolicies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,23 @@ spec:
type: object
http:
properties:
headers:
items:
properties:
header:
type: string
operator:
type: string
values:
items:
type: string
type: array
required:
- header
- operator
- values
type: object
type: array
methods:
items:
type: string
Expand Down Expand Up @@ -256,6 +273,23 @@ spec:
type: object
http:
properties:
headers:
items:
properties:
header:
type: string
operator:
type: string
values:
items:
type: string
type: array
required:
- header
- operator
- values
type: object
type: array
methods:
items:
type: string
Expand Down
34 changes: 34 additions & 0 deletions pkg/crds/enterprise/crd.projectcalico.org_stagednetworkpolicies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,23 @@ spec:
type: object
http:
properties:
headers:
items:
properties:
header:
type: string
operator:
type: string
values:
items:
type: string
type: array
required:
- header
- operator
- values
type: object
type: array
methods:
items:
type: string
Expand Down Expand Up @@ -252,6 +269,23 @@ spec:
type: object
http:
properties:
headers:
items:
properties:
header:
type: string
operator:
type: string
values:
items:
type: string
type: array
required:
- header
- operator
- values
type: object
type: array
methods:
items:
type: string
Expand Down
5 changes: 5 additions & 0 deletions pkg/crds/operator/operator.tigera.io_logcollectors.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,11 @@ spec:
Configuration for exporting flow, audit, and DNS logs
to external storage.
properties:
nonClusterLogsOnly:
description:
If true, only logs from NonClusterHost instances
will be forwarded to additional log stores defined in this spec.
type: boolean
s3:
description:
If specified, enables exporting of flow, audit, and
Expand Down
Loading
Loading