Ingest non-cluster flows through fluentd #3954
Conversation
This changeset facilitates voltron sending non-cluster flows to fluentd, rather than Linseed. This change is being made to enable support for forwarding of non-cluster flows to additional log stores like Splunk, since fluentd already supports such forwarding. When non-cluster hosts are enabled, the operator will now create a Service that exposes the HTTP ingestion point of fluentd, and creates the network policy and certificate updates required to allow this network flow. The operator also supports a new API option to forward only non-cluster flow logs to additional log stores.
| } | ||
| if nonclusterhost != nil { | ||
| if _, _, _, err := url.ParseEndpoint(nonclusterhost.Spec.Endpoint); err != nil { | ||
| r.status.SetDegraded(operatorv1.ResourceReadError, "Failed to read parse endpoint from NonClusterHost resource", err, reqLogger) |
There was a problem hiding this comment.
Might be helpful to add nonclusterhost.Spec.Endpoint to the message for the user's convenience.
api/v1/logcollector_types.go
Outdated
| @@ -77,6 +77,9 @@ type AdditionalLogStoreSpec struct { | |||
| // If specified, enables exporting of flow, audit, and DNS logs to splunk. | |||
| // +optional | |||
| Splunk *SplunkStoreSpec `json:"splunk,omitempty"` | |||
| // If true, only logs from NonClusterHost instances will be forwarded to additional log stores defined in this spec. | |||
| // +optional | |||
| NonClusterLogsOnly bool `json:"nonClusterLogsOnly,omitempty"` | |||
There was a problem hiding this comment.
Think twice about bool fields. Many ideas start as boolean but eventually trend towards a small set of mutually exclusive options. Plan for future expansions by describing the policy options explicitly as a string type alias (e.g. TerminationMessagePolicy). source: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md
Today, the only situation where we want to omit part of the data is nch vs standard, but in the future we may have other such scenarios. For this reason it may be better to think of a different field name with string type enum values that describe the modes that we want to support.
| . "github.com/onsi/ginkgo" | ||
| . "github.com/onsi/ginkgo/extensions/table" | ||
| . "github.com/onsi/gomega" | ||
| "github.com/tigera/operator/pkg/render/common/networkpolicy" |
Description
This changeset facilitates voltron sending non-cluster flows to fluentd, rather than Linseed. This change is being made to enable support for forwarding of non-cluster flows to additional log stores like Splunk, since fluentd already supports such forwarding. When non-cluster hosts are enabled, the operator will now create a Service that exposes the HTTP ingestion point of fluentd, and creates the network policy and certificate updates required to allow this network flow. The operator also supports a new API option to forward only non-cluster flow logs to additional log stores.
For PR author
make gen-filesmake gen-versionsFor PR reviewers
A note for code reviewers - all pull requests must have the following:
kind/bugif this is a bugfix.kind/enhancementif this is a a new feature.enterpriseif this PR applies to Calico Enterprise only.