NETOBSERV-1377: Add controller to deploy netobserv network policy #690

OlivierCazade · 2024-06-25T14:51:28Z

Description

Add controller to deploy netobserv network policy

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

codecov · 2024-06-25T15:33:13Z

Codecov Report

Attention: Patch coverage is 62.42038% with 59 lines in your changes missing coverage. Please review.

Project coverage is 65.98%. Comparing base (bc0bcef) to head (5a3c92b).
Report is 31 commits behind head on main.

Files	Patch %	Lines
controllers/networkpolicy/np_objects.go	58.88%	35 Missing and 2 partials ⚠️
...pis/flowcollector/v1beta2/zz_generated.deepcopy.go	43.75%	9 Missing ⚠️
controllers/reconcilers/reconcilers.go	46.66%	8 Missing ⚠️
controllers/networkpolicy/np_controller.go	86.11%	2 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #690      +/-   ##
==========================================
- Coverage   66.60%   65.98%   -0.62%     
==========================================
  Files          70       74       +4     
  Lines        8115     8541     +426     
==========================================
+ Hits         5405     5636     +231     
- Misses       2315     2503     +188     
- Partials      395      402       +7

Flag	Coverage Δ
unittests	`65.98% <62.42%> (-0.62%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
...s/flowcollector/v1beta1/zz_generated.conversion.go	`39.27% <ø> (ø)`
apis/flowcollector/v1beta2/flowcollector_types.go	`100.00% <ø> (ø)`
pkg/manager/manager.go	`66.66% <ø> (ø)`
pkg/manager/status/status_manager.go	`89.47% <ø> (+1.75%)`	⬆️
controllers/networkpolicy/np_controller.go	`86.11% <86.11%> (ø)`
controllers/reconcilers/reconcilers.go	`59.39% <46.66%> (-1.62%)`	⬇️
...pis/flowcollector/v1beta2/zz_generated.deepcopy.go	`42.72% <43.75%> (-1.05%)`	⬇️
controllers/networkpolicy/np_objects.go	`58.88% <58.88%> (ø)`

... and 17 files with indirect coverage changes

jotak · 2024-06-26T08:38:06Z

apis/flowcollector/v1beta2/flowcollector_types.go

+	NetworkPolicy NetworkPolycy `json:"networkPolicy,omitempty"`
+}
+
+type NetworkPolycy struct {


jotak · 2024-06-26T08:43:56Z

apis/flowcollector/v1beta2/flowcollector_types.go

+}
+
+type NetworkPolycy struct {
+	// Set `deploy` to `false` to disable network policy deployment. It is enabled by default.


We could add more information about what this policy does, such as:

This network policy better isolates the NetObserv components to prevent undesired connections to them. It is recommended to install it.

(or something in that vein)

Also, shouldn't we name that Enable, like we do generally for anything that can be turned on/off ?

jotak · 2024-06-26T08:44:54Z

apis/flowcollector/v1beta2/flowcollector_types.go

+	// +optional
+	Deploy *bool `json:"deploy,omitempty"`
+
+	// `additionalNamespaces` contains the interface names from where flows are collected. If empty, the agent


hmm I guess it's not the good text here :-)

We should mention here in particular what users need to care about:

If they use Kafka and installed it in a separate namespace, they must add it here

Same for any exporter

We can expect users misconfigurations here... Probably something that we'll have to worry about in docs & troubleshooting

Noted as something to add to docs :)

I think in @Amoghrd's testing having Loki and Kafka in separate NS didn't turn out to be a problem since this was an Ingress policy and only affecting inbound connections.

Cool, thanks Mehul!

jotak · 2024-06-26T08:47:48Z

controllers/networkpolicy/np_controller.go

+	return ctrl.NewControllerManagedBy(mgr).
+		For(&flowslatest.FlowCollector{}, reconcilers.IgnoreStatusChange).
+		Named("networkPolicy").
+		Owns(&corev1.Namespace{}).


I don't think Namespace is required here?

jotak · 2024-06-26T08:52:23Z

controllers/networkpolicy/np_controller.go

+	client.Client
+	mgr              *manager.Manager
+	status           status.Instance
+	currentNamespace string


looking at the code it doesn't seem that currentNamespace is useful here

jotak · 2024-06-26T09:59:22Z

controllers/networkpolicy/np_controller_test.go

+						Advanced: &flowslatest.AdvancedProcessorConfig{
+							Env: map[string]string{
+								"GOGC": "200",
+							},
+							ConversationHeartbeatInterval: &metav1.Duration{
+								Duration: conntrackHeartbeatInterval,
+							},
+							ConversationEndTimeout: &metav1.Duration{
+								Duration: conntrackEndTimeout,
+							},
+							ConversationTerminatingTimeout: &metav1.Duration{
+								Duration: conntrackTerminatingTimeout,
+							},
+						},


I don't think conn-track settings has any use here, right? I guess you can remove them

jpinsonneau

Code looks good ! Thanks @OlivierCazade

Amoghrd · 2024-07-24T17:33:20Z

/ok-to-test

github-actions · 2024-07-24T17:36:35Z

New images:

quay.io/netobserv/network-observability-operator:8afb9c0
quay.io/netobserv/network-observability-operator-bundle:v0.0.0-8afb9c0
quay.io/netobserv/network-observability-operator-catalog:v0.0.0-8afb9c0

They will expire after two weeks.

To deploy this build:

# Direct deployment, from operator repo
IMAGE=quay.io/netobserv/network-observability-operator:8afb9c0 make deploy

# Or using operator-sdk
operator-sdk run bundle quay.io/netobserv/network-observability-operator-bundle:v0.0.0-8afb9c0

Or as a Catalog Source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-8afb9c0
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

Amoghrd · 2024-07-24T19:22:22Z

@OlivierCazade networkPolicy seems to be disabled by default

controllers/networkpolicy/np_objects.go

jotak · 2024-07-25T10:25:17Z

pkg/manager/manager.go

@@ -30,6 +30,7 @@ import (
 //+kubebuilder:rbac:groups=monitoring.coreos.com,resources=servicemonitors;prometheusrules,verbs=get;create;delete;update;patch;list;watch
 //+kubebuilder:rbac:groups=config.openshift.io,resources=clusterversions;networks,verbs=get;list;watch
 //+kubebuilder:rbac:groups=loki.grafana.com,resources=network,resourceNames=logs,verbs=get;create
+//+kubebuilder:rbac:groups=networking.k8s.io,resources=networkpolicies,verbs=get;list;watch;create;update


delete is missing here; and I'd also add patch, just in case (we never know exactly what the controller-runtime client does under the cover)

Suggested change

//+kubebuilder:rbac:groups=networking.k8s.io,resources=networkpolicies,verbs=get;list;watch;create;update

//+kubebuilder:rbac:groups=networking.k8s.io,resources=networkpolicies,verbs=get;list;watch;create;update;patch;delete

- Add DELETE and PATCH roles to operator - Make additionalNamespaces only affect the main namespace - Add allowed namespaces based on context knowledge (is openshift? is console enabled? etc.) - Add port restriction for console access

jotak · 2024-07-26T10:44:56Z

hey @Amoghrd I think this is ok to test
/lgtm

openshift-ci-robot · 2024-07-26T10:45:25Z

Amoghrd · 2024-07-26T13:16:26Z

/ok-to-test

github-actions · 2024-07-26T13:19:32Z

New images:

quay.io/netobserv/network-observability-operator:e618cc3
quay.io/netobserv/network-observability-operator-bundle:v0.0.0-e618cc3
quay.io/netobserv/network-observability-operator-catalog:v0.0.0-e618cc3

They will expire after two weeks.

To deploy this build:

# Direct deployment, from operator repo
IMAGE=quay.io/netobserv/network-observability-operator:e618cc3 make deploy

# Or using operator-sdk
operator-sdk run bundle quay.io/netobserv/network-observability-operator-bundle:v0.0.0-e618cc3

Or as a Catalog Source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-e618cc3
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

Amoghrd · 2024-07-31T13:13:02Z

/label qe-approved

openshift-ci-robot · 2024-07-31T13:13:06Z

memodi

could we clarify its the Ingress policy that it deploys?

bundle/manifests/flows.netobserv.io_flowcollectors.yaml

config/crd/bases/flows.netobserv.io_flowcollectors.yaml

jotak · 2024-08-01T14:40:55Z

/lgtm
/approve

openshift-ci · 2024-08-01T14:41:13Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jotak

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [jotak]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci · 2024-08-01T14:41:19Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jotak

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [jotak]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

OlivierCazade force-pushed the network-policy branch 2 times, most recently from bbc161d to ea0cfec Compare June 25, 2024 15:16

OlivierCazade force-pushed the network-policy branch from ea0cfec to 1ce7586 Compare June 25, 2024 15:51

OlivierCazade requested a review from jotak June 26, 2024 08:17

jotak reviewed Jun 26, 2024

View reviewed changes

OlivierCazade force-pushed the network-policy branch from 1ce7586 to f8d5c37 Compare June 28, 2024 16:20

Add controller to deploy netobserv network policy

73bf6b9

OlivierCazade force-pushed the network-policy branch from f8d5c37 to 73bf6b9 Compare June 28, 2024 16:24

OlivierCazade requested a review from jotak June 28, 2024 18:16

jpinsonneau approved these changes Jul 11, 2024

View reviewed changes

openshift-ci bot assigned jpinsonneau Jul 11, 2024

openshift-ci bot added the lgtm label Jul 11, 2024

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Jul 24, 2024

jotak reviewed Jul 25, 2024

View reviewed changes

controllers/networkpolicy/np_objects.go Show resolved Hide resolved

jotak reviewed Jul 25, 2024

View reviewed changes

netpol updates:

3a92926

- Add DELETE and PATCH roles to operator - Make additionalNamespaces only affect the main namespace - Add allowed namespaces based on context knowledge (is openshift? is console enabled? etc.) - Add port restriction for console access

openshift-ci bot removed the lgtm label Jul 25, 2024

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Jul 25, 2024

fix test

5a3c92b

openshift-ci bot assigned jotak Jul 26, 2024

openshift-ci bot added the lgtm label Jul 26, 2024

jotak changed the title ~~NETOBSERV-1377 Add controller to deploy netobserv network policy~~ NETOBSERV-1377: Add controller to deploy netobserv network policy Jul 26, 2024

openshift-ci-robot added the jira/valid-reference label Jul 26, 2024

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Jul 26, 2024

openshift-ci bot added the qe-approved QE has approved this pull request label Jul 31, 2024

memodi reviewed Jul 31, 2024

View reviewed changes

bundle/manifests/flows.netobserv.io_flowcollectors.yaml Outdated Show resolved Hide resolved

config/crd/bases/flows.netobserv.io_flowcollectors.yaml Outdated Show resolved Hide resolved

clarify it's an ingress policy

7cce8ac

openshift-ci bot removed the lgtm label Aug 1, 2024

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Aug 1, 2024

openshift-ci bot added the lgtm label Aug 1, 2024

openshift-ci bot added the approved label Aug 1, 2024

openshift-merge-bot bot merged commit d14d142 into netobserv:main Aug 1, 2024
8 of 9 checks passed

	//+kubebuilder:rbac:groups=networking.k8s.io,resources=networkpolicies,verbs=get;list;watch;create;update
	//+kubebuilder:rbac:groups=networking.k8s.io,resources=networkpolicies,verbs=get;list;watch;create;update;patch;delete

NETOBSERV-1377: Add controller to deploy netobserv network policy #690

NETOBSERV-1377: Add controller to deploy netobserv network policy #690

Uh oh!

Conversation

OlivierCazade commented Jun 25, 2024

Description

Dependencies

Checklist

Uh oh!

codecov bot commented Jun 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jpinsonneau left a comment

Choose a reason for hiding this comment

Uh oh!

Amoghrd commented Jul 24, 2024

Uh oh!

github-actions bot commented Jul 24, 2024

Uh oh!

Amoghrd commented Jul 24, 2024

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jotak commented Jul 26, 2024

Uh oh!

openshift-ci-robot commented Jul 26, 2024 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Dependencies

Checklist

Uh oh!

Amoghrd commented Jul 26, 2024

Uh oh!

github-actions bot commented Jul 26, 2024

Uh oh!

Amoghrd commented Jul 31, 2024

Uh oh!

openshift-ci-robot commented Jul 31, 2024 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Dependencies

Checklist

Uh oh!

memodi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jotak commented Aug 1, 2024

Uh oh!

openshift-ci bot commented Aug 1, 2024

Uh oh!

openshift-ci bot commented Aug 1, 2024

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Jun 25, 2024 •

edited

Loading

openshift-ci-robot commented Jul 26, 2024 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Jul 31, 2024 •

edited by openshift-ci bot

Loading