NETOBSERV-1642: Ovs monitoring feature #664

msherif1234 · 2024-05-29T20:20:41Z

Description

changes to support ovs monitoring config and console bits

FC config to enable ovs monitoring fearure

agent:
    type: eBPF
    ebpf:   
      privileged: true
      features:
       - "OvsMonitor"

Dependencies

netobserv/netobserv-ebpf-agent#286

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

openshift-ci-robot · 2024-05-29T20:20:45Z

openshift-ci-robot · 2024-05-29T20:21:21Z

jpinsonneau · 2024-05-30T11:43:43Z

controllers/consoleplugin/config/static-frontend-config.yaml

+  - name: OvsMonitorMD[0]
+    type: string
+    description: OVS monitor metadata for the 1st event
+    cardinalityWarn: avoid


You should also add these into columns and filters such as:

columns: - id: OvsMonitorMD0 name: 1st OVS metadata tooltip: OVS monitor metadata for the 1st event field: OvsMonitorMD[0] filter: ovs_metadata_0 default: true width: 5 feature: ovsMonitor ... filters: - id: ovs_metadata_0 name: OVS monitor metadata for the 1st event component: text hint: Specify a single metadata. ...

if we want to make somether smarter that concatenate the events, it will require some code changes in the plugin

I believe there is still a need for plugin work anyway to handle the new feature if that is accurate optimizing the above can be part of that too ? for now I will see how far this takes me

updated both agent & operator PRs to allow single field with an array
netobserv/netobserv-ebpf-agent#286 (comment)

jpinsonneau · 2024-05-30T11:57:51Z

controllers/constants/constants.go

+	LokiCRWriter        = "netobserv-writer"
+	LokiCRBWriter       = "netobserv-writer-flp"
+	LokiCRReader        = "netobserv-reader"
+	MaxOvsMonitorEvents = 4


I'm not sure to get the impact of this yet. What would be the advantage to get more / less events metadatas ?

for the same flow we can have multiple events md if we wanted to trace different feature for example we wanted to trace ACL, NAT, QoS for example we will see array populated with 4 metadata strings each for one of the above features, while if we trace ACL only for example then only one event md will be populated

ok makes sense. So are we going to expose which features to trace in the FlowCollector CR ?

If so we could adapt the console plugin config dynamically and bind each field with its proper name / type etc

msherif1234 · 2024-05-30T12:11:03Z

/ok-to-test

github-actions · 2024-05-30T12:14:31Z

New images:

quay.io/netobserv/network-observability-operator:90f3501
quay.io/netobserv/network-observability-operator-bundle:v0.0.0-90f3501
quay.io/netobserv/network-observability-operator-catalog:v0.0.0-90f3501

They will expire after two weeks.

To deploy this build:

# Direct deployment, from operator repo
IMAGE=quay.io/netobserv/network-observability-operator:90f3501 make deploy

# Or using operator-sdk
operator-sdk run bundle quay.io/netobserv/network-observability-operator-bundle:v0.0.0-90f3501

Or as a Catalog Source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-90f3501
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

codecov · 2024-05-30T12:22:34Z

Codecov Report

Attention: Patch coverage is 7.84314% with 47 lines in your changes missing coverage. Please review.

Project coverage is 66.88%. Comparing base (690bb50) to head (75b75f2).

Files with missing lines	Patch %	Lines
controllers/ebpf/agent_controller.go	4.87%	36 Missing and 3 partials ⚠️
controllers/flp/flp_pipeline_builder.go	0.00%	5 Missing and 1 partial ⚠️
controllers/consoleplugin/consoleplugin_objects.go	0.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #664      +/-   ##
==========================================
- Coverage   67.21%   66.88%   -0.33%     
==========================================
  Files          74       74              
  Lines        8598     8649      +51     
==========================================
+ Hits         5779     5785       +6     
- Misses       2416     2457      +41     
- Partials      403      407       +4

Flag	Coverage Δ
unittests	`66.88% <7.84%> (-0.33%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
apis/flowcollector/v1beta1/flowcollector_types.go	`100.00% <ø> (ø)`
apis/flowcollector/v1beta2/flowcollector_types.go	`100.00% <ø> (ø)`
pkg/helper/flowcollector.go	`83.33% <100.00%> (+0.15%)`	⬆️
controllers/consoleplugin/consoleplugin_objects.go	`90.40% <0.00%> (-0.44%)`	⬇️
controllers/flp/flp_pipeline_builder.go	`83.67% <0.00%> (-1.31%)`	⬇️
controllers/ebpf/agent_controller.go	`53.95% <4.87%> (-5.01%)`	⬇️

... and 1 file with indirect coverage changes

msherif1234 · 2024-05-31T19:36:09Z

/ok-to-test

github-actions · 2024-05-31T19:39:28Z

New images:

quay.io/netobserv/network-observability-operator:495b735
quay.io/netobserv/network-observability-operator-bundle:v0.0.0-495b735
quay.io/netobserv/network-observability-operator-catalog:v0.0.0-495b735

They will expire after two weeks.

To deploy this build:

# Direct deployment, from operator repo
IMAGE=quay.io/netobserv/network-observability-operator:495b735 make deploy

# Or using operator-sdk
operator-sdk run bundle quay.io/netobserv/network-observability-operator-bundle:v0.0.0-495b735

Or as a Catalog Source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-495b735
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

msherif1234 · 2024-06-03T14:39:20Z

/ok-to-test

github-actions · 2024-06-03T14:42:54Z

New images:

quay.io/netobserv/network-observability-operator:32ec69e
quay.io/netobserv/network-observability-operator-bundle:v0.0.0-32ec69e
quay.io/netobserv/network-observability-operator-catalog:v0.0.0-32ec69e

They will expire after two weeks.

To deploy this build:

# Direct deployment, from operator repo
IMAGE=quay.io/netobserv/network-observability-operator:32ec69e make deploy

# Or using operator-sdk
operator-sdk run bundle quay.io/netobserv/network-observability-operator-bundle:v0.0.0-32ec69e

Or as a Catalog Source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-32ec69e
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

openshift-ci-robot · 2024-06-04T11:54:16Z

msherif1234 · 2024-07-15T12:55:04Z

/ok-to-test

msherif1234 · 2024-08-12T13:06:37Z

/ok-to-test

msherif1234 · 2024-08-12T13:10:51Z

/ok-to-test

github-actions · 2024-08-12T13:14:09Z

New images:

quay.io/netobserv/network-observability-operator:5890842
quay.io/netobserv/network-observability-operator-bundle:v0.0.0-5890842
quay.io/netobserv/network-observability-operator-catalog:v0.0.0-5890842

They will expire after two weeks.

To deploy this build:

# Direct deployment, from operator repo
IMAGE=quay.io/netobserv/network-observability-operator:5890842 make deploy

# Or using operator-sdk
operator-sdk run bundle quay.io/netobserv/network-observability-operator-bundle:v0.0.0-5890842

Or as a Catalog Source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-5890842
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

jotak · 2024-08-14T13:19:47Z

apis/flowcollector/v1beta1/flowcollector_types.go

-// - `FlowRTT`, to track TCP latency. [Unsupported (*)].<br>
-// +kubebuilder:validation:Enum:="PacketDrop";"DNSTracking";"FlowRTT"
+// - `FlowRTT`, to track TCP latency [Unsupported (*)].<br>
+// - `NetworkEventsMonitor`, to track Network events.<br>


just wondering if naming it just NetworkEvents would be sufficient and easier to remember, no ?

jotak · 2024-08-14T13:24:55Z

apis/flowcollector/v1beta1/flowcollector_types.go

@@ -325,6 +327,7 @@ type FlowCollectorEBPF struct {
 	// If the `spec.agent.ebpf.privileged` parameter is not set, an error is reported.<br>
 	// - `DNSTracking`: enable the DNS tracking feature.<br>
 	// - `FlowRTT`: enable flow latency (sRTT) extraction in the eBPF agent from TCP traffic.<br>
+	// - `NetworkEventsMonitor`: enable the Network events monitoring feature.<br>


like in PacketDrop, we should mention here if Privileged is required
Also, maybe add more details right here because it's not super obvious what are network events. like ".. such as network policy verdicts" ?

msherif1234 · 2024-08-17T02:32:16Z

/ok-to-test

github-actions · 2024-08-17T02:35:26Z

New images:

quay.io/netobserv/network-observability-operator:75f3f4c
quay.io/netobserv/network-observability-operator-bundle:v0.0.0-75f3f4c
quay.io/netobserv/network-observability-operator-catalog:v0.0.0-75f3f4c

They will expire after two weeks.

To deploy this build:

# Direct deployment, from operator repo
IMAGE=quay.io/netobserv/network-observability-operator:75f3f4c make deploy

# Or using operator-sdk
operator-sdk run bundle quay.io/netobserv/network-observability-operator-bundle:v0.0.0-75f3f4c

Or as a Catalog Source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-75f3f4c
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

Signed-off-by: Mohamed Mahmoud <[email protected]>

msherif1234 · 2024-09-04T17:15:40Z

/ok-to-test

github-actions · 2024-09-04T17:19:41Z

New images:

quay.io/netobserv/network-observability-operator:f0c1ef3
quay.io/netobserv/network-observability-operator-bundle:v0.0.0-f0c1ef3
quay.io/netobserv/network-observability-operator-catalog:v0.0.0-f0c1ef3

They will expire after two weeks.

To deploy this build:

# Direct deployment, from operator repo
IMAGE=quay.io/netobserv/network-observability-operator:f0c1ef3 make deploy

# Or using operator-sdk
operator-sdk run bundle quay.io/netobserv/network-observability-operator-bundle:v0.0.0-f0c1ef3

Or as a Catalog Source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-f0c1ef3
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

msherif1234 · 2024-09-05T12:54:40Z

/approve

openshift-ci · 2024-09-05T12:54:45Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: msherif1234

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [msherif1234]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot added the jira/valid-reference label May 29, 2024

openshift-ci bot added the do-not-merge/work-in-progress label May 29, 2024

msherif1234 marked this pull request as draft May 29, 2024 20:25

jpinsonneau reviewed May 30, 2024

View reviewed changes

msherif1234 force-pushed the ovs_monitor branch from b8c4325 to 49c4081 Compare May 30, 2024 12:10

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label May 30, 2024

msherif1234 force-pushed the ovs_monitor branch from 49c4081 to 014fdda Compare May 31, 2024 19:33

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label May 31, 2024

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label May 31, 2024

msherif1234 force-pushed the ovs_monitor branch from 014fdda to d14bb5f Compare June 3, 2024 14:15

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Jun 3, 2024

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Jun 3, 2024

jpinsonneau mentioned this pull request Jun 3, 2024

NETOBSERV-1636: add OvsMonitorMDs as field array netobserv/network-observability-console-plugin#535

Merged

10 tasks

msherif1234 mentioned this pull request Jun 3, 2024

NETOBSERV-1637: OVS monitoring ebpf hook netobserv/netobserv-ebpf-agent#286

Merged

11 tasks

openshift-merge-robot added the needs-rebase label Jun 12, 2024

msherif1234 force-pushed the ovs_monitor branch from 582052d to 87a3a90 Compare June 24, 2024 11:58

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Jun 24, 2024

openshift-merge-robot removed the needs-rebase label Jun 24, 2024

msherif1234 marked this pull request as ready for review August 9, 2024 11:12

msherif1234 changed the title ~~WIP: NETOBSERV-1642: Ovs monitoring feature~~ NETOBSERV-1642: Ovs monitoring feature Aug 9, 2024

openshift-ci bot removed the do-not-merge/work-in-progress label Aug 9, 2024

msherif1234 removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Aug 12, 2024

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Aug 12, 2024

jotak reviewed Aug 14, 2024

View reviewed changes

msherif1234 force-pushed the ovs_monitor branch from 4b420a6 to 7d40f20 Compare August 15, 2024 00:35

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Aug 15, 2024

msherif1234 force-pushed the ovs_monitor branch from 7d40f20 to ffd1dc3 Compare August 16, 2024 21:19

msherif1234 requested a review from jotak August 16, 2024 21:20

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Aug 17, 2024

NETOBSERV-1642: Ovs monitoring feature

75b75f2

Signed-off-by: Mohamed Mahmoud <[email protected]>

msherif1234 force-pushed the ovs_monitor branch from ffd1dc3 to 75b75f2 Compare September 4, 2024 17:13

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Sep 4, 2024

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Sep 4, 2024

openshift-ci bot added the approved label Sep 5, 2024

jpinsonneau approved these changes Sep 5, 2024

View reviewed changes

openshift-ci bot assigned jpinsonneau Sep 5, 2024

openshift-ci bot added the lgtm label Sep 5, 2024

openshift-merge-bot bot merged commit 8807110 into netobserv:main Sep 5, 2024
18 checks passed

NETOBSERV-1642: Ovs monitoring feature #664

NETOBSERV-1642: Ovs monitoring feature #664

Uh oh!

Conversation

msherif1234 commented May 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Dependencies

Checklist

Uh oh!

openshift-ci-robot commented May 29, 2024 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Dependencies

Checklist

Uh oh!

openshift-ci-robot commented May 29, 2024 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Dependencies

Checklist

Uh oh!

jpinsonneau May 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

msherif1234 commented May 30, 2024

Uh oh!

github-actions bot commented May 30, 2024

Uh oh!

codecov bot commented May 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

msherif1234 commented May 31, 2024

Uh oh!

github-actions bot commented May 31, 2024

Uh oh!

msherif1234 commented Jun 3, 2024

Uh oh!

github-actions bot commented Jun 3, 2024

Uh oh!

openshift-ci-robot commented Jun 4, 2024 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Dependencies

Checklist

Uh oh!

msherif1234 commented Jul 15, 2024

Uh oh!

msherif1234 commented Aug 12, 2024

Uh oh!

msherif1234 commented Aug 12, 2024

Uh oh!

github-actions bot commented Aug 12, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

msherif1234 commented Aug 17, 2024

Uh oh!

github-actions bot commented Aug 17, 2024

Uh oh!

msherif1234 commented Sep 4, 2024

Uh oh!

github-actions bot commented Sep 4, 2024

Uh oh!

msherif1234 commented May 29, 2024 •

edited

Loading

openshift-ci-robot commented May 29, 2024 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented May 29, 2024 •

edited by openshift-ci bot

Loading

jpinsonneau May 30, 2024 •

edited

Loading

codecov bot commented May 30, 2024 •

edited

Loading

openshift-ci-robot commented Jun 4, 2024 •

edited by openshift-ci bot

Loading