chore: easy setup fleets for lpt #3125

NagyZoltanPeter · 2024-10-17T08:19:15Z

Description

A greater upgrade for liteprotocoltester.

Changes

Added various metrics to follow test performance and failures.
Dashboard defined to follow test performance and failure counts.
Added ENR support for service and bootstrap peer configuration.
Added bootstrap capabilities using peer exchange.
If bootstrap node is defined, max 100 nodes will be gathered via PeerExchange.
PeerExchange nodes will be tested for connectivity and report created.
If further nodes are available (via bootstrap node and PX) in case of failure during test, the service node will be switched to another service node. (derived through internal thresholds).
jenkins job defined to build and create and deploy image of liteprotocoltester

Extension

A new repository https://github.com/waku-org/lpt-runner created to ease the usage and run against various waku networks and fleets.

Issue

#2999

…g service peers if failed with original

…ity and availability, dashboard adjustments

github-actions · 2024-10-17T10:49:11Z

You can find the image built from this PR at

quay.io/wakuorg/nwaku-pr:3125

Built from 3d8c20a

…ager's peerStore to wakuPeerStore

gabrielmer

Woooow amazing PR! 🔥 🔥 🔥 Thanks so much!

Left a bunch of random comments but mostly nitpicks.

Really looking forward to deploy the LPT and use it on the fleets! 😍

apps/liteprotocoltester/tester_config.nim

gabrielmer · 2024-10-18T07:16:24Z

apps/liteprotocoltester/service_peer_management.nim

+
+  return ok(peerInfo)
+
+randomize()


Do we want this call on a global scope or inside a proc? if it's globally, better to put it before every proc definition so it's easier to see

apps/liteprotocoltester/service_peer_management.nim

gabrielmer · 2024-10-18T07:26:02Z

apps/liteprotocoltester/service_peer_management.nim

+  elif codec.contains("filter"):
+    capability = Capabilities.Filter


isn't this elif redundant if capability is already initialized to Capabilities.Filter?

Same in selectRandomCapablePeer

gabrielmer · 2024-10-18T07:32:07Z

apps/liteprotocoltester/service_peer_management.nim

+  var found = none(RemotePeerInfo)
+  var okPeers: seq[RemotePeerInfo] = @[]
+
+  while found.isNone() and supportivePeers.len > 0:


I think we don't even update found in this proc, it's very similar to selectRandomCapablePeer - not sure if it's better to avoid the code duplication by having all that same logic in a same place

Actually, I think we don't use selectRandomCapablePeer anywhere 😶

gabrielmer · 2024-10-18T07:33:34Z

apps/liteprotocoltester/service_peer_management.nim

+      if connOpt.value().isSome():
+        found = some(randomPeer)
+        debug "Dialing successful",
+          peer = constructMultiaddrStr(randomPeer), codec = codec
+      else:
+        debug "Dialing failed", peer = constructMultiaddrStr(randomPeer), codec = codec


Shouldn't we do here

lpt_dialed_peers.inc()

and

lpt_dial_failures.inc()

like in tryCallAllPxPeers?

gabrielmer · 2024-10-18T07:47:08Z

apps/liteprotocoltester/liteprotocoltester.nim

+    if conf.bootstrapNode.len > 0:
+      info "Bootstrapping with PeerExchange to gather random service node"
+      let futForServiceNode = pxLookupServiceNode(wakuApp.node, conf)
+      if not (waitFor futForServiceNode.withTimeout(20.minutes)):


waitFor with 20 mins timeout? 😱

gabrielmer · 2024-10-18T07:48:40Z

apps/liteprotocoltester/liteprotocoltester.nim

+  # so we will mount PeerExchange to gather possible service peers, if got some
+  # we will mount the client protocols afterward.
+  wakuConf.peerExchange = false


not sure I understand - according to the comment, shouldn't wakuConf.peerExchange be set to true?

gabrielmer · 2024-10-18T08:03:53Z

apps/liteprotocoltester/diagnose_connections.nim

+proc `$`*(cap: Capabilities): string =
+  case cap
+  of Capabilities.Relay:
+    return "Relay"
+  of Capabilities.Store:
+    return "Store"
+  of Capabilities.Filter:
+    return "Filter"
+  of Capabilities.Lightpush:
+    return "Lightpush"
+  of Capabilities.Sync:
+    return "Sync"


It might be worth having this proc in waku/waku_enr/capabilities.nim so it can be used in other places too :)

Co-authored-by: gabrielmer <[email protected]>

…ring lightpublish

Ivansete-status

LGTM! Thanks for it! 💯
I just added some nitpick comments that I hope you find useful ;P

Ivansete-status · 2024-10-24T10:53:50Z

apps/liteprotocoltester/README.md

+### Using bootstrap nodes
+
+There are multiple benefits of using bootstrap nodes. By using them liteprotocoltester will use Peer Exchange protocol to get possible peers from the network that are capable to serve as service peers for testing. Additionally it will test dial them to verify their connectivity - this will be reported in the logs and on dashboard metrics.
+Also by using bootstrap node and peer exchange discovery, litprotocoltester will be able to simulate service peer switch in case of failures. There are built in tresholds for service peer failures during test and service peer can be switched during the test. Also these service peer failures are reported, thus extening network reliability measures.


Also by using bootstrap node and peer exchange discovery, litprotocoltester will be able to simulate service peer switch in case of failures.

This is super interesting! Maybe in the future we could set a list of service-nodes and then validate that the peer switch works well from a fixed service-node set.

There are built in tresholds for service peer failures during test and service peer can be switched during the test. Also these service peer failures are reported, thus extening network reliability measures

Sorry, but I don't fully get the first part of the previous sentence. Aside, there's a tiny typo in entending

Ivansete-status · 2024-10-24T11:02:03Z

apps/liteprotocoltester/README.md

@@ -205,7 +194,18 @@ docker build -t liteprotocoltester:latest -f Dockerfile.liteprotocoltester ../..

 # edit and adjust .env file to your needs and for the network configuration

-docker run --env-file .env liteprotocoltester:latest RECEIVER <service-node-ip4-peer-address>
+docker run --env-file .env liteprotocoltester:latest RECEIVER <service-node-peer-address>


I think it'd help to have an example on each command so that an not-experienced reader can learn about the expected address format (that tiny suggestion also applies to the following lines as well.)

apps/liteprotocoltester/diagnose_connections.nim

apps/liteprotocoltester/filter_subscriber.nim

apps/liteprotocoltester/lightpush_publisher.nim

apps/liteprotocoltester/service_peer_management.nim

waku/node/waku_node.nim

NagyZoltanPeter · 2024-10-25T14:11:10Z

Added more detailed examples section about how to use it in different ways and conditions.
For infra deployment metrics server port configurability added.
All review comments addressed.

NagyZoltanPeter self-assigned this Oct 17, 2024

NagyZoltanPeter requested review from Ivansete-status, gabrielmer, fryorcraken, darshankabariya and richard-ramos October 17, 2024 10:08

NagyZoltanPeter added 5 commits October 17, 2024 12:11

Added bootstrap peer exchange discovery option for easy setup ltp

3e19e46

Extended with PX discovery, auto-dial of PX cap peers, added switchin…

8859c58

…g service peers if failed with original

Added peer-exchange, found capable peers test, metrics on peer stabil…

7febe57

…ity and availability, dashboard adjustments

Updated and actualized README.md for liteprotocoltester

33c6ed0

Created jenkinsfile for liteprotocoltester deployment

460f561

NagyZoltanPeter force-pushed the chore-easy-setup-fleets-for-lpt branch from b0a1a05 to 460f561 Compare October 17, 2024 10:44

NagyZoltanPeter marked this pull request as ready for review October 17, 2024 10:45

NagyZoltanPeter mentioned this pull request Oct 17, 2024

[Deliverable] Tooling: filter and light push protocols waku-org/pm#178

Open

2 tasks

NagyZoltanPeter added 2 commits October 17, 2024 15:18

Fix build after rebase to latest master. Applied renames from PeerMan…

e9fb3c2

…ager's peerStore to wakuPeerStore

Fix Jenkinsfile.lpt choice param of LOGLEVEL

d781bb2

gabrielmer approved these changes Oct 18, 2024

View reviewed changes

NagyZoltanPeter and others added 10 commits October 18, 2024 11:19

Fix image push for jenkins job

47e2e04

More fix for deployment jenkins job

77238bf

More fix to Jenkinsfile.lpt

2f90d5f

More fix to Jenkinsfile.lpt

1fd679b

Fix Jenkinsfile.lpt post job cleanup

613f5bd

Apply suggestions from code review

b83eaa5

Co-authored-by: gabrielmer <[email protected]>

Address review comment with more explanation. Fixed dial exception du…

d591eb2

…ring lightpublish

Add configuration for requesting and testing peer exchange peers

286c4d4

Remove debug leftover

6d2a51a

Fix lint issues

9a1cd42

Ivansete-status approved these changes Oct 24, 2024

View reviewed changes

Addressing review comments, extended examples added to Readme

6ca28a2

NagyZoltanPeter added 2 commits October 25, 2024 11:23

Better explaination in Readme, added metrics port configurability

b212921

Fix cli arg description

2a863d4

Fix linting problem

a808619

NagyZoltanPeter merged commit 268e7e6 into master Oct 25, 2024
10 of 11 checks passed

NagyZoltanPeter deleted the chore-easy-setup-fleets-for-lpt branch October 25, 2024 20:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: easy setup fleets for lpt #3125

chore: easy setup fleets for lpt #3125

NagyZoltanPeter commented Oct 17, 2024 •

edited

Loading

github-actions bot commented Oct 17, 2024 •

edited

Loading

gabrielmer left a comment

gabrielmer Oct 18, 2024

gabrielmer Oct 18, 2024

gabrielmer Oct 18, 2024

gabrielmer Oct 18, 2024

gabrielmer Oct 18, 2024

gabrielmer Oct 18, 2024

gabrielmer Oct 18, 2024

Ivansete-status left a comment

Ivansete-status Oct 24, 2024

Ivansete-status Oct 24, 2024

NagyZoltanPeter commented Oct 25, 2024

		elif codec.contains("filter"):
		capability = Capabilities.Filter

chore: easy setup fleets for lpt #3125

chore: easy setup fleets for lpt #3125

Conversation

NagyZoltanPeter commented Oct 17, 2024 • edited Loading

Description

Changes

Extension

Issue

github-actions bot commented Oct 17, 2024 • edited Loading

gabrielmer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ivansete-status left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NagyZoltanPeter commented Oct 25, 2024

NagyZoltanPeter commented Oct 17, 2024 •

edited

Loading

github-actions bot commented Oct 17, 2024 •

edited

Loading