[explorer/nodewatch]: statistics service migration #1421

AnthonyLaw · 2025-06-14T20:01:22Z

Problem: missing some of the features and endpoints to migrate from the statistic service to Node Watch.
Solution: Added those endpoints and features.

… key Problem: The endpoint is missing when using the main public key to query node info. Solution: Added a new endpoint to query node info by the main public key. Added error handler for 400 and 404

… key Problem: The endpoint is missing when using the node public key to query node info. Solution: A new endpoint was added to query node info using the node public key.

Problem: The API node status field is missing Solution: Added API node status properties in NodeDescriptor

Problem: The filter node is missing on the API node list endpoint Solution: Added query params only_ssl, limit, and order only_ssl: to filter out SSL enable node, it needs explorer and wallet limit: the number of numbers returned from the endpoint order: shuffle node list when require

Problem: missing geo-location information in the node list Solution: load the geo location file from the specific path, and create a mapping for node

Problem: The nodes count endpoint is missing. Solution: Read the JSON file `symbol_time_series_nodes_count` and allow query data from the new endpoint.

Problem: missing finalizedEpoch, finalizedHash, and finalizedPoint in node descriptor Solution: added finalized properties in node descriptor

codecov · 2025-06-14T20:09:02Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.27%. Comparing base (be149f3) to head (39618e7).
Report is 6 commits behind head on dev.

Additional details and impacted files

@@           Coverage Diff           @@
##              dev    #1421   +/-   ##
=======================================
  Coverage   98.26%   98.27%           
=======================================
  Files         163      158    -5     
  Lines        6691     6648   -43     
  Branches      143      143           
=======================================
- Hits         6575     6533   -42     
+ Misses        116      115    -1

Flag	Coverage Δ
explorer-nodewatch	`98.64% <100.00%> (+0.24%)`	⬆️
explorer-rest	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
explorer/nodewatch/nodewatch/HeightChartBuilder.py	`100.00% <100.00%> (ø)`
explorer/nodewatch/nodewatch/NetworkRepository.py	`100.00% <100.00%> (ø)`
explorer/nodewatch/nodewatch/RoutesFacade.py	`96.82% <100.00%> (+0.63%)`	⬆️
explorer/nodewatch/nodewatch/__init__.py	`97.03% <100.00%> (+1.03%)`	⬆️
explorer/nodewatch/nodewatch/chart_utils.py	`100.00% <100.00%> (ø)`

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Wayonb · 2025-06-23T18:58:49Z

explorer/nodewatch/nodewatch/NetworkRepository.py

@@ -23,6 +23,13 @@ def __init__(
 		height=0,
 		finalized_height=0,
 		balance=0,
+		finalized_epoch=0,
+		finalized_hash=None,
+		finalized_point=0,


Does it make sense to group all the finalized properties into a namedtuple?

Yes, sound good to me 👍🏼
I'll update it.

An update.

I'm unsure it's a good practice to use nested way to get attr in Python.

Used namedtuple to group finalized_height, finalized_epoch, finalized_hash, and finalized_point as finalized_info.

Found out that chart_utlis.py's VersionAggregator consumes finalized_height directly from the object instead of finalized_info.height. Because of the namedtuple, it required a nested approach to retrieve the attribute.

Here is my commit d5f6ed7

Wayonb · 2025-06-23T20:41:22Z

explorer/nodewatch/tests/test_NetworkRepository.py

+		# Act:
+		json_object = repository.node_descriptors[3].to_json()
+
+		print(json_object)


Is the print still needed?

Good catch! 👍🏼

Wayonb · 2025-06-23T20:46:20Z

explorer/nodewatch/tests/test_NetworkRepository.py

+		self.assertEqual(None, json_object['geoLocation'])
+
+	# endregion
+


should be 2 spaces between # endregion and the next.

Wayonb · 2025-06-23T20:51:50Z

explorer/nodewatch/tests/test_RoutesFacade.py

+		full_node_names = list(map(lambda descriptor: descriptor['name'], all_node_descriptors))
+		random_node_names = list(map(lambda descriptor: descriptor['name'], node_descriptors))
+
+		self.assertEqual(2, len(node_descriptors))


might also want to verify that all_node_descriptors is not 2 nodes

Wayonb · 2025-06-23T20:58:52Z

explorer/nodewatch/tests/test_RoutesFacade.py

+		for name in random_node_names:
+			self.assertIn(name, full_node_names)
+
+	def test_can_generate_nodes_json_filtered_limit_5(self):


This seems similar to the above? Not sure if checking the roles below matters for a limit test.

Good catch! I was missing order='random' params in the test.

Wayonb · 2025-06-23T21:08:58Z

explorer/nodewatch/tests/test_app.py

-	] == list(map(lambda descriptor: descriptor['name'], response_json))
+
+def test_get_api_symbol_nodes_peer_with_only_ssl(client):  # pylint: disable=redefined-outer-name
+	_assert_get_api_nodes(client.get('/api/symbol/nodes/peer?only_ssl'), ['ibone74'])


I thought only_ssl flag would only work wtih API endpoint /api/symbol/nodes/api? 🤔

Nope. It belongs to both endpoints.

/api/symbol/nodes/api: returns only roles=2. For other roles, such as roles = 3, it also includes only_ssl.

AnthonyLaw added 7 commits June 15, 2025 04:03

[explorer/nodewatch]: added new endpoint to query node by main public…

37ededd

… key Problem: The endpoint is missing when using the main public key to query node info. Solution: Added a new endpoint to query node info by the main public key. Added error handler for 400 and 404

[explorer/nodewatch]: added new endpoint to query node by node public…

19d3de9

… key Problem: The endpoint is missing when using the node public key to query node info. Solution: A new endpoint was added to query node info using the node public key.

[explorer/nodewatch]: added api node status field

864ac03

Problem: The API node status field is missing Solution: Added API node status properties in NodeDescriptor

[explorer/nodewatch]: Added geo location in node descriptor

d04592b

Problem: missing geo-location information in the node list Solution: load the geo location file from the specific path, and create a mapping for node

[explorer/nodewatch]: Added time series nodes count endpoint

2024682

Problem: The nodes count endpoint is missing. Solution: Read the JSON file `symbol_time_series_nodes_count` and allow query data from the new endpoint.

[explorer/nodewatch]: Added finalized properties in node descriptor

0842cb0

Problem: missing finalizedEpoch, finalizedHash, and finalizedPoint in node descriptor Solution: added finalized properties in node descriptor

AnthonyLaw force-pushed the nodewatch/statistics-service-migration branch from fb779b2 to 0842cb0 Compare June 14, 2025 20:03

AnthonyLaw requested review from Wayonb and Jaguar0625 June 14, 2025 20:49

Wayonb requested changes Jun 23, 2025

View reviewed changes

AnthonyLaw added 5 commits June 25, 2025 02:51

[explorer/nodewatch]: added missing params in test

7206198

[explorer/nodewatch]: fixed space and remove print

971ff56

[explorer/nodewatch]: group finalized info

a700b22

[explorer/nodewatch]: added get nested attr method.

d5f6ed7

[explorer/nodewatch]: fix lint

190a9cf

AnthonyLaw mentioned this pull request Jun 27, 2025

[explorer/nodewatch] task: add new route #343

Closed

AnthonyLaw added 3 commits June 28, 2025 04:40

[explorer/nodewatch]: convert method to staticmethod

b719d0c

[explorer/nodewatch]: fix unit test

30f18f3

[explorer/nodewatch]: fix lint

39618e7

AnthonyLaw requested a review from Wayonb July 21, 2025 17:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[explorer/nodewatch]: statistics service migration #1421

[explorer/nodewatch]: statistics service migration #1421

Uh oh!

AnthonyLaw commented Jun 14, 2025

Uh oh!

codecov bot commented Jun 14, 2025 •

edited

Loading

Uh oh!

Wayonb Jun 23, 2025

Uh oh!

AnthonyLaw Jun 25, 2025

Uh oh!

AnthonyLaw Jun 25, 2025

Uh oh!

Wayonb Jun 23, 2025

Uh oh!

AnthonyLaw Jun 24, 2025

Uh oh!

Wayonb Jun 23, 2025

Uh oh!

Wayonb Jun 23, 2025

Uh oh!

Wayonb Jun 23, 2025

Uh oh!

AnthonyLaw Jun 24, 2025

Uh oh!

Wayonb Jun 23, 2025

Uh oh!

AnthonyLaw Jun 25, 2025

Uh oh!

Uh oh!

		self.assertEqual(None, json_object['geoLocation'])

		# endregion

[explorer/nodewatch]: statistics service migration #1421

Are you sure you want to change the base?

[explorer/nodewatch]: statistics service migration #1421

Uh oh!

Conversation

AnthonyLaw commented Jun 14, 2025

Uh oh!

codecov bot commented Jun 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov bot commented Jun 14, 2025 •

edited

Loading