Bridges - revert-back and improve congestion #6231

bkontur · 2024-10-25T08:51:14Z

Closes: #5551
Closes: #5550

Context

Before permissionless lanes, bridges only supported hard-coded, static lanes. The congestion mechanism was based on sending Transact(report_bridge_status(is_congested)) from pallet-xcm-bridge-hub to pallet-xcm-bridge-hub-router. Depending on is_congested, we adjusted the fee factor to increase or decrease fees. This congestion mechanism relied on monitoring XCMP queues, which could cause issues like suspending the entire XCMP queue rather than just the affected bridge.

Additionally, we are progressing with deploying bridge message pallets/routing directly on AssetHub, where we don’t interact with XCMP to perform ExportXcm locally.

Description

This PR re-introduces and improves congestion for bridges:

Enhanced Bridge Congestion Mechanism: The bridge queue mechanism has been restructured to operate independently of XCMP, with a refined protocol for congestion detection and suspension management.
Bridge-Specific Channel Suspension: pallet-xcm-bridge-hub and pallet-xcm-bridge-hub-router now use BridgeId to identify specific bridges, enabling selective suspension and resumption of individual bridge channels.
Dynamic Congestion Detection: pallet-xcm-bridge-hub now includes callbacks for fn suspend_bridge and fn resume_bridge based on congestion status:
- For sibling chains, the router sends xcm::Transact(report_bridge_status(bridge_id, is_congested)) using the stored callback information.
- For local chain deployments, the router manages state directly.
New Stop Threshold: A stop_threshold limit in pallet-xcm-bridge-hub enables or disables ExportXcm::validate, providing a fallback mechanism when the router does not adhere to the suspend signal.
Flexible Message Routing: pallet-xcm-bridge-hub-router has been refactored to support message routing for both sibling chains (ExportMessage) and local deployment (ExportXcm).

These updates improve modularity, allow more granular bridge congestion handling, and support diverse deployment scenarios.

bkontur · 2024-10-25T11:25:35Z

bot fmt
/cmd prdoc --audience runtime_dev --bump patch

prdoc/pr_6231.prdoc

bkontur · 2024-10-26T20:30:05Z

bot fmt

bkontur · 2024-10-28T15:04:20Z

bot fmt

bkontur · 2024-11-05T12:40:39Z

bot fmt

bkontur · 2024-11-07T15:57:44Z

/cmd bench --runtime asset-hub-westend asset-hub-rococo --pallet pallet_xcm_bridge_hub_router

bkontur · 2024-11-07T16:18:32Z

bot bench cumulus-assets --runtime=asset-hub-westend --pallet=pallet_xcm_bridge_hub_router
bot bench cumulus-assets --runtime=asset-hub-rococo --pallet=pallet_xcm_bridge_hub_router

bkontur · 2024-11-07T16:51:47Z

bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-assets --runtime=asset-hub-westend --pallet=pallet_xcm_bridge_hub_router
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-assets --runtime=asset-hub-rococo --pallet=pallet_xcm_bridge_hub_router

bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_messages
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_messages

bkontur · 2024-11-07T21:51:26Z

bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_messages
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_messages
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --subcommand=xcm --runtime=bridge-hub-rococo --pallet=pallet_xcm_benchmarks::generic
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --subcommand=xcm --runtime=bridge-hub-westend --pallet=pallet_xcm_benchmarks::generic

bkontur · 2024-11-08T23:55:04Z

bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_messages
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_messages
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_xcm_bridge_hub
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_xcm_bridge_hub

bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --subcommand=xcm --runtime=bridge-hub-rococo --pallet=pallet_xcm_benchmarks::generic
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --subcommand=xcm --runtime=bridge-hub-westend --pallet=pallet_xcm_benchmarks::generic

bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-assets --runtime=asset-hub-westend --pallet=pallet_xcm_bridge_hub_router
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-assets --runtime=asset-hub-rococo --pallet=pallet_xcm_bridge_hub_router

bkontur · 2024-11-15T14:55:26Z

/cmd --clean

bkontur · 2024-11-16T14:48:46Z

bot clean

serban300

Did just a first pass

serban300 · 2024-11-18T07:28:58Z

bridges/modules/xcm-bridge-hub-router/src/lib.rs

+						{
+							break;
+						}
+						bridges_to_update.push((bridge_id, previous_factor, bridge_state));


Why not process it on the spot ?

Why not process it on the spot ?

Well, at least for bridges_to_remove I can't/shouldn't do that according to the documentation:

/// Enumerate all elements in the map in no particular order.
///
/// If you alter the map while doing this, you'll get undefined results.

I don't know, maybe inserting the same key with different value while iter would work (I didn't try), but I assume that it is also "alter the map", so I better used the same pattern for bridges_to_update.

bridges/modules/xcm-bridge-hub-router/src/lib.rs

serban300 · 2024-11-18T08:06:03Z

bridges/modules/xcm-bridge-hub-router/src/lib.rs

+			// handle congestion and fee factor (if detected)
+			let increased =
+				Bridges::<T, I>::mutate_exists(&bridge_id, |bridge_state| match bridge_state {
+					Some(ref mut bridge_state) if bridge_state.is_congested => {


Nit: Something like

if !bridge_state.is_congested return None;

seems easier to follow

I fixed here: 416e2e0 to process everything on the spot as you requested in the previous comment.

@serban300 I am not sure how to fix this, is this still relevant? Could you please provide more verbose suggestion?

bridges/modules/xcm-bridge-hub-router/src/impls.rs

serban300 · 2024-11-18T13:33:52Z

bridges/modules/xcm-bridge-hub-router/src/lib.rs

+		/// Notification about congested bridge queue.
+		#[pallet::call_index(0)]
+		#[pallet::weight(T::WeightInfo::report_bridge_status())]
+		pub fn report_bridge_status(


Nit: Would update_bridge_status() be more suggestive ?

report_bridge_status is the original name, which I am reverting back (it was removed here: https://github.com/paritytech/polkadot-sdk/pull/4949/files#diff-9c9793c594a3cd93997c4b8fb651bd250d49b1b85068cd555aaf8c0809b500e1L158).

Well, I am ok with "report" in this case - somebody (pallet_xcm_bridge_hub) is just telling us that it has problem with congestion or not. Actually, yes, the pallet_xcm_bridge_hub_router updates some state, but it is just implementation detail. The pallet_xcm_bridge_hub does not need to reason about this if it is update or whatever.

serban300 · 2024-11-18T14:04:59Z

bridges/modules/xcm-bridge-hub/src/lib.rs

 		pub fn open_bridge(
 			origin: OriginFor<T>,
 			bridge_destination_universal_location: Box<VersionedInteriorLocation>,
+			maybe_notify: Option<Receiver>,


maybe_notify doesn't seem very suggestive. Maybe something like congestion_notif_receiver would be better.

Well, actually, I re-used maybe_notify name/pattern from the pallet_xcm's QueryStatus maybe_notify: Option<(u8, u8)>, :), which does exactly the same, Receiver is basically the same as (u8, u8).

bkontur added the T15-bridges This PR/Issue is related to bridges. label Oct 25, 2024

bkontur self-assigned this Oct 25, 2024

bkontur force-pushed the bko-bridges-congestion branch 2 times, most recently from 659be89 to b48b8a5 Compare October 25, 2024 21:14

bkontur commented Oct 26, 2024

View reviewed changes

prdoc/pr_6231.prdoc Outdated Show resolved Hide resolved

bkontur force-pushed the bko-bridges-congestion branch from a663bc2 to cbc6ae7 Compare October 26, 2024 20:29

bkontur force-pushed the bko-bridges-congestion branch from c78f9bc to 501a5c0 Compare October 28, 2024 15:01

bkontur force-pushed the bko-bridges-congestion branch 7 times, most recently from c78e707 to 152389a Compare November 5, 2024 12:33

bkontur force-pushed the bko-bridges-congestion branch 3 times, most recently from edd9c5c to 38f1bb3 Compare November 7, 2024 13:33

bkontur force-pushed the bko-bridges-congestion branch from f06433a to d329dec Compare November 7, 2024 16:44

bkontur force-pushed the bko-bridges-congestion branch from 21baa7f to c00ff6b Compare November 8, 2024 17:44

bkontur added the A4-needs-backport Pull request must be backported to all maintained releases. label Nov 11, 2024

Nits

ad0931c

actions-user and others added 2 commits November 14, 2024 12:11

Update from bkontur running command 'fmt'

f643ec7

PR review - updated docs

8518de3

bkontur requested a review from franciscoaguirre November 14, 2024 15:46

bkontur and others added 3 commits November 15, 2024 09:41

clippy

8a082c1

Merge remote-tracking branch 'origin/master' into bko-bridges-congestion

0ccfef2

Update from bkontur running command 'fmt'

465db5f

github-actions bot deleted a comment from bkontur Nov 15, 2024

Merge branch 'master' into bko-bridges-congestion

37974be

command-bot bot deleted a comment from github-actions bot Nov 16, 2024

serban300 reviewed Nov 18, 2024

View reviewed changes

bkontur added 4 commits November 19, 2024 22:50

PR review

416e2e0

Merge remote-tracking branch 'origin/master' into bko-bridges-congestion

9f66e49

PR review - removed brackets

15ef0a4

PR review

296476e

paritytech-review-bot bot requested a review from a team November 19, 2024 22:05

bkontur requested a review from serban300 November 19, 2024 22:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bridges - revert-back and improve congestion #6231

Bridges - revert-back and improve congestion #6231

bkontur commented Oct 25, 2024 •

edited

Loading

bkontur commented Oct 25, 2024

bkontur commented Oct 26, 2024

bkontur commented Oct 28, 2024

bkontur commented Nov 5, 2024

bkontur commented Nov 7, 2024

bkontur commented Nov 7, 2024

bkontur commented Nov 7, 2024

bkontur commented Nov 7, 2024

bkontur commented Nov 8, 2024

bkontur commented Nov 15, 2024

bkontur commented Nov 16, 2024

serban300 left a comment

serban300 Nov 18, 2024

bkontur Nov 18, 2024

serban300 Nov 18, 2024

bkontur Nov 19, 2024

serban300 Nov 18, 2024

bkontur Nov 18, 2024

serban300 Nov 18, 2024

bkontur Nov 19, 2024

Bridges - revert-back and improve congestion #6231

Are you sure you want to change the base?

Bridges - revert-back and improve congestion #6231

Conversation

bkontur commented Oct 25, 2024 • edited Loading

Context

Description

bkontur commented Oct 25, 2024

bkontur commented Oct 26, 2024

bkontur commented Oct 28, 2024

bkontur commented Nov 5, 2024

bkontur commented Nov 7, 2024

bkontur commented Nov 7, 2024

bkontur commented Nov 7, 2024

bkontur commented Nov 7, 2024

bkontur commented Nov 8, 2024

bkontur commented Nov 15, 2024

bkontur commented Nov 16, 2024

serban300 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bkontur commented Oct 25, 2024 •

edited

Loading