# Description

This is a continuation of
paritytech#5666 that finally fixes
paritytech#5333.

This should allow developers to create custom syncing strategies or even
the whole syncing engine if they so desire. It also moved syncing engine
creation and addition of corresponding protocol outside
`build_network_advanced` method, which is something Bastian expressed as
desired in
paritytech#5 (comment)

Here I replaced strategy-specific types and methods in `SyncingStrategy`
trait with generic ones. Specifically `SyncingAction` is now used by all
strategies instead of strategy-specific types with conversions.
`StrategyKey` was an enum with a fixed set of options and now replaced
with an opaque type that strategies create privately and send to upper
layers as an opaque type. Requests and responses are now handled in a
generic way regardless of the strategy, which reduced and simplified
strategy API.

`PolkadotSyncingStrategy` now lives in its dedicated module (had to edit
.gitignore for this) like other strategies.

`build_network_advanced` takes generic `SyncingService` as an argument
alongside with a few other low-level types (that can probably be
extracted in the future as well) without any notion of specifics of the
way syncing is actually done. All the protocol and tasks are created
outside and not a part of the network anymore. It still adds a bunch of
protocols like for light client and some others that should eventually
be restructured making `build_network_advanced` just building generic
network and not application-specific protocols handling.

## Integration

Just like paritytech#5666
introduced `build_polkadot_syncing_strategy`, this PR introduces
`build_default_block_downloader`, but for convenience and to avoid
typical boilerplate a simpler high-level function
`build_default_syncing_engine` is added that will take care of creating
typical block downloader, syncing strategy and syncing engine, which is
what most users will be using going forward. `build_network` towards the
end of the PR was renamed to `build_network_advanced` and
`build_network`'s API was reverted to
pre-paritytech#5666, so most users
will not see much of a difference during upgrade unless they opt-in to
use new API.

## Review Notes

For `StrategyKey` I was thinking about using something like private type
and then storing `TypeId` inside instead of a static string in it, let
me know if that would preferred.

The biggest change happened to requests that different strategies make
and how their responses are handled. The most annoying thing here is
that block response decoding, in contrast to all other responses, is
dependent on request. This meant request had to be sent throughout the
system. While originally `Response` was `Vec<u8>`, I didn't want to
re-encode/decode request and response just to fit into that API, so I
ended up with `Box<dyn Any + Send>`. This allows responses to be truly
generic and each strategy will know how to downcast it back to the
concrete type when handling the response.

Import queue refactoring was needed to move `SyncingEngine` construction
out of `build_network` that awkwardly implemented for `SyncingService`,
but due to `&mut self` wasn't usable on `Arc<SyncingService>` for no
good reason. `Arc<SyncingService>` itself is of course useless, but
refactoring to replace it with just `SyncingService` was unfortunately
rejected in paritytech#5454

As usual I recommend to review this PR as a series of commits instead of
as the final diff, it'll make more sense that way.

# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [x] My PR follows the [labeling requirements](

https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process
) of this project (at minimum one label for `T` required)
* External contributors: ask maintainers to put the right label on your
PR.
* [x] I have made corresponding changes to the documentation (if
applicable)

)

fixes paritytech#5966

---------

Co-authored-by: Guillaume Thiolliere <[email protected]>

…aritytech#6380)

This PR ensures that external addresses with different PeerIDs are not
propagated to the higher layer of the network code.

While at it, this ensures that libp2p only adds the `/p2p/peerid` part
to the discovered address if it does not contain it already.

This is a followup from:
- paritytech#6298

cc @paritytech/networking

---------

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: Dmitry Markin <[email protected]>

Fixes paritytech#5530

This PR introduces the removal of backing jobs that have been back
pressured for longer than `allowedAncestryLen`, as these candidates are
no longer viable.

It is reasonable to expect a result for a backing job execution within
`allowedAncestryLen` blocks. Therefore, we set the job TTL as a relay
block number and synchronize the validation host by sending activated
leaves.

---------

Co-authored-by: Andrei Sandu <[email protected]>
Co-authored-by: Branislav Kontur <[email protected]>

)

This PR has small addition to the db package version. As `cargodeb`
takes the version from the `*.toml` file, this PR adds an extra flag to
the `cargodeb` command so that the version of the deb package matches
the `polkadot` node version.

…ech#6382)

This PR changes the behavior of `json_patch::merge` function which no
longer removes any keys from the base JSON object.

fixes: paritytech#6306

---------

Co-authored-by: GitHub Action <[email protected]>

# Description

E2E tests to Polkadot/Kusama's people chains (in
open-web3-stack/polkadot-ecosystem-tests#63)
revealed that 2 of the identity pallet's extrinsics did not emit events
in case of success:
* `pallet_identity::rename_sub`, and
* `pallet_identity::set_subs`

This PR fixes that.

## Integration

Other than 2 extrinsics emiting an event when previously they did not,
no other behavior in pallets/extrinsics was modified, so no integration
is needed.

## Review Notes

N/A

# Description

Implemented benchmarks for Notifications and RequestResponse protocols
with libp2p implementation. These benchmarks allow us to monitor
regressions and implement fixes before they are observed in real chain.
In the future, they can be used for targeted optimizations of litep2p
compared to libp2p.

Part of paritytech#5220

Next steps:
- Add benchmarks for litep2p implementation
- Optimize load to get better results
- Add benchmarks to CI to catch regressions



## Integration

Benchmarks don't affect downstream projects.

---------

Co-authored-by: alvicsam <[email protected]>
Co-authored-by: GitHub Action <[email protected]>

Fixes paritytech#6172

…ritytech#6314)

Part of:

- paritytech#6202.

---------

Co-authored-by: Giuseppe Re <[email protected]>
Co-authored-by: GitHub Action <[email protected]>

…ch#6163)

This PR exposes additional syncing types to facilitate the development
of a custom syncing strategy based on the existing Polkadot syncing
strategy. Specifically, my goal is to isolate the state sync and chain
sync components, allowing the state to be downloaded from the P2P
network without running a full regular Substrate node. I also need to
intercept the state responses during the state sync process.

The newly exposed types are necessary to implement this custom syncing
strategy.

This PR introduces usage of `build_struct_json_patch` macro in all
runtimes (also guides) within the code base.

It also fixes macro to support _field init shorthand_, and _Struct
Update_ syntax which were missing in original implementation.

Follow up of paritytech#5700 and paritytech#5813

…paritytech#6290)

Part of:
paritytech#6202

---------

Co-authored-by: Giuseppe Re <[email protected]>
Co-authored-by: GitHub Action <[email protected]>

Signed-off-by: Iulian Barbu <[email protected]>

Signed-off-by: Iulian Barbu <[email protected]>