v1.7.0

October 20th, 2023

This v1.7 release introduces new features and improvements to Hermes.

One of the key highlights is the addition of new misbehavior detection features.

• Hermes now includes a new command called evidence, which monitors the blocks emitted by a chain for any presence of misbehavior evidence.
• If misbehavior is detected, the CLI will report that evidence to all counterparty clients of that chain.
  On top of that, misbehavior evidence detected on a chain that is a CCV (Cross-Chain Validation) consumer is now sent to its provider chain, alerting it directly of the misbehaving consumer chain.
• Furthermore, when misbehavior is detected from an on-chain client, such as a light client attack or a double-sign, the evidence is now submitted to all counterparty clients of the misbehaving chain, rather than just the counterparty client of the misbehaving client.

In addition, the REST server of Hermes now has a /clear_packets endpoint which allows triggering packet clearing for a specific chain or all chains if no specific chain is provided.

Another notable improvement is the ability to change tracing directives at runtime.
This feature lets users adjust tracing settings dynamically as needed, providing a more customizable and efficient debugging experience.

Overall, the new misbehavior detection features in Hermes contribute to a more robust and secure environment, enabling timely identification and response to potential misbehaving actors.

FEATURES

• Relayer CLI
  • Add a new evidence command for monitoring the blocks emitted by a chain for the presence of a misbehaviour evidence, and report that evidence to all counteparty clients of that chain. (#3456)
  • Add a /clear_packets?chain=CHAIN_ID endpoint to the built-in REST server to trigger packet clear for the chain specified in the chain query param or for all chains if the query param is omitted. (#3398)
  • Add support for changing tracing directives at runtime. ([#3564]
    Please see the corresponding page in the Hermes guide for more information.(#3564))

IMPROVEMENTS

• Relayer Library
  • When Hermes detects a misbehaviour on a chain that is CCV consumer, it will now send the misbehaviour evidence to the provider chain using the new IcsConsumerMisbehaviour message. (#3219)
  • When Hermes detects a misbehaviour from a on-chain client, eg. a light client attack or a double-sign, it will now submit the misbehaviour evidence to all counterparty clients of the misbehaving chain instead of to the counterparty client of the misbehaving client only. (#3223)
  • Improve error message when scanning unsupported client (#3531)
  • Regard the finalize_block_events field of the block_results RPC endpoint, added in CometBFT 0.38 (#3548)
  • Change fallback compatibility version for CometBFT from v0.37 to v0.34 (#3666)
• Relayer CLI
  • The listen command now works with both push and pull event sources (#3501)

BUG FIXES

• Relayer CLI
  • Revert Docker image to Ubuntu LTS and set the UID and GID explicitly (#3580)
• IBC Data structures
  • Fix build of ibc-relayer-types documentation on docs.rs (#3549)

What's Changed

• Add date to release in changelog by @romac in #3490
• Add Stride v12.1.0 to ICS tests by @ljoss17 in #3492
• Add ICA transfer test to interchain-security tests by @ljoss17 in #3493
• Add ICQ tests for interchain-security tests by @ljoss17 in #3494
• Bump dyn-clone from 1.0.11 to 1.0.12 by @dependabot in #3497
• Bump signal-hook from 0.3.15 to 0.3.17 by @dependabot in #3500
• Bump async-trait from 0.1.71 to 0.1.72 by @dependabot in #3515
• Bump bitcoin from 0.30.0 to 0.30.1 by @dependabot in #3512
• Bump erased-serde from 0.3.27 to 0.3.28 by @dependabot in #3511
• Bump Swatinem/rust-cache from 1 to 2 by @dependabot in #3517
• Fix interchain-security tests flakiness by @ljoss17 in #3518
• Bump moka from 0.11.2 to 0.11.3 by @dependabot in #3526
• Bump ibc-proto from 0.32.0 to 0.32.1 by @dependabot in #3536
• Enable listen command with pull event source by @romac in #3502
• Bump ed25519-dalek from 1.0.1 to 2.0.0 by @dependabot in #3537
• Reduce CI flakiness by using cargo-nextest by @soareschen in #3544
• Update to tendermint-rs 0.33.0 and adapt to RPC changes in CometBFT 0.38 by @mzabaluev in #3372
• Bump ics23 from 0.10.1 to 0.10.2 by @dependabot in #3551
• Remove dependency on safe-regex for regex by @romac in #3556
• Bump rustls-webpki from 0.101.1 to 0.101.4 by @dependabot in #3557
• Small fix in the config.toml``trust_threshold for chain ibc-1 by @ljoss17 in #3560
• Fix Clippy warnings on Rust 1.72 by @romac in #3562
• Add KeyFile associated type to SigningKeyPair trait by @DaviRain-Su in #3568
• Improve error message when scanning unsupported client by @DaviRain-Su in #3567
• Use lychee to check markdown links by @romac in #3576
• Limit link checker concurrency and use GitHub token by @romac in #3577
• Revert Docker image to Ubuntu LTS and set the UID and GID explicitly by @romac in #3581
• Bump actions/checkout from 3 to 4 by @dependabot in #3583
• Bump regex from 1.9.1 to 1.9.5 by @dependabot in #3599
• Split Docker build into its own workflow by @romac in #3602
• Add support for changing tracing directives at runtime by @ljoss17 in #3565
• Add REST endpoint /clear_packets to trigger packet clearing by @romac in #3401
• Bump docker/setup-qemu-action from 2 to 3 by @dependabot in #3608
• Bump docker/login-action from 2 to 3 by @dependabot in #3607
• Bump docker/metadata-action from 4 to 5 by @dependabot in #3606
• Bump docker/setup-buildx-action from 2 to 3 by @dependabot in #3605
• Bump docker/build-push-action from 4 to 5 by @dependabot in #3604
• Use only Gaia v11 on CI by @romac in #3546
• Bump ibc-proto from 0.33.0 to 0.34.0 by @dependabot in #3552
• docs: fix # of crates by @rootulp in #3621
• fix: link to gaiad-manager by @rootulp in #3613
• Document discard_abci_responses and pull-mode relaying interaction in the Hermes guide by @seanchen1991 in #3635
• Document Hermes' gRPC endpoint requirements by @seanchen1991 in #3529
• Update gaiad version and config.toml example in Hermes guide by @seanchen1991 in #3640
• Document how to read a key/mnemonic from stdin in keys add command by @romac in #3645
• Remove file committed by mistake by @romac in #3652
• Update tendermint-rs to v0.34, prost to v0.12 and tonic to v0.10 by @romac in #3651
• Fix new clippy warnings on Rust 1.74 by @romac in #3653
• Improve ICS test bootstrapping by @ljoss17 in #3657
• Add evidence command for monitoring and reporting misbehavior evidence to counterparty clients by @romac in #3456
• Bump Swatinem/rust-cache from 1 to 2 by @dependabot in https://github.com/informalsystems/hermes...

v1.6.0

July 19th, 2023

This release of Hermes notably features a new pull-based event source which fetches events from the chain periodically using
the /block_results RPC endpoint instead of getting them over WebSocket.

To use the new pull-based event source, set

event_source = { mode = 'pull', interval = '1s' }`

in the per-chain configuration.

Check the event_source setting in the example config.toml file in the Hermes repository for more details.

Additionally, it is now possible to skip the scanning phase during Hermes startup, by disabling clients, connections and channels workers, and setting clear_on_start to false. This significantly improve startup time when relaying on chains with hundreds or thousands of open channels, connections and clients.
See the Performance tuning page of the guide for more information.

Note for operators

⚠️ Be aware that this release contains breaking changes to the Hermes configuration.
⚠️ Please consult the UPGRADING.md document for more details.

BREAKING CHANGES

• The websocket_addr configuration option has been removed in favour of the new event_source setting.
  Please consult the UPGRADING.md document for more details.
• Bump MSRV to 1.71 (#3478)

BUG FIXES

• Relayer CLI
  • Install CA certificates in Docker image for Hermes to be able to connect to TLS endpoints (#3423)
• Relayer Library
  • Fix a bug where Hermes would discard the client updates corresponding to the supporting headers returned by the light client when assembling messages to relay from the operational data (#3465)

FEATURES

• Relayer Library

  • Add a pull-based event source which fetches events from the chain periodically using the /block_results RPC endpoint instead of getting them over WebSocket.

    To use the pull-based event source, set event_source = { mode = 'pull', interval = '1s' } in the per-chain configuration.

    Warning
    Only use this if you think Hermes is not getting all the events it should, eg. when relaying for a CosmWasm-enabled blockchain which emits IBC events in a smart contract where the events lack the message attribute key.
    See #3190 and #2809 for more background information.
    (#2850)

• Integration Test Framework
  • Add integration tests for the Fee Grant module. (#3416)
  • Add a new trait InterchainSecurityChainTest and two functions run_binary_interchain_security_node_test and run_binary_interchain_security_channel_test which can be used to bootstrap a Provider and Consumer chain for integration tests. Add a CI job to run tests with Gaia as a provider chain and Neutron as a Consumer chain.
    (#3450) (#3387)

IMPROVEMENTS

• Relayer CLI
  • By disabling clients, connections and channels workers, and setting clear_on_start to false, Hermes will now skip the scanning phase during startup, significantly improve startup time when relaying on chains with hundreds or thousands of open channels, connections and/or clients. See the Performance tuning page of the guide for more information. (#3403)
• Relayer Library
  • Hermes will now automatically shutdown inactive workers to reduce the consumption of host system resources (#3385)
  • Query the /genesis RPC endpoint to retrieve the value of max_expected_time_per_block and use it for max_block_time. (#3211)
• Telemetry & Metrics
  • Add two new configurations for the telemetry buckets:
  • latency_submitted used to specify the range and number of buckets for the tx_latency_submitted metric.
  • latency_confirmed used to specify the range and number of buckets for the tx_latency_confirmed metric.
    (#3366)
• Integration Test Framework
  • Update ICA tests to use ibc-go's simd instead of interchain-accounts-demo. (#3353)
  • Update simd version used in integration tests from v7.0.0 to v7.1.0 (#3434)

What's Changed

• Add poll-based event source by @romac in #3323
• Remove debug noise when deadline elapses by @romac in #3400
• Update disclaimer in guide by @romac in #3406
• Build Docker multi-platform images in parallel by @romac in #3405
• Use CometBFT 0.37 compatibility mode for unsupported Tendermint versions (eg. Tendermint 0.35 and 0.36) by @seanchen1991 in #3404
• Add documentation on relaying for consumer chains to the Hermes guide by @seanchen1991 in #3407
• Fix link to prometheus.yml file in guide by @ljoss17 in #3409
• Bump moka from 0.11.1 to 0.11.2 by @dependabot in #3413
• Bump time from 0.3.20 to 0.3.22 by @dependabot in #3412
• Bump tempfile from 3.5.0 to 3.6.0 by @dependabot in #3411
• Bump serde from 1.0.163 to 1.0.164 by @dependabot in #3410
• Disable scanning when client, channel and connection workers are disabled and clear_on_start = false by @ljoss17 in #3365
• Configurable range for histogram buckets for latency metrics by @ljoss17 in #3382
• Improve ICA tests by @ljoss17 in #3415
• Add integration tests for fee granters by @ljoss17 in #3418
• Bump actions/checkout from 2 to 3 by @dependabot in #3425
• Fix some CI flakiness by ensuring we don't bind the same port twice on chain startup by @romac in #3422
• Set chain's gas_price based off of fetched chain data by @seanchen1991 in #3426
• Bump cachix/install-nix-action from 21 to 22 by @dependabot in #3433
• Install CA certificates in container by @romac in #3424
• Update simd v7 used in integration tests by @ljoss17 in #3427
• Increase the timeout on CI by @romac in #3436
• Misc cleanups by @romac in #3438
• Add CCV chain bootstrap to CI with Neutron and Gaia by @ljoss17 in #3451
• Bump serde from 1.0.164 to 1.0.166 by @dependabot in #3458
• Bump async-trait from 0.1.68 to 0.1.69 by @dependabot in #3459
• Bump erased-serde from 0.3.25 to 0.3.26 by @dependabot in #3460
• Document clock drift parameters in guide by @seanchen1991 in #3420
• Bump uuid from 1.3.3 to 1.4.0 by @dependabot in #3461
• Fix wrong sequence number in MsgTimeoutOnClose by @puneet2019 in #3440
• Include client updates for supporting messages when assembling messages to relay from the operational data by @romac in #3468
• Use max_expected_time_per_block value for the max_block_time by @ljoss17 in #3467
• config auto now generates a config file even when it encounters an error by @seanchen1991 in #3466
• Fix clippy warnings on Rust 1.71 by @romac in #3477
• Bump MSRV to 1.70 by @romac in #3479
• Fix: typos by @omahs in #3481
• Update guide templates by @romac in #3482
• Release Hermes v1.6.0 by @romac in #3476

New Contributors

• @puneet2019 made their first contribution in #3440

Full changes: v1.5.1...v1.6.0

v1.5.1

June 5th, 2023

This is a patch release for Hermes, which includes a single bugfix and enables overflow checks in production builds.

BUG FIXES

• Fix a panic which can occur when querying connections filtered by counterparty chain using hermes query connections (#3381)

IMPROVEMENTS

• Overflow checks are now enabled when Hermes is built in release mode, in order to better catch and address potential logic errors leading to overflows (#3390)

What's Changed

• Fix Docker release job by installing QEMU on the worker by @romac in #3367
• Remove a couple tools from the workspace, to cut down on dependencies needing to be built for releases by @romac in #3369
• Bump cachix/install-nix-action from 20 to 21 by @dependabot in #3368
• Fix guide checker by @romac in #3371
• Remove debug log noise from event_monitor by @ancazamfir in #3373
• Update dependencies by @romac in #3379
• Small cleanups by @romac in #3384
• Work around Clippy false positives on 1.70 by @romac in #3389
• Fix panic when querying connections filtered by counterparty chain by @romac in #3381
• Clean code by @DaviRain-Su in #3383
• Panic on overflow in release mode by @romac in #3390
• Release Hermes v1.5.1 by @romac in #3393

Full Changelog: v1.5.0...v1.5.1

v1.5.0

May 24th, 2023

🎉 Hermes v1.5.0 is here, packed with a slew of exciting updates, including breaking changes💥, brand-new features🎁, performance enhancements🚀, and sweeping improvements✨.

The one breaking change is the removal of the unbonding_period setting from the chain configuration. This is now replaced by a fresh ccv_consumer_chain setting for Cross-Chain Validation (CCV) consumer chains.

Also, Hermes has strengthened its misbehavior detection. With the mode.misbehaviour.enabled setting enabled (now the case by default) the relayer was already closely monitoring on-chain client updates, comparing submitted headers with those fetched from its RPC node. In the event of any discrepancy, Hermes would report the misbehaviour to the chain hosting the IBC client. As of this version, Hermes will also report the misbehaviour evidence to the reference chain.

This version rolls out a string of performance enhancements. Event batches are now delivered after a configurable delay, greatly trimming down latency when relaying, particularly on high-traffic channels. This can be adjusted using the batch_delay setting in the per-chain configuration. Plus, packet acknowledgments are only queried when there are packet commitments on the counterparty, resulting in a major speed boost for packet clearing and on-start scanning! 🚀

In addition, the trusted_node setting can now specify whether the full node Hermes connects to is trusted or not. If untrusted, the light client will verify headers included in the ClientUpdate message. However, a word of caution: configuring the full node as trusted may cut down latency but could risk sending invalid client updates to the chain. Use wisely! ⚠️
Our Hermes guide has been re-organized a bit, now featuring a new Performance Tuning page that details the settings for optimizing the performance of the relayer.

For all the debuggers out there, Hermes now equips a new --debug global flag with several selectable values, and two bonus flags, --archive-address and --restart-height that enable a client update following a genesis restart without an IBC upgrade proposal.

When it comes to telemetry, the destination chain is now added to the labels of the confirmed packet metrics.

Take note that some metrics now have the suffix _total. If you're using a running a Grafana dashboard or any other tool relying on the metric names or labels, an update might be needed. The corresponding page in the guide reflects the new metric names and labels for your convenience.

There's also a fresh configuration option to specify the directory used for the keyring store.

From this version onwards, multi-platform (arm64 and amd64) images will be distributed both on Docker Hub and the GitHub Content Repository.

Note for operators

⚠️ Be aware that this release contains a couple breaking
⚠️ changes to the Hermes configuration and telemetry metrics.
⚠️ Please consult the UPGRADING.md document for more details.

What's changed

• Improved tests for 'hermes config auto' by @ljoss17 in #3287
• Use dependabot for Github Actions by @faddat in #3304
• Publish multi-platform (arm64/amd64) images to Docker Hub and GHCR by @faddat in #3305
• Add changelog entry for #3303 by @romac in #3316
• Output profiling data as JSON, with additional metadata by @romac in #3329
• Allow gRPC calls with larger query responses by @ljoss17 in #3335
• Add destination chain to confirmed packet metrics by @ljoss17 in #3302
• Document metric label updates by @ljoss17 in #3343
• Add migaloo ics29 tests by @ljoss17 in #3346
• Emit events batch after a configurable delay by @romac in #3327
• Detect light client attacks and report them to both the chain and the RPC node by @romac in #3224
• Only query for packet acks when there are packet commitments on the counterparty by @romac in #3347
• Allow disabling header verification by setting trusted_node = true in the chain config by @romac in #3328
• Improve visibility of config auto command by @seanchen1991 in #3301
• Add --debug=rpc and --debug=profiling flags for enabling debug output by @romac in #3352
• Handle duplicate packet events and perform full clearing even in the presence of errors by @romac in #3361
• Revert "Send consumer misbehaviour message IcsConsumerMisbehaviour to the provider" by @romac in #3242
• Release Hermes v1.5.0 by @romac in #3363
  Full Changelog: v1.4.1...v1.5.0

v1.4.1

📖 CHANGELOG

v1.4.0

March 27th, 2023

Hermes v1.4.0 brings compatibility with chains based on Tendermint/CometBFT 0.37, while retaining compatibility with Tendermint/CometBFT 0.34. This is transparent and does not require any additional configuration.

The relayer now supports ICS consumer chains, which only requires operators to specify the unbonding_period parameter in the chain settings. This is a temporary requirement, in the future Hermes will seamlessy support consumer chains with minimal changes to the configuration.

This release also deprecates support for chains based on Cosmos SDK 0.43.x and lower, and bumps the compatibility to Cosmos SDK 0.47.x.

The relayer now also allows operators to filter out packets to relay based on whether or not they contain a fee, and the minimal amount of such fee.
Please check the relevant documentation in the Hermes guide for more information.
Additionally, Hermes now also tracks metrics for ICS29 fees.

This release includes a new query client status CLI to quickly check whether a client is active, expired or frozen.

---

📖 CHANGELOG

v1.3.0

February 17th, 2023

Hermes v1.3.0 adds support for Cross-chain Queries (ICS 031), implements optimistic channel handshake for lower latency, and comes with a major performance improvement when clearing packets on busy channels for chains using a recent version of IBC-Go.
This release also brings a few bug fixes related to closing ordered channels and upgrading clients.

---

📖 CHANGELOG

v1.2.0

Hermes v1.2.0

December 14th, 2022

Hermes v1.2.0 brings a bunch of new features and other improvements, such as support for Ed25519 keys, more robust health check which takes into account the Tendermint min_gas_price setting, and various bug fixes related to the handling of begin- and end-block events in the Tendermint indexer.

Additionally, IBC clients with a trust level between 1/3 and 2/3 inclusive are now allowed.

---

📖 CHANGELOG

v1.1.0

Hermes v1.1.0

August 29th, 2022

Note to developers

The ibc and ibc-proto crates have been split into their own repositories under the cosmos organization.

Moreover, Hermes will not be using the original ibc crate anymore, and will from now on use instead the ibc-relayer-types crate, which is a trimmed down version of the ibc crate that contains only the data structures used by Hermes.

This change does not impact end-users of Hermes, but may affect downstream consumers of the ibc-relayer library in some cases.

Please reach out to us if you encounter any issue following from this reorganization of the repository.

---

📖 CHANGELOG

v1.0.0

Hermes v1.0.0

August 22nd, 2022

After more than 2 years in the works, this is the first stable release of the Hermes relayer! 🎉

For reaching this milestone, we thank the valuable contributions of over 50 individuals, spread across more than 800 documented & resolved issues. Beside Cosmos-SDK and Tendermint, we are fortunate to maintain some of the most active and intense repository in the Cosmos ecosystem. Most importantly, we thank the relentless work of relayer operators that have already relayed billions worth of value in IBC production networks, and have provided us with invaluable feedback on improving Hermes and raising the overall stability of IBC. Kudos to everyone!

Note for operators

⚠️ If upgrading from Hermes v0.15.0, be aware that this release contains multiple breaking
⚠️ changes to the Hermes command-line interface and configuration.
⚠️ Please consult the UPGRADING document for instructions for more details.

Highlights

• The performance and reliability of the relayer has been greatly improved
• Merged commands keys add and keys restore into single command keys add
  The flag to specify the key name for the CLI command keys add has been changed
  from -n to -k. Restoring a key now takes a file containing the mnemonic as
  input instead of directly taking the mnemonic
• Deprecated gas_adjustment setting in favor of new gas_multiplier setting.
  Check out the upgrading instructions for more details about the new setting.
• Updated all CLI commands to take flags instead of positional arguments
• Renamed query packet unreceived-packets to query packet pending-sends
  and query packet unreceived-acks to query packet pending-acks
• Added CLI command keys balance which outputs the balance of an account associated with a key
• Added CLI command query channel client which outputs the channel's client state
• Added CLI command query transfer denom-trace which outputs the base denomination and path of a given trace hash
• Dropped the raw prefix from all the tx raw commands
• Remove the four duplicate commands:
  • tx raw update-client, which is the same as update client
  • tx raw upgrade-client, which is the same as upgrade client
  • tx raw upgrade-clients, which is the same as upgrade clients
  • tx raw create-client, which is the same as create client
• [A new section was added to guide][telemetry-guide] which describes how the new metrics
  can be used to observe both the current state of the relayer and the networks it is connected to
• Added many new metrics to the telemetry. The full list can be found in new the guide section linked above

---

📖 CHANGELOG