Skip to content

Commit b6301aa

Browse files
authored
feat: Introduce retry mechanism for Email notifier (#282)
1 parent f5e8c02 commit b6301aa

File tree

30 files changed

+402
-334
lines changed

30 files changed

+402
-334
lines changed

Cargo.lock

Lines changed: 27 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ actix-web = "4"
2121
alloy = { version = "0.15.6", features = ["full"] }
2222
anyhow = { version = "1.0.97", features = ["std"] }
2323
async-trait = "0.1"
24+
backon = "1.5.1"
2425
base64 = "0.22"
2526
byte-unit = "5.1.6"
2627
chrono = "0.4"
@@ -34,7 +35,7 @@ glob = "0.3"
3435
hex = "0.4"
3536
hmac = "0.12.0"
3637
lazy_static = "1.5"
37-
lettre = "0.11.11"
38+
lettre = { version = "0.11.11", features = ["tokio1", "tokio1-native-tls"] }
3839
libc = "0.2"
3940
log = "0.4"
4041
oz-keystore = "0.1.4"

docs/modules/ROOT/pages/index.adoc

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -246,6 +246,7 @@ Following notification protocols support retry policies:
246246
* Discord
247247
* Telegram
248248
* Webhook
249+
* Email
249250

250251
Default retry policy is using exponential backoff with the following parameters:
251252
[cols="1,1,1"]
@@ -258,7 +259,7 @@ Default retry policy is using exponential backoff with the following parameters:
258259
| `jitter` | `Full` | Jitter strategy to apply to the backoff duration, currently supports `Full` and `None`
259260
|===
260261

261-
These parameters can be overridden by providing custom `HttpRetryConfig` struct in `retry_policy` field in trigger configuration.
262+
These parameters can be overridden by providing custom `RetryConfig` struct in `retry_policy` field in trigger configuration.
262263

263264
===== Script Security
264265

docs/modules/ROOT/pages/rpc.adoc

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -111,7 +111,7 @@ This middleware handles:
111111

112112
* Automatic retries for transient HTTP errors (e.g., 5xx server errors, network timeouts) for requests made to the *currently active RPC URL*.
113113
* An exponential backoff policy between these retry attempts.
114-
* Parameters like the number of retries, backoff durations, and jitter are defined in an `HttpRetryConfig` struct (see <<Configuration Options>>).
114+
* Parameters like the number of retries, backoff durations, and jitter are defined in an `RetryConfig` struct (see <<Configuration Options>>).
115115
* This same-endpoint retry mechanism is independent of, and operates before, the endpoint rotation logic. If all same-endpoint retries fail for the current URL, the error is then processed by the `EndpointManager`.
116116

117117
==== Endpoint Rotation (via `EndpointManager`)
@@ -120,26 +120,26 @@ If all same-endpoint retries fail for the currently active RPC URL, or if certai
120120

121121
==== Configuration Options
122122

123-
The same-endpoint retry behavior is configured via the `HttpRetryConfig` struct, which is used by `create_retryable_http_client` to set up the `ExponentialBackoff` policy for `reqwest-retry`.
123+
The same-endpoint retry behavior is configured via the `RetryConfig` struct, which is used by `create_retryable_http_client` to set up the `ExponentialBackoff` policy for `reqwest-retry`.
124124

125-
The default settings for `HttpRetryConfig` result in an `ExponentialBackoff` policy approximately equivalent to:
125+
The default settings for `RetryConfig` result in an `ExponentialBackoff` policy approximately equivalent to:
126126
[source,rust]
127127
----
128-
// This illustrates the default policy created by HttpRetryConfig::default()
128+
// This illustrates the default policy created by RetryConfig::default()
129129
// and create_retryable_http_client.
130-
let http_retry_config = HttpRetryConfig::default();
130+
let http_retry_config = RetryConfig::default();
131131
let retry_policy = ExponentialBackoff::builder()
132132
.base(http_retry_config.base_for_backoff)
133133
.retry_bounds(http_retry_config.initial_backoff, http_retry_config.max_backoff)
134134
.jitter(http_retry_config.jitter)
135135
.build_with_max_retries(http_retry_config.max_retries);
136136
----
137137

138-
The configurable options are defined in the `HttpRetryConfig` struct:
138+
The configurable options are defined in the `RetryConfig` struct:
139139
[source,rust]
140140
----
141141
// In utils::http
142-
pub struct HttpRetryConfig {
142+
pub struct RetryConfig {
143143
/// Maximum number of retries for transient errors (after the initial attempt).
144144
pub max_retries: u32,
145145
/// Initial backoff duration before the first retry.
@@ -156,7 +156,7 @@ pub struct HttpRetryConfig {
156156
The client architecture ensures efficient resource use and consistent retry behavior:
157157

158158
1. A single base `reqwest::Client` is created by `HttpTransportClient` with optimized connection pool settings. This base client is shared.
159-
2. The `create_retryable_http_client` utility takes this base client and an `HttpRetryConfig` to produce a `ClientWithMiddleware`.
159+
2. The `create_retryable_http_client` utility takes this base client and an `RetryConfig` to produce a `ClientWithMiddleware`.
160160
3. This `ClientWithMiddleware` (the "retryable client") is then used for all HTTP operations within `HttpTransportClient`, including initial health checks, requests sent via `EndpointManager`, and `try_connect` calls during rotation. This ensures all operations benefit from the configured retry policy and the shared connection pool.
161161

162162
Each transport client may define its own retry policy:
@@ -172,7 +172,7 @@ pub struct HttpTransportClient {
172172
173173
// Example of client creation with retry mechanism
174174
// Use default retry policy
175-
let http_retry_config = HttpRetryConfig::default();
175+
let http_retry_config = RetryConfig::default();
176176
177177
// Create the base HTTP client
178178
let base_http_client = reqwest::ClientBuilder::new()

src/models/config/trigger_config.rs

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -295,6 +295,7 @@ impl ConfigLoader for Trigger {
295295
message,
296296
sender,
297297
recipients,
298+
retry_policy: _,
298299
} = &self.config
299300
{
300301
// Validate host
@@ -703,7 +704,7 @@ mod tests {
703704
use crate::models::NotificationMessage;
704705
use crate::models::{core::Trigger, ScriptLanguage, SecretString};
705706
use crate::utils::tests::builders::trigger::TriggerBuilder;
706-
use crate::utils::HttpRetryConfig;
707+
use crate::utils::RetryConfig;
707708
use std::{fs::File, io::Write, os::unix::fs::PermissionsExt};
708709
use tempfile::TempDir;
709710
use tracing_test::traced_test;
@@ -1446,7 +1447,7 @@ mod tests {
14461447
title: "Test".to_string(),
14471448
body: "x".repeat(TELEGRAM_MAX_BODY_LENGTH + 1), // Exceeds max length
14481449
},
1449-
retry_policy: HttpRetryConfig::default(),
1450+
retry_policy: RetryConfig::default(),
14501451
},
14511452
};
14521453
assert!(max_body_length.validate().is_err());
@@ -1465,7 +1466,7 @@ mod tests {
14651466
title: "Test".to_string(),
14661467
body: "z".repeat(DISCORD_MAX_BODY_LENGTH + 1), // Exceeds max length
14671468
},
1468-
retry_policy: HttpRetryConfig::default(),
1469+
retry_policy: RetryConfig::default(),
14691470
},
14701471
};
14711472
assert!(max_body_length.validate().is_err());

src/models/core/trigger.rs

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
use crate::{
22
models::{core::ScriptLanguage, SecretValue},
3-
utils::HttpRetryConfig,
3+
utils::RetryConfig,
44
};
55
use email_address::EmailAddress;
66
use serde::{Deserialize, Serialize};
@@ -61,7 +61,7 @@ pub enum TriggerTypeConfig {
6161
message: NotificationMessage,
6262
/// Retry policy for HTTP requests
6363
#[serde(default)]
64-
retry_policy: HttpRetryConfig,
64+
retry_policy: RetryConfig,
6565
},
6666
/// Email notification configuration
6767
Email {
@@ -79,6 +79,9 @@ pub enum TriggerTypeConfig {
7979
sender: EmailAddress,
8080
/// Email recipients
8181
recipients: Vec<EmailAddress>,
82+
/// Retry policy for SMTP requests
83+
#[serde(default)]
84+
retry_policy: RetryConfig,
8285
},
8386
/// Webhook configuration
8487
Webhook {
@@ -94,7 +97,7 @@ pub enum TriggerTypeConfig {
9497
message: NotificationMessage,
9598
/// Retry policy for HTTP requests
9699
#[serde(default)]
97-
retry_policy: HttpRetryConfig,
100+
retry_policy: RetryConfig,
98101
},
99102
/// Telegram notification configuration
100103
Telegram {
@@ -108,7 +111,7 @@ pub enum TriggerTypeConfig {
108111
message: NotificationMessage,
109112
/// Retry policy for HTTP requests
110113
#[serde(default)]
111-
retry_policy: HttpRetryConfig,
114+
retry_policy: RetryConfig,
112115
},
113116
/// Discord notification configuration
114117
Discord {
@@ -118,7 +121,7 @@ pub enum TriggerTypeConfig {
118121
message: NotificationMessage,
119122
/// Retry policy for HTTP requests
120123
#[serde(default)]
121-
retry_policy: HttpRetryConfig,
124+
retry_policy: RetryConfig,
122125
},
123126
/// Script execution configuration
124127
Script {
@@ -136,7 +139,7 @@ pub enum TriggerTypeConfig {
136139

137140
impl TriggerTypeConfig {
138141
/// Get the retry policy for the trigger type, if applicable.
139-
pub fn get_retry_policy(&self) -> Option<HttpRetryConfig> {
142+
pub fn get_retry_policy(&self) -> Option<RetryConfig> {
140143
match self {
141144
Self::Slack { retry_policy, .. } => Some(retry_policy.clone()),
142145
Self::Discord { retry_policy, .. } => Some(retry_policy.clone()),

src/services/blockchain/transports/http.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ use crate::{
2222
BlockchainTransport, EndpointManager, RotatingTransport, TransientErrorRetryStrategy,
2323
TransportError,
2424
},
25-
utils::http::{create_retryable_http_client, HttpRetryConfig},
25+
utils::http::{create_retryable_http_client, RetryConfig},
2626
};
2727

2828
/// Basic HTTP transport client for blockchain interactions
@@ -71,7 +71,7 @@ impl HttpTransportClient {
7171

7272
// Create a retry policy with default settings
7373
// Shared config for endpoint manager and test connection
74-
let http_retry_config = HttpRetryConfig::default();
74+
let http_retry_config = RetryConfig::default();
7575

7676
// Create the base HTTP client
7777
let base_http_client = Arc::new(

src/services/notification/discord.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -185,7 +185,7 @@ impl Notifier for DiscordNotifier {
185185
mod tests {
186186
use crate::{
187187
models::{NotificationMessage, SecretString, SecretValue},
188-
utils::{tests::create_test_http_client, HttpRetryConfig},
188+
utils::{tests::create_test_http_client, RetryConfig},
189189
};
190190

191191
use super::*;
@@ -209,7 +209,7 @@ mod tests {
209209
title: "Test Alert".to_string(),
210210
body: "Test message ${value}".to_string(),
211211
},
212-
retry_policy: HttpRetryConfig::default(),
212+
retry_policy: RetryConfig::default(),
213213
}
214214
}
215215

0 commit comments

Comments
 (0)