Skip to content

Conversation

@heemankv
Copy link
Contributor

@heemankv heemankv commented Jun 18, 2025

Resolves #590: Optimize Orchestrator Chain Settlement Logic

Problem

The Orchestrator currently halts the entire chain settlement process when any job reaches Failed or Verification Timeout status, leading to inefficient processing and unnecessary blocking of independent operations.

Current problematic behavior:

  • Chain settlement stops completely on any job failure
  • Independent jobs are blocked unnecessarily (e.g., StateTransition for block 5 is halted when SnosRun for block 10 fails)
  • Cascading delays affect the entire pipeline

Solution

This PR optimizes the worker logic to continue processing independent jobs even when unrelated jobs fail, improving overall system resilience and throughput.

@heemankv heemankv self-assigned this Jun 18, 2025
@heemankv heemankv marked this pull request as ready for review June 24, 2025 04:29
@prkpndy prkpndy self-requested a review July 5, 2025 12:22
Copy link
Member

@Mohiiit Mohiiit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you clarify the difference between context implementation and the trigger implementation here? I see it's been used interchangeably or maybe I am missing something and do we really need that segregation?

@Trantorian1 Trantorian1 moved this to In review in Madara Jul 10, 2025
Copy link
Contributor

@Trantorian1 Trantorian1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my limited experience with orchestrator, lgtm 👌

Comment on lines +173 to +176
pub struct ProcessingContext {
pub config: Arc<Config>,
pub earliest_failed_block: Option<u64>,
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate struct?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

struct ProcessingContext {
config: Arc<Config>,
earliest_failed_block: Option<u64>,
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

Orchestrator Prod Readiness : Optimise stopping on Failed jobs.

5 participants