Skip to content

Test case fetch_retries_the_request_on_50x_errors failed randomly #492

Open
@tienvx

Description

@tienvx

The test case put_json_retries_the_request_on_50x_errors failed on musl-build job (with minimal log):

https://github.com/pact-foundation/pact-reference/actions/runs/15090810803/job/42419086482

thread 'pact_broker::tests::put_json_retries_the_request_on_50x_errors' panicked at /usr/local/cargo/registry/src/index.crates.io-1949cf8c6b5b557f/expectest-0.12.0/src/core/result.rs:81:13:
assertion failed: `expected to be equal to <3>, got <2>`, pact_verifier/src/pact_broker.rs:1403:5

Then on the next workflow run, it passed on musl-build job, but failed on v2-engine job (with more log):

https://github.com/pact-foundation/pact-reference/actions/runs/15093572402/job/42425278928?pr=486



failures:

---- pact_broker::tests::fetch_retries_the_request_on_50x_errors stdout ----
2025-05-18T07:25:08.197779Z DEBUG pact_consumer::mock_server::http_mock_server: Starting mock server from pact RequestResponsePact { consumer: Consumer { name: "RustPactVerifier" }, provider: Provider { name: "PactBrokerStub" }, interactions: [RequestResponseInteraction { id: None, description: "a request to a hal resource", provider_states: [ProviderState { name: "server returns a gateway error", params: {} }], request: Request { method: "GET", path: "/", query: None, headers: None, body: Missing, matching_rules: MatchingRules { rules: {PATH: MatchingRuleCategory { name: PATH, rules: {} }} }, generators: Generators { categories: {} } }, response: Response { status: 503, headers: None, body: Missing, matching_rules: MatchingRules { rules: {} }, generators: Generators { categories: {} } } }], metadata: {"pactRust": {"consumer": "1.3.4", "models": "1.3.1"}, "pactSpecification": {"version": "3.0.0"}}, specification_version: V3 }
2025-05-18T07:25:08.236125Z  INFO pact_verifier::pact_broker: Fetching path '/' from pact broker
2025-05-18T07:25:08.236178Z DEBUG pact_verifier::pact_broker: Final broker URL: http://127.0.0.1:41115/
2025-05-18T07:25:08.236311Z DEBUG reqwest::connect: starting new connection: http://127.0.0.1:41115/    
2025-05-18T07:25:08.236345Z DEBUG hyper_util::client::legacy::connect::http: connecting to 127.0.0.1:41115
2025-05-18T07:25:08.236504Z DEBUG pact_mock_server::hyper_server: Received connection from remote 127.0.0.1:44536
2025-05-18T07:25:08.236666Z DEBUG hyper_util::client::legacy::connect::http: connected to 127.0.0.1:41115
2025-05-18T07:25:08.237047Z DEBUG pact_mock_server::hyper_server: Creating pact request from hyper request path=/
2025-05-18T07:25:08.237075Z DEBUG pact_mock_server::hyper_server: Extracting query from uri /
2025-05-18T07:25:08.237136Z  INFO pact_mock_server::hyper_server: Received request GET /
2025-05-18T07:25:08.237144Z DEBUG pact_mock_server::hyper_server: 
      ----------------------------------------------------------------------------------------
       method: GET
       path: /
       query: None
       headers: Some({"user-agent": ["pact_verifier/1.2.6"], "accept-encoding": ["gzip", "deflate"], "accept": ["application/hal+json", "application/json"], "host": ["127.0.0.1:41115"]})
       body: Empty
      ----------------------------------------------------------------------------------------
      
2025-05-18T07:25:08.237227Z DEBUG pact_matching: comparing to expected HTTP Request ( method: GET, path: /, query: None, headers: None, body: Missing )
2025-05-18T07:25:08.237243Z DEBUG pact_matching:      body: ''
2025-05-18T07:25:08.237249Z DEBUG pact_matching:      matching_rules: MatchingRules { rules: {PATH: MatchingRuleCategory { name: PATH, rules: {} }} }
2025-05-18T07:25:08.237257Z DEBUG pact_matching:      generators: Generators { categories: {} }
2025-05-18T07:25:08.237289Z DEBUG pact_matching::matchers: String -> String: comparing '/' to '/' ==> true cascaded=false matcher=Equality
2025-05-18T07:25:08.237304Z DEBUG pact_matching: expected content type = '*/*', actual content type = '*/*'
2025-05-18T07:25:08.237341Z DEBUG pact_matching: content type header matcher = 'RuleList { rules: [], rule_logic: And, cascaded: false }'
2025-05-18T07:25:08.237369Z DEBUG pact_matching: --> Mismatches: []
2025-05-18T07:25:08.237433Z DEBUG pact_mock_server::hyper_server: Test context = {"mockServer": Object {"port": Number(41115), "url": String("127.0.0.1:41115")}}
2025-05-18T07:25:08.237448Z  INFO pact_mock_server::hyper_server: Request matched, sending response
2025-05-18T07:25:08.237454Z DEBUG pact_mock_server::hyper_server: 
          ----------------------------------------------------------------------------------------
           status: 503
           headers: None
           body: Missing
          ----------------------------------------------------------------------------------------
          
2025-05-18T07:25:08.237704Z DEBUG hyper_util::client::legacy::pool: pooling idle connection for ("http", 127.0.0.1:41115)
2025-05-18T07:25:08.339140Z DEBUG hyper_util::client::legacy::pool: reuse idle connection for ("http", 127.0.0.1:41115)
2025-05-18T07:25:08.339375Z DEBUG pact_mock_server::hyper_server: Creating pact request from hyper request path=/
2025-05-18T07:25:08.339402Z DEBUG pact_mock_server::hyper_server: Extracting query from uri /
2025-05-18T07:25:08.339466Z  INFO pact_mock_server::hyper_server: Received request GET /
2025-05-18T07:25:08.339477Z DEBUG pact_mock_server::hyper_server: 
      ----------------------------------------------------------------------------------------
       method: GET
       path: /
       query: None
       headers: Some({"host": ["127.0.0.1:41115"], "user-agent": ["pact_verifier/1.2.6"], "accept": ["application/hal+json", "application/json"], "accept-encoding": ["gzip", "deflate"]})
       body: Empty
      ----------------------------------------------------------------------------------------
      
2025-05-18T07:25:08.339518Z DEBUG pact_matching: comparing to expected HTTP Request ( method: GET, path: /, query: None, headers: None, body: Missing )
2025-05-18T07:25:08.339529Z DEBUG pact_matching:      body: ''
2025-05-18T07:25:08.339535Z DEBUG pact_matching:      matching_rules: MatchingRules { rules: {PATH: MatchingRuleCategory { name: PATH, rules: {} }} }
2025-05-18T07:25:08.339543Z DEBUG pact_matching:      generators: Generators { categories: {} }
2025-05-18T07:25:08.339606Z DEBUG pact_matching::matchers: String -> String: comparing '/' to '/' ==> true cascaded=false matcher=Equality
2025-05-18T07:25:08.339624Z DEBUG pact_matching: expected content type = '*/*', actual content type = '*/*'
2025-05-18T07:25:08.339666Z DEBUG pact_matching: content type header matcher = 'RuleList { rules: [], rule_logic: And, cascaded: false }'
2025-05-18T07:25:08.339687Z DEBUG pact_matching: --> Mismatches: []
2025-05-18T07:25:08.339760Z DEBUG pact_mock_server::hyper_server: Test context = {"mockServer": Object {"port": Number(41115), "url": String("127.0.0.1:41115")}}
2025-05-18T07:25:08.339779Z  INFO pact_mock_server::hyper_server: Request matched, sending response
2025-05-18T07:25:08.339785Z DEBUG pact_mock_server::hyper_server: 
          ----------------------------------------------------------------------------------------
           status: 503
           headers: None
             at /rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/core/src/ptr/mod.rs:523:1
   6: pact_verifier::pact_broker::tests::fetch_retries_the_request_on_50x_errors::{{closure}}
             at ./src/pact_broker.rs:1331:80
   7: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/core/src/future/future.rs:124:9
   8: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/core/src/future/future.rs:124:9
   9: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
             at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/scheduler/current_thread/mod.rs:733:54
  10: tokio::task::coop::with_budget
             at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/task/coop/mod.rs:167:5
  11: tokio::task::coop::budget
             at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/task/coop/mod.rs:133:5
  12: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
             at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/scheduler/current_thread/mod.rs:733:25
  13: tokio::runtime::scheduler::current_thread::Context::enter
             at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/scheduler/current_thread/mod.rs:432:19
  14: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
             at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/scheduler/current_thread/mod.rs:732:36
  15: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
             at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/scheduler/current_thread/mod.rs:820:68
  16: tokio::runtime::context::scoped::Scoped<T>::set
             at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/context/scoped.rs:40:9
  17: tokio::runtime::context::set_scheduler::{{closure}}
             at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/context.rs:180:26
  18: std::thread::local::LocalKey<T>::try_with
             at /rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/std/src/thread/local.rs:311:12
  19: std::thread::local::LocalKey<T>::with
             at /rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/std/src/thread/local.rs:275:15
  20: tokio::runtime::context::set_scheduler
             at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/context.rs:180:9
  21: tokio::runtime::scheduler::current_thread::CoreGuard::enter
             at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/scheduler/current_thread/mod.rs:820:27
  22: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
             at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/scheduler/current_thread/mod.rs:720:19
  23: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
             at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/scheduler/current_thread/mod.rs:200:28
  24: tokio::runtime::context::runtime::enter_runtime
             at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/context/runtime.rs:65:16
  25: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
             at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/scheduler/current_thread/mod.rs:188:9
  26: tokio::runtime::runtime::Runtime::block_on_inner
             at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/runtime.rs:368:47
  27: tokio::runtime::runtime::Runtime::block_on
             at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/runtime.rs:342:13
  28: pact_verifier::pact_broker::tests::fetch_retries_the_request_on_50x_errors
             at ./src/pact_broker.rs:1317:54
  29: pact_verifier::pact_broker::tests::fetch_retries_the_request_on_50x_errors::{{closure}}
             at ./src/pact_broker.rs:1317:53
  30: core::ops::function::FnOnce::call_once
             at /rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/core/src/ops/function.rs:250:5
  31: core::ops::function::FnOnce::call_once
             at /rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/core/src/ops/function.rs:250:5

On the next workflow run, they all passed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions