Per-thread variables for parallel jobs

[PhilShaughnes]

Is there a way to provide a variable that is unique or specified for each thread?
For example, say I am testing some jobs that require locking behavior: https://url/job/namespace
Where:

• each namespace is locked while the job is running
• subsequent requests will just wait to try and get a lock

When running multiple jobs, this means I can't actually run tests against multiple namespaces simultaneously(e.g. namespace-one, namespace-two, namespace-three, etc) without wrapping hurl in parallel or something similar.

Ideally each file could have something like POST https://url/job/test_{{SUFFIX}}
and then I could specify --job-variables SUFFIX="red,green,blue" or something (indicates run a max of 3 jobs)?

[lepapareil]

Hi @PhilShaughnes :)

If you're talking about running variations of the same Hurl file like in a matrix strategy, for now, you can create a Hurl template like request-path.hurl.tpl, and with a simple loop, generate static files like request-path1.hurl, request-pat2.hurl and then just run in parallel as usual with hurl request-path*.hurl.

Either way, we can have a debate about the need of having a new --matrix name=[value1,value2,value3,...] option that makes Hurl execute input files using this var on a matrix strategy way.

[PhilShaughnes]

Thanks for the suggestion! Generating static files from a template is a great workaround, but it still requires preprocessing or external parallelization, and doesn’t handle dynamically-determined variables easily.

While a full combinatorial matrix might be useful in some cases, that’s more complex than the simple parameterized parallel execution I’m proposing. My main use case is running multiple files in parallel with a variable assigned per worker. For example:
hurl --job-var ORG="red,green,blue" --test ./tests/*.hurl

This would let Hurl assign a variable per worker internally and run everything in parallel, without generating temporary files or separately managing parallel execution (e.g., background processes or GNU parallel).

[fabricereix]

If we had array variable colors=["red", "green", "blue"] and implicit variable THREAD_INDEX set automatically for each thread,
you could use something like {{colors nth THREAD_INDEX}}

[lepapareil]

As discussed with @fabricereix, maybe we can take it to another level, inspired by all features coming soon as #3629 and #3209.

Why not just creating a --option-file <option file>, in which we could declare any options for the run, and then just give Hurl the power to make multiple exec on multiline input.

We already have --variable-file which contains a list of variables, allowing us to manage profiles of variables, I suggest the same logic but to manage profiles of options.

Here is an example that can resolve this use case, and even going further with multi-level data :

hurl file test.hurl :

GET https://fake.api/{{team_color}}/{{player_id}}
HTTP 200

The option file team_data.scenario with the multi-level data :

--variable team_color=red --variable player_id=1
--variable team_color=red --variable player_id=2
--variable team_color=green --variable player_id=1
--variable team_color=green --variable player_id=2

The run :

$ hurl --option-file=team_data.scenario test.hurl

This will execute 4 times test.hurl, each thread with options list derived from each line of team_data.scenario.

Another example for another need, instead of chaining 4 Hurl command with different --variable-file values :

hurl --variable-file=prod.hurl-vars test.hurl --report-html=report
hurl --variable-file=staging.hurl-vars test.hurl --report-html=report
hurl --variable-file=ua.hurl-vars test.hurl --retry 2
hurl --variable-file=ba.hurl-vars test.hurl --retry 2

We should be able to write this option file all_envs.scenario:

--variable-file=prod.hurl-vars --report-html=report
--variable-file=staging.hurl-vars --report-html=report
--variable-file=ua.hurl-vars --retry 2
--variable-file=ba.hurl-vars --retry 2

And then run :

hurl --option-file=all_envs.scenario test.hurl

Maybe just too complicated, but I'm using this chat to make sure I don't lose the details of our oral discussions. ;)

[PhilShaughnes]

These are both really interesting ideas — I especially like the direction of THREAD_INDEX, and the --option-file concept sounds powerful for defining more complex scenarios.

That said, my original proposal was a bit narrower: a per-worker variable assignment (e.g. --job-var ORG="red,green,blue") that ties into the existing --jobs mechanism. The nice thing about that model is that the .hurl files themselves don’t need to know anything about threading or execution details — they just use {{ORG}}, and Hurl distributes those values internally.

The (colors nth THREAD_INDEX) approach technically works, but it’s a bit obtuse — it requires every file that might need parallel variables to explicitly know about and use thread-specific logic. The --job-var model keeps that concern outside the test definition, which feels like a cleaner boundary.

The --option-file idea also seems really cool, but I’m not sure it would solve this use case — or at least it’s ambiguous. It’s not clear whether it would run each file in test/*.hurl multiple times (once per option line) or assign one option line per worker/file in sequence. My goal is more the latter: keep the current multi-file behavior, but allow each worker to have unique variables while processing its share of files.

[jcamiel]

Hi all, just my two cents.

If you want to add a unique variable value per file you can use an [Options] section. For example, let's say we have three files:

a.hurl

GET http://localhost:8000/hello
[Options]
variable: job_name=a
HTTP 200


GET http://localhost:8000/echo?t={{job_name}}
HTTP 200

b.hurl

GET http://localhost:8000/hello
[Options]
variable: job_name=b
HTTP 200


GET http://localhost:8000/echo?t={{job_name}}
HTTP 200

c.hurl

GET http://localhost:8000/hello
[Options]
variable: job_name=c
HTTP 200


GET http://localhost:8000/echo?t={{job_name}}
HTTP 200

The first option section in each file:

[Options]
variable: job_name=x

Will declare a variable job_name with value x. This variable is valid for the whole file.
Running it :

$ hurl --repeat 2 a.hurl b.hurl c.hurl

Will produces:

a
b
c
a
b
c

Now in parallel hurl --parallel --repeat 2 a.hurl b.hurl c.hurl:

c
b
a
b
a
c

Now, it you repeat this sequence hurl --parallel --repeat 2 a.hurl b.hurl c.hurl each file will reuse its job value. If you want to have unique value for each file, even repeated, you can use functions:

GET http://localhost:8000/hello
[Options]
variable: job_name=a_{{newUuid}}
HTTP 200


GET http://localhost:8000/echo?t={{job_name}}
HTTP 200

c_25315209-4509-4ea4-a575-b5ad00c15825
a_3ed70c95-84a0-4ca2-899d-89fe57c01865
b_71bcc29b-4311-4a60-a99b-56070be23d9e
c_9e9abfd3-1864-4d90-b663-e41874fe3c06
b_d76b5e7c-5e92-4c2a-ac51-2050cae13500
a_be12e50d-fe2c-4c36-b046-f90a55796fef