Skip to content

Dynamic worker_args within the same pool #151

@hickscorp

Description

@hickscorp

Hello,

Requirements

The requirements of my request would sound like this:

  • As a user of Poolex who would like to have different arguments passed for workers within the same pool,
  • I would like to have an option to pass a function or MFA to worker_args,
  • So that I could round-robin things without having to spin-up more pools.

TL;DR

Here's an interesting use-case. Imagine having a bunch of Antivirus hosts. Say 4. They are identified within Elixir land by a {host, port} tuple. I'm looking for a way to do something like this?

  def pool_spec(hosts)
      when is_list(hosts),
      do: [
        pool_id: pool_id(),
        worker_module: @me,
        worker_args: WHAT_GOES_HERE?,
        workers_count: 4,
        max_overflow: 4
      ]

My goal would be to maybe pass a function rather than a literal in worker_args, expecting that it would be called by Poolex at runtime when starting each worker along with some sort of identifier for that worker, so that I could "round-robin" the hosts - imagine something like this:

  def pool_spec(hosts)
      when is_list(hosts),
      do: [
        pool_id: pool_id(),
        worker_module: @me,
        worker_args: &[Enum.at(rem(length(hosts), &1))],
        workers_count: 4,
        max_overflow: 4
      ]

Or something like that... I saw that there's a PR about this (maybe?) and I was wondering if there was already a way to do so?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions