How can we set this correlation_id in celery if we call .apply() to trigger a task?

[lukehsiao]

Suppose we're running in a celery task already, and want to additional run another celery task based on some condition. A common way to do this is to call the other task from within the first with .apply(). However, doing so loses the correlation_id if set up as described in the README.

[lukehsiao]

Was able to solve this by just providing headers= to the .apply() call and grabbing that in

@task_prerun.connect()
def load_correlation_id(task) -> None:
    # This is called when the worker picks up the task
    # Here we're able to load the correlation ID from the headers
    id_value = task.request.get(header_key)
    correlation_id.set(id_value)

However, note that we needed to use task.request.headers.get rather than task.request.get directly.