Skip to content

Update lock hierarchy devdocs #58884

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: master
Choose a base branch
from

Conversation

xal-0
Copy link
Member

@xal-0 xal-0 commented Jul 3, 2025

Brings the devdocs up to date for the lock hierarchy we have now, documenting the deadlock avoidance algorithm we use for method locks. The information about which locks to take to update global data has been removed because it was out of date, and some basic comments about locking has been added to julia.h.

Also removes an unused mutex, extern_c_lock, which I found while looking for locks.

TODO:

  • A few fields that have questionable locking (usings_backedges, jl_binding_t.backedges) will be fixed later.
  • Document how the atomic fields are updated/read.

@xal-0 xal-0 added the docs This change adds or pertains to documentation label Jul 3, 2025
* `world_counter_lock`
* `precomp_statement_out_lock`
* `dispatch_statement_out_lock`

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file still appears to have a lot of problems, and so is unsafe to load code when starting julia with threads.

  * loading.jl: `require` and `register_root_module`

    > This file potentially has numerous problems.
    >
    > fix: needs locks

Comment on lines -108 to -112
* toplevel

> doesn't exist right now
>
> fix: create it
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also may deserve a call out still, since this synchronization for staticdata.c does exist now, but is quite unusual: there is a call to a function called jl_safepoint_suspend_all_threads, which causes all other threads to block at their next safepoint/safe-region until that lock is released. It therefore sits at about level 2 of the hierarchy, in that no lock is permitted to be acquired inside, unless that lock disallows having safepoints (which is level 1).

* `cfun_lock`

### Level 7
* `world_counter_lock`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want to now document the world_counter update synchronization logic too. In particular, all inference and backedges are inserted without holding any locks against the current-world as of the start of inference. Then it does a call to jl_promote_cis_to_current, which atomically changes all code instances to use an unbounded world cap, only if there has not been a change to the world counter (as protected by the world counter lock). Since it would be impossible to correctly insert edges to Julia code while concurrently holding the world_counter lock, recursive acquisition of world_counter_lock here needs to be handled very carefully.

This also means this is the wrong place for this lock in the hierarchy, since it frequently gets called while holding several of those Level 6 locks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs This change adds or pertains to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants