Thread safety tracker

[ViralBShah]

One of the first things we need to do is make the runtime thread safe. This work is on the threads branch, and this tracker predates the new GC. I thought it is worth capturing a tracker that @StefanKarpinski prepared earlier in an issue to ease the thread safety work.

This list is organized as Variable; Approach

builtins.c

• extern size_t jl_page_size; constant
• extern int jl_in_inference; lock
• extern int jl_boot_file_loaded; constant
• int in_jl_ = 0; thread-local

ccall.cpp

• static std::mapstd::stringstd::string sonameMap; lock
• static bool got_sonames = false; lock, write-once
• static std::mapstd::stringuv_lib_t* libMap; lock
• static std::mapstd::stringGlobalVariable* libMapGV; lock
• static std::mapstd::stringGlobalVariable* symMapGV; lock
• static char *temp_arg_area; thread-local (will be deleted very soon)
• static const uint32_t arg_area_sz = 4196; constant (will be deleted very soon)
• static uint32_t arg_area_loc; thread-local (will be deleted very soon)
• static void *temp_arg_blocks[N_TEMP_ARG_BLOCKS]; thread-local (will be deleted very soon)
• static uint32_t arg_block_n = 0; thread-local (will be deleted very soon)
• static Function *save_arg_area_loc_func; constant (will be deleted very soon)
• static Function *restore_arg_area_loc_func; constant (will be deleted very soon)

cgutils.cpp

• static std::map<const std::stringGlobalVariable*> stringConstants; lock
• static std::map<void*jl_value_llvm> jl_value_to_llvm; lock
• static std::map<Value void> llvm_to_jl_value; lock
• static std::vector<Constant*> jl_sysimg_gvars; lock
• static std::map<intjl_value_t*> typeIdToType; lock
• jl_array_t *typeToTypeId; lock
• static int cur_type_id = 1; lock

codegen.cpp

• void *__stack_chk_guard = NULL; thread-local (jwn: why is this on the list? it's a constant and not thread local)

debuginfo.cpp

• extern "C" volatile int jl_in_stackwalk;
• JuliaJITEventListener *jl_jit_events;
• static obfiletype objfilemap;
• extern char *jl_sysimage_name; constant
• static logdata_t coverageData;
• static logdata_t mallocData;

dump.c

• static jl_array_t *tree_literal_values=NULL; thread-local
• static jl_value_t *jl_idtable_type=NULL; constant
• static jl_array_t *datatype_list=NULL; thread 0 only
• jl_value_t ***sysimg_gvars = NULL; thread 0 only
• extern int globalUnique; thread 0 only
• static size_t delayed_fptrs_n = 0; thread 0 only
• static size_t delayed_fptrs_max = 0; thread 0 only

gc.c

• static volatile size_t allocd_bytes = 0; thread-local
• static volatile int64_t total_allocd_bytes = 0; thread-local
• static int64_t last_gc_total_bytes = 0; thread-local
• static size_t freed_bytes = 0; barrier
• static uint64_t total_gc_time=0; barrier
• int jl_in_gc=0; * referenced from switchto task.c barrier
• static htable_t obj_counts; barrier
• static size_t total_freed_bytes=0; barrier
• static arraylist_t to_finalize; barrier
• static jl_value_t **mark_stack = NULL; barrier
• static size_t mark_stack_size = 0; barrier
• static size_t mark_sp = 0; barrier
• extern jl_module_t *jl_old_base_module; constant
• extern jl_array_t *typeToTypeId; barrier
• extern jl_array_t *jl_module_init_order; barrier
• static int is_gc_enabled = 1; atomic
• static double process_t0; constant

init.c

• char *jl_stack_lo; thread-local
• char *jl_stack_hi; thread-local
• volatile sig_atomic_t jl_signal_pending = 0; thread-local
• volatile sig_atomic_t jl_defer_signal = 0; thread-local
• uv_loop_t *jl_io_loop; I/O thread ?
• static void *signal_stack; thread-local (see Random CI failures rather intense right now #9763 (comment))
• static mach_port_t segv_port = 0; constant
• extern void * __stack_chk_guard; thread-local (duplicate of above)

jltypes.c

• int inside_typedef = 0; thread-local
• static int match_intersection_mode = 0; thread-local
• static int has_ntuple_intersect_tuple = 0; thread-local
• static int t_uid_ctr = 1; lock

llvm-simdloop.cpp

• static unsigned simd_loop_mdkind = 0; constant
• static MDNode* simd_loop_md = NULL; constant
• char LowerSIMDLoop::ID = 0; lock

module.c

• jl_module_t *jl_main_module=NULL; constant
• jl_module_t *jl_core_module=NULL; constant
• jl_module_t *jl_base_module=NULL; constant
• jl_module_t *jl_current_module=NULL; thread-local
• jl_array_t *jl_module_init_order = NULL; lock (this code is bady broken anyways: module init order is wrong and can cause segfaults #9799)

profile.c

• static volatile ptrint_t* bt_data_prof = NULL;
• static volatile size_t bt_size_max = 0;
• static volatile size_t bt_size_cur = 0;
• static volatile u_int64_t nsecprof = 0;
• static volatile int running = 0;
• volatile HANDLE hBtThread = 0;
• static pthread_t profiler_thread;
• static mach_port_t main_thread;
• clock_serv_t clk;
• static int profile_started = 0;
• static mach_port_t profile_port = 0;
• volatile static int forceDwarf = -2;
• volatile mach_port_t mach_profiler_thread = 0;
• static unw_context_t profiler_uc;
• mach_timespec_t timerprof;
• struct itimerval timerprof;
• static timer_t timerprof;
• static struct itimerspec itsprof;

sys.c

• JL_STREAM *JL_STDIN=0; constant
• JL_STREAM *JL_STDOUT=0; constant
• JL_STREAM *JL_STDERR=0; constant

task.c

• volatile int jl_in_stackwalk = 0; thread-local
• static size_t _frame_offset; constant
• DLLEXPORT jl_task_t * volatile jl_current_task; thread-local
• jl_task_t *jl_root_task; constant
• jl_value_t * volatile jl_task_arg_in_transit; thread-local
• jl_value_t *jl_exception_in_transit; thread-local
• __JL_THREAD jl_gcframe_t *jl_pgcstack = NULL; thread-local
• jl_jmp_buf * volatile jl_jmp_target; thread-local
• extern int jl_in_gc; barrier
• static jl_function_t *task__hook_func=NULL; constant
• ptrint_t bt_data[MAX_BT_SIZE+1]; thread-local
• size_t bt_size = 0; thread-local
• int needsSymRefreshModuleList; lock
• jl_function_t *jl_unprotect_stack_func; constant

toplevel.c

• int jl_lineno = 0; thread-local
• jl_module_t *jl_old_base_module = NULL; constant
• jl_module_t *jl_internal_main_module = NULL; constant
• extern int jl_in_inference; lock

[StefanKarpinski]

Don't we have a spreadsheet of this somewhere? Maybe make it public?

[ViralBShah]

https://docs.google.com/a/mayin.org/spreadsheets/d/1FLrB90u0ORvBDmJZtxUfXhSt8HVdKMM0rqYjoOeMvGo/edit#gid=0

Don't know if it is public - but I thought it would be easier to track the work here in the issue.

[StefanKarpinski]

I fancied up the spreadsheet a bit so that it's easier to organize and sort.

[vtjnash]

i've tried to help knock a few off the list based off existing issues and upcoming changes

[JeffBezanson]

That's true, #9986 helps with this.

[ViralBShah]

It would be nice to get basic printing to work:

julia> using Base.Threading
julia> @threads all for i=1:100; println(i) ; end

signal (11): Segmentation fault
Segmentation fault (core dumped)

[ArchRobison]

Are these changes being made in the master, or a branch? I see items checked off for llvm-simdloop.cpp, but don't see them in the master branch.

[StefanKarpinski]

I think this is for the threading branch, right, @JeffBezanson?

[tknopp]

would also be interesting which is the current threading branch, "threading" or "threads"

[StefanKarpinski]

"threading" is newer than "threads"; not sure why we changed branches.

[tknopp]

that why I asked :-) I have seen changes being made to both branches at a time and thus was not sure about it.

Would be also interesting to have an issue what the blockers are for merging into master. IIUC at some point llvm svn was required but this might be solved if we switch to llvm3.6? Further there is still the pthreads dependency.

[ArchRobison]

Moving to C11/C++11 threads would remove the pthreads dependency. Using C11/C++11 would also give us portable atomic ops with good memory consistency controls. LLVM is moving aggressively to depend on C++11 anyway.

[tknopp]

I am not 100% sure why the pthread dependency was added but it seems to be required to set thread affinity. Is this possible with C++11? For atomic ops we have written some macros that use the compiler dependent intrinsics.