CRASH and RANK ORDER VIOLATION with -exit_after_tracing #3346
Description
When I was gathering the new test trace
clients/drcachesim/tests/drmemtrace.threadsig.x64.tracedir/drmemtrace.threadsig.10506.7343.trace.gz
I hit problems when I used -exit_after_tracing but only with certain values.
This only happens with a large -trace_after_instrs: so if we start tracing
near the end.
$ rm -rf drmemtrace.*.dir; gdb --args bin64/drrun -msgbox_mask 12 -t drcachesim -offline -trace_after_instrs 2180K -exit_after_tracing 120K -- ~/dr/test/threadsig 8 2000
add-symbol-file '/home/bruening/dr/git/build_x64_dbg_tests/bin64/../clients/lib64/debug/libdrmemtrace.so' 0x00007fa4afa97c10
add-symbol-file '/home/bruening/dr/git/build_x64_dbg_tests/ext/lib64/debug/libdrsyms.so' 0x00007fa4afcd0260
add-symbol-file '/home/bruening/dr/git/build_x64_dbg_tests/ext/lib64/debug/libdrwrap.so' 0x00007fa4aff18020
add-symbol-file '/home/bruening/dr/git/build_x64_dbg_tests/ext/lib64/debug/libdrmgr.so' 0x00007fa4b0126570
add-symbol-file '/home/bruening/dr/git/build_x64_dbg_tests/ext/lib64/debug/libdrutil.so' 0x00007fa4b03316a0
add-symbol-file '/home/bruening/dr/git/build_x64_dbg_tests/ext/lib64/debug/libdrcovlib.so' 0x00007fa4b0537dc0
add-symbol-file '/home/bruening/dr/git/build_x64_dbg_tests/ext/lib64/debug/libdrx.so' 0x00007fa4b07431f0
add-symbol-file '/home/bruening/dr/git/build_x64_dbg_tests/ext/lib64/debug/libdrreg.so' 0x00007fa4b094cb10
add-symbol-file '/lib/x86_64-linux-gnu/libgcc_s.so.1' 0x00007fa5329f4ac0
add-symbol-file '/lib64/ld-linux-x86-64.so.2' 0x00007fa532c0baa0
add-symbol-file '/lib/x86_64-linux-gnu/libc.so.6' 0x00007fa532e51910
add-symbol-file '/lib/x86_64-linux-gnu/libm.so.6' 0x00007fa5331d7680
add-symbol-file '/usr/lib/x86_64-linux-gnu/libstdc++.so.6' 0x00007fa533563090
add-symbol-file '/home/bruening/dr/git/build_x64_dbg_tests/lib64/debug/libdynamorio.so' 0x00007fa533ab3688
SIGSEGV
(gdb) bt
#0 __cxa_finalize (d=0x7fa4afcc0020) at cxa_finalize.c:56
#1 0x00007fa4afa97cc3 in __do_global_dtors_aux ()
#2 0x00007fa51fab29f0 in ?? ()
#3 0x00007fa533d49a48 in privload_call_lib_func (func=0x7fa51f96fe68) at /home/bruening/dr/git/src/core/unix/loader.c:883
<rank order violation shared_vm_areas(readwrite)@/home/bruening/dr/git/src/core/vmareas.c:1568 acquired after privload_lock(recursive)@/home/bruening/dr/git/src/core/loader_shared.c:61 in tid:390cf>
(gdb) bt
#0 0x00007fa533b5011f in deadlock_avoidance_lock (lock=0x7fa51f96c8a0, acquired=true, ownable=false) at /home/bruening/dr/git/src/core/utils.c:618
#1 0x00007fa533b51242 in read_lock (rw=0x7fa51f96c8a0) at /home/bruening/dr/git/src/core/utils.c:1225
#2 0x00007fa533c0bf1c in check_in_last_thread_vm_area (dcontext=0x7fa51f97ccc0, pc=0x0) at /home/bruening/dr/git/src/core/vmareas.c:8243
#3 0x00007fa533d3b90b in master_signal_handler_C (sig=11, siginfo=0x7fa51fb96af0, ucxt=0x7fa51fb969c0, xsp=0x7fa51fb969b8 "q\264\317\063\245\177")
at /home/bruening/dr/git/src/core/unix/signal.c:5056
#4 0x00007fa533cfb471 in xfer_to_new_libdr () at /home/bruening/dr/git/src/core/arch/x86/x86.asm:1211
More info on the crash (from different run so addrs different):
(gdb) x/1i $pc
=> 0x7fcd95099c7d <__cxa_finalize+141>: callq *%rdx
(gdb) p /x $rdx
$1 = 0xa2ecc8dd87fdd68d
For the rank order: that lock acquisition looks bad in general. No, we
can't reorder to avoid this violation. We probably want to have
check_in_last_thread_vm_area() do a first read w/o the lock and only grab
it to confirm.