Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port A bunch of checkers useful in some RAJA / CHAI/ CARE codes from clang-query to clang-tidy #7

Open
robinson96 opened this issue Jun 24, 2020 · 1 comment

Comments

@robinson96
Copy link

let comment "### Disable default name for errors"
set bind-root false

let comment "### Identify device lambda contexts"
let b1 callExpr(hasArgument(0,hasType(asString("struct care::gpu"))))
let b2 callExpr(hasArgument(0,hasType(asString("struct care::parallel"))))
let b3 cxxMemberCallExpr(on(hasType(pointsTo(cxxRecordDecl(hasName("LoopFuser"))))))
let careHostDevicePtrCallee callee(cxxMethodDecl(ofClass(classTemplateSpecializationDecl(matchesName("care::host_device_ptr")))))
let rajaDeviceContext callExpr(anyOf(b1,b2,b3))
let inDeviceLambda hasAncestor(lambdaExpr(hasAncestor(rajaDeviceContext)))
let comment "### Identify code outside lambda contexts"
let notInsideLambda unless(hasAncestor(lambdaExpr()))
let comment "### Ignore implicit casts (to prevent duplicate matches)"
let notInImplicitCast unless(hasAncestor(implicitCastExpr()))
let hasCareHostDevPtrType hasType(qualType(hasCanonicalType(hasDeclaration(namedDecl(matchesName("care::host_device_ptr"))))))

######## USEFUL FOR RAJA USERS ###################

let comment "### Set up detection of assignment of pointer to 0 or NULL (instead of nullptr), except for with explicit casts"
match stmt(isExpansionInMainFile(), castExpr(hasCastKind("NullToPointer"), unless(hasAncestor(explicitCastExpr())), unless(hasSourceExpression(cxxNullPtrLiteralExpr())))).bind("assignment_of_pointer_to_0")

let comment "### Set up detection of implicit this within device RAJA context"
match cxxThisExpr(inDeviceLambda, notInImplicitCast).bind("implicit_use_of_this_inside_lambda_function")


#### USEFUL FOR CARE USERS (https://github.com/LLNL/CARE) ##################

let comment "### Set up detection of host_device_ptr operator[] outside RAJA context"
match cxxOperatorCallExpr(careHostDevicePtrCallee, notInsideLambda, hasOverloadedOperatorName("[]")).bind("use_of_host_device_ptr_operator[]_outside_lambda_function")

let comment "### Set up detection of host_device_ptr operator[] from pointer within inside lambda context (as a stand-in for RAJA contexts)"
let careHostDevPtrArrow stmt(allOf(memberExpr(isArrow()), expr(hasCareHostDevPtrType)))
match cxxOperatorCallExpr(careHostDevicePtrCallee, hasAncestor(lambdaExpr()), hasOverloadedOperatorName("[]"), hasDescendant(careHostDevPtrArrow)).bind("use_of_host_device_ptr_operator[]_from_pointer_inside_lambda_function")

let comment "### Set up detection of dereference of host_device_ptr, ignoring system headers (from vector<host_device_ptr>) and explicit casts"
match stmt(unaryOperator(unless(hasDescendant(explicitCastExpr())), unless(isExpansionInSystemHeader()), allOf(hasOperatorName("*"), hasUnaryOperand(hasDescendant(expr(hasCareHostDevPtrType)))))).bind("dereference_host_device_ptr")

let comment "### Set up detection of KeyValueSorter functions outside RAJA context"
let k1 cxxMethodDecl(hasName("key"))
let k2 cxxMethodDecl(hasName("setKey"))
let k3 cxxMethodDecl(hasName("value"))
let k4 cxxMethodDecl(hasName("setValue"))
let KeyValueSorterRAJACalls cxxMethodDecl(anyOf(k1,k2,k3,k4))
match cxxMemberCallExpr(on(hasType(cxxRecordDecl(hasName("KeyValueSorter")))), callee(KeyValueSorterRAJACalls), notInsideLambda).bind("use_of_KeyValueSort_RAJA_function_outside_lambda_function")

let comment "### Set up detection of assignment of raw pointer to host_device_ptr"
let comment "### This is searching for the constructor with bool as the third argument"
let comment "### Ignore when the raw pointer is explicitly cast"
match cxxConstructExpr(hasCareHostDevPtrType, hasArgument(2, hasType(asString("_Bool"))), unless(hasArgument(0, explicitCastExpr()))).bind("construction_of_host_device_ptr_from_raw_pointer")

############# USEFUL FOR CHAI USERS ########################

let comment "### Set up detection of reference of raw pointer in device lambda"
let comment "### ignoringImplicit and no implicitCastExpr ancestor is required to prevent duplicate matches"
match expr(inDeviceLambda, notInImplicitCast, ignoringImplicit(declRefExpr(to(varDecl(hasType(isAnyPointer()), unless(inDeviceLambda)))))).bind("capture_of_raw_pointer_in_lambda")

@calewis
Copy link

calewis commented Jun 25, 2020

@robinson96 was there some concern that the clang-query tests can be slow? I wonder because someone asked in our call if clang-tidy would be faster than clang-query. I noticed pattern(hasAncestor(lambdaExpr(hasAncestor(rajaDeviceContext)))) and wondered if this leads to a lot more work than something like (from memory, probably doesn't compile) rajaDeviceCall(hasAnyArgument(lambdaExpr(forEachDescendent(pattern())))) that way you only search for pattern in the context you care about instead of finding every pattern and searching backwards to see if we are in the correct context?

Also I noticed that by looking for lambdaExpr(hasAncestor(rajaDeviceContext))) you can only match temporary lambdas like:

RAJA::some_raja_func(..., [=]{});

Without testing I don't think that pattern can match the similar:

auto lambda = [=]{};
RAJA::some_raja_func(..., lambda);

crtrott pushed a commit that referenced this issue Jul 13, 2023
…tput

The crash happens in clang::driver::tools::SplitDebugName when Output is
InputInfo::Nothing. It doesn't happen with standalone clang driver because
output is created in Driver::BuildJobsForActionNoCache.

Example backtrace:
```
* thread #1, name = 'clangd', stop reason = hit program assert
  * frame #0: 0x00007ffff5c4eacf libc.so.6`raise + 271
    frame #1: 0x00007ffff5c21ea5 libc.so.6`abort + 295
    frame #2: 0x00007ffff5c21d79 libc.so.6`__assert_fail_base.cold.0 + 15
    frame #3: 0x00007ffff5c47426 libc.so.6`__assert_fail + 70
    frame #4: 0x000055555dc0923c clangd`clang::driver::InputInfo::getFilename(this=0x00007fffffff9398) const at InputInfo.h:84:5
    frame #5: 0x000055555dcd0d8d clangd`clang::driver::tools::SplitDebugName(JA=0x000055555f6c6a50, Args=0x000055555f6d0b80, Input=0x00007fffffff9678, Output=0x00007fffffff9398) at CommonArgs.cpp:1275:40
    frame #6: 0x000055555dc955a5 clangd`clang::driver::tools::Clang::ConstructJob(this=0x000055555f6c69d0, C=0x000055555f6c64a0, JA=0x000055555f6c6a50, Output=0x00007fffffff9398, Inputs=0x00007fffffff9668, Args=0x000055555f6d0b80, LinkingOutput=0x0000000000000000) const at Clang.cpp:5690:33
    frame #7: 0x000055555dbf6b54 clangd`clang::driver::Driver::BuildJobsForActionNoCache(this=0x00007fffffffb5e0, C=0x000055555f6c64a0, A=0x000055555f6c6a50, TC=0x000055555f6c4be0, BoundArch=(Data = 0x0000000000000000, Length = 0), AtTopLevel=true, MultipleArchs=false, LinkingOutput=0x0000000000000000, CachedResults=size=1, TargetDeviceOffloadKind=OFK_None) const at Driver.cpp:5618:10
    frame #8: 0x000055555dbf4ef0 clangd`clang::driver::Driver::BuildJobsForAction(this=0x00007fffffffb5e0, C=0x000055555f6c64a0, A=0x000055555f6c6a50, TC=0x000055555f6c4be0, BoundArch=(Data = 0x0000000000000000, Length = 0), AtTopLevel=true, MultipleArchs=false, LinkingOutput=0x0000000000000000, CachedResults=size=1, TargetDeviceOffloadKind=OFK_None) const at Driver.cpp:5306:26
    frame #9: 0x000055555dbeb590 clangd`clang::driver::Driver::BuildJobs(this=0x00007fffffffb5e0, C=0x000055555f6c64a0) const at Driver.cpp:4844:5
    frame #10: 0x000055555dbe6b0f clangd`clang::driver::Driver::BuildCompilation(this=0x00007fffffffb5e0, ArgList=ArrayRef<const char *> @ 0x00007fffffffb268) at Driver.cpp:1496:3
    frame #11: 0x000055555b0cc0d9 clangd`clang::createInvocation(ArgList=ArrayRef<const char *> @ 0x00007fffffffbb38, Opts=CreateInvocationOptions @ 0x00007fffffffbb90) at CreateInvocationFromCommandLine.cpp:53:52
    frame #12: 0x000055555b378e7b clangd`clang::clangd::buildCompilerInvocation(Inputs=0x00007fffffffca58, D=0x00007fffffffc158, CC1Args=size=0) at Compiler.cpp:116:44
    frame #13: 0x000055555895a6c8 clangd`clang::clangd::(anonymous namespace)::Checker::buildInvocation(this=0x00007fffffffc760, TFS=0x00007fffffffe570, Contents= Has Value=false ) at Check.cpp:212:9
    frame #14: 0x0000555558959cec clangd`clang::clangd::check(File=(Data = "build/test.cpp", Length = 64), TFS=0x00007fffffffe570, Opts=0x00007fffffffe600) at Check.cpp:486:34
    frame #15: 0x000055555892164a clangd`main(argc=4, argv=0x00007fffffffecd8) at ClangdMain.cpp:993:12
    frame #16: 0x00007ffff5c3ad85 libc.so.6`__libc_start_main + 229
    frame #17: 0x00005555585bbe9e clangd`_start + 46
```

Test Plan: ninja ClangDriverTests && tools/clang/unittests/Driver/ClangDriverTests

Differential Revision: https://reviews.llvm.org/D154602
crtrott pushed a commit that referenced this issue Aug 16, 2023
TSan reports the following race:

  Write of size 8 at 0x000107707ee8 by main thread:
    #0 lldb_private::ThreadedCommunication::StartReadThread(...) ThreadedCommunication.cpp:175
    #1 lldb_private::Process::SetSTDIOFileDescriptor(...) Process.cpp:4533
    #2 lldb_private::Platform::DebugProcess(...) Platform.cpp:1121
    #3 lldb_private::PlatformDarwin::DebugProcess(...) PlatformDarwin.cpp:711
    #4 lldb_private::Target::Launch(...) Target.cpp:3235
    #5 CommandObjectProcessLaunch::DoExecute(...) CommandObjectProcess.cpp:256
    #6 lldb_private::CommandObjectParsed::Execute(...) CommandObject.cpp:751
    #7 lldb_private::CommandInterpreter::HandleCommand(...) CommandInterpreter.cpp:2054

  Previous read of size 8 at 0x000107707ee8 by thread T5:
    #0 lldb_private::HostThread::IsJoinable(...) const HostThread.cpp:30
    #1 lldb_private::ThreadedCommunication::StopReadThread(...) ThreadedCommunication.cpp:192
    #2 lldb_private::Process::ShouldBroadcastEvent(...) Process.cpp:3420
    #3 lldb_private::Process::HandlePrivateEvent(...) Process.cpp:3728
    #4 lldb_private::Process::RunPrivateStateThread(...) Process.cpp:3914
    #5 std::__1::__function::__func<lldb_private::Process::StartPrivateStateThread(...) function.h:356
    #6 lldb_private::HostNativeThreadBase::ThreadCreateTrampoline(...) HostNativeThreadBase.cpp:62
    #7 lldb_private::HostThreadMacOSX::ThreadCreateTrampoline(...) HostThreadMacOSX.mm:18

The problem is the lack of synchronization between starting and stopping
the read thread. This patch fixes that by protecting those operations
with a mutex.

Differential revision: https://reviews.llvm.org/D157361
crtrott pushed a commit that referenced this issue Aug 31, 2023
This reverts commit 0e63f1a.

clang-format started to crash with contents like:
a.h:
```
```
$ clang-format a.h
```
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.      Program arguments: ../llvm/build/bin/clang-format a.h
 #0 0x0000560b689fe177 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /usr/local/google/home/kadircet/repos/llvm/llvm/lib/Support/Unix/Signals.inc:723:13
 #1 0x0000560b689fbfbe llvm::sys::RunSignalHandlers() /usr/local/google/home/kadircet/repos/llvm/llvm/lib/Support/Signals.cpp:106:18
 #2 0x0000560b689feaca SignalHandler(int) /usr/local/google/home/kadircet/repos/llvm/llvm/lib/Support/Unix/Signals.inc:413:1
 #3 0x00007f030405a540 (/lib/x86_64-linux-gnu/libc.so.6+0x3c540)
 #4 0x0000560b68a9a980 is /usr/local/google/home/kadircet/repos/llvm/clang/include/clang/Lex/Token.h:98:44
 #5 0x0000560b68a9a980 is /usr/local/google/home/kadircet/repos/llvm/clang/lib/Format/FormatToken.h:562:51
 #6 0x0000560b68a9a980 startsSequenceInternal<clang::tok::TokenKind, clang::tok::TokenKind> /usr/local/google/home/kadircet/repos/llvm/clang/lib/Format/FormatToken.h:831:9
 #7 0x0000560b68a9a980 startsSequence<clang::tok::TokenKind, clang::tok::TokenKind> /usr/local/google/home/kadircet/repos/llvm/clang/lib/Format/FormatToken.h:600:12
 #8 0x0000560b68a9a980 getFunctionName /usr/local/google/home/kadircet/repos/llvm/clang/lib/Format/TokenAnnotator.cpp:3131:17
 #9 0x0000560b68a9a980 clang::format::TokenAnnotator::annotate(clang::format::AnnotatedLine&) /usr/local/google/home/kadircet/repos/llvm/clang/lib/Format/TokenAnnotator.cpp:3191:17
Segmentation fault
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants