CODEQL - Scan passed even if alert introduced in PR, new alert can be found in Code Scanning tab

[Treders]

Hello,

I'm struggling with issue with CodeQL.

Setup:
Fork of the repository.
CodeQL Advanced enabled with codeql.yml

Steps:

1. Add custom query.
2. Prepare PR with new file with vulnerability.
3. Push the PR
4. Scan started, ongoing and then passed (even if there is new vulnerability introduced)
5. Check details of scan, it states "No new alerts found"
6. Go to "Code scanning" tab
7. New vulnerability is visible on that tab but for PR is not visible. By that it's not blocking PR

Has some of you encounter this problem?
Setup seems to work, it's finding problems but not showing them on PR, but in "Code scanning" tab.

[sampart]

Hi Adam, thanks for getting in touch. Would you be happy sharing the URL of your fork to help us dig into what's going on here? Thanks

(PS I'm away all next week so don't be surprised if you don't hear from me, or if you hear from one of my colleagues instead.)

[Treders]

Hi, unfortunatelly it's not my private repository so can't do it
My workflow looks like default one with
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
with:
category: "/language:go"

at the end, sarif file is uploaded but there is no info if some problem found and job in Github Action is passing

[sampart]

Hi again Adam,

Talking to my GitHub colleagues, one point they reminded me of is that alerts in Pull Requests only show up on the PR if they're present in the diff for the PR. Otherwise they only show up on the Code Scanning page as you mentioned.

Could this explain your situation?

If this is not the case and you are not able to share the repo name here so we can investigate further, then you've got a couple of options: You could either try to reproduce in a repo you are willing to share or have the repo owner open a support ticket which will keep information confidential between them and us. Thanks

[Treders]

Hi, thanks for suggestion.
I found out that when I introduced vulnerability from "default" pack it is shown correctly in PR, but when it should find vulnerability from custom_query, it's not finding it.

Here is my part of code for it:
yml file:

queries: +./.github/codeql/codeql-custom-queries-go # Include custom queries

custom query:

/**
 * @name Missing Copyright in .go file
 * @description Finds and lists all Go files (.go) that do not contain the string "Copyrights" in their comments.
 * @kind problem
 * @problem.severity error
 * @security-severity 5.0
 * @id go/find-missing-copyrights
 */

 import go

 /**
  * This query identifies Go files (.go) that do not contain the string "Copyrights" in comments.
  */
 
 // Check if a file contains the "Copyrights" string in comments
 predicate containsCopyrights(File f) {
   exists(Comment c |
     c.getFile() = f and
     c.getText().matches("%Copyright%")
   )
 }
 
 // Select Go files without the specified string in comments
 from File f
 where f.getBaseName().regexpMatch(".+\\.go$")
      and not containsCopyrights(f)
 select f, "Missing string: Copyrights in comments"

[sampart]

Hmm, interesting, thanks! Is your new query only in the PR's branch and not in the repo's default branch? That seems quite like to be relevant here.

[Treders]

It's one the master branch, I created new PR, added new file, introduced there SQL injection problem and missing copyrights. From those 2 only 1 with sql injection problem is found.