Need help to create a model pack for go

[hugo-syn]

I have this go function:

func (s *Server) ListTriggers(ctx context.Context, q *notification.TriggersListRequest) (*notification.TriggerList, error) {
	[...]
}

You can find it here: https://github.com/argoproj/argo-cd/blob/v2.13.3/server/notification/notification.go#L25

I want to add the q parameter as an additional source using the codeql threat model feature. I have the following ext.yml file:

extensions:
  - addsTo:
      pack: codeql/go-all
      extensible: sourceModel
    data:
      - ["github.com/argoproj/argo-cd/v2/server/notification", "", True, "ListTriggers", "", "", "Argument[1]", "remote", "manual" ]
  - addsTo:
      pack: codeql/go-all
      extensible: sinkModel
    data: []
  - addsTo:
      pack: codeql/go-all
      extensible: summaryModel
    data: []

I'm running the following query to check if I have additional entries in the output with the model:

 import semmle.go.security.FlowSources

 from RemoteFlowSource::Range source
 where not source.getFile().getRelativePath().matches("%/test/%")
 select source, ""

But I get the same number of result with and without my model pack. So is my model pack correct ? How can I test if it's really adding sources ?

Here is my command line:

$ codeql database analyze /../argo-cd-v2.13.3 --format=sarif-latest --output=/../sarif-remote-flow-sources.sarif --additional-packs /.../argocd-cd-ext/ --additional-packs /../argocd-cd/ --model-packs synacktiv/argocd-cd-ext synacktiv/argocd-cd:query.ql --rerun

[owen-mc]

Thank you for your detailed question. Looking at your Go model, it is missing the receiver type in the Type column. Try this instead.

      - ["github.com/argoproj/argo-cd/v2/server/notification", "Server", True, "ListTriggers", "", "", "Argument[1]", "remote", "manual" ]

If that does not work then it may be a model pack issue.

[owen-mc]

I don't think we use Parameter[...] in Go models. I think that might be for lambda modeling, which we don't use in Go.

I'm not sure what you mean about there being no code. If there are no calls to a function then we may not be able to model it using a yml file, but we might be able to do it in CodeQL. Can you give me an example of something you'd want CodeQL to find as a source of taint?

[hugo-syn]

Ok so now I have an additional result:

Yes in my case I would like to restrict it to the case when the receiver is a Server. But why is it not working then ? If I restrict the type of the receiver with the right one ?

      - ["github.com/argoproj/argo-cd/v2/server/notification", "Server", True, "ListTriggers", "", "", "Argument[1]", "remote", "manual" ]

[hugo-syn]

I want to taint my parameter q to know if it flows to a sink method something like this:

func (s *Server) ListTriggers(ctx context.Context, q *notification.TriggersListRequest) (*notification.TriggerList, error) {
	foo.callMethodA(q)
}

func ... callMethodA(q *notification.TriggersListRequest){
	bar.callMethodB(q)
}

func ... callMethodB(q *notification.TriggersListRequest){
	ba.sinkMethod(q)
}

In my case I know that I can control q and that ListTriggers will eventually be called so I want to extend the source model so that other queries can track this as a RemoteFlowSource.

I want to do this with the model feature to avoid adding Customization.qll and recompile all the go-queries.

[owen-mc]

Thanks for explaining your use case. This does indeed seem to be what "Parameter", which @hvitved mentioned earlier, does. I haven't used it before, and it isn't currently in use in the Go library, but it is possible that it currrently works, or that minimal changes are needed to make it work. I will look into it. In the meantime, have you tried the following model?

      - ["github.com/argoproj/argo-cd/v2/server/notification", "Server", True, "ListTriggers", "", "", "Parameter[1]", "remote", "manual" ]

[hugo-syn]

Thanks for the information, yes following the comment of @hvitved I tried this, but it does not return additional sources. When you say minimal changes are needed, do you have an idea of how to do this ? I know nothing about the codeql internals.

In the java/ql folder (of the codeql source code) I can see a lot of test case related to this.

[owen-mc]

Thanks for confirming that it doesn't work for you. I found the same with some internal tests.

To be clear, I do not expect you to get "Parameter" working. It is something that we should fix. If it is easy to fix then I will try to do it soon. Otherwise it will have to be prioritised alongside other work. I believe it shouldn't be too much work as most of the logic is in a shared library, and that clearly works for other languages. Once we have found a fix and a PR is merged, it will work with the next release of CodeQL. Releases are made roughly every 2 weeks. Thank you for highlighting this shortcoming.

[hugo-syn]

Alright, thank you very much! Can I leave this unresolved and wait for the feature to be implemented, so you can update me once it's done? Otherwise, I have no problem closing it.

Anyway, thank you again for taking the time to help me and for providing all the detailed information. This isn’t the first time I’ve posted a message in the CodeQL discussions, and every time, people take the time to explain things clearly and thoroughly, which I truly appreciate.

[owen-mc]

I believe I've found the problem and this PR should fix it.

Yes, feel free to leave this discussion open.

[hugo-syn]

Hi @owen-mc I have an additional issue with the model pack. I've applied your patch and I'm trying to use my model pack with the CommandInjection.ql query. But I have no result if I run the query from the downloaded pack (I've removed the qlx file and the compiled cache):

$ codeql query run -d /.../argo-cd/files/codeql/argo-cd-v2.13.3 --additional-packs /.../argocd-cd-ext/ --model-packs="synacktiv/argocd-cd-ext@latest" /.../.codeql/packages/codeql/go-queries/1.1.5/Security/CWE-078/CommandInjection.ql
Compiling query plan for /.../.codeql/packages/codeql/go-queries/1.1.5/Security/CWE-078/CommandInjection.ql.
[1/1] Found in cache: /.../.codeql/packages/codeql/go-queries/1.1.5/Security/CWE-078/CommandInjection.ql.
Starting evaluation of codeql/go-queries/Security/CWE-078/CommandInjection.ql.
Evaluation completed (12.6s).
Result set: edges
| a | b | key | val |
+---+---+-----+-----+

Result set: nodes
| n | key | val |
+---+-----+-----+

Result set: subpaths
| arg | par | ret | out |
+-----+-----+-----+-----+

Result set: #select
| col0 | source | sink | col3 | col4 | col5 |
+------+--------+------+------+------+------+
Shutting down query evaluator.

But If I copy the same query in my custom pack I got some results:

codeql query run -d /.../codeql/argo-cd-v2.13.3 --additional-packs /.../codeql/queries/argocd-cd-ext/ --model-packs="synacktiv/argocd-cd-ext@latest" /.../codeql/queries/argocd-cd/CommandInjection.ql | head -n 200
Compiling query plan for /.../codeql/queries/argocd-cd/CommandInjection.ql.
[1/1] Found in cache: /.../argocd-cd/CommandInjection.ql.
Starting evaluation of synacktiv/argocd-cd/CommandInjection.ql.
Evaluation completed (15s).
Result set: edges
|                             a                              |                             b                              |    key     |         val          |
+------------------------------------------------------------+------------------------------------------------------------+------------+----------------------+
| in                                                         | definition of q                                            | provenance |                      |
| in                                                         | definition of q                                            | provenance |                      |
| in                                                         | definition of q                                            | provenance |                      |
| in                                                         | definition of request                                      | provenance |                      |
| in                                                         | definition of request                                      | provenance |                      |
| in                                                         | definition of request                                      | provenance |                      |
| revision                                                   | definition of revision                                     | provenance |                      |
| selection of Revision                                      | definition of revision                                     | provenance |                      |
[...]

What's the difference ?

I've also tried to use the syntax codeql database analyze ....

[hvitved]

Sorry, I'm also no expert on model packs, but can you try to take a look at the logs in both cases, to see if they both actually load the model pack correctly?

[hugo-syn]

Yes they are all loaded in both cases:

Loading data extensions in /.../argocd-cd-ext/ext/ext.yml.
  Found extension targeting codeql/go-all:sourceModel: 143 tuples.
  Found extension targeting codeql/go-all:sinkModel: 3 tuples.
  Found extension targeting codeql/go-all:summaryModel: 0 tuples.

[hvitved]

I'm suspecting that the patch you applied from @owen-mc is perhaps not picked up when you run the query /.../.codeql/packages/codeql/go-queries/1.1.5/Security/CWE-078/CommandInjection.ql. One way to verify is to run

codeql query compile /.../.codeql/packages/codeql/go-queries/1.1.5/Security/CWE-078/CommandInjection.ql --dump-dil dil1.txt
codeql query compile /.../codeql/queries/argocd-cd/CommandInjection.ql --dump-dil dil2.txt
diff dil1.txt dil2.txt

[hugo-syn]

You're right, I downloaded the codeql/go-queries pack and forgot that it comes with its own codeql/go-all libraries. So, the modification wasn't applied. It's working much better now. Thanks so much for your help!

[owen-mc]

I defer to @hvitved as I am not very familiar with model packs.