Optimize CodeQL analysis step in CI builds

[richscott]

Optimize the CodeQL processing in the CI checks for Armada - currently, the CodeQL analysis for Go code can take up to 5 minutes.

[Bharadwajshivam28]

I would like to work on this issue can you guide me on how to get started with this to solve the issue

[pavlovic-ivan]

Hi @Bharadwajshivam28 . Yes, you can take over. You need to take a look at the CodeQL workflow, in the file codeql-analysis.yml. Pay attention to the matrix used analyze job. You will see that codeql is run for csharp, go, and javascript. Since that the matrix is used to run 3 parallel jobs, you need to pay attention to the Autobuild step that needs to run conditionally for csharp and javascript, and have another step running conditionally for go. This step for go needs to execute the go build commands seen in other workflows

[Bharadwajshivam28]

Hi @Bharadwajshivam28 . Yes, you can take over. You need to take a look at the CodeQL workflow, in the file codeql-analysis.yml. Pay attention to the matrix used analyze job. You will see that codeql is run for csharp, go, and javascript. Since that the matrix is used to run 3 parallel jobs, you need to pay attention to the Autobuild step that needs to run conditionally for csharp and javascript, and have another step running conditionally for go. This step for go needs to execute the go build commands seen in other workflows

I went through the file and understood the workflows. our end goal is to optimize the analysis for Go code which currently takes upto 5 minutes. Can you share some more details how can i tackle it? @pavlovic-ivan

[pavlovic-ivan]

The idea is to replace the autobuild step, with another that will execute the build natively as it is done in other workflows, and leave other steps in the analyse job as is. Please take a look at how conditional expressions are added to steps, so that autobuild step is executed for csharp and javascript, and how to add another step called (eg) Go build, that is run only when go comes from the matrix. That would be the first step. We can discuss further improvements after this. How does that sound?

[Bharadwajshivam28]

The idea is to replace the autobuild step, with another that will execute the build natively as it is done in other workflows, and leave other steps in the analyse job as is. Please take a look at how conditional expressions are added to steps, so that autobuild step is executed for csharp and javascript, and how to add another step called (eg) Go build, that is run only when go comes from the matrix. That would be the first step. We can discuss further improvements after this. How does that sound?

I have implemented condition expressions before but I am confused about what type of conditions we need to apply in the workflows....

And I understood that we need a step which will run only when go comes out of the Matrix.

Ya we can improve it... Let me work and give the updates..

[Bharadwajshivam28]

Hey @pavlovic-ivan we need to add some condition that when matrix language is csharp or javascript and for go we will have a seperate step which ensures that it will run only when the matrix language is go.
also we can run the Mage Proto step only when the matrix language is golang..

Is it what we need right? This will overall improve the performance i guess... and we can make more changes if needed.

correct me please if i missed something

[pavlovic-ivan]

Hi @Bharadwajshivam28 let me take over, as i was already assigned to the issue, and afterwards we can sync on the work i would do, and that will give you some insights for the future. How does that sound?

[Bharadwajshivam28]

Hi @Bharadwajshivam28 let me take over, as i was already assigned to the issue, and afterwards we can sync on the work i would do, and that will give you some insights for the future. How does that sound?

okay works for me ....