[SKIP SOFCI-TEST] topology: build using a local ALSA install #9870

lgirdwood · 2025-03-03T16:43:10Z

Build topologies using a locally installed ALSA lib and ALSA utils without having to use Docker or upgrade the system ALSA installation.

lgirdwood · 2025-03-03T16:48:23Z

This wont pass CI yet, pending a alsa-utils fix to upstream 1st before merge.

kv2019i

Looks good, Windows CI jobs looking green, so seems topology builds are attempted at limited places. We do need to update the CI flows were topologies are built before merge.

scripts/build-alsa-tools.sh

tools/topology/CMakeLists.txt

singalsu

It seemed to work for me. I wonder if the alsa-utils build question "Press Return to acknowledge the previous three paragraphs." would cause issues in CI batch env?

lgirdwood · 2025-04-08T18:00:06Z

It seemed to work for me. I wonder if the alsa-utils build question "Press Return to acknowledge the previous three paragraphs." would cause issues in CI batch env?

Tested that - it passes straight thru as no console stdin is available.

jsarha

Works for me too, and did not see anything wrong with the commits either.

Used to validate docker behavior in PR thesofproject#9870 Signed-off-by: Christopher Turner <[email protected]>

Used to validate docker behavior in PR thesofproject#9870 docker-run.sh: update Docker image tag to '20250410_no-alsa' Signed-off-by: Christopher Turner <[email protected]>

kv2019i · 2025-04-24T14:20:48Z

scripts/build-alsa-tools.sh

+# Directory where repositories will be cloned/updated.
+if [[ -z "$SOF_WORKSPACE" ]]; then
+  # Environment variable is empty or unset so use default
+  BASE_DIR="$HOME/work/sof"


I think this should be

BASE_DIR="$HOME/work"

For the default case, the BASE_DIR should point to parent directory of sof as this is the directory assumed in tools/topology/CMakeLists.txt rules:

set(SOF_ALSA_TOOLS_DIR "${SOF_ROOT_SOURCE_DIRECTORY}/../tools/bin")

So BASE_DIR should match "${SOF_ROOT_SOURCE_DIRECTORY}".

This is causing failuress e.g. when we try to use "build-tools.sh -A" in docker in github (and might be a reason why @lgirdwood you "open-coded" the build recipe for github testbench workflow in this PR).

This is correct. We are cloning into the ~/work/sof workspace directory with contains the ~/work/sof/sof FW directory. e.g. my SOF workspace directory looks like

~/work/sof$ ls alsa-lib build-sof-staging LICENSE.workspace ruy sof-docs zephyr alsa-utils flatbuffers modules sof sof-test build-mtl gemmlowp nnlib-hifi4 sof-bin tflite-micro build-rimage keys README.md sof.code-workspace tools

The workspace should include all ingredients used to build FW, topologies, tooling i.e. everything a developer needs to create a new module and deploy it. I'll fix the other makefiles.

@lgirdwood Right so "$ZEPHYR_WORKSPACE=work/sof" is terms of
https://thesofproject.github.io/latest/getting_started/build-guide/build-with-zephyr.html

Ok I didn't realize that, I never use "sof" as the workspace directory name, but indeed then the problem is elsewhere setting the workspace path.

UPDATE2: ahaa, the build-from SOF source instructions do have "work/sof/sof". https://thesofproject.github.io/latest/getting_started/build-guide/build-from-scratch.html#step-1-set-up-the-workspace-directory . so then this is indeed aligned!
UPDATE: our docker images have used workspace path "/home/sof/work" with sof git checked out in "/home/sof/work/sof". This just means the workspace has to be manually be set with "--env SOF_WORKSPACE=/home/sof/work/".

cgturner1 · 2025-04-28T22:36:59Z

@lrgirdwo In addition to the changes for the BASE_DIR i was testing this in the internal CI build and all the one specific flag does not work after installing the alsa tools and that is for the sof-ctl portion

so for example '.scripts/build-tools.sh -A -c' the tools install great and I'm able to build topologies but the sof-ctl returns:

tools/ctl/ctl.c:13:10: fatal error: alsa/asoundlib.h: No such file or directory
   13 | #include <alsa/asoundlib.h>
      |          ^~~~~~~~~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.

Is there a separate cmake file for sof-ctl that needs to be updated like the one for the topologies to work with the local ALSA install?

lgirdwood · 2025-05-01T13:12:48Z

Is there a separate cmake file for sof-ctl that needs to be updated like the one for the topologies to work with the local ALSA install?

Thanks - now fixed, was actually using system ALSA path

This script builds a local installation of ALSA lib and associated ALSA utilities that does not impact the system ALSA installation. This will enable a later update to locally build topologies without using the docker container. Signed-off-by: Liam Girdwood <[email protected]>

Add a command line option to rebuild the ALSA libraries and tools required to build all topologies. Signed-off-by: Liam Girdwood <[email protected]>

Update testbench to build without docker and use a local ALSA lib and ALSA utils version from SOF ALSA git to align with latest topology developments. Signed-off-by: Liam Girdwood <[email protected]>

kv2019i · 2025-05-02T07:54:00Z

tools/ctl/CMakeLists.txt

@@ -13,6 +13,7 @@ target_compile_options(sof-ctl PRIVATE
 )

 target_include_directories(sof-ctl PRIVATE
+	"${SOF_ROOT_SOURCE_DIRECTORY}/../tools/include"


I tested this and works in my local environment, but with @cgturner1 's test docker image (thesofproject/sof:20250410_no-alsa) with no system ALSA installed, the build fails as linkage still uses system libasound:

: && /usr/bin/cc -g ctl/CMakeFiles/sof-ctl.dir/ctl.c.o -o ctl/sof-ctl -lasound && : /usr/bin/ld: cannot find -lasound: No such file or directory collect2: error: ld returned 1 exit status ninja: build stopped: subcommand failed.

@kv2019i now fixed

Dont use the system ALSA installation which may be out of date with current topology development. Use the locally installed ALSA lib and utils installed in the SOF workspace. Signed-off-by: Liam Girdwood <[email protected]>

The installer workflow uses the legacy SDK build tools and is no longer valid for Zephyr based SDK migration. This also reduces effort in maintaining a SOF Docker container. Signed-off-by: Liam Girdwood <[email protected]>

lgirdwood · 2025-05-02T10:25:04Z

@cgturner1 thinking aloud - is there a hard need for a docker container now after this update is working/merged ? This has been quite painful to make changes here - it would be great if CI could build topologies and tools without Docker in the future.

cgturner1 · 2025-05-05T16:22:43Z

@cgturner1 thinking aloud - is there a hard need for a docker container now after this update is working/merged ? This has been quite painful to make changes here - it would be great if CI could build topologies and tools without Docker in the future.

After this is merged then there wont really be a need to continually update the docker but a lot of the current CI scripts are written around docker and the docker workspaces that removing that is a much bigger change than adding the command to build things using the -A script and build it locally inside the docker (which currently is working after the latest updates)

So I think for step 1 is to just get these local builds working in a docker that doesn't have Alsa-tools since it will require the least amount of changes to the jenkins and then longer term can remove the dependency all together but that touches way more CI scripts and dependencies.

One last open/question @lrgirdwo was doing final tests of this and was wondering if it can be possible to set the DO_BUILD_alsa=true when running ./scripts/build-tools.sh with no flags set? I've run into instances where we have it set to run ./scripts/build-tools.sh without any flags after but i don't know if the intended behavior is for alsa-build to be optional or included in the build-all scenario

marc-hb · 2025-05-05T23:24:17Z

I think this topic was already discussed there:

Add document to clarify BKM of user space packages: alsa-lib, alsa-ucm-conf, PulseAudio, GNOME etc linux#2211

... and in internal issue sofcitest/issues/692

I don't have time to look at this PR but the sheer length of it gives me the impression that a Docker container was a good idea. Sorry...

EDIT: and also

add User space test cases sof-test#363

+ many others.

lgirdwood · 2025-05-07T13:42:09Z

One last open/question @lrgirdwo was doing final tests of this and was wondering if it can be possible to set the DO_BUILD_alsa=true when running ./scripts/build-tools.sh with no flags set? I've run into instances where we have it set to run ./scripts/build-tools.sh without any flags after but i don't know if the intended behavior is for alsa-build to be optional or included in the build-all scenario

@cgturner1 Ok, so by default do you want the flags inverted i.e. always build ALSA unless a flag says dont build ALSA ? I can do that if preferred by you.

cgturner1 · 2025-05-07T17:49:45Z

One last open/question @lrgirdwo was doing final tests of this and was wondering if it can be possible to set the DO_BUILD_alsa=true when running ./scripts/build-tools.sh with no flags set? I've run into instances where we have it set to run ./scripts/build-tools.sh without any flags after but i don't know if the intended behavior is for alsa-build to be optional or included in the build-all scenario

@cgturner1 Ok, so by default do you want the flags inverted i.e. always build ALSA unless a flag says dont build ALSA ? I can do that if preferred by you.

ah didn't think about it that way i think the current optional way makes more sense to me where the flag includes it (to match the behavior of the other flags)

for now i have it that when we want to build all tools + install rather than just running

.scripts/buils-tools.sh

i have:

.scripts/build-tools.sh -A -c -l -p -t -T

I think for now we can leave as is and we can always revisit... I am trying to get time carved out tomorrow to have Kai merge as is and try to see if the internal CI changes work out worst case we back out on Friday and I figure out what changes were missing on the Jenkins side while not impacting production CI runs any more than a day

kv2019i · 2025-05-08T14:50:13Z

@cgturner1 Merged now. We need to do this in steps and now proceed to update the jenkins task.

lgirdwood requested review from ranj063, plbossart, mmaka1, lbetlej, dbaluta and kv2019i as code owners March 3, 2025 16:43

lgirdwood added the DNM Do Not Merge tag label Mar 3, 2025

kv2019i reviewed Mar 3, 2025

View reviewed changes

scripts/build-alsa-tools.sh Outdated Show resolved Hide resolved

scripts/build-alsa-tools.sh Outdated Show resolved Hide resolved

tools/topology/CMakeLists.txt Show resolved Hide resolved

lgirdwood force-pushed the lrg/topic/local-alsa branch 2 times, most recently from e5c19f6 to 1cf54d0 Compare March 5, 2025 13:01

lgirdwood mentioned this pull request Mar 5, 2025

testbench: dont build testbench, tools and topologies in the source directory #9871

Open

lgirdwood force-pushed the lrg/topic/local-alsa branch 8 times, most recently from 838b80a to b3670cb Compare March 8, 2025 10:03

lgirdwood changed the title ~~topology: build using a local ALSA install~~ [SKIP SOFCI-TEST] topology: build using a local ALSA install Mar 8, 2025

lgirdwood force-pushed the lrg/topic/local-alsa branch 9 times, most recently from ac138c8 to 8f6c110 Compare March 8, 2025 14:51

singalsu approved these changes Apr 8, 2025

View reviewed changes

jsarha approved these changes Apr 8, 2025

View reviewed changes

lgirdwood force-pushed the lrg/topic/local-alsa branch from 136240c to 8b165a3 Compare April 11, 2025 16:28

cgturner1 added a commit to cgturner1/sof that referenced this pull request Apr 11, 2025

[DNM] Verify no alsa docker install in testbench workflow

c2930bd

Used to validate docker behavior in PR thesofproject#9870 Signed-off-by: Christopher Turner <[email protected]>

cgturner1 mentioned this pull request Apr 11, 2025

[SKIP SOFCI-TEST] validate no alsa docker for PR#9870 [DNM] #9968

Draft

kv2019i reviewed Apr 24, 2025

View reviewed changes

lgirdwood and others added 3 commits May 1, 2025 14:47

scripts: build-tools: Add option to build alsa tools

cfc661d

Add a command line option to rebuild the ALSA libraries and tools required to build all topologies. Signed-off-by: Liam Girdwood <[email protected]>

github: workflow: update testbench workflow to use local ALSA tools

c0f6e16

Update testbench to build without docker and use a local ALSA lib and ALSA utils version from SOF ALSA git to align with latest topology developments. Signed-off-by: Liam Girdwood <[email protected]>

lgirdwood force-pushed the lrg/topic/local-alsa branch from 8b165a3 to ddf1ce1 Compare May 1, 2025 13:56

kv2019i reviewed May 2, 2025

View reviewed changes

lgirdwood and others added 2 commits May 2, 2025 10:36

topology: cmake: use local ALSA utilities to build topology

6c2f49a

Dont use the system ALSA installation which may be out of date with current topology development. Use the locally installed ALSA lib and utils installed in the SOF workspace. Signed-off-by: Liam Girdwood <[email protected]>

github: workflows: remove the legacy installer workflow

60ba53a

The installer workflow uses the legacy SDK build tools and is no longer valid for Zephyr based SDK migration. This also reduces effort in maintaining a SOF Docker container. Signed-off-by: Liam Girdwood <[email protected]>

lgirdwood force-pushed the lrg/topic/local-alsa branch from ddf1ce1 to 60ba53a Compare May 2, 2025 09:36

lgirdwood removed the DNM Do Not Merge tag label May 5, 2025

marc-hb mentioned this pull request May 5, 2025

Add document to clarify BKM of user space packages: alsa-lib, alsa-ucm-conf, PulseAudio, GNOME etc thesofproject/linux#2211

Closed

kv2019i merged commit 137c9be into thesofproject:main May 8, 2025
39 of 44 checks passed

This was referenced May 9, 2025

[FEATURE] add alsa-utils and alsa-lib to SOF manifest #9791

Closed

[BUG] IMX: Topology build fails after recent change #9997

Closed

[SKIP SOFCI-TEST] topology: build using a local ALSA install #9870

[SKIP SOFCI-TEST] topology: build using a local ALSA install #9870

Uh oh!

Conversation

lgirdwood commented Mar 3, 2025

Uh oh!

lgirdwood commented Mar 3, 2025

Uh oh!

kv2019i left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

singalsu left a comment

Choose a reason for hiding this comment

Uh oh!

lgirdwood commented Apr 8, 2025

Uh oh!

jsarha left a comment

Choose a reason for hiding this comment

Uh oh!

kv2019i Apr 24, 2025

Choose a reason for hiding this comment

Uh oh!

lgirdwood May 1, 2025

Choose a reason for hiding this comment

Uh oh!

kv2019i May 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cgturner1 commented Apr 28, 2025

Uh oh!

lgirdwood commented May 1, 2025

Uh oh!

kv2019i May 2, 2025

Choose a reason for hiding this comment

Uh oh!

lgirdwood May 2, 2025

Choose a reason for hiding this comment

Uh oh!

lgirdwood commented May 2, 2025

Uh oh!

cgturner1 commented May 5, 2025

Uh oh!

marc-hb commented May 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lgirdwood commented May 7, 2025

Uh oh!

cgturner1 commented May 7, 2025

Uh oh!

Uh oh!

kv2019i commented May 8, 2025

Uh oh!

Uh oh!

kv2019i May 2, 2025 •

edited

Loading

marc-hb commented May 5, 2025 •

edited

Loading