Add a second eosdev tree

[ssssam]

This adds a new eosdev filesystem, which has the same dependencies as eos, but includes everything in the "doc" and "debug" split-rules domains as well. So it is larger, and is intended mainly as a developer toolkit or SDK.

All trees are built and exported as a single repo.bst element. This is the same as before except now the repo.bst has multiple branches, and I moved it out of the eos/ directory.

It makes sense to combine all trees in one repo because:

• In CI we want to build and export all of them
• Each tree has many objects in common, so it should be more efficient in a single OStree repo where the objects are deduplicated

[ssssam]

Here are timings from job 51796508698:

[--:--:--][][   build:eos/filesystem.bst            ] START   endlessos/eos-filesystem/-build.20251002-132845.log
[--:--:--][][   build:eos/filesystem.bst            ] START   Staging dependencies
[00:00:06][][   build:eos/filesystem.bst            ] SUCCESS Staging dependencies
[--:--:--][][   build:eos/filesystem.bst            ] START   Computing split
[00:00:09][][   build:eos/filesystem.bst            ] SUCCESS Computing split
[--:--:--][][   build:eos/filesystem.bst            ] START   Integrating sandbox
[--:--:--][][   build:eos/filesystem.bst            ] START   Running commands
[00:00:54][][   build:eos/filesystem.bst            ] SUCCESS Running commands
[--:--:--][][   build:eos/filesystem.bst            ] INFO    Integration added 1435 and removed 0 files
[00:01:22][][   build:eos/filesystem.bst            ] SUCCESS Integrating sandbox
[--:--:--][][   build:eos/filesystem.bst            ] START   Creating composition
    Including files from all domains
    Excluding files from domains: debug, devel, doc
    Including orphaned files
[--:--:--][][   build:eos/filesystem.bst            ] INFO    Composing 126221 files
[00:00:01][][   build:eos/filesystem.bst            ] SUCCESS Creating composition
[--:--:--][][   build:eos/filesystem.bst            ] START   Caching artifact
[00:00:10][][   build:eos/filesystem.bst            ] SUCCESS Caching artifact
[00:01:51][][   build:eos/filesystem.bst            ] SUCCESS endlessos/eos-filesystem/-build.20251002-132845.log
[--:--:--][][    push:eos/filesystem.bst            ] START   endlessos/eos-filesystem/-push.20251002-133036.log
[--:--:--][][    push:eos/filesystem.bst            ] INFO    Pushed data from artifact  -> https://bstcache.endlessos.org
[--:--:--][][    push:eos/filesystem.bst            ] INFO    Pushed artifact  -> https://bstcache.endlessos.org
[00:00:11][][    push:eos/filesystem.bst            ] SUCCESS endlessos/eos-filesystem/-push.20251002-133036.log

[--:--:--][][   build:eosdev/filesystem.bst         ] START   endlessos/eosdev-filesystem/-build.20251002-132845.log
[--:--:--][][   build:eosdev/filesystem.bst         ] START   Staging dependencies
[00:00:06][][   build:eosdev/filesystem.bst         ] SUCCESS Staging dependencies
[--:--:--][][   build:eosdev/filesystem.bst         ] START   Computing split
[00:00:08][][   build:eosdev/filesystem.bst         ] SUCCESS Computing split
[--:--:--][][   build:eosdev/filesystem.bst         ] START   Integrating sandbox
[--:--:--][][   build:eosdev/filesystem.bst         ] START   Running commands
[00:01:24][][   build:eosdev/filesystem.bst         ] SUCCESS Running commands
[--:--:--][][   build:eosdev/filesystem.bst         ] INFO    Integration added 1435 and removed 0 files
[00:01:28][][   build:eosdev/filesystem.bst         ] SUCCESS Integrating sandbox
[--:--:--][][   build:eosdev/filesystem.bst         ] START   Creating composition
    Including files from all domains
    Excluding files from domains: debug
    Including orphaned files
[--:--:--][][   build:eosdev/filesystem.bst         ] INFO    Composing 225327 files
[00:00:00][][   build:eosdev/filesystem.bst         ] SUCCESS Creating composition
[--:--:--][][   build:eosdev/filesystem.bst         ] START   Caching artifact
[00:00:09][][   build:eosdev/filesystem.bst         ] SUCCESS Caching artifact
[00:01:54][][   build:eosdev/filesystem.bst         ] SUCCESS endlessos/eosdev-filesystem/-build.20251002-132845.log
[--:--:--][][    push:eosdev/filesystem.bst         ] START   endlessos/eosdev-filesystem/-push.20251002-133039.log
[--:--:--][][    push:eosdev/filesystem.bst         ] INFO    Pushed data from artifact  -> https://bstcache.endlessos.org
[--:--:--][][    push:eosdev/filesystem.bst         ] INFO    Pushed artifact  -> https://bstcache.endlessos.org
[00:00:11][][    push:eosdev/filesystem.bst         ] SUCCESS endlessos/eosdev-filesystem/-push.20251002-133039.log

[--:--:--][][   build:repo.bst                      ] START   endlessos/repo/-build.20251002-133039.log
[--:--:--][][   build:repo.bst                      ] START   Staging dependencies at: /
[00:00:01][][   build:repo.bst                      ] SUCCESS Staging dependencies at: /
[--:--:--][][   build:repo.bst                      ] START   Staging dependencies at: /sysroot-eos
[00:00:01][][   build:repo.bst                      ] SUCCESS Staging dependencies at: /sysroot-eos
[--:--:--][][   build:repo.bst                      ] START   Staging dependencies at: /sysroot-eosdev
[00:00:01][][   build:repo.bst                      ] SUCCESS Staging dependencies at: /sysroot-eosdev
[--:--:--][][   build:repo.bst                      ] START   Integrating sandbox
[00:00:41][][   build:repo.bst                      ] SUCCESS Integrating sandbox
[--:--:--][][   build:repo.bst                      ] START   Running commands
[00:17:40][][   build:repo.bst                      ] SUCCESS Running commands
[--:--:--][][   build:repo.bst                      ] START   Caching artifact
[00:00:01][][   build:repo.bst                      ] SUCCESS Caching artifact
[00:18:28][][   build:repo.bst                      ] SUCCESS endlessos/repo/-build.20251002-133039.log

The new eosdev/filesystem.bst element adds 2 minutes to ~all builds, and the repo.bst build takes 10 minutes longer than before.

Compare with the last build of 'main' (job 51758579199 which took the following to build repo.bst:

[00:08:19][][   build:eos/repo.bst                  ] SUCCESS endlessos/eos-repo/-build.20251002-024640.log

The reason repo.bst is slow to build is twofold:

• OSTree compresses each object in sequence on a single core, so the build is bound by compute speed
• Single-thread performance on the AWS build workers is not amazing. (On my AMD Ryzen 9 9900X machine the current repo.bst element builds in less than 4 minutes>

A possible workaround could be to add an optimization in the Github Actions workflow as follows:

• On pushes to main, always build repo.bst
• On pull requests
  • If repo.bst, or */filesystem.bst changed, built repo.bst
  • Otherwise, build only eos/deps.bst and eosdev/deps.bst

[ssssam]

Contents of the repo with this change:

> du -sh ./ostree
3.1G	./ostree

> ostree refs --repo ./ostree/repo/
os/eos/amd64/master
os/eosdev/amd64/master

> ostree ls -R --repo ./ostree/repo/ os/eos/amd64/master|wc -l
126398
> ostree ls -R --repo ./ostree/repo/ os/eosdev/amd64/master|wc -l
225502

> ostree ls -R --repo ./ostree/repo/ os/eos/amd64/master|grep bin/gcc

> ostree ls -R --repo ./ostree/repo/ os/eosdev/amd64/master|grep bin/gcc
l00777 0 0      0 /usr/bin/gcc -> x86_64-unknown-linux-gnu-gcc
l00777 0 0      0 /usr/bin/gcc-ar -> x86_64-unknown-linux-gnu-gcc-ar
l00777 0 0      0 /usr/bin/gcc-nm -> x86_64-unknown-linux-gnu-gcc-nm
l00777 0 0      0 /usr/bin/gcc-ranlib -> x86_64-unknown-linux-gnu-gcc-ranlib

And so on.

[starnight]

• On pushes to main, always build repo.bst
• On pull requests
  • If repo.bst, or */filesystem.bst changed, built repo.bst
  • Otherwise, build only eos/deps.bst and eosdev/deps.bst

Makes sense and should be.