[java] cdxgen fails to generate an sbom while a commercial tool does

[prabhu]

Received a report that cdxgen fails to generate an SBOM for a repository, while a commercial tool works and produces results. Although reports like these are not unusual, what intrigued me was that this particular project fails to build correctly (*), has no lock files (thanks, Maven!), and yet the commercial tool displays an SBOM with numerous components (around 250) and a dependency tree. How?

The repo in question:

https://github.com/apache/flex-blazeds

Investigation

Running cdxgen with default settings (recurse=true) shows a number of errors like below, but leads to an incomplete sbom with some components (~ 50) and a partial tree.

[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR]   The project org.apache.flex.blazeds:flex-messaging-opt-tomcat-6:4.8.0-SNAPSHOT (/Volumes/Work/sandbox/flex-blazeds-master/opt/tomcat/tomcat-6/pom.xml) has 1 error
[ERROR]     Non-resolvable parent POM for org.apache.flex.blazeds:flex-messaging-opt-tomcat-6:4.8.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.flex.blazeds:flex-messagi
ng-opt-tomcat:pom:4.8.0-SNAPSHOT (absent): Could not find artifact org.apache.flex.blazeds:flex-messaging-opt-tomcat:pom:4.8.0-SNAPSHOT and 'parent.relativePath' points at wrong local POM @ lin
e 23, column 13 -> [Help 2]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException
 Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 -Djna.library.path=/Users/prabhu/miniconda3/envs/chenpy/lib

The above build errors could be due to:

1. Check if the pom.xml contains valid settings for parent and modules. Some projects can be built only from a specific directory.
2. Private dependencies cannot be downloaded: Check if any additional arguments must be passed to maven and set them via MVN_ARGS environment variable.
3. Check if all required environment variables including any maven profile arguments are passed correctly to this tool.

There is genuinely a problem in this repo's master branch. The developers have forgotten to update the version number correctly in all places, so pom.xml files are referring to 4.8.0-SNAPSHOT (non-existent) instead of 4.9.0-SNAPSHOT. This is correctly shown in the error messages along with some troubleshooting tips (!).

After fixing the issues with the version numbers, cdxgen generates a complete SBOM with 128 components and 148 dependencies. However, this is still only half of what the commercial tool reports. So, does that mean cdxgen is bad?

Introducing Guess BOM

What the commercial tool appears to do is commonly referred to as "guessing" (or using a custom solver). When a build fails, instead of displaying Maven errors (which marketing wouldn't approve), the tool simply parses the dependencies attributes from the pom.xml files, downloads the jars, and continues parsing until it gathers several levels of information. Unfortunately, such naive algorithms are often incompatible with the package manager's dependency solver algorithms (DFS, BF, Skipper) and might even fail to account for any custom overrides, settings, or external factors (e.g., weather, time :)).

Is this wrong?

Not necessarily. Every SBOM tool, including cdxgen, makes up some information as it goes. However, users have the right to know the technique, the tool's confidence level, and the associated evidence for transparency and fair assessment. The particular commercial tool in question doesn't include evidence or confidence, making it impossible for end users to evaluate its precision. Additionally, it reinforces a misleading narrative: open-source is inferior, while commercial is superior.

What should end users do?

Learn terminologies such as lifecycles and aggregate.

• To generate a pre-build SBOM, lock files must be available. Most package managers and repositories might not include one.
• To generate a build SBOM, a buildable environment with the correct build tools installed is required. If the versions of these tools do not align, the SBOM may be imprecise (referred to as the reproducibility problem).
• To generate a post-build SBOM, the artifacts must be built and generated correctly, retaining all necessary debug and library information (e.g., unshaded and debug builds).

More importantly, stop expecting magic from sbom tools. Generating xBOMs is a complex process that requires active user participation. You should understand both the project and the capabilities of the tools.

Steps to reproduce

git clone https://github.com/apache/flex-blazeds.git
cd flex-blazeds
sdk use java 8.0.442-tem
mvn compile

# sbom generated with errors
cdxgen -t java -o bom.json $(pwd)

git apply version-update.patch
# sbom generated without errors
cdxgen -t java -o bom.json $(pwd) -p

Attachments

• Patch file
• bom after patching