jsoninfo: introduce inventory-artifacts cli
#8910
Open
+53
−9
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rnel inventory-ing - really the same as targets-default.yaml but sans desktops and only `kernel` arfifacts
…rtifact_input_variables
- stops after artifacts-reduced and adds kernels.ndjson and uboots.njdson via jq - if one runs `./compile.sh inventory-artifacts` we shall get: - `output/info/kernels.ndjson`: one line for each kernel build - `output/info/uboots.ndjson`: one line for each u-boot build - this is meant as base for other tooling that acts "for each kernel" or "for each u-boot" such as patch/config rewrites, etc - cli-jsoninfo: better kernels.ndjson; new kernels-duplicate-config.json - so we can detect mistakes ref LINUXCONFIG - maybe one day the "Rewrite Kernel Config" GHA can re-use kernels.ndjson - previous commit introduced targets-all-kernels.yaml, so one can run - `./compile.sh inventory-artifacts TARGETS_TEMPLATE=targets-all-kernels.yaml`
github-actions
bot
added
11
Contributor
WalkthroughNew kernel inventory feature is added with a YAML target template for building all kernels across non-eos boards, kernel artifact variables (LINUXCONFIG, ARMBIAN_KERNEL_DEB_NAME), a new inventory-artifacts command, and duplicate LINUXCONFIG detection across kernel artifacts. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant CLI
participant Reducer
participant DupCheck
participant Output
User->>CLI: inventory-artifacts command
CLI->>Reducer: run artifact reducer
Reducer-->>CLI: artifacts processed
CLI->>Output: create kernels.ndjson<br/>(jq extract kernel artifacts)
CLI->>Output: create uboots.ndjson<br/>(jq extract u-boot artifacts)
CLI->>DupCheck: check duplicate LINUXCONFIG<br/>across kernels
alt Duplicates found
DupCheck-->>CLI: duplicates detected
CLI->>Output: create kernels-duplicate-config.json
CLI-->>User: error alert logged
else No duplicates
DupCheck-->>CLI: no duplicates
CLI-->>User: info message logged
end
CLI-->>User: return 0
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Comment |
github-actions
bot
added
Needs review
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
lib/functions/cli/cli-jsoninfo.sh (1)
219-224: Minor: File size check for duplicate detection could be more explicit.Line 219 uses
stat -c%s > 3to detect whether duplicates were found. An empty JSON array[]is ~2 bytes, so this should work in practice. However, the check is somewhat implicit and fragile if file format changes (e.g., trailing whitespace). Consider a more explicit check:if [[ $(jq 'length' "${REDUCED_KERNELS_DUPLICATE_LINUXCONFIG_FILE}") -gt 0 ]]; thenThis reads the array length directly, making the intent clearer and more maintainable. That said, the current approach is functional.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (4)
config/templates/targets-all-kernels.yaml(1 hunks)lib/functions/artifacts/artifact-kernel.sh(1 hunks)lib/functions/cli/cli-jsoninfo.sh(3 hunks)lib/functions/cli/commands.sh(2 hunks)
🧰 Additional context used
🧠 Learnings (18)
📓 Common learnings
Learnt from: amazingfate
Repo: armbian/build PR: 8617
File: lib/functions/compilation/kernel.sh:126-127
Timestamp: 2025-09-12T09:49:31.957Z
Learning: In the Armbian build system, the "all" target in kernel build has been used for a long time without issues. The "all" target builds vmlinux/Image/Image.gz defaults for the architecture and works correctly alongside KERNEL_IMAGE_TYPE in the build_targets_build array.
Learnt from: tabrisnet
Repo: armbian/build PR: 8678
File: config/kernel/linux-bcm2711-edge.config:859-861
Timestamp: 2025-09-27T21:47:58.020Z
Learning: In the Armbian build system, kernel configuration files in config/kernel/ are generated through an automated process: taking previous config → applying scripted changes from armbian-kernel.sh → running kernel's `make oldconfig` → processing by Armbian machinery back into config files. This automated process properly handles kernel configuration dependencies and reduces the likelihood of manual configuration errors.
Learnt from: EvilOlaf
Repo: armbian/build PR: 0
File: :0-0
Timestamp: 2025-08-02T05:46:10.664Z
Learning: In the Armbian build system, the modern recommended approach for kernel configuration is to use the kernel-config command via "./compile.sh BOARD=boardname BRANCH=branchname kernel-config" instead of the deprecated KERNEL_CONFIGURE=yes flag. This provides a two-step workflow: configure using menuconfig, then build, with better transparency and control over configuration changes.
Learnt from: EvilOlaf
Repo: armbian/build PR: 0
File: :0-0
Timestamp: 2025-08-02T05:46:10.664Z
Learning: In the Armbian build system, the KERNEL_CONFIGURE=yes switch is deprecated. The recommended approach for kernel configuration is now to use the `kernel-config` build command, which provides a more structured way to configure kernel options.
Learnt from: tabrisnet
Repo: armbian/build PR: 8678
File: config/kernel/linux-sm8250-edge.config:80-82
Timestamp: 2025-09-27T21:50:04.845Z
Learning: In the Armbian build system, kernel configuration files are generated through this automated process: taking previous config → applying scripted changes from armbian-kernel.sh → running kernel's `make oldconfig` → processing by Armbian machinery back into config files. This automated process properly handles kernel configuration dependencies and reduces the likelihood of manual configuration errors.
Learnt from: tabrisnet
Repo: armbian/build PR: 8678
File: config/kernel/linux-sm8250-current.config:78-80
Timestamp: 2025-09-27T21:49:55.796Z
Learning: In the Armbian build system, kernel configuration files are generated through an automated process: taking previous config → applying scripted changes from armbian-kernel.sh → running kernel's `make oldconfig` → processing by Armbian machinery back into config files. This automated process properly handles kernel configuration dependencies and reduces the likelihood of manual configuration errors.
Learnt from: igorpecovnik
Repo: armbian/build PR: 8849
File: config/boards/radxa-e54c.csc:14-28
Timestamp: 2025-11-02T20:49:56.719Z
Learning: In Armbian board configuration files (config/boards/*.conf, *.csc, etc.), do not use kernel_config_set, kernel_config_set_m, kernel_config_set_y, or custom_kernel_config__* functions to modify kernel configuration. Kernel configuration is associated with LINUXFAMILY/BOARDFAMILY, not individual BOARD. Board-specific kernel modifications cause inconsistency in kernel packages published to the apt repository because boards within a family share the same kernel packages. Kernel configuration changes must be made in the appropriate kernel config file (e.g., config/kernel/linux-*-*.config) or in family configuration files (config/sources/families/*.conf, *.inc) instead.
Learnt from: leggewie
Repo: armbian/build PR: 8524
File: config/boards/orangepi2.csc:6-6
Timestamp: 2025-08-21T08:10:59.502Z
Learning: Not all Armbian boards support all kernel versions (legacy, current, edge). Some boards may only support specific kernel versions due to hardware limitations or lack of mainline support, which is why their KERNEL_TARGET contains only the supported options (e.g., just "legacy").
Learnt from: leggewie
Repo: armbian/build PR: 8524
File: config/boards/nanopiduo2.csc:12-12
Timestamp: 2025-08-21T08:10:25.459Z
Learning: KERNEL_TARGET and KERNEL_TEST_TARGET in Armbian build system do not have enforced validation relationships. KERNEL_TARGET is used for interactive menu ordering and branch validation, while KERNEL_TEST_TARGET is used for build list generation and recorded in /etc/armbian-release. No existing codebase validation enforces that KERNEL_TEST_TARGET values must be present in KERNEL_TARGET.
Learnt from: EvilOlaf
Repo: armbian/build PR: 8330
File: config/sources/families/sun55iw3.conf:32-36
Timestamp: 2025-06-25T03:42:09.086Z
Learning: In Armbian build system configuration files like config/sources/families/*.conf, KERNELSOURCE is explicitly declared when using unofficial or 3rd party kernel repositories (like the "dev" branch using https://github.com/apritzel/linux), but can be omitted when using the standard mainline kernel (like the "edge" branch) since it will fall back to the default mainline source.
📚 Learning: 2025-09-27T21:50:04.845Z
Learnt from: tabrisnet
Repo: armbian/build PR: 8678
File: config/kernel/linux-sm8250-edge.config:80-82
Timestamp: 2025-09-27T21:50:04.845Z
Learning: In the Armbian build system, kernel configuration files are generated through this automated process: taking previous config → applying scripted changes from armbian-kernel.sh → running kernel's `make oldconfig` → processing by Armbian machinery back into config files. This automated process properly handles kernel configuration dependencies and reduces the likelihood of manual configuration errors.
Applied to files:
lib/functions/artifacts/artifact-kernel.shlib/functions/cli/cli-jsoninfo.sh
📚 Learning: 2025-09-27T21:47:58.020Z
Learnt from: tabrisnet
Repo: armbian/build PR: 8678
File: config/kernel/linux-bcm2711-edge.config:859-861
Timestamp: 2025-09-27T21:47:58.020Z
Learning: In the Armbian build system, kernel configuration files in config/kernel/ are generated through an automated process: taking previous config → applying scripted changes from armbian-kernel.sh → running kernel's `make oldconfig` → processing by Armbian machinery back into config files. This automated process properly handles kernel configuration dependencies and reduces the likelihood of manual configuration errors.
Applied to files:
lib/functions/artifacts/artifact-kernel.sh
📚 Learning: 2025-11-02T20:49:56.719Z
Learnt from: igorpecovnik
Repo: armbian/build PR: 8849
File: config/boards/radxa-e54c.csc:14-28
Timestamp: 2025-11-02T20:49:56.719Z
Learning: In Armbian board configuration files (config/boards/*.conf, *.csc, etc.), do not use kernel_config_set, kernel_config_set_m, kernel_config_set_y, or custom_kernel_config__* functions to modify kernel configuration. Kernel configuration is associated with LINUXFAMILY/BOARDFAMILY, not individual BOARD. Board-specific kernel modifications cause inconsistency in kernel packages published to the apt repository because boards within a family share the same kernel packages. Kernel configuration changes must be made in the appropriate kernel config file (e.g., config/kernel/linux-*-*.config) or in family configuration files (config/sources/families/*.conf, *.inc) instead.
Applied to files:
lib/functions/artifacts/artifact-kernel.sh
📚 Learning: 2025-09-27T21:49:55.796Z
Learnt from: tabrisnet
Repo: armbian/build PR: 8678
File: config/kernel/linux-sm8250-current.config:78-80
Timestamp: 2025-09-27T21:49:55.796Z
Learning: In the Armbian build system, kernel configuration files are generated through an automated process: taking previous config → applying scripted changes from armbian-kernel.sh → running kernel's `make oldconfig` → processing by Armbian machinery back into config files. This automated process properly handles kernel configuration dependencies and reduces the likelihood of manual configuration errors.
Applied to files:
lib/functions/artifacts/artifact-kernel.sh
📚 Learning: 2025-09-27T21:50:15.915Z
Learnt from: tabrisnet
Repo: armbian/build PR: 8678
File: config/kernel/linux-sunxi64-current.config:94-94
Timestamp: 2025-09-27T21:50:15.915Z
Learning: When kernel config files are generated through Armbian's automated process (previous config → armbian-kernel.sh changes → make oldconfig → Armbian machinery processing), manual config file edits are not appropriate since they would be overwritten. Deprecated option handling should be implemented in the automated tooling instead.
Applied to files:
lib/functions/artifacts/artifact-kernel.shlib/functions/cli/cli-jsoninfo.sh
📚 Learning: 2025-05-05T12:35:07.143Z
Learnt from: Grippy98
Repo: armbian/build PR: 8152
File: lib/functions/configuration/interactive.sh:209-266
Timestamp: 2025-05-05T12:35:07.143Z
Learning: For the interactive kernel selection in Armbian, KERNEL_MAJOR_MINOR and KERNEL_DESCRIPTION are parsed from family.conf but deliberately not set as environment variables to avoid potential interference with other parts of the build system.
Applied to files:
lib/functions/artifacts/artifact-kernel.sh
📚 Learning: 2025-09-22T21:52:01.225Z
Learnt from: tabrisnet
Repo: armbian/build PR: 8661
File: lib/functions/compilation/armbian-kernel.sh:194-201
Timestamp: 2025-09-22T21:52:01.225Z
Learning: In lib/functions/compilation/armbian-kernel.sh, the kernel_config_modifying_hashes array is not universally required for all kernel configuration functions - some functions like armbian_kernel_config__netkit() operate without it, and adding entries with '=m' would be incorrect when the actual result might be '=y' for options already built-in.
Applied to files:
lib/functions/artifacts/artifact-kernel.sh
📚 Learning: 2025-09-22T21:59:36.334Z
Learnt from: tabrisnet
Repo: armbian/build PR: 8661
File: lib/functions/compilation/armbian-kernel.sh:200-200
Timestamp: 2025-09-22T21:59:36.334Z
Learning: Functions named with the pattern `armbian_kernel_config__*` in lib/functions/compilation/armbian-kernel.sh are automatically discovered and invoked by Armbian's extension system via metaprogramming using `compgen -A function`, without requiring explicit registration or calls.
Applied to files:
lib/functions/artifacts/artifact-kernel.sh
📚 Learning: 2025-08-02T05:46:10.664Z
Learnt from: EvilOlaf
Repo: armbian/build PR: 0
File: :0-0
Timestamp: 2025-08-02T05:46:10.664Z
Learning: In the Armbian build system, the modern recommended approach for kernel configuration is to use the kernel-config command via "./compile.sh BOARD=boardname BRANCH=branchname kernel-config" instead of the deprecated KERNEL_CONFIGURE=yes flag. This provides a two-step workflow: configure using menuconfig, then build, with better transparency and control over configuration changes.
Applied to files:
lib/functions/artifacts/artifact-kernel.sh
📚 Learning: 2025-10-14T05:08:11.785Z
Learnt from: EvilOlaf
Repo: armbian/build PR: 8754
File: config/boards/bestv-r3300-l.csc:14-16
Timestamp: 2025-10-14T05:08:11.785Z
Learning: In the Armbian build system, BOOTBRANCH_BOARD is a valid framework variable used as a fallback when BOOTBRANCH is unset. The framework checks BOOTBRANCH_BOARD before applying the default bootloader branch value (see config/sources/common.conf). Board configuration files can use BOOTBRANCH_BOARD to specify the bootloader branch.
Applied to files:
lib/functions/artifacts/artifact-kernel.sh
📚 Learning: 2025-06-25T03:42:09.086Z
Learnt from: EvilOlaf
Repo: armbian/build PR: 8330
File: config/sources/families/sun55iw3.conf:32-36
Timestamp: 2025-06-25T03:42:09.086Z
Learning: In Armbian build system configuration files like config/sources/families/*.conf, KERNELSOURCE is explicitly declared when using unofficial or 3rd party kernel repositories (like the "dev" branch using https://github.com/apritzel/linux), but can be omitted when using the standard mainline kernel (like the "edge" branch) since it will fall back to the default mainline source.
Applied to files:
lib/functions/artifacts/artifact-kernel.sh
📚 Learning: 2025-06-25T03:40:52.109Z
Learnt from: EvilOlaf
Repo: armbian/build PR: 8330
File: config/sources/families/sun55iw3.conf:32-36
Timestamp: 2025-06-25T03:40:52.109Z
Learning: In Armbian build system configuration files like config/sources/families/*.conf, when KERNELSOURCE is not explicitly declared in a case branch, the system falls back to a default KERNELSOURCE value, so missing KERNELSOURCE declarations are not critical issues.
Applied to files:
lib/functions/artifacts/artifact-kernel.sh
📚 Learning: 2025-09-07T17:39:32.272Z
Learnt from: EvilOlaf
Repo: armbian/build PR: 8586
File: config/boards/nanopi-r76s.conf:15-21
Timestamp: 2025-09-07T17:39:32.272Z
Learning: In the Armbian build system, the variables $BOARD and $SDCARD are always set by the build framework, so guard checks for these variables are unnecessary in board configuration files and hook functions.
Applied to files:
lib/functions/artifacts/artifact-kernel.sh
📚 Learning: 2025-09-14T11:37:35.089Z
Learnt from: amazingfate
Repo: armbian/build PR: 8619
File: config/sources/families/rockchip.conf:65-72
Timestamp: 2025-09-14T11:37:35.089Z
Learning: In the Armbian build system, patch directories referenced in BOOTPATCHDIR and KERNELPATCHDIR configurations can be non-existent without causing build failures. Empty patch directories are also ignored by git, so missing patch directories should not be flagged as errors during code review.
Applied to files:
lib/functions/cli/commands.sh
📚 Learning: 2025-09-12T09:49:31.957Z
Learnt from: amazingfate
Repo: armbian/build PR: 8617
File: lib/functions/compilation/kernel.sh:126-127
Timestamp: 2025-09-12T09:49:31.957Z
Learning: In the Armbian build system, the "all" target in kernel build has been used for a long time without issues. The "all" target builds vmlinux/Image/Image.gz defaults for the architecture and works correctly alongside KERNEL_IMAGE_TYPE in the build_targets_build array.
Applied to files:
config/templates/targets-all-kernels.yaml
📚 Learning: 2025-08-21T08:10:25.459Z
Learnt from: leggewie
Repo: armbian/build PR: 8524
File: config/boards/nanopiduo2.csc:12-12
Timestamp: 2025-08-21T08:10:25.459Z
Learning: KERNEL_TARGET and KERNEL_TEST_TARGET in Armbian build system do not have enforced validation relationships. KERNEL_TARGET is used for interactive menu ordering and branch validation, while KERNEL_TEST_TARGET is used for build list generation and recorded in /etc/armbian-release. No existing codebase validation enforces that KERNEL_TEST_TARGET values must be present in KERNEL_TARGET.
Applied to files:
config/templates/targets-all-kernels.yaml
📚 Learning: 2025-08-21T08:10:59.502Z
Learnt from: leggewie
Repo: armbian/build PR: 8524
File: config/boards/orangepi2.csc:6-6
Timestamp: 2025-08-21T08:10:59.502Z
Learning: Not all Armbian boards support all kernel versions (legacy, current, edge). Some boards may only support specific kernel versions due to hardware limitations or lack of mainline support, which is why their KERNEL_TARGET contains only the supported options (e.g., just "legacy").
Applied to files:
config/templates/targets-all-kernels.yaml
🧬 Code graph analysis (3)
lib/functions/artifacts/artifact-kernel.sh (5)
lib/functions/artifacts/artifact-armbian-base-files.sh (1)
artifact_armbian-base-files_config_dump(13-16)lib/functions/artifacts/artifact-rootfs.sh (1)
artifact_rootfs_config_dump(10-23)lib/functions/artifacts/artifact-armbian-bsp-desktop.sh (1)
artifact_armbian-bsp-desktop_config_dump(10-18)lib/functions/artifacts/artifact-uboot.sh (1)
artifact_uboot_config_dump(10-17)lib/functions/artifacts/artifact-armbian-bsp-cli.sh (1)
artifact_armbian-bsp-cli_config_dump(10-14)
config/templates/targets-all-kernels.yaml (2)
lib/functions/compilation/kernel.sh (1)
kernel_prepare_build_and_package(119-174)lib/functions/configuration/main-config.sh (1)
write_config_summary_output_file(488-541)
lib/functions/cli/cli-jsoninfo.sh (1)
lib/functions/logging/runners.sh (1)
run_host_command_logged(204-206)
🔇 Additional comments (7)
lib/functions/artifacts/artifact-kernel.sh (1)
13-14: New kernel inventory variables integrate well.The addition of
LINUXCONFIGandARMBIAN_KERNEL_DEB_NAMEtoartifact_input_variablesaligns with the existing pattern used inartifact-uboot.shandartifact-rootfs.sh. TheARMBIAN_KERNEL_DEB_NAMEderivation (${LINUXFAMILY}-${BRANCH}) correctly mirrors legacy kernel package naming and will be used by downstream inventory processing incli-jsoninfo.sh(line 209, 216).config/templates/targets-all-kernels.yaml (1)
1-12: Well-designed kernel-only inventory template.The
all-kernelstarget correctly restricts to kernel artifacts, disables desktop builds, and targets non-eos boards. This template cleanly implements the PR objective: optimized kernel inventorying. Users can activate it via./compile.sh inventory-artifacts TARGETS_TEMPLATE=targets-all-kernels.yaml.lib/functions/cli/cli-jsoninfo.sh (4)
118-120: Output file path declarations are clear and well-organized.The new NDJSON and JSON output files (
kernels.ndjson,uboots.ndjson,kernels-duplicate-config.json) follow established naming and location conventions (underBASE_INFO_OUTPUT_DIR).
20-20: Shellcheck disables are appropriately scoped.The
SC2317disables correctly suppress "command appears unreachable" warnings for the nested function definitions (json_info_logged,json_info_only), which are invoked viado_with_default_buildat line 314. This pattern is correct for the framework.Also applies to: 25-25
207-225: jq pipelines correctly extract and validate kernel/u-boot artifacts.Line 209's jq extracts kernel artifacts with BOARD, BRANCH, and newly-added LINUXCONFIG and ARMBIAN_KERNEL_DEB_NAME fields to NDJSON. Line 212 extracts u-boot artifacts similarly. Line 216's duplicate detection correctly groups kernels by LINUXCONFIG and flags groups with length > 1. The duplicate-checking logic (group_by, filter, map) is sound.
227-230: Early return for inventory-artifacts command is correct.The
inventory-artifactscommand now exits cleanly after generatingkernels.ndjson,uboots.ndjson, and the duplicate-config report, without proceeding to further artifact and image processing steps. This aligns with the PR's goal of providing a standalone inventory output.lib/functions/cli/commands.sh (1)
34-34: inventory-artifacts command registration follows established patterns.The new command is correctly mapped to the existing
json_infohandler (line 34) and configured with a dummyTARGETS_FILEto trigger template defaults (line 122), matching the pattern used forinventory-boards(line 121). The handler functionscli_json_info_pre_runandcli_json_info_runalready exist and support this command.Also applies to: 122-122
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
jsoninfo: introduce
inventory-artifactsclikernelarfifactsinventory-artifactscli./compile.sh inventory-artifactswe shall get:output/info/kernels.ndjson: one line for each kernel buildoutput/info/uboots.ndjson: one line for each u-boot build./compile.sh inventory-artifacts TARGETS_TEMPLATE=targets-all-kernels.yamlSummary by CodeRabbit
Release Notes
inventory-artifactscommand to query kernel and u-boot artifact inventory