Skip to content

Commit f36e678

Browse files
munishchouhanmunishchouhan
authored
Merge branch 'master' into add-nodeselectors
2 parents 9554227 + 36c4b42 commit f36e678

File tree

92 files changed

+1927
-2154
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

92 files changed

+1927
-2154
lines changed

‎.github/workflows/build-plugin-scanner.yml‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
name: Build and publish Scanner container image
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
trivy_version:
7+
description: 'trivy version (e.g., 0.65.0)'
8+
required: true
9+
default: '0.65.0'
10+
oras_version :
11+
description : 'oras version (e.g., 1.3.0)'
12+
required : true
13+
default : '1.3.0'
14+
15+
jobs:
16+
build-and-push:
17+
runs-on: ubuntu-latest
18+
steps:
19+
- name: Checkout code
20+
uses: actions/checkout@v4
21+
22+
- name: Set up Docker Buildx
23+
id: buildx
24+
uses: docker/setup-buildx-action@v3
25+
26+
- name: Docker Login
27+
uses: docker/login-action@v3
28+
with:
29+
registry: public.cr.seqera.io
30+
username: ${{ vars.SEQERA_PUBLIC_CR_USERNAME }}
31+
password: ${{ secrets.SEQERA_PUBLIC_CR_PASSWORD }}
32+
33+
- name: Build and Push Image to public.cr.seqera.io
34+
run: |
35+
cd plugin-scanner
36+
make all trivy_version=${{ github.event.inputs.trivy_version }} oras_version=${{ github.event.inputs.oras_version }}

‎VERSION‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
1.25.2
1+
1.26.1

‎adr/20251016-plugin-scanning.md‎

Lines changed: 304 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,304 @@
1+
# Security Scanning for Nextflow Plugins in OCI Registries
2+
3+
## Summary
4+
5+
This document describes the implementation of security scanning for Nextflow plugins hosted in OCI registries. The unified scanner can now detect and scan both traditional container images and Nextflow plugin artifacts, providing comprehensive vulnerability analysis across the entire Wave ecosystem.
6+
7+
## Context
8+
9+
Nextflow plugins are distributed as OCI artifacts in container registries, similar to container images but with different content types. These plugins contain JAR files and dependencies that need security scanning to detect vulnerabilities, just like container images. However, the scanning approach differs since plugins are not container images but rather ZIP archives containing Java artifacts.
10+
11+
## Feature Overview
12+
13+
### What it does
14+
- **Unified Scanning Infrastructure**: Single scanner implementation handles both container images and Nextflow plugins
15+
- **OCI Registry Support**: Pulls plugin artifacts from any OCI-compliant registry using ORAS
16+
- **Vulnerability Detection**: Scans plugin dependencies and JAR libraries for known CVEs using Trivy
17+
- **Multiple Output Formats**: Generates reports in JSON, SPDX, and CycloneDX formats
18+
- **Multi-Architecture Support**: Scanner image works on both AMD64 and ARM64 platforms
19+
20+
### How it works
21+
22+
**Detection Flow:**
23+
1. Scan request comes in with target image name
24+
2. System detects scan type based on image name pattern (`nextflow/plugin` → plugin scan)
25+
3. Scanner executes appropriate workflow:
26+
27+
**For Plugins:**
28+
- Downloads artifact using ORAS from OCI registry
29+
- Extracts plugin ZIP to temporary filesystem
30+
- Performs rootfs scan with Trivy (`trivy rootfs --scanners vuln`)
31+
- Generates vulnerability reports
32+
33+
**For Containers:**
34+
- Performs standard container image scan with Trivy (`trivy image`)
35+
- Analyzes all layers for vulnerabilities
36+
- Generates vulnerability reports
37+
38+
## Decision Drivers
39+
40+
1. **Unified Infrastructure**: Need to minimize operational complexity by using a single scanner for all artifact types
41+
2. **Flexibility**: Must support scanning of both standard OCI images and non-image OCI artifacts (plugins)
42+
3. **Consistency**: Same vulnerability database and reporting format across all scan types
43+
4. **Performance**: Efficient caching and resource utilization
44+
5. **Multi-Architecture**: Support for both AMD64 and ARM64 platforms
45+
46+
## Technical Decisions
47+
48+
### 1. Unified Scanner Script
49+
50+
**Decision:** Refactored `scanner/scan.sh` into a unified script with `--type` parameter
51+
- `--type container` → Container image scan
52+
- `--type plugin` → Nextflow plugin scan
53+
54+
**Rationale:**
55+
- Single script reduces maintenance overhead
56+
- Ensures consistent scanning behavior across types
57+
- Simplifies testing and deployment
58+
- Easier to add new scan types in the future
59+
60+
**Alternatives Considered:**
61+
- Separate scanner images for each type (rejected: increases maintenance)
62+
- Runtime auto-detection without explicit type (rejected: ambiguous for edge cases)
63+
64+
### 2. ORAS for Plugin Downloads
65+
66+
**Decision:** Integrated [ORAS](https://oras.land/) (OCI Registry As Storage) tool into scanner image
67+
68+
**Rationale:**
69+
- Industry-standard tool for OCI artifact management
70+
- Native support for non-container artifacts (plugins, charts, etc.)
71+
- Simple CLI interface integrates easily with bash scripts
72+
- Maintained by CNCF, ensuring long-term support
73+
74+
**Alternatives Considered:**
75+
- Custom OCI client implementation (rejected: reinventing the wheel)
76+
- Using docker/skopeo for artifact pull (rejected: designed for container images)
77+
78+
### 3. Filesystem Scanning for Plugins
79+
80+
**Decision:** Plugins scanned using Trivy's `rootfs` scanner mode with `--scanners vuln` flag
81+
82+
**Rationale:**
83+
- Plugins are JAR/ZIP artifacts, not container images
84+
- Filesystem scanning detects vulnerabilities in Java dependencies
85+
- Uses same Trivy vulnerability database as container scans
86+
- Provides consistent security posture across all artifacts
87+
88+
**Alternatives Considered:**
89+
- Container image wrapping (rejected: unnecessary complexity)
90+
- Custom vulnerability scanner (rejected: maintenance burden)
91+
92+
### 4. Explicit Cache Directory Configuration
93+
94+
**Decision:** Added `--cache-dir` option to configure Trivy's cache location explicitly
95+
96+
**Rationale:**
97+
- Improves reproducibility across environments
98+
- Allows mounting external cache volumes in production for persistence
99+
- Uses constant `Trivy.CACHE_MOUNT_PATH` for consistency across codebase
100+
- Better control over disk usage and cache lifecycle
101+
102+
**Alternatives Considered:**
103+
- Environment variable only (rejected: less explicit)
104+
- Hardcoded path (rejected: inflexible for different environments)
105+
106+
### 5. Multi-Architecture Scanner Image
107+
108+
**Decision:** Fixed Dockerfile to properly support both `amd64` and `arm64` architectures using Docker's `TARGETARCH` build arg
109+
110+
**Rationale:**
111+
- Critical for ARM-based infrastructure (AWS Graviton, Apple Silicon)
112+
- Single image tag works across all architectures
113+
- Reduces image management complexity
114+
- Improves deployment flexibility
115+
116+
**Alternatives Considered:**
117+
- Separate images per architecture (rejected: complex tag management)
118+
- AMD64 only (rejected: excludes ARM environments)
119+
120+
### 6. Pattern-Based Scan Type Detection
121+
122+
**Decision:** Detects plugin scans by checking if image name contains `nextflow/plugin`
123+
124+
**Rationale:**
125+
- Simple and effective for current use case
126+
- No additional registry roundtrips required
127+
- Provides clear migration path to manifest media type detection
128+
- Works immediately without registry changes
129+
130+
**Alternatives Considered:**
131+
- Manifest media type inspection (planned for future: #919)
132+
- User-provided scan type parameter (rejected: additional API complexity)
133+
134+
## Architecture
135+
136+
```
137+
┌─────────────────────────────────────────────────────────────┐
138+
│ Wave Application │
139+
│ │
140+
│ ┌────────────────────────────────────────────────────┐ │
141+
│ │ ContainerScanServiceImpl │ │
142+
│ │ • fromBuild() │ │
143+
│ │ • fromMirror() │ │
144+
│ │ • fromContainer() │ │
145+
│ └──────────────────┬──────────────────────────────────┘ │
146+
│ │ │
147+
│ ▼ │
148+
│ ┌────────────────────────────────────────────────────┐ │
149+
│ │ ScanStrategy │ │
150+
│ │ • buildScanCommand() │ │
151+
│ │ • Detects: container vs plugin │ │
152+
│ └──────────────────┬──────────────────────────────────┘ │
153+
│ │ │
154+
│ ▼ │
155+
│ ┌────────────────────────────────────────────────────┐ │
156+
│ │ KubeScanStrategy / DockerScanStrategy │ │
157+
│ │ • Launches scanner container │ │
158+
│ └──────────────────┬──────────────────────────────────┘ │
159+
└───────────────────────┼────────────────────────────────────┘
160+
│
161+
▼
162+
┌─────────────────────────────────────────────────────────────┐
163+
│ Scanner Container Image │
164+
│ (public.cr.seqera.io/wave/scanner:v1) │
165+
│ │
166+
│ ┌────────────────────────────────────────────────────┐ │
167+
│ │ /usr/local/bin/scan.sh │ │
168+
│ │ │ │
169+
│ │ if [type == container]: │ │
170+
│ │ trivy image --platform $PLATFORM $TARGET │ │
171+
│ │ │ │
172+
│ │ if [type == plugin]: │ │
173+
│ │ oras pull $TARGET │ │
174+
│ │ unzip plugin.zip │ │
175+
│ │ trivy rootfs --scanners vuln /extracted │ │
176+
│ └────────────────────────────────────────────────────┘ │
177+
│ │
178+
│ Components: │
179+
│ • Trivy 0.65.0 (vulnerability scanner) │
180+
│ • ORAS 1.3.0 (OCI artifact client) │
181+
│ • bash, unzip (utilities) │
182+
└─────────────────────────────────────────────────────────────┘
183+
```
184+
185+
## Implementation Details
186+
187+
### Files Changed
188+
189+
**Scanner Implementation:**
190+
- `scanner/scan.sh` - Unified bash script with plugin support and `--cache-dir` option
191+
- `scanner/Dockerfile` - Multi-arch build with ORAS integration and `TARGETARCH` support
192+
- `scanner/Makefile` - Build configuration for multi-platform images
193+
194+
**Application Code:**
195+
- `ScanStrategy.groovy` - Added `buildScanCommand()` with scan type detection and `--cache-dir` parameter
196+
- `KubeScanStrategy.groovy` - Simplified to use unified command builder
197+
- `DockerScanStrategy.groovy` - Simplified to use unified command builder
198+
- `Trivy.groovy` - Added `CACHE_MOUNT_PATH` constant
199+
200+
**Tests:**
201+
- `ScanStrategyTest.groovy` - Updated command expectations with `--cache-dir`
202+
- `BuildScanCommandTest.groovy` - Updated command expectations with `--cache-dir`
203+
204+
### Testing
205+
206+
All tests passing:
207+
```bash
208+
./gradlew test --tests 'io.seqera.wave.service.scan.*Test'
209+
```
210+
211+
Test coverage includes:
212+
- Container scans with/without platform and severity
213+
- Plugin scans with/without severity
214+
- Scan type auto-detection logic
215+
- Timeout conversions and parameter handling
216+
- Cache directory configuration
217+
218+
### Deployment
219+
220+
**Building the Scanner Image:**
221+
222+
```bash
223+
cd scanner
224+
make build # Builds for linux/amd64 and linux/arm64
225+
```
226+
227+
This publishes:
228+
- `public.cr.seqera.io/wave/scanner:v1-0.65.0-oras-1.3.0`
229+
- `public.cr.seqera.io/wave/scanner:v1` (latest)
230+
231+
**Configuration:**
232+
233+
The application automatically uses the scanner via existing configuration:
234+
```yaml
235+
wave:
236+
scan:
237+
image:
238+
name: public.cr.seqera.io/wave/scanner:v1
239+
```
240+
241+
## Example Usage
242+
243+
**Container Scan:**
244+
```bash
245+
/usr/local/bin/scan.sh \
246+
--type container \
247+
--target ubuntu:latest \
248+
--work-dir /tmp/scan \
249+
--platform linux/amd64 \
250+
--timeout 15 \
251+
--cache-dir /root/.cache/
252+
```
253+
254+
**Plugin Scan:**
255+
```bash
256+
/usr/local/bin/scan.sh \
257+
--type plugin \
258+
--target ghcr.io/nextflow-io/nextflow/plugins/nf-amazon:1.0.0 \
259+
--work-dir /tmp/scan \
260+
--timeout 15 \
261+
--cache-dir /root/.cache/
262+
```
263+
264+
## Benefits
265+
266+
1. ✅ **Unified Infrastructure** - Single image handles all security scanning
267+
2. ✅ **Simplified Maintenance** - One scanner to build, test, and deploy
268+
3. ✅ **Consistent Security** - Same Trivy version and vulnerability database
269+
4. ✅ **Better Performance** - Explicit cache configuration and reuse
270+
5. ✅ **Multi-Architecture** - Works on AMD64 and ARM64 platforms
271+
6. ✅ **Future-Ready** - Easy to extend for new artifact types
272+
273+
## Consequences
274+
275+
### Positive
276+
- Reduced operational complexity with single scanner image
277+
- Consistent vulnerability detection across all artifact types
278+
- Better resource utilization through unified caching
279+
- Multi-architecture support enables broader platform compatibility
280+
- Clear path for future artifact type additions
281+
282+
### Negative
283+
- Pattern-based detection less robust than media type inspection (mitigated: planned for #919)
284+
- Additional runtime dependency (ORAS) increases image size by ~10MB
285+
- Plugin extraction requires temporary disk space during scanning
286+
287+
### Neutral
288+
- Scanner image size increased from ~300MB to ~310MB (ORAS addition)
289+
- Slight increase in scan time for plugins due to extraction step (~2-3 seconds)
290+
291+
## References
292+
293+
- [ORAS Project](https://oras.land/)
294+
- [Trivy Documentation](https://aquasecurity.github.io/trivy/)
295+
- [OCI Distribution Spec](https://github.com/opencontainers/distribution-spec)
296+
- [Nextflow Plugins Documentation](https://www.nextflow.io/docs/latest/plugins.html)
297+
- Wave Issue #919: Manifest media type detection
298+
299+
---
300+
301+
**Status:** Implemented
302+
**Date:** 2025-10-16
303+
**Authors:** Wave Team
304+
**Related PR:** #912

‎build.gradle‎

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -32,11 +32,13 @@ dependencies {
3232
compileOnly 'io.micronaut:micronaut-inject-groovy'
3333
compileOnly 'io.micronaut:micronaut-http-validation'
3434
implementation 'jakarta.persistence:jakarta.persistence-api:3.0.0'
35-
implementation 'io.seqera:lib-lang:1.0.0'
36-
implementation 'io.seqera:lib-serde:1.0.0'
35+
implementation 'io.seqera:lib-cache-tiered-redis:1.0.0'
36+
implementation 'io.seqera:lib-lang:1.1.0'
37+
implementation 'io.seqera:lib-serde:1.1.0'
38+
implementation 'io.seqera:lib-serde-moshi:1.0.0'
3739
implementation 'io.seqera:lib-mail:1.3.0'
3840
implementation 'io.seqera:lib-pool:1.0.0'
39-
implementation 'io.seqera:lib-retry:1.1.0'
41+
implementation 'io.seqera:lib-retry:2.0.0'
4042
implementation 'io.seqera:lib-random:1.0.0'
4143
implementation 'io.seqera:lib-activator:1.0.0'
4244
implementation 'io.seqera:wave-api:0.16.0'
@@ -67,8 +69,7 @@ dependencies {
6769
implementation 'com.google.code.gson:gson:2.10.1'
6870
implementation 'com.fasterxml.jackson.datatype:jackson-datatype-jsr310'
6971
implementation 'com.fasterxml.jackson.dataformat:jackson-dataformat-yaml'
70-
implementation 'com.squareup.moshi:moshi:1.15.1'
71-
implementation 'com.squareup.moshi:moshi-adapters:1.15.1'
72+
// Moshi dependencies now provided by lib-serde-moshi
7273
implementation 'redis.clients:jedis:5.1.3'
7374
implementation 'io.github.resilience4j:resilience4j-ratelimiter:0.17.0'
7475
implementation 'io.micronaut:micronaut-retry'
@@ -115,7 +116,7 @@ dependencies {
115116
implementation 'io.micronaut.views:micronaut-views-handlebars'
116117
// bump version to prevent security issue
117118
runtimeOnly 'org.apache.commons:commons-lang3:3.18.0'
118-
runtimeOnly "io.netty:netty-bom:4.2.4.Final"
119+
runtimeOnly "io.netty:netty-bom:4.2.5.Final"
119120
}
120121

121122
application {

‎changelog.txt‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,18 @@
11
# Wave changelog
2+
1.26.1 - 23 Oct 2025
3+
- Refactor MoshiEncodeStrategy to remove Wave-specific polymorphic type adapters (#922) [abc71280d]
4+
- Bump lib-retry version 2.0.0 [63edabf95]
5+
- docs: Fix error in docs last_update (#921) [650c39415]
6+
- Upgrade Netty to 4.2.5.Final to fix CVE-2025-58057 (#920) [485ce7cd9]
7+
- docs: Update Wave use cases and FAQ (#917) [1b137e4ce]
8+
9+
1.26.0 - 16 Oct 2025
10+
- Add security scanning for Nextflow plugins in OCI registries (#912) [5acca4367]
11+
- Bump Micronaut to version 4.9.4 [b056aff91]
12+
- docs: Replace cards that loose formatting in production (#913) [c6155c9dc]
13+
- docs: Style Wave db migration page (#915) [5af1639b0]
14+
- docs: Remove Wave metrics page (#909) [95e1cc32b]
15+
216
1.25.2 - 25 Sep 2025
317
- Add CLAUDE.md documentation file [573c06ae]
418
- Bump corretto-25-al2023-jemalloc as base image [715c594a]

0 commit comments

Comments
 (0)