A container registry that uses the AT Protocol for manifest storage and S3 for blob storage. atcr.io
docker container atproto go
at refactor 568 lines 16 kB view raw view rendered
1# SBOM Scanning 2 3ATCR supports optional Software Bill of Materials (SBOM) generation for container images stored in holds. This feature enables automated security scanning and vulnerability analysis while maintaining the decentralized architecture. 4 5## Overview 6 7When enabled, holds automatically generate SBOMs for uploaded container images in the background. The scanning process: 8 9- **Async execution**: Scanning happens after upload completes (non-blocking) 10- **ORAS artifacts**: SBOMs stored as OCI Registry as Storage (ORAS) artifacts 11- **ATProto integration**: Scan results stored as `io.atcr.manifest` records in hold's embedded PDS 12- **Tool agnostic**: Results accessible via XRPC, ATProto queries, and direct blob URLs 13- **Opt-in**: Disabled by default, enabled per-hold via configuration 14 15### Default Scanner: Syft 16 17ATCR uses [Anchore Syft](https://github.com/anchore/syft) for SBOM generation: 18- Industry-standard SBOM generator 19- Supports SPDX and CycloneDX formats 20- Comprehensive package detection (OS packages, language libraries, etc.) 21- Active maintenance and CVE database updates 22 23Future enhancements may include [Grype](https://github.com/anchore/grype) for vulnerability scanning and [Trivy](https://github.com/aquasecurity/trivy) for comprehensive security analysis. 24 25## Trust Model 26 27### Same Trust as Docker Hub 28 29SBOM scanning follows the same trust model as Docker Hub or other centralized registries: 30 31**Docker Hub model:** 32- Docker Hub scans your image on their infrastructure 33- Results stored in their database 34- You trust Docker Hub's scanner version and scan integrity 35 36**ATCR hold model:** 37- Hold scans image on their infrastructure 38- Results stored in hold's embedded PDS 39- You trust hold operator's scanner version and scan integrity 40 41The security comes from **reproducibility** and **transparency**, not storage location: 42- Anyone can re-scan the same digest and verify results 43- Multiple holds scanning the same image provide independent verification 44- Scanner version and scan timestamp are recorded in ATProto records 45 46### Why Hold's PDS? 47 48Scan results are stored in the **hold's embedded PDS** rather than the user's PDS: 49 50**Advantages:** 511. **No OAuth expiry issues**: Hold owns its PDS, no service tokens needed 522. **Hold-scoped metadata**: Scanner version, scan time, hold configuration 533. **Multiple perspectives**: Different holds can scan the same image independently 544. **Simpler auth**: Hold writes directly to its own PDS 555. **Keeps user PDS lean**: Potentially large SBOM data doesn't bloat user's repo 56 57**Security properties:** 58- Same trust level as trusting hold to serve correct blobs 59- DID signatures prove which hold generated the SBOM 60- Reproducible scans enable independent verification 61- Multiple holds scanning same digest → compare results for tampering detection 62 63## ORAS Manifest Format 64 65SBOMs are stored as ORAS artifacts that reference their subject image using the OCI referrers specification. 66 67### Example Manifest Record 68 69```json 70{ 71 "$type": "io.atcr.manifest", 72 "repository": "alice/myapp", 73 "digest": "sha256:4a5e...", 74 "holdDid": "did:web:hold01.atcr.io", 75 "holdEndpoint": "https://hold01.atcr.io", 76 "schemaVersion": 2, 77 "mediaType": "application/vnd.oci.image.manifest.v1+json", 78 "artifactType": "application/spdx+json", 79 "subject": { 80 "mediaType": "application/vnd.oci.image.manifest.v1+json", 81 "digest": "sha256:abc123...", 82 "size": 1234 83 }, 84 "config": { 85 "mediaType": "application/vnd.oci.empty.v1+json", 86 "digest": "sha256:44136f...", 87 "size": 2 88 }, 89 "layers": [ 90 { 91 "mediaType": "application/spdx+json", 92 "digest": "sha256:def456...", 93 "size": 5678, 94 "annotations": { 95 "org.opencontainers.image.title": "sbom.spdx.json" 96 } 97 } 98 ], 99 "manifestBlob": { 100 "$type": "blob", 101 "ref": { "$link": "bafyrei..." }, 102 "mimeType": "application/vnd.oci.image.manifest.v1+json", 103 "size": 789 104 }, 105 "ownerDid": "did:plc:alice123", 106 "scannedAt": "2025-10-20T12:34:56.789Z", 107 "scannerVersion": "syft-v1.0.0", 108 "createdAt": "2025-10-20T12:34:56.789Z" 109} 110``` 111 112### Key Fields 113 114- `artifactType`: Distinguishes SBOM artifact from regular image manifest 115 - `application/spdx+json` for SPDX format 116 - `application/vnd.cyclonedx+json` for CycloneDX format 117- `subject`: Reference to the original image manifest 118- `ownerDid`: DID of the image owner (for multi-tenant holds) 119- `scannedAt`: ISO 8601 timestamp of when scan completed 120- `scannerVersion`: Tool version for reproducibility tracking 121 122### SBOM Blob 123 124The actual SBOM document is stored as a blob in the hold's storage backend and referenced in the manifest's `layers` array. The blob contains the full SPDX or CycloneDX JSON document. 125 126## Configuration 127 128SBOM scanning is configured via environment variables on the hold service. 129 130### Environment Variables 131 132```bash 133# Enable SBOM scanning (opt-in) 134HOLD_SBOM_ENABLED=true 135 136# Number of concurrent scan workers (default: 2) 137# Higher values = faster scanning, more CPU/memory usage 138HOLD_SBOM_WORKERS=4 139 140# SBOM output format (default: spdx-json) 141# Options: spdx-json, cyclonedx-json 142HOLD_SBOM_FORMAT=spdx-json 143 144# Future: Enable vulnerability scanning with Grype 145# HOLD_VULN_ENABLED=true 146``` 147 148### Example Configuration 149 150```bash 151# .env.hold 152HOLD_PUBLIC_URL=https://hold01.atcr.io 153STORAGE_DRIVER=s3 154S3_BUCKET=my-hold-blobs 155HOLD_OWNER=did:plc:xyz123 156HOLD_DATABASE_PATH=/var/lib/atcr/hold.db 157 158# Enable SBOM scanning 159HOLD_SBOM_ENABLED=true 160HOLD_SBOM_WORKERS=2 161HOLD_SBOM_FORMAT=spdx-json 162``` 163 164## Scanning Workflow 165 166### 1. Upload Completes 167 168When a container image is successfully pushed to a hold: 169 170``` 1711. Client: docker push atcr.io/alice/myapp:latest 1722. AppView routes blobs to hold service 1733. Hold receives multipart upload via XRPC 1744. Hold completes upload and stores blobs 1755. Hold checks: HOLD_SBOM_ENABLED=true? 1766. If yes: enqueue scan job (non-blocking) 1777. Upload completes immediately 178``` 179 180### 2. Background Scanning 181 182Scan workers process jobs from the queue: 183 184``` 1851. Worker pulls job from queue 1862. Extracts image layers from storage 1873. Runs Syft on extracted filesystem 1884. Generates SBOM in configured format 1895. Uploads SBOM blob to storage 1906. Creates ORAS manifest record in hold's PDS 1917. Job complete 192``` 193 194### 3. Result Storage 195 196SBOM results are stored in two places: 197 1981. **SBOM blob**: Full JSON document in hold's blob storage 1992. **ORAS manifest**: Metadata record in hold's embedded PDS 200 - Collection: `io.atcr.manifest` 201 - Record key: SBOM manifest digest 202 - Contains reference to subject image 203 204## Accessing SBOMs 205 206Multiple methods for discovering and retrieving SBOM data. 207 208### 1. XRPC Query Endpoint 209 210Query for SBOMs by image digest: 211 212```bash 213# Get SBOM for a specific image 214curl "https://hold01.atcr.io/xrpc/io.atcr.hold.getSBOM?\ 215 digest=sha256:abc123&\ 216 ownerDid=did:plc:alice123&\ 217 repository=alice/myapp" 218 219# Response: ORAS manifest JSON 220{ 221 "manifest": { 222 "schemaVersion": 2, 223 "mediaType": "application/vnd.oci.image.manifest.v1+json", 224 "artifactType": "application/spdx+json", 225 "subject": { "digest": "sha256:abc123...", ... }, 226 "layers": [ { "digest": "sha256:def456...", ... } ] 227 }, 228 "scannedAt": "2025-10-20T12:34:56.789Z", 229 "scannerVersion": "syft-v1.0.0" 230} 231``` 232 233### 2. ATProto Repository Queries 234 235Use standard ATProto XRPC to list all SBOMs: 236 237```bash 238# List all SBOM manifests in hold's PDS 239curl "https://hold01.atcr.io/xrpc/com.atproto.repo.listRecords?\ 240 repo=did:web:hold01.atcr.io&\ 241 collection=io.atcr.manifest" 242 243# Filter by artifactType (requires AppView indexing) 244# Returns all SBOM artifacts 245``` 246 247### 3. Direct SBOM Blob Download 248 249Download the full SBOM JSON file: 250 251```bash 252# Get SBOM blob CID from manifest layers[0].digest 253SBOM_DIGEST="sha256:def456..." 254 255# Request presigned download URL 256curl "https://hold01.atcr.io/xrpc/com.atproto.sync.getBlob?\ 257 did=did:web:hold01.atcr.io&\ 258 cid=$SBOM_DIGEST" 259 260# Response: presigned S3 URL or direct blob 261{ 262 "url": "https://s3.amazonaws.com/bucket/blob?signature=...", 263 "expiresAt": "2025-10-20T12:49:56Z" 264} 265 266# Download SBOM JSON 267curl "$URL" > sbom.spdx.json 268``` 269 270### 4. ORAS CLI Integration 271 272Use the ORAS CLI to discover and pull SBOMs: 273 274```bash 275# Discover referrers (SBOMs) for an image 276oras discover atcr.io/alice/myapp:latest 277 278# Output shows SBOM artifacts: 279# digest: sha256:abc123... 280# referrers: 281# - artifactType: application/spdx+json 282# digest: sha256:4a5e... 283 284# Pull SBOM artifact 285oras pull atcr.io/alice/myapp@sha256:4a5e... 286 287# Downloads sbom.spdx.json to current directory 288``` 289 290### 5. AppView Web UI (Future) 291 292Future enhancement: AppView web interface will display SBOM information on repository pages: 293 294- Link to SBOM JSON download 295- Vulnerability count (if Grype enabled) 296- Scanner version and scan timestamp 297- Comparison across multiple holds 298 299## Tool Integration 300 301### SPDX/CycloneDX Tools 302 303Any tool that understands SPDX or CycloneDX formats can consume the SBOMs: 304 305**Example tools:** 306- [OSV Scanner](https://github.com/google/osv-scanner) - Vulnerability scanning 307- [Grype](https://github.com/anchore/grype) - Vulnerability scanning 308- [Dependency-Track](https://dependencytrack.org/) - Software composition analysis 309- [SBOM Quality Score](https://github.com/eBay/sbom-scorecard) - SBOM completeness 310 311**Usage:** 312```bash 313# Download SBOM 314curl "https://hold01.atcr.io/xrpc/io.atcr.hold.getSBOM?..." | \ 315 jq -r '.manifest.layers[0].digest' | \ 316 # ... fetch blob ... > sbom.spdx.json 317 318# Scan with OSV 319osv-scanner --sbom sbom.spdx.json 320 321# Scan with Grype 322grype sbom:./sbom.spdx.json 323``` 324 325### OCI Registry API 326 327ORAS manifests are fully OCI-compliant and discoverable via standard registry APIs: 328 329```bash 330# Discover referrers for an image 331curl -H "Accept: application/vnd.oci.image.index.v1+json" \ 332 "https://atcr.io/v2/alice/myapp/referrers/sha256:abc123" 333 334# Returns referrers index with SBOM manifests 335{ 336 "schemaVersion": 2, 337 "mediaType": "application/vnd.oci.image.index.v1+json", 338 "manifests": [ 339 { 340 "mediaType": "application/vnd.oci.image.manifest.v1+json", 341 "digest": "sha256:4a5e...", 342 "artifactType": "application/spdx+json" 343 } 344 ] 345} 346``` 347 348### Programmatic Access 349 350Use the ATProto SDK to query SBOMs: 351 352```go 353import "github.com/bluesky-social/indigo/atproto" 354 355// List all SBOMs for a hold 356records, err := client.RepoListRecords(ctx, 357 "did:web:hold01.atcr.io", 358 "io.atcr.manifest", 359 100, // limit 360 "", // cursor 361) 362 363// Filter for SBOM artifacts 364for _, record := range records.Records { 365 manifest := record.Value.(ManifestRecord) 366 if manifest.ArtifactType == "application/spdx+json" { 367 // Process SBOM manifest 368 } 369} 370``` 371 372## Future Enhancements 373 374### Vulnerability Scanning (Grype) 375 376Add vulnerability scanning to SBOM generation: 377 378```bash 379# Configuration 380HOLD_VULN_ENABLED=true 381HOLD_VULN_DB_UPDATE_INTERVAL=24h 382 383# Extended manifest with vulnerability count 384{ 385 "artifactType": "application/spdx+json", 386 "annotations": { 387 "io.atcr.vuln.critical": "2", 388 "io.atcr.vuln.high": "15", 389 "io.atcr.vuln.medium": "42", 390 "io.atcr.vuln.low": "8", 391 "io.atcr.vuln.scannedWith": "grype-v0.74.0", 392 "io.atcr.vuln.dbVersion": "2025-10-20" 393 } 394} 395``` 396 397### Multi-Scanner Support (Trivy) 398 399Support multiple scanner backends: 400 401```bash 402HOLD_SBOM_SCANNER=trivy # syft (default), trivy, grype 403HOLD_TRIVY_SCAN_TYPE=os,library,config,secret 404``` 405 406### Multi-Hold Verification 407 408Compare SBOMs from different holds for the same image: 409 410```bash 411# Alice pushes to hold1 and hold2 412docker push atcr.io/alice/myapp:latest 413 414# Both holds scan independently 415# Compare results: 416atcr-cli compare-sboms \ 417 --image atcr.io/alice/myapp:latest \ 418 --holds hold1.atcr.io,hold2.atcr.io 419 420# Output: Package count differences, version mismatches, etc. 421``` 422 423### Signature Verification (Cosign) 424 425Sign SBOMs with Sigstore Cosign: 426 427```bash 428HOLD_SBOM_SIGN=true 429HOLD_COSIGN_KEY_PATH=/var/lib/atcr/cosign.key 430 431# SBOM artifacts get signed 432# Verification: 433cosign verify --key cosign.pub atcr.io/alice/myapp@sha256:4a5e... 434``` 435 436## Security Considerations 437 438### Reproducibility 439 440SBOMs should be reproducible for the same image digest: 441 442**Best practices:** 443- Pin scanner versions in production holds 444- Record scanner version in manifest annotations 445- Document vulnerability database versions 446- Re-scan periodically to catch new CVEs 447 448**Validation:** 449```bash 450# Compare SBOMs from different holds 451diff <(curl hold1/sbom.json | jq -S) \ 452 <(curl hold2/sbom.json | jq -S) 453 454# Differences indicate: 455# - Different scanner versions 456# - Different scan times (new CVEs discovered) 457# - Potential tampering (investigate) 458``` 459 460### Multiple Hold Verification 461 462Running multiple holds provides defense in depth: 463 4641. User pushes to hold1 (uses hold1 by default) 4652. User also pushes to hold2 (backup/verification) 4663. Both holds scan independently 4674. Compare SBOM results: 468 - Similar results = confidence in accuracy 469 - Divergent results = investigate discrepancy 470 471### Transparency 472 473Hold operators should publish scanning policies: 474 475- Scanner version and update schedule 476- Vulnerability database update frequency 477- SBOM format and schema version 478- Data retention policies 479 480### Trust Anchors 481 482Users can verify scanner integrity: 483 4841. **Scanner version**: Check `scannerVersion` field matches expected version 4852. **DID signature**: ATProto record signed by hold's DID 4863. **Timestamp**: Check `scannedAt` for stale scans 4874. **Reproducibility**: Re-scan locally and compare results 488 489## Example Workflows 490 491### Enable Scanning on Your Hold 492 493```bash 494# 1. Configure hold with SBOM enabled 495cat > .env.hold <<EOF 496HOLD_PUBLIC_URL=https://myhold.example.com 497STORAGE_DRIVER=s3 498S3_BUCKET=my-blobs 499HOLD_OWNER=did:plc:myid 500 501# Enable SBOM scanning 502HOLD_SBOM_ENABLED=true 503HOLD_SBOM_WORKERS=2 504HOLD_SBOM_FORMAT=spdx-json 505EOF 506 507# 2. Start hold service 508./bin/atcr-hold 509 510# 3. Push an image 511docker push atcr.io/alice/myapp:latest 512 513# 4. Wait for background scan (check logs) 514# 2025-10-20T12:34:56Z INFO Scanning image sha256:abc123... 515# 2025-10-20T12:35:12Z INFO SBOM generated sha256:def456... 516 517# 5. Query for SBOM 518curl "https://myhold.example.com/xrpc/io.atcr.hold.getSBOM?..." 519``` 520 521### Consume SBOMs in CI/CD 522 523```yaml 524# .github/workflows/security-scan.yml 525name: Security Scan 526on: push 527 528jobs: 529 scan: 530 runs-on: ubuntu-latest 531 steps: 532 - name: Pull image 533 run: docker pull atcr.io/alice/myapp:latest 534 535 - name: Get SBOM from hold 536 run: | 537 IMAGE_DIGEST=$(docker inspect atcr.io/alice/myapp:latest \ 538 --format='{{.RepoDigests}}') 539 540 curl "https://hold01.atcr.io/xrpc/io.atcr.hold.getSBOM?\ 541 digest=$IMAGE_DIGEST&\ 542 ownerDid=did:plc:alice123&\ 543 repository=alice/myapp" \ 544 -o sbom-manifest.json 545 546 SBOM_DIGEST=$(jq -r '.manifest.layers[0].digest' sbom-manifest.json) 547 548 curl "https://hold01.atcr.io/xrpc/com.atproto.sync.getBlob?\ 549 did=did:web:hold01.atcr.io&\ 550 cid=$SBOM_DIGEST" \ 551 | jq -r '.url' | xargs curl -o sbom.spdx.json 552 553 - name: Scan with Grype 554 uses: anchore/scan-action@v3 555 with: 556 sbom: sbom.spdx.json 557 fail-build: true 558 severity-cutoff: high 559``` 560 561## References 562 563- [ORAS Specification](https://oras.land/) 564- [OCI Artifacts](https://github.com/opencontainers/artifacts) 565- [SPDX Specification](https://spdx.dev/) 566- [CycloneDX Specification](https://cyclonedx.org/) 567- [Syft Documentation](https://github.com/anchore/syft) 568- [ATProto Specification](https://atproto.com/)