The SBOM Quality Problem: Existence ≠ Usefulness
Generating an SBOM has become trivial. A single command with Syft, Trivy or a comparable tool is enough to produce a JSON file that formally qualifies as a Software Bill of Materials. That is precisely the problem.
In practice, many of these SBOMs look alarmingly similar: hundreds of components without version numbers, missing license fields, no Package URLs (PURLs), no information on dependency relationships. In a real audit case, an SBOM from a mid-sized software vendor contained 312 components – 67 % of which had no valid PURL, 41 % had no license information, and transitive dependencies were missing entirely. The document existed, but was practically worthless for vulnerability management, legal review or supply chain analysis.
This pattern is not an isolated case. As long as SBOM generation is understood as a checkbox activity – generated for compliance, filed in the repository, never touched again – the real value creation fails to materialise. The first step toward improvement is making quality measurable.
Quality Dimensions: What Makes an SBOM Truly Good?
As a conceptual framework, five quality dimensions have become established that can be derived from the NTIA minimum requirements, the CISA recommendations and the German BSI Technical Guideline TR-03183-2:
Completeness describes whether all relevant components – including transitive dependencies – are captured. An SBOM that only contains direct dependencies often represents less than 30 % of the packages actually shipped in modern software stacks.
Accuracy asks whether the information contained is correct: do versions, hashes and identifiers match the artefact actually used?
Depth concerns the richness of metadata per component – license, copyright, supplier, hash values, PURL, CPE.
Freshness addresses the age of the SBOM: Does it correspond to the current build state, or was it last generated six months ago?
Machine-Readability, finally, describes whether the document is available in a standardised format and can be processed automatically by tools – without manual interpretation.
These five dimensions form the basis for concrete, verifiable metrics.
Concrete Metrics and Thresholds: Making SBOM Quality Measurable
For each dimension, metrics can be defined that are operationalisable in audits and automated pipelines:
Completeness
- Transitive Dependency Coverage Ratio: Share of captured transitive dependencies measured against the actually installed package set (e.g. via lock-file reconciliation).
- ✅ Good: > 90 % | ⚠️ Acceptable: 70–90 % | ❌ Insufficient: < 70 %
Accuracy
- PURL Validity Rate: Share of components with a syntactically valid and resolvable Package URL.
- ✅ Good: > 95 % | ⚠️ Acceptable: 80–95 % | ❌ Insufficient: < 80 %
- Hash Presence Rate: Share of components with at least one cryptographic hash (SHA-256 or stronger).
- ✅ Good: > 90 % | ⚠️ Acceptable: 75–90 % | ❌ Insufficient: < 75 %
Depth
- License Field Completeness: Share of components with a valid SPDX-compliant license statement.
- ✅ Good: > 95 % | ⚠️ Acceptable: 80–95 % | ❌ Insufficient: < 80 %
- Supplier Completeness: Share of components with supplier or author information.
- ✅ Good: > 85 % | ⚠️ Acceptable: 60–85 % | ❌ Insufficient: < 60 %
Freshness
- SBOM Age: Time since the last build reference of the SBOM.
- ✅ Good: < 7 days | ⚠️ Acceptable: 7–30 days | ❌ Insufficient: > 30 days
Machine-Readability
- Format Compliance: Validated against the official schema (CycloneDX JSON Schema, SPDX 2.3 Spec).
- ✅ Good: 0 validation errors | ❌ Insufficient: ≥ 1 error
These thresholds are not absolute truths, but they serve well as a starting point for internal SLAs and supplier requirements. It should also be noted at this point that compliance does not recognise proportional assessment. There, the binary 0 or 1 principle applies: either compliant or not. And that only with certainty, given complete information.
Format Matters: CycloneDX vs. SPDX from a Quality Perspective
Both standards – CycloneDX and SPDX – are established and recognised in regulatory requirements. For quality assurance, however, structural differences are relevant.
SPDX originally emerged from the licensing use case and offers very mature semantics here: SPDX license expressions, snippets, copyright texts. For legal compliance workflows, SPDX remains strong. The format is weaker when it comes to vulnerability metadata, build context and the native representation of complex dependency graphs.
CycloneDX was designed from the ground up as a security SBOM. It natively supports Vulnerability Disclosure Reports (VEX), Services, Hardware Components, Cryptographic Assets and Formulas (build reproducibility). This makes it the better basis for quality metrics that go beyond license compliance.
Personal assessment: For most modern use cases – particularly vulnerability management, CRA compliance and supply chain audits – CycloneDX is the leaner, more expressive format. SPDX remains the first choice when the primary requirement is legal compliance and license clearance. When in doubt: provide both formats in parallel, which most modern tools support.
Tooling for SBOM Quality Assessment: Automated Scoring in Practice
Without tooling, every quality metric remains academic. The following tools are relevant in practice:
sbom-scorecard (Adolfo García Veytia / Chainguard): Evaluates SBOMs against the NTIA minimum requirements and returns a numeric score. Well integrable into CI/CD, but the metrics are relatively coarse and not configurable.
SBOM Quality Score (Interlynk): Cloud-based tool with more differentiated metrics and dashboard functionality. Its strength lies in visualisation and trend tracking. Blind spot: proprietary platform, data sensitivity for commercial SBOMs must be evaluated.
ts-sbom-checker (TrustSource, Open Source): The tool available at github.com/trustsource/ts-sbom-checker specifically checks conformity with BSI TR-03183-2 – making it directly relevant for European compliance requirements. It is CLI-based, lightweight and can be integrated into build pipelines without external dependencies. Recommended for all teams focused on the German or European market.
For CI/CD integration: at minimum, PURL Validity and Schema Compliance should be implemented as hard gate criteria. Freshness checks can be automated via build timestamps. A combination of ts-sbom-checker for regulatory conformity and sbom-scorecard for the NTIA baseline covers most requirements.
Audit-Ready SBOM: A Threshold-Based Checklist
The following criteria are suitable as a pass/fail framework for internal audits, supplier audits and regulatory requirements (CRA, Executive Order 14028):
| Criterion | Minimum Requirement | Recommended |
|---|---|---|
| Format validation | 0 schema errors | CycloneDX 1.6+ or SPDX 2.3+ |
| PURL Validity Rate | > 80 % | > 95 % |
| License Completeness | > 80 % | > 95 % |
| Hash Presence | > 75 % | > 90 % |
| Transitive Coverage | > 70 % | > 90 % |
| SBOM age | < 30 days | < 7 days (per release) |
| TR-03183-2 conformity | Mandatory (DE/EU) | ts-sbom-checker: 0 errors |
Beyond this baseline, a high-quality SBOM opens up further dimensions of use: vulnerability management through PURL-based matching against OSV, NVD or EUVD, crypto agility through capturing cryptographic dependencies (CycloneDX supports this natively), and legal compliance through complete license metadata for open-source obligations.
From Compliance Theatre to Real Usefulness: Next Steps
The transition from formal SBOM generation to real quality assurance is not a tooling problem – it is a process and culture problem. Process and mindset must be established accordingly within organisations. In addition to technical implementation, organisational anchoring must also be created. A possible path could look like this:
Short term (0–4 weeks): Scan existing SBOMs with sbom-scorecard or ts-sbom-checker, collect baseline metrics, identify the most critical gaps (typically missing PURLs and licenses).
Medium term (1–3 months): Embed SBOM generation in CI/CD pipelines, set up automatic quality gates, clearly assign responsibility for SBOM quality within the team.
Long term: Understand SBOM quality as a continuous process – with regular review on dependency changes, inclusion in supplier contracts and linkage with the vulnerability management process. Gradually raise quality targets.
Organisationally, two things are needed: a person who takes ownership of SBOM quality, and a clear internal quality target that goes beyond “SBOM exists”. In our experience, it also makes sense – where different teams are involved – to offer an incentive for the first team to reach the defined quality targets.
Three key takeaways:
- An SBOM without measurable quality metrics is not an asset but compliance theatre – PURL Validity Rate, License Completeness and Transitive Coverage are the decisive indicators.
- CycloneDX is the structurally superior basis for security and supply chain use cases; for legal compliance workflows, SPDX remains strong.
- Tools such as ts-sbom-checker (TR-03183-2) and sbom-scorecard enable automated scoring in CI/CD – the bottleneck is not tooling, but organisational commitment to continuous quality assurance.