-
Notifications
You must be signed in to change notification settings - Fork 16
Add sbom generation tooling (#2232) #106
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Add sbom generation tooling (#2232) #106
Conversation
|
@Lukasz-Juranek this looks interesting! Can you describe a little why we need custom code? No native bazel support etc. |
|
Hi right now i'm not aware of any out of box support for bazel that would really cover all c++ imports and rust code. But IMO tooling itself is not important that much, this can be replaced later on with anything What is important is to start getting the SBOM data for 3rd party dependencies in any meaningful format when there is not much deps, and to build in SCORE developers this behavior that when you import some stuff to your project you think about SBOM. If SBOM data will be available then you can do conversion to some mature solution. |
ffca1f1 to
e308870
Compare
|
Updated PR according discussion that we had https://github.com/orgs/eclipse-score/discussions/2226#discussioncomment-15669973 Now manual generation of is removed , sbom data is generated via
Added mandatory fields from
For details see updated readme https://github.com/Lukasz-Juranek/score-tooling/blob/8e9cb957c32b7b2da9b4faf607be172ba9b14a6f/sbom/SBOM_Readme.md @masc2023 you asked for list of tools here it is https://github.com/Lukasz-Juranek/score-tooling/blob/8e9cb957c32b7b2da9b4faf607be172ba9b14a6f/sbom/SBOM_Readme.md#core-tools |
cad2c59 to
8e9cb95
Compare
9382341 to
c8d2a39
Compare
sbom/SBOM_Readme.md
Outdated
| | [@cyclonedx/cdxgen](https://github.com/CycloneDX/cdxgen) | C++ dependency scanner and license discovery tool | C++ metadata extraction when `auto_cdxgen = True` | | ||
| | [Node.js / npm](https://nodejs.org) | Runtime for cdxgen | C++ metadata extraction when `auto_cdxgen = True` | | ||
|
|
||
| ### Five-Phase Architecture |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Which of these actually depend on bazel? Can we use this tool without bazel?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@AlexanderLanin thx for review.
cyclonedx/cdxgen you can use without bazel, both NPM and cdxgen are currently required on system to be installed on system they are not deployed via bazel.
Bazel is only needed to collect dependencies that are needed to build the target,
This data is then hold in sbom_aspect then tool is matching data from this structure to data from cdxgen and bazel lockfile
But if you have all data that is needed in dash-license-scan then we should use this as source of data, not cdxgen and bazel lockfile
Maybe lets make some short sync on this just ping me on slack when you will have time
For think that we can't use only the dash-licensese-scan but we can extract license data from it
| Field | crates_metadata.json provides | dash-license-scan provides |
|---|---|---|
| version | Yes | Yes (from Cargo.lock parse) |
| license | Yes (from crates.io API) | Yes (from Eclipse/ClearlyDefined) |
| checksum (SHA-256) | Yes | No |
| purl | Yes (pkg:cargo/name@ver) | No (uses crate/cratesio/-/ format) |
| repository | Yes (from crates.io API) | No |
| #!/usr/bin/env python3 | ||
| """Generate crates.io metadata cache for SBOM generation. | ||
|
|
||
| This script parses Cargo.lock files and/or MODULE.bazel.lock files, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cargo.lock parsing has also been implemented in dash-license-scan. Probably needs alignment.
| lockfile_path: Path to MODULE.bazel.lock file | ||
|
|
||
| Returns: | ||
| Dict mapping crate name to {version, checksum, source} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how do you know which version to use? Thats the main reason I have not implemented bazel lockfile parsing in dash-license-scan, since there is no way to know which version is actually relevant?!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no i was not aware of this problem
| @@ -0,0 +1,142 @@ | |||
| """Tests for CycloneDX 1.6 formatter.""" | |||
|
|
|||
| import unittest | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
would you consider pytest? I don't know if its better or worse. But we use pytest everywhere.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok i will change framework
sbom/crates_licenses_common.json
Outdated
| @@ -0,0 +1,78 @@ | |||
| { | |||
| "serde": {"license": "MIT OR Apache-2.0"}, | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note that different versions can have different licenses
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this file is not used ,
The actual license data now comes from crates_metadata.json (generated by the cache script from crates.io API + MODULE.bazel.lock).
i need to clean this trash
| targets, traversing their transitive dependencies and generating output | ||
| in SPDX 2.3 and/or CycloneDX 1.6 format. | ||
|
|
||
| License metadata is collected automatically: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is where dash-license-scan could somehow fit it. It collects verified license information.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i will check it if we can match get license field from dash output makes sense.
sbom/SBOM_Readme.md
Outdated
| # bazel_dep module (version from module graph) | ||
| sbom_ext.license( | ||
| name = "googletest", | ||
| license = "BSD-3-Clause", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we should use dash-licene-scan and not write hand collected license infos
e208237 to
3c0af62
Compare
3c0af62 to
35c92b1
Compare
|
@AlexanderLanin i've updated SBOM tooling to get license from dash you can take a look at some generated files sbom_communication.cdx.json Two things to confirm
|
This PR adds basic sbom bazel rules for details see
https://github.com/Lukasz-Juranek/score-tooling/blob/8e9cb957c32b7b2da9b4faf607be172ba9b14a6f/sbom/SBOM_Readme.md
Here are generated SBOMs for reference_integration targets:
sbom_kyron.cdx.json
sbom_kyron.spdx.json
sbom_orch_per.cdx.json
sbom_orch_per.spdx.json
Here is SBOM target definition: eclipse-score/reference_integration#83
Sbom is automatically generated no modiffications are needed for SCORE
Depending-on: eclipse-score/score-crates#24
This is not full solution but rather sth to begin with.
Happy to discuss this in details.