Document matching algorithm interface

[tschmidtb51]

We need to document the matching algorithm interface for CSAF asset matching systems and CSAF SBOM matching systems. Both work similar:

Input

match(product_tree, asset_database_connection, matching_threshold)

 - resp. - 
 
match(product_tree, sbom_database_connection, matching_threshold)

Output

for each product_id in product_tree:
   a list of tuples (asset_id, probability, matching_reason)

- resp -

for each product_id in product_tree:
   a list of tuples (sbom_component_id, probability, matching_reason)

Note 1: The invoking function needs to store the context - as product ids are document local in a CSAF document.
Note 2: This covers only the one CSAF document to many/one assets/SBOMs. Please note that view of many matched CSAF documents to one assets/SBOMs is also a part of the requirements. It should be easy to generate that view once the matching is done - just by selecting the corresponding query criteria over all matches.

Matching

Algorithm - priorities:

1. Match based on product_identification_helper. Different ones might imply a different confidence: An sbom_url or serial_number might be stronger than a cpe.
2. Match based on the categorized strings (value of name) in the branches (e. g. vendor, product_name, product_version).
3. Match on the human-readable full_product_name_t/name.

The algorithm may end after it created a sufficient result - it can, but does not have to go through all steps.

Edit: The experience shows, we also want to provide a matching_threshold that allows us to fine tune what the lowest probability is that we get results for (a matching_threshold of 0 would give for each asset/SBOM component the probability that it matched with (which might be 0 if those are completely different)) and the matching_reason which provides insights into the confidence and helps debugging (a direct match on a serial number would potentially better than a match on the human-readable string).

[oxisto]

Current version can be found in #953

[oxisto]

I have tried to put the work that we have done in kotlin-csaf to implement an (initial) matching between CSAF documents and SBOM documents specified as protobom in the above post. Please see this is a starting point of the discussion. I have some open questions/issues such as

• I want to merge cpe, purl, etc. into a generic software identifier property
• What other properties should we consider? We have many more information specified in CSAF but they might lack a counterpart in SBOM formats
• Specific for PURL: should we consider the "namespace" as a candidate for the vendor?
• Currently, we regard all property sources as the same and then try to account for that in the matching (using the DifferentSources confidence). But it might also make sense to assign a confidence to the "extraction" of a property, e.g., in the above example extracting a "vendor" from the PURL "namespace" might be less confident than extracting a "vendor" from the CSAF branch category value
• Where should we put such a specification? Would it be part of the standard at some point or probably an accompanying document? I think such as specification would REALLY benefit anyone out there who tries to match CSAF advisories to some sort of SBOM and/or asset system
• Currently, this is all rather static, the specification would need more adaptation that certain things MUST be done but still allow for users to do some custom things like matching based on AI / rules, etc.

[tschmidtb51]

@oxisto: I'm not sure how easy it is to follow - please see some remarks from my side below:

• advisories: The CSAF security advisories to match against.

We should talk about CSAF documents here as a matcher should work independent of the profile.

Matches the advisories against one component in the SBOM and returns a Match
[...]
The result of a matching operation. It contains the matched SBOMs and their corresponding CSAF document.

It is correct that the CSAF documents provides the context. However, please be aware, that the matching happens on the product_tree.

• vulnerableProduct: The CSAF product that generated the match

This is incorrect. Not all matches refer to vulnerable products. They could have any product status.

• affectedNode: The node (component) that is affected by this advisory

As affected has a special meaning, I would suggest to use a different wording.

[oxisto]

@oxisto: I'm not sure how easy it is to follow - please see some remarks from my side below:

• advisories: The CSAF security advisories to match against.

We should talk about CSAF documents here as a matcher should work independent of the profile.

Good point. I was trying to avoid "document" as in protoBOM the SBOM is also defined as "document", but we can just avoid the protobom terminology.

However, in the conformance target (of CSAF 2.0), we also alternate between "advisories" and "document", so we should probably avoid that terminology in the conformance target as well.

Matches the advisories against one component in the SBOM and returns a Match
[...]
The result of a matching operation. It contains the matched SBOMs and their corresponding CSAF document.

It is correct that the CSAF documents provides the context. However, please be aware, that the matching happens on the product_tree.

• vulnerableProduct: The CSAF product that generated the match

This is incorrect. Not all matches refer to vulnerable products. They could have any product status.

True, I will change that to just product.

• affectedNode: The node (component) that is affected by this advisory

As affected has a special meaning, I would suggest to use a different wording.

-> matchedComponent

[tschmidtb51]

We should talk about CSAF documents here as a matcher should work independent of the profile.

However, in the conformance target (of CSAF 2.0), we also alternate between "advisories" and "document", so we should probably avoid that terminology in the conformance target as well.

Sorry - my phrasing seems to be misleading - I was aiming at the CSAF security advisories with the profile comment... When I talk about CSAF documents, we talk about a dedicated instance, a CSAF advisory is characterized by is publisher namespace and tracking id (and therefore usually refers to the latest available instance in the context). The latter is more or less the set of CSAF documents in different versions.

I hope that was a little clearer - and not more confusing.

[tschmidtb51]

Some additional comments:

Indicates that the node is definitely not affected by the advisory.

As stated above: we have to consider all product status - not just affected.

Confidence Levels

Sounds okay to me. I'm not sure yet, whether those values will always result in the best match, but I guess we can start with them.

Maybe, define them in one place to be sure that we can easily update them if needed.

The value of the property is a string containing the vendor of the vulnerable product.

Please update all phrases that indicate the matching would only work for vulnerable/affected.

The value of the property is a string containing the product name of the vulnerable product.

How do we handle product_family? The string that is in there might be considered a part of the product name in CPE/PURL or the SBOM data...

Note: the cpe and purl properties might be merged into a more generalized property in the future and extended to other software identifiers.

As they (may) come from different sources and have different challenges, I currently don't think that combining them is a good idea..

• First, the properties cpe and purl are matched against each other.

I guess, I would value purl over CPE due to the data quality currently...

If it is greater than the threshold, a Match structure is returned containing the CSAF advisory, the vulnerable product that matched, the SBOM node and the confidence value.

See comment about context above. Also, there should be a simple, interchangeable function that implements the interface to be able to test different approaches.

---

Overall, the idea sounds reasonable. Please make sure to:

1. Address the product status topic
2. Clearly state that this is only for SBOMs
3. Incorporate the missing PIH
4. Update the comment above

[tschmidtb51]

I have tried to put the work that we have done in kotlin-csaf to implement an (initial) matching between CSAF documents and SBOM documents specified as protobom in the above post. Please see this is a starting point of the discussion. I have some open questions/issues such as

* I want to merge cpe, purl, etc. into a generic software identifier property

I think that is a bad idea: Both come from different backgrounds and have different confidence level. In purl, I guess you can be almost certain that the package name given in a purl is the correct component name (as this is enforced by the package manager).

* What other properties should we consider? We have many more information specified in CSAF but they might lack a counterpart in SBOM formats

* Specific for PURL: should we consider the "namespace" as a candidate for the vendor?

Maybe as candidate, but I wouldn't argue that there need to match.

* Currently, we regard all property sources as the same and then try to account for that in the matching (using the `DifferentSources` confidence). But it might also make sense to assign a confidence to the "extraction" of a property, e.g., in the above example extracting a "vendor" from the PURL "namespace" might be less confident than extracting a "vendor" from the CSAF branch category value

Sound like an idea that should be done some research on.

* Where should we put such a specification? Would it be part of the standard at some point or probably an accompanying document? I think such as specification would REALLY benefit anyone out there who tries to match CSAF advisories to some sort of SBOM and/or asset system

I don't believe that we are at the point, where this should be in den specification. However, I guess we can either do that as committee note or just a small webpage.

* Currently, this is all rather static, the specification would need more adaptation that certain things MUST be done but still allow for users to do some custom things like matching based on AI / rules, etc.

Yes.

[oxisto]

See #953