Make every green-check PR mergeable

[mbauman]

This is a bit of a brainstorming issue to improve the sustainability of this project. Up til now, I've been checking a few things in every PR, and we can automate many of them:

• Double-check that the matched projects make sense. This isn't automatable, but isn't too hard and it's pretty rare (and often obvious) that it goes wrong. An example where it did is Manually import advisories for glib-networking #189.
• Ensure GeneralMetadata doesn't have a version of "*" on its oldest or newest registered version. This is fully automatable, and probably one of the bigger improvements we could make.
• Ensure the versions themselves "make sense". Most commonly the problem here is that the version extraction finds a date or commit sha instead of a version number (in either the advisory or the download URL). This is more heuristic-y, but we could have the bot report information about Levenshtein distances or some such, even if it's not directly tied to a binary CI ✅. Detect and do something smart with git SHA ranges and dates #89 and Check Levenshtein distance between version numbers for upstream projects #308
• Implement a mechanism to ignore/dismiss/reject upstream advisories #171
• Only propose updates to advisories when the upstream advisory has changed #170
• Automatically bound version ranges when a new non-vulnerable version is released #178

[mbauman]

One key desiderata is that I want to be able to know my knowns and my unknowns and make them clearer:

• If a package repository supports advisories
• If a package has artifacts
• If those artifacts point directly to a known upstream project
• If those artifacts were built with and redistribute something from a known upstream project, and I know how they were built
• Or, that I know I don't know what those artifacts are

So, turning that on its head, there's effectively a six-state here:

• No artifacts (just check the package repository for advisories and that's that)
• Has artifacts that directly match a known Repology project
• Has artifacts built from a known/trusted build-system with all sources matched against known Repology projects
• Has artifacts built from a known/trusted build-system with some matched sources and some unmatched
• Has untrusted artifacts with all sources matched against known Repology projects
• Has untrusted artifacts with some matched sources and some unmatched
• Has unknown artifacts

Or, more succinctly, there are four orthogonal characteristics:

• First-class repository advisories or not
• Artifacts or not
• Artifacts that are fully matched or partially matched or not matched at all
• Trusted build system or not

[mbauman]

OK, all these unknowns are now largely "known unknowns" and their state is exposed by GeneralMetadata. I think we should detangle alias searches from upstream searches, and it'd be really great if we could further refactor the machinery to constrain its pull requests to be one-per-package.

So we'd have the following automations:

• Alias searches
  • Repository GHSA fetch
  • NVD and EUVD plaintext PackageName.jl searches
• Upstream searches
  • Upstream repository GHSA fetch
  • Upstream CPE and vendor/part search

All of these are expensive and it's all per-package sorts of APIs, so that's why we'd want to track the search history on a per-package basis. Doing that allows us to iterate until we find a hit/update, create the PR, and bail out... to resume on the next scheduled call.

This would, then, give up on the "recently published" NVD/EUVD searches... which makes me a little sad because I worked so hard to make that work, but I think it'll make life so much easier.

And, actually, we should probably punt the "tracked" state entirely to GeneralMetadata.jl. Maybe SecurityAdvisories doesn't care at all if things are well-tracked. It'll just search them if they are, and you ask GeneralMetadata if things are in good enough shape.

[mbauman]

Oh interesting; one consequence of how I retrofit GeneralMetadata here (#290) is that we've drastically reduced the number of invented "*" upstream versions that we use in considering advisory applicability.

Previously, if any package had an upstream project at any of its versions, all versions were assumed to have it with an unknown "*" version. And when the oldest version (the least likely to have good data) had a "*" version, we'd consider all advisories. This is what led to disastrous imports where we pulled in dozens of old upstream advisories dating back to the year 2000.

Now, we only invent "*"s in cases where we've partially tracked the upstreams — cases where we have a known project but no version. This is most common with arbitrary commits against Git repos... and as long as it's not the lower bound we handle it slightly more gracefully.

This is keeping with the mantra of "doing due diligence for packages and projects that have done theirs..." and it's beautifully asymmetric between missing low versions and missing high versions. See, for example libpng_jll, which currently only has a few versions "in the middle" tracked: https://github.com/JuliaRegistries/GeneralMetadata.jl/blob/6467ba6ec9b2e7b30c8d4f12f82bd7579275a8c6/metadata/L/libpng_jll.toml. This led to an import of only a handful of issues that are present in those versions (#293) albeit with overzealous bounds.