-
-
Notifications
You must be signed in to change notification settings - Fork 17.6k
workflows: cleanup & refactor checkouts #410791
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
In PRs with multiple commits and merge conflicts the logic "targetSha == immediate parent of mergedSha" doesn't hold anymore. The head and base commits of the PR's branch have some commits inbetween them, instead. Before this change, we'd get a "fatal: invalid reference" on the "worktree add". Now, not anymore, because we fetch the right commit directly.
Using core.setOutput is much nicer than having to parse the json "result" on the outside. This also avoids some very odd errors, when the result can, for unknown reasons, *not* be parsed as JSON later on. Also avoiding a bit of duplication between the "if mergeable" branches.
By consistently checking out nixpkgs into the same location in every workflow, it's easier to reason about the different workflows at once. We also use crystal-clear names to make clear, which checkouts are considered trusted, because they only contain target-branch-code and which checkouts are untrusted, because they contain code from the head branch. By naming the checkout directories trusted/untrusted, it's obvious at the call-site. One example of where we likely did the wrong thing is the nixpkgs-vet workflow: Fetching the toolVersion from the untrusted checkout opens the door for an injection into the download URL, thus code could be downloaded from anywhere. This is not a problem, because this workflow does not run with elevated privileges, but it's a scary oversight nonetheless.
We have added nixpkgs-vet as a regular package to nixpkgs a while ago, so we can now use it from pinned nixpkgs. This avoids pulling a platform-specific binary version from upstream. This change also allows to run the tool easily locally, the same way as other tools: nix-build ci -A nixpkgs-vet This will do a full check of the repo with the exception of nixpkgs-vet's "ratchet" checks: Those depend on having two branches to compare, but the default is to only look at the head branch. Those ratchet checks will still be run in CI, though.
cd5349a to
228170a
Compare
This makes checking out the nixpkgs repo even more consistent and almost forces us to use the trusted/untrusted path pattern.
228170a to
0e1c284
Compare
|
Successfully created backport PR for |
|
@wolfgangwalther why do you actually need a .git? Could you not get rid of this before running nix instead of whatever breaks with the current shallow clone worktree? Hopefully the fix in nix is soon there, but just in case. |
|
When using worktrees, two things break:
So this does depend on the git repo via It only makes a small difference of a few seconds, so I wouldn't want to introduce more workarounds for it. Once this is fixed in Nix, we can take advantage, though. But it's not blocking anything else either. |
This actually sets
|
Ah, OK. I was about to test that again, because I thought I actually tried adding |
The core idea here is to make it harder to accidentally do bad things when checking out nixpkgs in CI. The most obvious thing is to avoid running code from the untrusted PR branch, of course. Just using the terms "merge commit" and "target commit" is not really telling that story, though. So, I renamed the checkout locations to
untrusted/andtrusted/instead. This means you'll have to explicitly write either "trusted" or "untrusted" when referring to any of them, which makes it mostly obvious when executing untrusted code. To encourage this pattern, I'm also moving the the checkouts intoactions/get-merge-commit. You'll have to explicitly do something very different to break it now.Found some inconsistencies, too:
actions/get-merge-committo usecore.setOutput.nixpkgs-vetworkflow... actually executes untrusted code, because it creates a download URL from the pinned version which is taken from the head branch... It doesn't have access to any secrets, so I don't think this is critical. By moving to the pinned nixpkgs instance,provided by the trusted target commitwhich is run in the nix sandbox, this is avoided. Also a regularnix-build ciwill now include nixpkgs-vet, too, albeit only running the basic checks, not the "ratchet" checks.git worktree, but hit a Nix bug. Slightly annoying to debug, because thelixI am using locally works just fine with worktrees, so that was confusing.Things done
Add a 👍 reaction to pull requests you find important.