-
Notifications
You must be signed in to change notification settings - Fork 49
tests: always make sure we return an LLB state #909
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
afbcce1 to
6501e6a
Compare
Signed-off-by: Brian Goff <[email protected]>
5cfdf4e to
6f64bda
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR refactors the test execution flow to ensure that an LLB state is always returned, even when tests fail. This enables proper debugging support with docker buildx dap by ensuring breakpoints can attach to the state. Instead of returning errors directly to BuildKit when tests fail, the PR adds a new LLB exec operation that prints test errors to stderr and exits non-zero, maintaining the build failure behavior while preserving the state chain.
Key changes:
- Modified
RunTeststo returnllb.StateOptioninstead of(Reference, error), allowing tests to be composed into the state chain - Introduced
withTestErrorto create an error-reporting state that outputs test failures before exiting - Added
TestErrorCmdas a new frontend command to handle test error output
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| test/linux_target_test.go | Updates test to consume logs via SolveStatus callback and verify test errors appear in output rather than as direct function errors |
| targets/linux/rpm/distro/pkg.go | Changes RunTests signature from returning (Reference, error) to StateOption, composing test dependencies and execution into the state chain |
| targets/linux/deb/distro/pkg.go | Matches RPM implementation with same RunTests signature change to StateOption pattern |
| targets/linux/distro_handler.go | Updates handlers to use new StateOption-based test execution and adds getRef helper to simplify state marshalling and solving |
| frontend/test_runner.go | Core refactoring: RunTests now returns StateOption, adds withTestError for error reporting, TestErrorCmd for outputting errors, and evalState helper |
| cmd/frontend/main.go | Wires up TestErrorCmd in the frontend command router |
| .github/workflows/ci.yml | Adds Docker system prune before tests to clean up resources |
Before this we are returning errors to buildkit when there are test
errors.
Or for that matter, if there is an error during evaluation of the stuff
running up to the tests.
This causes issues with the debug workflow ("docker buildx dap") since
there is no LLB state to attach to, so it can't set breakpoints
properly.
With this change even when we get a solve error when evaluating state,
we return the bad state so that the debugger can attach to it.
When a test fails, we add a new LLB exec op which prints the joined
test errors and exits non-zero so that the build will fail.
There's also some minor changes to make it simpler to discard any
changes to the filesystem from running tests, or in the case of
non-container outputs return the original state, while keeping the
dependency chain in tact.
Signed-off-by: Brian Goff <[email protected]>
Before this we are returning errors to buildkit when there are test errors.
Or for that matter, if there is an error during evaluation of the stuff running up to the tests.
This causes issues with the debug workflow ("docker buildx dap") since there is no LLB state to attach to, so it can't set breakpoints properly.
With this change even when we get a solve error when evaluating state, we return the bad state so that the debugger can attach to it. When a test fails, we add a new LLB exec op which prints the joined test errors and exits non-zero so that the build will fail.
There's also some minor changes to make it simpler to discard any changes to the filesystem from running tests, or in the case of non-container outputs return the original state, while keeping the dependency chain in tact.
This is a simpler alternative to #898.
#898 is hitting some race condition in buildkit (or at least that's what it looks like).
Additionally this can actually be backported to older release branches as the changes are minor.