Skip to content

Conversation

@cpuguy83
Copy link
Collaborator

Before this we are returning errors to buildkit when there are test errors.
Or for that matter, if there is an error during evaluation of the stuff running up to the tests.
This causes issues with the debug workflow ("docker buildx dap") since there is no LLB state to attach to, so it can't set breakpoints properly.

With this change even when we get a solve error when evaluating state, we return the bad state so that the debugger can attach to it. When a test fails, we add a new LLB exec op which prints the joined test errors and exits non-zero so that the build will fail.

There's also some minor changes to make it simpler to discard any changes to the filesystem from running tests, or in the case of non-container outputs return the original state, while keeping the dependency chain in tact.

This is a simpler alternative to #898.
#898 is hitting some race condition in buildkit (or at least that's what it looks like).
Additionally this can actually be backported to older release branches as the changes are minor.

@cpuguy83 cpuguy83 force-pushed the be_lazyish branch 2 times, most recently from afbcce1 to 6501e6a Compare December 23, 2025 00:00
@cpuguy83 cpuguy83 force-pushed the be_lazyish branch 7 times, most recently from 5cfdf4e to 6f64bda Compare December 23, 2025 20:23
@cpuguy83 cpuguy83 marked this pull request as ready for review December 23, 2025 21:07
Copilot AI review requested due to automatic review settings December 23, 2025 21:07
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the test execution flow to ensure that an LLB state is always returned, even when tests fail. This enables proper debugging support with docker buildx dap by ensuring breakpoints can attach to the state. Instead of returning errors directly to BuildKit when tests fail, the PR adds a new LLB exec operation that prints test errors to stderr and exits non-zero, maintaining the build failure behavior while preserving the state chain.

Key changes:

  • Modified RunTests to return llb.StateOption instead of (Reference, error), allowing tests to be composed into the state chain
  • Introduced withTestError to create an error-reporting state that outputs test failures before exiting
  • Added TestErrorCmd as a new frontend command to handle test error output

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
test/linux_target_test.go Updates test to consume logs via SolveStatus callback and verify test errors appear in output rather than as direct function errors
targets/linux/rpm/distro/pkg.go Changes RunTests signature from returning (Reference, error) to StateOption, composing test dependencies and execution into the state chain
targets/linux/deb/distro/pkg.go Matches RPM implementation with same RunTests signature change to StateOption pattern
targets/linux/distro_handler.go Updates handlers to use new StateOption-based test execution and adds getRef helper to simplify state marshalling and solving
frontend/test_runner.go Core refactoring: RunTests now returns StateOption, adds withTestError for error reporting, TestErrorCmd for outputting errors, and evalState helper
cmd/frontend/main.go Wires up TestErrorCmd in the frontend command router
.github/workflows/ci.yml Adds Docker system prune before tests to clean up resources

Before this we are returning errors to buildkit when there are test
errors.
Or for that matter, if there is an error during evaluation of the stuff
running up to the tests.
This causes issues with the debug workflow ("docker buildx dap") since
there is no LLB state to attach to, so it can't set breakpoints
properly.

With this change even when we get a solve error when evaluating state,
we return the bad state so that the debugger can attach to it.
When a test fails, we add a new LLB exec op which prints the joined
test errors and exits non-zero so that the build will fail.

There's also some minor changes to make it simpler to discard any
changes to the filesystem from running tests, or in the case of
non-container outputs return the original state, while keeping the
dependency chain in tact.

Signed-off-by: Brian Goff <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants