Skip to content

Commit af27717

Browse files
samstokesclaude
andauthored
fix(dashboard): ensure query_end fires when generator is abandoned early (Eventual-Inc#6326)
## Summary - When `.show()` collects enough rows, it breaks out of the `run_iter` generator early. Python sends `GeneratorExit` (a `BaseException`) which none of the existing exception handlers caught, so `_notify_query_end` was never called and the dashboard stayed stuck in "Finalizing" forever. - Handle `GeneratorExit` in `run_iter` to notify subscribers before returning. - Add regression test that verifies `on_query_end` fires for both `.collect()` and `.show()`. Closes DF-1664 ## Test plan - [x] New test `test_show_fires_query_end` reproduces the bug (fails without the fix, passes with it) - [x] All existing subscriber tests pass (`tests/test_subscribers.py`) - [x] Manually verified with the dashboard — query transitions from Finalizing to Finished 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 623e8b1 commit af27717

File tree

2 files changed

+44
-0
lines changed

2 files changed

+44
-0
lines changed

daft/runners/native_runner.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -139,6 +139,15 @@ def run_iter(
139139
except Exception:
140140
pass
141141
yield result
142+
except GeneratorExit:
143+
# Generator was abandoned (e.g., .show() breaking out early after collecting
144+
# enough rows). Notify subscribers so the dashboard transitions out of Finalizing.
145+
try:
146+
query_result = PyQueryResult(QueryEndState.Finished, "Query finished")
147+
ctx._notify_query_end(query_id, query_result)
148+
except Exception as e:
149+
logger.warning("Failed to send query end notification: %s", e)
150+
return # type: ignore[return-value]
142151
except StopIteration as e:
143152
query_result = PyQueryResult(QueryEndState.Finished, "Query finished")
144153
ctx._notify_query_end(query_id, query_result)

tests/test_subscribers.py

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -139,6 +139,41 @@ def long_running_udf(s: daft.Series):
139139
assert "Query canceled by the user" in subscriber.end_messages[query_id]
140140

141141

142+
def test_show_fires_query_end():
143+
"""Regression test for DF-1664.
144+
145+
.show() abandons the run_iter generator early, which must still fire
146+
on_query_end so the dashboard transitions out of Finalizing.
147+
"""
148+
subscriber = MockSubscriber()
149+
ctx = daft.context.get_context()
150+
ctx.attach_subscriber("mock", subscriber)
151+
152+
try:
153+
# Sanity check: .collect() on a transformed DataFrame fires on_query_end.
154+
df = daft.from_pydict({"x": list(range(100))})
155+
df = df.with_column("y", daft.col("x") + 1)
156+
df.collect()
157+
158+
query_id = subscriber.query_ids[-1]
159+
assert query_id in subscriber.end_states, "collect() should fire on_query_end"
160+
assert subscriber.end_states[query_id] == QueryEndState.Finished
161+
162+
# The actual bug: .show() on a transformed DataFrame does NOT fire on_query_end,
163+
# because _construct_show_preview breaks out of the run_iter generator early.
164+
df = daft.from_pydict({"x": list(range(100))})
165+
df = df.with_column("y", daft.col("x") + 1)
166+
df.show()
167+
168+
query_id = subscriber.query_ids[-1]
169+
assert query_id in subscriber.end_states, (
170+
"on_query_end was not called — query would be stuck in Finalizing on the dashboard"
171+
)
172+
assert subscriber.end_states[query_id] == QueryEndState.Finished
173+
finally:
174+
ctx.detach_subscriber("mock")
175+
176+
142177
def test_subscriber_template():
143178
subscriber = MockSubscriber()
144179
ctx = daft.context.get_context()

0 commit comments

Comments
 (0)