Skip to content

Re-enable tests for FIRST/LAST #1646

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
andygrove opened this issue Apr 14, 2025 · 0 comments
Open

Re-enable tests for FIRST/LAST #1646

andygrove opened this issue Apr 14, 2025 · 0 comments
Labels
enhancement New feature or request
Milestone

Comments

@andygrove
Copy link
Member

andygrove commented Apr 14, 2025

What is the problem the feature request solves?

During the upgrade to DataFusion 47.0.0 () it was necessary to disable some tests that use FIRST and LAST because the behavior of these functions changed. These functions are non-determistic in Spark but previous versions of DataFusion had the same behavior as Spark for the single partition case used in the unit tests.

This issue is for re-enabling the tests somehow, or for implementing a Comet-specific implementation of FIRST/LAST to match Spark behavior (just for the single partition case used in tests).

These are the tests (all in CometAggregateSuite):

  ignore("single group-by column + aggregate column, multiple batches, no null") {
  ignore("multiple group-by columns + single aggregate column (first/last), with nulls") {
  ignore("multiple group-by columns + multiple aggregate column (first/last), with nulls") {
  ignore("all types first/last, with nulls") {
  ignore("first/last") {

Describe the potential solution

No response

Additional context

No response

@andygrove andygrove added the enhancement New feature or request label Apr 14, 2025
@andygrove andygrove changed the title Re-enable tests for FIRS/LAST Re-enable tests for FIRST/LAST Apr 14, 2025
@andygrove andygrove added this to the 0.8.0 milestone Apr 14, 2025
@andygrove andygrove modified the milestones: 0.8.0, 0.9.0 May 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant