Skip to content

Conversation

@sanchit122006
Copy link

@sanchit122006 sanchit122006 commented Dec 12, 2025

  • closes #xxxx (Replace xxxx with the GitHub issue number)
  • Tests added and passed if fixing a bug or adding a new feature
  • All code checks passed.
  • Added type annotations to new arguments/methods/functions.
  • Added an entry in the latest tests\series\test_indexing.py file if fixing a bug or adding a new feature.
  • If I used AI to develop this pull request, I prompted it to follow AGENTS.md.

When performing partial slicing on a three-level MultiIndex where:

The first level (L1) has values of type datetime.date (from the standard library).

The lookup key for L1 is of type np.datetime64.

The lookup key tuple only includes L1 and L2 (e.g., df.loc[(key_L1, key_L2)]).

Pandas incorrectly returns data matching only the L1 key, effectively ignoring the L2 key. This partial slicing failure occurs because the lookup logic seems to fail the key match comparison due to the type mismatch, causing it to fall back to a less specific search across the MultiIndex.

The bug is absent if the lookup explicitly includes slice(None) for the third level (e.g., df.loc[(date, "A", slice(None))]).

Reproducible Example (Failing Code):-
'''
import pandas as pd
import numpy as np
import datetime as dt

Index created with standard library datetime.date objects

dates = [dt.datetime(2023, 11, 1).date(), dt.datetime(2023, 11, 1).date(), dt.datetime(2023, 11, 2).date()]
t1 = ["A", "B", "C"]
t2 = ["C", "D", "E"]
vals = np.random.uniform(size=len(dates))

df = pd.DataFrame(data=np.array([dates, t1, t2, vals]).T, columns=["dates", "t1", "t2", "vals"])
df.set_index(["dates", "t1", "t2"], inplace=True)

Lookup key is np.datetime64

date = np.datetime64("2023-11-01")

INCORRECT BEHAVIOR:

df.loc[(date, "A")] - Expected 1 row, but returns 2 rows (matching date, ignoring 'A')

df.loc[(date, "C")] - Expected KeyError, but returns 1 row (matching date, ignoring 'C')

'''

Expected Behavior
The expected behavior should be one of the following:

Strict Type Check: Raise a TypeError or KeyError due to the strict mismatch between the datetime.date index and the np.datetime64 key, forcing the user to cast keys explicitly.

Correct Coercion/Match: Coerce the np.datetime64 key to the index type (datetime.date) and correctly match only the row(s) that match both L1 (date) and L2 ('A' or 'B').

This PR implements the necessary fix to ensure the correct behavior (Option 2) when performing partial slicing with this specific type mismatch.

@sanchit122006 sanchit122006 deleted the fix/multiindex_partial_slice_datetime_type branch December 12, 2025 13:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant