Skip to content

BUG: Handle unit count when converting from np.datetime64 #58915

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v3.0.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -453,6 +453,7 @@ Categorical
Datetimelike
^^^^^^^^^^^^
- Bug in :class:`Timestamp` constructor failing to raise when ``tz=None`` is explicitly specified in conjunction with timezone-aware ``tzinfo`` or data (:issue:`48688`)
- Bug in :class:`Timestamp` constructor ignoring the unit multiplier of a ``datetime64`` (:issue:`39977`)
- Bug in :func:`date_range` where the last valid timestamp would sometimes not be produced (:issue:`56134`)
- Bug in :func:`date_range` where using a negative frequency value would not include all points between the start and end values (:issue:`56382`)
- Bug in :func:`tseries.api.guess_datetime_format` would fail to infer time format when "%Y" == "%H%M" (:issue:`57452`)
Expand Down
6 changes: 4 additions & 2 deletions pandas/_libs/tslibs/conversion.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ from pandas._libs.tslibs.np_datetime cimport (
dts_to_iso_string,
get_conversion_factor,
get_datetime64_unit,
get_datetime64_unit_num,
get_implementation_bounds,
import_pandas_datetime,
npy_datetime,
Expand Down Expand Up @@ -287,9 +288,10 @@ cdef int64_t get_datetime64_nanos(object val, NPY_DATETIMEUNIT reso) except? -1:
return NPY_NAT

unit = get_datetime64_unit(val)
nums = get_datetime64_unit_num(val)

if unit != reso:
pandas_datetime_to_datetimestruct(ival, unit, &dts)
if unit != reso or nums != 1:
pandas_datetime_to_datetimestruct(ival * nums, unit, &dts)
try:
ival = npy_datetimestruct_to_datetime(reso, &dts)
except OverflowError as err:
Expand Down
1 change: 1 addition & 0 deletions pandas/_libs/tslibs/np_datetime.pxd
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ cdef int64_t pydate_to_dt64(
cdef void pydate_to_dtstruct(date val, npy_datetimestruct *dts) noexcept

cdef NPY_DATETIMEUNIT get_datetime64_unit(object obj) noexcept nogil
cdef int get_datetime64_unit_num(object obj) noexcept nogil

cdef int string_to_dts(
str val,
Expand Down
7 changes: 7 additions & 0 deletions pandas/_libs/tslibs/np_datetime.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,13 @@ cdef NPY_DATETIMEUNIT get_datetime64_unit(object obj) noexcept nogil:
return <NPY_DATETIMEUNIT>(<PyDatetimeScalarObject*>obj).obmeta.base


cdef int get_datetime64_unit_num(object obj) noexcept nogil:
"""
returns the number of units of the dtype for a numpy datetime64 object.
"""
return (<PyDatetimeScalarObject*>obj).obmeta.num
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does number of units mean exactly? And is there a public way to get this - I am wary of relying up internals llike this



cdef NPY_DATETIMEUNIT get_unit_from_dtype(cnp.dtype dtype):
# NB: caller is responsible for ensuring this is *some* datetime64 or
# timedelta64 dtype, otherwise we can segfault
Expand Down
6 changes: 6 additions & 0 deletions pandas/tests/scalar/timestamp/test_timestamp.py
Original file line number Diff line number Diff line change
Expand Up @@ -633,6 +633,12 @@ def test_comparison(self, dt64, ts):
assert ts <= alt
assert alt <= ts

def test_datetime64_unit_count(self):
# GH 39977
ts = Timestamp(np.datetime64("2000-01-01", "10ns"))

assert ts == Timestamp("2000-01-01")

def test_cmp_cross_reso(self):
# numpy gets this wrong because of silent overflow
dt64 = np.datetime64(9223372800, "s") # won't fit in M8[ns]
Expand Down
Loading