Repository navigation
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1407 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 10 10
Lines 2315 2315
Branches 358 358
=========================================
Hits 2315 2315 ☔ View full report in Codecov by Harness. |
datetime.max.timestamp() raises a ValueError or OverflowError when run in timezones east of UTC (e.g. Asia/Kolkata, Europe/Berlin) because the local time maps to year 10000. This caused the fallback to year 3000 timestamp to trigger prematurely on 64-bit systems, which broke normalize_timestamp() for valid large timestamps (like year 9999). Using datetime.max.replace(tzinfo=timezone.utc).timestamp() computes the true POSIX timestamp upper bound safely and deterministically across all timezones. Fixes arrow-py#1159
Rajpal14
force-pushed
the
fix/normalize-timestamp-large-values
branch
from
November 9, 2026 20:45
d58c12f to
acf2c99
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Fixes #1159.
In
arrow/constants.py,_MAX_TIMESTAMPwas computed using naivedatetime.max.timestamp().When run on systems located in timezones east of UTC (such as
Asia/KolkataorEurope/Berlin),datetime.max.timestamp()converts to local time and raises:This caused the exception handler to trigger the 64-bit fallback (
year 3000timestamp32503746600.0). Consequently, valid large timestamps like year9999-12-31(253402214400.0) were misidentified as millisecond timestamps innormalize_timestamp()and erroneously divided by 1000, converting year 9999 into year 1978.Solution
constants.pyto compute_MAX_TIMESTAMPusingdatetime.max.replace(tzinfo=timezone.utc).timestamp(). This prevents timezone shifts past year 9999 and gives the true POSIX maximum timestamp platform-independently.tzinfo=timezone.utcto the fallback expressions for consistency.tests/test_util.pyensuringnormalize_timestamp()preserves year 9999 timestamps without dividing by 1000.Verification
1904 passed, 99.93% coverage).arrow.get('9999-12-31').timestamp()reconstructed viaarrow.get()successfully returns9999-12-31.