datetime: pure Python implementation of fromisoformat() handles times with trailing spaces inconsistently with the C extension

[mgorny]

Bug report

Bug description:

Discovered primarily because PyPy uses the Python version of datetime from the stdlib, and django.utils.dateparse.parse_datetime() supports wider range of values than the C version of datetime.datetime.fromisoformat(), and falls back to their own parser.

>>> datetime.datetime.fromisoformat("2012-04-23T10:20:30.400")
datetime.datetime(2012, 4, 23, 10, 20, 30, 400000)
>>> datetime.datetime.fromisoformat("2012-04-23T10:20:30.400 ")
datetime.datetime(2012, 4, 23, 10, 20, 30, 40000)
>>> datetime.datetime.fromisoformat("2012-04-23T10:20:30.400  ")
datetime.datetime(2012, 4, 23, 10, 20, 30, 4000)
>>> datetime.datetime.fromisoformat("2012-04-23T10:20:30.400   ")
datetime.datetime(2012, 4, 23, 10, 20, 30, 400)
>>> datetime.datetime.fromisoformat("2012-04-23T10:20:30.400    ")
Traceback (most recent call last):
  File "<python-input-9>", line 1, in <module>
    datetime.datetime.fromisoformat("2012-04-23T10:20:30.400    ")
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mgorny/git/cpython/Lib/_pydatetime.py", line 1949, in fromisoformat
    raise ValueError(
        f'Invalid isoformat string: {date_string!r}') from None
ValueError: Invalid isoformat string: '2012-04-23T10:20:30.400    '
>>> datetime.datetime.fromisoformat("2012-04-23T10:20:30.400 +02:30")
datetime.datetime(2012, 4, 23, 10, 20, 30, 40000, tzinfo=datetime.timezone(datetime.timedelta(seconds=9000)))

For comparison, the C extension rejects all variants containing spaces, causing Django to use its own parser.

PyPy bug report: pypy/pypy#5240

I'm going to try preparing a patch.

CPython versions tested on:

3.11, 3.13, CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-130959: Reject whitespace in fractions, in pure Python fromisoformat() #130962

[mgorny]

Hmm, another difference is that when giving 6 digits of microseconds, C implementation accepts them but Python implementation does not:

>>> datetime.datetime.fromisoformat("2021-12-20T05:05:05.600000 +02:30")
datetime.datetime(2021, 12, 20, 5, 5, 5, 600000, tzinfo=datetime.timezone(datetime.timedelta(seconds=9000)))
>>> _pydatetime.datetime.fromisoformat("2021-12-20T05:05:05.600000 +02:30")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mgorny/miniforge3/lib/python3.12/_pydatetime.py", line 1874, in fromisoformat
    raise ValueError(
ValueError: Invalid isoformat string: '2021-12-20T05:05:05.600000 +02:30'

[pganssle]

Hmm, another difference is that when giving 6 digits of microseconds, C implementation accepts them but Python implementation does not:

I think the spec doesn't allow for whitespace, right? If so, we should do whatever the spec says. It's slightly more likely to break things if we make changes to the C implementation than the Python implementation (since the vast majority of people are using the C implementation).

I think we can fix the issue with the Python implementation and backport the changes to 3.12+, since right now it just gives the wrong answer, and it's better to start raising exceptions here. If we reject the version with a space from the C implementation, I think we would want that to target only 3.14+, because that will take something that currently parses and gives the "correct" answer and start raising exceptions. We don't have to go as far as deprecating the behavior, since the documented scope of the function says that this should be rejected, but I don't think on net it serves users to make that kind of change in a bugfix version.

[mgorny]

I think the spec doesn't allow for whitespace, right?

Yes, the standard doesn't permit whitespace, and it's not listed in "exceptions" in the documentation.