python3-3.12.4

[Bruce Dubbs]

New point version.

[Bruce Dubbs]

What's New in Python 3.12.4 final?
==================================

*Release date: 2024-06-06*

Security
--------

- gh-issue-118486: :func:`os.mkdir` on Windows now accepts *mode* of
  ``0o700`` to restrict the new directory to the current user. This fixes
  :cve:`2024-4030` affecting :func:`tempfile.mkdtemp` in scenarios where the
  base temporary directory is more permissive than the default.

- gh-issue-116741: Update bundled libexpat to 2.6.2

- gh-issue-117233: Detect BLAKE2, SHA3, Shake, & truncated SHA512 support in
  the OpenSSL-ish libcrypto library at build time.  This allows
  :mod:`hashlib` to be used with libraries that do not to support every
  algorithm that upstream OpenSSL does.

Core and Builtins
-----------------

- gh-issue-119821: Fix execution of :ref:`annotation scopes
  <annotation-scopes>` within classes when ``globals`` is set to a non-dict.
  Patch by Jelle Zijlstra.

- gh-issue-118263: Speed up :func:`os.path.normpath` with a direct C call.

- gh-issue-119311: Fix bug where names are unexpectedly mangled in the bases
  of generic classes.

- gh-issue-119395: Fix bug where names appearing after a generic class are
  mangled as if they are in the generic class.

- gh-issue-118507: Fix :func:`os.path.isfile` on Windows for pipes.
  
- gh-issue-119213: Non-builtin modules built with argument clinic were
  crashing if used in a subinterpreter before the main interpreter.  The
  objects that were causing the problem by leaking between interpreters
  carelessly have been fixed.

- gh-issue-119011: Fixes ``type.__type_params__`` to return an empty tuple
  instead of a descriptor.
  
- gh-issue-118997: Fix _Py_ClearImmortal() assertion: use _Py_IsImmortal()
  to tolerate reference count lower than _Py_IMMORTAL_REFCNT. Fix the
  assertion for the stable ABI, when a C extension is built with Python 3.11
  or lower. Patch by Victor Stinner.

- gh-issue-118513: Fix incorrect :exc:`UnboundLocalError` when two
  comprehensions in the same function both reference the same name, and in
  one comprehension the name is bound while in the other it's an implicit
  global.

- gh-issue-118164: Break a loop between the Python implementation of the
  :mod:`decimal` module and the Python code for integer to string
  conversion. Also optimize integer to string conversion for values in the
  range from 9_000 to 135_000 decimal digits.

- gh-issue-118272: Fix bug where ``generator.close`` does not free the
  generator frame's locals.

- gh-issue-116767: Fix crash in compiler on 'async with' that has many
  context managers.

- gh-issue-117894: Prevent ``agen.aclose()`` objects being re-used after
  ``.throw()``.

- gh-issue-117881: prevent concurrent access to an async generator via
  athrow().throw() or asend().throw()

- gh-issue-115874: Fixed a possible segfault during garbage collection of
  ``_asyncio.FutureIter`` objects

Library
-------

- gh-issue-119819: Fix regression to allow logging configuration with
  multiprocessing queue types.

- gh-issue-89727: Fix issue with :func:`shutil.rmtree` where a
  :exc:`RecursionError` is raised on deep directory trees.

- gh-issue-89727: Partially fix issue with :func:`shutil.rmtree` where a
  :exc:`RecursionError` is raised on deep directory trees. A recursion error
  is no longer raised when :data:`!rmtree.avoids_symlink_attacks` is false.

- gh-issue-119118: Fix performance regression in the :mod:`tokenize` module
  by caching the ``line`` token attribute and calculating the column offset
  more efficiently.
- gh-issue-89727: Fix issue with :func:`os.fwalk` where a
  :exc:`RecursionError` was raised on deep directory trees by adjusting the
  implementation to be iterative instead of recursive.

- gh-issue-113892: Now, the method ``sock_connect`` of
  :class:`asyncio.ProactorEventLoop` raises a :exc:`ValueError` if given
  socket is not in non-blocking mode, as well as in other loop
  implementations.

- gh-issue-119174: Fix high DPI causes turtledemo(turtle-graphics examples)
  windows blurry Patch by Wulian233 and Terry Jan Reedy

- gh-issue-118643: Fix an AttributeError in the :mod:`email` module when
  re-fold a long address list. Also fix more cases of incorrect encoding of
  the address separator in the address list.

- gh-issue-58933: Make :mod:`pdb` return to caller frame correctly when
  ``f_trace`` of the caller frame is not set

- gh-issue-118868: Fixed issue where kwargs were no longer passed to the
  logging handler QueueHandler

- gh-issue-118164: The Python implementation of the ``decimal`` module could
  appear to hang in relatively small power cases (like ``2**117``) if
  context precision was set to a very high value. A different method to
  check for exactly representable results is used now that doesn't rely on
  computing ``10**precision`` (which could be effectively too large to
  compute).

- gh-issue-118404: Fix :func:`inspect.signature` for non-comparable
  callables.

- gh-issue-118314: Fix an edge case in :func:`binascii.a2b_base64` strict
  mode, where excessive padding is not detected when no padding is
  necessary.

- gh-issue-118042: Fix an unraisable exception in
  :meth:`!telnetlib.Telnet.__del__` when the ``__init__()`` method was not
  called.

- gh-issue-118221: Fix a bug where :func:`!sqlite3.iterdump` could fail if a
  custom :attr:`row factory <sqlite3.Connection.row_factory>` was used.
  Patch by Erlend Aasland.
- gh-issue-118013: Fix regression introduced in gh-103193 that meant that
  calling :func:`inspect.getattr_static` on an instance would cause a strong
  reference to that instance's class to persist in an internal cache in the
  :mod:`inspect` module. This caused unexpected memory consumption if the
  class was dynamically created, the class held strong references to other
  objects which took up a significant amount of memory, and the cache
  contained the sole strong reference to the class. The fix for the
  regression leads to a slowdown in :func:`!getattr_static`, but the
  function should still be significantly faster than it was in Python 3.11.
  Patch by Alex Waygood.

- gh-issue-90848: Fixed :func:`unittest.mock.create_autospec` to configure
  parent mock with keyword arguments.

- gh-issue-118168: Fix incorrect argument substitution when
  :data:`typing.Unpack` is used with the builtin :class:`tuple`.
  :data:`!typing.Unpack` now raises :exc:`TypeError` when used with certain
  invalid types. Patch by Jelle Zijlstra.

- gh-issue-118033: Fix :func:`dataclasses.dataclass` not creating a
  ``__weakref__`` slot when subclassing :class:`typing.Generic`.

- gh-issue-117535: Do not try to get the source line for made up file name
  "sys" in :mod:`warnings`.

- gh-issue-114053: Fix erroneous :exc:`NameError` when calling
  :func:`typing.get_type_hints` on a class that made use of :pep:`695` type
  parameters in a module that had ``from __future__ import annotations`` at
  the top of the file. Patch by Alex Waygood.

- gh-issue-117995: Don't raise :exc:`DeprecationWarning` when a
  :term:`sequence` of parameters is used to bind indexed, nameless
  placeholders. See also :gh:`100668`.

- gh-issue-80361: Fix TypeError in :func:`email.message.Message.get_payload`
  when the charset is :rfc:`2231` encoded.

- gh-issue-86650: Fix IndexError when parse some emails with invalid
  Message-ID (including one-off addresses generated by Microsoft Outlook).

- gh-issue-117691: Improve the error messages emitted by :mod:`tarfile`
  deprecation warnings relating to PEP 706. If a ``filter`` argument is not
  provided to ``extract()`` or ``extractall``, the deprecation warning now
  points to the line in the user's code where the relevant function was
  called. Patch by Alex Waygood.

- gh-issue-77102: :mod:`site` module now parses ``.pth`` file with UTF-8
  first, and :term:`locale encoding` if ``UnicodeDecodeError`` happened. It
  supported only locale encoding before.

- gh-issue-117692: Fixes a bug when :class:`doctest.DocTestFinder` was
  failing on wrapped ``builtin_function_or_method``.

- gh-issue-117566: :meth:`ipaddress.IPv6Address.is_loopback` will now return
  ``True`` for IPv4-mapped loopback addresses, i.e. addresses in the
  ``::ffff:127.0.0.0/104`` address space.

- gh-issue-117503: Fix support of non-ASCII user names in bytes paths in
  :func:`os.path.expanduser` on Posix.

- gh-issue-117313: Only treat ``'\n'``, ``'\r'`` and ``'\r\n'`` as line
  separators in re-folding the :mod:`email` messages. Preserve control
  characters ``'\v'``, ``'\f'``, ``'\x1c'``, ``'\x1d'`` and ``'\x1e'`` and
  Unicode line separators ``'\x85'``, ``'\u2028'`` and ``'\u2029'`` as is.

- gh-issue-113171: Fixed various false positives and false negatives in

  * :attr:`ipaddress.IPv4Address.is_private` (see these docs for details)
  * :attr:`ipaddress.IPv4Address.is_global`
  * :attr:`ipaddress.IPv6Address.is_private`
  * :attr:`ipaddress.IPv6Address.is_global`

  Also in the corresponding :class:`ipaddress.IPv4Network` and
  :class:`ipaddress.IPv6Network` attributes.

- gh-issue-103956: Fix lack of newline characters in :mod:`trace` module
  output when line tracing is enabled but source code line for current frame
  is not available.

- gh-issue-92081: Fix missing spaces in email headers when the spaces are
  mixed with encoded 8-bit characters.

- gh-issue-103194: Prepare Tkinter for C API changes in Tcl 8.7/9.0 to avoid
  :class:`_tkinter.Tcl_Obj` being unexpectedly returned instead of
  :class:`bool`, :class:`str`, :class:`bytearray`, or :class:`int`.

- gh-issue-87106: Fixed handling in :meth:`inspect.Signature.bind` of
  keyword arguments having the same name as positional-only arguments when a
  variadic keyword argument (e.g. ``**kwargs``) is present.

- bpo-45767: Fix integer conversion in :func:`os.major`, :func:`os.minor`,
  and :func:`os.makedev`. Support device numbers larger than ``2**63-1``.
  Support non-existent device number (``NODEV``).

- bpo-40943: Fix several IndexError when parse emails with truncated
  Message-ID, address, routes, etc, e.g. ``example@``.

- bpo-30988: Fix parsing of emails with invalid address headers having a
  leading or trailing dot. Patch by tsufeki.

- gh-issue-67693: Fix :func:`urllib.parse.urlunparse` and
  :func:`urllib.parse.urlunsplit` for URIs with path starting with multiple
  slashes and no authority. Based on patch by Ashwin Ramaswami.

- bpo-15010: :meth:`unittest.TestLoader.discover` now saves the original
  value of ``unittest.TestLoader._top_level_dir`` and restores it at the end
  of the call.

Documentation
-------------

- gh-issue-117928: The minimum Sphinx version required for the documentation
  is now 6.2.1.

- gh-issue-91565: Changes to documentation files and config outputs to
  reflect the new location for reporting bugs - i.e. GitHub rather than
  bugs.python.org.

Tests
-----

- gh-issue-119050: regrtest test runner: Add XML support to the refleak
  checker (-R option). Patch by Victor Stinner.

IDLE
----

- bpo-34774: Use user-selected color theme for Help => IDLE Doc.

C API
-----

- gh-issue-119585: Fix crash when a thread state that was created by
  :c:func:`PyGILState_Ensure` calls a destructor that during
  :c:func:`PyThreadState_Clear` that calls back into
  :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release`. This might
  occur when in the free-threaded build or when using thread-local variables
  whose destructors call :c:func:`PyGILState_Ensure`.

- gh-issue-117534: Improve validation logic in the C implementation of
  :meth:`datetime.fromisoformat` to better handle invalid years. Patch by
  Vlad Efanov.

[Bruce Dubbs]

Fixed at commit 58180415d.

[Douglas R. Reno]

While going through my email, I just encountered a mail from the Python security team about a CVE fixed here that doesn't seem to be in the release notes.

Here's the contents of that email:

-------- Forwarded Message --------
Subject:     [Security-announce][CVE-2024-4032] Incorrect IPv4 and IPv6 private ranges
Date:     Mon, 17 Jun 2024 09:01:18 -0500
From:     Seth Larson <seth@python.org>
Reply-To:     security-sig@python.org
To:     security-announce@python.org

The “ipaddress” module contained incorrect information about whether certain IPv4 and 
IPv6 addresses were designated as “globally reachable” or “private”. This affected the 
'is_private' and 'is_global' properties of the ipaddress.IPv4Address, 
ipaddress.IPv4Network, ipaddress.IPv6Address, and ipaddress.IPv6Network classes, where 
values wouldn’t be returned in accordance with the latest information from the IANA 
Special-Purpose Address Registries.

CPython 3.12.4 and 3.13.0a6 contain updated information from these registries and thus 
have the intended behavior.

Severity: Medium

References

  * https://github.com/python/cpython/issues/113171
  * https://github.com/python/cpython/pull/113179
  * https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml
  * https://www.iana.org/assignments/iana-ipv6-special-registry/iana-ipv6-special-registry.xhtml

Note that CPython is what we use, it's the C/C++ implementation of the Python language. Users who use older versions of Python will need to backport the fix.

[Douglas R. Reno]

SA-12.1-069 issued