Cython-3.0.10 (Python module) (Wait until PyYAML can use this or archive PyYAML?)

[Bruce Dubbs]

New major version.

This version is not compatible with PyYAML-6.

[Bruce Dubbs]

3.0.0 (2023-07-17)

Bugs fixed

• A crash in Python 2.7 was fixed when cleaning up extension type instances at program end.

3.0.0 rc 2 (2023-07-13)

Bugs fixed

• Parser crash on hex/oct enum values.

• 'cython --version' now prints the version to both stdout and stderr (unless that is a TTY).

3.0.0 rc 1 (2023-07-12)

Features added

• 'with gil' and 'with nogil(flag)' now accept their flag argument also in Python code.

• A new decorator '@…_gil' is available in Python code to match the 'with gil' function declaration in Cython syntax.

• Assigning a list to a ctuple is slightly faster.

Bugs fixed

• The reference counting of memory views involved useless overhead.

• Duplicate values in a cpdef enum could lead to invalid switch statements.

• Handling freshly raised exceptions that didn't have a traceback yet could crash.

• Reverse iteration in C++ no longer removes the const qualifier from the item type.

• C++ containers of item type bint could conflict with those of item type int.

• With MSVC, Cython no longer enables C-Complex support by accident (which is not supported there).

• The Python implementation of cimport cython.cimports… could raise an ImportError instead of an AttributeError when looking up package variable names.

• Passing a language level and directives on the command line lost the language level setting.

• Some typedef declarations for libc function types were fixed.

• Some C compiler warnings and errors in CPython 3.12 were resolved.

• The deprecated _PyGC_FINALIZED() C-API macro is no longer used.

• A compile error when using debug was resolved.

• A module loading problem with cython.inline() on Windows was resolved.

• cython --version now prints the version to stdout instead of stderr.

• Includes all bug-fixes and features from the 0.29 maintenance branch up to the :ref:'0.29.36' release.

Other changes

• The FAQ page was moved from the GitHub Wiki to the regular documentation to make it more visible.

• 'np.long_t' and np.ulong_t were removed from the NumPy declarations, synching Cython with upstream NumPy v1.25.0. The aliases were confusing since they could mean different things on different platforms.

3.0.0 beta 3 (2023-05-24)

Features added

• Custom buffer slot methods are now supported in the Limited C-API of Python 3.9+.

• The extern "C" and extern "C++" markers that Cython generates for public functions can now be controlled by setting the C macro CYTHON_EXTERN_C.

• The Python int handling code was adapted to make use of the new PyLong internals in CPython 3.12.

• Conversion of Python ints to C int128 is now always supported, although slow if dedicated C-API support is missing (_PyLong_AsByteArray()), specifically in the Limited C-API.

• The exception handling code was adapted to CPython 3.12.

• The dataclass implementation was adapted to support Python 3.12.

• The normal @dataclasses.dataclass and @functools.total_ordering decorators can now be used on extension types. Using the corresponding @cython.* decorator will automatically turn a Python class into an extension type (no need for @cclass).

• Multiplying a sequence by a C integer avoids creating and intermediate Python integer.

• ctuples can now be assigned from arbitrary sequences, not just Python tuples.

• A new directive embedsignature.format was added to select the format of the docstring embedded signatures between python, c and argument clinic.

• Some invalid directive usages are now detected and rejected, e.g. using @ccall together with @cfunc, and applying @cfunc to a @ufunc. Cython also warns now when a directive is applied needlessly.

• Unicode identifier names now allow all letters defined in CPython 3.12.

Bugs fixed

• Some C compile failures in CPython 3.12.0a6/a7 were resolved.

• Cascaded comparisons between integer constants and Python types could fail to compile.

• The internal macro PYX_IS_UNSIGNED was accidentally duplicated in beta 2 which lead to C compile errors.

• Memoryviews with typedef item types could fail to match the non-typedef item types.

• Fused memory views could raise a TypeError instead of a ValueError on creation.

• Cython could crash when finding import files with dots in their names.

• Selecting a context manager in parentheses and then calling it directly failed to parse.

• qualname and module were not available inside of class bodies.

• noexcept was not automatically applied to function pointer attributes in extern structs.

• Function signatures containing a type like tuple[()] could not be printed.

• Extension type hierarchies were generated in the wrong order, thus leading to compile issues.

• Using the --working option could lead to sources not being found.

• Some C compiler warnings were resolved.

• Includes all bug-fixes and features from the 0.29 maintenance branch up to the :ref:0.29.35 release.

Other changes

• For-loops now release the internal reference to their list/tuple iterable before instead of after the else: clause. This probably has no practical impact.

• Simple tuple types like (int, int) are no longer accepted in Python annotations and require the Python notation instead (e.g. tuple[cython.int, cython.int]).

• The code except +nogil (declaring a C++ exception handler function called nogil) is now rejected because it is almost certainly a typo from except + nogil.

3.0.0 beta 2 (2023-03-27)

Features added

• C++ declarations for <cmath>, <numbers> and std::any were added.

Bugs fixed

• Unintended internal exception handling lead to a visible performance regression for nogil memoryview code in 3.0.0b1.

• None default arguments for arguments with fused memoryview types could select a different implementation in 3.0 than in 0.29.x. The selection behaviour is generally considered suboptimal but was at least reverted to the old behaviour for now.

• The new complex vs. floating point behaviour of the power operator accidentally added a dependency on the GIL, which was really only required on failures.

• from cython cimport … as … could lead to imported names not being found in annotations.

• Generated NumPy ufuncs could crash for large arrays due to incorrect GIL handling.

• Very long Python integer constants could exceed the maximum C name length of MSVC.

• cimport_from_pyx could miss some declarations.

• Fully qualified C++ names prefixed by a cimported module name could fail to compile.

• Cython generated C++ code accidentally used C++11 features in some cases.

• Some C++ warnings regarding const usage in internally generated utility code were resolved.

• With language_level=2, imports of modules in packages could return the wrong module in Python 3.

• typing.Optional could fail on tuple types.

• Auto-generated utility code didn't always have all required user defined types available.

• Type checks for Python's memoryview type generated incorrect C code.

• Some issues with depfile generation were resolved.

• Some C code issue were resolved for the Limited API target.

• The C code shown in the annotated HTML output could lack the last C code line(s).

3.0.0 beta 1 (2023-02-25)

Features added

• Cython implemented C functions now propagate exceptions by default, rather than swallowing them in non-object returning function if the user forgot to add an except declaration to the signature. This was a long-standing source of bugs, but can require adding the noexcept declaration to existing functions if exception propagation is really undesired.

• To opt out of the new, safer exception handling behaviour, legacy code can set the new directive legacy_implicit_noexcept=True for a transition period to keep the previous, unsafe behaviour. This directive will eventually be removed in a later release.

• A new function decorator @cython.ufunc automatically generates a (NumPy) ufunc that applies the calculation function to an entire memoryview.

• The power operator now behaves more like in Python by returning the correct complex result if required by math. A new cpow directive was added to turn on the previous C-like behaviour.

• The special *pow methods now support the 2- and 3-argument variants.

• Unknown type annotations (e.g. because of typos) now emit a warning at compile time.

• Subscripted builtin types in type declarations (like list[float]) are now better supported.

• Python's memoryview is now a known builtin type with optimised properties.

• The call-time dispatch for fused memoryview types is less slow.

• Integer comparisons avoid Python coercions if possible.

• The Python Enum of a cpdef enum now inherits from IntFlag to better match both Python and C semantics of enums.

• PEP-614 <https://peps.python.org/pep-0614/>_: decorators can now be arbitrary Python expressions.

• cpdef enums can now be pickled.

• Bound C methods can now coerce to Python objects.

• C arrays can be initialised inside of nogil functions.

• reversed() can now be used together with C++ iteration.

• Standard C/C++ atomic operations are now used for memory views, if available.

• C11 complex.h is now properly detected.

• Nested cppclass definitions are supported.

• cpp_locals no longer have to be "assignable".

• cythonize --help now also prints information about the supported environment variables.

• Declarations were added for the C++ bit operations, some other parts of C++20 and CPython APIs.

Bugs fixed

• Generator expressions and comprehensions now look up their outer-most iterable on creation, as Python does, and not later on start, as they did previously.

• Type annotations for Python int rejected long under Py2 in the alpha-11 release. They are now ignored again (as always before) when language_level=2, and accept both int and long in Py2 (and only int in Py3) otherwise.

• Calling bound classmethods of builtin types could fail trying to call the unbound method.

• int(Py_UCS4) returned the code point instead of the parsed digit value.

• Several problems with CPython 3.12 were resolved.

• The C float type was not inferred on assignments.

• Memoryviews with object item type were not supported in Python type declarations.

• Iterating over memoryviews in generator expressions could leak a buffer reference.

• Memory views and the internal Cython array type now identify as collections.abc.Sequence by setting the Py_TPFLAGS_SEQUENCE type flag directly.

• del finaliser methods were not always called if they were only inherited.

• Extension types are now explicitly marked as immutable types to prevent them from being considered mutable.

• const types could not be returned from functions.

• cdef public functions declared in .pxd files could use an incorrectly mangled C name.

• cdef public functions used an incorrect linkage declaration in C++.
• C++ post-increment/-decrement operators were not correctly looked up on declared C++ classes, thus allowing Cython declarations to be missing for them and incorrect C++ code to be generated.

• C++ iteration more safely stores the iterable in temporary variables.

• C++ references did not work on fused types.

• The module state struct was not initialised in correct C (before C23), leading to compile errors on Windows.

• Structs that contained an array field resulted in incorrect C code. Their initialisation now uses memcpy().

• Nesting fused types in other fused types could fail to specialise the inner type.

• The special methods matmul, truediv, floordiv failed to type their self argument.

• Coverage analysis failed in projects with a separate source subdirectory.

• The annotation_typing directive was missing in pure Python mode.

• The @dataclass directive was accidentally inherited by methods and subclasses.

• Some issues with Cython @dataclass arguments, hashing, inheritance and repr() were resolved.

• cpdef enums no longer use OrderedDict but dict in Python 3.6 and later.

• Larger numbers of extension types with multiple subclasses could take very long to compile.

• Relative imports failed in compiled init.py package modules.
• Some old usages of the deprecated Python imp module were replaced with importlib.

• The cython and cythonize commands ignored non-existing input files without error.

• Invalid and misspelled cython.* module names were not reported as errors.

• Unused kwargs arguments did not show up in locals().

• Extended glob paths with // and \\ for finding source files failed on Windows.

• Annotated HTML generation was missing newlines in 3.0.0a11.

• Some parser issues were resolved.

• setup.cfg was missing from the source distribution.

• Some C/C++ warnings were resolved.

• The embedding code no longer calls deprecated C-API functions but uses the new PyConfig API instead on CPython versions that support it (3.8+).

• Intel C compilers could complain about unsupported gcc pragmas.

• Includes all bug-fixes and features from the 0.29 maintenance branch up to the :ref:0.29.33 release.

Other changes

• The undocumented, untested and apparently useless syntax from somemodule cimport class/struct/union somename was removed. The type modifier is not needed here and a plain cimport of the name will do.

• C-style array declarations (cdef int a[4]) are now (silently) deprecated in favour of the Java-style cdef int[4] a form. The latter was always available and the Python type declaration syntax already used it exclusively (a: int[4]).

• The wheel building process was migrated to use the cibuildwheel tool.

• Wheels now include a compiled parser again, which increases their size a little but gives about a 10% speed-up when running Cython.

• The Tempita module no longer contains HTML processing capabilities, which were found to be broken in Python 3.8 and later.

• The Emacs Cython mode file cython-mode.el is now maintained in a separate repo: ​https://github.com/cython/emacs-cython-mode

• Cython now uses a .dev0 version suffix for unreleased source installations.

See CHANGES.rst for alpha level changes.

[Xi Ruoyao]

PyYAML issue: ​https://github.com/yaml/pyyaml/issues/601

Maybe the fix can arrive for 12.0 or not.

[Joe Locash]

Here's a sed that will get PyYAML-6.0.1 to build with Cython-3:

sed -i 's/Cython.Distutils import build_ext/Cython.Distutils.old_build_ext import old_build_ext/' setup.py

It's a hack but I don't see any movement from the PyYAML team so why hold up updating other packages because of it.

[Xi Ruoyao]

Now 3.0.10.

[Bruce Dubbs]

Replying to Joe Locash:

Here's a sed that will get PyYAML-6.0.1 to build with Cython-3:

sed -i 's/Cython.Distutils import build_ext/Cython.Distutils.old_build_ext import old_build_ext/' setup.py

It's a hack but I don't see any movement from the PyYAML team so why hold up updating other packages because of it.

For some reason I just noticed this, but I like to play around with sed. Here is an equivalent sed that is a little shorter, but I'm not sure it is more clear.

sed -E '/Distutils /{s/utils/&.old_build_ext/;s/\ (build_ext)/ old_\1/}' \
    -i setup.py

It does the same thing, but I have not tested the change for either PyYAML or Cython-3.

[Xi Ruoyao]

Per the recent blfs-dev discussion it seems we can just archive pyyaml.

[Bruce Dubbs]

I'll archive pyyaml and update to Cython-3.0.10.

[Bruce Dubbs]

Fixed at commit c4a9428ef1.