xz-5.6.0

[Bruce Dubbs]

New minor version.

[Xi Ruoyao]

5.6.0 (2024-02-24)

This bumps the minor version of liblzma because new features were added. The API and ABI are still backward compatible with liblzma 5.4.x and 5.2.x and 5.0.x.

NOTE: As described in the NEWS for 5.5.2beta, the core components are now under the BSD Zero Clause License (0BSD).

Since 5.5.2beta:

• liblzma:

• Disabled the branchless C variant in the LZMA decoder based on the benchmark results from the community.

• Disabled x86-64 inline assembly on x32 to fix the build.

• Sandboxing support in xz:

• Landlock is now used even when xz needs to create files. In this case the sandbox is has to be more permissive than when no files need to be created. A similar thing was already in use with pledge(2) since 5.3.4alpha.

• Landlock and pledge(2) are now stricter when reading from more than one input file and only writing to standard output.

• Added support for Landlock ABI version 4.

• CMake:

• Default to -O2 instead of -O3 with CMAKE_BUILD_TYPE=Release. -O3 is not useful for speed and makes the code larger.

• Now builds lzmainfo and lzmadec.

• xzdiff, xzgrep, xzless, xzmore, and their symlinks are now installed. The scripts are also tested during "make test".

• Added translation support for xz, lzmainfo, and the man pages.

• Applied the symbol versioning workaround for MicroBlaze that is used in the Autotools build.

• The general XZ Utils and liblzma API documentation is now installed.

• The CMake component names were changed a little and several were added. liblzma_Runtime and liblzma_Development are unchanged.

• Minimum required CMake version is now 3.14. However, translation support is disabled with CMake versions older than 3.20.

• The CMake-based build is now close to feature parity with the Autotools-based build. Most importantly a few tests aren't run yet. Testing the CMake-based build on different operating systems would be welcome now. See the comment at the top of CMakeLists.txt.

• Fixed a bug in the Autotools feature test for ARM64 CRC32 instruction support for old versions of Clang. This did not affect the CMake build.

• Windows:

• The build instructions in INSTALL and windows/INSTALL*.txt were revised completely.

• windows/build-with-cmake.bat along with the instructions in windows/INSTALL-MinGW-w64_with_CMake.txt should make it very easy to build liblzma.dll and xz.exe on Windows using CMake and MinGW-w64 with either GCC or Clang/LLVM.

• windows/build.bash was updated. It now works on MSYS2 and on GNU/Linux (cross-compiling) to create a .zip and .7z package for 32-bit and 64-bit x86 using GCC + MinGW-w64.

• The TODO file is no longer installed as part of the documentation. The file is out of date and does not reflect the actual tasks that will be completed in the future.

• Translations:

• Translated lzmainfo man pages are now installed. These had been forgotten in earlier versions.

• Updated Croatian, Esperanto, German, Hungarian, Korean, Polish, Romanian, Spanish, Swedish, Vietnamese, and Ukrainian translations.

• Updated German, Korean, Romanian, and Ukrainian man page translations.

• Added a few tests.

Summary of new features added in the 5.5.x development releases:

• liblzma:

• LZMA decoder: Speed optimizations to the C code and added GCC & Clang compatible inline assembly for x86-64.

• Added lzma_mt_block_size() to recommend a Block size for multithreaded encoding.

• Added CLMUL-based CRC32 on x86-64 and E2K with runtime processor detection. Similar to CRC64, on 32-bit x86 it isn't available unless --disable-assembler is used.

• Optimized the CRC32 calculation on ARM64 platforms using the CRC32 instructions. Runtime detection for the instruction is used on GNU/Linux, FreeBSD, Windows, and macOS. If the compiler flags indicate unconditional CRC32 instruction support (+crc) then the generic version is not built.

• Added definitions of mask values like LZMA_INDEX_CHECK_MASK_CRC32 to <lzma/index.h>.

• xz:

• Multithreaded mode is now the default. This improves compression speed and creates .xz files that can be decompressed in multithreaded mode. The downsides are increased memory usage and slightly worse compression ratio.

• Added a new command line option --filters to set the filter chain using the liblzma filter string syntax.

• Added new command line options --filters1 ... --filters9 to set additional filter chains using the liblzma filter string syntax. The --block-list option now allows specifying filter chains that were set using these new options.

• Ported the command line tools to Windows MSVC. Visual Studio 2015 or later is required.

• Added lz4 support to xzdiff/xzcmp and xzgrep.

5.5.2beta (2024-02-14)

• Licensing change: The core components are now under the BSD Zero Clause License (0BSD). In XZ Utils 5.4.6 and older and 5.5.1alpha these components are in the public domain and obviously remain so; the change affects the new releases only.

0BSD is an extremely permissive license which doesn't require retaining or reproducing copyright or license notices when distributing the code, thus in practice there is extremely little difference to public domain.

• liblzma

• Significant speed optimizations to the LZMA decoder were made. There are now three variants that can be chosen at build time:

• Basic C version: This is a few percent faster than 5.4.x due to some new optimizations.

• Branchless C: This is currently the default on platforms for which there is no assembly code. This should be a few percent faster than the basic C version.

• x86-64 inline assembly. This works with GCC and Clang.

The default choice can currently be overridden by setting LZMA_RANGE_DECODER_CONFIG in CPPFLAGS: 0 means the basic version and 3 means that branchless C version.

• Optimized the CRC32 calculation on ARM64 platforms using the CRC32 instructions. The instructions are optional in ARMv8.0 and are required in ARMv8.1 and later. Runtime detection for the instruction is used on GNU/Linux, FreeBSD, Windows, and macOS. If the compiler flags indicate unconditional CRC32 instruction support (+crc) then the generic version is not built.

• Added lz4 support to xzdiff/xzcmp and xzgrep.

• Man pages of xzdiff/xzcmp, xzgrep, and xzmore were rewritten to simplify licensing of the man page translations.

• Translations:

• Updated Chinese (simplified), German, Korean, Polish, Romanian, Spanish, Swedish, and Ukrainian translations.

• Updated German, Korean, Romanian, and Ukrainian man page translations.

• Small improvements to the tests.

• Added doc/examples/11_file_info.c. It was added to the Git repository in 2017 but forgotten to be added into distribution tarballs.

• Removed doc/examples_old. These were from 2012.

• Removed the macos/build.sh script. It had not been updated since 2013.

5.5.1alpha (2024-01-26)

• Added a new filter for RISC-V binaries. The filter can be used for 32-bit and 64-bit binaries with either little or big endianness. In liblzma, the Filter ID is LZMA_FILTER_RISCV (0x0B) and the xz option is --riscv. liblzma filter string syntax recognizes this filter as "riscv".

• liblzma:

• Added lzma_mt_block_size() to recommend a Block size for multithreaded encoding

• Added CLMUL-based CRC32 on x86-64 and E2K with runtime processor detection. Similar to CRC64, on 32-bit x86 it isn't available unless --disable-assembler is used.

• Implemented GNU indirect function (IFUNC) as a runtime function dispatching method for CRC32 and CRC64 fast implementations on x86. Only GNU/Linux (glibc) and FreeBSD builds will use IFUNC, unless --enable-ifunc is specified to configure.

• Added definitions of mask values like LZMA_INDEX_CHECK_MASK_CRC32 to <lzma/index.h>.

• The XZ logo is now included in the Doxygen generated documentation. It is licensed under Creative Commons Attribution-ShareAlike 4.0.

• xz:

• Multithreaded mode is now the default. This improves compression speed and creates .xz files that can be decompressed multithreaded at the cost of increased memory usage and slightly worse compression ratio.

• Added new command line option --filters to set the filter chain using liblzma filter string syntax.

• Added new command line options --filters1 ... --filters9 to set additional filter chains using liblzma filter string syntax. The --block-list option now allows specifying filter chains that were set using these new options.

• Added support for Linux Landlock as a sandboxing method.

• xzdec now supports pledge(2), Capsicum, and Linux Landlock as sandboxing methods.

• Progress indicator time stats remain accurate after pausing xz with SIGTSTP.

• Ported xz and xzdec to Windows MSVC. Visual Studio 2015 or later is required.

• CMake Build:

• Supports pledge(2), Capsicum, and Linux Landlock sandboxing methods.

• Replacement functions for getopt_long() are used on platforms that do not have it.

• Enabled unaligned access by default on PowerPC64LE and on RISC-V targets that define riscv_misaligned_fast.

• Tests:

• Added two new fuzz targets to OSS-Fuzz.

• Implemented Continuous Integration (CI) testing using GitHub Actions.

• Changed quoting style from `...' to '...' in all messages, scripts, and documentation.

• Added basic Codespell support to help catch typo errors.

[Bruce Dubbs]

Fixed at commits

    Update to iana-etc-20240222.
    Update to vim-9.1.0145.
    Update to xz-5.6.0.
    Update to tcl-8.6.14.
    Update to shadow-4.14.6.
    Update to setuptools-69.1.1.
    Update to linux-6.7.7.
    Update to libffi-3.4.6.
    Update to gettext-0.22.5.
    Update to expat-2.6.1.