rsync-3.2.4

[Bruce Dubbs]

New point version.

[Bruce Dubbs]

NEWS for rsync 3.2.4 (15 Apr 2022)

## Changes in this version:

### BEHAVIOR CHANGES:

• A new form of arg protection was added that works similarly to the older [--protect-args](rsync.1#opt) (-s) option but in a way that avoids breaking things like rrsync (the restricted rsync script): rsync now uses backslash escaping for sending "shell-active" characters to the remote shell. This includes spaces, so fetching a remote file via a simple quoted filename value now works by default without any extra quoting:

     ```shell
         rsync -aiv host:'a simple file.pdf' .
     ```
  

Wildcards are not escaped in filename args, but they are escaped in options like the [--suffix](rsync.1#opt) and [--usermap](rsync.1#opt) values. If your rsync script depends on the old arg-splitting behavior, either run it with the [--old-args](rsync.1#opt) option or export RSYNC_OLD_ARGS=1 in the script's environment. See also the [ADVANCED USAGE](rsync.1#) section of rsync's manpage for how to use a more modern arg style.

• A long-standing bug was preventing rsync from figuring out the current locale's decimal point character, which made rsync always output numbers using the "C" locale. Since this is now fixed in 3.2.4, a script that parses rsync's decimal numbers (e.g. from the verbose footer) may want to setup the environment in a way that the output continues to be in the C locale. For instance, one of the following should work fine:

     ```shell
         export LC_ALL=C.UTF-8
     ```
  

  or if iconv translations are needed:

     ```shell
         if [ "${LC_ALL:-}" ]; then
             export LANG="$LC_ALL"
             export LC_CTYPE="$LC_ALL"
             unset LC_ALL
         fi
         export LC_NUMERIC=C.UTF-8
     ```
  

### SECURITY FIXES:

• A fix for CVE-2018-25032 in the bundled zlib (memory corruption issue). Note that BLFS does not use the bundled zlib.

### BUG FIXES:

• Fixed a bug with [--inplace](rsync.1#opt) + [--sparse](rsync.1#opt) (and a lack of [--whole-file](rsync.1#opt)) where the destination file could get reconstructed with bogus data. Since the bug can also be avoided by using (the seemingly redundant) [--no-W](rsync.1#opt) on the receiving side, the latest rsync will now send --no-W to a remote receiver when this option combination occurs. If your client rsync is not new enough to do this for you (or if you're just paranoid), you can manually specify `--no-W -M--no-W (when not using [--whole-file`](rsync.1#opt)) to make sure the bug is avoided.

• Fixed a bug with [--mkpath](rsync.1#opt) if a single-file copy specifies an existing destination dir with a non-existing destination filename.

• Fixed --update -vv to output "is uptodate" instead of "is newer" messages for files that are being skipped due to an identical modify time. (This was a new output quirk in 3.2.3.)

• When doing an append transfer, the sending side's file must not get shorter or it is skipped. Fixes a crash that could occur when the size changes to 0 in the middle of the send negotiations.

• When dealing with special files (see [--specials](rsync.1#opt)) in an alt-dest hierarchy, rsync now checks the non-permission mode bits to ensure that the 2 special files are really the same before hard-linking them together.

• Fixed a bug where [--delay-updates](rsync.1#opt) with stale partial data could cause a file to fail to update.

• Fixed a few places that would output an INFO message with [--info=NAME](rsync.1#opt) that should only have been output given [--verbose](rsync.1#opt) or [--itemize-changes](rsync.1#opt).

• Avoid a weird failure if you run a local copy with a (useless) [--rsh](rsync.1#opt) option that contains a V in the command.

• Fixed a long-standing compression bug where the compression level of the first file transferred affected the level for all future files. Also, the per-file compression skipping has apparently never worked, so it is now documented as being ineffective.

• Fixed a truncate error when a --write-devices copy wrote a file onto a device that was shorter than the device.

• Made --write-devices support both --checksum and --no-whole-file when copying to a device.

• Improved how the [--stop-at](rsync.1#opt), [--stop-after](rsync.1#opt), and (the deprecated) [--time-limit](rsync.1#opt) options check to see if the allowed time is over, which should make rsync exit more consistently.

• Tweak --progress to display "??:??:??" when the time-remaining value is so large as to be meaningless.

• Silence some chmod warnings about symlinks when it looks like we have a function to set their permissions but they can't really be set.

• Fixed a potential issue in git-set-file-times when handling commits with high-bit characters in the description & when handling a description that might mimick the git raw-commit deliniators. (See the support dir.)

• The bundled systemd/rsync.service file now includes Restart=on-failure.

### ENHANCEMENTS:

• Use openssl's -verify_hostname option in the rsync-ssl script.

• Added extra info to the "FILENAME exists" output of [--ignore-existing](rsync.1#opt) when [--info=skip2](rsync.1#opt) is used. The skip message becomes "FILENAME exists (INFO)" where the INFO is one of "type change", "sum change" (requires [--checksum](rsync.1#opt)), "file change" (based on the quick check), "attr change", or "uptodate". Prior versions only supported --info=skip1.

• Added the [--fsync](rsync.1#opt) option (promoted from the patches repo).

• Added the [--copy-devices](rsync.1#opt) option. Compared to the historical version from the rsync-patches repo, this version: properly handles --checksum; fixes a truncation bug when doing an --inplace copy onto a longer file; fixes several bugs in the --itemize output; and only the sending side needs the enhanced rsync for the copy to work.

• Reduced memory usage for an incremental transfer that has a bunch of small directories.

• The rsync daemon can now handle a client address with an implied "%scope" suffix.

• Added support for [--atimes](rsync.1#opt) on macOS and fixed a bug where it wouldn't work without [--times](rsync.1#opt).

• Rsync can now update the xattrs on a read-only file when your user can temporarily add user-write permission to the file. (It always worked for a root transfer.)
• Rsync can now work around an [--inplace](rsync.1#opt) update of a file that is being refused due to the Linux fs.protected_regular sysctl setting.

• When [--chown](rsync.1#opt), [--usermap](rsync.1#opt), or [--groupmap](rsync.1#opt) is specified, rsync now makes sure that the appropriate [--owner](rsync.1#opt) and/or [--group](rsync.1#opt) options are enabled.

• Added the [--info=NONREG](rsync.1#opt) setting to control if rsync should warn about non-regular files in the transfer. This is enabled by default (keeping the behavior the same as before), so specifying --info=nonreg0 can be used to turn the warnings off.

• An optional asm optimization for the rolling checksum from Shark64. Enable it with ./configure --enable-roll-asm.

• Using --debug=FILTER now outputs a caution message if a filter rule has trailing whitespace.

• Transformed rrsync into a python script with improvements:
  • Security has been beefed up.
  • The known rsync options were updated to include recent additions.
  • Make rrsync reject [--copy-links](rsync.1#opt) (-L), [--copy-dirlinks](rsync.1#opt) (-k), & [--keep-dirlinks](rsync.1#opt) (-K) by default to make it harder to exploit any out-of-subdir symlinks.
  • A new rrsync option of [-munge](rrsync.1#opt) tells rrsync to always enable rsync's [--munge-links](rsync.1#opt) option on the server side.
  • A new rrsync option of [-no-lock](rrsync.1#opt) disables a new single-use locking idiom that is the default when [-ro](rrsync.1#opt) is not used (useful with [-munge](rrsync.1#opt)).
  • A new rrsync option of [-no-del](rrsync.1#opt) disables all --remove* and --delete* rsync options on the server side.
  • The log format has been tweaked slightly to add seconds to the timestamp and to output the command executed as a tuple (making the args clearer).
  • An rrsync.1 manpage was added (in the support dir with rrsync).

• Added options to the lsh script to facilitate rrsync testing. (See the support dir.)

• Transformed the atomic-rsync script into a python script and added the ability to ignore one or more non-zero exit codes. By default, it now ignores code 24, the file-vanished exit code. (See the support dir.)

• Transformed the munge-symlinks script into python. (See the support dir.)

• Improved the rsync-no-vanished script to not join stdout & stderr together. (See the support dir.)

• Work around a glibc bug where lchmod() breaks in a chroot w/o /proc mounted.

• Try to support a client that sent a remote rsync a wacko stderr file handle (such as an older File::RsyncP perl library used by BackupPC).

• Lots of manpage improvements, including better HTML versions.

### PACKAGING RELATED:

• Give configure the --with-rrsync option if you want make install to install the (now python3) rrsync script and its new manpage.

• If the rrsync script is installed, its package should be changed to depend on python3 and the (suggested but not mandatory) python3 braceexpand lib.

• When creating a package from a non-release version (w/o a git checkout), the packager can elect to create git-version.h and define RSYNC_GITVER to the string they want --version to output. (The file is still auto-generated using the output of git describe when building inside a non-shallow git checkout, though.)

• Renamed configure's --enable-simd option to --enable-roll-simd and added the option --enable-roll-asm to use the new asm version of the code. Both are x86_64/amd64 only.

• Renamed configure's --enable-asm option to --enable-md5-asm to avoid confusion with the asm option for the rolling checksum. It is also honored even when openssl crypto is in use. This allows: normal MD4 & MD5, normal MD4 + asm MD5, openssl MD4 & MD5, or openssl MD4 + asm MD5 depending on the configure options selected.

• Made SIMD & asm configure checks default to "no" on non-Linux hosts due to various reports of problems on NetBSD & macOS hosts. These were also tweaked to allow enabling the feature on a host_cpu of amd64 (was only allowed on x86_64 before).

• Fixed configure to not fail at the SIMD check when cross-compiling.

• Improved the IPv6 determination in configure.

• Compile the C files with -pedantic-errors (when possible) so that we will get warned if a static initialization overflows in the future (among other things).

• When linking with an external zlib, rsync renames its read_buf() function to read_buf_() to avoid a symbol clash on an unpatched zlib.

• Added a SECURITY.md file.

### DEVELOPER RELATED:

• Made it easier to write rsync tests that diff the output while also checking the status code, and used the idiom to improve the existing tests. (See the checkdiff and checkdiff2 idioms in the testsuite/*.test files.

• The packaging scripts & related python lib got some minor enhancements.

### INTERNAL

• Use setenv() instead of putenv() when it is available.

• Improve the logic in compat.c so that we don't need to try to remember to sprinkle !local_server exceptions throughout the protocol logic.

• One more C99 Flexible Array improvement (started in the last release) and make use of the C99 %zd format string when printing size_t values (when possible).

• Use mallinfo2() instead of mallinfo(), when available.

[Bruce Dubbs]

Fixed at commits

118e8ef57e Update to pcre2-10.40.
c98a330c2a Update to rsync-3.2.4.