curl-8.9.0

[Douglas R. Reno]

New minor version

This release comes with a ton of bugfixes as well as two security advisories

[Douglas R. Reno]

Release Notes:

This release includes the following changes:

 o curl: add --ip-tos (IP Type of Service / Traffic Class) [42]
 o curl: add --mptcp [29]
 o curl: add --vlan-priority [107]
 o curl: add -w '%{num_retries} [65]
 o gnutls: support CA caching [90]
 o mbedtls: support CURLOPT_CERTINFO [116]
 o noproxy: patterns need to be comma separated [75]
 o socket: support binding to interface *AND* IP [80]
 o tcpkeepalive: add CURLOPT_TCP_KEEPCNT and --keepalive-cnt [103]
 o urlapi: add CURLU_NO_GUESS_SCHEME [72]
 o wolfssl: support CA caching [73]

This release includes the following bugfixes:

 o (lib)curl.rc: set debug flag also for `CURLDEBUG` and `UNITTESTS` [2]
 o asyn-thread: avoid using GetAddrInfoExW with impersonation [7]
 o aws-sigv4: url encode the canonical path [55]
 o BINDINGS: update java link to one that exists [115]
 o build: add Debug, TrackMemory, ECH to feature list [218]
 o build: add more supported attributes to the IAR compiler [46]
 o build: fix llvm 16 or older + Xcode 15 or newer, and gcc [240]
 o build: fix llvm 17 and older + macOS SDK 14.4 and newer [230]
 o build: sync warning options between autotools, cmake & compilers [244]
 o build: tidy up `__builtin_available` feature checks (Apple) [241]
 o build: untangle `CURLDEBUG` and `DEBUGBUILD` macros [9]
 o build: use `#error` instead of invalid syntax [212]
 o cd2nroff: convert two warnings to errors [135]
 o cd2nroff: use an empty "##" to signal end of .IP sequence [56]
 o cf-socket: improve SO_SNDBUF update for Winsock [27]
 o cf-socket: optimize curlx_nonblock() and check its return error [151]
 o cf-socket: remove obsolete recvbuf [203]
 o cf-socket: remove two "useless" assignments [238]
 o cfilters: make Curl_conn_connect always assign 'done' [60]
 o cmake: add CURL_USE_GSASL option with detection + CI test [133]
 o cmake: allow `ENABLE_CURLDEBUG=OFF` with `ENABLE_DEBUG=ON` [26]
 o cmake: allow SOVERSION override with `CURL_LIBCURL_SOVERSION` [120]
 o cmake: alpha-sort feature list [161]
 o cmake: always build unit tests with the `testdeps` target [20]
 o cmake: bring `curl-config.cmake` closer to `FindCURL` [130]
 o cmake: create `configurehelp.pm` like autotools does [252]
 o cmake: delete unused `HAVE_LIBSSH2`, `HAVE_LIBSOCKET` macros [251]
 o cmake: detect `libidn2` also via `pkg-config` [239]
 o cmake: enable SOVERSION for Cygwin and `CMAKE_DLL_NAME_WITH_SOVERSION` [119]
 o cmake: fix `-Wredundant-decls` in unity/mingw-w64 builds [15]
 o cmake: fix brotli lib order [3]
 o cmake: fix building `unit1600` due to missing `ssl/openssl.h` [222]
 o cmake: fix building in unity mode [4]
 o cmake: fix building with both md4 and md5 in unity mode [13]
 o cmake: fix builds with detected libidn2 lib but undetected header [221]
 o cmake: fix feature and protocol lists for SecureTransport [194]
 o cmake: fix quotes when appending multiple options (SecureTransport) [139]
 o cmake: fix test 1013 with websockets enabled and no TLS [47]
 o cmake: improve wolfSSL detection [190]
 o cmake: show protocols, then features [180]
 o cmake: stop setting SOVERSION for the static lib target [127]
 o cmake: sync CA bundle/path detection with autotools [253]
 o cmake: sync protocol/feature list with `curl -V` output [182]
 o cmake: use `APPLE` instead of `CMAKE_SYSTEM_NAME` string [24]
 o cmake: whitespace, formatting/tidy-up in comments [25]
 o cmdline-docs: "added in" cleanups [171]
 o cmdline-docs: fix `--proxy-ca-native` example + tidy-ups [181]
 o cmdline-opts/_PROTOCOLS.md: mention WS(S) [94]
 o cmdline-opts/ech.md: shorten the help text [93]
 o cmdline-opts/fail.md: expand and clarify [95]
 o cmdline-opts/interface.md: expand the documentation [66]
 o cmdline-opts: category cleanup [196]
 o cmdline-opts: expand the parallel explanations [98]
 o cmdline-opts: shorten six help texts [178]
 o cmdline: expand proxy option explanations [97]
 o code: language cleanup in comments [186]
 o configure: CA bundle/path detection fixes [254]
 o configure: fix `SystemConfiguration` detection [243]
 o configure: fix pkg-config library name 'libnghttp3' [138]
 o configure: fix pkg-config names (zstd, ngtcp2*) [170]
 o configure: limit `SystemConfiguration` test to non-c-ares, IPv6 builds [242]
 o configure: remove 'deeper' checks for `AC_CHECK_FUNCS` [23]
 o configure: require a QUIC library if nghttp3 is used [142]
 o configure: sort feature list, lowercase protocols, use backticks [206]
 o configure: use `$EGREP` in place of `grep -E` [41]
 o configure: use AC_MSG_WARN for TLS/experimental warning texts [122]
 o connect-to.md: expand with examples [147]
 o connection: shutdown TLS (for FTP) better [104]
 o cookie-jar.md: see also --junk-session-cookies [144]
 o curl-config: revert to backticks to support old target envs [88]
 o curl: allow etag and content-disposition for 3xx reply [117]
 o curl: bsearch the --write-out variable name [102]
 o curl: check for --disable case *sensitively* [199]
 o curl: list categories in --help [219]
 o curl: make warnings and other messages aware of terminal width [58]
 o curl: output "flying saucers" with leading carriage return [121]
 o curl_easy_escape: elaborate a little on encoding a URL [193]
 o curl_mprintf.md: add missing comma
 o curl_multi_poll.md: expand the example with an custom file descriptor [21]
 o curl_str[n]equal.md: tidy up text to make them stand-alone [195]
 o curl_url_set.md: libcurl only parses :// URLs [48]
 o curl_url_set: elaborate on scheme guessing [191]
 o curldown: make 'added-in:' a mandatory header field [226]
 o CURLOPT_CONNECTTIMEOUT*: clarify, document the milliseond version [105]
 o CURLOPT_ECH.md: remove repeated 'if' [109]
 o CURLOPT_NETRC.md: clarify what it does on Windows [140]
 o CURLOPT_RESOLVE.md: mention hostname can be wildcard ('*') [150]
 o CURLOPT_SSL_VERIFYHOST.md: refresh [224]
 o CURLOPT_TLSAUTH_PASSWORD/USERNAME.md: language fixups [155]
 o DISTROS: add a link to the list archive [22]
 o DISTROS: add AlmaLinux package source link
 o DISTROS: add MSYS2 (native) links [100]
 o docs/cmdline-opts: fix mail-auth example TLD typo [35]
 o docs/cmdline-opts: remove two superfluous "Added in" mentions [143]
 o docs/libcurl: polish the single-line descriptions [159]
 o docs/Makefile.am: make curl-config.1 install [14]
 o docs: reference non deprecated libcurl options [113]
 o docs: start markdown headers with capital letter where applicable [236]
 o doh-insecure.md: expand [96]
 o doh: fix cleanup [228]
 o doh: fix leak and zero-length HTTPS RR crash [227]
 o dump-header.md: mention minus for stdout [149]
 o examples/threaded-ssl: remove locking callback code [83]
 o examples: add missing binaries to .gitignore [106]
 o examples: delete unused includes [10]
 o examples: fix compiling with MSVC [34]
 o examples: suppress deprecation warnings locally [211]
 o FEATURES.md: refresh [208]
 o file: separate fake headers and body with a stand-alone CRLF [137]
 o ftp: remove redundant null pointer check in loop condition [256]
 o get.d: clarify the explanation [32]
 o GHA/windows: add MSVC wolfSSL job with test [250]
 o GHA/windows: ignore FTP test results for old-mingw-w64
 o GHA: add MSVC UWP job, expand jobs with more options [216]
 o GHA: detect and warn for more English contractions [123]
 o GHA: disable MQTT and WebSocket tests in Windows jobs [63]
 o GHA: disable TFTP tests in Windows jobs
 o GHA: enable tests 1139, 1177, 1477 on Windows [59]
 o GHA: improve vcpkg cache, add BoringSSL ECH and LibreSSL MSVC jobs [215]
 o GHA: unify http3 workflows into one [77]
 o GHA: use vcpkg to install packages for MSVC jobs [145]
 o GIT-INFO.md: remove version requirements [209]
 o gnutls: improve TLS shutdown [62]
 o gnutls: pass in SNI name, not hostname when checking cert [114]
 o help: add flags to output and ssh categories [202]
 o hostip: skip error check for infallible function call [237]
 o http/3: add shutdown support [154]
 o http/3: resume upload on ack if we have more data to send [232]
 o http: remove "struct HTTP" [134]
 o http: write last header line late [44]
 o idn: fix ß with AppleIDN [220]
 o idn: make macidn fail before trying conversion if name too long [235]
 o idn: tweak buffer use when converting with macidn [245]
 o lib/v*: tidy up types and casts [64]
 o lib: add a few DEBUGASSERT(data) to aid code analyzers [187]
 o lib: add failure reason on bind errors [247]
 o lib: fix gcc warning in certain debug builds [19]
 o lib: fix thread entry point to return `DWORD` on WinCE [85]
 o lib: graceful connection shutdown [162]
 o lib: prefer `var = time(NULL)` over `time(&var)` [52]
 o lib: tidy up types and casts [92]
 o lib: xfer_setup and non-blocking shutdown [111]
 o libcurl-docs: make option lists alpha-sorted [214]
 o libcurl-easy.md: now *more* than 300 options [233]
 o libcurl.pc: add `Requires.private`, `Requires` for static linking [129]
 o libcurl.pc: add more `Requires.private`/`Requires` dependencies [189]
 o libssh: remove CURLOPT_SSL_VERIFYHOST check [36]
 o macos: add workaround for gcc, non-c-ares, IPv6, compile error [213]
 o macos: undo `availability` macro enabled by Homebrew gcc [231]
 o managen: "added in" fixes [131]
 o managen: cleanups to generate nicer-looking output [141]
 o managen: error on trailing blank lines in input files [165]
 o managen: fix removing backticks from subtitles [179]
 o managen: insert final .fi for files ending with a quote [174]
 o managen: introduce "Multi: per-URL" [176]
 o managen: only output .RE for manpage output [156]
 o managen: output tabs for each 8 leading spaces [164]
 o managen: warn on excessively long help texts [87]
 o MANUAL.md: wrap two example urls that overrun styling [234]
 o mbedtls: check version before getting tls version [261]
 o mbedtls: check version for cipher id [12]
 o mbedtls: correct the error message for cert blob parsing failure [225]
 o mbedtls: send close-notify on close [11]
 o mbedtls: v3.6.0 workarounds [89]
 o md4: fix compilation with OpenSSL 1.x with md4 disabled [255]
 o misc: fix typos [108]
 o mk-ca-bundle.pl: delay 'curl -V' execution until it is needed [168]
 o multi: add multi->proto_hash, a key-value store for protocol data [37]
 o multi: do a final progress update on connect failure [248]
 o multi: fix multi_wait() timeout handling [51]
 o multi: fix pollset during RESOLVING phase [166]
 o multi: multi_getsock(), check correct socket [167]
 o ngtcp2+quictls: fix cert-status use [173]
 o noproxy: test bad ipv6 net size first [82]
 o openssl/gnutls: rectify the TLS version checks for QUIC [61]
 o openssl: fix %-specifier in infof() call [57]
 o openssl: fix hostname handling when using ECH [78]
 o openssl: stop duplicate ssl key logging for legacy OpenSSL [49]
 o os400: make it compilable again [128]
 o pytest: add ftp upload tests [16]
 o pytest: include testenv/vsftpd.py in dist tarball [99]
 o quic: enable UDP GRO [157]
 o quic: openssl quic, cmake and doc version update to 3.3.0 [148]
 o quic: require at least OpenSSL 3.3 for QUIC [158]
 o quic: update to quiche 0.22.0 [175]
 o quiche: fix operand of ‘?:’ changes signedness [177]
 o request.md: language fix [70]
 o request: change the struct field bodywrites to a bool, only for hyper [132]
 o reuse: switch to REUSE 3.2 and REUSE.toml [184]
 o runtests: show name and keywords for failed tests in summary [249]
 o runtests: sort test IDs in summary lines [33]
 o runtests: support %DATE for YYYY-MM-DD of right now
 o runtests: support %VERNUM
 o runtests: support crlf="yes" for the <stderr> section
 o sectransp: fix `HAVE_BUILTIN_AVAILABLE` checks to not emit warnings [210]
 o sectransp: fix clang compiler warnings, stop silencing them [223]
 o sectransp: remove large cipher table [76]
 o sectransp: use common code for cipher suite lookup [54]
 o sendf: fix CRLF conversion of input [258]
 o smtp: for starttls, do full upgrade [260]
 o socket: change TCP keepalive from ms to seconds on DragonFly BSD [74]
 o socket: use SOCK_NONBLOCK to eliminate extra system call [86]
 o socketpair: add `eventfd` and use `SOCK_NONBLOCK` for `socketpair()` [81]
 o src/Makefile.am: remove SUBDIRS assignment [172]
 o system_win32: add missing curl.h include [160]
 o tcpkeepalive: support TCP keep-alive parameters on Solaris <11.4 [91]
 o test1119: adapt for `.md` input [204]
 o test1139: scan .md files instead of .3 ones [197]
 o test1175: scan libcurl-errors.md, not the generated .3 version [188]
 o test1486: verify that write-out.md and tool_writeout.c are in sync [112]
 o test2600: disable on win32 [259]
 o test: add test1484, for HEAD with content [18]
 o test: add test1546, chunked not last transfer encoding [17]
 o tests/scripts: call it 'manpage' (single word) [229]
 o tests: add pytest for --ciphers and --tls13-ciphers options [38]
 o tests: delete `CharConv` remains [201]
 o tests: delete redundant `!MSDOS` guard [84]
 o tests: extend user/password parsing test1620 [40]
 o tests: fix sshd IdentityFile path for MinGW/Cygwin [217]
 o tests: fix sshd UserKnownHostsFile path for MinGW/Cygwin
 o tests: include current directory when running test Perl commands [205]
 o tests: log "Throwing away" messages before throwing away
 o tests: run with "--trace-config all" to provide even more info [6]
 o tests: sync feature names with `curl -V` [257]
 o tests: test_17_ssl_use.py clarify mbedTLS TLSv1.3 support [43]
 o tests: use exec when spawning nghttpx [45]
 o tidy-up: use consistent casing for Windows directories [28]
 o TODO: remove some old, clarify, add something [31]
 o tool_cb_hdr: return error for failed header writes [30]
 o tool_operate: avoid explicitly setting verifypeer to 1 [39]
 o tool_operate: simplify return code handling from url_proto() [198]
 o tool_writeout: get certinfo only when needing it [101]
 o trace-ascii.md: mention "%" for stderr [146]
 o transfer: avoid polling socket every transfer loop [200]
 o transfer: conn close on paused upload [8]
 o transfer: do not use EXPIRE_NOW while blocked [124]
 o transfer: remove curl_upload_refill_watermark, no longer used [50]
 o transfer: set CSELECT_IN if there is data pending [118]
 o unit2604: use 'unitfail' instead of 'error' variable [153]
 o url: allow DoH transfers to override max connection limit [68]
 o urlapi: remove unused definition of HOST_BAD [262]
 o variable.md: make example use expand [207]
 o verify-synopsis.pl: work with .md files [185]
 o vms: fixed language in comment [110]
 o vtls: deprioritize Secure Transport [71]
 o vtls: replace addsessionid with set_sessionid [183]
 o winbuild: fix PE version info debug flag [1]
 o winbuild: MS-DOS batch tidy-ups [163]
 o winbuild: remove outdated WIN32 defines [5]
 o windows: fix UWP builds, add GHA job [79]
 o winsock: move SO_SNDBUF update into cf-socket [53]
 o wolfssl: assume key_file equal to clientcert if no key_file [169]
 o wolfssl: use larger error buffer when formatting errors [246]
 o x509asn1: add some common ECDSA OIDs [67]
 o x509asn1: ASN1tostr() should fail when 'constructed' is set [125]
 o x509asn1: fallback to dotted OID representation [69]
 o x509asn1: make Curl_extract_certinfo store error message [136]
 o x509asn1: prevent NULL dereference [152]
 o x509asn1: remove superfluous free()
 o x509asn1: remove two static variables [126] 

CVE-2024-6197

freeing stack buffer in utf8asn1str
===================================

Project curl Security Advisory, July 24th 2024 -
[Permalink](https://curl.se/docs/CVE-2024-6197.html)

VULNERABILITY
-------------

libcurl's ASN1 parser has this utf8asn1str() function used for parsing an
ASN.1 UTF-8 string. It can detect an invalid field and return error.
Unfortunately, when doing so it also invokes `free()` on a 4 byte local stack
buffer.

Most modern malloc implementations detect this error and immediately abort.
Some however accept the input pointer and add that memory to its list of
available chunks. This leads to the overwriting of nearby stack memory. The
content of the overwrite is decided by the `free()` implementation; likely to
be memory pointers and a set of flags.

The most likely outcome of exploting this flaw is a crash, although it cannot
be ruled out that more serious results can be had in special circumstances.

INFO
----

The vulnerable code path can be triggered by a malicious server offering an
especially crafted TLS certificate.

This bug was introduced in a code refactor shipped in the curl 8.6.0 release
and is considered a *C mistake* (likely to have been avoided had we not been
using C).

This flaw also affects the curl command line tool.

The Common Vulnerabilities and Exposures (CVE) project has assigned the name
CVE-2024-6197 to this issue.

CWE-590: Free of Memory not on the Heap

Severity: Medium

AFFECTED VERSIONS
-----------------

The vulnerable code can only be reached when curl is built to use GnuTLS,
wolfSSL, Schannel or Secure Transport. Builds using other TLS backends are not
vulnerable.

- Affected versions: curl 8.6.0 to and including 8.8.0
- Not affected versions: curl < 8.6.0 and >= 8.9.0
- Introduced-in: https://github.com/curl/curl/commit/623c3a8fa0bdb2751f1

libcurl is used by many applications, but not always advertised as such!

SOLUTION
------------

- Fixed-in: https://github.com/curl/curl/commit/3a537a4db9e65e545

RECOMMENDATIONS
---------------

We suggest you take one of the following actions immediately, in order of
preference:

 A - Upgrade curl and libcurl to version 8.9.0

 B - Apply the patch to your version and rebuild

 C - Build your libcurl with an unaffected TLS backend 

The other security fix is Macintosh specific

[Douglas R. Reno]

Fixed at 8d57f6d4732b866a4a60435ad78aa09def5c0bf4

SA-12.1-081 issued