curl-7.74.0

[Douglas R. Reno]

New minor version

[Douglas R. Reno]

curl and libcurl 7.74.0

 Public curl releases:         196
 Command line options:         235
 curl_easy_setopt() options:   284
 Public functions in libcurl:  85
 Contributors:                 2287

This release includes the following changes:

 o hsts: add experimental support for Strict-Transport-Security [37]

This release includes the following bugfixes:

 o CVE-2020-8286: Inferior OCSP verification [93]
 o CVE-2020-8285: FTP wildcard stack overflow [95]
 o CVE-2020-8284: trusting FTP PASV responses [97]
 o acinclude: detect manually set minimum macos/ipod version [46]
 o alt-svc: enable (in the build) by default [20]
 o alt-svc: minimize variable scope and avoid "DEAD_STORE" [51]
 o asyn: use 'struct thread_data *' instead of 'void *' [84]
 o checksrc: warn on empty line before open brace [13]
 o CI/appveyor: disable test 571 in two cmake builds [22]
 o CI/azure: improve on flakiness by avoiding libtool wrappers [7]
 o CI/tests: enable test target on TravisCI for CMake builds [38]
 o CI/travis: add brotli and zstd to the libssh2 build [27]
 o cirrus: build with FreeBSD 12.2 in CirrusCI [80]
 o cmake: call the feature unixsockets without dash [26]
 o cmake: check for linux/tcp.h [91]
 o cmake: correctly handle linker flags for static libs [52]
 o cmake: don't pass -fvisibility=hidden to clang-cl on Windows [53]
 o cmake: don't use reserved target name 'test' [79]
 o cmake: make BUILD_TESTING dependent option [30]
 o cmake: make CURL_ZLIB a tri-state variable [70]
 o cmake: set the unicode feature in curl-config on Windows [23]
 o cmake: store IDN2 information in curl_config.h [25]
 o cmake: use libcurl.rc in all Windows builds [69]
 o configure: pass -pthread to Libs.private for pkg-config [50]
 o configure: use pkgconfig to find openSSL when cross-compiling [28]
 o connect: repair build without ipv6 availability [19]
 o curl.1: add an "OUTPUT" section at the top of the manpage [32]
 o curl.se: new home [59]
 o curl: add compatibility for Amiga and GCC 6.5 [61]
 o curl: only warn not fail, if not finding the home dir [15]
 o curl_easy_escape: limit output string length to 3 * max input [55]
 o Curl_pgrsStartNow: init speed limit time stamps at start [48]
 o curl_setup: USE_RESOLVE_ON_IPS is for Apple native resolver use
 o curl_url_set.3: fix typo in the RETURN VALUE section [3]
 o CURLOPT_DNS_USE_GLOBAL_CACHE.3: fix typo [34]
 o CURLOPT_HSTS.3: document the file format [82]
 o CURLOPT_NOBODY.3: fix typo [6]
 o CURLOPT_TCP_NODELAY.3: fix comment in example code [8]
 o CURLOPT_URL.3: clarify SCP/SFTP URLs are for uploads as well
 o docs: document the 8MB input string limit [57]
 o docs: fix typos and markup in ETag manpage sections [87]
 o docs: Fix various typos in documentation [58]
 o examples/httpput: remove use of CURLOPT_PUT [39]
 o FAQ: refreshed [56]
 o file: avoid duplicated code sequence [77]
 o ftp: retry getpeername for FTP with TCP_FASTOPEN [100]
 o gnutls: fix memory leaks (certfields memory wasn't released) [41]
 o header.d: mention the "Transfer-Encoding: chunked" handling [45]
 o HISTORY: the new domain
 o http3: fix two build errors, silence warnings [10]
 o http3: use the master branch of GnuTLS for testing [88]
 o http: pass correct header size to debug callback for chunked post [44]
 o http_proxy: use enum with state names for 'keepon' [54]
 o httpput-postfields.c: new example doing PUT with POSTFIELDS [35]
 o infof/failf calls: fix format specifiers [78]
 o libssh2: fix build with disabled proxy support [17]
 o libssh2: fix transport over HTTPS proxy [31]
 o libssh2: require version 1.0 or later [24]
 o Makefile.m32: add support for HTTP/3 via ngtcp2+nghttp3 [11]
 o Makefile.m32: add support for UNICODE builds [85]
 o mqttd: fclose test file when done [60]
 o NEW-PROTOCOL: document what needs to be done to add one [92]
 o ngtcp2: adapt to recent nghttp3 updates [49]
 o ngtcp2: advertise h3 ALPN unconditionally [72]
 o ngtcp2: Fix build error due to symbol name change [90]
 o ngtcp2: use the minimal version of QUIC supported by ngtcp2 [67]
 o ntlm: avoid malloc(0) on zero length user and domain [96]
 o openssl: acknowledge SRP disabling in configure properly [9]
 o openssl: free mem_buf in error path [94]
 o openssl: guard against OOM on context creation [68]
 o openssl: use OPENSSL_init_ssl() with >= 1.1.0 [66]
 o os400: Sync libcurl API options [5]
 o packages/OS400: make the source code-style compliant [4]
 o quiche: close the connection [89]
 o quiche: remove 'static' from local buffer [71]
 o range.d: clarify that curl will not parse multipart responses [36]
 o range.d: fix typo
 o Revert "multi: implement wait using winsock events" [99]
 o rtsp: error out on empty Session ID, unified the code
 o rtsp: fixed Session ID comparison to refuse prefix [65]
 o rtsp: fixed the RTST Session ID mismatch in test 570 [64]
 o runtests: return error if no tests ran [16]
 o runtests: revert the mistaken edit of $CURL
 o runtests: show keywords when no tests ran [33]
 o scripts/completion.pl: parse all opts [101]
 o socks: check for DNS entries with the right port number [74]
 o src/tool_filetime: disable -Wformat on mingw for this file [2]
 o strerror: use 'const' as the string should never be modified [18]
 o test122[12]: remove these two tests [1]
 o test506: make it not run in c-ares builds [75]
 o tests/*server.py: close log file after each log line [81]
 o tests/server/tftpd.c: close upload file right after transfer [62]
 o tests/util.py: fix compatibility with Python 2 [83]
 o tests: add missing global_init/cleanup calls [42]
 o tests: fix some http/2 tests for older versions of nghttpx [47]
 o tool_debug_cb: do not assume zero-terminated data
 o tool_help: make "output" description less confusing [21]
 o tool_operate: --retry for HTTP 408 responses too [43]
 o tool_operate: bail out proper on errors during parallel transfers [29]
 o tool_operate: fix compiler warning when --libcurl is disabled [12]
 o tool_writeout: use off_t getinfo-types instead of doubles [76]
 o travis: use ninja-build for CMake builds [63]
 o travis: use valgrind when running tests for debug builds [40]
 o urlapi: don't accept blank port number field without scheme [98]
 o urlapi: URL encode a '+' in the query part [14]
 o urldata: remove 'void *protop' and create the union 'p' [86]
 o vquic/ngtcp2.h: define local_addr as sockaddr_storage [73]

[Douglas R. Reno]

cURL Security Advisory for CVE-2020-8284

trusting FTP PASV responses
===========================

Project curl Security Advisory, December 9th 2020 -
[Permalink](https://curl.se/docs/CVE-2020-8284.html)

VULNERABILITY
-------------

When curl performs a passive FTP transfer, it first tries the `EPSV` command
and if that is not supported, it falls back to using `PASV`.  Passive mode is
what curl uses by default.

A server response to a `PASV` command includes the (IPv4) address and port
number for the client to connect back to in order to perform the actual data
transfer.

This is how the FTP protocol is designed to work.

A malicious server can use the `PASV` response to trick curl into connecting
back to a given IP address and port, and this way potentially make curl
extract information about services that are otherwise private and not
disclosed, for example doing port scanning and service banner extractions.

If curl operates on a URL provided by a user (which by all means is an unwise
setup), a user can exploit that and pass in a URL to a malicious FTP server
instance without needing any server breach to perform the attack.

We are not aware of any exploit of this flaw.

INFO
----

This issue has existed in curl for as long as FTP has been supported, since
day 1.

The flaw only exists for IPv4 since `PASV` doesn't work for IPv6 and curl will
prefer `EPSV`. The passive mode setup for FTP is used for both uploads and
downloads.

curl can be built without FTP support and applications can explicitly disable
FTP for single transfers.

curl users could already mitigate this flaw with `CURLOPT_FTP_SKIP_PASV_IP`
and `--ftp-skip-pasv-ip`.

Other FTP clients have in the past also had this flaw and have fixed it at
different points in time. Firefox fixed it in 2007: CVE-2007-1562.

The Common Vulnerabilities and Exposures (CVE) project has assigned the name
CVE-2020-8284 to this issue.

CWE-200: Exposure of Sensitive Information to an Unauthorized Actor

Severity: Low

AFFECTED VERSIONS
-----------------

- Affected versions: curl 4.0 to and including 7.73.0
- Not affected versions: curl >= 7.74.0

Also note that (lib)curl is used by many applications, and not always
advertised as such.

THE SOLUTION
------------

The IP address part of the response is now ignored by default, by making
`CURLOPT_FTP_SKIP_PASV_IP` default to `1L` instead of previously being `0L`.

This has the minor drawback that a small fraction of use cases might break,
when a server truly needs the client to connect back to a different IP address
than what the control connection uses and for those `CURLOPT_FTP_SKIP_PASV_IP`
can be set to `0L`.

The same goes for the command line tool, which then might need
`--no-ftp-skip-pasv-ip` set to prevent curl from ignoring the address in the
server response.

A [fix for CVE-2020-8284](https://github.com/curl/curl/commit/ec9cc725d598ac)

RECOMMENDATIONS
--------------

We suggest you take one of the following actions immediately, in order of
preference:

 A - Upgrade curl to version 7.74.0

 B - Set `CURLOPT_FTP_SKIP_PASV_IP` to `1L` or use `--ftp-skip-pasv-ip`

 C - Disable FTP availability for your transfers

TIMELINE
--------

This issue was first reported to the curl project on November 21, 2020.

This advisory was posted on December 9th 2020.

[Douglas R. Reno]

cURL Security Advisory for CVE-2020-8285

FTP wildcard stack overflow
===========================

Project curl Security Advisory, December 9th 2020 -
[Permalink](https://curl.se/docs/CVE-2020-8285.html)

VULNERABILITY
-------------

libcurl offers a wildcard matching functionality, which allows a callback (set
with `CURLOPT_CHUNK_BGN_FUNCTION`) to return information back to libcurl on
how to handle a specific entry in a directory when libcurl iterates over a
list of all available entries.

When this callback returns `CURL_CHUNK_BGN_FUNC_SKIP`, to tell libcurl to not
deal with that file, the internal function in libcurl then calls itself
recursively to handle the next directory entry.

If there's a sufficient amount of file entries and if the callback returns
"skip" enough number of times, libcurl runs out of stack space. The exact
amount will of course vary with platforms, compilers and other environmental
factors.

The content of the remote directory is not kept on the stack, so it seems hard
for the attacker to control exactly what data that overwrites the stack -
however it remains a Denial-Of-Service vector as a malicious user who controls
a server that a libcurl-using application works with under these premises can
trigger a crash.

(There is also a few other ways the function can be made to call itself and
trigger this problem.)

We are not aware of any exploit of this flaw.

INFO
----

This issue was unfortunately reported publicly in the curl GitHub issue
tracker as [issue 6255](https://github.com/curl/curl/issues/6255).

This flaw has existed in curl since commit
[0825cd80a](https://github.com/curl/curl/commit/0825cd80a) in curl 7.21.0.

This functionality is not used by the curl tool so it is not affected.
Further: it is not a very widely used feature.

The Common Vulnerabilities and Exposures (CVE) project has assigned the name
CVE-2020-8285 to this issue.

CWE-674: Uncontrolled Recursion

Severity: Medium

AFFECTED VERSIONS
-----------------

- Affected versions: libcurl 7.21.0 to and including 7.73.0
- Not affected versions: libcurl < 7.21.0 and libcurl >= 7.74.0

Also note that libcurl is used by many applications, and not always
advertised as such.

THE SOLUTION
------------

The internal function is rewritten to instead and more appropriately use an
ordinary loop instead of the recursive approach. This way, the stack use will
remain the same no matter how many files that are skipped.

A [fix for CVE-2020-8285](https://github.com/curl/curl/commit/69a358f2186e04)

RECOMMENDATIONS
--------------

We suggest you take one of the following actions immediately, in order of
preference:

 A - Upgrade curl to version 7.74.0

 B - Disable FTP wildcard use (`CURLOPT_WILDCARDMATCH`)

 C - Make sure your `CURLOPT_CHUNK_BGN_FUNCTION` callback doesn't do multiple skips.

TIMELINE
--------

This issue was first reported to the curl project on November 27, 2020.

This advisory was posted on December 9th 2020.

[Douglas R. Reno]

cURL Security Advisory for CVE-2020-8286

Inferior OCSP verification
==========================

Project curl Security Advisory, December 9th 2020 -
[Permalink](https://curl.se/docs/CVE-2020-8286.html)

VULNERABILITY
-------------

libcurl offers "OCSP stapling" via the `CURLOPT_SSL_VERIFYSTATUS` option. When
set, libcurl verifies the OCSP response that a server responds with as part of
the TLS handshake. It then aborts the TLS negotiation if something is wrong
with the response. The same feature can be enabled with `--cert-status` using
the curl tool.

As part of the OCSP response verification, a client should verify that the
response is indeed set out for the correct certificate. This step was not
performed by libcurl when built or told to use OpenSSL as TLS backend.

This flaw would allow an attacker, who perhaps could have breached a TLS
server, to provide a fraudulent OCSP response that would appear fine, instead
of the real one. Like if the original certificate actually has been revoked.

We are not aware of any exploit of this flaw.

INFO
----

This flaw has existed in curl since commit
[d1cf5d570663d](https://github.com/curl/curl/commit/d1cf5d570663d) in curl
7.41.0.

The vulnerability is present only if OpenSSL is the designated TLS backend.
OCSP stapling is not enabled by default by libcurl, it needs to be explicitly
enabled by the application to get used.

OCSP Stapling can be used with any of the TLS based protocols curl supports,
including HTTPS, FTPS, SMTPS, POP3S, IMAPS, HTTPS-proxy and more.

The Common Vulnerabilities and Exposures (CVE) project has assigned the name
CVE-2020-8286 to this issue.

CWE-299: Improper Check for Certificate Revocation

Severity: Medium

AFFECTED VERSIONS
-----------------

- Affected versions: libcurl 7.41.0 to and including 7.73.0
- Not affected versions: libcurl < 7.41.0 and libcurl >= 7.74.0

Also note that libcurl is used by many applications, and not always
advertised as such.

THE SOLUTION
------------

The OCSP response checker function now also verifies that the certificate id
is the correct one.

A [fix for CVE-2020-8286](https://github.com/curl/curl/commit/d9d01672785b)

RECOMMENDATIONS
--------------

We suggest you take one of the following actions immediately, in order of
preference:

 A - Upgrade curl to version 7.74.0

 B - Don't rely on OCSP

TIMELINE
--------

This issue was first reported to the curl project on December 2, 2020.

This advisory was posted on December 9th 2020.

[Douglas R. Reno]

We need to adjust the tests due to the removal of our python symlink. The following test servers are affected:

renodr [ /sources/curl-7.74.0/curl-7.74.0/tests ]$ grep -r "/usr/bin/env python" *
dictserver.py:#!/usr/bin/env python
negtelnetserver.py:#!/usr/bin/env python
smbserver.py:#!/usr/bin/env python
util.py:#!/usr/bin/env python

I'm going to put the following 'sed' in to make this work again:

grep -rl '^{#!.*python$' | xargs sed -i '1s/python/&3/'}

[Douglas R. Reno]

Fixed at r23975