curl-7.19.7 doesn't find the BLFS-ca-bundle

[Trent Shea]

Following the openssl setup curl doesn't find the certificates:

checking default CA cert bundle/path... no

It looks like this can be worked around a number of ways: --with-ca-bundle=/etc/ssl/ca-bundle.crt, --with-ca-path=/etc/ssl/certs, or a link /etc/ssl/certs/ca-certificates.crt -> ../ca-bundle.crt

The configure results look good from any of the above, but I don't know which is most appropriate.

[DJ Lucas]

The problem with the link is that you'd have a duplicate chain. If you were to add a cert to the directory and the ca-bundle.crt is not updated, then you don't pick up on it. One switch or the other is sufficient, personally, I prefer having individual certificates in the directory, however some application(s) (and unfortunately I don't recall which one(s)) needed the single file, which is why we do both in separate directories. Lets use '--with-ca-path=/etc/ssl/certs' on this one.

[Randy McMurchy]

I updated BLFS to cURL-7.20.0 and the problem does not exist. Here is the relevant output from the cURL configure.log. I didn't do anything special during the cURL or SSL installations:

checking default CA cert bundle/path... /etc/ssl/certs/ (capath)

Closing the ticket.

[Trent Shea]

contents of /etc/ssl

[Trent Shea]

7.20.0 still gives me: checking default CA cert bundle/path... no The only optional dependency I have installed is openssl. I've attached the contents of /etc/ssl maybe I missed something.

[Ag. Hatzimanikas]

I'm getting the same with Trent, even with 7.20.0.

The proper way to fix this is to use "--with-ca-bundle=/etc/ssl/ca-bundle.crt".

Dj we can't use "--with-ca-path=/etc/ssl/certs", because it doesn't work when linking curl with gnutls, see:

as_fn_error "--with-ca-path only works with openSSL" "$LINENO" 5

We can also (probably) fix this, with a sed the loop in the configure script, where it searches for certs.

Anyway, it doesn't look an important error, because it can by-passed:

• while using the curl utility in runtime, with a number of command line options

• while using libcurl bindings by setting for example the CURLOPT_CAINFO or the CURLOPT_CAPATH options

[Randy McMurchy]

I don't know what you guys are doing different than I (wrong?), but it works perfect for me without anything. Just FBBG and cURL finds the cert bundle just fine.

You obviously can see from the output of my configure that it is picking it up, right?

Not sure what to do. Works for me. Can you tell me a way to make cURL actually *use* the cert bundle in some operation?

[Ag. Hatzimanikas]

Try:

curl ​https://fedorahosted.org/releases/x/m/xmlto/xmlto-0.0.23.tar.bz2

You will probably take an error, unless in your case curl hardcoded with a way the path to the cert bundle.

Now (if you got the error) try again with:

curl --cacert /etc/ssl/ca-bundle.crt -O ​https://fedorahosted.org/releases/x/m/xmlto/xmlto-0.0.23.tar.bz2

I think the right thing to do is to provide the optional option "--with-ca-bundle=/etc/ssl/ca-bundle.crt", if a user is linking curl with openssl/gnutls.

In the case of gnutls however, we'll have to tell the reader with a way, to install the ca-bundle.crt, with a similar way as in the case of openssl, e.g.,

tar -vxf ../BLFS-ca-bundle-3.12.5.tar.bz2
for pem in *pem; do                                                                              
cat $pem
   echo ""
done > /etc/ssl/ca-bundle.crt

I really can't think right now, how this text can be written with a clean way, unless (maybe) we dedicate a page for the certificates installation, then it will be easy.

[Trent Shea]

gets even better

if test -d "$a" && ls "$a"/[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f].0 >/dev/null 2>/dev/null; then

capath="$a"

mkdir /etc/ssl/certs/12345678.0

and I'll get

ca cert path: /etc/ssl/certs/

I'm pretty sure I didn't miss anything with openssl instructions, but I do notice that ubuntu has these 2468acdf.0 soft links, thoughts anyone?

[Trent Shea]

Replying to trent.shea:

I'm pretty sure I didn't miss anything with openssl instructions, but I do notice that ubuntu has these 2468acdf.0 soft links, thoughts anyone?

I've made a bit of progress, but I have _no_ experience with this...

From ssleay.txt I'll quote:

There is also a link from the file a8556381.0 to eay1024.PEM. The value a8556381 is returned by 'x509 -hash -noout <eay1024.pem' and is the value used by X509 verification routines to 'find' this certificte when search a directory for it.

if i run

openssl x509 -hash -noout <23f4c490.pem

I get

23f4c490

It looks like this may have nothing to do with curl now. This ticket should be closed and a openssl one started?

[DJ Lucas]

Trent, we simply do not use the same policy as Debian for certificates. We do our best by trusting the folks at Mozilla.org to create a minimum set. The certificate with the hash value of 2468acdf is not trusted by the folks at Mozilla.org (and not by BLFS because of our choice to follow Mozilla devs for this). Open up the file in an editor and see who it belongs to. If you trust it, great, add it to your system and rerun the commands at the end of the OpenSSL page.

Anyone can add additional certificates, for instance, I add StartCom's intermediate certificates to my systems as I do use the free certificates, but I don't feel that it is appropriate to define policy for BLFS to use them (it'd probably be dangerous to some extent even). The crew at Mozilla.org does a very thorough verification/investigation process, that we take advantage of. Seems the problem is linking with gnutls as it works just fine with OpenSSL, and that the certificates should be installed from their own book page. There should already be one easily retrievable from SVN some time back as it was decided to simply tack it onto the OpenSSL page. I was not aware at the time that gnutls could be used as a /replacement/ for OpenSSL (for clients only, or even what gnutls can and cannot do honestly).

[DJ Lucas]

Replying to randy@…:

I don't know what you guys are doing different than I (wrong?), but it works perfect for me without anything. Just FBBG and cURL finds the cert bundle just fine.

No, it is finding the cert path, not the bundle. As Ag mentioned earlier, the certificate path is not valid when linking against gnutls. Again, also as already mentioned above by Ag, the correct solution is to separate out the ca-bundle, call it an optional dependency for both gnutls and OpenSSL, and add the "--with-ca-bundle=/etc/ssl/ca-bundle.crt" line to curl's configure arguments. The alternate is to move ca-bundle.crt to /etc/ssl/certs/ca-certificates.crt, but I don't like that idea because it results in double matching hash values (and potentially triple if you aren't careful about the commands used to create the bundle), which breaks some of OpenSSL's verification tools output (sorry, don't recall which tools, think it might have been s_client, but no time to verify right now), the very reason I did not follow the Debian example when adding the certs.

[Trent Shea]

Replying to dj@…:

Trent, we simply do not use the same policy as Debian for certificates. We do our best by trusting the folks at Mozilla.org to create a minimum set. The certificate with the hash value of 2468acdf is not trusted by the folks at Mozilla.org (and not by BLFS because of our choice to follow Mozilla devs for this). Open up the file in an editor and see who it belongs to. If you trust it, great, add it to your system and rerun the commands at the end of the OpenSSL page.

Sorry, the 2468acdf was just an example xxxxxxxx.0 > xxxxxxxx.pem may be more appropriate.

We are missing the links. I'm not suggesting that we do anything else here.

Anyone can add additional certificates, for instance, I add StartCom's intermediate certificates to my systems as I do use the free certificates, but I don't feel that it is appropriate to define policy for BLFS to use them (it'd probably be dangerous to some extent even). The crew at Mozilla.org does a very thorough verification/investigation process, that we take advantage of. Seems the problem is linking with gnutls as it works just fine with OpenSSL, and that the certificates should be installed from their own book page. There should already be one easily retrievable from SVN some time back as it was decided to simply tack it onto the OpenSSL page. I was not aware at the time that gnutls could be used as a /replacement/ for OpenSSL (for clients only, or even what gnutls can and cannot do honestly).

I have not touched gnutls. I believe that the problem I have/had is that curl expects there to be at least one xxxxxxxx.0 file to detect the capath automatically.

[Trent Shea]

Replying to dj@…:

Replying to randy@…:

I don't know what you guys are doing different than I (wrong?), but it works perfect for me without anything. Just FBBG and cURL finds the cert bundle just fine.

No, it is finding the cert path, not the bundle.

Just to reiterate, it doesn't find either for me. Randy, would you mind listing your certs directory? From what I can tell in configure you must have at least one xxxxxxxx.0 file present for the cert path to be detected automatically.

[Trent Shea]

Replying to trent.shea:

Replying to dj@…:

Replying to randy@…:

I don't know what you guys are doing different than I (wrong?), but it works perfect for me without anything. Just FBBG and cURL finds the cert bundle just fine.

No, it is finding the cert path, not the bundle.

Just to reiterate, it doesn't find either for me. Randy, would you mind listing your certs directory? From what I can tell in configure you must have at least one xxxxxxxx.0 file present for the cert path to be detected automatically.

Actually, this is irrelevant at this point. if capath doesn't work with gnutls. The bundle flag should be used (and they can't be combined) so I guess the links etc are a moot point.

[DJ Lucas]

Replying to trent.shea:

Just to reiterate, it doesn't find either for me. Randy, would you mind listing your certs directory? From what I can tell in configure you must have at least one xxxxxxxx.0 file present for the cert path to be detected automatically.

Ahh..now I understand. gnutls has no c_rehash, so the links will not be created as when installed with OpenSSL...but yes, as you stated, it is "a moot point." gnutls can use only the CAFile whereas OpenSSL can use either CAFile or CAPath.

On my report of breakage earlier, IIRC, it was the certificate information output that is broken when using CAFile with the s_client command. I think it displayed the first cert in the file, regardless of which cert was being verified when you display info, or something to that effect. At any rate, c_rehash will break with ca-certificates.crt in /etc/ssl/certs because some links will point to the file and others to the individual certs (giving the same output errors). Maybe it's been fixed since, but I still think they should be separated as there is no way to fix c_rehash.

[DJ Lucas]

The previous version would be from r7801 and r7803...maybe more.

Something like 'svn diff -c 7801' and 'svn diff -c 7803' should make a nice starting point for the new page. :-)

Bed for me.

[DJ Lucas]

Crud...sorry for the ticket spam. Anyway, don't forget to add a c_rehash command to OpenSSL if they are separated. I probably could have just done the edits by now. I'll try to get to it sometime in the coming weeks if one of yas don't get to it first. Had a rather major family incident recently. All is well now, but I've no time for fun things ATM, unless I deprive myself of sleep (as I already have tonight..ugh, this morning).

[Trent Shea]

Replying to dj@…:

Replying to trent.shea:

Just to reiterate, it doesn't find either for me. Randy, would you mind listing your certs directory? From what I can tell in configure you must have at least one xxxxxxxx.0 file present for the cert path to be detected automatically.

Ahh..now I understand. gnutls has no c_rehash, so the links will not be created as when installed with OpenSSL...but yes, as you stated, it is "a moot point." gnutls can use only the CAFile whereas OpenSSL can use either CAFile or CAPath.

Ah, there's the smack in the head I needed... openssl creates the links during make! I was packaging the certs separately... FBBG... but, I really wouldn't have learned anything this time, things would have just worked!

[DJ Lucas]

I suppose we could run c_rehash on the links prior to packaging them, however, the hash value will likely be different for gnutls anyway. I'll assign myself the ticket, but I'm not sure what can be done here if anything above pointing to the cafile and making it a required dep.

[Randy McMurchy]

Updated milestone to 6.7

[DJ Lucas]

This was partially fixed by the CA Certificates being separated, in that OpenSSL is required to manage the certs. I'm not sure if we can close this now or not. Should a method be provided to utilize GNUTLS, or just make OpenSSL a recommended dependency?

[bdubbs@…]

Fixed in revision 9481 by Andy. Added the configure option

--with-ca-path=/etc/ssl/certs and an explanation that mentions the

CA certs.

[bdubbs@…]

Milestone 6.7 deleted