Zlib issues

[chris@…]

There are a couple problems with the installation of the new version of Zlib...

1. It now installs a .pc file, but since libdir=/lib, it's installed in /lib/pkgconfig (thanks to Ratrophy on IRC for pointing this out).

2. It still installs a static library, but with current installation instructions it's kept in /lib.

One fix for #1 would be to add "pkgconfigdir=/usr/lib/pkgconfig" to the "make install" command. For #2, obviously /lib/libz.a would need to be moved to /usr/lib.

However, I think it would be easiest to fix both issues just by removing the --libdir parameter from configure, then simply mv'ing /usr/lib/libz.so.* to /lib and recreating the /usr/lib/libz.so.

Also since the static and shared libs are both built by a single configure/make, there are some small tweaks needed to the text.

[Gilles Espinasse]

Removing libdir may not be supported by module-init-tools because with --enable-zlib-dynamic, zlib has to be found in /lib.

I think the solution is to change perl sed to not search zlib in /usr/lib. That way, there is no more need to hack zlib installation in this curious way.

I made the symlink wrong at the first time and some program may mostly silently link to the static version. So I had the idea to remove zlib.a but that's wrong too. Some binutils tests will fail for that reason (as signaled by Robert Connolly)

[chris@…]

I didn't say to put Zlib's shared lib in /usr/lib, only to remove the --libdir parameter and manually mv it to /lib. I just think that would be easier than using libdir and having to add an extra parameter to make (for the pkgconfig file location) + an extra command to mv libz.a to /usr/lib.

[Gilles Espinasse]

Why do you need to manually move the shared lib to /lib? those instructions

./configure --prefix=/usr --libdir=/lib
make
make install

result in those files installed

lib/libz.a
lib/libz.so
lib/libz.so.1
lib/libz.so.1.2.4
lib/pkgconfig
lib/pkgconfig/zlib.pc
usr/include/zconf.h
usr/include/zlib.h
usr/share/man/man3/zlib.3

No need to rm and ln the shared lib if perl sed is adjusted to search zlib into lib instead of /usr/lib

[bdubbs@…]

I think I agree with Chris. I like the instructions:

./configure --prefix=/usr 
make
make install
mv /usr/lib/libz.so* /lib
ln -sfv  ../../lib/libz.so.1.2.4 /usr/lib/libz.so

I don't think we need to worry about libz.so.1. Thee are no other major versions of the library so it is highly unlikely that anyone links a program against that specific version. We could add

ln -sfv ../../lib/libz.so.1.2.4 /usr/lib/libz.so.1

for completeness.

[Gilles Espinasse]

The question is why to continue those complicated instructions that split part of the install in /lib and another in /usr/lib?

What is the real reason?

I didn't care specially of zlib.so.1, the original makefile care for me for at no cost.

If you upgrade zlib, you have to ajust the symlink instruction. If you install the normal way supported by the makefile in one directory alone for the lib, you don't need that extra work.

[chris@…]

First, static libraries should not be in /lib, as dictacted by the FHS. Second, the .pc file should not be in /lib either, both because you don't need it there (again the FHS - /lib is only for stuff required when booting) and because pkg-config doesn't look there.

[bdubbs@…]

Replying to gespinasse:

The question is why to continue those complicated instructions that split part of the install in /lib and another in /usr/lib?

What is the real reason?

If doing a build, the location of static libraries and other support files like a .pc or include file should be in the /usr tree. Dynamic libraries can be in either /lib or /usr/lib or possibly other directories specified by -L. These other libraries need to be available at run time via /etc/ld.so.cache or LD_LIBRARY_PATH.

If running a program where /usr is not mounted, such as a recovery situation, the administrator may need tar and tar may want libz to decompress tarballs.

Therefore libz needs to be in /lib along with tar in /bin.

There may be other issues that I don't remember right now.

[Bryan Kadzban]

Chiming in here in support of keeping things the way they are. :-)

As Bruce said, we need libz in /lib because of possibly tar, and definitely because of module-init-tools (unless they pull the static version, but then you run into problems with security updates). Also, auditing binaries in /bin and /sbin on my machine, the cramfs utilities from util-linux-ng link against zlib.

However...

I don't think we need to worry about libz.so.1. Thee are no other major versions of the library so it is highly unlikely that anyone links a program against that specific version.

No, nobody will do a "-l /lib/libz.so.1" in a compilation. However, every single binary that gets linked against libz.so, will include a reference to libz.so.1 (in the DT_NEEDED section). So we definitely need that symlink in a directory such that ld-linux.so.2 can find it, when those programs are started.

Whether that directory is /lib or /usr/lib depends on when those programs run, obviously.

[Gilles Espinasse]

Thank for the info for the .pc file location.

I think it's wrong to remove --libdir=/lib and manually move the shared lib because the .pc file is different and define libdir not where the shared lib will be.

I tested

./configure --prefix=/usr
make
make install pkgconfigdir=/usr/lib/pkgconfig

This change the .pc file to

prefix=/usr
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: zlib
Description: zlib compression library
Version: 1.2.4

Requires:
Libs: -L${libdir} -lz
Cflags: -I${includedir}

when with --libdir=/lib, it was

prefix=/usr
exec_prefix=${prefix}
libdir=/lib
includedir=${prefix}/include

Name: zlib
Description: zlib compression library
Version: 1.2.4

Requires:
Libs: -L${libdir} -lz
Cflags: -I${includedir}

So libdir define the wrong place in the .pc file when --libdir is not used and the shared lib is moved to /lib manually.

Maybe those instructions could make everyone happy with

./configure --prefix=/usr --libdir=/lib
make
make install pkgconfigdir=/usr/lib/pkgconfig
mv /lib/libz.a /usr/lib

(and change the perl sed or perl may link against libz.a)

[bdubbs@…]

The .pc file with libdir=${exec_prefix}/lib works fine because of the symlinks.

[willimm]

Lets just merge Chris' patch into LFS so that I don't have to worry when JHALFSing the development book.

[bdubbs@…]

Fixed at revision 9239.

I did need to make one small change to the patch. The command

mv -v /usr/lib/libz.so.* /lib

needed to have the last dot removed so libz.so gets moved also.

[willimm]

Replying to bdubbs@…:

Fixed at revision 9239.

I did need to make one small change to the patch. The command

mv -v /usr/lib/libz.so.* /lib

needed to have the last dot removed so libz.so gets moved also.

One question: Why are you also moving libz.so to /lib? You remove .so files in /lib, right?

[bdubbs@…]

Replying to willimm:

One question: Why are you also moving libz.so to /lib? You remove .so files in /lib, right?

No, all the .so* files are in /lib, but we have another .so file in /usr/lib for the .pc file to reference. Both point to the same place.

[willimm]

Replying to bdubbs@…:

Replying to willimm:

One question: Why are you also moving libz.so to /lib? You remove .so files in /lib, right?

No, all the .so* files are in /lib, but we have another .so file in /usr/lib for the .pc file to reference. Both point to the same place.

What I was asking is that usually you move .so files (the ones without version numbers) to /usr/lib, I'm just wondering why you did that here too.

[willimm]

Replying to willimm:

Replying to bdubbs@…:

Replying to willimm:

One question: Why are you also moving libz.so to /lib? You remove .so files in /lib, right?

No, all the .so* files are in /lib, but we have another .so file in /usr/lib for the .pc file to reference. Both point to the same place.

What I was asking is that usually you move .so files (the ones without version numbers) to /usr/lib, I'm just wondering why you did that here too.

Err,whoops, I meant:

I'm just wondering why you moved to .so file to /lib along with the others.

[willimm]

Just want to point out to you. .so files do not belong in /lib, but .so.* files can be in /lib.

[Gilles Espinasse]

That was right in the mv command with the final dot

There is no need for libz.so at two different places. Unsure if that will not be supported by a packaging system to have twice the same file name in /lib and /usr/lib.

Both Ubuntu/Debian and Fedora have the same layout (with 1.2.3) libz.so.* are in /lib in standard package libz.so is in /usr/lib, like libz.a and both are include in dev or devel package with headers.

You don't need the duplicate. And pkg-config work pkg-config --cflags --libs zlib

-lz

The requirement for libz in /lib come from /sbin/modprobe because of --enable-zlib-dynamic

[bdubbs@…]

I appreciate all the comments. It forced me to understand the difference between soname, real name, and linker name for libraries. It's always good to learn something new.

I've committed the change to keep libz.so in /usr/lib.

Actually, the update was made before the book generates, so the erroneous command never made it into the generated html.

Note: /sbin/fsck.cramfs also uses libz.

[Gilles Espinasse]

Just for reference, we find that not having libz.so on the running machine may cause trouble. The reason should be in openssl/crypto/dso/dso_dlfnc.c where the function *dlfcn_name_converter calculate lib%s.so name

Using SSLeay (from a perl script), when libz.so does not exist, you have this error

Dynamic DNS ip-update for x.y.z : failure (HTTP/1.0 900 NET OR SSL
ERROR#015#012#015#012CTX_new 1460: 1 - error:25066067:DSO support
routines:DLFCN_LOAD:could not load the shared library#012CTX_new 1460: 2
- error:25070067:DSO support routines:DSO_load:could not load the shared
library#012)

[Bryan Kadzban]

Replying to gespinasse:

Using SSLeay (from a perl script), when libz.so does not exist,

If SSLeay-in-perl is having problems when libz.so does not exist, then SSLeay-in-perl will *also* have problems as soon as the zlib developers introduce a binary-incompatible libz. This will be have soname libz.so.2, and everything will continue to work, *except* pieces of broken code like this. (Existing binaries will use libz.so.1, which will remain as it is today.)

If you're using dlopen, you *NEED* to open the soname. *NOT* the compile-time .so file; there is *NO* guarantee that this filename will stay binary-compatible, and dlopen() *requires* binary compatibility.

In short: this is not a reason to move libz.so; it's a broken piece of code.