Zlib 1.2.5 issues

[willimm]

These are some issues with Zlib 1.2.5 that I think need fixing:

• Zlib 1.2.5 segfaults a lot when built with GCC 4.5.0, either while building, or while a program using it is running (IE: with Firefox). The solution: add -mstackrealign to CFLAGS (You need to do this on the configure step, not on the make step), as documented at ​http://linuxfromscratch.org/pipermail/blfs-support/2010-June/067355.html.
• Also, when building a shared libary Perl (using -Duseshrplib) on 64bit systems, the Compress::Raw::Zlib module fails because of the module attempting to use the static Zlib, but because it's not compiled with -fPIC, the build fails. To avoid this, -fPIC should be added to CFLAGS, along with -mstackrealign.

[Gilles Espinasse]

The sed on the link should be wrong (--O changed to -O) Should be easier to do without sed using CFLAGS="$CFLAGS --mstackrealign" ./configure --prefix=/usr

See ​http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40838 for the controversed alignment issue.

Concerning the issue with perl, that should be an issue to find the shared lib, not that the static lib has to be compiled on a particular way adding -fPIC.

[willimm]

Replying to gespinasse:

The sed on the link should be wrong (--O changed to -O) Should be easier to do without sed using CFLAGS="$CFLAGS --mstackrealign" ./configure --prefix=/usr

I wasn't going to intend to use a sed (I used the link to show the gcc flag that works), the way you had it was like I had in mind. Also, -mstackrealign works fine.

See ​http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40838 for the controversed alignment issue.

Concerning the issue with perl, that should be an issue to find the shared lib, not that the static lib has to be compiled on a particular way adding -fPIC.

Sorry, I'm not sure how to hack perl to use libz.so and not libz.a. But, then again, other packages use libz.a, and the only way to fix this would be to add -fPIC to CFLAGS.

[bdubbs@…]

On my development system, I haven't run into a zlib problem, but my use of it has been very limited so far. The following seems to build without problem:

CFLAGS='-mstackrealign -fPIC -O3' ./configure --prefix=/usr
make
make check
make install
mv -v /usr/lib/libz.so.* /lib
ln -sfv ../../lib/libz.so.1.2.5 /usr/lib/libz.so

The only change to the book then is to add the CFLAGS='-mstackrealign -fPIC -O3' prefix.

[Gilles Espinasse]

That's wrong to fix the static compilation when you want perl linked to shared lib, no?

Compiler automaticly take the static libz.a when the shared libz.so is not found or the link to libz.so.1.2.5 is bad. At least I saw that with 5.10.1 when I made a mistake on the shared lib name.

I haven't try with perl-5.12 and I am compiling 32-bits only and with gcc-4.4.4. I have compiled ipcop entire package list with the static zlib.a removed, so I am sure the static libz.a was not include by a bug/mistake into perl.

I look at my perl compilation script and at the sed on perl. I am doing

-e "s|LIB\s*= ./zlib-src|LIB        = /lib|"

when LFS is using

-e "s|LIB\s*= ./zlib-src|LIB        = /usr/lib|"

I understand I should be be wrong setting LIB to /lib but that work.

Looking at the compilation log for -lz on LFS jhalf 094-perl build, I see

LD_RUN_PATH="/usr/lib" cc  -shared -O2 -L/usr/local/lib -fstack-protector Zlib.o  -o ../../lib/auto/Compress/Raw/Zlib/Zlib.so 	\
	   -L/usr/lib -lz  	\

Compiler should find libz.{a,so} in /usr/lib and select to use libz.so

What happen if you change the perl sed to set

 LIB        = /lib

Check the shared lib is used with

[chroot-i486] root:/$ find <your path>/usr/lib/perl5 -name Zlib.so -exec ldd {} \;
        linux-gate.so.1 =>  (0xb7f2f000)
        libz.so.1 => /lib/libz.so.1 (0xb7f0f000)
        libc.so.6 => /lib/libc.so.6 (0xb7dac000)
        /lib/ld-linux.so.2 (0xb7f30000)

[willimm]

Replying to gespinasse:

That's wrong to fix the static compilation when you want perl linked to shared lib, no?

Compiler automaticly take the static libz.a when the shared libz.so is not found or the link to libz.so.1.2.5 is bad. At least I saw that with 5.10.1 when I made a mistake on the shared lib name.

Really, just accept the fact that even though you can hack Perl to use the shared version of Zlib, lets face it, other packages might use libz.a and on 64-bit systems, if static libs are not compiled with -fPIC, then it won't build.

[Gilles Espinasse]

there is no problem to fix the static lib compilation once you know what you want is made at perl stage. But if you add -fPIC before fixing perl issue, you would not know the problem unless you use ldd against Zlib.so.

To my knowledge, there is only 2 packages that use libz.a. That's binutils and module-init-tools durings tests. For module-init-tools test, with one sed it is possible to change m-i-t to test the dynamic version that we use at the place of the static version that we try not to use..

This is what I made at module-init-tools stage not to use the static version.

	# that version use originally both --enable-zlib and --disable-zlib
	# prefer to use our shared libz.so for testing
	cd $(DIR_APP) && sed -i 's|--enable-zlib |--enable-zlib-dynamic |' tests/runtests

[willimm]

Replying to gespinasse:

To my knowledge, there is only 2 packages that use libz.a. That's binutils and module-init-tools durings tests. For module-init-tools test, with one sed it is possible to change m-i-t to test the dynamic version that we use at the place of the static version that we try not to use..

Just keep in mind that packages outside of LFS/BLFS sometimes link aganst the static version of Zlib. To make a permanent solution for this 64-bit issue, just build Zlib with CFLAGS="-fPIC -mstackrealign" and don't worry about solutions for other packages.

[Bryan Kadzban]

Replying to willimm:

Just keep in mind that packages outside of LFS/BLFS sometimes link aganst the static version of Zlib.

That's a bug in those packages, in my mind.

*NOTHING* should ever link against a static zlib IMO. Otherwise you have a rehash of the problems back in ... what was it, 2002 or so I think? ... when zlib had a buffer overflow fixed, and everything on the planet needed to be relinked.

Not using the shared lib is a Bad Idea; I'd probably support removing libz.a from the system completely after zlib is installed. Certainly binutils shouldn't be using it, but putting the .so.x.y.z file and .so.x link into /lib (and the .so link in /usr/lib) will allow both it and module-init-tools (if needed) to dynamically link libz and continue to work.

[Gilles Espinasse]

That's for the reason that I removed first zlib.a.

For module-init-tools, that's simple to change the test that try to link the static lib to the test that try with the shared lib. That's better as this is the configuration we use.

For binutils, that's will make some tests to fail, so I reverted the removal as as that time I had other issues to solve and I wanted to remove that cause of failure.

The right solution should be a patch that skip those tests if libz.a can't be found.

Concerning perl, the question for me is more why perl doesn't link to the shared lib with

-L/usr/lib -lz

It work for me with "LIB = /lib" in the sed but I am not exactly in the same condition as those with the failure (as gcc-4.4.4, perl-5.10.1 and 32-bits)

[willimm]

Replying to bryan@…:

Replying to willimm:

Just keep in mind that packages outside of LFS/BLFS sometimes link aganst the static version of Zlib.

That's a bug in those packages, in my mind.

*NOTHING* should ever link against a static zlib IMO. Otherwise you have a rehash of the problems back in ... what was it, 2002 or so I think? ... when zlib had a buffer overflow fixed, and everything on the planet needed to be relinked.

Thanks for bringing that attention to me. I was just assuming that some packages would use libz.a, but now that you brought it to my attention, it's a very bad idea. As for the Perl Compress::Raw::Zlib issue, let's do what gespinasse did and use "LIB = /lib" instead of "LIB = /usr/lib", and see if that works.

[bdubbs@…]

Replying to willimm:

Thanks for bringing that attention to me. I was just assuming that some packages would use libz.a, but now that you brought it to my attention, it's a very bad idea. As for the Perl Compress::Raw::Zlib issue, let's do what gespinasse did and use "LIB = /lib" instead of "LIB = /usr/lib", and see if that works.

A quick test build confirms that passing LIB = /lib builds fine and the tests. including the Zlib tests pass.

[Gilles Espinasse]

I just checked with an unchanged jhalf build (32-bits), so with LIB = /usr/lib

shared zlib is linked to Zlib.so but ldd complain about version ZLIB_1.2.3.3 not found.

lfs@a7n8x-e:~/jhalfs$ ldd /mnt/build_dir/usr/lib/perl5/5.12.1/i686-linux/auto/Compress/Raw/Zlib/Zlib.so /mnt/build_dir/usr/lib/perl5/5.12.1/i686-linux/auto/Compress/Raw/Zlib/Zlib.so: /usr/lib/libz.so.1: version `ZLIB_1.2.3.3' not found (required by /mnt/build_dir/usr/lib/perl5/5.12.1/i686-linux/auto/Compress/Raw/Zlib/Zlib.so)

linux-gate.so.1 => (0xb7f88000) libz.so.1 => /usr/lib/libz.so.1 (0xb7f63000) libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb7e04000) /lib/ld-linux.so.2 (0xb7f89000)

I checked in perl source for 'ZLIB_1.2.3.3' and found nothing There is many match in readelf -a /mnt/build_dir/lib/libz.so.1.2.5 | grep '1.2.3.3'

[Gilles Espinasse]

Another issue with zlib as this match the subject:

zlib.h is changed by gentoo, fedora-devel and Arch linux to fix one issue. See

​http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/sys-libs/zlib/files/zlib-1.2.5-lfs-decls.patch?view=log

Same effect of gentoo patch could be obtained with a

sed -i 's|ifdef _LARGEFILE64_SOURCE|ifndef _LARGEFILE64_SOURCE|' zlib.h

Not sure this is the /right thing/ But at least, this is reported to fix compilation failure with pdfedit, cgit, freecad, fatrat, libxlm2, luatex (probably only failure on 64-bits compilation, I have only warnings on 32-bits distrib with libxml2)

With that change, it remove for me on 32-bits compilation those warning when compiling libxml2

xmlIO.c:1175: warning: implicit declaration of function 'gzopen64'
xmlIO.c:1175: warning: nested extern declaration of 'gzopen64'

For more details ​http://bugs.gentoo.org/show_bug.cgi?id=316377 ​http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=439980

[Bryan Kadzban]

Replying to gespinasse:

shared zlib is linked to Zlib.so but ldd complain about version ZLIB_1.2.3.3 not found.

Wrong libz.so.1 file:

/mnt/build_dir/usr/lib/perl5/5.12.1/i686-linux/auto/Compress/Raw/Zlib/Zlib.so: /usr/lib/libz.so.1:

The perl lib is in /mnt/build_dir/usr/lib, but the libz that ldd found is in /usr/lib. This is wrong; the dynamic linker is pulling in host libs when trying to load a binary in the new partition. Either the compile-time linker is wrong (did readjusting the toolchain fail somehow?), or perl is being dumb, or this will only ever work inside chroot.

Is this chapter 6 perl, or chapter 5? If 6, then this can never work as you did it; you *must* be inside the chroot. If 5, it should be looking for /tools/lib/libz.so.1 instead.

[Tobias Gasser]

*NOTHING* should ever link against a static zlib IMO.

whenever possible, i add --disable-static to ./configure. i remove ALL remaining static libraries where a dynamic one is built (the 2 exceptions i mention below).

even libiberty (binutils) can be made dynamic:

cd /usr/lib mkdir -p TMP cd TMP ar x ../libiberty.a gcc -shared -o libiberty.so \

-Wl,-soname,libiberty-$D_VER *.o

strip -d -p libiberty.so mv libiberty.so .. cd .. rm -r TMP

i just have 2 issues:

• i can't convince sysvinit to use a dynamic libcrypt
• and modutils works fine with '--with-zlib-dynamic' but fails if no static libc is available.

those 2 static libs i move to a temp directory after building them, move them back to /usr/lib while building sysvinit or modutils and then i remove them.

tobias

[willimm]

So anyway, back on the origional subject, why don't we do this:

• Add CFLAGS="$CFLAGS -mstackrealign" to zlib's configure, and
• Use "LIB = /lib" in the Perl sed, like what gespinasse sugested, and
• Add "sed -i 's|ifdef _LARGEFILE64_SOURCE|ifndef _LARGEFILE64_SOURCE|' zlib.h" to the zlib instructions, to fix some compilation isues.

I think that would work out well.

[bdubbs@…]

Added CFLAGS to zlib configure instruction Also patched the herader file.

Fixed at revisions 9315 and 9316.

If we need to change perl, open another ticket.