Blanket notes for updated builds (SVN-DJL-20080825)

(randy) The version of GMP has been incremented to 4.2.3.

(randy) The URL to download the MPFR patch is broken. Best I can tell you need to use http://mpfr.org/mpfr-current/patches

This should be resolved, however, by putting the patch in LFS format and in the LFS patch repo.

(randy) The disk space used in the GMP and MPFR instructions seem to be much different than my logs show. I showed 23mb for GMP and 19mb for MPFR.

This has been discussed and we came to no real consensus but the Bash patches are now up to 39. Our LFS-7 patch stops at 33.

We could not determine if using these patches is the same as just pulling from CVS/SVN/GIT. I'm for using the latest available patches.

Related to this is a message on -dev about the same sort of thing for Ncurses.

(randy) Ran into a documented bug in Coreutils-6.12 when compiled in certain situations. Best I can tell coreutils uses a function that was not implemented into the Linux kernel until version 2.6.22.x.

This causes building some packages (I ran into it when building the E2fsprogs package in Chapter 5) to fail when attempting to use the 'touch' program. My host is an LFS with kernel 2.6.21.5.

Here's a Gentoo bug report which describes it: http://bugs.gentoo.org/show_bug.cgi?id=224483

However, I used this patch to coreutils and after building and installing coreutils with the patch, the touch program worked just fine. Here's the patch I used: http://bugs.gentoo.org/attachment.cgi?id=155835

I don't know what the final outcome is, nor which patch we should use (if any?), but I needed the patch.

(randy) As the following is new to the Util-Linux instructions: 'make BLKID_LIBS="-lblkid -luuid"', it should be explained why this is necessary.

(randy) There are now 6 patches available for VIM-7.2. I will roll them all into one patch and place it in the LFS repo.

Replying to randy@linuxfromscratch.org:

(randy) There are now 6 patches available for VIM-7.2. I will roll them all into one patch and place it in the LFS repo.

Is there a way to automate downloading and incorporating the patches? Rolling up the current list seems to be a lot of work that will need to be redone with every one of the relatively frequent patches.

I did this for autofs in BLFS and it should be considered for vim.

Replying to bdubbs@linuxfromscratch.org:

Is there a way to automate downloading and incorporating the patches? Rolling up the current list seems to be a lot of work that will need to be redone with every one of the relatively frequent patches. I did this for autofs in BLFS and it should be considered for vim.

I suppose we could, but keep in mind that vim is installed in LFS in Chapter 6 where we are in chroot. The wget program won't exist, but I suppose we could use ftp. Where to get the patches is ftp://ftp.vim.org/pub/vim/patches/7.2/

Problem is you don't have networking available in chroot while you are installing VIM (I think, but not positive), so ftp may be hard.

Let's discuss.

Replying to randy@linuxfromscratch.org:

Problem is you don't have networking available in chroot while you are installing VIM (I think, but not positive), so ftp may be hard. Let's discuss.

Networking is available as long as your host has a properly configured connection. The only thing you have to do over and above what is currently in the book is copy your host's /etc/resolv.conf file to chroot.

Even so, you still wouldn't be able to download anything with ftp until the ftp client is installed in section 6.40 (Inetutils). If you really wanted to go this route, you could install wget in chapter 5 as a part of the tools, or, I'm sure there's some way to write a bash download script that sends out HTTP requests.

I personally don't mind wget in chapter 5, but really, anything on this level may be a lot more work than it's worth.

Just realized that ftp would still be fine for this particular purpose since vim is way at the end of chapter 6. In that case, there is also a script in the LiveCD archives that makes it easy to script the downloading of multiple files using only the ftp client. I will look it up...

The man-pages package has incremented to 3.08

Replying to jhuntwork@linuxfromscratch.org:

Just realized that ftp would still be fine for this particular purpose since vim is way at the end of chapter 6. In that case, there is also a script in the LiveCD archives that makes it easy to script the downloading of multiple files using only the ftp client. I will look it up...

Downloading the patches should take place at the same time as downloading all the other packages and patches. Then they would be available in chroot. No networking is needed in chroot.

Also, I wouldn't bother with the patches in Chapter 5. Basic functionality will be available for the short time before building in Chapter 6.

Keep in mind for this build I used a Glibc-2.8 snapshot dated 08/25/2008/ --- Well, not a great sign, but it could be worse (Chapter 6 Glibc build):

root:/build/glibc-build# grep Error check.log
make[2]: *** [/build/glibc-build/iconvdata/bug-iconv6.out] Error 1
make[2]: *** [/build/glibc-build/iconvdata/tst-iconv7.out] Error 1
make[1]: *** [iconvdata/tests] Error 2
make[2]: *** [/build/glibc-build/math/test-ildoubl.out] Error 1
make[1]: *** [math/tests] Error 2
make[2]: [/build/glibc-build/posix/annexc.out] Error 1 (ignored)
make: *** [check] Error 2

I'm just going to drive on. Chapter 5 seemed flawless using this glibc version. More to follow.

Replying to bdubbs@linuxfromscratch.org:

Is there a way to automate downloading and incorporating the patches? Rolling up the current list seems to be a lot of work that will need to be redone with every one of the relatively frequent patches. I did this for autofs in BLFS and it should be considered for vim.

I ought to write this before.

This is not possible, unless:

a. we are going to extend the instructions to extract also the extra sources, otherwise there are going to be patches that don't apply or

b. in the hypothetically given loop, we are going to blacklist some patches or

c. maintain an external script to filter also some patches based on their header

I am sure most people here don't want to add bloat in their disk to extract mac, dos, vms or windows stuff, so I don't give many chances for the first possibility. b is possible, but we still have to keep updated the list with the blacklisted patches, if we don't want to have broken instructions and it's a little bit ugly. c is possible but it's also an extra step, so again there is bloat that people might want to avoid

However and if we follow one of the above solutions or something else that I don't have in mind right now, this will sets a precedent, with the meaning that:

- we are going to provide instructions for users, to apply patches without testing beforehand by an editor

- the codebase is going to be different from build to build, so support (especially) and testing is going to be also problematic

Another problem; what we are going to do in the case of a stable book release?

Replying to ag@linuxfromscratch.org:

- we are going to provide instructions for users, to apply patches without testing beforehand by an editor - the codebase is going to be different from build to build, so support (especially) and testing is going to be also problematic Another problem; what we are going to do in the case of a stable book release?

I have exactly the same concerns regarding quality management, and supportability of the book. I'll try and dig out my trivial(ish) script that does automatically create an LFS-formatted patch from the latest available patches. That helps editors to quickly create a patch that can be tested in the same manner as other updates. I don't see any particular advantages to having users build against what will quite likely be untested (by us) patches, and will certainly change throughout the lifetime of a stable book release.

Regards,

Matt.

(Randy) Some package updates after building Chapter 5:

1. GMP has a version increment 2. MPFR has a version increment 3. E2fsprogs has a version increment 4. Texinfo has a version increment 5. Util-Linux-NG has a version increment

(Randy) There is an issue with the GMP instructions in Chapter 5. The --enable-cxx parameter needs to be removed if you don't have GMP installed on the host. It isn't required anyway, so there's really no reason for it. It failed on me because we don't install the C++ compiler in Chapter 5 GCC Pass 1 instructions.

I even tried to pass CXX=/usr/bin/g++ to the configure script and it finds g++, but it fails later on in the configure process. Apparently due to incompatibilities with the older host compiler (I have 4.0.3 on my host).

(Randy) Noted that in Chapter 6 Vim instructions, you're not using the Vim patch I created and installed in the repo. I've created another patch for Vim which has patches 1-22. I'll also put it in the repo.

There is a known test failure in binutils-2.18 when using GCC-4.3.x (ld-shared test). Upstream has it fixed. I added a patch to the repo.

http://www.linuxfromscratch.org/patches/downloads/binutils/binutils-2.18-GCC43-1.patch

Additionally, because binutils is part of the "toolchain", I think we should show how to confirm the test suite ran as expected (as we do with Glibc and GCC). Simply issue

grep "# of" check.log; grep test- check.log

(assuming the output from the tests was output to "check.log") and the following should be returned:

# of expected passes 45 # of expected passes 185 # of expected passes 439 # of expected failures 4 ./test-demangle < ../../../binutils-2.18/libiberty/testsuite/demangle-expected ./test-demangle: 765 tests, 0 failures ./test-pexecute ./test-expandargv PASS: test-expandargv-0. PASS: test-expandargv-1. PASS: test-expandargv-2. PASS: test-expandargv-3.

Finally, we should pass parameters to configure so that the man and info files will be installed to /usr/share instead of /usr/man and /usr/info.

Replying to randy@linuxfromscratch.org:

Additionally, because binutils is part of the "toolchain", I think we should show how to confirm the test suite ran as expected (as we do with Glibc and GCC).

(snip the example)

I'm ambivalent about that, at least until we come to a release, but I don't have strong feelings either way (and I haven't yet built gcc-4.3). I suppose if it goes in it can always come out when it no longer applies.

Finally, we should pass parameters to configure so that the man and info files will be installed to /usr/share instead of /usr/man and /usr/info.

But this one leaves me puzzled - isn't this why we create /usr/{man,info} as symlinks ? That certainly seems to mitigate the problem on a lot of packages, both in LFS and later.

Replying to ken@linuxfromscratch.org:

I'm ambivalent about that, at least until we come to a release, but I don't have strong feelings either way (and I haven't yet built gcc-4.3). I suppose if it goes in it can always come out when it no longer applies.

I'm not following. What do you mean "no longer applies"? Won't we *always* have binutils as part of Chapter 6?

But this one leaves me puzzled - isn't this why we create /usr/{man,info} as symlinks ? That certainly seems to mitigate the problem on a lot of packages, both in LFS and later.

I realize that the symlinks make up for the misplaced files. But I thought our goal was to provide instructions to build packages and place the files in proper locations. IMO we should try to create instructions that place the files in the proper locations. No big deal, I just think it would look better on us to show that we know the files are misplaced.

Replying to randy@linuxfromscratch.org:

I'm not following. What do you mean "no longer applies"? Won't we *always* have binutils as part of Chapter 6?

When we upgrade the version of binutils, I assume any errors will change and maybe disappear entirely.

I realize that the symlinks make up for the misplaced files. But I thought our goal was to provide instructions to build packages and place the files in proper locations. IMO we should try to create instructions that place the files in the proper locations. No big deal, I just think it would look better on us to show that we know the files are misplaced.

It's a valid thought, and perhaps we ought to do it once in the book with an explanation, but I dread the implicit idea of noting this for every package which happens to use /usr/man or /usr/info (both in LFS and in BLFS).

Replying to ken@linuxfromscratch.org:

When we upgrade the version of binutils, I assume any errors will change and maybe disappear entirely.

I'm still not following. I don't think we are discussing the same thing here. What I'm saying is that we should show the reader how to parse the output from the test suite. We do this for GCC and Glibc, why not do it for the remaining toolchain package as well?

This has nothing to do with errors, there *never* should be any. What I'm saying to do is provide a method for the reader to confirm this.

Actually, we should also provide a way to parse the GMP test suite output as well. The maintainers suggest to do it, and we provide instructions in BLFS to do it. It could be said that GMP is also a "toolchain" package.

For that matter, it probably is important to put a blurb in the book on the MPFR page that running the test suite and examining the output is highly recommended.

Some notes about the MPFR package.

1. The patch is not required with the updated version.

2. I think we should pass --enable-thread-safe to configure.

3. I think we should add a 'make html' command so that the HTML docs are built and then add an installation command.

Not sure what to think about this one. I double-checked and confirmed that the fixinc.sh sed was ran. Indeed it was. But running the header sanity check after building Chapter 6 GCC, I get this returned:

{{{#include <...> search starts here:

/usr/lib/gcc/i686-pc-linux-gnu/4.3.2/include /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/include-fixed /usr/include}}}

I suppose I'm not seeing the /usr/local because I have no need for that directory and do not create it, therefore it doesn't exist on my system. What I find strange is the "include-fixed" directory (which is populated with 3 files): {{{-rw-r--r-- 1 root root 750 Sep 26 18:12 README -rw-r--r-- 1 root root 3470 Sep 26 18:12 limits.h -rw-r--r-- 1 root root 330 Sep 26 18:12 syslimits.h}}}

Is this normal behavior? Or did I somehow screw something up with the fixincludes process?

Trying again to see if I can get the formatting correct.

Not sure what to think about this one. I double-checked and confirmed that the fixinc.sh sed was ran. Indeed it was. But running the header sanity check after building Chapter 6 GCC, I get this returned:

#include <...> search starts here:

    /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/include
    /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/include-fixed
    /usr/include

I suppose I'm not seeing the /usr/local because I have no need for that directory and do not create it, therefore it doesn't exist on my system. What I find strange is the "include-fixed" directory (which is populated with 3 files):

-rw-r--r-- 1 root root 750 Sep 26 18:12 README
-rw-r--r-- 1 root root 3470 Sep 26 18:12 limits.h
-rw-r--r-- 1 root root 330 Sep 26 18:12 syslimits.h

Is this normal behavior? Or did I somehow screw something up with the fixincludes process?

The Berkeley DB package needs to have the upstream patch install put into the Chapter 6 instructions. Patch is at:

http://www.oracle.com/technology/products/berkeley-db/db/update/4.7.25/patch.4.7.25.1

I can't really see what is wrong in the original Makefile.in file, but the E2fsprogs package fails to build the blkid and fsck programs in the misc directory due to failing to find the libcom_err library. I know that in all other previous versions of LFS we don't build a portion of E2fsprogs in Chapter 5, but I can't see that has anything to do with it anyway.

I didn't check upstream as a quick glance at the appropriate Makefile.in file shows how easy it was to fix this. I had to run the following sed for the compilation to succeed.

sed -i -e 's/-o blkid.*/& $(LIBCOM_ERR)/' \
       -e 's/-o fsck.*/& $(LIBCOM_ERR)/'  \
    misc/Makefile.in

Note that I'm using the updated 1.41.1 version of E2fsprogs.

Note that the following commands (modified to suit LFS and not my DESTDIR installation) could be added to the E2fsprogs instructions to provide a bit more thorough documentation.

gunzip -v destdir/usr/share/info/libext2fs.info.gz

makeinfo -o      doc/com_err.info ../lib/et/com_err.texinfo
install -v -m644 doc/com_err.info destdir/usr/share/info

install -v -m755 -d destdir/usr/share/doc/${PACKAGE_NAME}-${PACKAGE_VERSION}
install -v -m644    ../doc/libblkid.txt \
                    destdir/usr/share/doc/${PACKAGE_NAME}-${PACKAGE_VERSION}

Note also that the /usr/share/info/dir file is not recreated as it should be.

As the chroot command is moved to /usr/sbin in the Coreutils instructions, the corresponding man page should be renamed to a .8 file and moved to the man8 directory.

there is a post 1.41.1 upstream patch to fix the libcom_err. http://sourceforge.net/tracker/index.php?func=detail&aid=2088537&group_id=2406&atid=102406

Something to consider is installing the documentation included with the Ncurses packages into an appropriate directory. Simply create a directory and cp -v -R all the files from the source tree 'doc' directory into it.

This list is getting really long, really fast. I think all of this should be happening in the development book now...save the man-db->man change. I'm going to update my local copy with all of the above changes (except the html docs for MPFR) and start breaking out patches (I know I said I'd do it 15 days ago). As for the HTML docs for MPRR, are the necessary tools available in chroot? I believe I'm current up to comment #17. Randy, if you continue on this path, would you mind attaching patches for LFS-Devs to begin applying? Give me till Monday night and I'll have everything so far broken out into individual pieces for applying to svn.

-- DJ

Okay...moved. http://www.linuxfromscratch.org/~dj/LFS-20080928/html/

Completely untested!!! Remember that the man-pages are still broken for anything but English locales. I will be posting patches to this bug.

-- DJ

After the above patch, somebody needs to verify man-db works correctly. I've not done a build with it yet.

Oops, md5 and size was of original upstream patch.

For the committing editor, please remember to replace my name in the changelog with your own.

Update to e2fsprogs-1.41.1. See to it that Randy gets credit for this and DB udpate too.

Update to findutils-4.4.0.

Update to iproute2-2.6.26

Update to linux-2.6.26.5.

Update to m4-1.4.11.

Update to man-pages-3.08.

I don't recall the removal of mktemp, but sure enough it's gone. Patch removes unused mktmp entities and adds missing "d" (in "Updated") to last few changelog entries.

Replying to dj@linuxfromscratch.org:

This list is getting really long, really fast. I think all of this should be happening in the development book now..

Yes it should, but that means there will be no release based on gcc-4,2*? The book is stable and it only needs the gcc update up to 4.2.4 and it will be ready for a release.

Upate to Perl-5.10.0

Update to shadow-4.1.2.1.

Update to tar-1.20

Update to tcl-8.5.4

Update to texinfo-4.13.

Update to udev-126.

Update to Util-Linux-NG-2.14.1.

Update to Vim-7.2.

OK, all of the mind numbing package updates are here for review and commit, 16 in all through vim. I will post big ones in the coming days (hopefully tomorrow or Monday).

-- DJ Lucas

Replying to dj@linuxfromscratch.org:

For the committing editor, please remember to replace my name in the changelog with your own.

DJ, would it not just be easier to give you commit privs to the LFS SVN repo? :)

Version increment to libtool-2.2.6

**NOTE**

http://savannah.gnu.org/forum/forum.php?forum_id=5442

There is already a Trac ticket where there is supposedly some incompatibilities with the 2.2 series of libtool. I'm going to use it and I'll report back.

Replying to matthew@linuxfromscratch.org:

DJ, would it not just be easier to give you commit privs to the LFS SVN repo? :)

Best I can tell from examining the /etc/group file on Quantum is that both DJ and I have commit privs to the LFS SVN repo, we've just chosen not to use them. :-)

Just FYI, though it probably wouldn't hurt to change the book as well, but the home page for Perl should actually be at cpan.org and not perl.com.

perl.com is an O'Reilly site. CPAN is the official Perl website.

Replying to randy@linuxfromscratch.org:

Replying to matthew@linuxfromscratch.org:

DJ, would it not just be easier to give you commit privs to the LFS SVN repo? :)

Best I can tell from examining the /etc/group file on Quantum is that both DJ and I have commit privs to the LFS SVN repo, we've just chosen not to use them. :-)

Yes, I've had the file permissions for a long time, since the separated bootscript days. I've just never used them beyond what they were intended for (bootscripts), but I can certainly apply these updates myself if you'd like.

-- DJ Lucas

Readline has postscript, dvi, pdf and html documentation in the 'doc' directory that could be installed.

the man-pages package has had a version increment to 3.10

There's a typo in the Berkeley db patch description on the "Needed patches" page. (Upstram)

There's a -8 patch for bash in the repo that needs to be added to the bash page to replace the -7 patch

The File package has been incremented to version 4.26

Additionally, the version in the text of the book is different than the version listed on the "All Packages" page.

Just a reminder (from the -dev list by Alexander) that the --enable-multibyte configure switch is not applicable to the groff package any longer.

Module-Init-Tools has a version increment to 3.4.1

By default the Automake package will install documentation to /usr/share/doc/automake. Passing --docdir=/usr/share/doc/automake-1.10.1 will cause the docs to be installed in a versioned directory, which is preferred.

In the Flex package there is a flex.pdf file in the 'doc' subdir that could be installed.

Simply because I've seen Alexander do magic with the awk (Gawk) program, and there's documentation in the doc subdir that is really entertaining, I recommend we install it.

If anything, the 'awkforai.txt' file and then .eps, .jpg, and .pdf files should be installed. I recommend to anyone to at least read the 'awkforai.txt' file. Very entertaining.

By default the Groff package will install docs to /usr/share/doc/groff/version. If we pass docdir=/usr/share/doc/groff-version, then the docs will be installed into a more standardized versioned directory.

A couple of notes about Iproute2.

1. Passing DOCDIR= to the install command can put the docs in a versioned directory. By default it is not a versioned directory.

2. This is speculation (I use DESTDIR and this package is *not* DESTDIR friendly). Anyway, it appears now that since DESTDIR is hard-coded in the Makefile, that passing SBIN= isn't going to do what you think it will as the default value for DESTDIR is /usr and that is where it will go. It appears that if you use DESTDIR="", then the binaries will go where you want them but the "share" dir will be directly off the root instead of /usr/share/. YMMV

Another Iproute2 note:

The arpd binary is still made and needs to be moved to /usr/sbin. The instructions and notes to do this have been removed in the DJ version of the book.

The entire doc directory in the KBD package should probably be copied into a versioned directory of /usr/share/doc.

Best I can tell the default behavior for the Shadow useradd program is to now correctly prepend /home to the user's account name so we shouldn't have to change the default useradd entry.

Replying to randy@linuxfromscratch.org:

Another Iproute2 note: The arpd binary is still made and needs to be moved to /usr/sbin. The instructions and notes to do this have been removed in the DJ version of the book.

Oops...leftover from half cocked DB removal...remove iproute2.xml changes from the update patch.

Randy, can you start creating new tickets please? All the patches I attached and we are at 62 messages long, 63 now. It's getting difficult to use it as a task list. Most of the notes above will apply to SVN book when it gets updated.

Thanks.

-- DJ

The following instruction from the Udev package is somewhat redundant in that the last filename doesn't need to be specified.

install -m644 -v rules/packages/40-pilot-links.rules \

/etc/udev/rules.d/40-pilot-links.rules

This should be about it DJ, it was just so much easier for me to add to this ticket than to open up new ones.

At least this stuff is documented, and I'll help you get it all into the book. It's almost as if it is you and I now if we want to get anything done.

Perhaps others will jump in.

Funny..I just sent you and Matt offline messages about that. ;-)

Throwing one more out there about Udev. There are two package config files in /lib/pkgconfig that need to be moved to /usr/lib/pkgconfig, instead of just one which the DJ book shows.

One last note (hopefully):

The .so files created by the Udev package that are installed in /lib need to be removed and recreated in /usr/lib.

Replying to randy@linuxfromscratch.org:

Trying again to see if I can get the formatting correct. Not sure what to think about this one. I double-checked and confirmed that the fixinc.sh sed was ran. Indeed it was. But running the header sanity check after building Chapter 6 GCC, I get this returned: {{{ #include <...> search starts here: /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/include /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/include-fixed /usr/include }}} I suppose I'm not seeing the /usr/local because I have no need for that directory and do not create it, therefore it doesn't exist on my system. What I find strange is the "include-fixed" directory (which is populated with 3 files): {{{ -rw-r--r-- 1 root root 750 Sep 26 18:12 README -rw-r--r-- 1 root root 3470 Sep 26 18:12 limits.h -rw-r--r-- 1 root root 330 Sep 26 18:12 syslimits.h }}} Is this normal behavior? Or did I somehow screw something up with the fixincludes process?

Other guys didn't catch this for us yet because the are building cross compilers (when cross compiling, fixincludes and fixproto cannot be run). This should do the trick, but I haven't tested it just yet.

cp -v gcc/configure{,.orig}
sed -e 's@stmp-fixproto@@' \
    -e 's@stmp-fixinc@@' \
    gcc/configure.orig > gcc/configure

Replying to dj@linuxfromscratch.org:

This should do the trick, but I haven't tested it just yet.

Yuck! Don't do that. ;-) I'll take care of it shortly.

Replying to dj@linuxfromscratch.org:

Replying to dj@linuxfromscratch.org:

This should do the trick, but I haven't tested it just yet.

Yuck! Don't do that. ;-) I'll take care of it shortly.

I've added the following just beneath the sed command in my local copy in chapter 5:

    <note><para>As of version 4.3.0, GCC now unconditionally installs the
    <filename>limits.h</filename> file into the private
    <filename class="directory">fixed-includes</filename> directory, and that
    directory is required to be in place.</para></note>

Is that sufficient explanation? Should we provide the link?

I've double checked all the stuff on this ticket and everything is either resolved, or is in another ticket (a couple of things I have notes on and I'll be making tickets for those). Closing this beast.

All packages updates are in (I'll double check this as well).