Util-Linux-NG-2.16

[Matthew Burgess]

New version. Release announcement at ​http://lkml.indiana.edu/hypermail/linux/kernel/0905.0/01715.html.

[Matthew Burgess]

Note that this version, by default, looks for blkid by invoking pkg-config. I don't think we can guarantee our host has blkid installed, so chapter 5's Util-Linux-NG instructions will need '--with-fsprobe=builtin' to override the default.

Once that's done though, the following is encountered:

gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I..  -include ../config.h -I../include \
-DLOCALEDIR=\"/home/matthew/util-linux-ng/share/locale\"   -fsigned-char \
-I../libs/blkid/src   -g -O2 -MT mount-setproctitle.o -MD -MP -MF \
.deps/mount-setproctitle.Tpo -c -o mount-setproctitle.o `test -f \
'../lib/setproctitle.c' || echo './'`../lib/setproctitle.c mv -f \
.deps/mount-setproctitle.Tpo .deps/mount-setproctitle.Po make -C 
make: option requires an argument -- 'C'
Usage: make [options] [target] ...
...

This has been reported upstream at ​http://marc.info/?l=util-linux-ng&m=124182554005293&w=2.

For chapter 6, this seems like a prime opportunity to migrate pkg-config from BLFS to LFS. There didn't seem to be any objections when Randy suggested this at ​http://www.linuxfromscratch.org/pipermail/lfs-dev/2009-February/062627.html.

[Matthew Burgess]

The above issue with 'make -C' is fixed with the patch at ​http://marc.info/?l=util-linux-ng&m=124203585112415&w=2, also in an LFS-style patch at ​http://www.linuxfromscratch.org/patches/downloads/util-linux-ng/util-linux-ng-2.15-compile_fix-1.patch.

Now we hit the problem reported at ​http://marc.info/?l=linux-fsdevel&m=124160979414771&w=2. That's fixed upstream at ​http://marc.info/?l=util-linux-ng&m=124204848630344&w=2, and a fully re-autotooled version is available from ​http://www.linuxfromscratch.org/patches/downloads/util-linux-ng/util-linux-ng-2.15-blkid_detection-1.patch.

With those 2 patches applied, and pkg-config installed, Util-Linux-NG-2.15 compiles and installs cleanly.

[Bryan Kadzban]

Also, we'll want to disable blkid / libblkid from e2fsprogs, to avoid having multiple packages fighting over the binary and library. --disable-libblkid is supposed to do that.

Last, I should note that the next udev version will require util-linux-ng-2.15's libblkid: they've removed vol_id, trying to settle on a single program doing all the disk probing. See ​http://marc.info/?l=linux-hotplug&m=124203739214725&w=2 for info.

[Matthew Burgess]

Thanks Bryan. That's an interesting thread indeed. Another patch in that thread at ​http://marc.info/?l=util-linux-ng&m=124215770729246&w=2 enables util-linux-ng to build prior to e2fsprogs as it has a dependency on libuuid.

So, we have a circular dependency if we were to migrate to upstream's preferred instructions:

Util-Linux-NG requires libuuid from E2fsprogs E2fsprogs requires libblkid from Util-Linux-NG

They're proposing to move libuuid into Util-Linux-NG, but until that's done, upstream's suggestion is to do the following:

Util-Linux-NG: --with-fsprobe=builtin (+ additional commands to install libblkid) E2fsprogs: --disable-libblkid Util-Linux-NG: --with-fsprobe=builtin

My question is, is this worth it at this point? Could we just wait until libuuid has been migrated into Util-Linux-NG?

RFC-1: Roll up the 2 current util-linux-ng patches with the 3rd mentioned at the start of this comment into one catch-all compile_fixes patch.

RFC-2: Mark this ticket as WONTFIX, and wait until 2.15.1 :-)

[bdubbs@…]

Replying to matthew@…:

So, we have a circular dependency if we were to migrate to upstream's preferred instructions:

Util-Linux-NG: --with-fsprobe=builtin (+ additional commands to install libblkid) E2fsprogs: --disable-libblkid Util-Linux-NG: --with-fsprobe=builtin

My question is, is this worth it at this point?

We don't know how long it's going to take for upstream to fix things. Why don't we move Util-Linux-NG to before E2fsprogs in Chapter 5 and then build as normal (with appropriate switches) in Chapter 6?

[Bryan Kadzban]

Replying to bdubbs@…:

Why don't we move Util-Linux-NG to before E2fsprogs in Chapter 5 and then build as normal (with appropriate switches) in Chapter 6?

Using chapter 5 to get around the circular dependency sounds good to me. But we might want to keep the chapter 5 order as-is (and not worry about packages fighting about libblkid; just install it from both), and just use chapter 5's libuuid for the util-linux-ng build in chapter 6.

Or at least do that for now...

[Matthew Burgess]

Also note that configure now checks for ncurses and fails if it doesn't find ncurses.h. Fixed by moving ncurses up to immediately prior to Util-Linux-NG in the build order.

[Matthew Burgess]

Util-Linux-NG in chapter 6 still complains that it can't find -luuid. So, it looks like we're going to have to move e2fsprogs as well. If anyone gets a chance, could you see if it's possible to get Util-Linux-NG to pick up chapter 5's lib please? I couldn't see anything obvious in the configure options.

If not, we'll have to go with the suggestion in comment 4, re-formatted here for clarity:

Util-Linux-NG: --with-fsprobe=builtin (+ additional commands to install libblkid)
E2fsprogs: --disable-libblkid
Util-Linux-NG: --with-fsprobe=builtin

[Matthew Burgess]

I'd suggest we wait until 2.16 comes out. As mentioned on ​http://marc.info/?l=util-linux-ng&m=124298241223068&w=2, it includes libuuid from e2fsprogs thus removing the circular dependency issue. So, do we wait until this release before we branch for 6.5-RC1, do we branch anyway and make an exception for pushing 2.16 into 6.5-Final, or do we not hold up our release for 2.16?

[Bryan Kadzban]

Udev-142 (at runtime) *requires* at least util-linux-ng 2.15. (I.e., the current book build is broken.) We can't release 6.5 without either (a) upgrading to util-linux-ng 2.15/2.16, or (b) rolling back udev to -141.

The issue with not finding libuuid might just be fixable by overriding CPPFLAGS/LDFLAGS:

./configure --enable-arch --enable-partx --enable-write \
    CPPFLAGS="-I/tools/include" LDFLAGS="-L/tools/lib"

or something like that. Haven't tried this, though; I believe specifying the flags like this (as arguments to ./configure) should work, whereas specifying them in the environment might mess up the "normal" CPPFLAGS/LDFLAGS values in the Makefiles. (This might be backwards though.) It'd be worth running a couple tests to see; I might have time tomorrow to do that.

[bdubbs@…]

Right now I'm thinking that we should hold off -rc1 for util-linux-2.16. The mailing list says they will try to get both util-linux and e2fsprogs updated in the next 2-3 weeks:

​http://article.gmane.org/gmane.linux.utilities.util-linux-ng/2260

In the meantime, I don't see a reason to hold off changing -dev to util-linux-2.15 with the workarounds. The extra time will let us address some of the other tickets that are now targeted for the 7.0 release. I'll make a post to th e-dev mailing list to discuss this.

[Bryan Kadzban]

So I've done a bit of testing with util-linux-ng's build setup, though this might not be entirely valid (details below).

First, I moved /tools out of the way and recreated it as an empty directory. It had been holding the /tools from this system's build, a couple years ago. (This is one reason these results might not be valid: normally this directory has a lot of other stuff in it.)

Then, I moved libblkid and libuuid from e2fsprogs out of their normal homes and into my home directory somewhere, where gcc won't be able to find them. (Both the 32-bit libraries and the 64-bit libraries got moved. This is another reason these results might not be valid: this is a multilib setup. Oh, I also moved the pkg-config files.)

Then, I compiled e2fsprogs, without any special flags (except the standard flag to get it to use -m64), and did a "make install-libs". This created a libblkid file (simulating section 5.19: e2fsprogs in chapter 5).

Then, I extracted and patched util-linux-ng with the -fixes-1.patch in the patches repo. Then, I did a whole bunch of testing to try to figure out where it was getting flag values from. It looks like it *never* runs pkg-config to find libuuid; this seems really wrong to me, since pkg-config was giving the correct flags. In the absence of pkg-config, I had to specify flags manually to get it to find libuuid:

CC="gcc -m64" CPPFLAGS=-I/tools/include/uuid LDFLAGS=/tools/lib64 \
    ./configure --prefix=/tools --with-fsprobe=builtin

finally worked:

checking for uuid_is_null in -luuid... yes

Now, this was a chapter-5-type build (--prefix=/tools), which is the last reason this might not work in chapter 6. I don't see any particular reason that would have any effect, but it is a difference.

Also, note that the variables had to be set in the environment; I thought they could be passed as arguments to configure, but apparently not. (They had no effect: I was echoing out the values inside the script just before it tried to compile the test program, and they didn't show up. That was also how I found out that pkg-config was not getting called.)

Finally, Bruce: I think holding off for 2.16 is probably a good idea, unless we want to get something temporary in place before then for some other reason. The libuuid inclusion means we don't have to go through any of this CPPFLAGS/LDFLAGS stuff, plus they got rid of the --with-fsprobe=builtin requirement. The only problem is that it doesn't quite exist yet... :-)

[Matthew Burgess]

2.15.1-rc1 is now out. See ​http://lkml.indiana.edu/hypermail/linux/kernel/0905.3/02135.html. This does not include the migration of libuuid from e2fsprogs so we'd still have the circular dependency issue, but it does contain the rest of the build fixes we're waiting on. The libuuid migration was done on trunk, i.e. what will become 2.16.

[bdubbs@…]

New version.

​ftp://ftp.kernel.org/pub/linux/utils/util-linux-ng/v2.15/v2.15-ReleaseNotes

Critical point:

libblkid, blkid(8) and findfs(8):

The libblkid library has been moved from e2fsprogs to util-linux-ng. The library has been extended and now includes:

• low-level probing API that is useful for example for udev rules (cmdline: blkid -p -o udev <device>)

• very high-level API that provides portable interface for LABELs and UUIDs evaluation on 2.4, 2.6 and udev-based system. It's recommended

to use "blkid -LU" in your scripts rather than directly read

/dev/disk/by-* symlinks.

• the old ABI and API is backwardly compatible with the current version from e2fsprogs.

[bdubbs@…]

Will add Util-Linux-NG-2.16-rc1 for testing and then update to 2.16 when released.

[bdubbs@…]

I'm going to hold off a bit on this ticket due to recent activity on the util-linux-ng mailing list.

[Matthew Burgess]

So, I've just read ​http://www.linuxfromscratch.org/lfs/view/development/chapter05/e2fsprogs.html again and it looks like the only reason we install e2fsprogs in chapter 5 is because it was a dependency of Util-Linux-NG. So, with the upgrade to 2.16 we should be able to remove e2fsprogs from chapter 5.

[DJ Lucas]

Replying to matthew@…:

So, with the upgrade to 2.16 we should be able to remove e2fsprogs from chapter 5.

Not that it is probably an issue now days, but I am unsure. Verify that the host requirements include a version of mount that supports --bind. I don't remember how old that flag is. IIRC, that was why chapter 5 mount/umount was added years ago.

[Matthew Burgess]

Replying to dj@…:

Verify that the host requirements include a version of mount that supports --bind.

I'm not removing Util-Linux-NG from chapter 5, just its old dependency of e2fsprogs. So, 'mount --bind' shouldn't require anything more from the host than at present.

[DJ Lucas]

Whoops...ID ten T error. My apologies.

[bdubbs@…]

From Guy Dalziel:

Under previous versions of Util-linux-ng we've always complied with the FHS by changing /etc/adjtime to /var/lib/hwclock/adjtime. It now seems that 2.16-rc2 uses /var/lib/hwclock/adjtime and makes the sed command defunct. We can remove therefore remove the FHS compliance notes for 2.16.

[Matthew Burgess]

Replying to bdubbs@…:

It now seems that 2.16-rc2 uses /var/lib/hwclock/adjtime and makes the sed command defunct.

From the Util-Linux-NG-2.16-rc2 sources:

$ grep '#define ADJPATH' hwclock/{clock-ppc,hwclock}.c hwclock/clock-ppc.c:#define ADJPATH /etc/adjtime hwclock/hwclock.c:#define ADJPATH /etc/adjtime

Is this overridden anywhere?

[Matthew Burgess]

From the Util-Linux-NG-2.16-rc2 sources (and this time with formatting!):

$ grep '#define ADJPATH' hwclock/{clock-ppc,hwclock}.c
hwclock/clock-ppc.c:#define ADJPATH /etc/adjtime
hwclock/hwclock.c:#define ADJPATH /etc/adjtime

Is this overridden anywhere?

[Bryan Kadzban]

According to the git repository at ​http://git.kernel.org/?p=utils/util-linux-ng/util-linux-ng.git;a=summary, that appears to be the only mention of "adjtime" anywhere (in code).

Grepping through the util-linux-ng-2.16-rc2 directory for "adjtime", and then "var", produces only one line, in the translations ("var" shows up as part of one of the words in the Danish translation).

It doesn't look to me like this change has been made upstream yet... maybe it will be made before the final release.

[bdubbs@…]

Fixed at revision 8959.

Removed util-linux-ng from Chapter 5.