Udev-122

[Matthew Burgess]

New version. From RELEASE-NOTES:

Bugfixes.

Dynamic rules can be created in /dev/.udev/rules.d/ to trigger actions by dynamically created rules.

SYMLINK=="<value>" matches agains the entries in the list of currently defined symlinks. The links are not created in the filesystem at that point in time, but the values can be matched.

RUN{ignore_error}+="<program>" will ignore any exit code from the program and not record as a failed event.

[Matthew Burgess]

Not sure whether the following is a bug or not, but it breaks a a jhalfs-based build:

# /lib/udev/write_net_rules all_interfaces
# Missing $INTERFACE.

[bryan@linuxfromscratch.org]

Sigh. ;-)

They completely removed the "installation-time" part of the script. According to the commit messages (​http://git.kernel.org/?p=linux/hotplug/udev.git;a=commitdiff;h=9b23e594bf2533b7fadeea1680b186e4775cfe17 and ​http://git.kernel.org/?p=linux/hotplug/udev.git;a=commitdiff;h=dcfa2acce30c0574bcec761b72cad4141da78e1b are the commits: if we revert this, we'll have to revert chunks of both), it looks like they removed it because it was "keeping a second blacklist outside the rules files".

They also had some comment about "this should be called by triggering events, not by looping", but I have no idea what that's supposed to mean. Obviously it needs to be called in a loop when a distro gets installed, otherwise the user isn't going to have a clue what names their devices will have.

Er, hold on, I think I get it now. They're saying that instead of running what we have in the book, you need to run something like:

for dir in /sys/class/net/*/ ; do
    echo "add" >$dir/uevent
done

after installing the files, to pre-generate the rules. That seems a lot more direct (closer to what actually should be happening) anyway.

The only problem is, udevd is running outside the chroot, so it won't update the new system's rules files. It'll only process the host's already-in-place rules, and the script won't ever get called. If we kill udevd and restart it from inside chroot, the process could end up yanking devices out from under the host and renaming them. Neither choice is good.

I'll see if I can find out from the -devel list how we're supposed to handle this from inside chroot...

[bryan@linuxfromscratch.org]

And now -115 is out, but no official solution for the chroot issue.

The only change in -115 that I think will significantly affect us is the addition of the default rules. (It's "a merge of rules between Fedora and SuSE". I suspect we can replace most of 25-lfs.rules with the rules provided.) They also fixed a few bugs that were introduced into -114, and one patch removes the "policy" from the CD rule script (i.e., which types of devices default to by-id, and which default to by-path).

I think all of this should be evaluated for 7.0 -- but I think it's too intrusive to get into 6.3. (Of course, this ticket targets 7.0 anyway, so that's fine.)

[Matthew Burgess]

Udev-116 is now out.

[Matthew Burgess]

Udev-117 has been released. Release announcement at ​http://marc.info/?l=linux-hotplug-devel&m=119499039011244&w=2.

[Matthew Burgess]

Now Udev-118:

udev 118
========
Bugfixes.

Udevstart is removed from the tree, it did not get installed for a long time now, and is long replaced by trigger and settle.

[Julio Meca Hansen]

What is the current status for this package?

Last time this package was commented about was 2 months ago, and I've seen there's a new version of the bootscripts. Maybe this should be the time to revise the actual situation and finally integrate udev-118 in the LFS build?

Julio

[Matthew Burgess]

Replying to LydianKnight:

What is the current status for this package?

Comment #1 is still blocking this package update from occurring. Basically, we have to figure out the best way to get the network devices detected whilst in the chroot environment.

[Jeremy Huntwork]

Bryan, any headway with this one? I'm looking at udev-120 now to see what I can make of it. But in all honesty, this is going to be like starting fresh with udev for me (I've avoided it for so long, so it's probably a good thing). I'm anxious to get this moving, it feels like we're getting left behind here.

[Jeremy Huntwork]

Ok, after some good reading, I think I grasp most of what is happening here now. :) The essential installation of udev is done in the udev section itself. We don't really need to configure persistent rules for a network device until after the first boot, do we?

[bryan@linuxfromscratch.org]

Proposed patch against current udev git

[bryan@linuxfromscratch.org]

Replying to jhuntwork@linuxfromscratch.org:

We don't really need to configure persistent rules for a network device until after the first boot, do we?

Wouldn't that be nice. ;-)

Section 7.13 (Configuring the network Script) requires the user know the interface name that will be assigned to the NIC when the system boots, because that's what the ifconfig.XXXX directory's name is based on. And if there's more than one ethX device, then the assignment will be (or at least, can be) random.

The real problem is that the more recent you get with udev, the more of the logic gets pushed out from the script and into the rules file. This isn't really bad, but when the newest daemon won't start inside chroot (because the host's daemon is running), and nothing will be able to interpret the rules in a compatible way, it's a problem.

(Looks like there have been a few useful changes made since -120, as well. Not that we want to wait even longer, but -121 might be good.)

I've been waffling back and forth between the following options for several months now:

1. Patch udev so it's able to start up inside a chroot, even if the host's udev is running. Involves adding an --in-chroot option to udevd, then firing it up for the shortest time possible (just long enough to udevadm trigger --subsystem-match=net and udevadm settle) to create the rules. The issue would be whether we want to build udev twice (once with the patch, then install it, then create the rules, then go back and recompile / reinstall it without the patch), or we want to keep the extra code hanging around in udevd forever. But we don't have to copy all the logic that's already in udevd into anywhere else with this option.

2. Write a script that copies the device whitelist from the current rules file, and gets updated every time the list of environment variables used for communication (see write_net_rules) changes. Possibly tons of maintenance, but this is entirely independent of udev; no patching or anything crazy like that.

(I've tried posting the patch to upstream once; nobody commented. It's attached.)

Opinions here?

[randy@…]

Replying to Bryan Kadzban:

Opinions here?

Earlier Jeremy said: "I'm anxious to get this moving, it feels like we're getting left behind here."

With all due respect to Jeremy's knowlege and desire, don't let his emotions get in the way of *your* (Bryan's) thoughts on this subject. I've followed this ticket, and have trusted Bryan's judgement.

I say let Bryan make the final call.

Bryan: I only say these things so that you won't let pressure influence your decision.

Jeremy: As always, it isn't personal. I'm just trying to get across that your 'pushing' might make a decision come because of pressure, instead of a well thought out plan.

[bryan@linuxfromscratch.org]

Well, I should also say that I'm getting antsy to get something put together that works too. Jeremy's not the only one. ;-)

Getting an idea of what people think about patching versus scripting would be helpful, though. At this point I'm leaning (back) toward the patch, given how little it actually changes in udevd.

[Jeremy Huntwork]

Replying to Bryan Kadzban:

Wouldn't that be nice. ;-)

Yes. :) And I'm still not sure why it isn't an option, see below.

Section 7.13 (Configuring the network Script) requires the user know the interface name that will be assigned to the NIC when the system boots, because that's what the ifconfig.XXXX directory's name is based on. And if there's more than one ethX device, then the assignment will be (or at least, can be) random.

Right. I'm aware of this one. The only instance that I can think of where this will pose a major problem is when you are building a system over ssh to which you have no physical access to. (My dedicated servers have been like this - thankfully they've only had one ethernet port.) But even this can be overcome. For example, if only one port has a link, simply assign both of them the same ip configuration for the initial boot.

Anyway, I know it's not the prettiest solution, but it's often acceptable (when setting up a machine for the first time) to run through some initial configurations after the first boot. While we could totally move onto BLFS from within chroot, what percentage of users do that? I guess I'm still not sure why we couldn't move some of that configuration of 7.13 to post-first-boot.

The real problem is that the more recent you get with udeRighv, the more of the logic gets pushed out from the script and into the rules file. This isn't really bad, but when the newest daemon won't start inside chroot (because the host's daemon is running), and nothing will be able to interpret the rules in a compatible way, it's a problem.

Understood.

Opinions here?

Well... based on what I've heard so far, I prefer the script option to patching udev.

Actually, what is our main goal here? To be able to configure one working network card and have it's configuration be persistent? If this absolutely must be done inside chroot, wouldn't it be simpler to just replicate the minimal amount of code needed to generate one rule for a chosen port, and then let the rest be generated post-boot?

Is there something else I'm missing? I'll be the first to admit that my head makes for a good cluebat target. ;)

[Jeremy Huntwork]

Replying to randy@linuxfromscratch.org:

Jeremy: As always, it isn't personal. I'm just trying to get across that your 'pushing' might make a decision come because of pressure, instead of a well thought out plan.

If my note above sounded 'emotional' then certainly I didn't phrase it correctly. My intention was just to ping this ticket to bring it some attention, hopefully get updated on Bryan's more recent thoughts about it, and even possibly spark some more discussion about our options.

[bryan@linuxfromscratch.org]

Possible script to call write_net_rules for each NIC (should work with current udev git)

[bryan@linuxfromscratch.org]

Replying to jhuntwork@linuxfromscratch.org:

Section 7.13 ![...]

Right. I'm aware of this one. The only instance that I can think of where this will pose a major problem is when you are building a system over ssh to which you have no physical access to.

Probably, maybe, yeah. ;-) I don't like trying to enumerate all the ways that a change like this could fail, because I know I'm not any good at it (I usually leave a few out).

I think that if the book had always only supported one NIC, then leaving it that way might be fine. It just makes me a bit uncomfortable to change it now that we've released N different book versions that didn't care...

While we could totally move onto BLFS from within chroot, what percentage of users do that?

Not sure about anyone else, but I do that every time I build. I never actually boot up a new build (except once, to make sure it actually boots) until I've installed at least X, firefox, thunderbird, and transcode from inside chroot.

But maybe I'm the only one. :-)

move some of that configuration of 7.13 to post-first-boot.

We might be able to. There's currently nothing that gets run "post-first-boot", though, so it'd be a new section at least. It'd also probably be a pain for any kind of scripted build.

I prefer the script option to patching udev.

I've attached the script that I'm looking at; I probably should have attached it last night as well. This is a horrific hack, is probably completely unsupported by upstream, and will very likely break after a few kernel releases -- but it works under 2.6.24.x, at least for PCI NICs. The more of it that I wrote, the less I liked it.

(When I started writing it, I wasn't thinking about the sysfs stuff that it would need to do to get values like what goes into the rule comment, or the driver name, or the subsystem.)

Actually, what is our main goal here?

I'd say the main goal would be setting stuff up so the system will come up configured properly once the book is done. But that's only because it used to work, and I don't like changing stuff like that unless we need to. ;-)

I guess it depends on which of the script or the patch is a worse hack, and whether the less-hackish one is better than forcing a boot. If neither is better than forcing a boot, then maybe forcing a boot is what we need to do.

wouldn't it be simpler to just replicate the minimal amount of code needed to generate one rule for a chosen port, and then let the rest be generated post-boot?

There's not much more code that would be required to get all NICs working than the code that would be required for one -- see the script. The only extra thing that the script does to get it to work for all NICs is a loop. :-)

The problem with the script (as I see it anyway) isn't the loop -- it's all the assumptions that go into the various readlink calls, and the fact that most of the body of the loop is the shell equivalent of the existing 75-persistent-net-generator.rules file.

For instance, is .../device/driver always valid? Is looking for a certain string in the .../device symlink a valid way to figure out what subsystem the device belongs to? (I doubt that it is.) How fragile is that particular case statement, anyway (are there cases where it misdetects the subsystem)? Is the fact that it doesn't support S/390 devices ever going to be a problem? (Perhaps not.)

[Jeremy Huntwork]

Replying to Bryan Kadzban:

The problem with the script (as I see it anyway) isn't the loop -- it's all the assumptions that go into the various readlink calls, and the fact that most of the body of the loop is the shell equivalent of the existing 75-persistent-net-generator.rules file.

For instance, is .../device/driver always valid? Is looking for a certain string in the .../device symlink a valid way to figure out what subsystem the device belongs to? (I doubt that it is.) How fragile is that particular case statement, anyway (are there cases where it misdetects the subsystem)? Is the fact that it doesn't support S/390 devices ever going to be a problem? (Perhaps not.)

I see what you mean about assumptions and re-using logic built into udev. So the patch is preferable to the script. Even so, I really don't like where that leaves us. Either we leave the hacked up functionality in our final udev, or we build it twice. Ugh to both.

Personally, I think I still prefer to let just let udev auto handle it at first boot. By far most of my systems have one NIC port which will always be named eth0. On other systems that have more than one, it doesn't matter to me which one is named what so long as it is consistent (it will be when udev generates the rule). In those circumstances, if I am only using one NIC then it's easy to find which one is named eth0. If I'm using both NICs, then it will be an advanced network setup that isn't really going to be covered in LFS, and I'll still be able to determine which one needs which configuration. And, I know a user might start doing BLFS stuff while in chroot, but technically, that's not by-the-book anyway. See section 9.3.

In any case, if more feel that it's better to patch up udev so that this configuration can be done in chroot, I'll conform. :)

[Julio Meca Hansen]

version 121 has been released as of May 9th of 2008

here is the list of changes from version 120 to version 121:

libvolume_id: recognize swap partitions with a tuxonice hibernate image
writing udev rules: fix rule typos
rules_generator: net rules - add "dev_id" value to generated rules
selinux: more context settings
udevinfo: do not replace chars when printing ATTR== matches
vol_id: add --offset option
cdrom_id: replace with version which also exports media properties
udevd: at startup write message including version number to kernel log
rules_generator: net rules - always add KERNEL== match to generated rules
selinux: fix missing includes
allow setting of MODE="0000"
path_id: remove subsystem whitelist
logging: add trailing newline to all strings
scsi_id: initialize serial strings
persistent device naming: also read unpartitioned media
cdrom_id: add more help text
add $links substitution
fstab_import: add program to IMPORT matching fstab entry
add OPTIONS+="event_timeout=<seconds>"
write "event_timeout" to db
udevadm: trigger - add --env= option
udevadm: control - fix --env key to accept --env=<KEY>=<value>
udevadm: info - do not print ATTR{dev}==
persistent device naming: update tape rules
rules: update md rules

as usual, it can be downloaded from the following locations:

​http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev-121.tar.gz ​http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev-121.tar.bz2

[Julio Meca Hansen]

and now comes version 122, released as of May 17th of 2008

here is the list of changes from version 121 to version 122:

scsi_id: remove all sysfs dependencies
scsi_id: add SGv4 support
volume_id: clean up linux_raid code
scsi_id:  update man page
scsi_id: remove bus_id option
scsi_id: add --sg-version= option
rules: adapt to new scsi_id
rules: adapt tape rules to new scsi_id
scsi_id: add bsg.h
volume_id: bump version
Makefile: do not create udevcontrol, udevtrigger symlinks
man: udevd- fix udev(8) reference
man: scsi_id
cdrom_id: fix segfault

as usual, it can be downloaded from the following locations:

​http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev-122.tar.gz ​http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev-122.tar.bz2

[dnicholson@…]

Replying to jhuntwork@linuxfromscratch.org:

In any case, if more feel that it's better to patch up udev so that this configuration can be done in chroot, I'll conform. :)

Just throwing out an opinion...

I think that the right thing to do, if the --in-chroot patch is rejected, would be to just right a stripped down udevd that just does what we want. Basically, just strip out all the parts of udevd.c that aren't applicable (like actual device creation and RUN rules), then add code for triggering the net events. You could even call it write_net_rules.

I'd rather see this move than wait for that to happen, but I think it would be fairly trivial for someone knowledgeable in C to copy udevd.c and strip to just the calls needed to trigger the net rules.

[bryan@linuxfromscratch.org]

Replying to dnicholson@linuxfromscratch.org:

I think that the right thing to do, if the --in-chroot patch is rejected,

I've tried sending it to the -devel list; the only response at the time was from the maintainer of the Debian package (not sure on his status as an official udev maintainer), and wasn't very favorable.

I didn't push it very hard upstream after that; I figured that if Debian had a way around it, we could probably figure something out too. (Turns out their "way around it" is to use the same version of udev outside chroot, and then copy the rules during the install. Not exactly going to work for us.)

would be to just right a stripped down udevd that just does what we want. Basically, just strip out all the parts of udevd.c that aren't applicable (like actual device creation and RUN rules), then add code for triggering the net events.

Hmm. So do the script, but do it in C so you have access to the data coming over the netlink socket for the uevent (including the subsystem, etc.), and don't have to guess about all of that info.

There's more code from udev that would be needed than just udevd.c, I think: you'd need the rule parsing code, the sysfs access code (if that's still part of the package; I believe it is but haven't looked in a couple years), possibly the SELinux code (though not likely), etc. If you kept it a daemon -- but there's no particular need to do that -- then you'd also need to keep the /dev/.udev/* handling code, so "udevadm settle" would work with it.

Otherwise, you'd have to make it trigger the uevents itself, but that's just a readdir loop and writing to the uevent file. Of course you'd have to somehow do that while it was listening for the uevents, which might be hard (otherwise it might miss them? maybe not though; depends on the semantics of netlink sockets).

The issue that I see would be keeping it in sync with the upstream code, so it can keep using the same set of rules. I'm not sure whether that would be more work than keeping the patch in sync or not (really all the patch does is add code to recognize the command line option, and check the stored value in two or three places; it doesn't really change any upstream code, it just adds a few checks to skip some of it in certain cases).

Hmm...

[alexander@…]

Debian doesn't have a way around it: they just:

• use udev in their installer CD

• let the user configure the network interfaces under the (random) names assigned by the installer

• copy the resulting configuration and persistent rules to the installed system

So they don't have a way to say "let eth1 be the same card that was named eth1 in my previous distro" (and that's, essentially, the goal of LFS). So my opinion is that we should write rules by hand, explaining the logic (it is not that hard), or, optionally, postpone the configuration until after the reboot (thus losing predictable names and the ability to do remote installations).

[Julio Meca Hansen]

I think it's the best option to follow... trying to give consistent names to network devices in the chroot phase using the variety of proposed methods seemed a bit strange for me (and the reason why I always used the CLFS book for udev)

LFS has an educational value, and trying to do the work for the users doesn't seem to be quite educational (from my personal point of view), although... device naming policies aren't specially the easiest part in the system configuration, but it's not that difficult. BLFS has a chapter for naming (for example) the CDRW drive and so on, giving a persistent name for the network card shouldn't be more difficult than that

I see more coherent to write some text indicating some examples of rules to accomodate for a certain piece of network hardware, less work needed to keep the book advancing and more time to overcome other tickets & problems

(but that's my opinion) ;)

Julio

[bryan@linuxfromscratch.org]

Replying to alexander@linuxfromscratch.org:

So they don't have a way to say "let eth1 be the same card that was named eth1 in my previous distro" (and that's, essentially, the goal of LFS).

Well, that's what the patch would actually get you. My goal was less specific than that, though; my goal was to generate some fixed name (randomly or not, it doesn't matter) before the user runs through section 7.13 and creates the configuration files based on the NIC's name. It doesn't have to be the same name as the parent distro (though it will be here); it just has to be known when the network script is set up (in chroot, currently), and fixed from that point forward.

So my goal is the same as what Debian actually does; the only reason their approach won't work for us is that they can rely on the version of udev on the installer CD (their host).

(If the host doesn't run udev, or runs an old version that doesn't do NICs, then the generated names could be random; that would be fine.)

So my opinion is that we should write rules by hand, explaining the logic

Which is what the book used to do, until write_net_rules came along.

Of course, the logic has gotten a lot more complicated since we yanked that section: excluding Xen devices, skipping devices that have the "local" bit set in their MAC, including comments so you know which device the rule is for when you look at the generated file later, and even letting external tools like biosdevname (for Dell server systems) generate a persistent name. I would rather not duplicate all of that.

But OTOH, if you generate a rule that doesn't have quite enough info in it, you'll start to get users that have *_rename devices because the rule that they wrote ended up matching two (or more) of their interfaces. Or they use a NIC driver that generates a random MAC on every boot, but don't know it, so their ethX name keeps changing.

I have a feeling that this could cause a lot more traffic on -support. Of course, I don't know for sure.

or, optionally, postpone the configuration until after the reboot (thus losing predictable names and the ability to do remote installations).

I wonder how many users require that (that is, remote installations). There's probably no way to know, except yanking it out and seeing who complains. (That's assuming people complain, instead of just moving to some other setup.) It's not like asking on -dev or -support is going to reach all the users.

[bryan@linuxfromscratch.org]

Replying to LydianKnight:

trying to give consistent names to network devices in the chroot phase using the variety of proposed methods seemed a bit strange for me (and the reason why I always used the CLFS book for udev)

You don't mind if, while you're running through CLFS's section 11.13, you don't know what name the card will have when you do boot?

I would definitely mind. But maybe that's just me. :-)

LFS has an educational value, and trying to do the work for the users doesn't seem to be quite educational

That is true. Ideally the user would go investigate what the commands do, which could lead into a lot of knowledge about netlink, select, udev's internals, etc., but that probably isn't very likely. Of course, the same could be said of all the package installation instructions: if someone just copies and pastes, they won't learn anything no matter how we do it. ;-)

That being said, I'm not sure how to make it more educational, while still not going overboard duplicating effort. Upstream already has a system that works in tons of cases, including systems that I haven't ever heard of (e.g. S/390); I'd rather not duplicate that if we can use it.

(True, the book doesn't support S/390. But there's all the work that went into wireless cards whose drivers create multiple interfaces, and Xen devices, and external-tool support. Plus there's the work that will go into the next change that needs to be made to handle some new type of card; I'd rather let udev figure all that out and just use it, rather than having to copy it.)

BLFS has a chapter for naming (for example) the CDRW drive and so on, giving a persistent name for the network card shouldn't be more difficult than that

It isn't difficult if you only have one card, or have physical access to the machine and don't care if networking comes up correctly on the first boot ("do nothing" would suffice there ;-) ). But it is more difficult if you have two wireless cards, or if you have a Dell server and want to use biosdevname for its NICs. Or if you want to use some completely different tool to generate the names.

Maybe none of that is needed in most cases. But I've never liked the 80/20 rule. :-P

write some text indicating some examples of rules to accomodate for a certain piece of network hardware,

This seems to be what we did in the 6.2 book (​http://www.linuxfromscratch.org/lfs/view/6.2/chapter07/network.html for instance)? Note all the exceptions below the example rules; that list is longer now. Should we keep that list in the book (including additions to it), or expect the user to find it on their own? Or not support that hardware (which I think is equivalent to "expect the user to find it on their own")? Or something else?

Since upstream has already written a set of rules that will work for all these cases already, I'd like to simply use them. :-)

(I realize I've basically disagreed with everyone that responded to my comment request. That doesn't mean I don't appreciate the comments, though: thank you all! :-) At minimum, they've helped me figure out what exactly was needed, and why I want to go the way I want to go.)

[Julio Meca Hansen]

Replying to Bryan Kadzban:

You don't mind if, while you're running through CLFS's section 11.13, you don't know what name the card will have when you do boot?

I would definitely mind. But maybe that's just me. :-)

I have to admit you have a point ;)

All my machines, whether it's a laptop or a desktop machine only have a single ethernet port and in the case of my laptop, a wireless adapter, so I don't have to face the plethora of situations a user with a server machine or just a machine with several network adapters have, although I think the vast majority of users won't have more than one network adapter, wired+wireless at best (setting up a network connection over an IR beam seems quite strange for me even if the kernel has some facilities for it... and that's not the common rule (I suppose)), with bluetooth, I couldn't say...

Anyway, the 'rule' I tend to follow is to take a quick look at the /sys hierarchy, and after I've found something interesting, I run udevtest with the desired parameter to inspect some of the capabilities and information for a given adapter (same case as the naming for a CDRW drive, for example)

Maybe none of that is needed in most cases. But I've never liked the 80/20 rule. :-P

Maybe my point of view sounded a bit selfish but that wasn't my intention, although I think the best option would be to 'prepare' the underlying infrastructure in terms of device configuration, but let the users be the ones who give the final touch to the configuration, I mean... we could add some text about the /sys hierarchy among some examples of current network device naming, how to properly modifying a written rule to add some of their devices, given some of the 'major' distributions just do a copy'n'paste of the host rules or part of them

(but frankly speaking, would be nice to see this effort reaching a stable solution for all, no other distro seems to have reached a proper solution (at least debian like it's mentioned) and hey, that would definitely be a great point for LFS)

This seems to be what we did in the 6.2 book (​http://www.linuxfromscratch.org/lfs/view/6.2/chapter07/network.html for instance)? Note all the exceptions below the example rules; that list is longer now. Should we keep that list in the book (including additions to it), or expect the user to find it on their own? Or not support that hardware (which I think is equivalent to "expect the user to find it on their own")? Or something else?

With the proper instructions (like I have mentioned before) I don't see why a LFS user can't find the correct names for his/her network devices, we're not asking users to patch/sed some lines of code in a given file to be able to accomodate their purposes, but to inspect a bit their new system to customize it a bit more

Anyway, it's just an oppinion (and I'm thrilling to see a final decision on this) :)

Julio

[bryan@linuxfromscratch.org]

Replying to LydianKnight:

although I think the vast majority of users won't have more than one network adapter, wired+wireless at best

And even if you have one wired/one wireless, you will usually have different basenames for the two interfaces (ethX versus wifiX/wlanX, athX, raX, etc.). :-)

take a quick look at the /sys hierarchy, and after I've found something interesting, I run udevtest with the desired parameter to inspect some of the capabilities and information for a given adapter (same case as the naming for a CDRW drive, for example)

Yeah; with some knowledge about all the different interfaces in the system (not just cards), and what makes each one unique, this works OK. The issue is, if you don't know the pitfalls for your driver(s), it's easy to fall into them. :-)

Maybe none of that is needed in most cases. But I've never liked the 80/20 rule. :-P

Maybe my point of view sounded a bit selfish

Not really -- that 80/20 comment was more because I hear a lot of it at work. I don't really agree with a "The last 20% doesn't matter enough to worry about!" type of a mindset.

It works out OK if you're looking at some physical system -- usually, anyway -- but software is usually a lot more complicated. There is a limit beyond which you shouldn't worry (and maybe these multi-NIC cases are beyond that limit), but the limit is almost never at 80%; if only 80% of some system works, it's still too buggy, IMO. :-)

(Plus I always seem to be using the last 5%: the stuff that got left buggy because of some other group's equivalent of 80/20. :-P )

we could add some text about the /sys hierarchy

This is probably a good idea anyway. At least an overview of what's where in current kernels, and how to find stuff starting in e.g. /sys/class or /sys/bus.

(but frankly speaking, would be nice to see this effort reaching a stable solution for all, no other distro seems to have reached a proper solution (at least debian like it's mentioned) and hey, that would definitely be a great point for LFS)

You're right, but I think the reason other distros don't need this is that they don't really care what happens if you move from another distro to them. You go via their CD (probably just like Debian), and their CD can generate the rules. LFS is fairly unique that way, I think.

we're not asking users to patch/sed some lines of code in a given file to be able to accomodate their purposes, but to inspect a bit their new system to customize it a bit more

Yeah, that's true, and maybe it's not terribly difficult.

I guess my biggest issue with having users write the rules themselves is probably that I don't expect users to need to know whether their card requires a "special" rule or not, which means we'd have to tell them (or have them get it wrong). We'd have to keep track of which type of setup would require extra match keys to work right, and if that gets missed (either we miss a new special setup, or if the user skips that section), then the whole system fails.

[alexander@…]

Replying to Bryan Kadzban:

This seems to be what we did in the 6.2 book (​http://www.linuxfromscratch.org/lfs/view/6.2/chapter07/network.html for instance)? Note all the exceptions below the example rules; that list is longer now. Should we keep that list in the book (including additions to it), or expect the user to find it on their own? Or not support that hardware (which I think is equivalent to "expect the user to find it on their own")? Or something else?

The long list of exceptions is a red herring. Moreover, it explains why modern versions of udev write rules this way, so it counts as educational content. What really matters is that the rule matches the correct device uniquely, and we should state this goal explicitly and provide a way to check this (e.g., say that the *_rename interface will appear after the reboot in the case of an error).

[alexander@…]

Also don't forget that the following case exists where the upstream generator doesn't provide the correct rules and the *_rename interface appears sometimes with unpatched udev (already reported, cantfix):

• one network card with a "real" MAC address
• one network card with a "local" MAC address

So the only way to get out of this situation is to write location-based or driver-based rules manually.

[Julio Meca Hansen]

I have an idea, but I don't know how well it could work

Instead of trying to create a cut version of udev or just copying the host rules... Could we try to, in the host and prior to begin the construction of the temporary LFS system, update the PCI & USB device database, make a quick run of the lspci & lsusb commands, grep the result for the words 'Ethernet' (for ethernet ports) and 'Network' (for wireless devices), add those results into a file, saved into $LFS/sources and run sort of a script to make use of the first characters in the file for each line?

I mean, the first characters in each file (at least for the lspci command) correspond to the pci address space id like this:

02:00.0 Ethernet controller: Intel Corporation 82573L Gigabit Ethernet Controller
03:00.0 Network controller: Intel Corporation PRO/Wireless 3945ABG Network Connection (rev 02)

we could then try to write a udev rule for the devices based on its pci address space id rather to try anything else, but... I don't know how efficient or inefficient this would be, but if this could work for all, the only requisite we should add to the host system would be to have a recent version of the pci.ids & ubs.ids (but I haven't tried the usb part, just an idea that bumped into my mind yesterday tonight and I have been thinking about it)

Any comments on this? :)

Julio

[alexander@…]

IMHO the discussion in this ticket became too diverged. Some people (e.g. Bryan Kadzban) try to formulate the problem (with words other than "jhalfs broke") and the goals. Other people (e.g., LydianKnight) present solutions to their own understanding of the problem, which may or may not be the same as, e.g., my understanding.

My idea is that the problem of configuring the network interfaces (i.e., getting the correct udev rules and writing the corresponding interface configuration files) has multiple solutions. One of them is to defer the configuration until the reboot (that's simple but doesn't work with the important case of remote installations), another is to write udev rules by hand or by script or even by copying from a suitable host (this is a bit more complicated to get right, and is overkill in most of cases). The book should not present only one of the methods (that would not be educational enough), but must present the goal, include an overview of some available methods to reach it, and provide some means to verify that it has been in fact reached.

[bryan@linuxfromscratch.org]

Replying to alexander@linuxfromscratch.org:

and we should state this goal explicitly and provide a way to check this

That sounds like a pretty good idea, actually.

(e.g., say that the *_rename interface will appear after the reboot in the case of an error).

Hmm; I'm not sure that after the reboot is the best time to test it: it's already too late if you don't have (easy) physical access to the machine. But that's just an example, too.

What may work is using udevtest, if we don't mind using it. (Not sure on the status of it upstream: at one point I think it was slated to disappear, but that was before the ability to read from the sysfs uevent file was added to the kernel.) If we run udevtest inside chroot for each NIC and make sure each one gets a different NAME applied, that may be sufficient. Catching bugs that have to do with cards getting random MACs would be a problem, though.

The rules won't be nearly as nice-looking as the ones that upstream generates, either. Whether that matters or not is a different issue though. :-)

(Actually, one thing I haven't tried is seeing whether udevtest runs IMPORT{program} keys. If it does, that may solve all the issues. Let me test it...)

HAH! That should work fine. :-) OK, a new option: Unless udevtest is evil for some other reason, we can just run:

for i in /sys/class/net/*[0-9] ; do
    udevtest $i
done

, and that will pre-generate everything. It will run the IMPORT programs just like udevd would. Unless I'm missing something critical, this should work as long as udevtest keeps working that way.

Replying to alexander@linuxfromscratch.org:

So the only way to get out of this situation is to write location-based or driver-based rules manually.

Well, that's one way to fix it. Another way could be to let all the "local" MAC address cards get randomly-assigned names that grow from the top of the numeric range, and let all the "real" MAC address cards get persistent names at the bottom of the numeric range.

For instance, say you have four cards, two of each type. Give the first "real" card that gets rules generated for it eth0, and give the second "real" card eth1. The other two cards will be given some name up higher; they could probably trade off between eth32767 and eth32766 (assuming the "top" of the range gets set to the maximum 16-bit signed integer).

(Of course, path-based rules are probably easier. They'd be even easier if path_id wouldn't require a "dev" attribute in sysfs for the DEVPATH in question; NICs don't have one. :-) )

[bryan@linuxfromscratch.org]

Replying to LydianKnight:

we could then try to write a udev rule for the devices based on its pci address space id rather to try anything else

Yep; that's basically what is required if your card's driver gives it a random MAC address on every boot.

The issue is that it fails once you move a card to a different slot. (Maybe that doesn't happen often, I'm not sure. At least not for PCI devices -- but it's a real problem for at least USB-connected NICs, if they get moved to another USB port.) And depending on how the rule gets written, two different USB NICs at the same position on two different USB buses (where, say, each bus has a unique PCI position because one is connected to a USB 2 add-in card, while the other is directly on the motherboard), could still collide. You'd have to use something like udev's path_id to generate the full path to the NIC, across all bus types.

Anyway, that's basically just adding another type of persistence (one which the user could -- maybe -- generate a rule for manually, so it doesn't fall victim to some of the same issues as other in-chroot-rule-generation methods).

But I still like udevtest, at least for the moment. :-)

[bryan@linuxfromscratch.org]

Replying to alexander@linuxfromscratch.org:

My idea is that the problem of configuring the network interfaces (i.e., getting the correct udev rules and writing the corresponding interface configuration files) has multiple solutions.

Yes, certainly.

The book should not present only one of the methods (that would not be educational enough), but must present the goal, include an overview of some available methods to reach it, and provide some means to verify that it has been in fact reached.

At the risk of harping on a bad solution, "or just use udevtest". :-P

Although actually, I do think that even if we do go with udevtest, it'd still be a good idea to present the goal (as you say) and say "this section explains how we recommend achieving this goal, but there are alternate solutions; see XXXX and YYYY". Those can be hints, or in-book paragraphs (of course then the sentence structure would change), or whatever. They could even be (in the future) alternate book renderings based on the user's choice, though we don't have the infrastructure for that yet.

And if we don't go with udevtest, then that kind of explanation should still certainly be there, yes.

[dnicholson@…]

s/udevtest/udevadm test/ for newer releases. I think the functionality is the same.

[alexander@…]

"udevadm test" explicitly says that it doesn't run programs specified in the RUN key (and this applies to IMPORT, too), and thus cannot write rules. Please stop pushing a solution that can't work.

[alexander@…]

OTOH, "./test-udev" (the program used by udev's own testsuite, but not installed by default) may work.

[alexander@…]

When testing ./test-udev starting from Debian Sarge (with kernel 2.6.8), I noticed that udev fails to get the subsystem from sysfs, thus causing the generator to be skipped. So, if we implement the solution based on ./test-udev or the --in-chroot patch, we will have to up the required kernel version of the host.

[alexander@…]

On Sarge, the following voodoo created almost-correct rules (i.e. only wrong comments):

# Compensate for the missing "subsystem" symlink
cp -a /sys /sys-copy
for a in /sys-copy/class/net/* ; do ln -s ../../../class/net $a/subsystem ; done
for SYS in /sys-copy/class/net/* ; do SYSFS_PATH=/sys-copy DEVPATH=${SYS#/sys-copy} ACTION=add SUBSYSTEM=net INTERFACE=${SYS##*/} ./test-udev net ; done

Will retry with a more recent kernel now... However, I don't think that abusing an undocumented test tool is a good idea.

[alexander@…]

2.6.12 is still too old (doesn't have the "subsystem" symlink)

[alexander@…]

2.6.16.38 is still too old, for the same reason.

[alexander@…]

With 2.6.22.5, the rules got generated correctly, including comments:

for SYS in /sys/class/net/* ; do \
    DEVPATH=${SYS#/sys} \
    ACTION=add \
    SUBSYSTEM=net \
    INTERFACE=${SYS##*/} \
    ./test-udev net ; \
done

However, that's a fairly recent kernel, not sure we want to require that (see above: a simple workaround exists, but loses PCI/USB information in comments). And my comment about abusing the testing tool still stands, so I don't recommend this solution.

[alexander@…]

In the previous example, the "net" argument after "./test-udev" is not needed (but harmless).

[alexander@…]

Actually, linux-2.6.18 (from Debian Etch) also produces correct udev rules without the subsystem hack. According to ​http://lwn.net/Articles/187921/ (search for "subsystem"), 2.6.18 is indeed the minimum required kernel version.

[bryan@linuxfromscratch.org]

Replying to alexander@linuxfromscratch.org:

"udevadm test" explicitly says that it doesn't run programs specified in the RUN key

Correct.

(and this applies to IMPORT, too)

Hmm; not with udevtest from udev-110, or with "udevadm test" from udev-122. I just deleted everything udev-related, installed -122, re-added my custom rules, and ran the test (I also ran it on -110 last night). This was without a generated rule file present, since I didn't move it back:

# udevadm test /class/net/eth0
This program is for debugging only, it does not run any program,
specified by a RUN key. It may show incorrect results, because
some values may be different, or not available at a simulation run.

parse_file: reading '<bunch of files>' as rules file
parse_file: reading '/etc/udev/rules.d/61-persistent-storage-edd.rules' as rules file
parse_file: reading '/etc/udev/rules.d/75-cd-aliases-generator.rules' as rules file
parse_file: reading '/etc/udev/rules.d/75-persistent-net-generator.rules' as rules file
parse_file: reading '<bunch more files>' as rules file
import_uevent_var: import into environment: 'INTERFACE=eth0'
import_uevent_var: import into environment: 'IFINDEX=2'
udevtest: looking at device '/devices/pci0000:00/0000:00:11.0/net/eth0' from subsystem 'net'
match_rule: set ENV 'MATCHADDR=00:13:8f:a6:9e:2b'
match_rule: set ENV 'MATCHIFTYPE=1'
match_rule: set ENV 'COMMENT=PCI device 0x10b9:0x5263 (uli526x)'
run_program: 'write_net_rules'
run_program: '/lib/udev/write_net_rules' returned with status 0
udev_rules_get_name: no node name set, will use kernel name 'eth0'
udevtest: run: 'socket:@/org/kernel/udev/monitor'
# udevadm test /class/net/eth0
This program is for debugging only, it does not run any program,
specified by a RUN key. It may show incorrect results, because
some values may be different, or not available at a simulation run.

parse_file: reading '<bunch of files>' as rules file
parse_file: reading '/etc/udev/rules.d/61-persistent-storage-edd.rules' as rules file
parse_file: reading '/etc/udev/rules.d/70-persistent-net.rules' as rules file
parse_file: reading '/etc/udev/rules.d/75-cd-aliases-generator.rules' as rules file
parse_file: reading '/etc/udev/rules.d/75-persistent-net-generator.rules' as rules file
parse_file: reading '<bunch more files>' as rules file
import_uevent_var: import into environment: 'INTERFACE=eth0'
import_uevent_var: import into environment: 'IFINDEX=2'
udevtest: looking at device '/devices/pci0000:00/0000:00:11.0/net/eth0' from subsystem 'net'
udev_rules_get_name: rule applied, 'eth0' becomes 'eth0'
udevtest: run: 'socket:@/org/kernel/udev/monitor'
#

It didn't parse a 70-persistent-net.rules file on the first run, and it did run the write_net_rules script according to the output. It did parse a 70-persistent-net.rules file on the second run, and the interface had a NAME assigned on the second run as well. It looks like it works fine. :-)

Now, it's possible they'll remove this support in the future, I'm not sure. But since it does work for now, I'd feel slightly more comfortable using it than the test-udev binary. (But only slightly.)

I'm guessing that 2.6.18 is probably the minimum required kernel for this method as well, unless "udevadm test" is able to find the required information using the old method. Unfortunately I don't have a bunch of kernels to test it on...

(Actually, the "read from the uevent attribute" code wasn't added to the kernel until something more recent than 2.6.18. But if needed, we should -- hopefully -- be able to work around that by setting INTERFACE manually before running udevadm. Google seems to think that the minimum kernel for reading from uevent is 2.6.24; I don't have anything older than that available to test.)

[alexander@…]

My bad. With 2.6.18, the following works:

for SYS in /sys/class/net/* ; do \
    INTERFACE=${SYS##*/} udevadm test --action=add --subsystem=net $SYS ; \
done

[bryan@linuxfromscratch.org]

Replying to alexander@linuxfromscratch.org:

With 2.6.18, the following works:

for SYS in /sys/class/net/* ; do \
    INTERFACE=${SYS##*/} udevadm test --action=add --subsystem=net $SYS ; \
done

Since you're providing --subsystem on the udevadm command line, might this work with kernels that don't provide the subsystem symlink on their own (i.e. older than 2.6.18)? It would be nice if it did (if you still have any of those kernels around to test). :-)

Although even if it does work on older kernels now, future changes to udev might break it, too. Hmm.

[alexander@…]

Replying to Bryan Kadzban:

Since you're providing --subsystem on the udevadm command line, might this work with kernels that don't provide the subsystem symlink on their own (i.e. older than 2.6.18)? It would be nice if it did (if you still have any of those kernels around to test). :-)

This does work on older kernels. As for --action=add, it is just the default and should be dropped to save space. So we have:

for SYS in /sys/class/net/* ; do \
    INTERFACE=${SYS##*/} udevadm test --subsystem=net $SYS ; \
done

plus a 2.6.18 kernel requirement for correct comments, and just 2.6.8 (or even below) for correct rules in the simplest case. Tested with the following kvm line:

kvm -hda hda.dsk \
    -net nic,macaddr=50:54:00:12:34:56,model=ne2k_pci -net user \
    -net nic,macaddr=50:54:00:12:34:57,model=rtl8139 \
    -net nic,macaddr=50:54:00:12:34:58,model=pcnet \
    -net nic,macaddr=50:54:00:12:34:59,model=pcnet

Since the SUBSYSTEMS keyword doesn't work with kernels below 2.6.18, the commands above, when applied to old kernels, create incorrect rules for the following two cases, neither of which are relevant to today's LFS:

• Xen
• S/390

and, of course, in all cases where upstream udev doesn't create sensible rules with modern kernels.

[bryan@linuxfromscratch.org]

OK, I think it's time to call this one fixed, at least for now. See r8545 and r8546 (plus a few supporting changes to udev-config and bootscripts: r8542, r8543, and r8544). I don't see a problem with removing the --action=add at some point, but it doesn't hurt anything to have it in for now. (I may remove it the next time I check something else in.)