shadow-4.1.2.1

[ken@…]

This moved from PLD to ​http://pkg-shadow.alioth.debian.org, current version is 4.1.0.

The page says:

2007-12-10 - New release: 4.1.0

A new version of shadow was released. It contains patches gathered from various distributions, the documentation of the login.defs variables, support for SHA password encryption methods, and lots of bug fixes.

The manpages are now generated depending on the configuration. It is recommended to re-generate them if you change some configure options.

and

2007-10-07 - New repository layout for upstream development

Upstream development of the shadow tool suite used to be hosted by the PLD project.

It will now be hosted on Alioth.

The pkg-shadow repository was reorganized to separate upstream maintenance from the Debian package mainenance.

The shadow history was imported using old archives and the last known snapshot of CVS repository on PLD.

[Matthew Burgess]

This version fixes the useradd/usermod groupname vs. groupid bug that can still be observed in 4.0.18.2 (see #2145).

Here's a couple of suggestions for the instructions:

sed -i -e 's@#ENCRYPT_METHOD DES@ENCRYPT_METHOD MD5@' etc/login.defs

instead of

sed -i -e 's@#MD5_CRYPT_ENAB.no@MD5_CRYPT_ENAB yes@' -e 's@/var/spool/mail@/var/mail@' etc/login.defs

According to the comments in login.defs, MD5_CRYPT_ENAB is deprecated, in favour of the new ENCRYPT_METHOD setting.

Then:

sed -i -e 's/yes/no/' etc/useradd

instead of

sed -i 's/yes/no/' /etc/default/useradd

This is just to make it more consistent with the login.defs edit, where the change is made prior to installation, rather than after.

Finally, we still see:

useradd: unknown GID 1000

This is because etc/useradd has

GROUP=1000

useradd apparently complains of the non-existent default group, even when a specific, existent group is provided on the command line. So, we could provide a group with GID=1000 (presumably this would be a catch-all 'users' group I've seen on occasion). Or we could change the default to a group we already have in /etc/passwd or a new more suitable one if that's deemed a better option. Or, we could remove the default group setting and all related handling of it in useradd & usermod (which is part of what the current Shadow patch does).

I'd rather not have a non-upstream patch though, if at all possible.

[bryan@linuxfromscratch.org]

Replying to matthew@linuxfromscratch.org:

Or, we could remove the default group setting and all related handling of it in useradd & usermod (which is part of what the current Shadow patch does).

Not sure if anything other than this would be needed or not, but how about a:

sed -i -e '/^GROUP=/d' etc/useradd

before installation? Or is that what you meant with the "related handling of it in useradd and usermod" comment?

If changes to the source of useradd and usermod are required, then I'd probably be in favor of adding a GID-1000 group. I don't think any of the groups that we have would be right for this setting, except *possibly* nogroup.

[ken@…]

Here's a couple of suggestions for the instructions:

sed -i -e 's@#ENCRYPT_METHOD DES@ENCRYPT_METHOD MD5@' etc/login.defs

instead of

sed -i -e 's@#MD5_CRYPT_ENAB.no@MD5_CRYPT_ENAB yes@' -e 's@/var/spool/mail@/var/mail@' etc/login.defs

I assume the loss of

 -e 's@/var/spool/mail@/var/mail@'

was an accident ?

[ken@…]

Replying to Bryan Kadzban:

Replying to matthew@linuxfromscratch.org:

Or, we could remove the default group setting and all related handling of it in useradd & usermod (which is part of what the current Shadow patch does).

Not sure if anything other than this would be needed or not, but how about a:

sed -i -e '/^GROUP=/d' etc/useradd

before installation? Or is that what you meant with the "related handling of it in useradd and usermod" comment?

If changes to the source of useradd and usermod are required, then I'd probably be in favor of adding a GID-1000 group. I don't think any of the groups that we have would be right for this setting, except *possibly* nogroup.

If we just sed it out, we get

root in chroot ~# useradd fubar
useradd: unknown GID 100

and the last line of /etc/group is now

fubar:x:1000:

This is partly coming from useradd.c, but I'm unsure how it generated 1000 in that case :

/*
 * Global variables
 */
/*
 * These defaults are used if there is no defaults file.
 */
static gid_t def_group = 100;
static const char *def_gname = "other";
static const char *def_home = "/home";
static const char *def_shell = "";
static const char *def_template = SKEL_DIR;
static const char *def_create_mail_spool = "no";

I could live with a group of 1000, called 'users', unless anybody objects.

And don't you love the way trac formatted what should have been code (for the sed) in the quote ?

[ken@…]

With a group of 1000, and omitting the sed to remove GROUP= from /etc/useradd, I now get no warning messages. The user was created in group 1001, dunno if that is what you guys expected ? I certainly didn't.

/# cat /etc/default/useradd 
# useradd defaults file
GROUP=1000
HOME=/home
INACTIVE=-1
EXPIRE=
SHELL=/bin/bash
SKEL=/etc/skel
CREATE_MAIL_SPOOL=no

'useradd -g users' works ok.

[bryan@linuxfromscratch.org]

Replying to ken@linuxfromscratch.org:

The user was created in group 1001, dunno if that is what you guys expected ?

By that do you mean that the user's primary GID (the one in /etc/passwd) was 1001? What group name is that?

I didn't really expect it, no...

[ken@…]

Replying to Bryan Kadzban:

Replying to ken@linuxfromscratch.org:

The user was created in group 1001, dunno if that is what you guys expected ?

By that do you mean that the user's primary GID (the one in /etc/passwd) was 1001? What group name is that?

I didn't really expect it, no...

First time, I added user fubar with just 'useradd fubar', then user barfu with -g users. From passwd:

fubar:x:1000:1001::/home/fubar:/bin/bash
barfu:x:1001:1000::/home/barfu:/bin/bash

and from group:

users:x:1000:
fubar:x:1001:

so, if I'm reading it correctly, useradd seems to have invented a group, given it the user's name, and made it the user's primary group. Yes, here's a cleaner example

root in chroot /# useradd third
root in chroot /# grep third /etc/passwd
third:x:1002:1002::/home/third:/bin/bash
root in chroot /# grep third /etc/group 
third:x:1002:

This is the expected behaviour for distros that put each user in their own group - from memory, all the rh-derived distros, not sure about debian-derived.

[bryan@linuxfromscratch.org]

Replying to ken@linuxfromscratch.org:

so, if I'm reading it correctly, useradd seems to have invented a group, given it the user's name, and made it the user's primary group.

That's what I see as well.

from memory, all the rh-derived distros, not sure about debian-derived.

Yep, that sounds correct regarding RH, anyway. The Debian box I have at work has a single (non-root) user, who is a member of a single group that has the same name as the user -- but that group has GID 1000, so I may have changed its name when I first added the user a couple years ago. I don't remember for sure.

But what I'd like to know now is, why force everyone to have a single common group, and then avoid using it? Maybe this is just an interim measure though; that could be.

[Matthew Burgess]

Replying to Bryan Kadzban:

But what I'd like to know now is, why force everyone to have a single common group, and then avoid using it? Maybe this is just an interim measure though; that could be.

See ​http://lists.alioth.debian.org/pipermail/pkg-shadow-devel/2008-February/006334.html - apparently this 'create same group as username' is a complete mess across distributions and upstream. I think the quickest fix for this issue is:

sed -i -e 's/nflg = 0/nflg = 1/' src/useradd.c

The fact that useradd complains about an invalid default group, even if '-g' is specified on the command line is an annoyance but is it worth fixing? If so, I'd think that moving the call to 'get_defaults()' to after 'process_flags()' then add a check in 'get_defaults()' to see if '-g' was specified. If it was, don't warn about the default group, as it isn't going to be used anyway. Haven't got time to look into this any further at the moment so I don't know whether 'get_defaults()' depends on 'get_defaults()' having been read.

[steve.crosby@…]

Is this ticket the place to be discussing the shadow suite administration *policy*? Whether or not the default operation of shadow is to create a group per user isn't a bug per se, and the behaviour can be modified with command line switches.

And should LFS be setting system administration policy for users? I have no issue with having an information box\hint (Here is how you can change the compiled default group creation policy for useradd) but having that in mainstream instructions is just a recipe for creating dissention as soon as someone who disagrees with you about what the default poicy should be.

[Matthew Burgess]

The issue here, Steve, is that the 'create a group with the same name as a user' policy *cannot* be controlled via a command line switch at the moment. Although the flag variables are in the source (see 'nflag' sed above), no code is present that handles a '-n' command-line switch. That means that the current version of Shadow is inflicting the policy (which is different than prior versions of Shadow) on its users with no means of changing it after compile/install time.

I'd much rather us change the policy back to how things were as that will lead to the least surprise amongst users. Once upstream provide a means of changing/overriding policy via the command line and/or config files, we can document that in the book.

Regards,

Matt.

[steve.crosby@…]

Now I'm confused - the behaviour of useradd from shadow 4.1.0 is the same as previous versions - if you do not specify a group on the command line using -g, then a group with the same name as the user is created by default. The "command line" override for this behaviour is to manually specify a group to use.

OR maybe I'm not understanding the issue....

tests from lfs6.3-20070731 rc2 show what I expected to see, and what the new version does as well. Note that GID 1000 (admin) was already created previously, and portions of /etc/passwd and /etc/group have been redacted.

-bash-3.2# cat /etc/lfs-release 6.3-20070731 - jhalfs build

-bash-3.2# cat /etc/default/useradd # useradd defaults file HOME=/home INACTIVE=-1 EXPIRE= SHELL=/bin/bash SKEL=/etc/skel CREATE_MAIL_SPOOL=no

-bash-3.2# useradd fubar -bash-3.2# useradd barfu -g admin

-bash-3.2# cat /etc/passwd fubar:x:1001:1001::/home/fubar:/bin/bash barfu:x:1002:1000::/home/barfu:/bin/bash

-bash-3.2# cat /etc/group admin:x:1000: fubar:x:1001:

So, fubar gets a new group, also called fubar barfu gets the main system group of admin, but no private group.

Is this not the behaviour of shadow 4.1.0? (the "group 1000 not found is a seperate issue)

[Julio Meca Hansen]

I always put the following line in /etc/groups:

users:x:1000:

and I always use shadow-4.1.0 (I know... I always tend to deviate from the book, but I want to test the latest packages to see if they actually work without any modification needed (from one hand) and to have the latest versions in my system (from the other hand)) without any problem, but I have to note everytime I need to create an user (and I only create my user for regular system operation and no more), I use the -g flag, so my command ends being this:

useradd -g users julio

As it works very well (at least from my personal experience), I think we should close this ticket, and delete the other ticket about shadow-4.0.18.2, given the fact version 4.1.0 solves some of the problems arisen in the 4.0.18 series

Julio

[Julio Meca Hansen]

Shadow-4.1.2 needs different instructions for installing, but frankly speaking, I don't know what's going on with this package, as it doesn't seem to be installing any of the shadow libraries, no trace of the libshadow.{a,la,so} in /lib or /usr/lib, instead of that, the libraries can be found inside shadow-4.1.2 source directory, maybe it has new installation instructions, but a plain 'make install' doesn't do the job, so I'll be browsing the web for a solution on this, and I'll report in case of finding a suitable solution, for now... I'm stuck with this

Any ideas?

Julio

[Julio Meca Hansen]

I've been having a talk in the #shadow irc channel at freenode, and the answer I've got have left me quite surprised... according to their response, the libraries installation has been suppressed recently, but for your reading pleasure, I'll paste the channel log, it's quite worth of reading :P

<LydianKnight> good afternoon, could I make a question about the latest shadow-4.1.2 package?
<LydianKnight> I compile shadow-4.1.2 with these parameters: ./configure --libdir=/lib --sysconfdir=/etc --enable-shared --without-selinux
<LydianKnight> and the compilation goes fine
<LydianKnight> but
<LydianKnight> at the time of performing 'make install'
<LydianKnight> it only installs the man pages
<LydianKnight> and the binaries
<LydianKnight> but the libraries doesn't get installed but residing in the source directory
<LydianKnight> so no libshadow.{a,la,so}
<LydianKnight> is this a bug or maybe I should issue an extra install command to place the libraries in their proper locations?
<LydianKnight> anyone could give me a hint on this?
<nekral> LydianKnight: Are the libraries really needed?
<nekral> They should not be necessary for the shadow utils. And I don't know any other utility which depends on these libraries.
<LydianKnight> I use LFS as my base system, and the installation instructions install the libraries in the standard /lib and /usr/lib locations
<LydianKnight> seemed to me a bit strange
<LydianKnight> for the libraries not be installed
<LydianKnight> hence my question about a change in the installation instructions
<LydianKnight> and
<LydianKnight> their late relation with the installation of Linux-PAM
<nekral> LydianKnight: I think you can ignore the installation of the libraries. They used to be installed, but this was disabled recently. It might be that the LFS was not updated since then.
<LydianKnight> as I'm collaborating in the book edition, I'll send a note to the main editors about this, thank you very much :)
<LydianKnight> oh, before disconnecting...
<LydianKnight> is this behaviour
<LydianKnight> compatible with the installation of the latest version of Linux-PAM? I mean... is it aware of this changes?
<nekral> The parsing of the /etc/passwd, /etc/group, /etc/shadow files is now dones by the glibc. Linux-PAM does not need the shadow libraries.
<nekral> s/dones/done/
<LydianKnight> oh, I see
<LydianKnight> thank you very much for your help :)

Comments? Ideas? Conclusions?

Julio

[bdubbs@…]

I haven't looked at it lately, but a spot check of some shadow utilities like passwd only use libcrypt, libc, and ld-linux.

This would indicate that libshadow isn't needed. Looking at a RHEL system, I can't find libshadow at all, so that would confirm the previous comment.

It looks like we just need to remove the stuff about moving the libraries and remove the library references in the contents section.

[dnicholson@…]

Exactly. Only the shadow utilities use libshadow. The only advantage of installing libshadow.so was to cut down on the size of the utilities. No functionality is lost if libshadow becomes an internal static library. It also means they don't have to stabilize the API.

[Julio Meca Hansen]

So we can remove the library part and convert the configure parameters from:

./configure --libdir=/lib --sysconfdir=/etc --enable-shared --without-selinux

To this?

./configure --sysconfdir=/etc --without-selinux

I mean... there's no use in passing the --libdir parameter if we're not gonna deploy any libraries at all... still surprising for me, but I'll try as I still have the final chapter pending, so I can continue reporting any modifications needed, if any

Julio

[bdubbs@…]

Replying to LydianKnight:

So we can remove the library part and convert the configure parameters

That looks right. If you test it out and everything looks OK, please post the results. We can go from there.

[Julio Meca Hansen]

For the configure part, even better, taking a look at the --help switch, it seems it only accepts adding selinux support if it's actually present, so less parameters, a simple:

./configure --sysconfdir=/etc

Will do the trick. Unfortunately, even having a very easy way to install this new version of shadow, this needs further investigation, as it fails at the time of creating the user home directory. Here is the console output for the commands I ran in order to test if shadow installation plays nice, and... it doesn't.

root:/sources/shadow-4.1.2# groupadd -g 1000 users
root:/sources/shadow-4.1.2# useradd -g users lydianknight
root:/sources/shadow-4.1.2# passwd lydianknight
Changing password for lydianknight
Enter the new password (minimum of 5 characters)
Please use a combination of upper and lower case letters and numbers.
New password:
Re-enter new password:
Password changed.
root:/sources/shadow-4.1.2# su - lydianknight
No directory, logging in with HOME=/
-su-3.2$ ls -la /home/
total 8
drwxr-xr-x  2 root root 4096 Jun  6 18:41 .
drwxr-xr-x 22 root root 4096 Jun  6 18:41 ..
-su-3.2$ exit
logout

I don't know if it's a bug or I had to sed or modify any source files prior compiling the source, took a look at the login.defs file but didn't found anything specially interesting, seems we still have some work to do in order to get shadow-4.1.2 in the book

Julio

[bdubbs@…]

Use -m to create the home directory. It should be

useradd -m -g users lydianknight

[Julio Meca Hansen]

Idiot me, you're right :)

I tested the user and group creation, deletion, and adding groups for the test user, and it all works fine, so the final instructions for compiling and installing shadow-4.1.2 are:

./configure --sysconfdir=/etc

sed -i 's/groups$(EXEEXT) //' src/Makefile
find man -name Makefile -exec sed -i 's/groups\.1 / /' {} \;

sed -i -e 's/ ko//' -e 's/ zh_CN zh_TW//' man/Makefile

for i in de es fi fr id it pt_BR; do
    convert-mans UTF-8 ISO-8859-1 man/${i}/*.?
done

for i in cs hu pl; do
    convert-mans UTF-8 ISO-8859-2 man/${i}/*.?
done

convert-mans UTF-8 EUC-JP man/ja/*.?
convert-mans UTF-8 KOI8-R man/ru/*.?
convert-mans UTF-8 ISO-8859-9 man/tr/*.?

sed -i -e 's@#MD5_CRYPT_ENAB.no@MD5_CRYPT_ENAB yes@' \
    -e 's@/var/spool/mail@/var/mail@' etc/login.defs

make

make install

mv -v /usr/bin/passwd /bin

pwconv

grpconv

useradd -D -b /home
sed -i 's/yes/no/' /etc/default/useradd

passwd root

Plus deleting any reference to library instructions and notes in the shadow page, although the grpconv command complains about the non-existence of groud id 1000, my suggestion would be to add this:

groupadd -g 1000 users

Before the grpconv command, or after all the instructions (the command doesn't seem to have secondary effects whether I run it before or after the grpconv command)

Except that debated subject, shadow-4.1.2 compiled and tested successfully.

Julio

[randy@…]

William,

Please follow etiquette in this community by educating yourself with the mailing lists. If you would bother to actually read the lists, you'd see that DJ rendered a version of the book that we are using as a guideline for the updates that are going on.

And DJ's book uses the 4.1.2.1 version.

William - Read more and talk less.

[randy@…]

Updated LFS to Shadow-4.1.2.1

Many, many changes. As there was no consensus about what to do with useradd, I did what I thought was best. I'll be hitting -dev with a message about this.

Anyway, I like the changes. I've moved all the config stuff to before the first configure command so that it appears to be a CMMI package.

I wrote quite a bit of detail about the useradd stuff. If there are comments or problems, please open a new ticket as this one is so long already.