Constructing a temporary system
Introduction
In this chapter we will compile and install a minimal
Linux system. This system will contain just enough tools to be able
to start constructing the final LFS system in the next chapter.
The building of this minimal system is done in two steps: first we
build a brand-new and host-independent toolchain (compiler, assembler,
linker and libraries), and then use this to build all the other essential
tools.
The files compiled in this chapter will be installed under the
$LFS/tools directory
to keep them separate from the files installed in the next chapter.
Since the packages compiled here are merely temporary, we don't want
them to pollute the soon-to-be LFS system.
Before issuing the build instructions for a package you are expected to
have already unpacked it as user lfs (explained shortly),
and to have performed a cd into the created directory.
The build instructions assume that you are using the bash
shell.
Several of the packages are patched before compilation, but only when
the patch is needed to circumvent a problem. Often the patch is needed in
both this and the next chapter, but sometimes in only one of them. Therefore,
don't worry when instructions for a downloaded patch seem to be missing. Also,
when applying a patch, you'll occasionally see warning messages about
offset or fuzz. These warnings are
nothing to worry about, as the patch was still successfully applied.
During the compilation of most packages you will see many warnings
scroll by on your screen. These are normal and can safely be ignored. They are
just what they say they are: warnings -- mostly about deprecated, but not
invalid, use of the C or C++ syntax. It's just that C standards have changed
rather often and some packages still use the older standard, which is not
really a problem.
After installing each package you should delete its source and build
directories, unless told otherwise. Deleting the sources
saves space, but also prevents misconfiguration when the same package is
reinstalled further on. Only for three packages you will need to keep the
source and build directories around for a while, so their contents can be used
by later commands. Do not miss the reminders.
Now first check that your LFS environment variable is set up
properly:
echo $LFS
Make sure the output shows the path to your LFS partition's mount
point, which is /mnt/lfs if you
followed our example.
Toolchain technical notes
This section attempts to explain some of the rationale and technical
details behind the overall build method. It's not essential that you understand
everything here immediately. Most of it will make sense once you have performed
an actual build. Feel free to refer back here at any time.
The overall goal of is to provide a sane,
temporary environment that we can chroot into, and from which we can produce a
clean, trouble-free build of the target LFS system in
. Along the way, we attempt to divorce ourselves
from the host system as much as possible, and in so doing build a
self-contained and self-hosted toolchain. It should be noted that the
build process has been designed in such a way so as to minimize the risks for
new readers and provide maximum educational value at the same time. In other
words, more advanced techniques could be used to build the system.
Before continuing, you really should be aware of the name of your working
platform, often also referred to as the target triplet. For
many folks the target triplet will probably be
i686-pc-linux-gnu. A simple way to determine your target
triplet is to run the config.guess script that comes with
the source for many packages. Unpack the Binutils sources and run the script:
./config.guess and note the output.
You'll also need to be aware of the name of your platform's
dynamic linker, often also referred to as the
dynamic loader, not to be confused with the standard linker
ld that is part of Binutils. The dynamic linker is provided
by Glibc and has the job of finding and loading the shared libraries needed by a
program, preparing the program to run and then running it. For most folks the
name of the dynamic linker will be ld-linux.so.2. On
platforms that are less prevalent, the name might be
ld.so.1 and newer 64 bit platforms might even have
something completely different. You should be able to determine the name
of your platform's dynamic linker by looking in the
/lib directory on your host system. A
surefire way is to inspect a random binary from your host system by running:
readelf -l <name of binary> | grep interpreter
and noting the output. The authoritative reference covering all platforms is in
the shlib-versions file in the root of the Glibc source
tree.
Some key technical points of how the build
method works:
Similar in principle to cross compiling whereby tools installed
into the same prefix work in cooperation and thus utilize a little GNU
"magic".
Careful manipulation of the standard linker's library search
path to ensure programs are linked only against libraries we
choose.
Careful manipulation of gcc's
specs file to tell the compiler which target dynamic
linker will be used.
Binutils is installed first because both GCC and Glibc perform various
feature tests on the assembler and linker during their respective runs of
./configure to determine which software features to enable
or disable. This is more important than one might first realize. An incorrectly
configured GCC or Glibc can result in a subtly broken toolchain where the impact
of such breakage might not show up until near the end of the build of a whole
distribution. Thankfully, a test suite failure will usually alert us before too
much time is wasted.
Binutils installs its assembler and linker into two locations,
/tools/bin and
/tools/$TARGET_TRIPLET/bin. In reality,
the tools in one location are hard linked to the other. An important facet of
the linker is its library search order. Detailed information can be obtained
from ld by passing it the --verbose
flag. For example: ld --verbose | grep SEARCH will
show you the current search paths and their order. You can see what files are
actually linked by ld by compiling a dummy program and
passing the --verbose switch to the linker. For example:
gcc dummy.c -Wl,--verbose 2>&1 | grep succeeded
will show you all the files successfully opened during the linking.
The next package installed is GCC and during its run of
./configure you'll see, for example:
checking what assembler to use... /tools/i686-pc-linux-gnu/bin/as
checking what linker to use... /tools/i686-pc-linux-gnu/bin/ld
This is important for the reasons mentioned above. It also demonstrates
that GCC's configure script does not search the PATH directories to find which
tools to use. However, during the actual operation of gcc
itself, the same search paths are not necessarily used. You can find out which
standard linker gcc will use by running:
gcc -print-prog-name=ld.
Detailed information can be obtained from gcc by passing
it the -v flag while compiling a dummy program. For
example: gcc -v dummy.c will show you detailed
information about the preprocessor, compilation and assembly stages, including
gcc's include search paths and their order.
The next package installed is Glibc. The most important considerations for
building Glibc are the compiler, binary tools and kernel headers. The compiler
is generally no problem as Glibc will always use the gcc
found in a PATH directory. The binary tools and kernel headers can be a little
more troublesome. Therefore we take no risks and use the available configure
switches to enforce the correct selections. After the run of
./configure you can check the contents of the
config.make file in the
glibc-build directory for all the
important details. You'll note some interesting items like the use of
CC="gcc -B/tools/bin/" to control which binary tools are
used, and also the use of the -nostdinc and
-isystem flags to control the compiler's include search
path. These items help to highlight an important aspect of the Glibc package:
it is very self-sufficient in terms of its build machinery and generally does
not rely on toolchain defaults.
After the Glibc installation, we make some adjustments to ensure that
searching and linking take place only within our /tools
prefix. We install an adjusted ld, which has a hard-wired
search path limited to /tools/lib. Then
we amend gcc's specs file to point to our new dynamic
linker in /tools/lib. This last step is
vital to the whole process. As mentioned above, a
hard-wired path to a dynamic linker is embedded into every ELF shared
executable. You can inspect this by running:
readelf -l <name of binary> | grep interpreter.
By amending gcc's specs file, we are ensuring that every
program compiled from here through the end of this chapter will use our new
dynamic linker in /tools/lib.
The need to use the new dynamic linker is also the reason why we apply the
Specs patch for the second pass of GCC. Failure to do so will result in the GCC
programs themselves having the name of the dynamic linker from the host system's
/lib directory embedded into them, which
would defeat our goal of getting away from the host.
During the second pass of Binutils, we are able to utilize the
--with-lib-path configure switch to control
ld's library search path. From this point onwards, the
core toolchain is self-contained and self-hosted. The remainder of the
packages all build against the new Glibc in
/tools and all is well.
Upon entering the chroot environment in , the
first major package we install is Glibc, due to its self-sufficient nature that
we mentioned above. Once this Glibc is installed into
/usr, we perform a quick changeover of
the toolchain defaults, then proceed for real in building the rest of the
target LFS system.
Notes on static linking
Most programs have to perform, beside their specific task, many rather
common and sometimes trivial operations. These include allocating memory,
searching directories, reading and writing files, string handling, pattern
matching, arithmetic and many other tasks. Instead of obliging each program to
reinvent the wheel, the GNU system provides all these basic functions in
ready-made libraries. The major library on any Linux system is
Glibc.
There are two primary ways of linking the functions from a library to a
program that uses them: statically or dynamically. When a program is linked
statically, the code of the used functions is included in the executable,
resulting in a rather bulky program. When a program is dynamically linked, what
is included is a reference to the dynamic linker, the name of the library, and
the name of the function, resulting in a much smaller executable. (A third way
is to use the programming interface of the dynamic linker. See the
dlopen man page for more information.)
Dynamic linking is the default on Linux and has three major advantages
over static linking. First, you need only one copy of the executable library
code on your hard disk, instead of having many copies of the same code included
into a whole bunch of programs -- thus saving disk space. Second, when several
programs use the same library function at the same time, only one copy of the
function's code is required in core -- thus saving memory space. Third, when a
library function gets a bug fixed or is otherwise improved, you only need to
recompile this one library, instead of having to recompile all the programs that
make use of the improved function.
If dynamic linking has several advantages, why then do we statically link
the first two packages in this chapter? The reasons are threefold: historical,
educational, and technical. Historical, because earlier versions of LFS
statically linked every program in this chapter. Educational, because knowing
the difference is useful. Technical, because we gain an element of independence
from the host in doing so, meaning that those programs can be used
independently of the host system. However, it's worth noting that an overall
successful LFS build can still be achieved when the first two packages are
built dynamically.
Creating the $LFS/tools directory
All programs compiled in this chapter will be installed under $LFS/tools to keep them separate from the
programs compiled in the next chapter. The programs compiled here are only
temporary tools and won't be a part of the final LFS system and by keeping them
in a separate directory, we can later easily throw them away.
Later on you might wish to search through the binaries of your system to
see what files they make use of or link against. To make this searching easier
you may want to choose a unique name for the directory in which the temporary
tools are stored. Instead of the simple "tools" you could use something like
"tools-for-lfs". However, you'll need to be careful to adjust all references to
"tools" throughout the book -- including those in any patches, notably the
GCC Specs Patch.
Create the required directory by running the following:
mkdir $LFS/tools
The next step is to create a /tools symlink on
your host system. It will point to the directory we just created on the LFS
partition:
ln -s $LFS/tools /
The above command is correct. The ln command
has a few syntactic variations, so be sure to check the info page before
reporting what you may think is an error.
The created symlink enables us to compile our toolchain so that it always
refers to /tools, meaning that the compiler, assembler
and linker will work both in this chapter (when we are still using some tools
from the host) and in the next (when we are chrooted to
the LFS partition).
Adding the user lfs
When logged in as root, making a single mistake
can damage or even wreck your system. Therefore we recommend that you
build the packages in this chapter as an unprivileged user. You could
of course use your own user name, but to make it easier to set up a clean
work environment we'll create a new user lfs and
use this one during the installation process. As root,
issue the following command to add the new user:
useradd -s /bin/bash -m -k /dev/null lfs
The meaning of the switches:
-s /bin/bash: This makes
bash the default shell for user
lfs.
-m -k /dev/null: These create a home
directory for lfs, while preventing the files from a
possible /etc/skel being copied into it.
If you want to be able to log in as lfs, then give
this new user a password:
passwd lfs
Now grant this new user lfs full access to
$LFS/tools by giving it ownership
of the directory:
chown lfs $LFS/tools
If you made a separate working directory as suggested, give user
lfs ownership of this directory too:
chown lfs $LFS/sources
Next, login as user lfs. This can be done via a
virtual console, through a display manager, or with the following substitute
user command:
su - lfs
The "-" instructs su to start a
login shell.
Setting up the environment
We're going to set up a good working environment by creating two new
startup files for the bash shell. While logged in as
user lfs, issue the following command to create a new
.bash_profile:
cat > ~/.bash_profile << "EOF"
exec env -i HOME=$HOME TERM=$TERM PS1='\u:\w\$ ' /bin/bash
EOF
Normally, when you log on as user lfs,
the initial shell is a login shell which reads the
/etc/profile of your host (probably containing some
settings of environment variables) and then .bash_profile.
The exec env -i ... /bin/bash command in the latter file
replaces the running shell with a new one with a completely empty environment,
except for the HOME, TERM and PS1 variables. This ensures that no unwanted and
potentially hazardous environment variables from the host system leak into our
build environment. The technique used here is a little strange, but it achieves
the goal of enforcing a clean environment.
The new instance of the shell is a non-login shell,
which doesn't read the /etc/profile or
.bash_profile files, but reads the
.bashrc file instead. Create this latter file now:
cat > ~/.bashrc << "EOF"
set +h
umask 022
LFS=/mnt/lfs
LC_ALL=POSIX
PATH=/tools/bin:/bin:/usr/bin
export LFS LC_ALL PATH
EOF
The set +h command turns off
bash's hash function. Normally hashing is a useful
feature: bash uses a hash table to remember the
full pathnames of executable files to avoid searching the PATH time and time
again to find the same executable. However, we'd like the new tools to be
used as soon as they are installed. By switching off the hash function, our
"interactive" commands (make,
patch, sed,
cp and so forth) will always use
the newest available version during the build process.
Setting the user file-creation mask to 022 ensures that newly created
files and directories are only writable for their owner, but readable and
executable for anyone.
The LFS variable should of course be set to the mount point you
chose.
The LC_ALL variable controls the localization of certain programs,
making their messages follow the conventions of a specified country. If your
host system uses a version of Glibc older than 2.2.4,
having LC_ALL set to something other than "POSIX" or "C" during this chapter
may cause trouble if you exit the chroot environment and wish to return later.
By setting LC_ALL to "POSIX" (or "C", the two are equivalent) we ensure that
everything will work as expected in the chroot environment.
We prepend /tools/bin to the standard PATH so
that, as we move along through this chapter, the tools we build will get used
during the rest of the building process.
Finally, to have our environment fully prepared for building the
temporary tools, source the just-created profile:
source ~/.bash_profile
&c5-binutils-pass1;
&c5-gcc-pass1;
&c5-kernelheaders;
&c5-glibc;
Adjusting the toolchain
Now that the temporary C libraries have been installed, we want all
the tools compiled in the rest of this chapter to be linked against these
libraries. To accomplish this, we need to adjust the linker and the compiler's
specs file. Some people would say that it is "black magic juju below
this line", but it is really very simple.
First install the adjusted linker (adjusted at the end of the first pass
of Binutils) by running the following command from within
the binutils-build directory:
make -C ld install
From this point onwards everything will link only
against the libraries in /tools/lib.
If you somehow missed the earlier warning to retain the Binutils
source and build directories from the first pass or otherwise accidentally
deleted them or just don't have access to them, don't worry, all is not lost.
Just ignore the above command. The result is a small chance of the subsequent
testing programs linking against libraries on the host. This is not ideal, but
it's not a major problem. The situation is corrected when we install the
second pass of Binutils a bit further on.
Now that the adjusted linker is installed, you have to
remove the Binutils build and source directories.
The next thing to do is to amend our GCC specs file so that it points
to the new dynamic linker. A simple sed will accomplish this:
SPECFILE=/tools/lib/gcc-lib/*/*/specs &&
sed -e 's@ /lib/ld-linux.so.2@ /tools/lib/ld-linux.so.2@g' \
$SPECFILE > tempspecfile &&
mv -f tempspecfile $SPECFILE &&
unset SPECFILE
We recommend that you cut-and-paste the above rather than try and type it
all in. Or you can edit the specs file by hand if you want to: just replace the
occurrence of "/lib/ld-linux.so.2" with "/tools/lib/ld-linux.so.2". Be sure to
visually inspect the specs file to verify the intended change was actually
made.
If you are working on a platform where the name of the dynamic
linker is something other than ld-linux.so.2, you
must substitute ld-linux.so.2 with the
name of your platform's dynamic linker in the above commands. Refer back to
if necessary.
Lastly, there is a possibility that some include files from the host
system have found their way into GCC's private include dir. This can happen
because of GCC's "fixincludes" process which runs as part of the GCC build.
We'll explain more about this further on in this chapter. For now, run the
following commands to eliminate this possibility:
rm -f /tools/lib/gcc-lib/*/*/include/{pthread.h,bits/sigthread.h}
It is imperative at this point to stop and ensure that the basic
functions (compiling and linking) of the new toolchain are working as expected.
For this we are going to perform a simple sanity check:
echo 'main(){}' > dummy.c
cc dummy.c
readelf -l a.out | grep ': /tools'
If everything is working correctly, there should be no errors, and the
output of the last command will be:
[Requesting program interpreter: /tools/lib/ld-linux.so.2]
(Of course allowing for platform specific differences in dynamic linker
name). Note especially that /tools/lib
appears as the prefix of our dynamic linker. If you did not receive the output
as shown above, or received no output at all, then something is seriously wrong.
You will need to investigate and retrace your steps to find out where the
problem is and correct it. There is no point in continuing until this is done.
First, redo the sanity check using gcc instead of
cc. If this works it means the
/tools/bin/cc symlink is missing. Revisit
and fix the symlink. Second, ensure your PATH
is correct. You can check this by running echo $PATH and
verifying that /tools/bin is at the head
of the list. If the PATH is wrong it could mean you're not logged in as user
lfs or something went wrong back in
. Third, something may have gone wrong with
the specs file amendment above. In this case redo the specs file amendment
ensuring to cut-and-paste the commands as was recommended.
Once you are satisfied that all is well, clean up the test files:
rm dummy.c a.out
&c5-tcl;
&c5-expect;
&c5-dejagnu;
&c5-gcc-pass2;
&c5-binutils-pass2;
&c5-gawk;
&c5-coreutils;
&c5-bzip2;
&c5-gzip;
&c5-diffutils;
&c5-findutils;
&c5-make;
&c5-grep;
&c5-sed;
&c5-gettext;
&c5-ncurses;
&c5-patch;
&c5-tar;
&c5-texinfo;
&c5-bash;
&c5-perl;
Stripping
The steps in this section are optional. If your LFS partition is rather
small, you will be glad to learn that you can throw away some unnecessary
things. The executables and libraries you have built so far contain about 130 MB
of unneeded debugging symbols. Remove those symbols like this:
strip --strip-unneeded /tools/{,s}bin/*
strip --strip-debug /tools/lib/*
The first of the above commands will skip some twenty files, reporting
that it doesn't recognize their file format. Most of them are scripts instead
of binaries.
Take care not to use
--strip-unneeded on the libraries -- the static ones
would be destroyed and you would have to build the three toolchain packages
all over again.
To save another couple of megabytes, you can throw away all the
documentation:
rm -rf /tools/{,share/}{doc,info,man}
You will now need to have at least 850 MB of free space on your LFS
file system to be able to build and install Glibc in the next phase. If you can
build and install Glibc, you can build and install the rest too.