Ignore:
Timestamp:
01/22/2006 01:13:52 PM (18 years ago)
Author:
Manuel Canales Esparcia <manuel@…>
Branches:
10.0, 10.0-rc1, 10.1, 10.1-rc1, 11.0, 11.0-rc1, 11.0-rc2, 11.0-rc3, 11.1, 11.1-rc1, 11.2, 11.2-rc1, 11.3, 11.3-rc1, 12.0, 12.0-rc1, 12.1, 12.1-rc1, 6.3, 6.4, 6.5, 6.6, 6.7, 6.8, 7.0, 7.1, 7.2, 7.3, 7.4, 7.5, 7.5-systemd, 7.6, 7.6-systemd, 7.7, 7.7-systemd, 7.8, 7.8-systemd, 7.9, 7.9-systemd, 8.0, 8.1, 8.2, 8.3, 8.4, 9.0, 9.1, arm, bdubbs/gcc13, ml-11.0, multilib, renodr/libudev-from-systemd, s6-init, trunk, xry111/arm64, xry111/arm64-12.0, xry111/clfs-ng, xry111/lfs-next, xry111/loongarch, xry111/loongarch-12.0, xry111/loongarch-12.1, xry111/mips64el, xry111/pip3, xry111/rust-wip-20221008, xry111/update-glibc
Children:
41ddb3b
Parents:
7b82580
Message:

Finished chapter 05 indentation.

git-svn-id: ​http://svn.linuxfromscratch.org/LFS/trunk/BOOK@7292 4aa44e1e-78dd-0310-a6d2-fbcd4c07a689

File:
1 edited

Legend:

Unmodified
Added
Removed

  • chapter05/toolchaintechnotes.xml

    r7b82580 rb28fd35  
    11<?xml version="1.0" encoding="ISO-8859-1"?>
    2 <!DOCTYPE sect1 PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN" "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd" [
     2<!DOCTYPE sect1 PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
     3  "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd" [
    34  <!ENTITY % general-entities SYSTEM "../general.ent">
    45  %general-entities;
    56]>
     7
    68<sect1 id="ch-tools-toolchaintechnotes">
    7 <title>Toolchain Technical Notes</title>
    8 <?dbhtml filename="toolchaintechnotes.html"?>
    9 
    10 <para>This section explains some of the rationale and technical
    11 details behind the overall build method. It is not essential to
    12 immediately understand everything in this section. Most of this
    13 information will be clearer after performing an actual build. This
    14 section can be referred back to at any time during the process.</para>
    15 
    16 <para>The overall goal of <xref linkend="chapter-temporary-tools"/> is to
    17 provide a temporary environment that can be chrooted into and from which can be
    18 produced a clean, trouble-free build of the target LFS system in <xref
    19 linkend="chapter-building-system"/>. Along the way, we separate the new system
    20 from the host system as much as possible, and in doing so, build a
    21 self-contained and self-hosted toolchain. It should be noted that the build
    22 process has been designed to minimize the risks for new readers and provide
    23 maximum educational value at the same time.</para>
    24 
    25 <important>
    26 <para>Before continuing, be aware of the name of the working platform,
    27 often referred to as the target triplet. Many times, the target
    28 triplet will probably be <emphasis>i686-pc-linux-gnu</emphasis>. A
    29 simple way to determine the name of the target triplet is to run the
    30 <command>config.guess</command> script that comes with the source for
    31 many packages. Unpack the Binutils sources and run the script:
    32 <userinput>./config.guess</userinput> and note the output.</para>
    33 
    34 <para>Also be aware of the name of the platform's dynamic linker,
    35 often referred to as the dynamic loader (not to be confused with the
    36 standard linker <command>ld</command> that is part of Binutils). The
    37 dynamic linker provided by Glibc finds and loads the shared libraries
    38 needed by a program, prepares the program to run, and then runs it.
    39 The name of the dynamic linker will usually be
    40 <filename class="libraryfile">ld-linux.so.2</filename>. On platforms that are less
    41 prevalent, the name might be <filename class="libraryfile">ld.so.1</filename>,
    42 and newer 64 bit platforms might be named something else entirely. The name of
    43 the platform's dynamic linker can be determined by looking in the
    44 <filename class="directory">/lib</filename> directory on the host
    45 system. A sure-fire way to determine the name is to inspect a random
    46 binary from the host system by running: <userinput>readelf -l &lt;name
    47 of binary&gt; | grep interpreter</userinput> and noting the output.
    48 The authoritative reference covering all platforms is in the
    49 <filename>shlib-versions</filename> file in the root of the Glibc
    50 source tree.</para>
    51 </important>
    52 
    53 <para>Some key technical points of how the <xref linkend="chapter-temporary-tools"/> build
    54 method works:</para>
    55 
    56 <itemizedlist>
    57 <listitem><para>The process is similar in principle to
    58 cross-compiling, whereby tools installed in the same prefix work in
    59 cooperation, and thus utilize a little GNU
    60 <quote>magic</quote></para></listitem>
    61 
    62 <listitem><para>Careful manipulation of the standard linker's library
    63 search path ensures programs are linked only against chosen
    64 libraries</para></listitem>
    65 
    66 <listitem><para>Careful manipulation of <command>gcc</command>'s
    67 <filename>specs</filename> file tells the compiler which target dynamic
    68 linker will be used</para></listitem>
    69 </itemizedlist>
    70 
    71 <para>Binutils is installed first because the
    72 <command>configure</command> runs of both GCC and Glibc perform
    73 various feature tests on the assembler and linker to determine which
    74 software features to enable or disable. This is more important than
    75 one might first realize. An incorrectly configured GCC or Glibc can
    76 result in a subtly broken toolchain, where the impact of such breakage
    77 might not show up until near the end of the build of an entire
    78 distribution. A test suite failure will usually highlight this error
    79 before too much additional work is performed.</para>
    80 
    81 <para>Binutils installs its assembler and linker in two locations,
    82 <filename class="directory">/tools/bin</filename> and <filename
    83 class="directory">/tools/$TARGET_TRIPLET/bin</filename>. The tools in
    84 one location are hard linked to the other. An important facet of the
    85 linker is its library search order. Detailed information can be
    86 obtained from <command>ld</command> by passing it the
    87 <parameter>--verbose</parameter> flag. For example, an <userinput>ld
    88 --verbose | grep SEARCH</userinput> will illustrate the current search
    89 paths and their order. It shows which files are linked by
    90 <command>ld</command> by compiling a dummy program and passing the
    91 <parameter>--verbose</parameter> switch to the linker. For example,
    92 <userinput>gcc dummy.c -Wl,--verbose 2&gt;&amp;1 | grep
    93 succeeded</userinput> will show all the files successfully opened
    94 during the linking.</para>
    95 
    96 <para>The next package installed is GCC. An example of what can be
    97 seen during its run of <command>configure</command> is:</para>
    98 
    99 <screen><computeroutput>checking what assembler to use...
     9  <?dbhtml filename="toolchaintechnotes.html"?>
     10
     11  <title>Toolchain Technical Notes</title>
     12
     13  <para>This section explains some of the rationale and technical details
     14  behind the overall build method. It is not essential to immediately
     15  understand everything in this section. Most of this information will be
     16  clearer after performing an actual build. This section can be referred
     17  back to at any time during the process.</para>
     18
     19  <para>The overall goal of <xref linkend="chapter-temporary-tools"/> is to
     20  provide a temporary environment that can be chrooted into and from which can be
     21  produced a clean, trouble-free build of the target LFS system in <xref
     22  linkend="chapter-building-system"/>. Along the way, we separate the new system
     23  from the host system as much as possible, and in doing so, build a
     24  self-contained and self-hosted toolchain. It should be noted that the build
     25  process has been designed to minimize the risks for new readers and provide
     26  maximum educational value at the same time.</para>
     27
     28  <important>
     29    <para>Before continuing, be aware of the name of the working platform,
     30    often referred to as the target triplet. Many times, the target
     31    triplet will probably be <emphasis>i686-pc-linux-gnu</emphasis>. A
     32    simple way to determine the name of the target triplet is to run the
     33    <command>config.guess</command> script that comes with the source for
     34    many packages. Unpack the Binutils sources and run the script:
     35    <userinput>./config.guess</userinput> and note the output.</para>
     36
     37    <para>Also be aware of the name of the platform's dynamic linker, often
     38    referred to as the dynamic loader (not to be confused with the standard
     39    linker <command>ld</command> that is part of Binutils). The dynamic linker
     40    provided by Glibc finds and loads the shared libraries needed by a program,
     41    prepares the program to run, and then runs it. The name of the dynamic
     42    linker will usually be <filename class="libraryfile">ld-linux.so.2</filename>.
     43    On platforms that are less prevalent, the name might be <filename
     44    class="libraryfile">ld.so.1</filename>, and newer 64 bit platforms might
     45    be named something else entirely. The name of the platform's dynamic linker
     46    can be determined by looking in the <filename class="directory">/lib</filename>
     47    directory on the host system. A sure-fire way to determine the name is to
     48    inspect a random binary from the host system by running:
     49    <userinput>readelf -l &lt;name of binary&gt; | grep interpreter</userinput>
     50    and noting the output. The authoritative reference covering all platforms
     51    is in the <filename>shlib-versions</filename> file in the root of the Glibc
     52    source tree.</para>
     53  </important>
     54
     55  <para>Some key technical points of how the <xref
     56  linkend="chapter-temporary-tools"/> build method works:</para>
     57
     58  <itemizedlist>
     59    <listitem>
     60      <para>The process is similar in principle to cross-compiling, whereby
     61      tools installed in the same prefix work in cooperation, and thus utilize
     62      a little GNU <quote>magic</quote></para>
     63    </listitem>
     64    <listitem>
     65      <para>Careful manipulation of the standard linker's library search path
     66      ensures programs are linked only against chosen libraries</para>
     67    </listitem>
     68    <listitem>
     69      <para>Careful manipulation of <command>gcc</command>'s
     70      <filename>specs</filename> file tells the compiler which target dynamic
     71      linker will be used</para>
     72    </listitem>
     73  </itemizedlist>
     74
     75  <para>Binutils is installed first because the <command>configure</command>
     76  runs of both GCC and Glibc perform various feature tests on the assembler
     77  and linker to determine which software features to enable or disable. This
     78  is more important than one might first realize. An incorrectly configured
     79  GCC or Glibc can result in a subtly broken toolchain, where the impact of
     80  such breakage might not show up until near the end of the build of an
     81  entire distribution. A test suite failure will usually highlight this error
     82  before too much additional work is performed.</para>
     83
     84  <para>Binutils installs its assembler and linker in two locations,
     85  <filename class="directory">/tools/bin</filename> and <filename
     86  class="directory">/tools/$TARGET_TRIPLET/bin</filename>. The tools in one
     87  location are hard linked to the other. An important facet of the linker is
     88  its library search order. Detailed information can be obtained from
     89  <command>ld</command> by passing it the <parameter>--verbose</parameter>
     90  flag. For example, an <userinput>ld --verbose | grep SEARCH</userinput>
     91  will illustrate the current search paths and their order. It shows which
     92  files are linked by <command>ld</command> by compiling a dummy program and
     93  passing the <parameter>--verbose</parameter> switch to the linker. For example,
     94  <userinput>gcc dummy.c -Wl,--verbose 2&gt;&amp;1 | grep succeeded</userinput>
     95  will show all the files successfully opened during the linking.</para>
     96
     97  <para>The next package installed is GCC. An example of what can be
     98  seen during its run of <command>configure</command> is:</para>
     99
     100<screen><computeroutput>checking what assembler to use...
    100101        /tools/i686-pc-linux-gnu/bin/as
    101102checking what linker to use... /tools/i686-pc-linux-gnu/bin/ld</computeroutput></screen>
    102103
    103 <para>This is important for the reasons mentioned above. It also
    104 demonstrates that GCC's configure script does not search the PATH
    105 directories to find which tools to use. However, during the actual
    106 operation of <command>gcc</command> itself, the same
    107 search paths are not necessarily used. To find out which standard
    108 linker <command>gcc</command> will use, run: <userinput>gcc
    109 -print-prog-name=ld</userinput>.</para>
    110 
    111 <para>Detailed information can be obtained from <command>gcc</command>
    112 by passing it the <parameter>-v</parameter> command line option while
    113 compiling a dummy program. For example, <userinput>gcc -v
    114 dummy.c</userinput> will show detailed information about the
    115 preprocessor, compilation, and assembly stages, including
    116 <command>gcc</command>'s included search paths and their order.</para>
    117 
    118 <para>The next package installed is Glibc. The most important
    119 considerations for building Glibc are the compiler, binary tools, and
    120 kernel headers. The compiler is generally not an issue since Glibc
    121 will always use the <command>gcc</command> found in a
    122 <envar>PATH</envar> directory.
    123 The binary tools and kernel headers can be a bit more complicated.
    124 Therefore, take no risks and use the available configure switches to
    125 enforce the correct selections. After the run of
    126 <command>configure</command>, check the contents of the
    127 <filename>config.make</filename> file in the <filename
    128 class="directory">glibc-build</filename> directory for all important
    129 details. Note the use of <parameter>CC="gcc -B/tools/bin/"</parameter>
    130 to control which binary tools are used and the use of the
    131 <parameter>-nostdinc</parameter> and <parameter>-isystem</parameter>
    132 flags to control the compiler's include search path. These items
    133 highlight an important aspect of the Glibc package&mdash;it is very
    134 self-sufficient in terms of its build machinery and generally does not
    135 rely on toolchain defaults.</para>
    136 
    137 <para>After the Glibc installation, make some adjustments to ensure
    138 that searching and linking take place only within the <filename
    139 class="directory">/tools</filename> prefix.  Install an adjusted
    140 <command>ld</command>, which has a hard-wired search path limited to
    141 <filename class="directory">/tools/lib</filename>. Then amend
    142 <command>gcc</command>'s specs file to point to the new dynamic linker
    143 in <filename class="directory">/tools/lib</filename>. This last step
    144 is vital to the whole process. As mentioned above, a hard-wired path
    145 to a dynamic linker is embedded into every Executable and Link Format
    146 (ELF)-shared executable.  This can be inspected by running:
    147 <userinput>readelf -l &lt;name of binary&gt; | grep
    148 interpreter</userinput>. Amending gcc's specs file
    149 ensures that every program compiled from here through the end of this
    150 chapter will use the new dynamic linker in <filename
    151 class="directory">/tools/lib</filename>.</para>
    152 
    153 <para>The need to use the new dynamic linker is also the reason why
    154 the Specs patch is applied for the second pass of GCC. Failure to do
    155 so will result in the GCC programs themselves having the name of the
    156 dynamic linker from the host system's <filename
    157 class="directory">/lib</filename> directory embedded into them, which
    158 would defeat the goal of getting away from the host.</para>
    159 
    160 <para>During the second pass of Binutils, we are able to utilize the
    161 <parameter>--with-lib-path</parameter> configure switch to control
    162 <command>ld</command>'s library search path.  From this point onwards,
    163 the core toolchain is self-contained and self-hosted. The remainder of
    164 the <xref linkend="chapter-temporary-tools"/> packages all build
    165 against the new Glibc in <filename
    166 class="directory">/tools</filename>.</para>
    167 
    168 <para>Upon entering the chroot environment in <xref
    169 linkend="chapter-building-system"/>, the first major package to be
    170 installed is Glibc, due to its self-sufficient nature mentioned above.
    171 Once this Glibc is installed into <filename
    172 class="directory">/usr</filename>, perform a quick changeover of the
    173 toolchain defaults, then proceed in building the rest of the target
    174 LFS system.</para>
    175 
    176 <!-- Removed as part of the fix for bug 1061 - we no longer build pass1
    177      packages statically, therefore this explanation isn't required -->
    178 
    179 <!--<sect2>
    180 <title>Notes on Static Linking</title>
    181 
    182 <para>Besides their specific task, most programs have to perform many
    183 common and sometimes trivial operations. These include allocating
    184 memory, searching directories, reading and writing files, string
    185 handling, pattern matching, arithmetic, and other tasks. Instead of
    186 obliging each program to reinvent the wheel, the GNU system provides
    187 all these basic functions in ready-made libraries. The major library
    188 on any Linux system is Glibc.</para>
    189 
    190 <para>There are two primary ways of linking the functions from a
    191 library to a program that uses them&mdash;statically or dynamically. When
    192 a program is linked statically, the code of the used functions is
    193 included in the executable, resulting in a rather bulky program. When
    194 a program is dynamically linked, it includes a reference to the
    195 dynamic linker, the name of the library, and the name of the function,
    196 resulting in a much smaller executable. A third option is to use the
    197 programming interface of the dynamic linker (see <filename>dlopen(3)</filename>
    198 for more information).</para>
    199 
    200 <para>Dynamic linking is the default on Linux and has three major
    201 advantages over static linking. First, only one copy of the executable
    202 library code is needed on the hard disk, instead of having multiple
    203 copies of the same code included in several programs, thus saving
    204 disk space. Second, when several programs use the same library
    205 function at the same time, only one copy of the function's code is
    206 required in core, thus saving memory space. Third, when a library
    207 function gets a bug fixed or is otherwise improved, only the one
    208 library needs to be recompiled instead of recompiling all programs
    209 that make use of the improved function.</para>
    210 
    211 <para>If dynamic linking has several advantages, why then do we
    212 statically link the first two packages in this chapter? The reasons
    213 are threefold&mdash;historical, educational, and technical. The
    214 historical reason is that earlier versions of LFS statically linked
    215 every program in this chapter. Educationally, knowing the difference
    216 between static and dynamic linking is useful. The technical benefit is
    217 a gained element of independence from the host, meaning that those
    218 programs can be used independently of the host system. However, it is
    219 worth noting that an overall successful LFS build can still be
    220 achieved when the first two packages are built dynamically.</para>
    221 
    222 </sect2>-->
     104  <para>This is important for the reasons mentioned above. It also demonstrates
     105  that GCC's configure script does not search the PATH directories to find which
     106  tools to use. However, during the actual operation of <command>gcc</command>
     107  itself, the same search paths are not necessarily used. To find out which
     108  standard linker <command>gcc</command> will use, run:
     109  <userinput>gcc -print-prog-name=ld</userinput>.</para>
     110
     111  <para>Detailed information can be obtained from <command>gcc</command> by
     112  passing it the <parameter>-v</parameter> command line option while compiling
     113  a dummy program. For example, <userinput>gcc -v dummy.c</userinput> will show
     114  detailed information about the preprocessor, compilation, and assembly stages,
     115  including <command>gcc</command>'s included search paths and their order.</para>
     116
     117  <para>The next package installed is Glibc. The most important considerations
     118  for building Glibc are the compiler, binary tools, and kernel headers. The
     119  compiler is generally not an issue since Glibc will always use the
     120  <command>gcc</command> found in a <envar>PATH</envar> directory. The binary
     121  tools and kernel headers can be a bit more complicated. Therefore, take no
     122  risks and use the available configure switches to enforce the correct
     123  selections. After the run of <command>configure</command>, check the contents
     124  of the <filename>config.make</filename> file in the <filename
     125  class="directory">glibc-build</filename> directory for all important details.
     126  Note the use of <parameter>CC="gcc -B/tools/bin/"</parameter> to control which
     127  binary tools are used and the use of the <parameter>-nostdinc</parameter>
     128  and <parameter>-isystem</parameter> flags to control the compiler's include
     129  search path. These items highlight an important aspect of the Glibc
     130  package&mdash;it is very self-sufficient in terms of its build machinery and
     131  generally does not rely on toolchain defaults.</para>
     132
     133  <para>After the Glibc installation, make some adjustments to ensure that
     134  searching and linking take place only within the <filename
     135  class="directory">/tools</filename> prefix.  Install an adjusted
     136  <command>ld</command>, which has a hard-wired search path limited to
     137  <filename class="directory">/tools/lib</filename>. Then amend
     138  <command>gcc</command>'s specs file to point to the new dynamic linker in
     139  <filename class="directory">/tools/lib</filename>. This last step is vital
     140  to the whole process. As mentioned above, a hard-wired path to a dynamic
     141  linker is embedded into every Executable and Link Format (ELF)-shared
     142  executable.  This can be inspected by running:
     143  <userinput>readelf -l &lt;name of binary&gt; | grep interpreter</userinput>.
     144  Amending gcc's specs file ensures that every program compiled from here
     145  through the end of this chapter will use the new dynamic linker in
     146  <filename class="directory">/tools/lib</filename>.</para>
     147
     148  <para>The need to use the new dynamic linker is also the reason why
     149  the Specs patch is applied for the second pass of GCC. Failure to do
     150  so will result in the GCC programs themselves having the name of the
     151  dynamic linker from the host system's <filename
     152  class="directory">/lib</filename> directory embedded into them, which
     153  would defeat the goal of getting away from the host.</para>
     154
     155  <para>During the second pass of Binutils, we are able to utilize the
     156  <parameter>--with-lib-path</parameter> configure switch to control
     157  <command>ld</command>'s library search path.  From this point onwards,
     158  the core toolchain is self-contained and self-hosted. The remainder of
     159  the <xref linkend="chapter-temporary-tools"/> packages all build against
     160  the new Glibc in <filename class="directory">/tools</filename>.</para>
     161
     162  <para>Upon entering the chroot environment in <xref
     163  linkend="chapter-building-system"/>, the first major package to be
     164  installed is Glibc, due to its self-sufficient nature mentioned above.
     165  Once this Glibc is installed into <filename
     166  class="directory">/usr</filename>, perform a quick changeover of the
     167  toolchain defaults, then proceed in building the rest of the target
     168  LFS system.</para>
     169
     170  <!-- Removed as part of the fix for bug 1061 - we no longer build pass1
     171      packages statically, therefore this explanation isn't required -->
     172
     173  <!--<sect2>
     174  <title>Notes on Static Linking</title>
     175
     176  <para>Besides their specific task, most programs have to perform many
     177  common and sometimes trivial operations. These include allocating
     178  memory, searching directories, reading and writing files, string
     179  handling, pattern matching, arithmetic, and other tasks. Instead of
     180  obliging each program to reinvent the wheel, the GNU system provides
     181  all these basic functions in ready-made libraries. The major library
     182  on any Linux system is Glibc.</para>
     183
     184  <para>There are two primary ways of linking the functions from a
     185  library to a program that uses them&mdash;statically or dynamically. When
     186  a program is linked statically, the code of the used functions is
     187  included in the executable, resulting in a rather bulky program. When
     188  a program is dynamically linked, it includes a reference to the
     189  dynamic linker, the name of the library, and the name of the function,
     190  resulting in a much smaller executable. A third option is to use the
     191  programming interface of the dynamic linker (see <filename>dlopen(3)</filename>
     192  for more information).</para>
     193
     194  <para>Dynamic linking is the default on Linux and has three major
     195  advantages over static linking. First, only one copy of the executable
     196  library code is needed on the hard disk, instead of having multiple
     197  copies of the same code included in several programs, thus saving
     198  disk space. Second, when several programs use the same library
     199  function at the same time, only one copy of the function's code is
     200  required in core, thus saving memory space. Third, when a library
     201  function gets a bug fixed or is otherwise improved, only the one
     202  library needs to be recompiled instead of recompiling all programs
     203  that make use of the improved function.</para>
     204
     205  <para>If dynamic linking has several advantages, why then do we
     206  statically link the first two packages in this chapter? The reasons
     207  are threefold&mdash;historical, educational, and technical. The
     208  historical reason is that earlier versions of LFS statically linked
     209  every program in this chapter. Educationally, knowing the difference
     210  between static and dynamic linking is useful. The technical benefit is
     211  a gained element of independence from the host, meaning that those
     212  programs can be used independently of the host system. However, it is
     213  worth noting that an overall successful LFS build can still be
     214  achieved when the first two packages are built dynamically.</para>
     215
     216  </sect2>-->
    223217
    224218</sect1>
    225 
Note: See TracChangeset for help on using the changeset viewer.