1 | <?xml version="1.0" encoding="ISO-8859-1"?>
|
---|
2 | <!DOCTYPE sect1 PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
|
---|
3 | "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
|
---|
4 | <!ENTITY % general-entities SYSTEM "../general.ent">
|
---|
5 | %general-entities;
|
---|
6 | ]>
|
---|
7 |
|
---|
8 | <sect1 id="ch-tools-toolchaintechnotes" xreflabel="Toolchain Technical Notes">
|
---|
9 | <?dbhtml filename="toolchaintechnotes.html"?>
|
---|
10 |
|
---|
11 | <title>Toolchain Technical Notes</title>
|
---|
12 |
|
---|
13 | <para>This section explains some of the rationale and technical details
|
---|
14 | behind the overall build method. It is not essential to immediately
|
---|
15 | understand everything in this section. Most of this information will be
|
---|
16 | clearer after performing an actual build. This section can be referred
|
---|
17 | to at any time during the process.</para>
|
---|
18 |
|
---|
19 | <para>The overall goal of <xref linkend="chapter-cross-tools"/> and <xref
|
---|
20 | linkend="chapter-temporary-tools"/> is to produce a temporary area that
|
---|
21 | contains a known-good set of tools that can be isolated from the host system.
|
---|
22 | By using <command>chroot</command>, the commands in the remaining chapters
|
---|
23 | will be contained within that environment, ensuring a clean, trouble-free
|
---|
24 | build of the target LFS system. The build process has been designed to
|
---|
25 | minimize the risks for new readers and to provide the most educational value
|
---|
26 | at the same time.</para>
|
---|
27 |
|
---|
28 | <para>The build process is based on the process of
|
---|
29 | <emphasis>cross-compilation</emphasis>. Cross-compilation is normally used
|
---|
30 | for building a compiler and its toolchain for a machine different from
|
---|
31 | the one that is used for the build. This is not strictly needed for LFS,
|
---|
32 | since the machine where the new system will run is the same as the one
|
---|
33 | used for the build. But cross-compilation has the great advantage that
|
---|
34 | anything that is cross-compiled cannot depend on the host environment.</para>
|
---|
35 |
|
---|
36 | <sect2 id="cross-compile" xreflabel="About Cross-Compilation">
|
---|
37 |
|
---|
38 | <title>About Cross-Compilation</title>
|
---|
39 |
|
---|
40 | <para>Cross-compilation involves some concepts that deserve a section on
|
---|
41 | their own. Although this section may be omitted in a first reading, it
|
---|
42 | is strongly suggested to come back to it later in order to get a full
|
---|
43 | grasp of the build process.</para>
|
---|
44 |
|
---|
45 | <para>Let us first define some terms used in this context:</para>
|
---|
46 |
|
---|
47 | <variablelist>
|
---|
48 | <varlistentry><term>build</term><listitem>
|
---|
49 | <para>is the machine where we build programs. Note that this machine
|
---|
50 | is referred to as the <quote>host</quote> in other
|
---|
51 | sections.</para></listitem>
|
---|
52 | </varlistentry>
|
---|
53 |
|
---|
54 | <varlistentry><term>host</term><listitem>
|
---|
55 | <para>is the machine/system where the built programs will run. Note
|
---|
56 | that this use of <quote>host</quote> is not the same as in other
|
---|
57 | sections.</para></listitem>
|
---|
58 | </varlistentry>
|
---|
59 |
|
---|
60 | <varlistentry><term>target</term><listitem>
|
---|
61 | <para>is only used for compilers. It is the machine the compiler
|
---|
62 | produces code for. It may be different from both build and
|
---|
63 | host.</para></listitem>
|
---|
64 | </varlistentry>
|
---|
65 |
|
---|
66 | </variablelist>
|
---|
67 |
|
---|
68 | <para>As an example, let us imagine the following scenario (sometimes
|
---|
69 | referred to as <quote>Canadian Cross</quote>): we may have a
|
---|
70 | compiler on a slow machine only, let's call the machine A, and the compiler
|
---|
71 | ccA. We may have also a fast machine (B), but with no compiler, and we may
|
---|
72 | want to produce code for another slow machine (C). To build a
|
---|
73 | compiler for machine C, we would have three stages:</para>
|
---|
74 |
|
---|
75 | <informaltable align="center">
|
---|
76 | <tgroup cols="5">
|
---|
77 | <colspec colnum="1" align="center"/>
|
---|
78 | <colspec colnum="2" align="center"/>
|
---|
79 | <colspec colnum="3" align="center"/>
|
---|
80 | <colspec colnum="4" align="center"/>
|
---|
81 | <colspec colnum="5" align="left"/>
|
---|
82 | <thead>
|
---|
83 | <row><entry>Stage</entry><entry>Build</entry><entry>Host</entry>
|
---|
84 | <entry>Target</entry><entry>Action</entry></row>
|
---|
85 | </thead>
|
---|
86 | <tbody>
|
---|
87 | <row>
|
---|
88 | <entry>1</entry><entry>A</entry><entry>A</entry><entry>B</entry>
|
---|
89 | <entry>build cross-compiler cc1 using ccA on machine A</entry>
|
---|
90 | </row>
|
---|
91 | <row>
|
---|
92 | <entry>2</entry><entry>A</entry><entry>B</entry><entry>C</entry>
|
---|
93 | <entry>build cross-compiler cc2 using cc1 on machine A</entry>
|
---|
94 | </row>
|
---|
95 | <row>
|
---|
96 | <entry>3</entry><entry>B</entry><entry>C</entry><entry>C</entry>
|
---|
97 | <entry>build compiler ccC using cc2 on machine B</entry>
|
---|
98 | </row>
|
---|
99 | </tbody>
|
---|
100 | </tgroup>
|
---|
101 | </informaltable>
|
---|
102 |
|
---|
103 | <para>Then, all the other programs needed by machine C can be compiled
|
---|
104 | using cc2 on the fast machine B. Note that unless B can run programs
|
---|
105 | produced for C, there is no way to test the built programs until machine
|
---|
106 | C itself is running. For example, for testing ccC, we may want to add a
|
---|
107 | fourth stage:</para>
|
---|
108 |
|
---|
109 | <informaltable align="center">
|
---|
110 | <tgroup cols="5">
|
---|
111 | <colspec colnum="1" align="center"/>
|
---|
112 | <colspec colnum="2" align="center"/>
|
---|
113 | <colspec colnum="3" align="center"/>
|
---|
114 | <colspec colnum="4" align="center"/>
|
---|
115 | <colspec colnum="5" align="left"/>
|
---|
116 | <thead>
|
---|
117 | <row><entry>Stage</entry><entry>Build</entry><entry>Host</entry>
|
---|
118 | <entry>Target</entry><entry>Action</entry></row>
|
---|
119 | </thead>
|
---|
120 | <tbody>
|
---|
121 | <row>
|
---|
122 | <entry>4</entry><entry>C</entry><entry>C</entry><entry>C</entry>
|
---|
123 | <entry>rebuild and test ccC using itself on machine C</entry>
|
---|
124 | </row>
|
---|
125 | </tbody>
|
---|
126 | </tgroup>
|
---|
127 | </informaltable>
|
---|
128 |
|
---|
129 | <para>In the example above, only cc1 and cc2 are cross-compilers, that is,
|
---|
130 | they produce code for a machine different from the one they are run on.
|
---|
131 | The other compilers ccA and ccC produce code for the machine they are run
|
---|
132 | on. Such compilers are called <emphasis>native</emphasis> compilers.</para>
|
---|
133 |
|
---|
134 | </sect2>
|
---|
135 |
|
---|
136 | <sect2 id="lfs-cross">
|
---|
137 | <title>Implementation of Cross-Compilation for LFS</title>
|
---|
138 |
|
---|
139 | <note>
|
---|
140 | <para>Almost all the build systems use names of the form
|
---|
141 | cpu-vendor-kernel-os referred to as the machine triplet. An astute
|
---|
142 | reader may wonder why a <quote>triplet</quote> refers to a four component
|
---|
143 | name. The reason is history: initially, three component names were enough
|
---|
144 | to designate unambiguously a machine, but with new machines and systems
|
---|
145 | appearing, that proved insufficient. The word <quote>triplet</quote>
|
---|
146 | remained. A simple way to determine your machine triplet is to run
|
---|
147 | the <command>config.guess</command>
|
---|
148 | script that comes with the source for many packages. Unpack the binutils
|
---|
149 | sources and run the script: <userinput>./config.guess</userinput> and note
|
---|
150 | the output. For example, for a 32-bit Intel processor the
|
---|
151 | output will be <emphasis>i686-pc-linux-gnu</emphasis>. On a 64-bit
|
---|
152 | system it will be <emphasis>x86_64-pc-linux-gnu</emphasis>.</para>
|
---|
153 |
|
---|
154 | <para>Also be aware of the name of the platform's dynamic linker, often
|
---|
155 | referred to as the dynamic loader (not to be confused with the standard
|
---|
156 | linker <command>ld</command> that is part of binutils). The dynamic linker
|
---|
157 | provided by Glibc finds and loads the shared libraries needed by a
|
---|
158 | program, prepares the program to run, and then runs it. The name of the
|
---|
159 | dynamic linker for a 32-bit Intel machine will be <filename
|
---|
160 | class="libraryfile">ld-linux.so.2</filename> (<filename
|
---|
161 | class="libraryfile">ld-linux-x86-64.so.2</filename> for 64-bit systems). A
|
---|
162 | sure-fire way to determine the name of the dynamic linker is to inspect a
|
---|
163 | random binary from the host system by running: <userinput>readelf -l
|
---|
164 | <name of binary> | grep interpreter</userinput> and noting the
|
---|
165 | output. The authoritative reference covering all platforms is in the
|
---|
166 | <filename>shlib-versions</filename> file in the root of the Glibc source
|
---|
167 | tree.</para>
|
---|
168 | </note>
|
---|
169 |
|
---|
170 | <para>In order to fake a cross compilation, the name of the host triplet
|
---|
171 | is slightly adjusted by changing the "vendor" field in the
|
---|
172 | <envar>LFS_TGT</envar> variable. We also use the
|
---|
173 | <parameter>--with-sysroot</parameter> option when building the cross linker and
|
---|
174 | cross compiler to tell them where to find the needed host files. This
|
---|
175 | ensures that none of the other programs built in <xref
|
---|
176 | linkend="chapter-temporary-tools"/> can link to libraries on the build
|
---|
177 | machine. Only two stages are mandatory, and one more for tests:</para>
|
---|
178 |
|
---|
179 | <informaltable align="center">
|
---|
180 | <tgroup cols="5">
|
---|
181 | <colspec colnum="1" align="center"/>
|
---|
182 | <colspec colnum="2" align="center"/>
|
---|
183 | <colspec colnum="3" align="center"/>
|
---|
184 | <colspec colnum="4" align="center"/>
|
---|
185 | <colspec colnum="5" align="left"/>
|
---|
186 | <thead>
|
---|
187 | <row><entry>Stage</entry><entry>Build</entry><entry>Host</entry>
|
---|
188 | <entry>Target</entry><entry>Action</entry></row>
|
---|
189 | </thead>
|
---|
190 | <tbody>
|
---|
191 | <row>
|
---|
192 | <entry>1</entry><entry>pc</entry><entry>pc</entry><entry>lfs</entry>
|
---|
193 | <entry>build cross-compiler cc1 using cc-pc on pc</entry>
|
---|
194 | </row>
|
---|
195 | <row>
|
---|
196 | <entry>2</entry><entry>pc</entry><entry>lfs</entry><entry>lfs</entry>
|
---|
197 | <entry>build compiler cc-lfs using cc1 on pc</entry>
|
---|
198 | </row>
|
---|
199 | <row>
|
---|
200 | <entry>3</entry><entry>lfs</entry><entry>lfs</entry><entry>lfs</entry>
|
---|
201 | <entry>rebuild and test cc-lfs using itself on lfs</entry>
|
---|
202 | </row>
|
---|
203 | </tbody>
|
---|
204 | </tgroup>
|
---|
205 | </informaltable>
|
---|
206 |
|
---|
207 | <para>In the above table, <quote>on pc</quote> means the commands are run
|
---|
208 | on a machine using the already installed distribution. <quote>On
|
---|
209 | lfs</quote> means the commands are run in a chrooted environment.</para>
|
---|
210 |
|
---|
211 | <para>Now, there is more about cross-compiling: the C language is not
|
---|
212 | just a compiler, but also defines a standard library. In this book, the
|
---|
213 | GNU C library, named glibc, is used. This library must
|
---|
214 | be compiled for the lfs machine, that is, using the cross compiler cc1.
|
---|
215 | But the compiler itself uses an internal library implementing complex
|
---|
216 | instructions not available in the assembler instruction set. This
|
---|
217 | internal library is named libgcc, and must be linked to the glibc
|
---|
218 | library to be fully functional! Furthermore, the standard library for
|
---|
219 | C++ (libstdc++) also needs being linked to glibc. The solution to this
|
---|
220 | chicken and egg problem is to first build a degraded cc1 based libgcc,
|
---|
221 | lacking some functionalities such as threads and exception handling, then
|
---|
222 | build glibc using this degraded compiler (glibc itself is not
|
---|
223 | degraded), then build libstdc++. But this last library will lack the
|
---|
224 | same functionalities as libgcc.</para>
|
---|
225 |
|
---|
226 | <para>This is not the end of the story: the conclusion of the preceding
|
---|
227 | paragraph is that cc1 is unable to build a fully functional libstdc++, but
|
---|
228 | this is the only compiler available for building the C/C++ libraries
|
---|
229 | during stage 2! Of course, the compiler built during stage 2, cc-lfs,
|
---|
230 | would be able to build those libraries, but (1) the build system of
|
---|
231 | GCC does not know that it is usable on pc, and (2) using it on pc
|
---|
232 | would be at risk of linking to the pc libraries, since cc-lfs is a native
|
---|
233 | compiler. So we have to build libstdc++ later, in chroot.</para>
|
---|
234 |
|
---|
235 | </sect2>
|
---|
236 |
|
---|
237 | <sect2 id="other-details">
|
---|
238 |
|
---|
239 | <title>Other procedural details</title>
|
---|
240 |
|
---|
241 | <para>The cross-compiler will be installed in a separate <filename
|
---|
242 | class="directory">$LFS/tools</filename> directory, since it will not
|
---|
243 | be part of the final system.</para>
|
---|
244 |
|
---|
245 | <para>Binutils is installed first because the <command>configure</command>
|
---|
246 | runs of both GCC and Glibc perform various feature tests on the assembler
|
---|
247 | and linker to determine which software features to enable or disable. This
|
---|
248 | is more important than one might first realize. An incorrectly configured
|
---|
249 | GCC or Glibc can result in a subtly broken toolchain, where the impact of
|
---|
250 | such breakage might not show up until near the end of the build of an
|
---|
251 | entire distribution. A test suite failure will usually highlight this error
|
---|
252 | before too much additional work is performed.</para>
|
---|
253 |
|
---|
254 | <para>Binutils installs its assembler and linker in two locations,
|
---|
255 | <filename class="directory">$LFS/tools/bin</filename> and <filename
|
---|
256 | class="directory">$LFS/tools/$LFS_TGT/bin</filename>. The tools in one
|
---|
257 | location are hard linked to the other. An important facet of the linker is
|
---|
258 | its library search order. Detailed information can be obtained from
|
---|
259 | <command>ld</command> by passing it the <parameter>--verbose</parameter>
|
---|
260 | flag. For example, <command>$LFS_TGT-ld --verbose | grep SEARCH</command>
|
---|
261 | will illustrate the current search paths and their order. It shows which
|
---|
262 | files are linked by <command>ld</command> by compiling a dummy program and
|
---|
263 | passing the <parameter>--verbose</parameter> switch to the linker. For
|
---|
264 | example,
|
---|
265 | <command>$LFS_TGT-gcc dummy.c -Wl,--verbose 2>&1 | grep succeeded</command>
|
---|
266 | will show all the files successfully opened during the linking.</para>
|
---|
267 |
|
---|
268 | <para>The next package installed is GCC. An example of what can be
|
---|
269 | seen during its run of <command>configure</command> is:</para>
|
---|
270 |
|
---|
271 | <screen><computeroutput>checking what assembler to use... /mnt/lfs/tools/i686-lfs-linux-gnu/bin/as
|
---|
272 | checking what linker to use... /mnt/lfs/tools/i686-lfs-linux-gnu/bin/ld</computeroutput></screen>
|
---|
273 |
|
---|
274 | <para>This is important for the reasons mentioned above. It also
|
---|
275 | demonstrates that GCC's configure script does not search the PATH
|
---|
276 | directories to find which tools to use. However, during the actual
|
---|
277 | operation of <command>gcc</command> itself, the same search paths are not
|
---|
278 | necessarily used. To find out which standard linker <command>gcc</command>
|
---|
279 | will use, run: <command>$LFS_TGT-gcc -print-prog-name=ld</command>.</para>
|
---|
280 |
|
---|
281 | <para>Detailed information can be obtained from <command>gcc</command> by
|
---|
282 | passing it the <parameter>-v</parameter> command line option while compiling
|
---|
283 | a dummy program. For example, <command>gcc -v dummy.c</command> will show
|
---|
284 | detailed information about the preprocessor, compilation, and assembly
|
---|
285 | stages, including <command>gcc</command>'s included search paths and their
|
---|
286 | order.</para>
|
---|
287 |
|
---|
288 | <para>Next installed are sanitized Linux API headers. These allow the
|
---|
289 | standard C library (Glibc) to interface with features that the Linux
|
---|
290 | kernel will provide.</para>
|
---|
291 |
|
---|
292 | <para>The next package installed is Glibc. The most important
|
---|
293 | considerations for building Glibc are the compiler, binary tools, and
|
---|
294 | kernel headers. The compiler is generally not an issue since Glibc will
|
---|
295 | always use the compiler relating to the <parameter>--host</parameter>
|
---|
296 | parameter passed to its configure script; e.g. in our case, the compiler
|
---|
297 | will be <command>$LFS_TGT-gcc</command>. The binary tools and kernel
|
---|
298 | headers can be a bit more complicated. Therefore, take no risks and use
|
---|
299 | the available configure switches to enforce the correct selections. After
|
---|
300 | the run of <command>configure</command>, check the contents of the
|
---|
301 | <filename>config.make</filename> file in the <filename
|
---|
302 | class="directory">build</filename> directory for all important details.
|
---|
303 | Note the use of <parameter>CC="$LFS_TGT-gcc"</parameter> (with
|
---|
304 | <envar>$LFS_TGT</envar> expanded) to control which binary tools are used
|
---|
305 | and the use of the <parameter>-nostdinc</parameter> and
|
---|
306 | <parameter>-isystem</parameter> flags to control the compiler's include
|
---|
307 | search path. These items highlight an important aspect of the Glibc
|
---|
308 | package—it is very self-sufficient in terms of its build machinery
|
---|
309 | and generally does not rely on toolchain defaults.</para>
|
---|
310 |
|
---|
311 | <para>As said above, the standard C++ library is compiled next, followed in
|
---|
312 | <xref linkend="chapter-temporary-tools"/> by all the programs that need
|
---|
313 | themselves to be built. The install step of all those packages uses the
|
---|
314 | <envar>DESTDIR</envar> variable to have the
|
---|
315 | programs land into the LFS filesystem.</para>
|
---|
316 |
|
---|
317 | <para>At the end of <xref linkend="chapter-temporary-tools"/> the native
|
---|
318 | lfs compiler is installed. First binutils-pass2 is built,
|
---|
319 | with the same <envar>DESTDIR</envar> install as the other programs,
|
---|
320 | then the second pass of GCC is constructed, omitting libstdc++
|
---|
321 | and other non-important libraries. Due to some weird logic in GCC's
|
---|
322 | configure script, <envar>CC_FOR_TARGET</envar> ends up as
|
---|
323 | <command>cc</command> when the host is the same as the target, but is
|
---|
324 | different from the build system. This is why
|
---|
325 | <parameter>CC_FOR_TARGET=$LFS_TGT-gcc</parameter> is put explicitly into
|
---|
326 | the configure options.</para>
|
---|
327 |
|
---|
328 | <para>Upon entering the chroot environment in <xref
|
---|
329 | linkend="chapter-chroot-temporary-tools"/>, the first task is to install
|
---|
330 | libstdc++. Then temporary installations of programs needed for the proper
|
---|
331 | operation of the toolchain are performed. From this point onwards, the
|
---|
332 | core toolchain is self-contained and self-hosted. In
|
---|
333 | <xref linkend="chapter-building-system"/>, final versions of all the
|
---|
334 | packages needed for a fully functional system are built, tested and
|
---|
335 | installed.</para>
|
---|
336 |
|
---|
337 | </sect2>
|
---|
338 |
|
---|
339 | </sect1>
|
---|