source: part3intro/toolchaintechnotes.xml

trunk
Last change on this file was 360fdfca, checked in by Pierre Labastie <pierre.labastie@…>, 3 months ago

Fix punctuation in quotes, and quote signs

  • period and comma inside quotes
  • " to <quote>
  • some " to <literal> when it is a var value
  • Property mode set to 100644
File size: 20.4 KB
Line 
1<?xml version="1.0" encoding="UTF-8"?>
2<!DOCTYPE sect1 PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
3 "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
4 <!ENTITY % general-entities SYSTEM "../general.ent">
5 %general-entities;
6]>
7
8<sect1 id="ch-tools-toolchaintechnotes" xreflabel="Toolchain Technical Notes">
9 <?dbhtml filename="toolchaintechnotes.html"?>
10
11 <title>Toolchain Technical Notes</title>
12
13 <para>This section explains some of the rationale and technical details
14 behind the overall build method. Don't try to immediately
15 understand everything in this section. Most of this information will be
16 clearer after performing an actual build. Come back and re-read this chapter
17 at any time during the build process.</para>
18
19 <para>The overall goal of <xref linkend="chapter-cross-tools"/> and <xref
20 linkend="chapter-temporary-tools"/> is to produce a temporary area
21 containing a set of tools that are known to be good, and that are isolated from the host system.
22 By using the <command>chroot</command> command, the compilations in the remaining chapters
23 will be isolated within that environment, ensuring a clean, trouble-free
24 build of the target LFS system. The build process has been designed to
25 minimize the risks for new readers, and to provide the most educational value
26 at the same time.</para>
27
28 <para>This build process is based on
29 <emphasis>cross-compilation</emphasis>. Cross-compilation is normally used
30 to build a compiler and its associated toolchain for a machine different from
31 the one that is used for the build. This is not strictly necessary for LFS,
32 since the machine where the new system will run is the same as the one
33 used for the build. But cross-compilation has one great advantage:
34 anything that is cross-compiled cannot depend on the host environment.</para>
35
36 <sect2 id="cross-compile" xreflabel="About Cross-Compilation">
37
38 <title>About Cross-Compilation</title>
39
40 <note>
41 <para>
42 The LFS book is not (and does not contain) a general tutorial to
43 build a cross- (or native) toolchain. Don't use the commands in the
44 book for a cross-toolchain for some purpose other
45 than building LFS, unless you really understand what you are doing.
46 </para>
47 </note>
48
49 <para>Cross-compilation involves some concepts that deserve a section of
50 their own. Although this section may be omitted on a first reading,
51 coming back to it later will help you gain a fuller understanding of
52 the process.</para>
53
54 <para>Let us first define some terms used in this context.</para>
55
56 <variablelist>
57 <varlistentry><term>The build</term><listitem>
58 <para>is the machine where we build programs. Note that this machine
59 is also referred to as the <quote>host.</quote></para></listitem>
60 </varlistentry>
61
62 <varlistentry><term>The host</term><listitem>
63 <para>is the machine/system where the built programs will run. Note
64 that this use of <quote>host</quote> is not the same as in other
65 sections.</para></listitem>
66 </varlistentry>
67
68 <varlistentry><term>The target</term><listitem>
69 <para>is only used for compilers. It is the machine the compiler
70 produces code for. It may be different from both the build and
71 the host.</para></listitem>
72 </varlistentry>
73
74 </variablelist>
75
76 <para>As an example, let us imagine the following scenario (sometimes
77 referred to as <quote>Canadian Cross</quote>). We have a
78 compiler on a slow machine only, let's call it machine A, and the compiler
79 ccA. We also have a fast machine (B), but no compiler for (B), and we
80 want to produce code for a third, slow machine (C). We will build a
81 compiler for machine C in three stages.</para>
82
83 <informaltable align="center">
84 <tgroup cols="5">
85 <colspec colnum="1" align="center"/>
86 <colspec colnum="2" align="center"/>
87 <colspec colnum="3" align="center"/>
88 <colspec colnum="4" align="center"/>
89 <colspec colnum="5" align="left"/>
90 <thead>
91 <row><entry>Stage</entry><entry>Build</entry><entry>Host</entry>
92 <entry>Target</entry><entry>Action</entry></row>
93 </thead>
94 <tbody>
95 <row>
96 <entry>1</entry><entry>A</entry><entry>A</entry><entry>B</entry>
97 <entry>Build cross-compiler cc1 using ccA on machine A.</entry>
98 </row>
99 <row>
100 <entry>2</entry><entry>A</entry><entry>B</entry><entry>C</entry>
101 <entry>Build cross-compiler cc2 using cc1 on machine A.</entry>
102 </row>
103 <row>
104 <entry>3</entry><entry>B</entry><entry>C</entry><entry>C</entry>
105 <entry>Build compiler ccC using cc2 on machine B.</entry>
106 </row>
107 </tbody>
108 </tgroup>
109 </informaltable>
110
111 <para>Then, all the programs needed by machine C can be compiled
112 using cc2 on the fast machine B. Note that unless B can run programs
113 produced for C, there is no way to test the newly built programs until machine
114 C itself is running. For example, to run a test suite on ccC, we may want to add a
115 fourth stage:</para>
116
117 <informaltable align="center">
118 <tgroup cols="5">
119 <colspec colnum="1" align="center"/>
120 <colspec colnum="2" align="center"/>
121 <colspec colnum="3" align="center"/>
122 <colspec colnum="4" align="center"/>
123 <colspec colnum="5" align="left"/>
124 <thead>
125 <row><entry>Stage</entry><entry>Build</entry><entry>Host</entry>
126 <entry>Target</entry><entry>Action</entry></row>
127 </thead>
128 <tbody>
129 <row>
130 <entry>4</entry><entry>C</entry><entry>C</entry><entry>C</entry>
131 <entry>Rebuild and test ccC using ccC on machine C.</entry>
132 </row>
133 </tbody>
134 </tgroup>
135 </informaltable>
136
137 <para>In the example above, only cc1 and cc2 are cross-compilers, that is,
138 they produce code for a machine different from the one they are run on.
139 The other compilers ccA and ccC produce code for the machine they are run
140 on. Such compilers are called <emphasis>native</emphasis> compilers.</para>
141
142 </sect2>
143
144 <sect2 id="lfs-cross">
145 <title>Implementation of Cross-Compilation for LFS</title>
146
147 <note>
148 <para>All the cross-compiled packages in this book use an
149 autoconf-based building system. The autoconf-based building system
150 accepts system types in the form cpu-vendor-kernel-os,
151 referred to as the system triplet. Since the vendor field is often
152 irrelevant, autoconf lets you omit it.</para>
153
154 <para>An astute reader may wonder
155 why a <quote>triplet</quote> refers to a four component name. The
156 kernel field and the os field began as a single
157 <quote>system</quote> field. Such a three-field form is still valid
158 today for some systems, for example,
159 <literal>x86_64-unknown-freebsd</literal>. But
160 two systems can share the same kernel and still be too different to
161 use the same triplet to describe them. For example, Android running on a
162 mobile phone is completely different from Ubuntu running on an ARM64
163 server, even though they are both running on the same type of CPU (ARM64) and
164 using the same kernel (Linux).</para>
165
166 <para>Without an emulation layer, you cannot run an
167 executable for a server on a mobile phone or vice versa. So the
168 <quote>system</quote> field has been divided into kernel and os fields, to
169 designate these systems unambiguously. In our example, the Android
170 system is designated <literal>aarch64-unknown-linux-android</literal>,
171 and the Ubuntu system is designated
172 <literal>aarch64-unknown-linux-gnu</literal>.</para>
173
174 <para>The word <quote>triplet</quote> remains embedded in the lexicon. A simple way to determine your
175 system triplet is to run the <command>config.guess</command>
176 script that comes with the source for many packages. Unpack the binutils
177 sources, run the script <userinput>./config.guess</userinput>, and note
178 the output. For example, for a 32-bit Intel processor the
179 output will be <emphasis>i686-pc-linux-gnu</emphasis>. On a 64-bit
180 system it will be <emphasis>x86_64-pc-linux-gnu</emphasis>. On most
181 Linux systems the even simpler <command>gcc -dumpmachine</command> command
182 will give you similar information.</para>
183
184 <para>You should also be aware of the name of the platform's dynamic linker, often
185 referred to as the dynamic loader (not to be confused with the standard
186 linker <command>ld</command> that is part of binutils). The dynamic linker
187 provided by package glibc finds and loads the shared libraries needed by a
188 program, prepares the program to run, and then runs it. The name of the
189 dynamic linker for a 32-bit Intel machine is <filename
190 class="libraryfile">ld-linux.so.2</filename>; it's <filename
191 class="libraryfile">ld-linux-x86-64.so.2</filename> on 64-bit systems. A
192 sure-fire way to determine the name of the dynamic linker is to inspect a
193 random binary from the host system by running: <userinput>readelf -l
194 &lt;name of binary&gt; | grep interpreter</userinput> and noting the
195 output. The authoritative reference covering all platforms is in the
196 <filename>shlib-versions</filename> file in the root of the glibc source
197 tree.</para>
198 </note>
199
200 <para>In order to fake a cross-compilation in LFS, the name of the host triplet
201 is slightly adjusted by changing the &quot;vendor&quot; field in the
202 <envar>LFS_TGT</envar> variable so it says &quot;lfs&quot;. We also use the
203 <parameter>--with-sysroot</parameter> option when building the cross-linker and
204 cross-compiler, to tell them where to find the needed host files. This
205 ensures that none of the other programs built in <xref
206 linkend="chapter-temporary-tools"/> can link to libraries on the build
207 machine. Only two stages are mandatory, plus one more for tests.</para>
208
209 <informaltable align="center">
210 <tgroup cols="5">
211 <colspec colnum="1" align="center"/>
212 <colspec colnum="2" align="center"/>
213 <colspec colnum="3" align="center"/>
214 <colspec colnum="4" align="center"/>
215 <colspec colnum="5" align="left"/>
216 <thead>
217 <row><entry>Stage</entry><entry>Build</entry><entry>Host</entry>
218 <entry>Target</entry><entry>Action</entry></row>
219 </thead>
220 <tbody>
221 <row>
222 <entry>1</entry><entry>pc</entry><entry>pc</entry><entry>lfs</entry>
223 <entry>Build cross-compiler cc1 using cc-pc on pc.</entry>
224 </row>
225 <row>
226 <entry>2</entry><entry>pc</entry><entry>lfs</entry><entry>lfs</entry>
227 <entry>Build compiler cc-lfs using cc1 on pc.</entry>
228 </row>
229 <row>
230 <entry>3</entry><entry>lfs</entry><entry>lfs</entry><entry>lfs</entry>
231 <entry>Rebuild and test cc-lfs using cc-lfs on lfs.</entry>
232 </row>
233 </tbody>
234 </tgroup>
235 </informaltable>
236
237 <para>In the preceding table, <quote>on pc</quote> means the commands are run
238 on a machine using the already installed distribution. <quote>On
239 lfs</quote> means the commands are run in a chrooted environment.</para>
240
241 <para>This is not yet the end of the story. The C language is not
242 merely a compiler; it also defines a standard library. In this book, the
243 GNU C library, named glibc, is used (there is an alternative, &quot;musl&quot;). This library must
244 be compiled for the LFS machine; that is, using the cross-compiler cc1.
245 But the compiler itself uses an internal library providing complex
246 subroutines for functions not available in the assembler instruction set. This
247 internal library is named libgcc, and it must be linked to the glibc
248 library to be fully functional. Furthermore, the standard library for
249 C++ (libstdc++) must also be linked with glibc. The solution to this
250 chicken and egg problem is first to build a degraded cc1-based libgcc,
251 lacking some functionalities such as threads and exception handling, and then
252 to build glibc using this degraded compiler (glibc itself is not
253 degraded), and also to build libstdc++. This last library will lack some of the
254 functionality of libgcc.</para>
255
256 <para>The upshot of the preceding paragraph is that cc1 is unable to
257 build a fully functional libstdc++ with the degraded libgcc, but cc1
258 is the only compiler available for building the C/C++ libraries
259 during stage 2. There are two reasons we don't immediately use the
260 compiler built in stage 2, cc-lfs, to build those libraries.</para>
261
262 <itemizedlist>
263 <listitem>
264 <para>
265 Generally speaking, cc-lfs cannot run on pc (the host system). Even though the
266 triplets for pc and lfs are compatible with each other, an executable
267 for lfs must depend on glibc-&glibc-version;; the host distro
268 may utilize either a different implementation of libc (for example, musl), or
269 a previous release of glibc (for example, glibc-2.13).
270 </para>
271 </listitem>
272 <listitem>
273 <para>
274 Even if cc-lfs can run on pc, using it on pc would create
275 a risk of linking to the pc libraries, since cc-lfs is a native
276 compiler.
277 </para>
278 </listitem>
279 </itemizedlist>
280
281 <para>So when we build gcc stage 2, we instruct the building system to
282 rebuild libgcc and libstdc++ with cc1, but we link libstdc++ to the newly
283 rebuilt libgcc instead of the old, degraded build. This makes the rebuilt
284 libstdc++ fully functional.</para>
285
286 <para>In &ch-final; (or <quote>stage 3</quote>), all the packages needed for
287 the LFS system are built. Even if a package has already been installed into
288 the LFS system in a previous chapter, we still rebuild the package. The main reason for
289 rebuilding these packages is to make them stable: if we reinstall an LFS
290 package on a completed LFS system, the reinstalled content of the package
291 should be the same as the content of the same package when first installed in
292 &ch-final;. The temporary packages installed in &ch-tmp-cross; or
293 &ch-tmp-chroot; cannot satisfy this requirement, because some of them
294 are built without optional dependencies, and autoconf cannot
295 perform some feature checks in &ch-tmp-cross; because of cross-compilation,
296 causing the temporary packages to lack optional features,
297 or use suboptimal code routines. Additionally, a minor reason for
298 rebuilding the packages is to run the test suites.</para>
299
300 </sect2>
301
302 <sect2 id="other-details">
303
304 <title>Other Procedural Details</title>
305
306 <para>The cross-compiler will be installed in a separate <filename
307 class="directory">$LFS/tools</filename> directory, since it will not
308 be part of the final system.</para>
309
310 <para>Binutils is installed first because the <command>configure</command>
311 runs of both gcc and glibc perform various feature tests on the assembler
312 and linker to determine which software features to enable or disable. This
313 is more important than one might realize at first. An incorrectly configured
314 gcc or glibc can result in a subtly broken toolchain, where the impact of
315 such breakage might not show up until near the end of the build of an
316 entire distribution. A test suite failure will usually highlight this error
317 before too much additional work is performed.</para>
318
319 <para>Binutils installs its assembler and linker in two locations,
320 <filename class="directory">$LFS/tools/bin</filename> and <filename
321 class="directory">$LFS/tools/$LFS_TGT/bin</filename>. The tools in one
322 location are hard linked to the other. An important facet of the linker is
323 its library search order. Detailed information can be obtained from
324 <command>ld</command> by passing it the <parameter>--verbose</parameter>
325 flag. For example, <command>$LFS_TGT-ld --verbose | grep SEARCH</command>
326 will illustrate the current search paths and their order. (Note that this
327 example can be run as shown only while logged in as user
328 <systemitem class="username">lfs</systemitem>. If you come back to this
329 page later, replace <command>$LFS_TGT-ld</command> with
330 <command>ld</command>).</para>
331
332 <para>The next package installed is gcc. An example of what can be
333 seen during its run of <command>configure</command> is:</para>
334
335<screen><computeroutput>checking what assembler to use... /mnt/lfs/tools/i686-lfs-linux-gnu/bin/as
336checking what linker to use... /mnt/lfs/tools/i686-lfs-linux-gnu/bin/ld</computeroutput></screen>
337
338 <para>This is important for the reasons mentioned above. It also
339 demonstrates that gcc's configure script does not search the PATH
340 directories to find which tools to use. However, during the actual
341 operation of <command>gcc</command> itself, the same search paths are not
342 necessarily used. To find out which standard linker <command>gcc</command>
343 will use, run: <command>$LFS_TGT-gcc -print-prog-name=ld</command>. (Again,
344 remove the <command>$LFS_TGT-</command> prefix if coming back to this
345 later.)</para>
346
347 <para>Detailed information can be obtained from <command>gcc</command> by
348 passing it the <parameter>-v</parameter> command line option while compiling
349 a program. For example, <command>$LFS_TGT-gcc -v
350 <replaceable>example.c</replaceable></command> (or without <command>
351 $LFS_TGT-</command> if coming back later) will show
352 detailed information about the preprocessor, compilation, and assembly
353 stages, including <command>gcc</command>'s search paths for included
354 headers and their order.</para>
355
356 <para>Next up: sanitized Linux API headers. These allow the
357 standard C library (glibc) to interface with features that the Linux
358 kernel will provide.</para>
359
360 <para>Next comes glibc. The most important
361 considerations for building glibc are the compiler, binary tools, and
362 kernel headers. The compiler is generally not an issue since glibc will
363 always use the compiler relating to the <parameter>--host</parameter>
364 parameter passed to its configure script; e.g., in our case, the compiler
365 will be <command>$LFS_TGT-gcc</command>. The binary tools and kernel
366 headers can be a bit more complicated. Therefore, we take no risks and use
367 the available configure switches to enforce the correct selections. After
368 the run of <command>configure</command>, check the contents of the
369 <filename>config.make</filename> file in the <filename
370 class="directory">build</filename> directory for all important details.
371 Note the use of <parameter>CC="$LFS_TGT-gcc"</parameter> (with
372 <envar>$LFS_TGT</envar> expanded) to control which binary tools are used
373 and the use of the <parameter>-nostdinc</parameter> and
374 <parameter>-isystem</parameter> flags to control the compiler's include
375 search path. These items highlight an important aspect of the glibc
376 package&mdash;it is very self-sufficient in terms of its build machinery,
377 and generally does not rely on toolchain defaults.</para>
378
379 <para>As mentioned above, the standard C++ library is compiled next, followed in
380 <xref linkend="chapter-temporary-tools"/> by other programs that must
381 be cross-compiled to break circular dependencies at build time.
382 The install step of all those packages uses the
383 <envar>DESTDIR</envar> variable to force installation
384 in the LFS filesystem.</para>
385
386 <para>At the end of <xref linkend="chapter-temporary-tools"/> the native
387 LFS compiler is installed. First binutils-pass2 is built,
388 in the same <envar>DESTDIR</envar> directory as the other programs,
389 then the second pass of gcc is constructed, omitting some
390 non-critical libraries. Due to some weird logic in gcc's
391 configure script, <envar>CC_FOR_TARGET</envar> ends up as
392 <command>cc</command> when the host is the same as the target, but
393 different from the build system. This is why
394 <parameter>CC_FOR_TARGET=$LFS_TGT-gcc</parameter> is declared explicitly
395 as one of the configuration options.</para>
396
397 <para>Upon entering the chroot environment in <xref
398 linkend="chapter-chroot-temporary-tools"/>,
399 the temporary installations of programs needed for the proper
400 operation of the toolchain are performed. From this point onwards, the
401 core toolchain is self-contained and self-hosted. In
402 <xref linkend="chapter-building-system"/>, final versions of all the
403 packages needed for a fully functional system are built, tested, and
404 installed.</para>
405
406 </sect2>
407
408</sect1>
Note: See TracBrowser for help on using the repository browser.