source: introduction/important/building-notes.xml@ 65877d86

trunk
Last change on this file since 65877d86 was 65877d86, checked in by Xi Ruoyao <xry111@…>, 7 weeks ago

update building-notes and meson buildtype xinclude

Some packages are using debugoptimized in meson.build.

  • Property mode set to 100644
File size: 34.9 KB
Line 
1<?xml version="1.0" encoding="ISO-8859-1"?>
2<!DOCTYPE sect1 PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
3 "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
4 <!ENTITY % general-entities SYSTEM "../../general.ent">
5 %general-entities;
6]>
7
8<sect1 id="unpacking">
9 <?dbhtml filename="notes-on-building.html"?>
10
11 <sect1info>
12 <date>$Date$</date>
13 </sect1info>
14
15 <title>Notes on Building Software</title>
16
17 <para>Those people who have built an LFS system may be aware
18 of the general principles of downloading and unpacking software. Some
19 of that information is repeated here for those new to building
20 their own software.</para>
21
22 <para>Each set of installation instructions contains a URL from which you
23 can download the package. The patches; however, are stored on the LFS
24 servers and are available via HTTP. These are referenced as needed in the
25 installation instructions.</para>
26
27 <para>While you can keep the source files anywhere you like, we assume that
28 you have unpacked the package and changed into the directory created by the
29 unpacking process (the 'build' directory). We also assume you have
30 uncompressed any required patches and they are in the directory immediately
31 above the 'build' directory.</para>
32
33 <para>We can not emphasize strongly enough that you should start from a
34 <emphasis>clean source tree</emphasis> each time. This means that if
35 you have had an error during configuration or compilation, it's usually
36 best to delete the source tree and
37 re-unpack it <emphasis>before</emphasis> trying again. This obviously
38 doesn't apply if you're an advanced user used to hacking
39 <filename>Makefile</filename>s and C code, but if in doubt, start from a
40 clean tree.</para>
41
42 <sect2>
43 <title>Building Software as an Unprivileged (non-root) User</title>
44
45 <para>The golden rule of Unix System Administration is to use your
46 superpowers only when necessary. Hence, BLFS recommends that you
47 build software as an unprivileged user and only become the
48 <systemitem class='username'>root</systemitem> user when installing the
49 software. This philosophy is followed in all the packages in this book.
50 Unless otherwise specified, all instructions should be executed as an
51 unprivileged user. The book will advise you on instructions that need
52 <systemitem class='username'>root</systemitem> privileges.</para>
53
54 </sect2>
55
56 <sect2>
57 <title>Unpacking the Software</title>
58
59 <para>If a file is in <filename class='extension'>.tar</filename> format
60 and compressed, it is unpacked by running one of the following
61 commands:</para>
62
63<screen><userinput>tar -xvf filename.tar.gz
64tar -xvf filename.tgz
65tar -xvf filename.tar.Z
66tar -xvf filename.tar.bz2</userinput></screen>
67
68 <note>
69 <para>You may omit using the <option>v</option> parameter in the commands
70 shown above and below if you wish to suppress the verbose listing of all
71 the files in the archive as they are extracted. This can help speed up the
72 extraction as well as make any errors produced during the extraction
73 more obvious to you.</para>
74 </note>
75
76 <para>You can also use a slightly different method:</para>
77
78<screen><userinput>bzcat filename.tar.bz2 | tar -xv</userinput></screen>
79
80 <para>Finally, you sometimes need to be able to unpack patches which are
81 generally not in <filename class='extension'>.tar</filename> format. The
82 best way to do this is to copy the patch file to the parent of the 'build'
83 directory and then run one of the following commands depending on whether
84 the file is a <filename class='extension'>.gz</filename> or <filename
85 class='extension'>.bz2</filename> file:</para>
86
87<screen><userinput>gunzip -v patchname.gz
88bunzip2 -v patchname.bz2</userinput></screen>
89
90 </sect2>
91
92 <sect2>
93 <title>Verifying File Integrity Using 'md5sum'</title>
94
95 <para>Generally, to verify that the downloaded file is genuine and complete,
96 many package maintainers also distribute md5sums of the files. To verify the
97 md5sum of the downloaded files, download both the file and the
98 corresponding md5sum file to the same directory (preferably from different
99 on-line locations), and (assuming <filename>file.md5sum</filename> is the
100 md5sum file downloaded) run the following command:</para>
101
102<screen><userinput>md5sum -c file.md5sum</userinput></screen>
103
104 <para>If there are any errors, they will be reported. Note that the BLFS
105 book includes md5sums for all the source files also. To use the BLFS
106 supplied md5sums, you can create a <filename>file.md5sum</filename> (place
107 the md5sum data and the exact name of the downloaded file on the same
108 line of a file, separated by white space) and run the command shown above.
109 Alternately, simply run the command shown below and compare the output
110 to the md5sum data shown in the BLFS book.</para>
111
112<screen><userinput>md5sum <replaceable>&lt;name_of_downloaded_file&gt;</replaceable></userinput></screen>
113
114 </sect2>
115
116 <sect2>
117 <title>Creating Log Files During Installation</title>
118
119 <para>For larger packages, it is convenient to create log files instead of
120 staring at the screen hoping to catch a particular error or warning. Log
121 files are also useful for debugging and keeping records. The following
122 command allows you to create an installation log. Replace
123 <replaceable>&lt;command&gt;</replaceable> with the command you intend to execute.</para>
124
125<screen><userinput>( <replaceable>&lt;command&gt;</replaceable> 2&gt;&amp;1 | tee compile.log &amp;&amp; exit $PIPESTATUS )</userinput></screen>
126
127 <para><option>2&gt;&amp;1</option> redirects error messages to the same
128 location as standard output. The <command>tee</command> command allows
129 viewing of the output while logging the results to a file. The parentheses
130 around the command run the entire command in a subshell and finally the
131 <command>exit $PIPESTATUS</command> command ensures the result of the
132 <replaceable>&lt;command&gt;</replaceable> is returned as the result and not the
133 result of the <command>tee</command> command.</para>
134
135 </sect2>
136
137 <sect2 id="parallel-builds" xreflabel="Using Multiple Processors">
138 <title>Using Multiple Processors</title>
139
140 <para>For many modern systems with multiple processors (or cores) the
141 compilation time for a package can be reduced by performing a "parallel
142 make" by either setting an environment variable or telling the make program
143 how many processors are available. For instance, a Core2Duo can support two
144 simultaneous processes with: </para>
145
146 <screen><userinput>export MAKEFLAGS='-j2'</userinput></screen>
147
148 <para>or just building with:</para>
149
150 <screen><userinput>make -j2</userinput></screen>
151
152 <para>Generally the number of processes should not exceed the number of
153 cores supported by the CPU. To list the processors on your
154 system, issue: <userinput>grep processor /proc/cpuinfo</userinput>.
155 </para>
156
157 <para>In some cases, using multiple processors may result in a 'race'
158 condition where the success of the build depends on the order of the
159 commands run by the <command>make</command> program. For instance, if an
160 executable needs File A and File B, attempting to link the program before
161 one of the dependent components is available will result in a failure.
162 This condition usually arises because the upstream developer has not
163 properly designated all the prerequisites needed to accomplish a step in the
164 Makefile.</para>
165
166 <para>If this occurs, the best way to proceed is to drop back to a
167 single processor build. Adding '-j1' to a make command will override
168 the similar setting in the MAKEFLAGS environment variable.</para>
169
170 <note><para>When running the package tests or the install portion of the
171 package build process, we do not recommend using an option greater than
172 '-j1' unless specified otherwise. The installation procedures or checks
173 have not been validated using parallel procedures and may fail with issues
174 that are difficult to debug.</para></note>
175
176 </sect2>
177
178 <sect2 id="automating-builds" xreflabel="Automated Building Procedures">
179 <title>Automated Building Procedures</title>
180
181 <para>There are times when automating the building of a package can come in
182 handy. Everyone has their own reasons for wanting to automate building,
183 and everyone goes about it in their own way. Creating
184 <filename>Makefile</filename>s, <application>Bash</application> scripts,
185 <application>Perl</application> scripts or simply a list of commands used
186 to cut and paste are just some of the methods you can use to automate
187 building BLFS packages. Detailing how and providing examples of the many
188 ways you can automate the building of packages is beyond the scope of this
189 section. This section will expose you to using file redirection and the
190 <command>yes</command> command to help provide ideas on how to automate
191 your builds.</para>
192
193 <bridgehead renderas="sect3">File Redirection to Automate Input</bridgehead>
194
195 <para>You will find times throughout your BLFS journey when you will come
196 across a package that has a command prompting you for information. This
197 information might be configuration details, a directory path, or a response
198 to a license agreement. This can present a challenge to automate the
199 building of that package. Occasionally, you will be prompted for different
200 information in a series of questions. One method to automate this type of
201 scenario requires putting the desired responses in a file and using
202 redirection so that the program uses the data in the file as the answers to
203 the questions.</para>
204
205 <para>Building the <application>CUPS</application> package is a good
206 example of how redirecting a file as input to prompts can help you automate
207 the build. If you run the test suite, you are asked to respond to a series
208 of questions regarding the type of test to run and if you have any
209 auxiliary programs the test can use. You can create a file with your
210 responses, one response per line, and use a command similar to the
211 one shown below to automate running the test suite:</para>
212
213<screen><userinput>make check &lt; ../cups-1.1.23-testsuite_parms</userinput></screen>
214
215 <para>This effectively makes the test suite use the responses in the file
216 as the input to the questions. Occasionally you may end up doing a bit of
217 trial and error determining the exact format of your input file for some
218 things, but once figured out and documented you can use this to automate
219 building the package.</para>
220
221 <bridgehead renderas="sect3">Using <command>yes</command> to Automate
222 Input</bridgehead>
223
224 <para>Sometimes you will only need to provide one response, or provide the
225 same response to many prompts. For these instances, the
226 <command>yes</command> command works really well. The
227 <command>yes</command> command can be used to provide a response (the same
228 one) to one or more instances of questions. It can be used to simulate
229 pressing just the <keycap>Enter</keycap> key, entering the
230 <keycap>Y</keycap> key or entering a string of text. Perhaps the easiest
231 way to show its use is in an example.</para>
232
233 <para>First, create a short <application>Bash</application> script by
234 entering the following commands:</para>
235
236<screen><userinput>cat &gt; blfs-yes-test1 &lt;&lt; "EOF"
237<literal>#!/bin/bash
238
239echo -n -e "\n\nPlease type something (or nothing) and press Enter ---> "
240
241read A_STRING
242
243if test "$A_STRING" = ""; then A_STRING="Just the Enter key was pressed"
244else A_STRING="You entered '$A_STRING'"
245fi
246
247echo -e "\n\n$A_STRING\n\n"</literal>
248EOF
249chmod 755 blfs-yes-test1</userinput></screen>
250
251 <para>Now run the script by issuing <command>./blfs-yes-test1</command> from
252 the command line. It will wait for a response, which can be anything (or
253 nothing) followed by the <keycap>Enter</keycap> key. After entering
254 something, the result will be echoed to the screen. Now use the
255 <command>yes</command> command to automate the entering of a
256 response:</para>
257
258<screen><userinput>yes | ./blfs-yes-test1</userinput></screen>
259
260 <para>Notice that piping <command>yes</command> by itself to the script
261 results in <keycap>y</keycap> being passed to the script. Now try it with a
262 string of text:</para>
263
264<screen><userinput>yes 'This is some text' | ./blfs-yes-test1</userinput></screen>
265
266 <para>The exact string was used as the response to the script. Finally,
267 try it using an empty (null) string:</para>
268
269<screen><userinput>yes '' | ./blfs-yes-test1</userinput></screen>
270
271 <para>Notice this results in passing just the press of the
272 <keycap>Enter</keycap> key to the script. This is useful for times when the
273 default answer to the prompt is sufficient. This syntax is used in the
274 <xref linkend="net-tools-automate-example"/> instructions to accept all the
275 defaults to the many prompts during the configuration step. You may now
276 remove the test script, if desired.</para>
277
278 <bridgehead renderas="sect3">File Redirection to Automate Output</bridgehead>
279
280 <para>In order to automate the building of some packages, especially those
281 that require you to read a license agreement one page at a time, requires
282 using a method that avoids having to press a key to display each page.
283 Redirecting the output to a file can be used in these instances to assist
284 with the automation. The previous section on this page touched on creating
285 log files of the build output. The redirection method shown there used the
286 <command>tee</command> command to redirect output to a file while also
287 displaying the output to the screen. Here, the output will only be sent to
288 a file.</para>
289
290 <para>Again, the easiest way to demonstrate the technique is to show an
291 example. First, issue the command:</para>
292
293<screen><userinput>ls -l /usr/bin | more</userinput></screen>
294
295 <para>Of course, you'll be required to view the output one page at a time
296 because the <command>more</command> filter was used. Now try the same
297 command, but this time redirect the output to a file. The special file
298 <filename>/dev/null</filename> can be used instead of the filename shown,
299 but you will have no log file to examine:</para>
300
301<screen><userinput>ls -l /usr/bin | more &gt; redirect_test.log 2&gt;&amp;1</userinput></screen>
302
303 <para>Notice that this time the command immediately returned to the shell
304 prompt without having to page through the output. You may now remove the
305 log file.</para>
306
307 <para>The last example will use the <command>yes</command> command in
308 combination with output redirection to bypass having to page through the
309 output and then provide a <keycap>y</keycap> to a prompt. This technique
310 could be used in instances when otherwise you would have to page through
311 the output of a file (such as a license agreement) and then answer the
312 question of <quote>do you accept the above?</quote>. For this example,
313 another short <application>Bash</application> script is required:</para>
314
315<screen><userinput>cat &gt; blfs-yes-test2 &lt;&lt; "EOF"
316<literal>#!/bin/bash
317
318ls -l /usr/bin | more
319
320echo -n -e "\n\nDid you enjoy reading this? (y,n) "
321
322read A_STRING
323
324if test "$A_STRING" = "y"; then A_STRING="You entered the 'y' key"
325else A_STRING="You did NOT enter the 'y' key"
326fi
327
328echo -e "\n\n$A_STRING\n\n"</literal>
329EOF
330chmod 755 blfs-yes-test2</userinput></screen>
331
332 <para>This script can be used to simulate a program that requires you to
333 read a license agreement, then respond appropriately to accept the
334 agreement before the program will install anything. First, run the script
335 without any automation techniques by issuing
336 <command>./blfs-yes-test2</command>.</para>
337
338 <para>Now issue the following command which uses two automation techniques,
339 making it suitable for use in an automated build script:</para>
340
341<screen><userinput>yes | ./blfs-yes-test2 &gt; blfs-yes-test2.log 2&gt;&amp;1</userinput></screen>
342
343 <para>If desired, issue <command>tail blfs-yes-test2.log</command> to see
344 the end of the paged output, and confirmation that <keycap>y</keycap> was
345 passed through to the script. Once satisfied that it works as it should,
346 you may remove the script and log file.</para>
347
348 <para>Finally, keep in mind that there are many ways to automate and/or
349 script the build commands. There is not a single <quote>correct</quote> way
350 to do it. Your imagination is the only limit.</para>
351
352 </sect2>
353
354 <sect2>
355 <title>Dependencies</title>
356
357 <para>For each package described, BLFS lists the known dependencies.
358 These are listed under several headings, whose meaning is as follows:</para>
359
360 <itemizedlist>
361 <listitem>
362 <para><emphasis>Required</emphasis> means that the target package
363 cannot be correctly built without the dependency having first been
364 installed.</para>
365 </listitem>
366 <listitem>
367 <para><emphasis>Recommended</emphasis> means that BLFS strongly
368 suggests this package is installed first for a clean and trouble-free
369 build, that won't have issues either during the build process, or at
370 run-time. The instructions in the book assume these packages are
371 installed. Some changes or workarounds may be required if these
372 packages are not installed.</para>
373 </listitem>
374 <listitem>
375 <para><emphasis>Optional</emphasis> means that this package might be
376 installed for added functionality. Often BLFS will describe the
377 dependency to explain the added functionality that will result.</para>
378 </listitem>
379 </itemizedlist>
380
381 </sect2>
382
383 <sect2 id="package_updates">
384 <title>Using the Most Current Package Sources</title>
385
386 <para>On occasion you may run into a situation in the book when a package
387 will not build or work properly. Though the Editors attempt to ensure
388 that every package in the book builds and works properly, sometimes a
389 package has been overlooked or was not tested with this particular version
390 of BLFS.</para>
391
392 <para>If you discover that a package will not build or work properly, you
393 should see if there is a more current version of the package. Typically
394 this means you go to the maintainer's web site and download the most current
395 tarball and attempt to build the package. If you cannot determine the
396 maintainer's web site by looking at the download URLs, use Google and query
397 the package's name. For example, in the Google search bar type:
398 'package_name download' (omit the quotes) or something similar. Sometimes
399 typing: 'package_name home page' will result in you finding the
400 maintainer's web site.</para>
401
402 </sect2>
403
404 <sect2 id="stripping">
405 <title>Stripping One More Time</title>
406
407 <para>
408 In LFS, stripping of debugging symbols was discussed a couple of
409 times. When building BLFS packages, there are generally no special
410 instructions that discuss stripping again. It is probably not a good
411 idea to strip an executable or a library while it is in use, so exiting
412 any windowing environment is a good idea. Then you can do:
413 </para>
414
415<screen><userinput>find /{,usr/}{bin,lib,sbin} \
416 -type f \( -name \*.so* -a ! -name \*dbg \) \
417 -exec strip --strip-unneeded {} \;</userinput></screen>
418
419 <para>
420 If you install programs in other directories such as <filename
421 class="directory">/opt</filename> or <filename
422 class="directory">/usr/local</filename>, you may want to strip the files
423 there too.
424 </para>
425
426 <para>
427 For more information on stripping, see <ulink
428 url="http://www.technovelty.org/linux/stripping-shared-libraries.html"/>.
429 </para>
430
431 </sect2>
432<!--
433 <sect2 id="libtool">
434 <title>Libtool files</title>
435
436 <para>
437 One of the side effects of packages that use Autotools, including
438 libtool, is that they create many files with an .la extension. These
439 files are not needed in an LFS environment. If there are conflicts with
440 pkgconfig entries, they can actually prevent successful builds. You
441 may want to consider removing these files periodically:
442 </para>
443
444<screen><userinput>find /lib /usr/lib -not -path "*Image*" -a -name \*.la -delete</userinput></screen>
445
446 <para>
447 The above command removes all .la files with the exception of those that
448 have <quote>Image</quote> or <quote>openldap</quote> as a part of the
449 path. These .la files are used by the ImageMagick and openldap programs,
450 respectively. There may be other exceptions by packages not in BLFS.
451 </para>
452
453 </sect2>
454-->
455 <sect2 id="buildsystems">
456 <title>Working with different build systems</title>
457
458 <para>
459 There are now three different build systems in common use for
460 converting C or C++ source code into compiled programs or
461 libraries and their details (particularly, finding out about available
462 options and their default values) differ. It may be easiest to understand
463 the issues caused by some choices (typically slow execution or
464 unexpected use of, or omission of, optimizatons) by starting with
465 the CFLAGS and CXXFLAGS environment variables. There are also some
466 programs which use rust.
467 </para>
468
469 <para>
470 Most LFS and BLFS builders are probably aware of the basics of CFLAGS
471 and CXXFLAGS for altering how a program is compiled. Typically, some
472 form of optimization is used by upstream developers (-O2 or -O3),
473 sometimes with the creation of debug symbols (-g), as defaults.
474 </para>
475
476 <para>
477 If there are contradictory flags (e.g. multiple different -O values),
478 the <emphasis>last</emphasis> value will be used. Sometimes this means
479 that flags specified in environment variables will be picked up before
480 values hardcoded in the Makefile, and therefore ignored. For example,
481 where a user specifies '-O2' and that is followed by '-O3' the build will
482 use '-O3'.
483 </para>
484
485 <para>
486 There are various other things which can be passed in CFLAGS or
487 CXXFLAGS, such as forcing compilation for a specific microarchitecture
488 (e.g. -march=amdfam10, -march=native) or specifying a specific standard
489 for C or C++ (-std=c++17 for example). But one thing which has now come
490 to light is that programmers might include debug assertions in their
491 code, expecting them to be disabled in releases by using -DNDEBUG.
492 Specifically, if <xref linkend="mesa"/> is built with these assertions
493 enabled, some activities such as loading levels of games can take
494 extremely long times, even on high-class video cards.
495 </para>
496
497 <bridgehead renderas="sect3" id="autotools-info">Autotools with Make</bridgehead>
498
499 <para>
500 This combination is often described as 'CMMI' (configure, make, make
501 install) and is used here to also cover the few packages which have a
502 configure script that is not generated by autotools.
503 </para>
504
505 <para>
506 Sometimes running <command>./configure --help</command> will produce
507 useful options about switches which might be used. At other times,
508 after looking at the output from configure you may need to look
509 at the details of the script to find out what it was actually searching
510 for.
511 </para>
512
513 <para>
514 Many configure scripts will pick up any CFLAGS or CXXFLAGS from the
515 environment, but CMMI packages vary about how these will be mixed with
516 any flags which would otherwise be used (<emphasis>variously</emphasis>:
517 ignored, used to replace the programmer's suggestion, used before the
518 programmer's suggestion, or used after the programmer's suggestion).
519 </para>
520
521 <para>
522 In most CMMI packages, running 'make' will list each command and run
523 it, interspersed with any warnings. But some packages try to be 'silent'
524 and only show which file they are compiling or linking instead of showing
525 the command line. If you need to inspect the command, either because of
526 an error, or just to see what options and flags are being used, adding
527 'V=1' to the make invocation may help.
528 </para>
529
530 <bridgehead renderas="sect3" id="cmake-info">CMake</bridgehead>
531
532 <para>
533 CMake works in a very different way, and it has two backends which can
534 be used on BLFS: 'make' and 'ninja'. The default backend is make, but
535 ninja can be faster on large packages with multiple processors. To
536 use ninja, specify '-G Ninja' in the cmake command. However, there are
537 some packages which create fatal errors in their ninja files but build
538 successfully using the default of Unix Makefiles.
539 </para>
540
541 <para>
542 The hardest part of using CMake is knowing what options you might wish
543 to specify. The only way to get a list of what the package knows about
544 is to run <command>cmake -LAH</command> and look at the output for that
545 default configuration.
546 </para>
547
548 <para>
549 Perhaps the most-important thing about CMake is that it has a variety
550 of CMAKE_BUILD_TYPE values, and these affect the flags. The default
551 is that this is not set and no flags are generated. Any CFLAGS or
552 CXXFLAGS in the environment will be used. If the programmer has coded
553 any debug assertions, those will be enabled unless -DNDEBUG is used.
554 The following CMAKE_BUILD_TYPE values will generate the flags shown,
555 and these will come <emphasis>after</emphasis> any flags in the
556 environment and therefore take precedence.
557 </para>
558
559 <itemizedlist>
560 <listitem>
561 <para>Debug : '-g'</para>
562 </listitem>
563 <listitem>
564 <para>Release : '-O3 -DNDEBUG'</para>
565 </listitem>
566 <listitem>
567 <para>RelWithDebInfo : '-O2 -g -DNDEBUG'</para>
568 </listitem>
569 <listitem>
570 <para>MinSizeRel : '-Os -DNDEBUG'</para>
571 </listitem>
572 </itemizedlist>
573
574 <para>
575 CMake tries to produce quiet builds. To see the details of the commands
576 which are being run, use 'make VERBOSE=1' or 'ninja -v'.
577 </para>
578
579 <bridgehead renderas="sect3" id="meson-info">Meson</bridgehead>
580
581 <para>
582 Meson has some similarities to CMake, but many differences. To get
583 details of the defines that you may wish to change you can look at
584 <filename>meson_options.txt</filename> which is usually in the
585 top-level directory.
586 </para>
587
588 <para>
589 If you have already configured the package by running
590 <command>meson</command> and now wish to change one or more settings,
591 you can either remove the build directory, recreate it, and use the
592 altered options, or within the build directory run <command>meson
593 configure</command>, e.g. to set an option:
594 </para>
595
596<screen><userinput>meson configure -D&lt;some_option&gt;=true</userinput></screen>
597
598 <para>
599 If you do that, the file <filename>meson-private/cmd_line.txt</filename>
600 will show the <emphasis>last</emphasis> commands which were used.
601 </para>
602
603 <para>
604 Meson provides the following buildtype values, and the flags they enable
605 come <emphasis>after</emphasis> any flags supplied in the environment and
606 therefore take precedence.
607 </para>
608
609 <itemizedlist>
610 <listitem>
611 <para>plain : no added flags. This is for distributors to supply their
612 own CLFAGS, CXXFLAGS and LDFLAGS. There is no obvious reason to use
613 this in BLFS.</para>
614 </listitem>
615 <listitem>
616 <para>debug : '-g' - this is the default if nothing is specified
617 in neither <filename>meson.build</filename> nor the command line.
618 However it results large and slow binaries, so we should override
619 it in BLFS.</para>
620 </listitem>
621 <listitem>
622 <para>debugoptimized : '-O2 -g' : this is the default specified in
623 <filename>meson.build</filename> of some packages.</para>
624 </listitem>
625 <listitem>
626 <para>release : '-O3 -DNDEBUG' (but occasionally a package will force
627 -O2 here)</para>
628 </listitem>
629 </itemizedlist>
630
631 <para>
632 Although the 'release' buildtype is described as enabling -DNDEBUG, and all
633 CMake Release builds pass that, it has so far only been observed (in
634 verbose builds) for <xref linkend="mesa"/>. That suggests that it might
635 only be used when there are debug assertions present.
636 </para>
637
638 <para>
639 The -DNDEBUG flag can also be provided by passing
640 <command>-Db_ndebug=true</command>.
641 </para>
642
643 <para>
644 To see the details of the commands which are being run in a package using
645 meson, use 'ninja -v'.
646 </para>
647
648 <bridgehead renderas="sect3" id="rust-info">Rustc and Cargo</bridgehead>
649
650 <para>
651 Most released rustc programs are provided as crates (source tarballs)
652 which will query a server to check current versions of dependencies
653 and then download them as necessary. These packages are built using
654 <command>cargo --release</command>. In theory, you can manipulate the
655 RUSTFLAGS to change the optimize-level (default is 3, like -O3, e.g.
656 <literal>-Copt-level=3</literal>) or to force it to build for the
657 machine it is being compiled on, using
658 <literal>-Ctarget-cpu=native</literal> but in practice this seems to
659 make no significant difference.
660 </para>
661
662 <para>
663 If you find an interesting rustc program which is only provided as
664 unpackaged source, you should at least specify
665 <literal>RUSTFLAGS=-Copt-level=2</literal> otherwise it will do an
666 unoptimized compile with debug info and run <emphasis>much</emphasis>
667 slower.
668 </para>
669
670 <para>
671 The rust developers seem to assume that everyone will compile on a
672 machine dedicated to producing builds, so by default all CPUs are used.
673 This can often be worked around, either by exporting
674 CARGO_BUILD_JOBS=&lt;N&gt; or passing --jobs &lt;N&gt; to cargo. For
675 compiling rustc itself, specifying --jobs &lt;N&gt; on invocations of
676 x.py (together with the <envar>CARGO_BUILD_JOBS</envar> environment
677 variable, which looks like a "belt and braces" approach but seems to be
678 necessary) mostly works. The exception is running the tests when building
679 rustc, some of them will nevertheless use all online CPUs, at least as of
680 rustc-1.42.0.
681 </para>
682
683 </sect2>
684
685 <sect2 id="optimizations">
686 <title>Optimizing the build</title>
687
688 <para>
689 Many people will prefer to optimize compiles as they see fit, by providing
690 CFLAGS or CXXFLAGS. For an introduction to the options available with gcc
691 and g++ see <ulink
692 url="https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html"/> and <ulink
693 url="https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html"/>
694 and <command>info gcc</command>.
695
696 </para>
697
698 <para>
699 Some packages default to '-O2 -g', others to '-O3 -g', and if CFLAGS or
700 CXXFLAGS are supplied they might be added to the package's defaults,
701 replace the package's defaults, or even be ignored. There are details
702 on some desktop packages which were mostly current in April 2019 at
703 <ulink url="https://www.linuxfromscratch.org/~ken/tuning/"/> - in
704 particular, README.txt, tuning-1-packages-and-notes.txt, and
705 tuning-notes-2B.txt. The particular thing to remember is that if you
706 want to try some of the more interesting flags you may need to force
707 verbose builds to confirm what is being used.
708 </para>
709
710 <para>
711 Clearly, if you are optimizing your own program you can spend time to
712 profile it and perhaps recode some of it if it is too slow. But for
713 building a whole system that approach is impractical. In general,
714 -O3 usually produces faster programs than -O2. Specifying
715 -march=native is also beneficial, but means that you cannot move the
716 binaries to an incompatible machine - this can also apply to newer
717 machines, not just to older machines. For example programs compiled for
718 'amdfam10' run on old Phenoms, Kaveris, and Ryzens : but programs
719 compiled for a Kaveri will not run on a Ryzen because certain op-codes
720 are not present. Similarly, if you build for a Haswell not everything
721 will run on a SandyBridge.
722 </para>
723
724 <para>
725 There are also various other options which some people claim are
726 beneficial. At worst, you get to recompile and test, and then
727 discover that in your usage the options do not provide a benefit.
728 </para>
729
730 <para>
731 If building Perl or Python modules, or Qt packages which use qmake,
732 in general the CFLAGS and CXXFLAGS used are those which were used by
733 those 'parent' packages.
734 </para>
735
736 </sect2>
737
738 <sect2 id="hardening">
739 <title>Options for hardening the build</title>
740
741 <para>
742 Even on desktop systems, there are still a lot of exploitable
743 vulnerabilities. For many of these, the attack comes via javascript
744 in a browser. Often, a series of vulnerabilities are used to gain
745 access to data (or sometimes to pwn, i.e. own, the machine and
746 install rootkits). Most commercial distros will apply various
747 hardening measures.
748 </para>
749
750 <para>
751 For hardening options which are reasonably cheap, there is some
752 discussion in the 'tuning' link above (occasionally, one or more
753 of these options might be inappropriate for a package). These
754 options are -D_FORTIFY_SOURCE=2, -fstack-protector=strong, and
755 (for C++) -D_GLIBCXX_ASSERTIONS. On modern machines these should
756 only have a little impact on how fast things run, and often they
757 will not be noticeable.
758 </para>
759
760 <para>
761 In the past, there was Hardened LFS where gcc (a much older version)
762 was forced to use hardening (with options to turn some of it off on a
763 per-package basis. What is being covered here is different - first you
764 have to make sure that the package is indeed using your added flags and
765 not over-riding them.
766 </para>
767
768 <para>
769 The main distros use much more, such as RELRO (Relocation Read Only)
770 and perhaps -fstack-clash-protection. You may also encounter the
771 so-called 'userspace retpoline' (-mindirect-branch=thunk etc.) which
772 is the equivalent of the spectre mitigations applied to the linux
773 kernel in late 2018). The kernel mitigations caused a lot of complaints
774 about lost performance, if you have a production server you might wish
775 to consider testing that, along with the other available options, to
776 see if performance is still sufficient.
777 </para>
778
779 <para>
780 Whilst gcc has many hardening options, clang/LLVM's strengths lie
781 elsewhere. Some options which gcc provides are said to be less effective
782 in clang/LLVM.
783 </para>
784
785 </sect2>
786
787</sect1>
Note: See TracBrowser for help on using the repository browser.