source: introduction/important/building-notes.xml@ 6333296

11.0 11.1 11.2 11.3 12.0 kea ken/inkscape-core-mods lazarus lxqt plabs/python-mods qt5new trunk upgradedb xry111/intltool xry111/soup3 xry111/test-20220226 xry111/xf86-video-removal
Last change on this file since 6333296 was 6333296, checked in by Xi Ruoyao <xry111@…>, 2 years ago

building notes: stop to use MD5 for genuity verification, ...

use SHA256 or GnuPG instead.

  • Property mode set to 100644
File size: 37.7 KB
1<?xml version="1.0" encoding="ISO-8859-1"?>
2<!DOCTYPE sect1 PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
3 "" [
4 <!ENTITY % general-entities SYSTEM "../../general.ent">
5 %general-entities;
8<sect1 id="unpacking">
9 <?dbhtml filename="notes-on-building.html"?>
11 <sect1info>
12 <date>$Date$</date>
13 </sect1info>
15 <title>Notes on Building Software</title>
17 <para>Those people who have built an LFS system may be aware
18 of the general principles of downloading and unpacking software. Some
19 of that information is repeated here for those new to building
20 their own software.</para>
22 <para>Each set of installation instructions contains a URL from which you
23 can download the package. The patches; however, are stored on the LFS
24 servers and are available via HTTP. These are referenced as needed in the
25 installation instructions.</para>
27 <para>While you can keep the source files anywhere you like, we assume that
28 you have unpacked the package and changed into the directory created by the
29 unpacking process (the 'build' directory). We also assume you have
30 uncompressed any required patches and they are in the directory immediately
31 above the 'build' directory.</para>
33 <para>We can not emphasize strongly enough that you should start from a
34 <emphasis>clean source tree</emphasis> each time. This means that if
35 you have had an error during configuration or compilation, it's usually
36 best to delete the source tree and
37 re-unpack it <emphasis>before</emphasis> trying again. This obviously
38 doesn't apply if you're an advanced user used to hacking
39 <filename>Makefile</filename>s and C code, but if in doubt, start from a
40 clean tree.</para>
42 <sect2>
43 <title>Building Software as an Unprivileged (non-root) User</title>
45 <para>The golden rule of Unix System Administration is to use your
46 superpowers only when necessary. Hence, BLFS recommends that you
47 build software as an unprivileged user and only become the
48 <systemitem class='username'>root</systemitem> user when installing the
49 software. This philosophy is followed in all the packages in this book.
50 Unless otherwise specified, all instructions should be executed as an
51 unprivileged user. The book will advise you on instructions that need
52 <systemitem class='username'>root</systemitem> privileges.</para>
54 </sect2>
56 <sect2>
57 <title>Unpacking the Software</title>
59 <para>If a file is in <filename class='extension'>.tar</filename> format
60 and compressed, it is unpacked by running one of the following
61 commands:</para>
63<screen><userinput>tar -xvf filename.tar.gz
64tar -xvf filename.tgz
65tar -xvf filename.tar.Z
66tar -xvf filename.tar.bz2</userinput></screen>
68 <note>
69 <para>You may omit using the <option>v</option> parameter in the commands
70 shown above and below if you wish to suppress the verbose listing of all
71 the files in the archive as they are extracted. This can help speed up the
72 extraction as well as make any errors produced during the extraction
73 more obvious to you.</para>
74 </note>
76 <para>You can also use a slightly different method:</para>
78<screen><userinput>bzcat filename.tar.bz2 | tar -xv</userinput></screen>
80 <para>Finally, you sometimes need to be able to unpack patches which are
81 generally not in <filename class='extension'>.tar</filename> format. The
82 best way to do this is to copy the patch file to the parent of the 'build'
83 directory and then run one of the following commands depending on whether
84 the file is a <filename class='extension'>.gz</filename> or <filename
85 class='extension'>.bz2</filename> file:</para>
87<screen><userinput>gunzip -v patchname.gz
88bunzip2 -v patchname.bz2</userinput></screen>
90 </sect2>
92 <sect2>
93 <title>Verifying File Integrity</title>
95 <para>Generally, to verify that the downloaded file is complete,
96 many package maintainers also distribute md5sums of the files. To verify the
97 md5sum of the downloaded files, download both the file and the
98 corresponding md5sum file to the same directory (preferably from different
99 on-line locations), and (assuming <filename>file.md5sum</filename> is the
100 md5sum file downloaded) run the following command:</para>
102<screen><userinput>md5sum -c file.md5sum</userinput></screen>
104 <para>If there are any errors, they will be reported. Note that the BLFS
105 book includes md5sums for all the source files also. To use the BLFS
106 supplied md5sums, you can create a <filename>file.md5sum</filename> (place
107 the md5sum data and the exact name of the downloaded file on the same
108 line of a file, separated by white space) and run the command shown above.
109 Alternately, simply run the command shown below and compare the output
110 to the md5sum data shown in the BLFS book.</para>
112<screen><userinput>md5sum <replaceable>&lt;name_of_downloaded_file&gt;</replaceable></userinput></screen>
114 <para>MD5 is not cryptographically secure, so the md5sums are only
115 provided for detecting random errors or truncations introduced during
116 network transfer. There is no <quote>100%</quote> secure way to make
117 sure the genuity of the source files. Assuming the upstream is managing
118 their website correctly (the private key is not leaked and the domain is
119 not hijacked), and the trust anchors have been set up correctly using
120 <xref linkend="make-ca"/> on the BLFS system, we can reasonably trust
121 download URLs to the upstream official website
122 <emphasis role="bold">with https protocol</emphasis>. Note that
123 BLFS book itself is published on a website with https, so you should
124 already have some confidence in https protocol or you wouldn't trust the
125 book content.</para>
127 <para>If the package is downloaded from an unofficial location (for
128 example a local mirror), checksums generated by cryptographically secure
129 digest algorithms (for example SHA256) can be used to verify the
130 genuity of the package. Download the checksum file from the upstream
131 <emphasis role="bold">official</emphasis> website (or somewhere
132 <emphasis role="bold">you can trust</emphasis>) and compare the
133 checksum of the package from unoffical location with it. For example,
134 SHA256 checksum can be checked with the command:</para>
136 <note>
137 <para>If the checksum and the package are downloaded from the same
138 untrusted location, you won't gain security enhancement by verifying
139 the package with the checksum. The attacker can fake the checksum as
140 well as compromising the package itself.</para>
141 </note>
143<screen><userinput>sha256sum -c <replaceable>file</replaceable>.sha256sum</userinput></screen>
145 <para>If <xref linkend="gnupg2"/> is installed, you can also verify the
146 genuity of the package with a GPG signature. Import the upstream GPG
147 public key with:</para>
149<screen><userinput>gpg --recv-key <replaceable>keyID</replaceable></userinput></screen>
151 <para><replaceable>keyID</replaceable> should be replaced with the key ID
152 from somewhere <emphasis role="bold">you can trust</emphasis> (for
153 example, copy it from the upstream official website using https). Now
154 you can verify the signature with:</para>
156<screen><userinput>gpg --recv-key <replaceable>file</replaceable>.sig <replaceable>file</replaceable></userinput></screen>
158 <para>The advantage of <application>GnuPG</application> signature is,
159 once you imported a public key which can be trusted, you can download
160 both the package and its signature from the same unofficial location and
161 verify them with the public key. So you won't need to connect to the
162 official upstream website to retrieve a checksum for each new release.
163 You only need to update the public key if it's expired or revoked.
164 </para>
166 </sect2>
168 <sect2>
169 <title>Creating Log Files During Installation</title>
171 <para>For larger packages, it is convenient to create log files instead of
172 staring at the screen hoping to catch a particular error or warning. Log
173 files are also useful for debugging and keeping records. The following
174 command allows you to create an installation log. Replace
175 <replaceable>&lt;command&gt;</replaceable> with the command you intend to execute.</para>
177<screen><userinput>( <replaceable>&lt;command&gt;</replaceable> 2&gt;&amp;1 | tee compile.log &amp;&amp; exit $PIPESTATUS )</userinput></screen>
179 <para><option>2&gt;&amp;1</option> redirects error messages to the same
180 location as standard output. The <command>tee</command> command allows
181 viewing of the output while logging the results to a file. The parentheses
182 around the command run the entire command in a subshell and finally the
183 <command>exit $PIPESTATUS</command> command ensures the result of the
184 <replaceable>&lt;command&gt;</replaceable> is returned as the result and not the
185 result of the <command>tee</command> command.</para>
187 </sect2>
189 <sect2 id="parallel-builds" xreflabel="Using Multiple Processors">
190 <title>Using Multiple Processors</title>
192 <para>For many modern systems with multiple processors (or cores) the
193 compilation time for a package can be reduced by performing a "parallel
194 make" by either setting an environment variable or telling the make program
195 how many processors are available. For instance, a Core2Duo can support two
196 simultaneous processes with: </para>
198 <screen><userinput>export MAKEFLAGS='-j2'</userinput></screen>
200 <para>or just building with:</para>
202 <screen><userinput>make -j2</userinput></screen>
204 <para>Generally the number of processes should not exceed the number of
205 cores supported by the CPU. To list the processors on your
206 system, issue: <userinput>grep processor /proc/cpuinfo</userinput>.
207 </para>
209 <para>In some cases, using multiple processors may result in a 'race'
210 condition where the success of the build depends on the order of the
211 commands run by the <command>make</command> program. For instance, if an
212 executable needs File A and File B, attempting to link the program before
213 one of the dependent components is available will result in a failure.
214 This condition usually arises because the upstream developer has not
215 properly designated all the prerequisites needed to accomplish a step in the
216 Makefile.</para>
218 <para>If this occurs, the best way to proceed is to drop back to a
219 single processor build. Adding '-j1' to a make command will override
220 the similar setting in the MAKEFLAGS environment variable.</para>
222 <note><para>When running the package tests or the install portion of the
223 package build process, we do not recommend using an option greater than
224 '-j1' unless specified otherwise. The installation procedures or checks
225 have not been validated using parallel procedures and may fail with issues
226 that are difficult to debug.</para></note>
228 </sect2>
230 <sect2 id="automating-builds" xreflabel="Automated Building Procedures">
231 <title>Automated Building Procedures</title>
233 <para>There are times when automating the building of a package can come in
234 handy. Everyone has their own reasons for wanting to automate building,
235 and everyone goes about it in their own way. Creating
236 <filename>Makefile</filename>s, <application>Bash</application> scripts,
237 <application>Perl</application> scripts or simply a list of commands used
238 to cut and paste are just some of the methods you can use to automate
239 building BLFS packages. Detailing how and providing examples of the many
240 ways you can automate the building of packages is beyond the scope of this
241 section. This section will expose you to using file redirection and the
242 <command>yes</command> command to help provide ideas on how to automate
243 your builds.</para>
245 <bridgehead renderas="sect3">File Redirection to Automate Input</bridgehead>
247 <para>You will find times throughout your BLFS journey when you will come
248 across a package that has a command prompting you for information. This
249 information might be configuration details, a directory path, or a response
250 to a license agreement. This can present a challenge to automate the
251 building of that package. Occasionally, you will be prompted for different
252 information in a series of questions. One method to automate this type of
253 scenario requires putting the desired responses in a file and using
254 redirection so that the program uses the data in the file as the answers to
255 the questions.</para>
257 <para>Building the <application>CUPS</application> package is a good
258 example of how redirecting a file as input to prompts can help you automate
259 the build. If you run the test suite, you are asked to respond to a series
260 of questions regarding the type of test to run and if you have any
261 auxiliary programs the test can use. You can create a file with your
262 responses, one response per line, and use a command similar to the
263 one shown below to automate running the test suite:</para>
265<screen><userinput>make check &lt; ../cups-1.1.23-testsuite_parms</userinput></screen>
267 <para>This effectively makes the test suite use the responses in the file
268 as the input to the questions. Occasionally you may end up doing a bit of
269 trial and error determining the exact format of your input file for some
270 things, but once figured out and documented you can use this to automate
271 building the package.</para>
273 <bridgehead renderas="sect3">Using <command>yes</command> to Automate
274 Input</bridgehead>
276 <para>Sometimes you will only need to provide one response, or provide the
277 same response to many prompts. For these instances, the
278 <command>yes</command> command works really well. The
279 <command>yes</command> command can be used to provide a response (the same
280 one) to one or more instances of questions. It can be used to simulate
281 pressing just the <keycap>Enter</keycap> key, entering the
282 <keycap>Y</keycap> key or entering a string of text. Perhaps the easiest
283 way to show its use is in an example.</para>
285 <para>First, create a short <application>Bash</application> script by
286 entering the following commands:</para>
288<screen><userinput>cat &gt; blfs-yes-test1 &lt;&lt; "EOF"
291echo -n -e "\n\nPlease type something (or nothing) and press Enter ---> "
293read A_STRING
295if test "$A_STRING" = ""; then A_STRING="Just the Enter key was pressed"
296else A_STRING="You entered '$A_STRING'"
299echo -e "\n\n$A_STRING\n\n"</literal>
301chmod 755 blfs-yes-test1</userinput></screen>
303 <para>Now run the script by issuing <command>./blfs-yes-test1</command> from
304 the command line. It will wait for a response, which can be anything (or
305 nothing) followed by the <keycap>Enter</keycap> key. After entering
306 something, the result will be echoed to the screen. Now use the
307 <command>yes</command> command to automate the entering of a
308 response:</para>
310<screen><userinput>yes | ./blfs-yes-test1</userinput></screen>
312 <para>Notice that piping <command>yes</command> by itself to the script
313 results in <keycap>y</keycap> being passed to the script. Now try it with a
314 string of text:</para>
316<screen><userinput>yes 'This is some text' | ./blfs-yes-test1</userinput></screen>
318 <para>The exact string was used as the response to the script. Finally,
319 try it using an empty (null) string:</para>
321<screen><userinput>yes '' | ./blfs-yes-test1</userinput></screen>
323 <para>Notice this results in passing just the press of the
324 <keycap>Enter</keycap> key to the script. This is useful for times when the
325 default answer to the prompt is sufficient. This syntax is used in the
326 <xref linkend="net-tools-automate-example"/> instructions to accept all the
327 defaults to the many prompts during the configuration step. You may now
328 remove the test script, if desired.</para>
330 <bridgehead renderas="sect3">File Redirection to Automate Output</bridgehead>
332 <para>In order to automate the building of some packages, especially those
333 that require you to read a license agreement one page at a time, requires
334 using a method that avoids having to press a key to display each page.
335 Redirecting the output to a file can be used in these instances to assist
336 with the automation. The previous section on this page touched on creating
337 log files of the build output. The redirection method shown there used the
338 <command>tee</command> command to redirect output to a file while also
339 displaying the output to the screen. Here, the output will only be sent to
340 a file.</para>
342 <para>Again, the easiest way to demonstrate the technique is to show an
343 example. First, issue the command:</para>
345<screen><userinput>ls -l /usr/bin | more</userinput></screen>
347 <para>Of course, you'll be required to view the output one page at a time
348 because the <command>more</command> filter was used. Now try the same
349 command, but this time redirect the output to a file. The special file
350 <filename>/dev/null</filename> can be used instead of the filename shown,
351 but you will have no log file to examine:</para>
353<screen><userinput>ls -l /usr/bin | more &gt; redirect_test.log 2&gt;&amp;1</userinput></screen>
355 <para>Notice that this time the command immediately returned to the shell
356 prompt without having to page through the output. You may now remove the
357 log file.</para>
359 <para>The last example will use the <command>yes</command> command in
360 combination with output redirection to bypass having to page through the
361 output and then provide a <keycap>y</keycap> to a prompt. This technique
362 could be used in instances when otherwise you would have to page through
363 the output of a file (such as a license agreement) and then answer the
364 question of <quote>do you accept the above?</quote>. For this example,
365 another short <application>Bash</application> script is required:</para>
367<screen><userinput>cat &gt; blfs-yes-test2 &lt;&lt; "EOF"
370ls -l /usr/bin | more
372echo -n -e "\n\nDid you enjoy reading this? (y,n) "
374read A_STRING
376if test "$A_STRING" = "y"; then A_STRING="You entered the 'y' key"
377else A_STRING="You did NOT enter the 'y' key"
380echo -e "\n\n$A_STRING\n\n"</literal>
382chmod 755 blfs-yes-test2</userinput></screen>
384 <para>This script can be used to simulate a program that requires you to
385 read a license agreement, then respond appropriately to accept the
386 agreement before the program will install anything. First, run the script
387 without any automation techniques by issuing
388 <command>./blfs-yes-test2</command>.</para>
390 <para>Now issue the following command which uses two automation techniques,
391 making it suitable for use in an automated build script:</para>
393<screen><userinput>yes | ./blfs-yes-test2 &gt; blfs-yes-test2.log 2&gt;&amp;1</userinput></screen>
395 <para>If desired, issue <command>tail blfs-yes-test2.log</command> to see
396 the end of the paged output, and confirmation that <keycap>y</keycap> was
397 passed through to the script. Once satisfied that it works as it should,
398 you may remove the script and log file.</para>
400 <para>Finally, keep in mind that there are many ways to automate and/or
401 script the build commands. There is not a single <quote>correct</quote> way
402 to do it. Your imagination is the only limit.</para>
404 </sect2>
406 <sect2>
407 <title>Dependencies</title>
409 <para>For each package described, BLFS lists the known dependencies.
410 These are listed under several headings, whose meaning is as follows:</para>
412 <itemizedlist>
413 <listitem>
414 <para><emphasis>Required</emphasis> means that the target package
415 cannot be correctly built without the dependency having first been
416 installed.</para>
417 </listitem>
418 <listitem>
419 <para><emphasis>Recommended</emphasis> means that BLFS strongly
420 suggests this package is installed first for a clean and trouble-free
421 build, that won't have issues either during the build process, or at
422 run-time. The instructions in the book assume these packages are
423 installed. Some changes or workarounds may be required if these
424 packages are not installed.</para>
425 </listitem>
426 <listitem>
427 <para><emphasis>Optional</emphasis> means that this package might be
428 installed for added functionality. Often BLFS will describe the
429 dependency to explain the added functionality that will result.</para>
430 </listitem>
431 </itemizedlist>
433 </sect2>
435 <sect2 id="package_updates">
436 <title>Using the Most Current Package Sources</title>
438 <para>On occasion you may run into a situation in the book when a package
439 will not build or work properly. Though the Editors attempt to ensure
440 that every package in the book builds and works properly, sometimes a
441 package has been overlooked or was not tested with this particular version
442 of BLFS.</para>
444 <para>If you discover that a package will not build or work properly, you
445 should see if there is a more current version of the package. Typically
446 this means you go to the maintainer's web site and download the most current
447 tarball and attempt to build the package. If you cannot determine the
448 maintainer's web site by looking at the download URLs, use Google and query
449 the package's name. For example, in the Google search bar type:
450 'package_name download' (omit the quotes) or something similar. Sometimes
451 typing: 'package_name home page' will result in you finding the
452 maintainer's web site.</para>
454 </sect2>
456 <sect2 id="stripping">
457 <title>Stripping One More Time</title>
459 <para>
460 In LFS, stripping of debugging symbols was discussed a couple of
461 times. When building BLFS packages, there are generally no special
462 instructions that discuss stripping again. It is probably not a good
463 idea to strip an executable or a library while it is in use, so exiting
464 any windowing environment is a good idea. Then you can do:
465 </para>
467<screen><userinput>find /{,usr/}{bin,lib,sbin} \
468 -type f \( -name \*.so* -a ! -name \*dbg \) \
469 -exec strip --strip-unneeded {} \;</userinput></screen>
471 <para>
472 If you install programs in other directories such as <filename
473 class="directory">/opt</filename> or <filename
474 class="directory">/usr/local</filename>, you may want to strip the files
475 there too.
476 </para>
478 <para>
479 For more information on stripping, see <ulink
480 url=""/>.
481 </para>
483 </sect2>
485 <sect2 id="libtool">
486 <title>Libtool files</title>
488 <para>
489 One of the side effects of packages that use Autotools, including
490 libtool, is that they create many files with an .la extension. These
491 files are not needed in an LFS environment. If there are conflicts with
492 pkgconfig entries, they can actually prevent successful builds. You
493 may want to consider removing these files periodically:
494 </para>
496<screen><userinput>find /lib /usr/lib -not -path "*Image*" -a -name \*.la -delete</userinput></screen>
498 <para>
499 The above command removes all .la files with the exception of those that
500 have <quote>Image</quote> or <quote>openldap</quote> as a part of the
501 path. These .la files are used by the ImageMagick and openldap programs,
502 respectively. There may be other exceptions by packages not in BLFS.
503 </para>
505 </sect2>
507 <sect2 id="buildsystems">
508 <title>Working with different build systems</title>
510 <para>
511 There are now three different build systems in common use for
512 converting C or C++ source code into compiled programs or
513 libraries and their details (particularly, finding out about available
514 options and their default values) differ. It may be easiest to understand
515 the issues caused by some choices (typically slow execution or
516 unexpected use of, or omission of, optimizatons) by starting with
517 the CFLAGS and CXXFLAGS environment variables. There are also some
518 programs which use rust.
519 </para>
521 <para>
522 Most LFS and BLFS builders are probably aware of the basics of CFLAGS
523 and CXXFLAGS for altering how a program is compiled. Typically, some
524 form of optimization is used by upstream developers (-O2 or -O3),
525 sometimes with the creation of debug symbols (-g), as defaults.
526 </para>
528 <para>
529 If there are contradictory flags (e.g. multiple different -O values),
530 the <emphasis>last</emphasis> value will be used. Sometimes this means
531 that flags specified in environment variables will be picked up before
532 values hardcoded in the Makefile, and therefore ignored. For example,
533 where a user specifies '-O2' and that is followed by '-O3' the build will
534 use '-O3'.
535 </para>
537 <para>
538 There are various other things which can be passed in CFLAGS or
539 CXXFLAGS, such as forcing compilation for a specific microarchitecture
540 (e.g. -march=amdfam10, -march=native) or specifying a specific standard
541 for C or C++ (-std=c++17 for example). But one thing which has now come
542 to light is that programmers might include debug assertions in their
543 code, expecting them to be disabled in releases by using -DNDEBUG.
544 Specifically, if <xref linkend="mesa"/> is built with these assertions
545 enabled, some activities such as loading levels of games can take
546 extremely long times, even on high-class video cards.
547 </para>
549 <bridgehead renderas="sect3" id="autotools-info">Autotools with Make</bridgehead>
551 <para>
552 This combination is often described as 'CMMI' (configure, make, make
553 install) and is used here to also cover the few packages which have a
554 configure script that is not generated by autotools.
555 </para>
557 <para>
558 Sometimes running <command>./configure --help</command> will produce
559 useful options about switches which might be used. At other times,
560 after looking at the output from configure you may need to look
561 at the details of the script to find out what it was actually searching
562 for.
563 </para>
565 <para>
566 Many configure scripts will pick up any CFLAGS or CXXFLAGS from the
567 environment, but CMMI packages vary about how these will be mixed with
568 any flags which would otherwise be used (<emphasis>variously</emphasis>:
569 ignored, used to replace the programmer's suggestion, used before the
570 programmer's suggestion, or used after the programmer's suggestion).
571 </para>
573 <para>
574 In most CMMI packages, running 'make' will list each command and run
575 it, interspersed with any warnings. But some packages try to be 'silent'
576 and only show which file they are compiling or linking instead of showing
577 the command line. If you need to inspect the command, either because of
578 an error, or just to see what options and flags are being used, adding
579 'V=1' to the make invocation may help.
580 </para>
582 <bridgehead renderas="sect3" id="cmake-info">CMake</bridgehead>
584 <para>
585 CMake works in a very different way, and it has two backends which can
586 be used on BLFS: 'make' and 'ninja'. The default backend is make, but
587 ninja can be faster on large packages with multiple processors. To
588 use ninja, specify '-G Ninja' in the cmake command. However, there are
589 some packages which create fatal errors in their ninja files but build
590 successfully using the default of Unix Makefiles.
591 </para>
593 <para>
594 The hardest part of using CMake is knowing what options you might wish
595 to specify. The only way to get a list of what the package knows about
596 is to run <command>cmake -LAH</command> and look at the output for that
597 default configuration.
598 </para>
600 <para>
601 Perhaps the most-important thing about CMake is that it has a variety
602 of CMAKE_BUILD_TYPE values, and these affect the flags. The default
603 is that this is not set and no flags are generated. Any CFLAGS or
604 CXXFLAGS in the environment will be used. If the programmer has coded
605 any debug assertions, those will be enabled unless -DNDEBUG is used.
606 The following CMAKE_BUILD_TYPE values will generate the flags shown,
607 and these will come <emphasis>after</emphasis> any flags in the
608 environment and therefore take precedence.
609 </para>
611 <itemizedlist>
612 <listitem>
613 <para>Debug : '-g'</para>
614 </listitem>
615 <listitem>
616 <para>Release : '-O3 -DNDEBUG'</para>
617 </listitem>
618 <listitem>
619 <para>RelWithDebInfo : '-O2 -g -DNDEBUG'</para>
620 </listitem>
621 <listitem>
622 <para>MinSizeRel : '-Os -DNDEBUG'</para>
623 </listitem>
624 </itemizedlist>
626 <para>
627 CMake tries to produce quiet builds. To see the details of the commands
628 which are being run, use 'make VERBOSE=1' or 'ninja -v'.
629 </para>
631 <bridgehead renderas="sect3" id="meson-info">Meson</bridgehead>
633 <para>
634 Meson has some similarities to CMake, but many differences. To get
635 details of the defines that you may wish to change you can look at
636 <filename>meson_options.txt</filename> which is usually in the
637 top-level directory.
638 </para>
640 <para>
641 If you have already configured the package by running
642 <command>meson</command> and now wish to change one or more settings,
643 you can either remove the build directory, recreate it, and use the
644 altered options, or within the build directory run <command>meson
645 configure</command>, e.g. to set an option:
646 </para>
648<screen><userinput>meson configure -D&lt;some_option&gt;=true</userinput></screen>
650 <para>
651 If you do that, the file <filename>meson-private/cmd_line.txt</filename>
652 will show the <emphasis>last</emphasis> commands which were used.
653 </para>
655 <para>
656 Meson provides the following buildtype values, and the flags they enable
657 come <emphasis>after</emphasis> any flags supplied in the environment and
658 therefore take precedence.
659 </para>
661 <itemizedlist>
662 <listitem>
663 <para>plain : no added flags. This is for distributors to supply their
664 own CLFAGS, CXXFLAGS and LDFLAGS. There is no obvious reason to use
665 this in BLFS.</para>
666 </listitem>
667 <listitem>
668 <para>debug : '-g' - this is the default if nothing is specified
669 in either <filename></filename> or the command line.
670 However it results large and slow binaries, so we should override
671 it in BLFS.</para>
672 </listitem>
673 <listitem>
674 <para>debugoptimized : '-O2 -g' : this is the default specified in
675 <filename></filename> of some packages.</para>
676 </listitem>
677 <listitem>
678 <para>release : '-O3 -DNDEBUG' (but occasionally a package will force
679 -O2 here)</para>
680 </listitem>
681 </itemizedlist>
683 <para>
684 Although the 'release' buildtype is described as enabling -DNDEBUG, and all
685 CMake Release builds pass that, it has so far only been observed (in
686 verbose builds) for <xref linkend="mesa"/>. That suggests that it might
687 only be used when there are debug assertions present.
688 </para>
690 <para>
691 The -DNDEBUG flag can also be provided by passing
692 <command>-Db_ndebug=true</command>.
693 </para>
695 <para>
696 To see the details of the commands which are being run in a package using
697 meson, use 'ninja -v'.
698 </para>
700 <bridgehead renderas="sect3" id="rust-info">Rustc and Cargo</bridgehead>
702 <para>
703 Most released rustc programs are provided as crates (source tarballs)
704 which will query a server to check current versions of dependencies
705 and then download them as necessary. These packages are built using
706 <command>cargo --release</command>. In theory, you can manipulate the
707 RUSTFLAGS to change the optimize-level (default is 3, like -O3, e.g.
708 <literal>-Copt-level=3</literal>) or to force it to build for the
709 machine it is being compiled on, using
710 <literal>-Ctarget-cpu=native</literal> but in practice this seems to
711 make no significant difference.
712 </para>
714 <para>
715 If you find an interesting rustc program which is only provided as
716 unpackaged source, you should at least specify
717 <literal>RUSTFLAGS=-Copt-level=2</literal> otherwise it will do an
718 unoptimized compile with debug info and run <emphasis>much</emphasis>
719 slower.
720 </para>
722 <para>
723 The rust developers seem to assume that everyone will compile on a
724 machine dedicated to producing builds, so by default all CPUs are used.
725 This can often be worked around, either by exporting
726 CARGO_BUILD_JOBS=&lt;N&gt; or passing --jobs &lt;N&gt; to cargo. For
727 compiling rustc itself, specifying --jobs &lt;N&gt; on invocations of
728 (together with the <envar>CARGO_BUILD_JOBS</envar> environment
729 variable, which looks like a "belt and braces" approach but seems to be
730 necessary) mostly works. The exception is running the tests when building
731 rustc, some of them will nevertheless use all online CPUs, at least as of
732 rustc-1.42.0.
733 </para>
735 </sect2>
737 <sect2 id="optimizations">
738 <title>Optimizing the build</title>
740 <para>
741 Many people will prefer to optimize compiles as they see fit, by providing
742 CFLAGS or CXXFLAGS. For an introduction to the options available with gcc
743 and g++ see <ulink
744 url=""/> and <ulink
745 url=""/>
746 and <command>info gcc</command>.
748 </para>
750 <para>
751 Some packages default to '-O2 -g', others to '-O3 -g', and if CFLAGS or
752 CXXFLAGS are supplied they might be added to the package's defaults,
753 replace the package's defaults, or even be ignored. There are details
754 on some desktop packages which were mostly current in April 2019 at
755 <ulink url=""/> - in
756 particular, README.txt, tuning-1-packages-and-notes.txt, and
757 tuning-notes-2B.txt. The particular thing to remember is that if you
758 want to try some of the more interesting flags you may need to force
759 verbose builds to confirm what is being used.
760 </para>
762 <para>
763 Clearly, if you are optimizing your own program you can spend time to
764 profile it and perhaps recode some of it if it is too slow. But for
765 building a whole system that approach is impractical. In general,
766 -O3 usually produces faster programs than -O2. Specifying
767 -march=native is also beneficial, but means that you cannot move the
768 binaries to an incompatible machine - this can also apply to newer
769 machines, not just to older machines. For example programs compiled for
770 'amdfam10' run on old Phenoms, Kaveris, and Ryzens : but programs
771 compiled for a Kaveri will not run on a Ryzen because certain op-codes
772 are not present. Similarly, if you build for a Haswell not everything
773 will run on a SandyBridge.
774 </para>
776 <para>
777 There are also various other options which some people claim are
778 beneficial. At worst, you get to recompile and test, and then
779 discover that in your usage the options do not provide a benefit.
780 </para>
782 <para>
783 If building Perl or Python modules, or Qt packages which use qmake,
784 in general the CFLAGS and CXXFLAGS used are those which were used by
785 those 'parent' packages.
786 </para>
788 </sect2>
790 <sect2 id="hardening">
791 <title>Options for hardening the build</title>
793 <para>
794 Even on desktop systems, there are still a lot of exploitable
795 vulnerabilities. For many of these, the attack comes via javascript
796 in a browser. Often, a series of vulnerabilities are used to gain
797 access to data (or sometimes to pwn, i.e. own, the machine and
798 install rootkits). Most commercial distros will apply various
799 hardening measures.
800 </para>
802 <para>
803 For hardening options which are reasonably cheap, there is some
804 discussion in the 'tuning' link above (occasionally, one or more
805 of these options might be inappropriate for a package). These
806 options are -D_FORTIFY_SOURCE=2, -fstack-protector=strong, and
807 (for C++) -D_GLIBCXX_ASSERTIONS. On modern machines these should
808 only have a little impact on how fast things run, and often they
809 will not be noticeable.
810 </para>
812 <para>
813 In the past, there was Hardened LFS where gcc (a much older version)
814 was forced to use hardening (with options to turn some of it off on a
815 per-package basis. What is being covered here is different - first you
816 have to make sure that the package is indeed using your added flags and
817 not over-riding them.
818 </para>
820 <para>
821 The main distros use much more, such as RELRO (Relocation Read Only)
822 and perhaps -fstack-clash-protection. You may also encounter the
823 so-called 'userspace retpoline' (-mindirect-branch=thunk etc.) which
824 is the equivalent of the spectre mitigations applied to the linux
825 kernel in late 2018). The kernel mitigations caused a lot of complaints
826 about lost performance, if you have a production server you might wish
827 to consider testing that, along with the other available options, to
828 see if performance is still sufficient.
829 </para>
831 <para>
832 Whilst gcc has many hardening options, clang/LLVM's strengths lie
833 elsewhere. Some options which gcc provides are said to be less effective
834 in clang/LLVM.
835 </para>
837 </sect2>
Note: See TracBrowser for help on using the repository browser.