source: introduction/important/building-notes.xml@ 88f20bb

Last change on this file since 88f20bb was 88f20bb, checked in by Ken Moffat <ken@…>, 4 years ago

Perl modules branch: merge revisions 20505 to 20531 from trunk.

git-svn-id: svn:// af4574ff-66df-0310-9fd7-8a98e5e911e0

  • Property mode set to 100644
File size: 20.4 KB
1<?xml version="1.0" encoding="ISO-8859-1"?>
2<!DOCTYPE sect1 PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
3 "" [
4 <!ENTITY % general-entities SYSTEM "../../general.ent">
5 %general-entities;
8<sect1 id="unpacking">
9 <?dbhtml filename="notes-on-building.html"?>
11 <sect1info>
12 <othername>$LastChangedBy$</othername>
13 <date>$Date$</date>
14 </sect1info>
16 <title>Notes on Building Software</title>
18 <para>Those people who have built an LFS system may be aware
19 of the general principles of downloading and unpacking software. Some
20 of that information is repeated here for those new to building
21 their own software.</para>
23 <para>Each set of installation instructions contains a URL from which you
24 can download the package. The patches; however, are stored on the LFS
25 servers and are available via HTTP. These are referenced as needed in the
26 installation instructions.</para>
28 <para>While you can keep the source files anywhere you like, we assume that
29 you have unpacked the package and changed into the directory created by the
30 unpacking process (the 'build' directory). We also assume you have
31 uncompressed any required patches and they are in the directory immediately
32 above the 'build' directory.</para>
34 <para>We can not emphasize strongly enough that you should start from a
35 <emphasis>clean source tree</emphasis> each time. This means that if
36 you have had an error during configuration or compilation, it's usually
37 best to delete the source tree and
38 re-unpack it <emphasis>before</emphasis> trying again. This obviously
39 doesn't apply if you're an advanced user used to hacking
40 <filename>Makefile</filename>s and C code, but if in doubt, start from a
41 clean tree.</para>
43 <sect2>
44 <title>Building Software as an Unprivileged (non-root) User</title>
46 <para>The golden rule of Unix System Administration is to use your
47 superpowers only when necessary. Hence, BLFS recommends that you
48 build software as an unprivileged user and only become the
49 <systemitem class='username'>root</systemitem> user when installing the
50 software. This philosophy is followed in all the packages in this book.
51 Unless otherwise specified, all instructions should be executed as an
52 unprivileged user. The book will advise you on instructions that need
53 <systemitem class='username'>root</systemitem> privileges.</para>
55 </sect2>
57 <sect2>
58 <title>Unpacking the Software</title>
60 <para>If a file is in <filename class='extension'>.tar</filename> format
61 and compressed, it is unpacked by running one of the following
62 commands:</para>
64<screen><userinput>tar -xvf filename.tar.gz
65tar -xvf filename.tgz
66tar -xvf filename.tar.Z
67tar -xvf filename.tar.bz2</userinput></screen>
69 <note>
70 <para>You may omit using the <option>v</option> parameter in the commands
71 shown above and below if you wish to suppress the verbose listing of all
72 the files in the archive as they are extracted. This can help speed up the
73 extraction as well as make any errors produced during the extraction
74 more obvious to you.</para>
75 </note>
77 <para>You can also use a slightly different method:</para>
79<screen><userinput>bzcat filename.tar.bz2 | tar -xv</userinput></screen>
81 <para>Finally, you sometimes need to be able to unpack patches which are
82 generally not in <filename class='extension'>.tar</filename> format. The
83 best way to do this is to copy the patch file to the parent of the 'build'
84 directory and then run one of the following commands depending on whether
85 the file is a <filename class='extension'>.gz</filename> or <filename
86 class='extension'>.bz2</filename> file:</para>
88<screen><userinput>gunzip -v patchname.gz
89bunzip2 -v patchname.bz2</userinput></screen>
91 </sect2>
93 <sect2>
94 <title>Verifying File Integrity Using 'md5sum'</title>
96 <para>Generally, to verify that the downloaded file is genuine and complete,
97 many package maintainers also distribute md5sums of the files. To verify the
98 md5sum of the downloaded files, download both the file and the
99 corresponding md5sum file to the same directory (preferably from different
100 on-line locations), and (assuming <filename>file.md5sum</filename> is the
101 md5sum file downloaded) run the following command:</para>
103<screen><userinput>md5sum -c file.md5sum</userinput></screen>
105 <para>If there are any errors, they will be reported. Note that the BLFS
106 book includes md5sums for all the source files also. To use the BLFS
107 supplied md5sums, you can create a <filename>file.md5sum</filename> (place
108 the md5sum data and the exact name of the downloaded file on the same
109 line of a file, separated by white space) and run the command shown above.
110 Alternately, simply run the command shown below and compare the output
111 to the md5sum data shown in the BLFS book.</para>
113<screen><userinput>md5sum <replaceable>&lt;name_of_downloaded_file&gt;</replaceable></userinput></screen>
115 </sect2>
117 <sect2>
118 <title>Creating Log Files During Installation</title>
120 <para>For larger packages, it is convenient to create log files instead of
121 staring at the screen hoping to catch a particular error or warning. Log
122 files are also useful for debugging and keeping records. The following
123 command allows you to create an installation log. Replace
124 <replaceable>&lt;command&gt;</replaceable> with the command you intend to execute.</para>
126<screen><userinput>( <replaceable>&lt;command&gt;</replaceable> 2&gt;&amp;1 | tee compile.log &amp;&amp; exit $PIPESTATUS )</userinput></screen>
128 <para><option>2&gt;&amp;1</option> redirects error messages to the same
129 location as standard output. The <command>tee</command> command allows
130 viewing of the output while logging the results to a file. The parentheses
131 around the command run the entire command in a subshell and finally the
132 <command>exit $PIPESTATUS</command> command ensures the result of the
133 <replaceable>&lt;command&gt;</replaceable> is returned as the result and not the
134 result of the <command>tee</command> command.</para>
136 </sect2>
138 <sect2 id="parallel-builds" xreflabel="Using Multiple Processors">
139 <title>Using Multiple Processors</title>
141 <para>For many modern systems with multiple processors (or cores) the
142 compilation time for a package can be reduced by performing a "parallel
143 make" by either setting an environment variable or telling the make program
144 how many processors are available. For instance, a Core2Duo can support two
145 simultaneous processes with: </para>
147 <screen><userinput>export MAKEFLAGS='-j2'</userinput></screen>
149 <para>or just building with:</para>
151 <screen><userinput>make -j2</userinput></screen>
153 <para>Generally the number of processes should not exceed the number of
154 cores supported by the CPU. To list the processors on your
155 system, issue: <userinput>grep processor /proc/cpuinfo</userinput>.
156 </para>
158 <para>In some cases, using multiple processors may result in a 'race'
159 condition where the success of the build depends on the order of the
160 commands run by the <command>make</command> program. For instance, if an
161 executable needs File A and File B, attempting to link the program before
162 one of the dependent components is available will result in a failure.
163 This condition usually arises because the upstream developer has not
164 properly designated all the prerequisites needed to accomplish a step in the
165 Makefile.</para>
167 <para>If this occurs, the best way to proceed is to drop back to a
168 single processor build. Adding '-j1' to a make command will override
169 the similar setting in the MAKEFLAGS environment variable.</para>
171 <note><para>When running the package tests or the install portion of the
172 package build process, we do not recommend using an option greater than
173 '-j1' unless specified otherwise. The installation procedures or checks
174 have not been validated using parallel procedures and may fail with issues
175 that are difficult to debug.</para></note>
177 </sect2>
179 <sect2 id="automating-builds" xreflabel="Automated Building Procedures">
180 <title>Automated Building Procedures</title>
182 <para>There are times when automating the building of a package can come in
183 handy. Everyone has their own reasons for wanting to automate building,
184 and everyone goes about it in their own way. Creating
185 <filename>Makefile</filename>s, <application>Bash</application> scripts,
186 <application>Perl</application> scripts or simply a list of commands used
187 to cut and paste are just some of the methods you can use to automate
188 building BLFS packages. Detailing how and providing examples of the many
189 ways you can automate the building of packages is beyond the scope of this
190 section. This section will expose you to using file redirection and the
191 <command>yes</command> command to help provide ideas on how to automate
192 your builds.</para>
194 <bridgehead renderas="sect3">File Redirection to Automate Input</bridgehead>
196 <para>You will find times throughout your BLFS journey when you will come
197 across a package that has a command prompting you for information. This
198 information might be configuration details, a directory path, or a response
199 to a license agreement. This can present a challenge to automate the
200 building of that package. Occasionally, you will be prompted for different
201 information in a series of questions. One method to automate this type of
202 scenario requires putting the desired responses in a file and using
203 redirection so that the program uses the data in the file as the answers to
204 the questions.</para>
206 <para>Building the <application>CUPS</application> package is a good
207 example of how redirecting a file as input to prompts can help you automate
208 the build. If you run the test suite, you are asked to respond to a series
209 of questions regarding the type of test to run and if you have any
210 auxiliary programs the test can use. You can create a file with your
211 responses, one response per line, and use a command similar to the
212 one shown below to automate running the test suite:</para>
214<screen><userinput>make check &lt; ../cups-1.1.23-testsuite_parms</userinput></screen>
216 <para>This effectively makes the test suite use the responses in the file
217 as the input to the questions. Occasionally you may end up doing a bit of
218 trial and error determining the exact format of your input file for some
219 things, but once figured out and documented you can use this to automate
220 building the package.</para>
222 <bridgehead renderas="sect3">Using <command>yes</command> to Automate
223 Input</bridgehead>
225 <para>Sometimes you will only need to provide one response, or provide the
226 same response to many prompts. For these instances, the
227 <command>yes</command> command works really well. The
228 <command>yes</command> command can be used to provide a response (the same
229 one) to one or more instances of questions. It can be used to simulate
230 pressing just the <keycap>Enter</keycap> key, entering the
231 <keycap>Y</keycap> key or entering a string of text. Perhaps the easiest
232 way to show its use is in an example.</para>
234 <para>First, create a short <application>Bash</application> script by
235 entering the following commands:</para>
237<screen><userinput>cat &gt; blfs-yes-test1 &lt;&lt; "EOF"
240echo -n -e "\n\nPlease type something (or nothing) and press Enter ---> "
242read A_STRING
244if test "$A_STRING" = ""; then A_STRING="Just the Enter key was pressed"
245else A_STRING="You entered '$A_STRING'"
248echo -e "\n\n$A_STRING\n\n"</literal>
250chmod 755 blfs-yes-test1</userinput></screen>
252 <para>Now run the script by issuing <command>./blfs-yes-test1</command> from
253 the command line. It will wait for a response, which can be anything (or
254 nothing) followed by the <keycap>Enter</keycap> key. After entering
255 something, the result will be echoed to the screen. Now use the
256 <command>yes</command> command to automate the entering of a
257 response:</para>
259<screen><userinput>yes | ./blfs-yes-test1</userinput></screen>
261 <para>Notice that piping <command>yes</command> by itself to the script
262 results in <keycap>y</keycap> being passed to the script. Now try it with a
263 string of text:</para>
265<screen><userinput>yes 'This is some text' | ./blfs-yes-test1</userinput></screen>
267 <para>The exact string was used as the response to the script. Finally,
268 try it using an empty (null) string:</para>
270<screen><userinput>yes '' | ./blfs-yes-test1</userinput></screen>
272 <para>Notice this results in passing just the press of the
273 <keycap>Enter</keycap> key to the script. This is useful for times when the
274 default answer to the prompt is sufficient. This syntax is used in the
275 <xref linkend="net-tools-automate-example"/> instructions to accept all the
276 defaults to the many prompts during the configuration step. You may now
277 remove the test script, if desired.</para>
279 <bridgehead renderas="sect3">File Redirection to Automate Output</bridgehead>
281 <para>In order to automate the building of some packages, especially those
282 that require you to read a license agreement one page at a time, requires
283 using a method that avoids having to press a key to display each page.
284 Redirecting the output to a file can be used in these instances to assist
285 with the automation. The previous section on this page touched on creating
286 log files of the build output. The redirection method shown there used the
287 <command>tee</command> command to redirect output to a file while also
288 displaying the output to the screen. Here, the output will only be sent to
289 a file.</para>
291 <para>Again, the easiest way to demonstrate the technique is to show an
292 example. First, issue the command:</para>
294<screen><userinput>ls -l /usr/bin | more</userinput></screen>
296 <para>Of course, you'll be required to view the output one page at a time
297 because the <command>more</command> filter was used. Now try the same
298 command, but this time redirect the output to a file. The special file
299 <filename>/dev/null</filename> can be used instead of the filename shown,
300 but you will have no log file to examine:</para>
302<screen><userinput>ls -l /usr/bin | more &gt; redirect_test.log 2&gt;&amp;1</userinput></screen>
304 <para>Notice that this time the command immediately returned to the shell
305 prompt without having to page through the output. You may now remove the
306 log file.</para>
308 <para>The last example will use the <command>yes</command> command in
309 combination with output redirection to bypass having to page through the
310 output and then provide a <keycap>y</keycap> to a prompt. This technique
311 could be used in instances when otherwise you would have to page through
312 the output of a file (such as a license agreement) and then answer the
313 question of <quote>do you accept the above?</quote>. For this example,
314 another short <application>Bash</application> script is required:</para>
316<screen><userinput>cat &gt; blfs-yes-test2 &lt;&lt; "EOF"
319ls -l /usr/bin | more
321echo -n -e "\n\nDid you enjoy reading this? (y,n) "
323read A_STRING
325if test "$A_STRING" = "y"; then A_STRING="You entered the 'y' key"
326else A_STRING="You did NOT enter the 'y' key"
329echo -e "\n\n$A_STRING\n\n"</literal>
331chmod 755 blfs-yes-test2</userinput></screen>
333 <para>This script can be used to simulate a program that requires you to
334 read a license agreement, then respond appropriately to accept the
335 agreement before the program will install anything. First, run the script
336 without any automation techniques by issuing
337 <command>./blfs-yes-test2</command>.</para>
339 <para>Now issue the following command which uses two automation techniques,
340 making it suitable for use in an automated build script:</para>
342<screen><userinput>yes | ./blfs-yes-test2 &gt; blfs-yes-test2.log 2&gt;&amp;1</userinput></screen>
344 <para>If desired, issue <command>tail blfs-yes-test2.log</command> to see
345 the end of the paged output, and confirmation that <keycap>y</keycap> was
346 passed through to the script. Once satisfied that it works as it should,
347 you may remove the script and log file.</para>
349 <para>Finally, keep in mind that there are many ways to automate and/or
350 script the build commands. There is not a single <quote>correct</quote> way
351 to do it. Your imagination is the only limit.</para>
353 </sect2>
355 <sect2>
356 <title>Dependencies</title>
358 <para>For each package described, BLFS lists the known dependencies.
359 These are listed under several headings, whose meaning is as follows:</para>
361 <itemizedlist>
362 <listitem>
363 <para><emphasis>Required</emphasis> means that the target package
364 cannot be correctly built without the dependency having first been
365 installed.</para>
366 </listitem>
367 <listitem>
368 <para><emphasis>Recommended</emphasis> means that BLFS strongly
369 suggests this package is installed first for a clean and trouble-free
370 build, that won't have issues either during the build process, or at
371 run-time. The instructions in the book assume these packages are
372 installed. Some changes or workarounds may be required if these
373 packages are not installed.</para>
374 </listitem>
375 <listitem>
376 <para><emphasis>Optional</emphasis> means that this package might be
377 installed for added functionality. Often BLFS will describe the
378 dependency to explain the added functionality that will result.</para>
379 </listitem>
380 </itemizedlist>
382 </sect2>
384 <sect2 id="package_updates">
385 <title>Using the Most Current Package Sources</title>
387 <para>On occasion you may run into a situation in the book when a package
388 will not build or work properly. Though the Editors attempt to ensure
389 that every package in the book builds and works properly, sometimes a
390 package has been overlooked or was not tested with this particular version
391 of BLFS.</para>
393 <para>If you discover that a package will not build or work properly, you
394 should see if there is a more current version of the package. Typically
395 this means you go to the maintainer's web site and download the most current
396 tarball and attempt to build the package. If you cannot determine the
397 maintainer's web site by looking at the download URLs, use Google and query
398 the package's name. For example, in the Google search bar type:
399 'package_name download' (omit the quotes) or something similar. Sometimes
400 typing: 'package_name home page' will result in you finding the
401 maintainer's web site.</para>
403 </sect2>
405 <sect2 id="stripping">
406 <title>Stripping One More Time</title>
408 <para>
409 In LFS, stripping of debugging symbols was discussed a couple of
410 times. When building BLFS packages, there are generally no special
411 instructions that discuss stripping again. It is probably not a good
412 idea to strip an executable or a library while it is in use, so exiting
413 any windowing environment is a good idea. Then you can do:
414 </para>
416<screen><userinput>find /{,usr/}{bin,lib,sbin} -type f -exec strip --strip-unneeded {} \;</userinput></screen>
418 <para>
419 If you install programs in other directories such as <filename
420 class="directory">/opt</filename> or <filename
421 class="directory">/usr/local</filename>, you may want to strip the files
422 there too.
423 </para>
425 <para>
426 For more information on stripping, see <ulink
427 url=""/>.
428 </para>
430 </sect2>
432 <sect2 id="libtool">
433 <title>Libtool files</title>
435 <para>
436 One of the side effects of packages that use Autotools, including
437 libtool, is that they create many files with an .la extension. These
438 files are not needed in an LFS environment. If there are conflicts with
439 pkgconfig entries, they can actually prevent successful builds. You
440 may want to consider removing these files periodically:
441 </para>
443<screen><userinput>find /lib /usr/lib -not -path "*Image*" -a -name \*.la -delete</userinput></screen>
445 <para>
446 The above command removes all .la files with the exception of those that
447 have <quote>Image</quote> or <quote>openldap</quote> as a part of the
448 path. These .la files are used by the ImageMagick and openldap programs,
449 respectively. There may be other exceptions by packages not in BLFS.
450 </para>
452 </sect2>
Note: See TracBrowser for help on using the repository browser.