Changeset 6efdfbe

Timestamp:

07/02/2023 11:33:10 AM (12 months ago)

Author:

Xi Ruoyao <xry111@…>

Branches:

12.0, 12.1, kea, ken/TL2024, ken/tuningfonts, lazarus, plabs/newcss, python3.11, rahul/power-profiles-daemon, renodr/vulkan-addition, trunk, xry111/llvm18, xry111/xf86-video-removal

Children:

9b61265

Parents:

cbffe27

Message:

building-notes: Format and update

• Do not use the single quote "'" for *everything*. Use a proper XML tag if applicable.
• Don't call the source directory "build" directory because we now often create a subdirectory named "build", so calling the source directory "build" can be puzzling.
• Update the method to use compressed patch (matching libpng instruction).
• Explain why ninja attempts to use N+1 or N+2 logical processors.
• Now we don't run test suites w/o parallel jobs.
• Use "less" instead of "more" as "less" is actually more powerful.
• Explain -mtune=.
• -DNDEBUG is not always implied by --buildtype=release, but it's sometimes implied.
• Alert that some "Skylake"s are actually not -march=skylake.
• For Rust, explain "-O" and "-g".

File:

• introduction/important/building-notes.xml (modified) (20 diffs)

introduction/important/building-notes.xml

@@ -24 +24 @@
   <para>While you can keep the source files anywhere you like, we assume that
   you have unpacked the package and changed into the directory created by the
-  unpacking process (the 'build' directory).  We also assume you have
-  uncompressed any required patches and they are in the directory immediately
-  above the 'build' directory.</para>
+  unpacking process (the source directory).  We also assume you have
+  uncompressed any required patches and they are in the directory
+  immediately above the source directory.</para>
 
   <para>We can not emphasize strongly enough that you should start from a
@@ -75 +75 @@
 <screen><userinput>bzcat filename.tar.bz2 | tar -xv</userinput></screen>
 
-    <para>Finally, you sometimes need to be able to unpack patches which are
-    generally not in <filename class='extension'>.tar</filename> format. The
-    best way to do this is to copy the patch file to the parent of the 'build'
-    directory and then run one of the following commands depending on whether
-    the file is a <filename class='extension'>.gz</filename> or <filename
-    class='extension'>.bz2</filename> file:</para>
-
-<screen><userinput>gunzip -v patchname.gz
-bunzip2 -v patchname.bz2</userinput></screen>
+    <para>
+      Finally, sometimes we have a compressed patch file in
+      <filename class='extension'>.patch.gz</filename> or
+      <filename class='extension'>.patch.bz2</filename> format.
+      The best way to apply the patch is piping the output of the
+      decompressor to the <command>patch</command> utility.  For example:
+    </para>
+
+    <screen><userinput>gzip -cd ../patchname.patch.gz | patch -p1</userinput></screen>
+
+    <para>
+      Or for a patch compressed with <command>bzip2</command>:
+    </para>
+
+    <screen><userinput>bzcat ../patchname.patch.bz2 | patch -p1</userinput></screen>
 
   </sect2>
@@ -218 +224 @@
 
     <para>
-      but for ninja, the default number of jobs is &lt;N&gt;+2, where &lt;N&gt;
-      is the number of processors available, so that using the above commands
-      is rather for limiting the number of jobs (see below for why this could
-      be necessary).
+      but for ninja, the default number of jobs is N + 2, if
+      the number of logical processors N is greater than 2; or N + 1 if
+      N is 1 or 2.  The reason to use a number of jobs slightly greater
+      than the number of logical processors is keeping all logical
+      processors busy even if some jobs are performing I/O operatations.
     </para>
 
     <para>Generally the number of processes should not exceed the number of
-    cores supported by the CPU.  To list the processors on your
+    cores supported by the CPU too much.  To list the processors on your
     system, issue: <userinput>grep processor /proc/cpuinfo</userinput>.
     </para>
 
-    <para>In some cases, using multiple processes may result in a 'race'
+    <para>In some cases, using multiple processes may result in a race
     condition where the success of the build depends on the order of the
     commands run by the <command>make</command> program.  For instance, if an
@@ -239 +246 @@
 
     <para>If this occurs, the best way to proceed is to drop back to a
-    single processor build.  Adding '-j1' to a make command will override
-    the similar setting in the <envar>MAKEFLAGS</envar> environment
-    variable.</para>
-
+    single processor build.  Adding <option>-j1</option> to a make command
+    will override the similar setting in the <envar>MAKEFLAGS</envar>
+    environment variable.</para>
+<!-- outdated
     <note><para>When running the package tests or the install portion of the
     package build process, we do not recommend using an option greater than
@@ -248 +255 @@
     have not been validated using parallel procedures and may fail with issues
     that are difficult to debug.</para></note>
-
+-->
     <important>
       <para>
@@ -293 +300 @@
     redirection so that the program uses the data in the file as the answers to
     the questions.</para>
-
+<!-- outdated
     <para>Building the <application>CUPS</application> package is a good
     example of how redirecting a file as input to prompts can help you automate
@@ -303 +310 @@
 
 <screen><userinput>make check &lt; ../cups-1.1.23-testsuite_parms</userinput></screen>
-
+-->
     <para>This effectively makes the test suite use the responses in the file
     as the input to the questions. Occasionally you may end up doing a bit of
@@ -382 +389 @@
     example. First, issue the command:</para>
 
-<screen><userinput>ls -l /usr/bin | more</userinput></screen>
+<screen><userinput>ls -l /usr/bin | less</userinput></screen>
 
     <para>Of course, you'll be required to view the output one page at a time
-    because the <command>more</command> filter was used. Now try the same
+    because the <command>less</command> filter was used. Now try the same
     command, but this time redirect the output to a file. The special file
     <filename>/dev/null</filename> can be used instead of the filename shown,
     but you will have no log file to examine:</para>
 
-<screen><userinput>ls -l /usr/bin | more &gt; redirect_test.log 2&gt;&amp;1</userinput></screen>
+<screen><userinput>ls -l /usr/bin | less &gt; redirect_test.log 2&gt;&amp;1</userinput></screen>
 
     <para>Notice that this time the command immediately returned to the shell
@@ -407 +414 @@
 <literal>#!/bin/bash
 
-ls -l /usr/bin | more
+ls -l /usr/bin | less
 
 echo -n -e "\n\nDid you enjoy reading this? (y,n) "
@@ -703 +710 @@
       the issues caused by some choices (typically slow execution or
       unexpected use of, or omission of, optimizatons) by starting with
-      the CFLAGS and CXXFLAGS environment variables.  There are also some
-      programs which use rust.
-    </para>
-
-    <para>
-      Most LFS and BLFS builders are probably aware of the basics of CFLAGS
-      and CXXFLAGS for altering how a program is compiled. Typically, some
-      form of optimization is used by upstream developers (-O2 or -O3),
-      sometimes with the creation of debug symbols (-g), as defaults.
-    </para>
-
-    <para>
-      If there are contradictory flags (e.g. multiple different -O values),
+      the <envar>CFLAGS</envar>, <envar>CXXFLAGS</envar>, and
+      <envar>LDFLAGS</envar> environment variables.  There are also some
+      programs which use Rust.
+    </para>
+
+    <para>
+      Most LFS and BLFS builders are probably aware of the basics of
+      <envar>CFLAGS</envar> and <envar>CXXFLAGS</envar> for altering how a
+      program is compiled. Typically, some form of optimization is used by
+      upstream developers (<option>-O2</option> or <option>-O3</option>),
+      sometimes with the creation of debug symbols (<option>-g</option>),
+      as defaults.
+    </para>
+
+    <para>
+      If there are contradictory flags (e.g. multiple different
+      <option>-O</option> values),
       the <emphasis>last</emphasis> value will be used. Sometimes this means
       that flags specified in environment variables will be picked up before
       values hardcoded in the Makefile, and therefore ignored.  For example,
-      where a user specifies '-O2' and that is followed by '-O3' the build will
-      use '-O3'.
+      where a user specifies <option>-O2</option> and that is followed by
+      <option>-O3</option> the build will use <option>-O3</option>.
     </para>
 
     <para>
       There are various other things which can be passed in CFLAGS or
-      CXXFLAGS, such as forcing compilation for a specific microarchitecture
-      (e.g. -march=amdfam10, -march=native) or specifying a specific standard
-      for C or C++ (-std=c++17 for example). But one thing which has now come
-      to light is that programmers might include debug assertions in their
-      code, expecting them to be disabled in releases by using -DNDEBUG.
-      Specifically, if <xref linkend="mesa"/> is built with these assertions
-      enabled, some activities such as loading levels of games can take
-      extremely long times, even on high-class video cards.
+      CXXFLAGS, such as allowing using the instruction set extensions
+      available with a specific microarchitecture (e.g.
+      <option>-march=amdfam10</option> or <option>-march=native</option>),
+      tune the generated code for a specific microarchitecture (e. g.
+      <option>-mtune=tigerlake</option> or <option>-mtune=native</option>,
+      if <option>-mtune=</option> is not used, the microarchitecture from
+      <option>-march=</option> setting will be used), or specifying a
+      specific standard for C or C++ (<option>-std=c++17</option> for
+      example).  But one thing which has now come to light is that
+      programmers might include debug assertions in their code, expecting
+      them to be disabled in releases by using <option>-DNDEBUG</option>.
+      Specifically, if <xref linkend="mesa"/> is built with these
+      assertions enabled, some activities such as loading levels of games
+      can take extremely long times, even on high-class video cards.
     </para>
 
@@ -738 +755 @@
 
       <para>
-       This combination is often described as 'CMMI' (configure, make, make
-       install) and is used here to also cover the few packages which have a
-       configure script that is not generated by autotools.
+        This combination is often described as <quote>CMMI</quote>
+        (configure, make, make install) and is used here to also cover
+        the few packages which have a configure script that is not
+        generated by autotools.
       </para>
 
@@ -760 +778 @@
 
       <para>
-       In most CMMI packages, running 'make' will list each command and run
-       it, interspersed with any warnings. But some packages try to be 'silent'
-       and only show which file they are compiling or linking instead of showing
-       the command line. If you need to inspect the command, either because of
-       an error, or just to see what options and flags are being used, adding
-       'V=1' to the make invocation may help.
-     </para>
+        In most CMMI packages, running <command>make</command> will list
+        each command and run it, interspersed with any warnings. But some
+        packages try to be <quote>silent</quote> and only show which file
+        they are compiling or linking instead of showing the command line.
+        If you need to inspect the command, either because of an error, or
+        just to see what options and flags are being used, adding
+        <option>V=1</option> to the make invocation may help.
+      </para>
 
     <bridgehead renderas="sect3" id="cmake-info">CMake</bridgehead>
 
       <para>
-        CMake works in a very different way, and it has two backends which can
-        be used on BLFS: 'make' and 'ninja'. The default backend is make, but
+        CMake works in a very different way, and it has two backends which
+        can be used on BLFS: <command>make</command> and
+        <command>ninja</command>. The default backend is make, but
         ninja can be faster on large packages with multiple processors. To
-        use ninja, specify '-G Ninja' in the cmake command. However, there are
-        some packages which create fatal errors in their ninja files but build
-        successfully using the default of Unix Makefiles.
+        use ninja, specify <option>-G Ninja</option> in the cmake command.
+        However, there are some packages which create fatal errors in their
+        ninja files but build successfully using the default of Unix
+        Makefiles.
       </para>
 
@@ -789 +810 @@
         Perhaps the most-important thing about CMake is that it has a variety
         of CMAKE_BUILD_TYPE values, and these affect the flags. The default
-        is that this is not set and no flags are generated. Any CFLAGS or
-        CXXFLAGS in the environment will be used. If the programmer has coded
-        any debug assertions, those will be enabled unless -DNDEBUG is used.
-        The following CMAKE_BUILD_TYPE values will generate the flags shown,
-        and these will come <emphasis>after</emphasis> any flags in the
-        environment and therefore take precedence.
+        is that this is not set and no flags are generated. Any
+        <envar>CFLAGS</envar> or <envar>CXXFLAGS</envar> in the environment
+        will be used. If the programmer has coded any debug assertions,
+        those will be enabled unless -DNDEBUG is used. The following
+        CMAKE_BUILD_TYPE values will generate the flags shown, and these
+        will come <emphasis>after</emphasis> any flags in the environment
+        and therefore take precedence.
       </para>
 
@@ -845 +867 @@
         details of the defines that you may wish to change you can look at
         <filename>meson_options.txt</filename> which is usually in the
-         top-level directory.
+        top-level directory.
       </para>
 
@@ -872 +894 @@
         <listitem>
           <para>plain : no added flags. This is for distributors to supply their
-          own CLFAGS, CXXFLAGS and LDFLAGS. There is no obvious reason to use
+          own <envar>CFLAGS</envar>, <envar>CXXFLAGS</envar> and
+          <envar>LDFLAGS</envar>. There is no obvious reason to use
           this in BLFS.</para>
         </listitem>
         <listitem>
-          <para>debug : '-g' - this is the default if nothing is specified
-          in either <filename>meson.build</filename> or the command line.
-          However it results large and slow binaries, so we should override
-          it in BLFS.</para>
+          <para>debug : <option>-g</option> - this is the default if
+          nothing is specified in either <filename>meson.build</filename>
+          or the command line. However it results large and slow binaries,
+          so we should override it in BLFS.</para>
         </listitem>
         <listitem>
-           <para>debugoptimized : '-O2 -g' : this is the default specified in
-           <filename>meson.build</filename> of some packages.</para>
+          <para>debugoptimized : <option>-O2 -g</option> - this is the
+          default specified in <filename>meson.build</filename> of some
+          packages.</para>
         </listitem>
         <listitem>
-           <para>release : '-O3 -DNDEBUG' (but occasionally a package will force
-           -O2 here)</para>
+          <para>release : <option>-O3</option> (occasionally a package will
+          force <option>-O2</option> here) - this is the buildtype we use
+          for most packages with Meson build system in BLFS.</para>
         </listitem>
       </itemizedlist>
 
-      <para>
-        Although the 'release' buildtype is described as enabling -DNDEBUG, and all
-        CMake Release builds pass that, it has so far only been observed (in
-        verbose builds) for <xref linkend="mesa"/>. That suggests that it might
-        only be used when there are debug assertions present.
-      </para>
-
-      <para>
-        The -DNDEBUG flag can also be provided by passing
-        <command>-Db_ndebug=true</command>.
+      <!-- From https://mesonbuild.com/Builtin-options.html#core-options:
+           b_ndebug: Default value = false, Possible values are
+           true, false, if-release.  Some packages sets it to if-release
+           so we mistakenly believed if-release had been the default.  -->
+      <para>
+        The <option>-DNDEBUG</option> flag is implied by the release
+        buildtype for some packages (for example <xref linkend='mesa'/>).
+        It can also be provided explicitly by passing
+        <option>-Db_ndebug=true</option>.
       </para>
 
       <para>
         To see the details of the commands which are being run in a package using
-        meson, use 'ninja -v'.
+        meson, use <command>ninja -v</command>.
       </para>
 
@@ -915 +939 @@
         and then download them as necessary.  These packages are built using
         <command>cargo --release</command>. In theory, you can manipulate the
-        RUSTFLAGS to change the optimize-level (default is 3, like -O3, e.g.
-        <literal>-Copt-level=3</literal>) or to force it to build for the
-        machine it is being compiled on, using
-        <literal>-Ctarget-cpu=native</literal> but in practice this seems to
+        RUSTFLAGS to change the optimize-level (default for
+        <option>--release</option> is 3, i. e.
+        <option>-Copt-level=3</option>, like <option>-O3</option>) or to
+        force it to build for the machine it is being compiled on, using
+        <option>-Ctarget-cpu=native</option> but in practice this seems to
         make no significant difference.
       </para>
 
       <para>
-        If you find an interesting rustc program which is only provided as
-        unpackaged source, you should at least specify
-        <literal>RUSTFLAGS=-Copt-level=2</literal> otherwise it will do an
-        unoptimized compile with debug info and run <emphasis>much</emphasis>
-        slower.
-      </para>
-
-      <para>
-        The rust developers seem to assume that everyone will compile on a
-        machine dedicated to producing builds, so by default all CPUs are used.
-        This can often be worked around, either by exporting
-        CARGO_BUILD_JOBS=&lt;N&gt; or passing --jobs &lt;N&gt; to cargo. For
-        compiling rustc itself, specifying --jobs &lt;N&gt; on invocations of
-        x.py (together with the <envar>CARGO_BUILD_JOBS</envar> environment
-        variable, which looks like a "belt and braces" approach but seems to be
-        necessary) mostly works. The exception is running the tests when building
-        rustc, some of them will nevertheless use all online CPUs, at least as of
-        rustc-1.42.0.
+        If you are compiling a standalone Rust program (as an unpackaged
+        <filename class='extension'>.rs</filename> file) by running
+        <command>rustc</command> directly, you should specify
+        <option>-O</option> (the abbreviation of
+        <option>-Copt-level=2</option>) or <option>-Copt-level=3</option>
+        otherwise it will do an unoptimized compile and run
+        <emphasis>much</emphasis> slower.  If are compiling the program
+        for debugging it, replace the <option>-O</option> or
+        <option>-Copt-level=</option> options with <option>-g</option> to
+        produce an unoptimized program with debug info.
+      </para>
+
+      <para>
+        Like <command>ninja</command>, by default <command>cargo</command>
+        uses all logical processors.  This can often be worked around,
+        either by exporting
+        <envar>CARGO_BUILD_JOBS=<replaceable>&lt;N&gt;</replaceable></envar>
+        or passing
+        <option>--jobs <replaceable>&lt;N&gt;</replaceable></option> to
+        <command>cargo</command>.
+        For compiling rustc itself, specifying
+        <option>--jobs <replaceable>&lt;N&gt;</replaceable></option> for
+        invocations of <command>x.py</command>
+        (together with the <envar>CARGO_BUILD_JOBS</envar> environment
+        variable, which looks like a <quote>belt and braces</quote>
+        approach but seems to be necessary) mostly works. The exception is
+        running the tests when building rustc, some of them will
+        nevertheless use all online CPUs, at least as of rustc-1.42.0.
       </para>
 
@@ -950 +985 @@
       <para>
         Many people will prefer to optimize compiles as they see fit, by providing
-        CFLAGS or CXXFLAGS. For an introduction to the options available with gcc
-        and g++ see <ulink
+        <envar>CFLAGS</envar> or <envar>CXXFLAGS</envar>. For an
+        introduction to the options available with gcc and g++ see <ulink
         url="https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html"/> and <ulink
         url="https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html"/>
@@ -959 +994 @@
 
       <para>
-        Some packages default to '-O2 -g', others to '-O3 -g', and if CFLAGS or
-        CXXFLAGS are supplied they might be added to the package's defaults,
-        replace the package's defaults, or even be ignored.  There are details
-        on some desktop packages which were mostly current in April 2019 at
+        Some packages default to <option>-O2 -g</option>, others to
+        <option>-O3 -g</option>, and if <envar>CFLAGS</envar> or
+        <envar>CXXFLAGS</envar> are supplied they might be added to the
+        package's defaults, replace the package's defaults, or even be
+        ignored.  There are details on some desktop packages which were
+        mostly current in April 2019 at
         <ulink url="https://www.linuxfromscratch.org/~ken/tuning/"/> - in
-        particular, README.txt, tuning-1-packages-and-notes.txt, and
-        tuning-notes-2B.txt. The particular thing to remember is that if you
-        want to try some of the more interesting flags you may need to force
-        verbose builds to confirm what is being used.
+        particular, <filename>README.txt</filename>,
+        <filename>tuning-1-packages-and-notes.txt</filename>, and
+        <filename>tuning-notes-2B.txt</filename>. The particular thing to
+        remember is that if you want to try some of the more interesting
+        flags you may need to force verbose builds to confirm what is being
+        used.
       </para>
 
@@ -974 +1013 @@
         profile it and perhaps recode some of it if it is too slow. But for
         building a whole system that approach is impractical. In general,
-        -O3 usually produces faster programs than -O2.  Specifying
-        -march=native is also beneficial, but means that you cannot move the
-        binaries to an incompatible machine - this can also apply to newer
-        machines, not just to older machines. For example programs compiled for
-        'amdfam10' run on old Phenoms, Kaveris, and Ryzens : but programs
-        compiled for a Kaveri will not run on a Ryzen because certain op-codes
-        are not present.  Similarly, if you build for a Haswell not everything
-        will run on a SandyBridge.
-      </para>
+        <option>-O3</option> usually produces faster programs than
+        <option>-O2</option>.  Specifying
+        <option>-march=native</option> is also beneficial, but means that
+        you cannot move the binaries to an incompatible machine - this can
+        also apply to newer machines, not just to older machines. For
+        example programs compiled for <literal>amdfam10</literal> run on
+        old Phenoms, Kaveris, and Ryzens : but programs compiled for a
+        Kaveri will not run on a Ryzen because certain op-codes are not
+        present.  Similarly, if you build for a Haswell not everything will
+        run on a SandyBridge.
+      </para>
+
+      <note>
+        <para>
+          Be careful that the name of a <option>-march</option> setting
+          does not always match the baseline of the microarchitecture
+          with the same name.  For example, the Skylake-based Intel Celeron
+          processors do not support AVX at all, but
+          <option>-march=skylake</option> assumes AVX and even AVX2.
+        </para>
+      </note>
 
       <para>
@@ -992 +1043 @@
       <para>
         If building Perl or Python modules, or Qt packages which use qmake,
-        in general the CFLAGS and CXXFLAGS used are those which were used by
-        those 'parent' packages.
+        in general the <envar>CFLAGS</envar> and <envar>CXXFLAGS</envar>
+        used are those which were used by those <quote>parent</quote>
+        packages.
       </para>