Changeset 8fd509c

07/24/2019 03:31:03 AM (2 years ago)
Ken Moffat <ken@…>
10.0, 10.1, 11.0, 9.0, 9.1, ken/refactor-virt, lazarus, qt5new, trunk, xry111/git-date, xry111/git-date-for-trunk, xry111/git-date-test

Expand Notes on Building Software.

git-svn-id: svn:// af4574ff-66df-0310-9fd7-8a98e5e911e0

2 edited


  • introduction/important/building-notes.xml

    r58b66af3 r8fd509c  
    463463      options and their default values) differ. It may be easiest to understand
    464464      the issues caused by some choices (typically slow execution or
    465       unexpected use of, or omission of, optimizatons) by starting with
    466       the CFLAGS and CXXFLAGS environment variables.
     465      unexpected use of, or omission of, optimizatons) by starting with
     466      the CFLAGS and CXXFLAGS environment variables.  There are also some
     467      programs which use rust.
    467468    </para>
    619620        </listitem>
    620621        <listitem>
    621            <para>release : '-O3 -DNDEBUG'</para>
     622           <para>release : '-O3 -DNDEBUG' (but occasionally a package will force
     623           -O2 here)</para>
    622624        </listitem>
    623625      </itemizedlist>
    640642      </para>
     644    <bridgehead renderas="sect3" id="rust-info">Rustc and Cargo</bridgehead>
     646      <para>
     647        Most released rustc programs are provided as crates (source tarballs)
     648        which will query a server to check current versions of dependencies
     649        and then download them as necessary.  These packages are built using
     650        <command>cargo --release</command>. In theory, you can manipulate the
     651        RUSTFLAGS to change the optimize-level (default is 3, like -O3, e.g.
     652        <literal>-Copt-level=3</literal>) or to force it to build for the
     653        machine it is being compiled on, using
     654        <literal>-Ctarget-cpu=native</literal> but in practice this seems to
     655        make no significant difference.
     656      </para>
     658      <para>
     659        If you find an interesting rustc program which is only provided as
     660        unpackaged source, you should at least specify
     661        <literal>RUSTFLAGS=-Copt-level=2</literal> otherwise it will do an
     662        unoptimized compile with debug info and run <emphasis>much</emphasis>
     663        slower.
     664      </para>
     666  </sect2>
     668  <sect2 id="optimizations">
     669    <title>Optimizing the build</title>
     671      <para>
     672        Many people will prefer to optimize compiles as they see fit, by providing
     673        CFLAGS or CXXFLAGS. For an introduction to the options available with gcc
     674        and g++ see <ulink
     675        url=""/> and <ulink
     676        url=""/>
     677        and <command>info gcc</command>.
     679      </para>
     681      <para>
     682        Some packages default to '-O2 -g', others to '-O3 -g', and if CFLAGS or
     683        CXXFLAGS are supplied they might be added to the package's defaults,
     684        replace the package's defaults, or even be ignored.  There are details
     685        on some desktop packages which were mostly current in April 2019 at
     686        <ulink url=""/> - in
     687        particular, README.txt, tuning-1-packages-and-notes.txt, and
     688        tuning-notes-2B.txt. The particular thing to remember is that if you
     689        want to try some of the more interesting flags yo may need to force
     690        verbose builds to confirm what is being used.
     691      </para>
     693      <para>
     694        Clearly, if you are optimizing your own program you can spend time to
     695        profile it and perhaps recode some of it if it is too slow. But for
     696        building a whole system that approach is impractical. In general,
     697        -O3 usually produces faster programs than -O2.  Specifying
     698        -march=native is also beneficial, but means that you cannot move the
     699        binaries to an incompatible machine - this can also apply to newer
     700        machines, not just to older machines. For example programs compiled for
     701        'amdfam10' run on old Phenoms, Kaveris, and Ryzens : but programs
     702        compiled for a Kaveri will not run on a Ryzen because certain op-codes
     703        are not present.  Similarly, if you build for a Haswell not everything
     704        will run on a SandyBridge.
     705      </para>
     707      <para>
     708        There are also various other options which some people claim are
     709        beneficial. At worst, you get to recompile and test, and then
     710        discover that in your usage the options do not provide a benefit.
     711      </para>
     713      <para>
     714        If building Perl or Python modules, or Qt packages which use qmake,
     715        in general the CFLAGS and CXXFLAGS used are those which were used by
     716        those 'parent' packages.
     717      </para>
     719  </sect2>
     721  <sect2 id="hardening">
     722    <title>Options for hardening the build</title>
     724      <para>
     725        Even on desktop systems, there are still a lot of exploitable
     726        vulnerabilities. For many of these, the attack comes via javascript
     727        in a browser. Often, a seris of vulnerabilities are used to gain
     728        access to data (or sometimes to pwn, i.e. own, the machine and
     729        install rootkits).  Most commercial distros will apply various
     730        hardening measures.
     731      </para>
     733      <para>
     734        For hardening options which are reasonably cheap, there is some
     735        discussion in the 'tuning' link above (occasionally, one or more
     736        of these options might be inappropriate for a package). These
     737        options are -D_FORTIFY_SOURCE=2, -fstack-protector=strong, and
     738        (for C++) -D_GLIBCXX_ASSERTIONS. On modern machines these should
     739        only have a little impact on how fast things run, and often they
     740        will not be noticeable.
     741      </para>
     743      <para>
     744        In the past, there was Hardened LFS where gcc (a much older version)
     745        was forced to use hardening (with options to turn some of it off on a
     746        per-package basis. What is being covered here is different - first you
     747        have to make sure that the package is indeed using your added flags and
     748        not over-riding them.
     749      </para>
     751      <para>
     752        The main distros use much more, such as RELRO (Relocation Read Only)
     753        and perhaps -fstack-clash-protection. You may also encounter the
     754        so-called 'userspace retpoline' (-mindirect-branch=thunk etc.) which
     755        is the equivalent of the spectre mitigations applied to the linux
     756        kernel in late 2018). The kernel mitigations casue a lot of complaints
     757        about lost performance, if you have a production server you might wish
     758        to consider testing that, along with the other available options, to
     759        see if performance is still sufficient.
     760      </para>
     762      <para>
     763        Whilst gcc has many hardening options, clang/LLVM's strengths lie
     764        elsewhere. Some options which gcc provides are said to be less effective
     765        in clang/LLVM, others are not available.
     766      </para>
    642768  </sect2>
  • introduction/welcome/changelog.xml

    r58b66af3 r8fd509c  
    4545      <para>July 23rd, 2019</para>
    4646      <itemizedlist>
     47        <listitem>
     48          <para>[ken] - In 'Notes on Building Software' add notes on rustc/cargo,
     49          optimizations, and hardening.</para>
     50        </listitem>
    4751        <listitem>
    4852          <para>[ken] - Update to firefox-68.0.1 and adapt it to API changes
Note: See TracChangeset for help on using the changeset viewer.