source: general/prog/rust.xml@ 2d0650b

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE sect1 PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
   "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
  <!ENTITY % general-entities SYSTEM "../../general.ent">
  %general-entities;

  <!ENTITY rust-download-http "https://static.rust-lang.org/dist/rustc-&rust-version;-src.tar.gz">
  <!ENTITY rust-download-ftp  " ">
  <!ENTITY rust-md5sum        "e67432b37150f13e186ebfb67e5192fb">
  <!ENTITY rust-size          "147 MB">
  <!-- Gentle Reminder: buildsize is how much the user requires for the real
       install, i.e. the source with its DESTDIR *plus* the DESTDIR.  You
       can 'mkdir /tmp/RUST ; cp -a install/* /tmp/RUST' and then run 'du -sch'
       to measure it. -->

  <!ENTITY rust-buildsize     "5.3GB (375 MB installed) including 429MB of ~/.cargo files for the user building this. Add 1.7 GB if running the tests">
  <!ENTITY rust-time          "24 SBU (add 13 SBU for tests, both with 4 processors)">
]>

<sect1 id="rust" xreflabel="rustc-&rust-version;">
  <?dbhtml filename="rust.html"?>

  <sect1info>
    <othername>$LastChangedBy$</othername>
    <date>$Date$</date>
  </sect1info>

  <title>Rustc-&rust-version;</title>

  <indexterm zone="rust">
    <primary sortas="a-rust">Rust</primary>
  </indexterm>

  <sect2 role="package">
    <title>Introduction to Rust</title>

    <para>
      The <application>Rust</application> programming language is designed
      to be a safe, concurrent, practical language.
    </para>

    <para>
      This package is updated on a six-weekly release cycle. Because it is
      such a large and slow package to build, and is at the moment only required
      by a few packages in this book, the BLFS editors take the view that it
      should only be updated when that is necessary (either to fix problems,
      or to allow a new version of <application>firefox</application> to build).
    </para>

    <para>
      As with many  other programming languages, rustc (the rust compiler)
      needs a binary from which to bootstrap. It will download a stage0 binary
      and many cargo crates (these are actually .tar.gz source archives) at
      the start of the build, so you cannot compile it without an internet
      connection.
    </para>

    <para>
      These crates will then remain in various forms (cache, directories of
      extracted source), in <filename class="directory">~/.cargo</filename> for
      ever more. It is common for large <application>rust</application> packages
      to use multiple versions of some crates. If you purge the files before
      updating this package, very few crates will need to be updated by the
      packages in this book which use it (and they will be downloaded as
      required). But if you retain an older version as a fallback option and
      then use it (when <emphasis>not</emphasis> building in
      <filename class="directory">/usr</filename>), it is likely that it will
      then have to re-download some crates. For a full download (i.e. starting
      with an empty or missing <filename class="directory">~/.cargo</filename>)
      downloading the external cargo files for this version only takes a minute
      or so on a fast network.
    </para>

    <note>
      <para>
        Although BLFS usually installs in <filename
        class="directory">/usr</filename>, when you later upgrade to a newer
        version of <application>rust</application> the old libraries in <filename
        class="directory">/usr/lib/rustlib</filename> will remain, with various
        hashes in their names, but will not be usable and will waste space. The
        editors recommend placing the files in the <filename
        class="directory">/opt</filename> directory.  In particular, if you
        have reason to rebuild with a modified configuration (e.g. using the
        shipped LLVM after building with shared LLVM, but perhaps also the
        reverse situation) it it possible for the install to leave a broken
        <command>cargo</command> program. In such a situation, either remove
        the existing installation first, or use a different prefix such as
        /opt/rustc-&rust-version;-build2.
      </para>

      <para>
        If you prefer, you can of course change the prefix to <filename
        class="directory">/usr</filename> and omit the
        <command>ldconfig</command> and the actions to add rustc to the PATH.
      </para>
    </note>

    <para>
      The current <application>rustbuild</application> build-system will use
      all available processors, although it does not scale well and often falls
      back to just using one core while waiting for a library to compile.
    </para>

    <para>
      At the moment <application>Rust</application> does not provide any
      guarantees of a stable ABI.
    </para>

    <note>
      <para>
        Rustc defaults to building for ALL supported architectures, using a
        shipped copy of LLVM. In BLFS the build is only for the X86
        architecture. Rustc still claims to require Python 2, but that is only
        really necessary when building some other architectures with the
        shipped LLVM.
        If you intend to develop rust crates, this build may not be good
        enough for your purposes.
      </para>
      <para>
        The build times of this version when repeated on the same machine are
        often reasonably consistent, but as with all compilations using
        <command>rustc</command> there can be some very slow outliers.
      </para>
      <para>
        Unusually, a DESTDIR-style method is being used to install this package.
        This is because running the install as root not only downloads all of the
        cargo files again (to <filename>/root/.cargo</filename>), it then spends
        a very long time recompiling. Using this method saves a lot of time, at
        the cost of extra disk space.
      </para>
    </note>

    &lfs90_checked;

    <bridgehead renderas="sect3">Package Information</bridgehead>
    <itemizedlist spacing="compact">
      <listitem>
        <para>
          Download (HTTP): <ulink url="&rust-download-http;"/>
        </para>
      </listitem>
      <listitem>
        <para>
          Download (FTP): <ulink url="&rust-download-ftp;"/>
        </para>
      </listitem>
      <listitem>
        <para>
          Download MD5 sum: &rust-md5sum;
        </para>
      </listitem>
      <listitem>
        <para>
          Download size: &rust-size;
        </para>
      </listitem>
      <listitem>
        <para>
          Estimated disk space required: &rust-buildsize;
        </para>
      </listitem>
      <listitem>
        <para>
          Estimated build time: &rust-time;
        </para>
      </listitem>
    </itemizedlist>

    <bridgehead renderas="sect3">Additional Downloads</bridgehead>
    <itemizedlist spacing="compact">
      <listitem>
        <para>
          Required patch:
          <ulink url="&patch-root;/rustc-&rust-version;-llvm9_fixes-1.patch"/>
        </para>
      </listitem>
    </itemizedlist>

    <bridgehead renderas="sect3">Rust Dependencies</bridgehead>

    <bridgehead renderas="sect4">Required</bridgehead>
    <para role="required">
      <xref linkend="curl"/>,
      <xref linkend="cmake"/>, and
      <xref linkend="libssh2"/>
    </para>

    <bridgehead renderas="sect4">Recommended</bridgehead>
    <para role="recommended">
      <package>clang</package> from <xref linkend="llvm"/>
      (built with -DLLVM_LINK_LLVM_DYLIB=ON so that rust can link to
      system LLVM instead of building its shipped version)
    </para>

    <bridgehead renderas="sect4">Optional</bridgehead>
    <para role="optional">
      <xref linkend="gdb"/> (used by the testsuite if it is present) and
      <xref linkend="python2"/> (used by the testsuite)
    </para>

    <para condition="html" role="usernotes">
      User Notes: <ulink url="&blfs-wiki;/rust"/>
    </para>
  </sect2>

  <sect2 role="installation">
    <title>Installation of Rust</title>

    <para>
      To install into the
      <filename class="directory">/opt</filename> directory, remove the symlink
      and create a new directory (i.e. with a different name if trying a
      modified build).
      As the <systemitem class="username">root</systemitem>
      user:
    </para>

<screen role="root"><userinput>mkdir /opt/rustc-&rust-version;             &amp;&amp;
ln -svfin rustc-&rust-version; /opt/rustc</userinput></screen>

    <note>
      <para>
        If multiple versions of <application>Rust</application> are installed
        in <filename class="directory">/opt</filename>, changing to another
        version only requires changing the <filename> /opt/rustc</filename>
        symbolic link and then running <command>ldconfig</command>.
      </para>
    </note>

    <para>
      Create a suitable <filename>config.toml</filename> file which will
      configure the build.
    </para>

<screen><userinput>cat &lt;&lt; EOF &gt; config.toml
<literal># see config.toml.example for more possible options
# See the 8.4 book for an example using shipped LLVM
# e.g. if not installing clang, or using a version before 8.0.
[llvm]
# by default, rust will build for a myriad of architectures
targets = "X86"

# When using system llvm prefer shared libraries
link-shared = true

[build]
# omit docs to save time and space (default is to build them)
docs = false

# install cargo as well as rust
extended = true

[install]
prefix = "/opt/rustc-&rust-version;"
docdir = "share/doc/rustc-&rust-version;"

[rust]
channel = "stable"
rpath = false

# BLFS does not install the FileCheck executable from llvm,
# so disable codegen tests
codegen-tests = false

[target.x86_64-unknown-linux-gnu]
# NB the output of llvm-config (i.e. help options) may be
# dumped to the screen when config.toml is parsed.
llvm-config = "/usr/bin/llvm-config"

[target.i686-unknown-linux-gnu]
# NB the output of llvm-config (i.e. help options) may be
# dumped to the screen when config.toml is parsed.
llvm-config = "/usr/bin/llvm-config"

</literal>
EOF</userinput></screen>

    <para>
     Adapt rustc to allow it to build and function with the changes in LLVM-9:
    </para>

<screen><userinput remap="pre">patch -Np1 -i ../rustc-&rust-version;-llvm9_fixes-1.patch</userinput></screen>

    <para>
      Now compile <application>Rust</application> by running the following
      commands:
    </para>

<screen><userinput>export RUSTFLAGS="$RUSTFLAGS -C link-args=-lffi" &amp;&amp;
python3 ./x.py build --exclude src/tools/miri</userinput></screen>

    <note>
      <para>
        The testsuite will generate some messages in the
        <phrase revision="sysv">system log</phrase>
        <phrase revision="systemd">systemd journal</phrase>
        for traps on invalid opcodes, and for segmentation faults.
        In themselves these are nothing to worry about, just a way for the
        test to be terminated.
      </para>
    </note>

    <para>
      To run the tests issue <command>python3 ./x.py test --verbose
      --no-fail-fast | tee rustc-testlog</command>: as with the build, that
      will use all available CPUs.
    </para>

    <para>
      The instructions above do not build ARM compilers, so the testsuite
      <emphasis>will</emphasis> fail and the tests will be reported to end in
      error, with a backtrace of the last failing test. On a good run, 3 tests
      which need Thumb (ARM) compilers will fail, all in <filename
      class="directory">ui/issues</filename> for issues 37131, 49851 and 50993.
      A fourth test,
      <filename>run-make-fulldeps/sysroot-crates-are-unstable</filename>
      fails, presumably because we are using only stable features.
      <!-- appears to be fixed now
      If gdb has been installed, in some circumstances tests in
      <filename class="directory">debuginfo</filename> may fail.--> As with all
      large testsuites, other tests might
      fail on some machines - if the number of failures is in the single digits,
      check the log for 'FAILED' and review lines above that, particularly the
      'stderr:' lines. Any mention of
      SIGSEGV or signal 11 in a failing test is a cause for concern.
      A fifth test, run-make-fulldeps/linker-output-non-utf8, is known to fail.
    </para>

    <para>
      Therefore, you should determine the number of tests, failures, etc. The
      total number of tests which were considered is found by running:
    </para>

<screen remap="test"><userinput>grep 'running .* tests' rustc-testlog | awk '{ sum += $2 } END { print sum }'</userinput></screen>

    <para>
      That should report 16499 tests. Similarly, the total tests which failed can
      be found by running:
    </para>

<screen remap="test"><userinput>grep '^test result:' rustc-testlog | awk  '{ sum += $6 } END { print sum }'</userinput></screen>

    <para>
      And similarly for the tests which passed use $4, for those which were ignored
      (i.e. skipped) use $8 (and $10 for 'measured', $12 for 'filtered out' but both
      are probably zero). The breakdown does not quite match the overall total.
    </para>

    <para>
      Still as your normal user, do a DESTDIR install:
    </para>

<screen><userinput>export LIBSSH2_SYS_USE_PKG_CONFIG=1 &amp;&amp;
DESTDIR=${PWD}/install python3 ./x.py install &amp;&amp;
unset LIBSSH2_SYS_USE_PKG_CONFIG</userinput></screen>

    <para>
      Now, as the <systemitem class="username">root</systemitem> user
      install the files from the DESTDIR:
    </para>

<screen role="root"><userinput>chown -R root:root install &amp;&amp;
cp -a install/* /</userinput></screen>

  </sect2>

  <sect2 role="commands">
    <title>Command Explanations</title>

    <para>
      <command>ln -svfn rustc-&rust-version; /opt/rustc</command>: if this is
      not the first use of the <filename class="directory">/opt/rustc</filename>
      symlink, overwrite it by forcing, and use the '-n' flag to avoid getting
      confusing results from e.g. <command>ls -l</command>.
    </para>

    <para>
      <command>targets = "X86"</command>: this avoids building all the available
      linux cross-compilers (Aarch64, MIPS, PowerPC, SystemZ, etc). Unfortunately,
      rust insists on installing source files for these below
      <filename class="directory">/opt/rustc/lib/src</filename>.
    </para>

    <para>
      <command>extended = true</command>: this installs Cargo alongside Rust.
    </para>

    <para>
      <command>channel = "stable"</command>: this ensures only stable features
      can be used, the default in <filename>config.toml</filename> is to use
      development features, which is not appropriate for a released version.
    </para>

    <para>
      <command>rpath = false</command>: by default, <command>rust</command> can
      be run from where it was built, without being installed. That adds DT_RPATH
      entries to all of the ELF files, which produces very messy output from
      <command>ldd</command>, showing the libraries in the place they were built,
      even if they have been deleted from there after the install.
    </para>

    <!-- comment while using shipped LLVM
    <para>
      <command>[target.x86_64-unknown-linux-gnu]</command>: the syntax of
      <filename>config.toml</filename> requires an <literal>llvm-config</literal>
      entry for each target for which system-llvm is to be used. Change the target
      to <literal>[target.i686-unknown-linux-gnu]</literal> if you are building
      on 32-bit x86. This whole section may be omitted if you wish to build
      against the shipped llvm, or do not have clang, but the resulting build will
      be larger and take longer.
    </para>-->

    <para>
      <command>export RUSTFLAGS="$RUSTFLAGS -C link-args=-lffi"</command>:
      This adds a link to libffi to any RUSTFLAGS you may already be passing
      to the build. On some systems, linking fails to include libffi unless
      this is used. The reason why this is needed is not clear.
    </para>

    <para>
      <command>--exclude src/tools/miri</command>: For a long time, the miri
      crate (an interpreter for the Midlevel Intermediate Representation)
      has failed to build on releases. It is optional, but the failure
      messages can persuade people that the whole build failed.  However,
      although it is not built in the main compile, with rustc-1.35.0 it
      now gets compiled during the install, but it  is broken in this version.
      <!-- might be unbroken in 1.36.0, if so remove broken from description
       of miri below. https://github.com/rust-lang/rust/issues/61830 -->
    </para>

    <para>
      <command>--verbose</command>: this switch can sometimes provide more
      information about a test which fails.
    </para>

    <para>
      <command>--no-fail-fast</command>: this switch ensures that the testsuite
      will not stop at the first error.
    </para>

    <para>
      <command>export LIBSSH2_SYS_USE_PKG_CONFIG=1</command>: On some systems,
      cairo fails to link during the install because it cannot find libssh2.
      This seems to fix it, but again the reason why the problem occurs is not
      understood.
    </para>

    <para>
      <command>DESTDIR=${PWD}/install python3 ./x.py install</command>: This
      effects a DESTDIR-style install in the source tree,creating an <filename
      class="directory">install</filename> directory. Note that DESTDIR installs
      need an absolute path, passing 'install' will not work.
    </para>

    <para>
      <command>chown -R root:root install</command>: the DESTDIR install
      was run by a regular user, who owns the files. For security, change their
      owner before doing a simple copy to install them.
    </para>

  </sect2>

  <sect2 role="configuration">
    <title>Configuring Rust</title>

    <sect3 id="rustc-config">
      <title>Configuration Information</title>

      <para>
        If you installed <application>rustc</application> in
        <filename class="directory">/opt</filename>, you need to update the
        following configuration files so that <application>rustc</application> is
        correctly found by other packages and system processes.
      </para>

      <para>
        As the <systemitem class="username">root</systemitem> user, update
        the <filename>/etc/ld.so.conf</filename> file and the dynamic linker's
        run-time cache file:
      </para>

<screen role="root"><userinput>cat &gt;&gt; /etc/ld.so.conf &lt;&lt; EOF
<literal># Begin rustc addition

/opt/rustc/lib

# End rustc addition</literal>
EOF

ldconfig</userinput></screen>

      <indexterm zone="rustc rustc-config">
        <primary sortas="e-etc-ld.so.conf">/etc/ld.so.conf</primary>
      </indexterm>

      <para>
        As the <systemitem class="username">root</systemitem> user, create
        the <filename>/etc/profile.d/rustc.sh</filename> file:
      </para>

<screen role="root"><userinput>cat &gt; /etc/profile.d/rustc.sh &lt;&lt; "EOF"
<literal># Begin /etc/profile.d/rustc.sh

pathprepend /opt/rustc/bin           PATH

# End /etc/profile.d/rustc.sh</literal>
EOF</userinput></screen>

      <para>Immediately after installation, update the current PATH 
      for your current shell as a normal user:</para>

<screen><userinput>source /etc/profile.d/rustc.sh</userinput></screen>

    </sect3>
  </sect2>


  <sect2 role="content">
    <title>Contents</title>

    <segmentedlist>
      <segtitle>Installed Programs</segtitle>
      <segtitle>Installed Libraries</segtitle>
      <segtitle>Installed Directories</segtitle>

      <seglistitem>
        <seg>
          cargo-clippy, cargo-fmt, cargo-miri, cargo, clippy-driver, miri, rls, rust-gdb, rust-lldb, rustc, rustdoc, rustfmt.
        </seg>
        <seg>
          Many lib*&lt;16-byte-hash&gt;.so libraries.
        </seg>
        <seg>
          ~/.cargo,
          /opt/rustc/lib/rustlib,
          /opt/rustc/share/doc/rustc-&rust-version;, and
          /opt/rustc/share/zsh/site-functions
        </seg>
      </seglistitem>
    </segmentedlist>

    <variablelist>
      <bridgehead renderas="sect3">Short Descriptions</bridgehead>
      <?dbfo list-presentation="list"?>
      <?dbhtml list-presentation="table"?>

      <varlistentry id="cargo-clippy">
        <term><command>cargo-clippy</command></term>
        <listitem>
          <para>
            provides lint checks for a cargo package.
          </para>
          <indexterm zone="rust cargo-clippy">
            <primary sortas="b-cargo-clippy">cargo-clippy</primary>
          </indexterm>
        </listitem>
      </varlistentry>

      <varlistentry id="cargo-fmt">
        <term><command>cargo-fmt</command></term>
        <listitem>
          <para>
            formats all bin and lib files of the current crate using
            rustfmt.
          </para>
          <indexterm zone="rust cargo-fmt">
            <primary sortas="b-cargo-fmt">cargo-fmt</primary>
          </indexterm>
        </listitem>
      </varlistentry>

      <varlistentry id="cargo-miri">
        <term><command>cargo-miri</command></term>
        <listitem>
          <para>
            <!-- FIXME reword to 'is used by' if Miri installed
             AND works enough to report its \-\-help -->
            is for use by Miri to interpret bin crates and tests
          </para>
          <indexterm zone="rust cargo-miri">
            <primary sortas="b-cargo-miri">cargo-miri</primary>
          </indexterm>
        </listitem>
      </varlistentry>

      <varlistentry id="cargo">
        <term><command>cargo</command></term>
        <listitem>
          <para>
            is the Package Manager for Rust.
          </para>
          <indexterm zone="rust cargo">
            <primary sortas="b-cargo">cargo</primary>
          </indexterm>
        </listitem>
      </varlistentry>

      <varlistentry id="clippy-driver">
        <term><command>clippy-driver</command></term>
        <listitem>
          <para>
            provides lint checks for Rust.
          </para>
          <indexterm zone="rust clippy-driver">
            <primary sortas="b-clippy-driver">clippy-driver</primary>
          </indexterm>
        </listitem>
      </varlistentry>

      <varlistentry id="miri">
        <term><command>miri</command></term>
        <listitem>
          <para>
            is an interpreter for Rust's mid-level intermediate representation
            (MIR). It is broken in this version.
          </para>
          <indexterm zone="rust miri">
            <primary sortas="b-miri">miri</primary>
          </indexterm>
        </listitem>
      </varlistentry>

      <varlistentry id="rls">
        <term><command>rls</command></term>
        <listitem>
          <para>
            is the Rust Language Server. This can run in the background to
            provide IDEs, editors, and other tools with information about Rust
            programs.
          </para>
          <indexterm zone="rust rls">
            <primary sortas="b-rls">rls</primary>
          </indexterm>
        </listitem>
      </varlistentry>

      <varlistentry id="rust-gdb">
        <term><command>rust-gdb</command></term>
        <listitem>
          <para>
            is a wrapper script for gdb, pulling in Python
            pretty-printing modules installed in <filename
            class="directory">/usr/lib/rustlib/etc</filename>.
          </para>
          <indexterm zone="rust rust-gdb">
            <primary sortas="b-rust-gdb">rust-gdb</primary>
          </indexterm>
        </listitem>
      </varlistentry>

      <varlistentry id="rust-lldb">
        <term><command>rust-lldb</command></term>
        <listitem>
          <para>
            is a wrapper script for LLDB (the LLVM debugger)
            pulling in the Python pretty-printing modules.
          </para>
          <indexterm zone="rust rust-lldb">
            <primary sortas="b-rust-lldb">rust=lldb</primary>
          </indexterm>
        </listitem>
      </varlistentry>

      <varlistentry id="rustc">
        <term><command>rustc</command></term>
        <listitem>
          <para>
            is the rust compiler.
          </para>
          <indexterm zone="rust rustc">
            <primary sortas="b-rustc">rustc</primary>
          </indexterm>
        </listitem>
      </varlistentry>

      <varlistentry id="rustdoc">
        <term><command>rustdoc</command></term>
        <listitem>
          <para>
            generates documentation from rust source code.
          </para>
          <indexterm zone="rust rustdoc">
            <primary sortas="b-rustdoc">rustdoc</primary>
          </indexterm>
        </listitem>
      </varlistentry>

      <varlistentry id="rustfmt">
        <term><command>rustfmt</command></term>
        <listitem>
          <para>
            formats rust code.
          </para>
          <indexterm zone="rust rustfmt">
            <primary sortas="b-rustfmt">rustfmt</primary>
          </indexterm>
        </listitem>
      </varlistentry>

      <varlistentry id="libstd">
        <term><filename class="libraryfile">libstd-&lt;16-byte-hash&gt;.so</filename></term>
        <listitem>
          <para>
            is the Rust Standard Library, the foundation of portable Rust software.
          </para>
          <indexterm zone="rust libstd">
            <primary sortas="c-libstd">libstd-&lt;16-byte-hash&gt;.so</primary>
          </indexterm>
        </listitem>
      </varlistentry>
    </variablelist>
  </sect2>
</sect1>