rustc-1.66.1

[ken@…]

From oss-security:

The Rust Security Response WG was notified that Cargo did not perform SSH host key verification when cloning indexes and dependencies via SSH. An attacker could exploit this to perform man-in-the-middle (MITM) attacks.

This vulnerability has been assigned CVE-2022-46176.

## Overview

When an SSH client establishes communication with a server, to prevent MITM attacks the client should check whether it already communicated with that server in the past and what the server's public key was back then. If the key changed since the last connection, the connection must be aborted as a MITM attack is likely taking place.

It was discovered that Cargo never implemented such checks, and performed no validation on the server's public key, leaving Cargo users vulnerable to MITM attacks. ## Affected Versions

All Rust versions containing Cargo before 1.66.1 are vulnerable.

Note that even if you don't explicitly use SSH for alternate registry indexes or crate dependencies, you might be affected by this vulnerability if you have configured git to replace HTTPS connections to GitHub with SSH (through git's [url.<base>.insteadOf][1] setting), as that'd cause you to clone the crates.io index through SSH.

## Mitigations

We will be releasing Rust 1.66.1 today, 2023-01-10, changing Cargo to check the SSH host key and abort the connection if the server's public key is not already trusted. We recommend everyone to upgrade as soon as possible.

Patch files for Rust 1.66.0 are also available [here][2] for custom-built toolchains.

For the time being Cargo will not ask the user whether to trust a server's public key during the first connection. Instead, Cargo will show an error message detailing how to add that public key to the list of trusted keys. Note that this might break your automated builds if the hosts you clone dependencies or indexes from are not already trusted.

## Acknowledgments

Thanks to the Julia Security Team for disclosing this to us according to our [security policy][1]! We also want to thank the members of the Rust project who contributed to fixing this issue. Thanks to Eric Huss and Weihang Lo for writing and reviewing the patch, Pietro Albini for coordinating the disclosure and writing this advisory, and Josh Stone, Josh Triplett and Jacob Finkelman for advising during the disclosure.

[ken@…]

Tests all pass, as with 1.64.1, but install fails:

thread 'main' panicked at 'missing miri', install.rs:205:14
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

This appears to be ​https://github.com/rust-lang/rust/issues/105816 which says that miri is not built in stable.

Claimed to be fixed, but obviously not in the 1.66 series.

The obvious original fix is ​https://github.com/rust-lang/rust/pull/105940/commits/e5c92bc2b6f0bd69856ad3c4d3056c7f0c0ad72d but trying to paste that into what is in 1.66.1 does not match what is there and I have no idea how to fix it.

Giving up.

[Bruce Dubbs]

I pulled the source and it built without issue: Build completed successfully in 0:09:27

But during the tests I got:

Traceback (most recent call last):
  File "/build/rustc/rustc-1.66.1-src/./x.py", line 28, in <module>
    bootstrap.main()
  File "/build/rustc/rustc-1.66.1-src/src/bootstrap/bootstrap.py", line 936, in main
    bootstrap(help_triggered)
  File "/build/rustc/rustc-1.66.1-src/src/bootstrap/bootstrap.py", line 922, in bootstrap
    run(args, env=env, verbose=build.verbose, is_bootstrap=True)
  File "/build/rustc/rustc-1.66.1-src/src/bootstrap/bootstrap.py", line 166, in run
    raise RuntimeError(err)
RuntimeError: failed to run: /build/rustc/rustc-1.66.1-src/build/bootstrap/debug/bootstrap test --verbose --no-fail-fast

I then ran 'build/bootstrap/debug/bootstrap test --verbose --no-fail-fast' and it seemed to complete OK.

Re-running 'python3 ./x.py test --verbose --no-fail-fast | tee rustc-testlog' still failed in the same way as above.

I then tried running just 'python3 ./x.py test' and got:

--- stderr
/build/rustc/rustc-1.66.1-src/build/x86_64-unknown-linux-gnu/stage1/bin/rustc: error while loading shared libraries: librustc_driver-70b5e3e437214976.so: cannot open shared object file: No such file or directory
', src/tools/compiletest/src/common.rs:472:13

I can't figure it out tonight. Maybe tomorrow.

[ken@…]

Replying to Bruce Dubbs:

I pulled the source and it built without issue: Build completed successfully in 0:09:27

But during the tests I got:

Traceback (most recent call last):
  File "/build/rustc/rustc-1.66.1-src/./x.py", line 28, in <module>
    bootstrap.main()
  File "/build/rustc/rustc-1.66.1-src/src/bootstrap/bootstrap.py", line 936, in main
    bootstrap(help_triggered)
  File "/build/rustc/rustc-1.66.1-src/src/bootstrap/bootstrap.py", line 922, in bootstrap
    run(args, env=env, verbose=build.verbose, is_bootstrap=True)
  File "/build/rustc/rustc-1.66.1-src/src/bootstrap/bootstrap.py", line 166, in run
    raise RuntimeError(err)
RuntimeError: failed to run: /build/rustc/rustc-1.66.1-src/build/bootstrap/debug/bootstrap test --verbose --no-fail-fast

I then ran 'build/bootstrap/debug/bootstrap test --verbose --no-fail-fast' and it seemed to complete OK.

Re-running 'python3 ./x.py test --verbose --no-fail-fast | tee rustc-testlog' still failed in the same way as above.

I then tried running just 'python3 ./x.py test' and got:

--- stderr
/build/rustc/rustc-1.66.1-src/build/x86_64-unknown-linux-gnu/stage1/bin/rustc: error while loading shared libraries: librustc_driver-70b5e3e437214976.so: cannot open shared object file: No such file or directory
', src/tools/compiletest/src/common.rs:472:13

I can't figure it out tonight. Maybe tomorrow.

Your build time seems excessively fast. For me, the build is ok, and yes the tests do seem to report an error like that. But using the book's attempt to work out the test results I had

11546 were claimed to be run
11334 tests passed
0 tests failed
212 tests ignored

The problem is durign the install.

Don't have my previous test log any more, have just started another build (below).

[ken@…]

Not sleeping, so back on the chain-gang.

I diff'd from 1.66.0 to 1.66.1 - a LOT of changes, but nothing in bootstrap/install.rs. Then I downloaded 1.65.0 which I knew build and installed ok, and diff'd just that one file - the only change is one hunk around the line which now rejects.

Now trying with a patch which reverts that.

[pierre]

I downloaded the fix and for me it applies (with some fuzz)... Could it be that you got some <space> vs <tab> problem?

[pierre]

Replying to Bruce Dubbs:

I pulled the source and it built without issue: Build completed successfully in 0:09:27

But during the tests I got:

Traceback (most recent call last):
  File "/build/rustc/rustc-1.66.1-src/./x.py", line 28, in <module>
    bootstrap.main()
  File "/build/rustc/rustc-1.66.1-src/src/bootstrap/bootstrap.py", line 936, in main
    bootstrap(help_triggered)
  File "/build/rustc/rustc-1.66.1-src/src/bootstrap/bootstrap.py", line 922, in bootstrap
    run(args, env=env, verbose=build.verbose, is_bootstrap=True)
  File "/build/rustc/rustc-1.66.1-src/src/bootstrap/bootstrap.py", line 166, in run
    raise RuntimeError(err)
RuntimeError: failed to run: /build/rustc/rustc-1.66.1-src/build/bootstrap/debug/bootstrap test --verbose --no-fail-fast

I then ran 'build/bootstrap/debug/bootstrap test --verbose --no-fail-fast' and it seemed to complete OK.

Re-running 'python3 ./x.py test --verbose --no-fail-fast | tee rustc-testlog' still failed in the same way as above.

I then tried running just 'python3 ./x.py test' and got:

--- stderr
/build/rustc/rustc-1.66.1-src/build/x86_64-unknown-linux-gnu/stage1/bin/rustc: error while loading shared libraries: librustc_driver-70b5e3e437214976.so: cannot open shared object file: No such file or directory
', src/tools/compiletest/src/common.rs:472:13

I can't figure it out tonight. Maybe tomorrow.

Guess it needs some LD_LIBRARY_PATH setting...

[pierre]

Replying to pierre:

Replying to Bruce Dubbs:

I pulled the source and it built without issue: Build completed successfully in 0:09:27

But during the tests I got:

Traceback (most recent call last):
  File "/build/rustc/rustc-1.66.1-src/./x.py", line 28, in <module>
    bootstrap.main()
  File "/build/rustc/rustc-1.66.1-src/src/bootstrap/bootstrap.py", line 936, in main
    bootstrap(help_triggered)
  File "/build/rustc/rustc-1.66.1-src/src/bootstrap/bootstrap.py", line 922, in bootstrap
    run(args, env=env, verbose=build.verbose, is_bootstrap=True)
  File "/build/rustc/rustc-1.66.1-src/src/bootstrap/bootstrap.py", line 166, in run
    raise RuntimeError(err)
RuntimeError: failed to run: /build/rustc/rustc-1.66.1-src/build/bootstrap/debug/bootstrap test --verbose --no-fail-fast

I then ran 'build/bootstrap/debug/bootstrap test --verbose --no-fail-fast' and it seemed to complete OK.

Re-running 'python3 ./x.py test --verbose --no-fail-fast | tee rustc-testlog' still failed in the same way as above.

I then tried running just 'python3 ./x.py test' and got:

--- stderr
/build/rustc/rustc-1.66.1-src/build/x86_64-unknown-linux-gnu/stage1/bin/rustc: error while loading shared libraries: librustc_driver-70b5e3e437214976.so: cannot open shared object file: No such file or directory
', src/tools/compiletest/src/common.rs:472:13

I can't figure it out tonight. Maybe tomorrow.

Guess it needs some LD_LIBRARY_PATH setting..

Two points:

• the "error while loading shared libraries" occurs while running a test. This test is then reported to fail. But this test is about "long linker lines" so the failure may come from anything and not be related to a bad library path.
• I've looked at file "/build/rustc/rustc-1.66.1-src/src/bootstrap/bootstrap.py", around line 166. The reason why the RuntimeError is raised is just because the tests exit with non zero status. This is expected, since some tests failed. But for some reason, when running a bootstrap, the rust devs have decided not to print the error (because it has already been printed) ... Hence the cryptic message. All in all this is not something to worry about.

[ken@…]

Replying to pierre:

I downloaded the fix and for me it applies (with some fuzz)... Could it be that you got some <space> vs <tab> problem?

Very possible, I could not see any option to download it from github so I tried to paste it - but the code looked very different. I'm planning to use my own patch which merely reverts the miri part (one successful build of that, one disk-full on the other machine).

[ken@…]

Replying to pierre:

Replying to pierre:

Replying to Bruce Dubbs:

Two points:

• the "error while loading shared libraries" occurs while running a test. This test is then reported to fail. But this test is about "long linker lines" so the failure may come from anything and not be related to a bad library path.
• I've looked at file "/build/rustc/rustc-1.66.1-src/src/bootstrap/bootstrap.py", around line 166. The reason why the RuntimeError is raised is just because the tests exit with non zero status. This is expected, since some tests failed. But for some reason, when running a bootstrap, the rust devs have decided not to print the error (because it has already been printed) ... Hence the cryptic message. All in all this is not something to worry about.

Thanks for that explanation.

[Bruce Dubbs]

The discussion of the tests needs to be expanded a bit. It's a bit confusing to have the python traceback right at the end of the output. To me it suggests that the test procedure failed before the end.

I also suggest that the awk portion of the line checking for failures be changed to something like:

awk '{sum1 += $4; sum2 += $6} END { print sum1 " passed; " sum2 " failed" }'

With that I get:

11334 passed; 0 failed

[ken@…]

I think we need a comment that the the tests will appear to fail. Thinking about the awk change.

Still trying to do a no-tests build, after saving the details from the build with tests, I forgot to remove /opt/rustc-1.66.1 (my script checks that does not exist, as a sanity check).

[ken@…]

The presence, or absence, of gdb no-longer seems to make a difference to the test results. On both systems I got the same totals:

11546 were claimed to be run
11334 tests passed
0 tests failed
212 tests ignored

In my log for the build, the only referenced to gdb are from extracting files such as rust-gdb when untarring 1.65.0 at the start. In the output from the tests there is a line for

* omit_gdb_pretty_printer_section  lang     unstable     1.5.0   

which is in a long list without any explanation - I thought at first it was a list of the error codes lacking tests, but that says there are 506 error codes and the list is a lot longer than that.

As with so much of rust, I'm unclear if gdb is actually used in the current tests.

[ken@…]

Looking at the source, the following test files mention gdb in their names, so I guess it is still used.

./src/test/codegen/gdb_debug_script_load.rs
./src/test/debuginfo/gdb-char.rs
./src/test/debuginfo/gdb-pretty-struct-and-enums.rs
./src/test/ui/feature-gates/feature-gate-omit-gdb-pretty-printer-section.stderr
./src/test/ui/feature-gates/feature-gate-omit-gdb-pretty-printer-section.rs

[pierre]

Replying to ken@…:

Replying to pierre:

I downloaded the fix and for me it applies (with some fuzz)... Could it be that you got some <space> vs <tab> problem?

Very possible, I could not see any option to download it from github so I tried to paste it - but the code looked very different.

Weird. It looked the same to me (maybe shifted one line or so).

I'm planning to use my own patch which merely reverts the miri part (one successful build of that, one disk-full on the other machine).

On github, to download the raw patch, take the url of the displayed one, and add ".patch". In this case

wget https://github.com/rust-lang/rust/pull/105940/commits/e5c92bc2b6f0bd69856ad3c4d3056c7f0c0ad72d.patch

should download it.

[Bruce Dubbs]

Replying to ken@…:

Replying to Bruce Dubbs:

I pulled the source and it built without issue: Build completed successfully in 0:09:27

Your build time seems excessively fast.

I was building with all 24 cores.

[ken@…]

Replying to Bruce Dubbs:

Replying to ken@…:

Replying to Bruce Dubbs:

I pulled the source and it built without issue: Build completed successfully in 0:09:27

Your build time seems excessively fast.

I was building with all 24 cores.

;-)

I don't think I've enough memory to try that on my best machine, and anyways that one is now mostly used for other things.

[ken@…]

Replying to pierre:

I'm planning to use my own patch which merely reverts the miri part (one successful build of that, one disk-full on the other machine).

On github, to download the raw patch, take the url of the displayed one, and add ".patch". In this case

wget https://github.com/rust-lang/rust/pull/105940/commits/e5c92bc2b6f0bd69856ad3c4d3056c7f0c0ad72d.patch

should download it.

Thanks, I keep forgetting how to do that. Got it now, it adds 8 lines (mine adds 7) and the context at the end differs. The extra line is the comment 'Miri is only available on nightly'. I thought the formatting in front of this had seemed different, but it was the altered context at the end which really threw me.

Firefox-102.7.0 first candidate builds.

Still not sure about the severity of the advisory - for the moment I'm not inclined to update to 1.66.1 on existing systems, but maybe I'm misreading upstream's advice ?

[pierre]

About tests: I get "16 command(s) did not execute successfully", although the total of failed tests is 0 when using the awk line from the book. It looks like upstream have changed their error reporting or something like that. Not all tests lead to a "test result" line, I guess.

[pierre]

I wrote something wrong about the "error while loading shared libraries": It makes a difference to export LD_LIBRARY_PATH=./build/x86_64-unknown-linux-gnu/stage1/lib/. Then I get only

3 command(s) did not execute successfully

and I get (using Bruce's proposition for the awk line)

26673 passed; 49 failed

Note that there are still instances of "error while loading shared libraries" even with the setting above.

[ken@…]

Replying to pierre:

About tests: I get "16 command(s) did not execute successfully", although the total of failed tests is 0 when using the awk line from the book. It looks like upstream have changed their error reporting or something like that. Not all tests lead to a "test result" line, I guess.

(I see you have replied while I was preparing this)

These seem to be failures to open a shared library:

ken@deluxe ~ $grep 'cannot open shared object file' /home/logs/LFS-git-20221228/desktop-rustc-1.66.1/rustc-1.66.1-src.log.check | sort -u

/scratch/working/rustc-1.66.1-src/build/x86_64-unknown-linux-gnu/stage1/bin/rustc: error while loading shared libraries: librustc_driver-70b5e3e437214976.so: cannot open shared object file: No such file or directory
/scratch/working/rustc-1.66.1-src/build/x86_64-unknown-linux-gnu/stage1/bin/rustdoc: error while loading shared libraries: libtest-b18d53981b5bc577.so: cannot open shared object file: No such file or directory

ken@deluxe ~ $grep 'cannot open shared object file' /home/logs/LFS-git-20221228/desktop-rustc-1.66.1/rustc-1.66.1-src.log.check | wc -l
16

That is 15 instances for librustc_driver-70b5e3e437214976.so and one for libtest-b18d53981b5bc577.so.

The 15 all reference src/tools/compiletest/src/common.rs:472:13, the libtest file is for something in rustdoc.

common.rs:472:13 a hunk of code in 'impl TargetCfg' to apparently panic after not getting the config for the target, the !output.status.success part of

        {
            Ok(output) => output,
            Err(e) => panic!("error: failed to get cfg info from {:?}: {e}", rustc_path),
        };
        if !output.status.success() {
            panic!(
                "error: failed to get cfg info from {:?}\n--- stdout\n{}\n--- stderr\n{}",
                rustc_path,
                String::from_utf8(output.stdout).unwrap(),
                String::from_utf8(output.stderr).unwrap(),
            );
        }

Those 15 each follow a '^{Check compiletest ' line with a whole load of parameters including the suite name, mode, lots of parameters, and then configuration information.}

Looks as if something in how it retrieves an item in the configuration information is not working. Among the many parms in the compiletest invocations I notice

 "--suite" "mir-opt" "--mode" "mir-opt"

which makes me suspect that this might be related to the assumption that miri is always present, but of course it could be that mir is not miri.

Not sure how best to describe these.

[ken@…]

Trying to get more info on the severity of the vulnerability, I got to the (last two) comments at ​https://lobste.rs/s/rwwglc/cve_2022_46176_cargo_does_not_check_ssh : it appears from those that an unusual setup is needed and therefore while the severity is indeed high, for most people it has no relevance.

[pierre]

Replying to ken@…:

Trying to get more info on the severity of the vulnerability, I got to the (last two) comments at ​https://lobste.rs/s/rwwglc/cve_2022_46176_cargo_does_not_check_ssh : it appears from those that an unusual setup is needed and therefore while the severity is indeed high, for most people it has no relevance.

Maybe we shouldn't rely on the fact that lfs users don't do unusual things: building linux from scratch is an unusual thing ;)

[ken@…]

Replying to pierre:

Replying to ken@…:

Trying to get more info on the severity of the vulnerability, I got to the (last two) comments at ​https://lobste.rs/s/rwwglc/cve_2022_46176_cargo_does_not_check_ssh : it appears from those that an unusual setup is needed and therefore while the severity is indeed high, for most people it has no relevance.

Maybe we shouldn't rely on the fact that lfs users don't do unusual things: building linux from scratch is an unusual thing ;)

We should update. What I mean is that the advisory should be clear about the limited circumstances in which the vulnerability exists.

Meanwhile, was running tests on my fast machine to look at the issues, thought I could build in /tmp for speed, filled it up. I suppose I'd better admit that I'm intending to do it.

[ken@…]

With that LD_LIBRARY_PATH I agree the totals are very different. On this not-all-latest machine (mid october) I get 26556 passed, 50 failed using the commands in the book.

I'll look at that, and the detail in the testlog, tomorrow or later depending on how things work out here.

[ken@…]

Replying to pierre:

I wrote something wrong about the "error while loading shared libraries": It makes a difference to export LD_LIBRARY_PATH=./build/x86_64-unknown-linux-gnu/stage1/lib/.

[snip]

Note that there are still instances of "error while loading shared libraries" even with the setting above.

I'm now looking at my testlog. The first linkage failure in my case is failing to link librustc_driver-70b5e3e437214976.so : that is in stage0 and stage1 while librustc_driver-08f69ab3ef246cd6.so is in stage1, stage2 and the final install (reminder: I did the install first to review how much space was added by the tests).

I'm also seeing tests which fail after trying to link static libs: /usr/lib/libm-2.36.a and /usr/lib/libmvec.a (two of each so far).

Using grep '^{test result:' rustc-testlog | grep -v ' 0 failed;' I can see that each result of interest has a large FAILED in it:}

test result: FAILED. 13587 passed; 1 failed; 160 ignored; 0 measured; 0 filtered out; finished in 121.15s
test result: FAILED. 0 passed; 47 failed; 99 ignored; 0 measured; 0 filtered out; finished in 0.29s
test result: FAILED. 212 passed; 1 failed; 21 ignored; 0 measured; 0 filtered out; finished in 25.49s
test result: FAILED. 42 passed; 1 failed; 26 ignored; 0 measured; 0 filtered out; finished in 2.45s

In checking the config.toml options last night (example no-longer provided) I think we can delete the first line.

As for the tests which always used to fail because we only build for native target, they made a change some months ago to default to only running tests for the target.

Posting now, before someone else posts here. Going to look at those FAILED items.

[ken@…]

OK, I think I've got the results for my build.

1. I have only ONE instance of 'error while loading shared libraries:', for

      [run-make] src/test/run-make-fulldeps/long-linker-command-lines
   

which tries to use the stage0 library I mentioned in my previous commit. I now regard that as a broken test (if all the others use stage1, it should do the same).

2. Link failures for static libs /usr/lib/libm-2.36.a and /usr/lib/libmvec.a result in

   [ui] src/test/ui/process/nofile-limit.rs
   [run-make] src/test/run-make/static-pie
   

I'm not sure if those two are both available in vanilla LFS ?

3. The 47 failures in 'assembly' :

The reason is well hidden in the voluminous output - retracing to one of them while I was half-way through the batch I found:

rc/test/assembly/stack-protector/stack-protector-heuristics-effect.rs#missing' panicked at 'failed to exec `"/usr/bin/FileCheck"

So all 47 tests of [assembly] fail because we do not install FileCheck.

I would be interested to known, if the details are still available, which one of these 50 failures is specific to this build of mine (compared to the 49 above) ?

[ken@…]

I'll also mention that neither Arch (llvm with added wasm, no tests of course) and gentoo do not need to patch the source to get it to build. Probably I'm missing something that caused me to fall down the same hole as the sparc builder who flagged the problem for 1.66.0 ? (oh, and if anyone finds this while searching, the MSVC fix appears to be in the 1.66.1 changes).

[pierre]

About patch, cannot tell... It fails for me too at install without the patch. For the tests: using an absolute path for the library, and copying the FileCheck file from LLVM to /usr/bin, I get the following grand totals:

26722 passed; 0 failed; 587 ignored

[pierre]

Completely unrelated, but I think we should add changelog-seen = 2 at the beginning of config.toml: x.py prints a warning at each invocation otherwise (twice: when beginning and when ending processing).

[pierre]

Maybe arch do not need the patch because they build a limited set of tools: in config.toml, they have:

tools = ["cargo", "rls", "clippy", "rustfmt", "analysis", "src", "rust-demangler"]

And gentoo also pass tools =, that may or may not contain miri (depending on their use flags). So it seems to me that we might want to remove the --exclude src/tools/miri, and pass tools into config.toml (what to put into tools is a good question :)

[ken@…]

Replying to pierre:

Maybe arch do not need the patch because they build a limited set of tools: in config.toml, they have:

tools = ["cargo", "rls", "clippy", "rustfmt", "analysis", "src", "rust-demangler"]

And gentoo also pass tools =, that may or may not contain miri (depending on their use flags). So it seems to me that we might want to remove the --exclude src/tools/miri, and pass tools into config.toml (what to put into tools is a good question :)

Thanks. For librustc_driver-70b5e3e437214976.so I can only say that I was in a maze of twisty hashed libraries, mostly alike. I now agree that it *is* also in stage1.

The two static glibc libs are down to me. Will be rerunning later.

Will look at tools =, but I can tell you now that rls was deprecated and is no-longer installed ​https://blog.rust-lang.org/2022/07/01/RLS-deprecation.html

[ken@…]

Replying to pierre:

About patch, cannot tell... It fails for me too at install without the patch. For the tests: using an absolute path for the library, and copying the FileCheck file from LLVM to /usr/bin, I get the following grand totals:

26722 passed; 0 failed; 587 ignored

You mean I need to rebuild LLVM to get the assembly tests to pass ?

[ken@…]

Replying to pierre:

Completely unrelated, but I think we should add changelog-seen = 2 at the beginning of config.toml: x.py prints a warning at each invocation otherwise (twice: when beginning and when ending processing).

Do you have a link for current documentation of config.toml in rust ? The old examples have been replaced, the new examples in src/bootstrap/defaults only show a few items and I found the new changelog unhelpful.

[ken@…]

Replying to ken@…:

Replying to pierre:

Completely unrelated, but I think we should add changelog-seen = 2 at the beginning of config.toml: x.py prints a warning at each invocation otherwise (twice: when beginning and when ending processing).

Do you have a link for current documentation of config.toml in rust ? The old examples have been replaced, the new examples in src/bootstrap/defaults only show a few items and I found the new changelog unhelpful.

Sorry, found it ​https://github.com/rust-lang/rust/blob/master/config.toml.example.

[pierre]

Tried two things:

• build without patch and without the --exclude miri option: failure at install
• build without patch, with the above tools = ... setting (from arch). No failure.

[pierre]

Replying to pierre:

Tried two things:

• build without patch and without the --exclude miri option: failure at install
• build without patch, with the above tools = ... setting (from arch). No failure.

Note that arch does not have "extended = true", which, according to config.toml.example (in the top level directory) is needed for tools to be used.

I'm also trying running the tests with "rpath = true" (the default). This is said to allow to use the tools from the build directory...

[pierre]

Replying to ken@…:

Replying to pierre:

About patch, cannot tell... It fails for me too at install without the patch. For the tests: using an absolute path for the library, and copying the FileCheck file from LLVM to /usr/bin, I get the following grand totals:

26722 passed; 0 failed; 587 ignored

You mean I need to rebuild LLVM to get the assembly tests to pass ?

Well, that's what I did (didn't install, just copied the file after running ninja). I suppose a suitable target passed to ninja could just build what is needed for FileCheck. OTOH, I think we should add the copy of FileCheck to the LLVM instructions, so that next time you (or somebody else) build LLVM, you (or somebody else) get the file...

[ken@…]

OK, so I was looking in src/bootstrap instead of top-level directory for the example.

That currently has

#tools = ["cargo", "rls", "clippy", "rustfmt", "analysis", "src"] # + "rust-demangler" if `profiler`

Which is the full set and does match the current upstream link - obviously out of date now that rls is no longer installed. From the link above about its deprecation: " Builds of RLS will continue to be released until at least the Rust 1.64 release (2022-09-22), after which no new releases will be made. This timeline may change if any issues arise."

Specifying them all does not affect miri. From the ticket re the install failure, miri is not built in stable although the option is accepted. I don't think there is any point in specifying the default tools.

(sorry if this is turning into a 'stream of consciousness')

[ken@…]

minimum llvm version: version_check in src/bootstrap/native.rs, currently 13.0. (affects the comment on when to use the shipped LLVM)

[pierre]

Replying to pierre:

Replying to pierre:

Tried two things:

• build without patch and without the --exclude miri option: failure at install
• build without patch, with the above tools = ... setting (from arch). No failure.

Note that arch does not have "extended = true", which, according to config.toml.example (in the top level directory) is needed for tools to be used.

I can confirm the extended = true line is not needed. The tools = [...] line is enough for a successful install.

I'm also trying running the tests with "rpath = true" (the default). This is said to allow to use the tools from the build directory...

With FileCheck installed and removing the rpath = false line from config.toml, there is no failure in tests.

[pierre]

Replying to ken@…:

OK, so I was looking in src/bootstrap instead of top-level directory for the example.

That currently has

#tools = ["cargo", "rls", "clippy", "rustfmt", "analysis", "src"] # + "rust-demangler" if `profiler`

Which is the full set and does match the current upstream link - obviously out of date now that rls is no longer installed. From the link above about its deprecation: " Builds of RLS will continue to be released until at least the Rust 1.64 release (2022-09-22), after which no new releases will be made. This timeline may change if any issues arise."

Specifying them all does not affect miri. From the ticket re the install failure, miri is not built in stable although the option is accepted. I don't think there is any point in specifying the default tools.

I suspect it does affect miri in the sense that if tools is set in config.toml, the install does not try to install it (which generates a panic if not found), even without the patch...

(sorry if this is turning into a 'stream of consciousness')

Well, I'm ok with the patch too. Just wanted to understand why arch and gentoo do not need a patch. BTW, in gentoo, miri can be added to the set of tools (not in the doc, visibly)!

[pierre]

Replying to pierre:

Replying to ken@…:

Replying to pierre:

About patch, cannot tell... It fails for me too at install without the patch. For the tests: using an absolute path for the library, and copying the FileCheck file from LLVM to /usr/bin, I get the following grand totals:

26722 passed; 0 failed; 587 ignored

You mean I need to rebuild LLVM to get the assembly tests to pass ?

Well, that's what I did (didn't install, just copied the file after running ninja). I suppose a suitable target passed to ninja could just build what is needed for FileCheck. OTOH, I think we should add the copy of FileCheck to the LLVM instructions, so that next time you (or somebody else) build LLVM, you (or somebody else) get the file...

You do not have to rebuild the full LLVM. Just extract llvm and llvm-cmake, create the build dir and change to it, run the cmake command, then run ninja bin/FileCheck. You get the FileCheck file built in about one minute (on the Haswell).

[pierre]

Summary of what we have found:

• Passing RUSTFLAGS is not needed anymore
• Passing LIBSSH2_SYS_USE_PKG_CONFIG=1 is still needed for cargo to use libssh2.
• Removing the rpath = false line from config.toml and installing FileCheck from LLVM allows the tests to pass
• Alternatively to removing the rpath = false, export LD_LIBRARY_PATH=<absolute path> (see above)
• to fix the miri problem at install: either add a tools = line or apply the patch

[ken@…]

Replying to pierre:

You do not have to rebuild the full LLVM. Just extract llvm and llvm-cmake, create the build dir and change to it, run the cmake command, then run ninja bin/FileCheck. You get the FileCheck file built in about one minute (on the Haswell).

Thanks, I've already built the version from 15.0.6 to confirm where it lives, will add that to the LLVM build.

And anyone who has already build llvm-15 (e.g. me on my other desktops) can follow these instructions if I remember to mention it in the changelog.

[ken@…]

Replying to pierre:

Summary of what we have found:

• Passing RUSTFLAGS is not needed anymore
• Passing LIBSSH2_SYS_USE_PKG_CONFIG=1 is still needed for cargo to use libssh2.
• Removing the rpath = false line from config.toml and installing FileCheck from LLVM allows the tests to pass
• Alternatively to removing the rpath = false, export LD_LIBRARY_PATH=<absolute path> (see above)
• to fix the miri problem at install: either add a tools = line or apply the patch

ISTR we eventually decided we wanted to disable rpath, so I'll keep it set to false.

My reading of the ticket for the patch is that miri is not built in the stable channel (but the option is valid if not building for the stable channel). Which item in 'tools = ' causes it to be ignored, please ?

[ken@…]

I tried removing '--exclude src/tools/miri' and omitting the patch: install still failed.

thread 'main' panicked at 'missing miri', install.rs:205:14

[pierre]

Replying to ken@…:

Replying to pierre:

Summary of what we have found:

• Passing RUSTFLAGS is not needed anymore
• Passing LIBSSH2_SYS_USE_PKG_CONFIG=1 is still needed for cargo to use libssh2.
• Removing the rpath = false line from config.toml and installing FileCheck from LLVM allows the tests to pass
• Alternatively to removing the rpath = false, export LD_LIBRARY_PATH=<absolute path> (see above)
• to fix the miri problem at install: either add a tools = line or apply the patch

ISTR we eventually decided we wanted to disable rpath, so I'll keep it set to false.

My reading of the ticket for the patch is that miri is not built in the stable channel (but the option is valid if not building for the stable channel). Which item in 'tools = ' causes it to be ignored, please ?

Setting 'tools' to anything is enough, I think. The ticket does not mention it, I agree, but both arch and gentoo set 'tools', and they don't need a patch. In gentoo, tools can be set to only [ "cargo" ] if no "use" flags are set, or [ "cargo", "clippy", "miri", "rust-demangler", "rustfmt", "rust-analyzer", "analysis", "src" ] if all the corresponding "use" flags are set. Note that I don't know where the documentation of those settings is!

[ken@…]

Replying to pierre:

Setting 'tools' to anything is enough, I think. The ticket does not mention it, I agree, but both arch and gentoo set 'tools', and they don't need a patch. In gentoo, tools can be set to only [ "cargo" ] if no "use" flags are set, or [ "cargo", "clippy", "miri", "rust-demangler", "rustfmt", "rust-analyzer", "analysis", "src" ] if all the corresponding "use" flags are set. Note that I don't know where the documentation of those settings is!

Well spotted. I guess that in rust, as in most projects including our own, documentation is often out of date or missing.

Omitting the patch and using

tools = ["cargo", "clippy", "rustfmt", "analysis", "src", "rust-demangler"]

completed, and the installed programs are: cargo, cargo-clippy, cargo-fmt, clippy-driver, rust-demangler, rust-gdb, rust-gdbgui, rust-lldb, rustc, rustdoc, rustfmt. I was going to say that was the complete set, but it is NOT. Missing rust-analyzer. <sigh/>.

Maybe enabling 'rls', despite that no-longer getting installed, enables rust-analyzer ? Another build coming up.

[ken@…]

Nope. I get the feeling that the 'how to navigate the source' file is "in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard.'”

Now trying to specify rust-analyzer directly.

And this makes me wonder what "analysis" and "src" actually enable.

[pierre]

Replying to ken@…:

Nope. I get the feeling that the 'how to navigate the source' file is "in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard.'”

Now trying to specify rust-analyzer directly.

And this makes me wonder what "analysis" and "src" actually enable.

Only thing I can tell is that the gentoo use flag analysis enables both "analysis" and "rust-analyzer" in tools. Let me make a build with

tools = ["cargo", "clippy", "rustfmt", "analysis", "src", "rust-demangler", "rust-analyzer"]

and see what I get in ${prefix}/bin

Got cargo, cargo-clippy, cargo-fmt, clippy-driver, rust-analyzer, rust-demangler, rustfmt, in addition to rust, rustdoc, rust-gdb, rust-gdbgui, and rust-lldb, that are the only binaries installed when extended=false.

"src" installs the source for rust in ${prefix}/lib/rustlib/src. "analysis" installs .json files with same names as libraries in ${prefix}/lib/rustlib/x86_64-unknown-linux-gnu/analysis.

[ken@…]

Thanks, I was just about to reply after looking at a series of builds.

(Edited to v2, changes at bottom)

I agree with what you say: Specifying 'rust-demangler' adds that. Adding 'analysis' adds json and rlb rust-analysis files formerly used by rls and maybe other old source code tools, a likely candidate for eventual removal, blog on speeding up the compiler ​PR101841. Adding 'rust-src' adds a local copy of the standard library source.

Looking at ​https://rust-lang.github.io/rustup/concepts/components.html the entry for rust-src says "rust-std — This is the Rust standard library. There is a separate rust-std component for each target that rustc supports, such as rust-std-x86_64-pc-windows-msvc. See the Cross-compilation chapter for more detail."

The page lists the various items and then points to the profiles page ​https://rust-lang.github.io/rustup/concepts/profiles.html : Quoting that selectively:

rustup has the concept of "profiles". They are groups of components you can choose to download while installing a new Rust toolchain. The profiles available at this time are minimal, default, and complete.

The minimal profile includes as few components as possible to get a working compiler (rustc, rust-std, and cargo).

The default profile includes all of components in the minimal profile, and adds rust-docs, rustfmt, and clippy. This profile will be used by rustup by default, and it's the one recommended for general use.

(end selective quotes)

From that, I think that we want a smaller build, with: rustc, rust-std, cargo, rustfmt, clippy. Of those, rustc and rust-std are not tools.

That should make a significantly smaller build.

v2: omit rust-docs, we do not currently specify that, clarify that neither rustc nor rust-std are tools.

[ken@…]

With 'tools = ["cargo", "clippy", "rustfmt"]' the -j16 build with tests was a bit smaller and 4 SBU faster. No change to the number of tests that passed.

Will try -j4 non-test and test builds on non-optimized haswell. Progs: cargo, cargo-clippy, cargo-fmt, clippy-driver, rustc, rustdoc, rustfmt,rust-gdb, rust-gdbgui, rust-lldb.

[ken@…]

On the machine where I ran those experimental builds I lack gdb, 26910 tests passed. On the machine where I measured with -j4, 27026 tests passed (0 failures in both cases), so gdb is definitely tested.

Starting to prepare the book changes.

[ken@…]

Updated in 7baeffeaadd6ed9e0bfa2ef470e5eb50d23828b3 11.2-803

Security Advisory to follow, hopefully within 24 hours.

[Xi Ruoyao]

Hi Ken,

I'd suggest to add

# don't download crates during build
locked-deps = true
vendor = true

into [build] section. But maybe it is too late now (I just came back from a not-so-happy trip).

[Xi Ruoyao]

Replying to Xi Ruoyao:

Hi Ken,

I'd suggest to add

# don't download crates during build
locked-deps = true
vendor = true

into [build] section. But maybe it is too late now (I just came back from a not-so-happy trip).

Note that this will also remove the necessity to use DESTDIR installation.

[pierre]

Not really needed, but could be nice: I also suggest to add:

description = "for BLFS &version;"

into the [rust] section: The description is added to the version when running rustc --version.

[ken@…]

Replying to Xi Ruoyao:

Hi Ken,

I'd suggest to add

# don't download crates during build
locked-deps = true
vendor = true

into [build] section. But maybe it is too late now (I just came back from a not-so-happy trip).

Sorry the trip was like that. For me, this is too late - I like to understand what the changes will do, and to try them. I've no plan to again build rust in the coming days (I've built it more times than I have fingers in the last week). Pierre's reply to this suggests that install can also change with this.

I've created #17520 for this and the description.

[ken@…]

At ​https://nvd.nist.gov/vuln/detail/CVE-2022-46176 the CVE is awaiting analysis, with an initial severity of Medium set by GitHub. SA-11.2-066 created.