Opened 3 years ago

Closed 3 years ago

#11975 closed enhancement (fixed)

rustc-1.35.0 (use system LLVM again)

Reported by: Xi Ruoyao Owned by: ken@…
Priority: normal Milestone: 9.0
Component: BOOK Version: SVN
Severity: normal Keywords:
Cc:

Description (last modified by Xi Ruoyao)

Change History (23)

comment:1 by Xi Ruoyao, 3 years ago

Description: modified (diff)

comment:2 by ken@…, 3 years ago

I regard rust as an absolute pain, both to build, and to use (e.g. at the moment I just get 'Error 101' when trying to build firefox 67 *beta*, but I had the same for much for 66 beta and that was ok before the release. So, I am reluctant to upgrade unnecessarily (ff67beta as of last week said that our current version was adequate).

But when we do have to upgrade, system llvm should certainly be tried again.

And when that happens, we also need to make sure that all the packages in the book which use rust are ok with the newer version.

in reply to:  2 comment:3 by Xi Ruoyao, 3 years ago

Replying to ken@…:

I regard rust as an absolute pain, both to build, and to use (e.g. at the moment I just get 'Error 101' when trying to build firefox 67 *beta*, but I had the same for much for 66 beta and that was ok before the release. So, I am reluctant to upgrade unnecessarily (ff67beta as of last week said that our current version was adequate).

I feel just like you. If I was skilled enough I would fork librsvg and remove all rust code.

But when we do have to upgrade, system llvm should certainly be tried again.

And when that happens, we also need to make sure that all the packages in the book which use rust are ok with the newer version.

Unfortunately I don't (and don't want to) build FF so I can't test this.

Last edited 3 years ago by Xi Ruoyao (previous) (diff)

comment:4 by Xi Ruoyao, 3 years ago

Summary: rustc-1.34.0 (use system LLVM again)rustc-1.34.1 (use system LLVM again)

Now 1.34.1.

This patch release fixes two false positives and a panic when checking macros in Clippy. Clippy is a tool which provides a collection of lints to catch common mistakes and improve your Rust code.

comment:5 by Douglas R. Reno, 3 years ago

Any ideas if this fixes the Skylake-specific PANIC when compiling audioipc-client in Firefox? I have to wait for the build to fail, and then restart it for it to finish. That has made preparing the update to 66.0.5 VERY difficult, to the point where I'm just now finishing verification with the Telemetry fix installed.

comment:6 by Douglas R. Reno, 3 years ago

11:51.09 [style 0.0.1] cargo:rerun-if-changed=/sources/firefox-66.0.5/firefox-66.0.5/firefox-build-dir/dist/include/js/TracingAPI.h
11:51.09 thread 'main' panicked at 'stack backtrace:
11:51.13    0:     0x557ddc373b6f - std::sys::unix::backtrace::tracing::imp::unwind_backtrace::he883b608a4bf6182
11:51.13                                at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
11:51.13    1:     0x557ddc365437 - std::sys_common::backtrace::print::hd9f260d7e660310a
11:51.13                                at src/libstd/sys_common/backtrace.rs:71
11:51.13                                at src/libstd/sys_common/backtrace.rs:59
11:51.13    2:     0x557ddc35cc1d - std::panicking::default_hook::{{closure}}::hcf6ec4be1606b352
11:51.13                                at src/libstd/panicking.rs:211
11:51.13    3:     0x557ddc35c994 - std::panicking::default_hook::hefef34c6159e3976
11:51.13                                at src/libstd/panicking.rs:227
11:51.13    4:     0x557ddc35d310 - std::panicking::rust_panic_with_hook::h74b2e2d37e4365c0
11:51.13                                at src/libstd/panicking.rs:491
11:51.13    5:     0x557ddc35cea1 - std::panicking::continue_panic_fmt::h1858f37bef5c109d
11:51.13                                at src/libstd/panicking.rs:398
11:51.13    6:     0x557ddc35cdee - std::panicking::begin_panic_fmt::h387a630da6f0833a
11:51.13                                at src/libstd/panicking.rs:353
11:51.13    7:     0x557ddc370143 - std::io::stdio::_print::h1a9fce701e810e37
11:51.13                                at src/libstd/io/stdio.rs:700
11:51.13                                at src/libstd/io/stdio.rs:709
11:51.13    8:     0x557ddbf89fa3 - cargo::core::compiler::job_queue::JobQueue::drain_the_queue::h76cc92eb08a3c302
11:51.13    9:     0x557ddc018774 - std::panicking::try::do_call::h901a62bb772ddb40
11:51.13   10:     0x557ddc385f39 - __rust_maybe_catch_panic
11:51.13                                at src/libpanic_unwind/lib.rs:102
11:51.13   11:     0x557ddc01818d - crossbeam_utils::thread::scope::ha3d66debb4d260c8
11:51.13   12:     0x557ddbf82364 - cargo::core::compiler::context::Context::compile::h376c5760e14831aa
11:51.13   13:     0x557ddbdc3c2b - cargo::ops::cargo_compile::compile_ws::h657461bd4cc5264f
11:51.13   14:     0x557ddbdc0108 - cargo::ops::cargo_compile::compile::hb3819b5782fcf9c7
11:51.13   15:     0x557ddbd26035 - cargo::commands::rustc::exec::h8fe0b856efbee4e1
11:51.13   16:     0x557ddbd0603e - cargo::cli::main::h0f07c6a7a6ce919b
11:51.13   17:     0x557ddbd26ecf - cargo::main::h64ba207a4def4601
11:51.13   18:     0x557ddbd1e4b2 - std::rt::lang_start::{{closure}}::hdb386df0415e9de5
11:51.13   19:     0x557ddc35cd22 - std::panicking::try::do_call::h05eeced72861b08b
11:51.13                                at src/libstd/rt.rs:59
11:51.13                                at src/libstd/panicking.rs:310
11:51.13   20:     0x557ddc385f39 - __rust_maybe_catch_panic
11:51.13                                at src/libpanic_unwind/lib.rs:102
11:51.13   21:     0x557ddc36bbaa - std::rt::lang_start_internal::h1aba9fa33a080cd0
11:51.13                                at src/libstd/panicking.rs:289
11:51.13                                at src/libstd/panic.rs:398
11:51.13                                at src/libstd/rt.rs:58
11:51.13   22:     0x557ddbd295c1 - main
11:51.13   23:     0x7fb0644efb6a - __libc_start_main
11:51.13                                at ../csu/libc-start.c:308
11:51.13   24:     0x557ddbd00b49 - _start
11:51.13                                at ../sysdeps/x86_64/start.S:120
11:51.13   25:                0x0 - <unknown>
11:51.15 [style 0.0.1] cargo:rerun-if-changed=/sources/firefox-66.0.5/firefox-66.0.5/firefox-build-dir/dist/include/nsRefPtrHashtable.h
11:51.15 make[4]: *** [/sources/firefox-66.0.5/firefox-66.0.5/config/rules.mk:1027: force-cargo-library-build] Error 101
11:51.15 make[4]: Leaving directory '/sources/firefox-66.0.5/firefox-66.0.5/firefox-build-dir/toolkit/library/rust'
11:51.15 make[3]: *** [/sources/firefox-66.0.5/firefox-66.0.5/config/recurse.mk:74: toolkit/library/rust/target] Error 2
11:51.15 make[3]: Leaving directory '/sources/firefox-66.0.5/firefox-66.0.5/firefox-build-dir'
11:51.15 make[2]: *** [/sources/firefox-66.0.5/firefox-66.0.5/config/recurse.mk:34: compile] Error 2
11:51.15 make[2]: Leaving directory '/sources/firefox-66.0.5/firefox-66.0.5/firefox-build-dir'
11:51.15 make[1]: *** [/sources/firefox-66.0.5/firefox-66.0.5/config/rules.mk:415: default] Error 2
11:51.15 make[1]: Leaving directory '/sources/firefox-66.0.5/firefox-66.0.5/firefox-build-dir'
11:51.15 make: *** [client.mk:125: build] Error 2
11:51.18 10 compiler warnings present.
11:51.19 /usr/bin/notify-send --app-name=Mozilla Build System Mozilla Build System Build failed

This is the error in particular I'm talking about. It seems to show up only 50% of the time. Temperatures and voltages are well within reasonable values, with the temperature being at 40C and the voltages on Vcore, 12v, 5v, EPS12v, and 3.3v being perfectly in the center of their ranges. Memory is good as well. I'm certain it's rust.

comment:7 by Bruce Dubbs, 3 years ago

What version of rust are you using? I am using rustc 1.32.0 and have never seen such problem.

in reply to:  7 comment:8 by ken@…, 3 years ago

Replying to bdubbs:

What version of rust are you using? I am using rustc 1.32.0 and have never seen such problem.

No idea what Douglas is using, but I'm using 1.32.0 until it becomes inadequate for firefox-beta.

comment:9 by Douglas R. Reno, 3 years ago

renodr [ /usr/src/jhalfs/logs ]$ rustc --version
rustc 1.32.0

This has been observed over two builds of LFS now. I'm certain it's a problem with the Skylake architetcure, possibly MPX-removal (in the kernel) related.

comment:10 by Xi Ruoyao, 3 years ago

Summary: rustc-1.34.1 (use system LLVM again)rustc-1.35.0 (use system LLVM again)

comment:11 by ken@…, 3 years ago

Owner: changed from blfs-book to ken@…
Status: newassigned

I'm starting to look at this (slowly, will be interrupted) because firefox-68 will need 1.34.0 or later (and yes, 1.34.0 had a CVE but would still be accepted). I plan to first try to build it as now (i.e. with the shipped llvm), and then see if it can build current versions of everything that is in the book.

If ok, I then hope to try with system llvm and find how the space and time compare.

comment:12 by ken@…, 3 years ago

For the current-style build using shipped LLVM: the size is bigger (of course), the timing is also a lot bigger (70.3 SBU including the tests).Test results seem similar to 1.32.0 (haven't these rustaceans learned arithmetic yet ?).

Part of the reason why the build is so slow might be from building LLVM twice: I noticed this initially when my first install failed because of a wetware problem, https://github.com/rust-lang/rust/issues/61206 but the slowness (if any) will be in the main build.

I've got the FreeBSD patch from that issue (<sigh>I seem to be building FreeBSD from Scratch for rust and firefox</sigh>), will give that a go. Meanwhile, with shipped llvm, 1.35.0 successfully builds rsvg, cbindgen, firefox. Will test thunderbird later.

For firefox, the build seems a bit faster - but I know enough about building with rust to discount a single build (on my "tuning" experiments the build times for rust packages vary quite a lot, even when just rebuilding exactly the same).

comment:13 by ken@…, 3 years ago

Hmm, the FreeBSD patch does not speed up building with shipped LLVM - my build actually took a little longer, but that is probably normal rust variation.

Thunderbird-60.7.0 does not build:

16:26.14 error: missing documentation for macro

I found the fedora srpms, in thunderbird they have a patch for rustc-1.33.0 which seems to fix this (still compiling).

Trying to find out when thunderbird will be updated, I read that it normally *follows* the firefox ESR (i.e. might be a little while after that). will upload the patch when build has completed. Looks as if thunderbird-60.8 will happen in July, with thunderbird-68 which presumably will also need newer rustc, and last thunderbird-60 in about September.

comment:14 by ken@…, 3 years ago

Side note: fedora still build rustc using the old configure script!

More to the point, they have a patch for building rustc with system LLVM-7 (sic) - and claim that system LLVM-6 can also be used, as well as system LLVM-8.

comment:15 by Bruce Dubbs, 3 years ago

Milestone: 8.59.0

Milestone renamed

comment:16 by ken@…, 3 years ago

Coming back to this since it's now only about 10 days until firefox-68.0

With shipped LLVM, a scripted build of rustc (probably with CFLAGS of -O2) took 70.3 SBU, 8750 MB total, 557 MB installed. With system LLVM (8.0.0) and using -O2 plus "cheap hardening" because I'd left that from exercising inkscape, the figures reduce to 59.5 SBU, 7155 MB total, 424 MB installed.

I'm hopeful that everything will build and run. Rustc-1.36.0 is due in the coming week (Releases file has been updated, labelled as 4th July), but given their recent release history (the 1.30, 1.31, 1.34 issues) I see no reason to rush to use it.

comment:17 by ken@…, 3 years ago

Forgot to mention that with system llvm one test moved from 'passed' to 'ignored'.

comment:18 by ken@…, 3 years ago

Looking at the test results, the format seems to have changed a bit (look for 'stderr:' in the log), but still 3 failures for missing thumb compilers. One other failure because it is looking in the wrong place for a library directory:

stderr:
------------------------------------------
Traceback (most recent call last):
  File "test.py", line 54, in <module>
    libs = get_all_libs(join(sysroot, 'lib/rustlib/{}/lib'.format(os.environ['TARGET'])))
  File "test.py", line 49, in get_all_libs
    for f in listdir(dir_path)
OSError: [Errno 2] No such file or directory: 'lib/rustlib/x86_64-unknown-linux-gnu/lib'
make: *** [Makefile:2: all] Error 1

Looking for 'rustlib' in my build, the actual place is

/scratch/ken/rustc-1.35.0-src/build/tmp/dist/rust-std-1.35.0-x86_64-unknown-linux-gnu/rust-std-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/

so I have no idea why it did not find it.

Also, all my systems have Python2.7 installed and it definitely seems to use that to run the tests.

comment:19 by ken@…, 3 years ago

Checking more details - my first build with system llvm, including tests, rounded to 60 SBU. The current manual build took 24 SBU including the DESTDIR install, running the tests after that to confirm the size and space took the total to 39 SBU. Repeated build followed by tests, times were within a few seconds so I conclude the 60 SBU was an outlier.

in reply to:  18 comment:20 by ken@…, 3 years ago

Replying to ken@…:

Looking at the test results, the format seems to have changed a bit (look for 'stderr:' in the log)

I now see that if I look for FAILED the old description of where to look is still applicable, and that test which cannot find the library directory is run-make-fulldeps/sysroot-crates-are-unstable : this was labelled as failing if P~ython2.7 is NOT installed, but I think a more likely reason (since I do have that!) is that we are using a stable build.

comment:21 by ken@…, 3 years ago

Although the main build excludes miri, the install nevertheless compiles and installs both miri and cargo-miri. These seem unwanted, and it looks as if miri itself cannot be successfully run. Adding --exclude src/tools/miri to the install command does not alter this.

 ~ $/opt/rustc/bin/miri  --help
thread 'main' panicked at 'could not find sysroot. Either set `MIRI_SYSROOT` at run-time, or at build-time specify `RUST_SYSROOT` env var or use rustup or multirust', src/libcore/option.rs:1034:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

Ahh, https://github.com/rust-lang/rust/issues/61830 so broken in this version but perhaps will be fixed in 1.36 (not that it is useful to use!).

in reply to:  21 comment:22 by ken@…, 3 years ago

Replying to ken@…:

(not that it is useful to use!).

s/use!/us!/

comment:23 by ken@…, 3 years ago

Resolution: fixed
Status: assignedclosed
Note: See TracTickets for help on using tickets.