Repository navigation
Allow using self-contained LLD in bootstrap - #135001
Conversation
…ntained"` is used Before, we just used the global `lld` anyway.
Not all targets support these flags, so we cannot just pass them to the tests unconditionally. Before, we were using a linker arg (`-Clink-arg=-fuse-ld=lld`), which circumvented this in a hacky way.
I have a 3970X (which may or may not qualify these days). On master, I see On this PR, I see running again: (the difference caused by this PR is close to jitter) |
commented
Jan 1, 2025
|
Thanks, it doesn't seem like there's any difference then. Actually, I wonder if the stage1 compiler uses LLD anyway, since it's now enabled by default on nightly 🤔 In any case, the second commit shouldn't regress perf. anyway. |
|
Measured on a 24c/48t Zen2 CPU Doc test ranges overlap. But it seems to be a small slowdown for UI tests.
Not sure what you mean? LLD was already used before to build the tests, that's why bootstrap has some handling to set the thread count for lld (UI tests should be running with |
|
I meant that the tests are already linked with LLD even without using Note: this is only true for stage 1+, since on stage0 the bootstrap compiler is beta, which doesn't use LLD by default, so |
|
I was going to check the time on native msvc, but it turns out with `tests\ui\asan-odr-win\asan_odr_windows.rs`EDIT: opened #135013 to track that test failure. |
|
|
||
| let mut targetflags = flags; | ||
| targetflags.push(format!("-Lnative={}", builder.test_helpers_out(target).display())); | ||
| targetflags.extend(linker_flags(builder, compiler.host, LldThreads::No)); |
There was a problem hiding this comment.
Shouldn't we design linker_flags in a way that preserves the old logic for tests instead of removing it entirely?
There was a problem hiding this comment.
I saw only very minimal gains with LLD vs the default Linux linker on UI tests, but it's true that on some other platforms the difference might be larger.
The problem with the old approach is that it cannot respect the "self-contained" mode. So when you use lld = "self-contained", the old approach ignores that and just uses the global lld for the tests.
I guess that a compromise could be to use the linker flag with use-lld = true and the new compiler flag with use-lld = "self-contained", but I would eventually like to set use-lld = "self-contained" to be the default on CI; with this approach, some tests would simply fail..
There was a problem hiding this comment.
I guess that a compromise could be to use the linker flag with use-lld = true and the new compiler flag with use-lld = "self-contained", but I would eventually like to set use-lld = "self-contained" to be the default on CI; with this approach, some tests would simply fail..
This sounds reasonable I think.
There was a problem hiding this comment.
But there's still the "some tests would fail" part 😆 Another alternative would be to actually modify the target configuration of the compiler when use-lld is enabled, but this means that the compiler itself would be modified and would use lld by default; right now, we just enable lld in a hacky way in bootstrap.
There was a problem hiding this comment.
You said "eventually", I am not sure why we should concern on that part right now.
There was a problem hiding this comment.
It's not just about CI. Right now, use-lld is quite useful to make local compiler builds faster. But if we actually make self-contained working as intended, and keep the compiletest target flags, then use-lld = "self-contained" will also cause UI tests to fail for people that use this config value.
There was a problem hiding this comment.
I see. In this case I am fine with landing it as is. We can create an issue to track that part.
commented
Jan 3, 2025
|
@bors r+ |
commented
Jan 3, 2025
commented
Jan 4, 2025
commented
Jan 9, 2025
|
This is already merged, bors. @bors retry r- |
In #116278, I added a
"self-contained"mode to therust.use-lldbootstrap option, which was designed for using the built-in LLD for linking compiler artifacts. However, this was later reverted in #118810.This PR brings the old logic back, which switches LLD in bootstrap from
-fuse-ld=lldto MCP510's way of passing linker flags to enable LLD (both external and self-contained). So this does two changes:-fuse-ld=lldto MCP510Regarding the second commit: Since #86113, we have been passing
-fuse-ld=lldas a target flag to all tests whenuse-lld = trueis enabled. This kind of worked for all tests, since it was just a linker argument, which has bypassed any compiler checks, and probably resulted only in some warning if the given target linker didn't actually support LLD. However, after the first commit, some tests actually start failing with this approach:So the second commit removes the passing of LLD flags as target flags to tests. I don't think that it's a good idea to pass specific compiler flags to all tests unconditionally, tbh. The doctest command from #86113 doesn't go through compiletest anymore, and doctests should be quite a lot faster since #126245 in general.
CC @the8472
If someone has a beefy machine, it would be nice to test whether this doesn't regress test execution speed. How to do that:
rust.use-lld = trueandrust.lld = trueinconfig.toml./x test tests/ui --force-rerunbetweenmasterand this PROnce this is tested in the wild, I would like to make the self-contained LLD the default in CI, hopefully to make CI builds faster.
r? @onur-ozkan