Repository navigation
fix: use the toolchain repository as the rustc sysroot to shorten Windows paths - #59
Open
titusfortner wants to merge 114 commits into
Open
titusfortner wants to merge 114 commits into
titusfortner wants to merge 114 commits into
Conversation
2 tasks done
This reverts commit 59a507e.
Add support for tier 3 targets bpfeb-unknown-none and bpfel-unknown-none (see https://github.com/rust-lang/rust/blob/f5e2df7/src/doc/rustc/src/platform-support.md?plain=1#L311-L312). This is modeled after bazelbuild#3507 and should probably be updated if/when bazelbuild/platforms#131 is merged. (please use rebase merge when landing this as the proper commit message is in the commit, rather than the PR description) /cc @avrabe
…elbuild#3829)" This reverts commit f198dde.
…() (bazelbuild#3816)" This reverts commit 9586468.
…hermeticbuild#3) * 0 * Add rust analyzer test coverage
…hollow rlibs: the RustcMetadata action runs rustc to completion with -Zno-codegen, emitting a .rlib archive. This approach mirrors the one used by buck2 and avoids needing to kill rustc mid-output in order to produce metadata. While not fixing problems with SVH mismatches when non-determinism, this does simplify the codepath and uses a production tested technique that doesn't have any of the dangers associated with killing the rustc process while it's still active.
Port the sharding wrapper feature from bazelbuild#3774 into the hermeticbuild fork. The implementation wraps rust_test executables when experimental_enable_sharding is set while keeping rustc_compile_action's existing provider-list API for internal and extension callers. rust_test now scans the returned providers to replace DefaultInfo for the wrapper, so extensions such as prost and wasm-bindgen continue to consume rustc_compile_action without API churn. Co-authored-by: Brian Duff <bduff@linkedin.com> Co-authored-by: Codex <noreply@openai.com>
Rustc emits GNU-like Windows staticlibs as lib<crate>.a, but rules_rust was stripping the lib prefix for all Windows non-rlib library outputs. Keep the prefix for staticlib outputs when the target ABI is gnu or gnullvm so declared outputs match rustc.
Use source_file_label to preserve filenames relative to the owning Bazel package when query_label_for builds the srcs query. Files such as src/adb.rs now match the target's source label instead of :adb.rs. Add a regression test for root packages, nested directories, and direct source files. The test fails with the old basename behavior. Based on bazelbuild#4238; addresses hermeticbuild/rules_rs#232. Validation: 10 Flycheck tests and 44 rust-analyzer library tests pass through rules_rs. --saved-file command checks pass for four package layouts.
…eticbuild#48) ``` On Windows, consolidate_dependency_search_paths (util/process_wrapper/main.rs:156) collapses every -Ldependency=<dir> rustc flag into one unified directory, to avoid Windows command-line size overruns (commit 3f613e4). It does so by read_dir-ing each search path and hardlinking (falling back to copying) every regular file it finds into that unified dir. However, builds failed with: Error: ProcessWrapperError("unable to copy .../ijent_util-subscriber_contract-test.exe.tmp8ceb5a4 into unified dependency dir ...: Access is denied. (os error 5)") .exe.tmpXXXXXXX file is an LLVM's atomic-output temp file. Every LLVM tool that writes output this way does it: lld-link / rust-lld (the PE image and the PDB), llvm-lib (import libs), llvm-objcopy. So this file is rust-lld writing the sibling target ijent_util-subscriber_contract-test.exe. It only occurs with toolchain.linker_preference == "rust" (rustc.bzl:445-448); MSVC's link.exe writes in place. Lifecycle of that temp file ┌────────────────┬───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Phase │ What happens │ ├────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤ │ Created │ At the start of the sibling's link step. TempFile::create passes OF_Delete, then immediately calls │ │ │ setDeleteDisposition(H, true) → DELETE_PENDING from birth. │ ├────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤ │ Alive │ For the entire duration of that link, while lld mmaps and writes the image into it. │ ├────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤ │ Deleted │ FileOutputBuffer::commit() → TempFile::keep(FinalPath): setDeleteDisposition(H, false), then rename_handle(H, │ │ (success) │ "…-test.exe"). The temp name vanishes by rename. │ ├────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤ │ Deleted │ TempFile::discard() — the already-set delete disposition does it on close; RemoveFileOnSignal covers crashes. │ │ (failure) │ │ └────────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ Why os error 5 and not os error 2 Because of DELETE_PENDING. On Windows a delete-pending file: - still appears in directory enumeration — fs::read_dir yields it and entry.file_type() succeeds (metadata comes from the dirent); - but every CreateFile on it fails with STATUS_DELETE_PENDING → ERROR_ACCESS_DENIED (5). So fs::hard_link (CreateHardLinkW) fails with 5, the fs::copy (CopyFileExW) fallback fails with 5, and the wrapper escalates to a fatal ProcessWrapperError. LLVM's own createUniqueEntity retries on errc::permission_denied for exactly this reason. This is not a narrow race. The window is the whole link of the sibling target, and the failure is deterministic once read_dir observes the temp file. Why the temp file is in a -L dependency= directory at all rust/private/rustc.bzl:2994-2998 derives the search paths from the dirnames of dep_info.transitive_crate_outputs (_get_dirname, rustc.bzl:3433). A dirname is an entire Bazel package bin dir (bazel-out/<cfg>/bin/fleet/native/ijent/ijent_util/) containing every output of that package — sibling .exes, .pdbs, .os, .rustc-outputs, tree artifacts, in-flight linker temps — not a curated set of crate artifacts. Bazel has no sandboxing on Windows, so those scanned directories are the live bazel-out tree with every concurrent action's activity visible. ```
Remove cargo_toml_env_vars inputs from the determinism BUILD files and match RustdocTest in the C++ runtime analysis test. Restore the dynamic standard-library default for construct_arguments callers and include C++ linker files in Windows GNU library compile inputs for dlltool. Add dynamic-standard-library assertions for Clippy and rustdoc tests, and cover dlltool inputs for GNU, GNU LLVM, MSVC, and Linux targets.
Rust generates import libraries in-process for Windows GNU LLVM and MSVC. Restrict the dlltool argument and C++ linker-file inputs to Windows GNU, where raw-dylib compilation invokes dlltool. Update the rlib input test so GNU LLVM retains no C++ linker files.
Expand C++ linker paths and flags in Args.map_each callbacks, so Bazel can apply the Rustc action's path mapper. Preserve the eager get_linker_and_args interface for Cargo build scripts and bindgen, and retain Rust linker selection and flag filtering. Defer Windows GNU dlltool paths as well. Add an analysis regression with a generated C++ linker, linker script, and LIB-derived search directory. It passes with mapping off on Bazel 9.1.0 and 9.3.0-dzbarsky18, and with strip on a 9.3.0-dzbarsky18 build containing bazelbuild/bazel#30789. It fails with strip on unpatched Bazel. All 21 selected linker/native-dependency/SDKROOT analysis tests pass with mapping off and on. Codex at 52e73e3a builds //codex-rs/cli:codex with strip after backporting this change to its rules_rust pin, preserving its patches. Unmodified rules_rust fails to find Scrt1.o and libc with strip, including with the Bazel backport alone.
Default experimental_use_allocator_libraries_with_mangled_symbols to true so cc_common.link and C++ consumers of Rust libraries receive allocator symbols generated by the selected rustc. Preserve the explicit opt-out for custom and legacy C++ allocator libraries and document the nightly requirement for the global allocator implementation. Remove redundant allocator flags from integration fixtures and ThinLTO tests. Cover heap allocation through C++ and per-target cc_common.link, default allocator CcInfo, and explicit opt-out. Use the C++ bootstrap wrapper in the mock codegen-units toolchain to avoid a dependency cycle. Validated with Bazel 9.1.0 on macOS: 20 cc_common.link integration tests, 32 CcInfo/allocator/ThinLTO unit tests, and the pinned nightly global allocator test. Per-target and C++ allocation tests also pass with the global cc_common.link setting disabled; the C++ test passes with dynamic libstd. Dynamic-libstd Rust tests encounter a getopts link failure also reproduced on unchanged main. Buildifier and git diff --check pass. Assisted-by: OpenAI Codex
Bootstrap actions pass rustc flags through @params, but the C++ bootstrap process wrapper substitutes only direct arguments. The literal remap placeholders leave execution directories in tinyjson crate metadata. Use -Zremap-cwd-prefix for bootstrap compilation so rustc resolves its own working directory. Default RUSTC_BOOTSTRAP to the bootstrap crate's name, preserving explicit environment settings. Keep the existing response files and SDKROOT expansion without changing the C++ wrapper. Add a regression test that reads the actual bootstrap tinyjson archive. The test fails before the fix. Sandboxed and local builds in different output bases now produce identical archives without execution directories. Validation: Bazel 9.1.0, Rust 1.98.0, macOS arm64; bazel test //... passed 621 tests and skipped 37 platform-incompatible tests. Refs hermeticbuild/rules_rs#233 and hermeticbuild#46. Co-authored-by: Codex <noreply@openai.com>
Sort flag variants and classify their argument forms once in _prefix_pwd_to_flag, rather than repeating both operations for every C/C++ toolchain argument. Preserve the existing ordered matching and path rewrites.
Replace the struct returned by _get_linker_config with _LinkerConfigInfo, declaring the seven fields consumed by the linker argument and environment helpers. Validation: buildifier -mode=check and git diff --check pass. Assisted-by: OpenAI Codex
) Include determine_output_hash in metadata filenames for rust_binary and both rust_test implementations. Targets sharing a crate_name otherwise declare the same metadata output when always_enable_metadata_output_groups is enabled. Add a regression with two binaries and two tests sharing crate names. The regression fails with conflicting RustcMetadata actions before the fix; all 11 metadata_output_groups tests pass after the fix on Bazel 9.0.0. Assisted-by: OpenAI Codex
Include --copt with --conlyopt or --cxxopt in the C/C++ toolchain variables used to derive CFLAGS and CXXFLAGS. This makes build script native compilation honor Bazel compiler options while keeping language-specific flags separate. Add an analysis regression for all three options. The regression fails before the fix; all 32 cc_args_and_env tests pass on Bazel 9 afterward. Assisted-by: OpenAI Codex
Pass skip_expanding_rustc_env through both construct_arguments calls so pipelined compilation does not expand literal environment values for custom rules. Add an analysis regression for literal make-variable syntax. The regression fails before the fix; all 14 pipelined_compilation tests pass on Bazel 9 afterward. Assisted-by: OpenAI Codex
…build#67) Only binaries, cdylibs, and proc macros linked by rustc register linkstamp compile actions. Pass that condition to collect_deps to avoid flattening C++ linker inputs for Rust libraries, and disable linkstamp collection for Clippy actions, which already discard linkstamps. Preserve transitive C++ linker inputs for downstream binaries and retain the existing default for other collect_deps callers. Assisted-by: Codex
Require CrateInfo, TestCrateInfo, CrateGroupInfo, or DepVariantInfo before applying rust_analyzer_aspect. Share PyO3 provider forwarding and advertise DepVariantInfo for both single-crate and crate-group toolchains. Advertise CrateInfo and DepInfo on the custom_alias test rule, and use named providers in the three changed files. The traversal fixture and rust-analyzer tool target required 24 aspect applications instead of 208 and produced the same 22 crate specifications on the original checkout. Alternating measurements did not demonstrate an overall analysis-time improvement. Validated PyO3 provider forwarding for both toolchain variants, Rust static/shared-library specifications, the proc-macro analysis test, buildifier, and git diff --check.
…ermeticbuild#76) Call ctx.expand_location only for environment values containing $(, since other values cannot contain location macros. Forward literal values unchanged to avoid unnecessary expansion calls during rust-analyzer aspect analysis.
…cbuild#75) Advertise CcInfo from rust_library and ToolchainInfo from rust_toolchain, and add CcSharedLibraryHintInfo to rust_stdlib_filegroup. This lets cc_shared_library visit the owners of Rust archives and standard library link flags instead of rejecting their linker inputs. Add integration coverage for a C shared library depending on rust_library.
Pass transitive_noncrates directly to Args and construct each linker input tuple during argument expansion. This avoids retaining a flattened list and repeated option tuples in every linking action while preserving the existing platform callbacks, library order, and execution-path mapping. Reuse global PIC and non-PIC directory callbacks and let Args deduplicate directory arguments.
…ld#73) Pass the linker_inputs list from collect_inputs to _disambiguate_libs instead of flattening transitive_noncrates again. This removes a redundant depset conversion without changing native library order or disambiguation.
Mark extra_rustc_flag as target-scoped so Bazel 9 drops its values before configuring toolchains, build scripts, and proc macros. rules_rust already omits these values from exec Rustc actions, so keeping them in exec configurations creates unnecessary configuration differences. Include the existing extra_rustc_flag test in its test suite.
Append t to PYO3_CROSS_PYTHON_VERSION when the Python runtime ABI flags contain t. PyO3 needs this suffix to select the free-threaded ABI because the toolchain disables interpreter detection with PYO3_NO_PYTHON. Default missing ABI flags to an empty string for older Python runtime providers.
…build#71) Native archives from different configurations can share short_path. Key ambiguous_libs by File so one configuration does not overwrite another, while preserving lookups during Bazel execution-path mapping. Add a regression test that links two configurations of the same cc_library and requires distinct native library arguments.
Add clippy_config to Rust rules so individual crates can select a clippy.toml without changing the workspace-wide setting. Keep the global configuration as the default and as a fallback for custom rules that provide CrateInfo without clippy_config. Add analysis coverage for global configuration, per-crate overrides, and custom-rule compatibility.
… `cc_common.link` (hermeticbuild#82) Keep `.a` files out of the unified dependency directory of the Windows process wrapper. On Windows, `process_wrapper` merges every `-Ldependency=` directory into one, to keep the command line short. It scans each directory with `read_dir`, so it also sees files that are no input of the action. `rules_rust` puts a `lib<name>.a` symlink next to each rlib, for `cc_common.link`. With `experimental_use_cc_common_link`, Bazel builds those symlinks while sibling `Rustc` actions scan the directory, and the copy fails with "Access is denied". `-L dependency=` only searches for crates, and no crate is a `.a` file, so skip the extension. Relates to hermeticbuild#48
titusfortner
force-pushed
the
c/rules-rust-windows-paths-104013
branch
from
October 6, 2026 15:48
9f91986 to
628dc24
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Windows builds hit MAX_PATH in
link.exeon paths into the Rust sysroot. Bazel unpacks each toolchain archive intoexternal/<repo>/, which already has the layout rustc expects.rust_toolchainthen symlinks every file from there intobazel-out/<config>/bin/external/<repo>/rust_toolchain/and points rustc at the copy, so every std rlib path is about 50 characters longer than the file it points at, and there is one copy per configuration. rustc's fixedlib/rustlib/<triple>/liblayout means those characters cannot be recovered anywhere else in the path.LongPathsEnableddoes not help becauselink.exehas nolongPathAwaremanifest.Solution
When every toolchain component is a source file under the directory holding
bin/rustc, which is true for every toolchain repository rules_rust downloads, use that directory as the sysroot directly.--sysrootnow points atexternal/<repo>instead of the bazel-out copy. Toolchains that mix generated files or files from other repositories still get the symlink tree.Example, with
--output_base=D:/b, before:After:
libstdrlib, target configurationThe saving is 51 characters in the target configuration and 66 in the exec configuration, and the sysroot path is now the same in every configuration.
Additional considerations
--sysrootis derived fromtoolchain.rustcinstead of the anchor file so path mapping still applies. The anchor file is only written for symlinked sysroots andsysroot_anchoris None otherwise.--output_base=C:/b, so it exercises the new sysroot but never reproduced the long-path failure. Confirming the fix needs a real project build on a Windows machine without a shortened output base._toolssuffix andrw-<hash>repo name (about 19 characters) and--output_user_root.