`available_parallelism` misbehaves on FreeBSD

[RalfJung]

The rustup in #5343 surfaced a regression in the behavior of https://github.com/rust-lang/miri/pull/5343 on FreeBSD. With -Zmiri-num-cpus=1024 it now shows 256 CPUs rather than 1024.

I temporarily disabled the test but we should figure out what is happening here. The problem probably began with some change in this upstream diff: rust-lang/rust@88638df...3670d25.

Cc @YohDeadfall @LorrensP-2158466

[RalfJung]

It's probably rust-lang/rust#120589.

[devnexen]

here my 2 cents: sched_getaffinity miri shim silently truncates the mask when the buffer is too small (cpuset_t is 256 bit), silently truncates but returns success. Miri needs to reflect real freebsd behavior (return EINVAL) when the highest cpu bit does not fit cpusetsize.

[YohDeadfall]

Will fix it latertoday.

@rustbot claim

[YohDeadfall]

@RalfJung, tell me please what I should use to print debug output from miri? And what would be the best way to reorganize the code, so sched_getaffinity can be routed to cpuset_getaffinity for FreeBSD? The difference is in checks like FreeBSD doesn't care about copying "words" but requires fitting the mask into the buffer. Moreover, FreeBSD replaces ERANGE with EINVAL for sched_getaffinity.

[LorrensP-2158466]

@YohDeadfall IIRC, you can just use eprintln, no?

And what would be the best way to reorganize the code, so sched_getaffinity can be routed to cpuset_getaffinity for FreeBSD?

I don't think that is the solution here, these are distinct API's (and won't work because of the reasons you gave).

I have been staring at the code for a while and I do not see any differences in the way both write the mask. I really think this is a version mismatch. The test expects 1024 cpus, but FreeBSD 13 and under only supports up to 256. No idea why it suddenly changed, but i don't think its the implementation it self (don't quote me, still looking).

[RalfJung]

FreeBSD version handling is certainly messy and confusing so that may well be it.

[RalfJung]

So the definition of cpuset_t is here. It depends on the configured FreeBSD version which, I think, is set to 12. So this is not just a Miri problem, the FreeBSD we distribute will also use the FreeBSD 12 version of this type.

[LorrensP-2158466]

I still find it weird that it didn't fail before. cpuset_getaffinity also uses cpuset_t and has the exact same code for copying the mask. If the type was the same, it should have failed with ERANGE.

[YohDeadfall]

I don't think that is the solution here, these are distinct API's (and won't work because of the reasons you gave).

I don't think that it's even coming from the current implementation too. Otherwise, it would affect Linux as well, the code is shared between the both systems. I suspect it being set in some other place to 256 and that's why I wanted to add some output to verify that.

If the type was the same, it should have failed with ERANGE.

And that's exactly how I spent an evening debugging the wrong shim. FreeBSD has its own functions starting cpuset_ while providing a compatibility layer via sched_. They both handle errors differently, and even then it seems not to be a fully compatible handling with Linux.

The shim that currently handles it is designed for Linux, so that's where the mess is coming from from the first place.

Currently I see it as moving sched_ functions out of the Unix like shared path and implementing them for each target separately with the shared logic if needed. If I'm not wrong then futexes behave like that, or at least some of my previous contributions did that.