Support raw-dylib link kind on ELF

[Noratrieb]

raw-dylib is a link kind that allows rustc to link against a library without having any library files present.
This currently only exists on Windows. rustc will take all the symbols from raw-dylib link blocks and put them in an import library, where they can then be resolved by the linker.

While import libraries don't exist on ELF, it would still be convenient to have this same functionality. Not having the libraries present at build-time can be convenient for several reasons, especially cross-compilation. With raw-dylib, code linking against a library can be cross-compiled without needing to have these libraries available on the build machine. If the libc crate makes use of this, it would allow cross-compilation without having any libc available on the build machine. This is not yet possible with this implementation, at least against libc's like glibc that use symbol versioning. The raw-dylib kind could be extended with support for symbol versioning in the future.

This implementation is very experimental and I have not tested it very well. I have tested it for a toy example and the lz4-sys crate, where it was able to successfully link a binary despite not having a corresponding library at build-time.

I was inspired by Björn's comments in https://internals.rust-lang.org/t/bundle-zig-cc-in-rustup-by-default/22096/27
Tracking issue: #135694

r? bjorn3

[rustbot]

These commits modify compiler targets.
(See the Target Tier Policy.)

Some changes occurred in src/tools/compiletest

cc @jieyouxu

[bjorn3]

This is fine as is for now, but once Apple targets support raw-dylib, this static deps skipping behavior should likely apply there too if you are allowed to put .tbd files in static libraries.

[bjorn3]

It should be possible to handle verbatim by emitting a SONAME in the stub shared library with the actual name of the shared library and then have the stub have a name starting with lib and ending with .so like usual. We will need to support verbatim for linking against shared libraries like libc.so.6 (glibc) or basically any shared library that is part of a distro.

[Noratrieb]

Do we actually need to set SONAME for verbatim? Couldn't we just name the stub library after the verbatim name and get it working directly?

[bjorn3]

No, because the linker requires the filename of every shared library to start with lib and end with .so, but with versioned shared libraries the name ends with .so.<version>. The way this normally works is by having libfoo.so.1 contains a SONAME of libfoo.so.1 and then have a symlink from libfoo.so to libfoo.so.1 in the -dev package. The linker will see libfoo.so and then encode libfoo.so.1 in the executable because of the SONAME.

[Noratrieb]

I'm not sure I understand. You mentioned verbatim, but with verbatim we can just pass the -l:libfoo.so.1 to the linker and it will pick that up for the NEEDED so name. So there isn't anything we need to do to make this work. Since we do need to know the library version from the source code, verbatim is needed anyways for all these cases.

[bjorn3]

but with verbatim we can just pass the -l:libfoo.so.1 to the linker and it will pick that up for the NEEDED so name.

What if the user passes an absolute path as verbatim name? NEEDED entries accept absolute paths just fine, but -l: would look at the wrong location. SONAME can encode arbitrary values to set as NEEDED without messing up the search logic of the linker. Also using SONAME + random filename for the stub library should still allow #[link(name = "foo")] to still look for the system libfoo.so even if there is a #[link_name = "foo", kind = "raw-dylib")] only encodes a subset of the functions available in libfoo.so (as the crate in question didn't need the rest). If the stub library is called libfoo.so it would take precedence over the system install and thus every raw-dylib block would need to exhaustively list all functions, including those not needed by the crate in question to avoid breaking someones build.

[Noratrieb]

I see, that makes sense. I have changed the strategy to accommodate this by always generating it with a random file name and setting the SONAME.

[jieyouxu]

compiletest changes LGTM, would be great if you could add the two new directives to the directives listing in r-d-g

[rustbot]

The rustc-dev-guide subtree was changed. If this PR only touches the dev guide consider submitting a PR directly to rust-lang/rustc-dev-guide otherwise thank you for updating the dev guide with your changes.

cc @BoxyUwU, @jieyouxu, @Kobzol

[rustbot]

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

[bors]

☔ The latest upstream changes (presumably #135164) made this pull request unmergeable. Please resolve the merge conflicts.

[bjorn3]

Maybe instead use a hash of load_filename and raw_dylib_imports? I'm afraid this non-determinism will leak into debuginfo.

[bjorn3]

Would have been nice if the object crate exposed a method for this.

[bjorn3]

Suggested change

	let mut stub = write::Writer::new(object::Endianness::Little, true, &mut stub_buf);
	let endianness = match sess.target.options.endian {
	Endian::Little => object::Endianness::Little,
	Endian::Big => object::Endianness::Big,
	};
	let mut stub = write::Writer::new(endianness, sess.target.pointer_width == 64, &mut stub_buf);