Use live apphost when publishing ilc as singlefile

[am11]

Using the following command to cross-build product for FreeBSD:

docker run --rm -v$(pwd)/runtime:/runtime -e ROOTFS_DIR=/crossrootfs/x64 \
  mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-cross-amd64-freebsd-14 \
  sh -c '/runtime/build.sh clr+libs+packs -c Debug -os freebsd -cross &&
  tdnf install -y file unzip && cd /tmp &&
  unzip /runtime/artifacts/packages/Debug/Shipping/runtime.freebsd-x64.Microsoft.DotNet.ILCompiler.9.0.0-dev.nupkg &&
  find . -name ilc -exec file {} \;'

before:

./tools/ilc: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=771fdbe539f221e5b1e4e6dc000846f10d503b3f, stripped

after:

./tools/ilc: ELF 64-bit LSB pie executable, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 14.0 (1400097), FreeBSD-style, BuildID[sha1]=4b265d82eb3e1de5315251f7a5db2713ced94d2e, stripped

---

It required defer building ILCompiler.csproj until after apphost sfxproj. Also, PackageRID is used to restore the "host" package to build the target, while OutputRID is used to create package for the target, which is what we need here.

Fix #104497

[jkotas]

The tricky ordering of the different build steps looks fragile.

I am wondering whether it would be better to use multiple build.sh invocations to bootstrap the FreeBSD cross build. The first invocation would skip building parts that depend on last-known-good packages for target arch, the second invocation would then build the rest and use the packages produced by the first invocation.

[am11]

Yup, it is fragile as in, when we are not building packs subset, it fails the build on platforms using PublishSingleFile. Pulling Microsoft.NETCore.App.Host.sfxproj in clr subset is also not an option because that depends on libs+host to build first.

There is an existing bug that on the systems where we use PublishAot for ILCompiler, it still looks for apphost (#80154 (comment), it was actually a regression in .net9 p1). That makes the refactoring those parts a degree more delicate.

[am11]

@jkotas, the ILCompiler.csproj build will be now deferred until after packs are built, only for freebsd/illumos/haiku OS set which do not provide Microsoft SDK. This requires single invocation of ./build.sh.

The current state of this PR (351856c) is such that if we still don't like Subsets.props changes, then we can revert Subsets.props (alone) and document the following steps for the devs of those platforms:

# invocation 1 - build everything without ILCompiler support
$ ./build.sh -os freebsd -p:NativeAotSupported=false

# invocation 2 - build clr.tools and packs subsets again without NativeAotSupported
#                condition so ILCompiler.csproj can use live-built apphost.
$ ./build.sh clr.tools+packs -os freebsd

[jkotas]

Ideally, the cross-arch and cross-OS bootstrapping builds would produce a full-featured bits that are equivalent (including how the tools get published) to the natively compiled packages. I think achieving it via multiple invocations of build.sh flow would be easier to maintain than the ordering of the individual build steps.

[am11]

@Thefrank, @sec, basically this is the new workflow for community platforms for cross-OS build (e.g. for freebsd on linux): #105004 (comment) (two steps). I tested it with this workflow https://github.com/am11/CrossRepoCITesting/actions/runs/10845732989/workflow, and artifacts are https://github.com/am11/CrossRepoCITesting/releases/tag/freebsd_10845732989. The ilc executable in runtime.freebsd-x64.Microsoft.DotNet.ILCompiler.10.0.0-ci.nupkg shows:

$ unzip ~/Downloads/runtime.freebsd-x64.Microsoft.DotNet.ILCompiler.10.0.0-ci.nupkg
$ find . -name ilc -exec file {} \; | fold
./tools/ilc: ELF 64-bit LSB pie executable, x86-64, version 1 (FreeBSD), dynamic
ally linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 14.0 (1400097), FreeB
SD-style, BuildID[sha1]=235f8e6220cbf08ec8c1966a415f82f6ed072e4e, with debug_inf
o, not stripped

[jkotas]

$ ./build.sh -os freebsd -p:NativeAotSupported=false
$ ./build.sh clr.tools+packs -os freebsd

I think we need to have a clean separation of responsibilities between the two commands. Otherwise, I expect it will collide with where #107772 is trying to go. A good split may be:

• The first command should be partial build that produces packages that the full build expects to come from LKG toolsets when there is one for target platform
• The second command should be a full build that uses the packages produced by the first command

We can define a new subset for the first command to make the command line UX simpler and stable over time. This subset would not be on by default. Building this subset can drop the packages in a new directory. The full build can automatically pick the LKG toolset from there if the directory exists.

Ideally, it would be even possible for the first command to be debug/checked build and the second command to be release build or vice versa, and it should work for non-cross builds too.

@agocke Thoughts?

[am11]

PublishAotUsingRuntimePack

It gives

/runtime/.dotnet/sdk/10.0.100-alpha.1.24610.7/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(497,5):
error NETSDK1082: There was no runtime pack for Microsoft.NETCore.App available for the specified RuntimeIdentifier 'linux-loongarch64'.
[/runtime/src/coreclr/tools/aot/crossgen2/crossgen2_publish.csproj]

We can look into UseNativeAotForComponents separately. Original issue #104497 was about corehost published ilc binary is using linux apphost on freebsd cross build, which is fixed.

[am11]

This is also needed to test riscv64 AOT #106223 (comment).

[akoeplinger]

@am11 the build failures on freebsd/loongarch/riscv look related

[jkotas]

@jkotas, if this looks ok, can this be merged?

I was hoping to get proper two stage (cross-plat) build for native AOT.

If the UseLocalTargetingRuntimePack changes look fine to @ViktorHofer, this can be merged. I expect that we will want to do more work on this over time.

[ViktorHofer]

@jkoritzinsky please also take a look

[am11]

I was hoping to get proper two stage (cross-plat) build for native AOT.

@jkotas, your ask was a two stage solution with NativeAotSupported+UseNativeAotForComponents, I have provided two stage solution with NativeAotSupported and sufficiently explained why UseNativeAotForComponents part is not an option by sharing what I tried and how it failed (also visible in the CI). Either it is possible and I have no idea how to go about it, or it is not possible and I am having a hard time explaining.

In any case, this is getting a bit out of scope for this PR as it was supposed to be fixing #104497.

I expect that we will want to do more work on this over time.

Definitely, like anything this can improve as well. Now that CI is capturing the usage pattern of Samsung and FreeBSD folks, it would be slightly harder to regress it in the first place.

I suggest we merge this PR to avoid continuing merge conflicts.

[jkoritzinsky]

/ba-g wasm failures and timeouts unrelated