Direct P/Invoke does not seem to work when "encapsulated" in a library

[sweemer]

Description

I have two C# projects in my solution - one library and one exe project. The library project exposes a native method using Direct P/Invoke and the exe project calls the method exposed in the library. I would like to "encapsulate" the DirectPInvoke and NativeLibrary properties within the library project, but as far as I can tell, they must be configured on the exe project in order to work. Is this true? If so, can I use this issue to request that "encapsulation" of Direct P/Invoke in a library project be supported? If not, then can somebody help point me in the right direction to get this working?

Reproduction Steps

First, verify that Direct P/Invoke works on the exe project:

$ git clone https://github.com/sweemer/NativeAotTest
$ cd NativeAotTest/Library
$ g++ -c main.cpp
$ ar qc libnativeaottest.a main.o
$ cd ..
$ dotnet publish && ./Exe/bin/Release/net8.0/linux-x64/publish/Exe # should print "Hello, world!" to the screen

Now move the following lines from Exe/Exe.csproj to Library/Library.csproj:

  <ItemGroup>
    <DirectPInvoke Include="nativeaottest" />
    <NativeLibrary Include="..\Library\libnativeaottest.a" />
  </ItemGroup>

Rerun the dotnet publish command from above and see that it produces the following exception:

Unhandled Exception: System.DllNotFoundException: Unable to load shared library 'nativeaottest' or one of its dependencies. In order to help diagnose loading problems, consider using a tool like strace. If you're using glibc, consider setting the LD_DEBUG environment variable:
nativeaottest.so: cannot open shared object file: No such file or directory
libnativeaottest.so: cannot open shared object file: No such file or directory
nativeaottest: cannot open shared object file: No such file or directory
libnativeaottest: cannot open shared object file: No such file or directory

   at System.Runtime.InteropServices.NativeLibrary.LoadLibErrorTracker.Throw(String) + 0x46
   at Internal.Runtime.CompilerHelpers.InteropHelpers.FixupModuleCell(InteropHelpers.ModuleFixupCell*) + 0x127
   at Internal.Runtime.CompilerHelpers.InteropHelpers.ResolvePInvokeSlow(InteropHelpers.MethodFixupCell*) + 0x35
   at NativeAotTest.Interop.hello_world() + 0x2d
   at Exe!<BaseAddress>+0xb3a29
Aborted (core dumped)

Expected behavior

When moving the DirectPInvoke and NativeLibrary properties from the exe project to the library project, I would expect them to be applied to the native methods exposed by the library. This is what I mean by "encapsulating" the Direct P/Invoke functionality within the library. Otherwise, the exe project needs to set these properties, which breaks the principle of encapsulation and tightly couples the exe project to the library project.

Actual behavior

DirectPInvoke and NativeLibrary properties apparently need to be set on the exe project, not on the library project. Please correct me if this is wrong.

Regression?

No response

Known Workarounds

No response

Configuration

$ dotnet --info
.NET SDK:
 Version:           8.0.401
 Commit:            811edcc344
 Workload version:  8.0.400-manifests.b6724b7a
 MSBuild version:   17.11.4+37eb419ad

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  22.04
 OS Platform: Linux
 RID:         linux-x64
 Base Path:   /usr/share/dotnet/sdk/8.0.401/

.NET workloads installed:
Configured to use loose manifests when installing new manifests.
There are no installed workloads to display.

Host:
  Version:      8.0.8
  Architecture: x64
  Commit:       08338fcaa5

.NET SDKs installed:
  8.0.401 [/usr/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 8.0.8 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 8.0.8 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  None

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

Other information

No response

[dotnet-policy-service]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

[MichalStrehovsky]

This looks like the same kind of problem one would have without native AOT in the picture and a project that has a native dependency.

Adding just a reference to the project doesn't lead to the native dependency to be binplaced/packaged where it's needed. It leaks out to the consuming project that needs to know there's extra files.

This can be fixed through e.g. packaging the referenced project into a NuGet package. The NuGet package can have msbuild targets/tasks to modify the build/publish of whoever references it. I think this should be solved through nuget as well - package the managed assembly into a NuGet, package the native static library, into a nuget, and add .targets/.props to the nuget package that do the right thing when building projects that reference the nuget.

[sweemer]

@MichalStrehovsky I see, thanks for the explanation. Now I am wondering, is there any advantage to setting PublishAot=true on a library project? It seems that even if PublishAot=true is set on a library project, most of the "magic" happens when the exe project is built.

Adding just a reference to the project doesn't lead to the native dependency to be binplaced/packaged where it's needed.

If I put my native library in runtimes/linux-x64/native and then run dotnet pack on my library project, then it is automatically included in the output directory when publishing an exe project that depends on the library - no additional targets or properties required. That's what I consider effective "encapsulation" of the native library and it's why I (incorrectly) assumed that the Native AOT mechanism would work in the same way. I think it would be useful to provide some examples of a .targets/.props file in the online docs to illustrate the difference between packaging plain old native dependencies versus Direct P/Invoke dependencies.

[MichalStrehovsky]

@MichalStrehovsky I see, thanks for the explanation. Now I am wondering, is there any advantage to setting PublishAot=true on a library project? It seems that even if PublishAot=true is set on a library project, most of the "magic" happens when the exe project is built.

If you set PublishAot on a library project, you can publish such library and you'll get a native library that can be consumed from non-.NET languages. If you don't intend to publish the library project, the presence of PublishAot=true in it only enables the trim/AOT/single-file analyzers. You can achieve the same thing by setting IsAotCompatible property to true.

If I put my native library in runtimes/linux-x64/native and then run dotnet pack on my library project, then it is automatically included in the output directory when publishing an exe project that depends on the library - no additional targets or properties required. That's what I consider effective "encapsulation" of the native library and it's why I (incorrectly) assumed that the Native AOT mechanism would work in the same way. I think it would be useful to provide some examples of a .targets/.props file in the online docs to illustrate the difference between packaging plain old native dependencies versus Direct P/Invoke dependencies.

I didn't realize NuGet now does this for DLLs. It could potentially do the same for static libraries, but that would have to be a request for NuGet. Absent a NuGet feature, it should be possible to do this by embedding a .props file in the NuGet package that adds the two items (DirectPInvoke and NativeLibrary) to any project that references the NuGet (I think you just need a .props file with the same name as the NuGet package, and place it in the build directory of the NuGet).