Duplicate native libraries in dotnet publish output

[Jackhammer88]

Duplicate native libraries in dotnet publish output (e.g. libtdjson.so and libtdjson.so.1.8.45)

Summary

When publishing a .NET project using dotnet publish with PublishSingleFile=true, native libraries may be duplicated in the output directory under multiple names — even when they are binary-identical. This unnecessarily increases the published size and causes confusion.

libtdjson.so
libtdjson.so.1.8.45

Both files are bit-for-bit identical, but they are copied as separate physical files instead of creating a symlink or eliminating the duplicate.

Steps to Reproduce

1. Add a native library such as libtdjson.so.1.8.45 to the project (via None or Native item group, e.g. for TDLib)

2. Publish the project using:

dotnet publish -c Release -r linux-x64 -p:PublishSingleFile=true

3. Observe the output directory (e.g. publish/)

Actual Behavior

Output contains two identical files:

$ ls -lah
-rwxr--r-- 1 user user  39M ... libtdjson.so
-rwxr--r-- 1 user user  39M ... libtdjson.so.1.8.45

$ cmp libtdjson.so libtdjson.so.1.8.45
(no output — files are identical)

$ du -h
150M .  # includes both files

$ rm libtdjson.so
$ du -h
111M .  # shows file was fully duplicated

Expected Behavior

Only one physical file should be included

Either:

The shorter name (e.g. libtdjson.so) should be a symlink to the full version (like most Linux distros do)

Or just include one version and allow user control via ItemGroup metadata

Or emit a warning if duplicate identical .so files are included

dotnet publish -c Release --self-contained -r linux-x64 -p:PublishSingleFile=true

Suggested Fix

Detect and de-duplicate native libraries by content hash (or ELF BuildID) during publish

Optionally replace duplicates with symlinks if filesystem supports it

Expose a diagnostic warning if duplicates are detected

I used csproj like this:

<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <OutputType>Exe</OutputType>
        <TargetFramework>net9.0</TargetFramework>
        <ImplicitUsings>enable</ImplicitUsings>
        <Nullable>enable</Nullable>
    </PropertyGroup>

    <ItemGroup>
      <PackageReference Include="TDLib" Version="1.8.45" />
      <PackageReference Include="tdlib.native" Version="1.8.45" />
    </ItemGroup>

</Project>

[ForNeVeR]

Hmm, actually, I don't think the numbered library (libtdjson.so.1.8.45) is required at all. We might wish to just get rid of it.

[Jackhammer88]

Absolutely. The output bundle size will shrink by 39M.

[Jackhammer88]

#133
I've added removal of all versioned libtdjson.so files and re-ran the CI/CD. At this point, only the unversioned libtdjson.so remains in tdlib.native.ubuntu-22.04.x86-64.zip.