Add support for LoongArch64. #1971

LuckyXu-HF · 2022-03-29T10:00:31Z

Add LoongArch64 architecture.

dnfadmin · 2022-03-29T10:00:44Z

All CLA requirements met.

adamsitnik

Hi @HFDude

Big thanks for your contribution! Overall the code looks good to me, but I don't have the possibility to test it. Is there any chance you have an access to LoongArch64 machine and could provide a screen shot with sample results?

cd .\samples\BenchmarkDotNet.Samples\
dotnet run -c Release -f net5.0 --filter *Basic* --job dry

LuckyXu-HF · 2022-03-30T07:44:31Z

Hi @HFDude

Big thanks for your contribution! Overall the code looks good to me, but I don't have the possibility to test it. Is there any chance you have an access to LoongArch64 machine and could provide a screen shot with sample results?
cd .\samples\BenchmarkDotNet.Samples\
dotnet run -c Release -f net5.0 --filter *Basic* --job dry

OK, thank you for your guidance and take time to review and give me suggestions.

By the way, may I ask why the size of BenchmarkDotNet.dll compiled by the local BenchmarkDotNet project is only 953K , but the size of the BenchmarkDotNet.dll which download from https://www.nuget.org/packages/BenchmarkDotNet/0.13.1 is 3.4M ? Is there any other projects need to be packed into BenchmarkDotNet.dll together?

Yesterday's tests on Ubuntu azure pipelines environment takes so long time, this is indeed abnormal. I have tested this on my local environment Debian10-amd64, and all test cases have passed. The full log can be seen in BenchmarkDotnet-buildandtest-Debian10-arm64.log

Local environment Debian10-amd64:

~$ lsb_release -a
No LSB modules are available.
Distributor ID:	Debian
Description:	Debian GNU/Linux 10 (buster)
Release:	10
Codename:	buster

~$ uname -a
Linux debian 4.19.0-20-amd64 #1 SMP Debian 4.19.235-1 (2022-03-17) x86_64 GNU/Linux

Currently we don’t support dotnet-sdk-5.0, so I add net6.0 TargetFrameworks in BenchmarkDotNet.Samples.csproj:

$ git diff samples/BenchmarkDotNet.Samples/BenchmarkDotNet.Samples.csproj
diff --git a/samples/BenchmarkDotNet.Samples/BenchmarkDotNet.Samples.csproj b/samples/BenchmarkDotNet.Samples/BenchmarkDotNet.Samples.csproj
index 299e4138..414035a9 100644
--- a/samples/BenchmarkDotNet.Samples/BenchmarkDotNet.Samples.csproj
+++ b/samples/BenchmarkDotNet.Samples/BenchmarkDotNet.Samples.csproj
@@ -2,7 +2,7 @@
   <Import Project="..\..\build\common.props" />
   <PropertyGroup>
     <AssemblyTitle>BenchmarkDotNet.Samples</AssemblyTitle>
-    <TargetFrameworks>net461;net5.0</TargetFrameworks>
+    <TargetFrameworks>net461;net5.0;net6.0</TargetFrameworks>
     <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
     <AssemblyName>BenchmarkDotNet.Samples</AssemblyName>
     <OutputType>Exe</OutputType>

Then run on LoongArch64 machine:

cd .\samples\BenchmarkDotNet.Samples\
dotnet run -c Release -f net6.0 --filter *Basic* --job dry

Then I choose BenchmarkDotNet.Samples.Algo_Md5VsSha256.Md5 to run, it seems that it needs another dll to support LoongArch64 to recognize LoongArch64 architecture. So I guess these testcases can't be tested on LA64 machine unless we add LoongArch64 architecture to some other related projects.

adamsitnik · 2022-03-30T08:05:06Z

@HFDude I am sorry, I have provided invalid filter. On Unix we need to escape the asterisks with ', otherwise the OS replaces the filter with list of files in current folder that match the filter (in your case InfoBasic.cs). Could you please re-run the following command in the ./samples/BenchmarkDotNet.Samples directory?

dotnet run -c Release -f net6.0 --filter '*Basic*' --job dry

adamsitnik · 2022-03-30T08:10:52Z

By the way, may I ask why the size of BenchmarkDotNet.dll compiled by the local BenchmarkDotNet project is only 953K , but the size of the BenchmarkDotNet.dll which download from https://www.nuget.org/packages/BenchmarkDotNet/0.13.1 is 3.4M ? Is there any other projects need to be packed into BenchmarkDotNet.dll together?

That is a great question! The official build is always using Windows machine, which builds the disassembler projects and copies the produced .exe files to the resources:

BenchmarkDotNet/src/BenchmarkDotNet/BenchmarkDotNet.csproj

Lines 32 to 43 in 09d9396

    
           <ItemGroup Condition="'$(OS)' == 'Windows_NT'"> 
        
             <ProjectReference Include="..\BenchmarkDotNet.Disassembler.x64\BenchmarkDotNet.Disassembler.x64.csproj"> 
        
               <ReferenceOutputAssembly>false</ReferenceOutputAssembly> 
        
             </ProjectReference> 
        
             <ProjectReference Include="..\BenchmarkDotNet.Disassembler.x86\BenchmarkDotNet.Disassembler.x86.csproj"> 
        
               <ReferenceOutputAssembly>false</ReferenceOutputAssembly> 
        
             </ProjectReference> 
        
             <EmbeddedResource Include="Disassemblers\net461\win7-x86\BenchmarkDotNet.Disassembler.x86.exe" /> 
        
             <EmbeddedResource Include="Disassemblers\net461\win7-x64\BenchmarkDotNet.Disassembler.x64.exe" /> 
        
             <EmbeddedResource Include="Disassemblers\net461\win7-x64\Microsoft.Diagnostics.Runtime.dll" /> 
        
             <EmbeddedResource Include="Disassemblers\net461\win7-x64\Iced.dll" /> 
        
           </ItemGroup>

The full answer is https://adamsitnik.com/Disassembly-Diagnoser/#desktop-net, but in general you should not need these exes if you don't run on Windows.

LuckyXu-HF · 2022-03-30T09:02:27Z

@HFDude I am sorry, I have provided invalid filter. On Unix we need to escape the asterisks with ', otherwise the OS replaces the filter with list of files in current folder that match the filter (in your case InfoBasic.cs). Could you please re-run the following command in the ./samples/BenchmarkDotNet.Samples directory?
dotnet run -c Release -f net6.0 --filter '*Basic*' --job dry

OK, it reports the same error:

It seems that it needs another dll to support LoongArch64 to recognize LoongArch64 architecture. So I guess these testcases can't be tested on LA64 machine unless we add LoongArch64 architecture to some other related projects.

adamsitnik · 2022-03-30T09:18:42Z

It seems that it needs another dll to support LoongArch64 to recognize LoongArch64 architecture. So I guess these testcases can't be tested on LA64 machine unless we add LoongArch64 architecture to some other related projects.

It seems that MSBuild does not recognize LOONGARCH64 as architecture name. We had similar issues with S390x in the past.

Could you please remove the following line:

BenchmarkDotNet/src/BenchmarkDotNet/Extensions/ConfigurationExtensions.cs

Lines 25 to 26 in 43eaf48

    
           case Platform.LoongArch64: 
        
               return "LOONGARCH64";

To see if it works when we use AnyCPU as the Platform?

LuckyXu-HF · 2022-03-30T09:39:57Z

By the way, may I ask why the size of BenchmarkDotNet.dll compiled by the local BenchmarkDotNet project is only 953K , but the size of the BenchmarkDotNet.dll which download from https://www.nuget.org/packages/BenchmarkDotNet/0.13.1 is 3.4M ? Is there any other projects need to be packed into BenchmarkDotNet.dll together?

That is a great question! The official build is always using Windows machine, which builds the disassembler projects and copies the produced .exe files to the resources:

BenchmarkDotNet/src/BenchmarkDotNet/BenchmarkDotNet.csproj

Lines 32 to 43 in 09d9396

<ItemGroup Condition="'$(OS)' == 'Windows_NT'">

<ProjectReference Include="..\BenchmarkDotNet.Disassembler.x64\BenchmarkDotNet.Disassembler.x64.csproj">

<ReferenceOutputAssembly>false</ReferenceOutputAssembly>

</ProjectReference>

<ProjectReference Include="..\BenchmarkDotNet.Disassembler.x86\BenchmarkDotNet.Disassembler.x86.csproj">

<ReferenceOutputAssembly>false</ReferenceOutputAssembly>

</ProjectReference>

<EmbeddedResource Include="Disassemblers\net461\win7-x86\BenchmarkDotNet.Disassembler.x86.exe" />

<EmbeddedResource Include="Disassemblers\net461\win7-x64\BenchmarkDotNet.Disassembler.x64.exe" />

<EmbeddedResource Include="Disassemblers\net461\win7-x64\Microsoft.Diagnostics.Runtime.dll" />

<EmbeddedResource Include="Disassemblers\net461\win7-x64\Iced.dll" />

</ItemGroup>

The full answer is https://adamsitnik.com/Disassembly-Diagnoser/#desktop-net, but in general you should not need these exes if you don't run on Windows.

@adamsitnik ,thanks for your patient and comprehensive guide.

The full answer is https://adamsitnik.com/Disassembly-Diagnoser/#desktop-net, but in general you should not need these exes if you don't run on Windows.

What puzzled me now is that I can't use BenchmarkDotNet.dll which is compiled in my local linux-x64 environment to run performance:release6.0 testcases, it will report the same errors on both linux-x64 and linux-LA64. （I have confirmed the The relevant version of BenchmarkDotNet.dll is right）

But while I use BenchmarkDotNet.dll from nuget , it's ok to run performance:release6.0 testcases on linux-x64.

adamsitnik · 2022-03-30T09:50:37Z

@HFDude if my suggestion from #1971 (comment) helps, then I am going to merge your PR immediately and our CI is going to publish a "proper" NuGet package for you to our CI feed

LuckyXu-HF · 2022-03-30T10:02:52Z

@HFDude if my suggestion from #1971 (comment) helps, then I am going to merge your PR immediately and our CI is going to publish a "proper" NuGet package for you to our CI feed

Yes, it worked. Full log: BenchmarkDotnet-linuxLA64-test1.log

Thank you very much!

src/BenchmarkDotNet/Extensions/ConfigurationExtensions.cs

adamsitnik

LGTM, big thanks for your contribution @HFDude !

LuckyXu-HF · 2022-03-30T10:28:50Z

LGTM, big thanks for your contribution @HFDude !

Big thanks again for your great help! @adamsitnik !

adamsitnik · 2022-03-30T10:46:12Z

The CI failure is unrelated (#1925), I am merging.

adamsitnik · 2022-03-30T10:49:15Z

@HFDude once this build finishes (30-60 minutes from now) and it's green (no flaky test failures) the CI is going to publish 0.13.1.1751 NuGet package(s) to our CI feed. You should be able to consume it by adding the feed to your nuget.config file.

<packageSources>
  <add key="bdn-ci" value="https://ci.appveyor.com/nuget/benchmarkdotnet" />
</packageSources>

LuckyXu-HF · 2022-03-30T12:16:32Z

@HFDude once this build finishes (30-60 minutes from now) and it's green (no flaky test failures) the CI is going to publish 0.13.1.1751 NuGet package(s) to our CI feed. You should be able to consume it by adding the feed to your nuget.config file.
<packageSources>
  <add key="bdn-ci" value="https://ci.appveyor.com/nuget/benchmarkdotnet" />
</packageSources>

OK, I got it ! And I can successful run performance:release6.0 testcases on linux-LoongArch64 now!

@adamsitnik ,Thank you very much!

~/performance/artifacts/bin/MicroBenchmarks/Release/net6.0$ dotnet MicroBenchmarks.dll
Available Benchmarks:
  #0   BilinearTest
  #1   Burgers
  #2   ByteMark
  #3   CscBench
  #4   LinqBenchmarks
...
...
 #438 Perf_XmlNode


You should select the target benchmark(s). Please, print a number of a benchmark (e.g. `0`) or a contained benchmark caption (e.g. `BilinearTest`).
If you want to select few, please separate them with space ` ` (e.g. `1 2 3`).
You can also provide the class name in console arguments by using --filter. (e.g. `--filter '*BilinearTest*'`).
3

// Validating benchmarks:
// ***** BenchmarkRunner: Start   *****
// ***** Found 406 benchmark(s) in total *****
// ***** Building 1 exe(s) in Parallel: Start   *****
// start dotnet  restore /p:DebugType=portable -bl:benchmarkdotnet.binlog /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true in /data/xuliangyu/aaa/performance/artifacts/bin/MicroBenchmarks/Release/net6.0/a6d63f1f-4dcb-45f1-8a70-f0cc010d50db
// command took 9.09s and exited with 0
// start dotnet  build -c Release /p:DebugType=portable -bl:benchmarkdotnet.binlog --no-restore /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true in /data/xuliangyu/aaa/performance/artifacts/bin/MicroBenchmarks/Release/net6.0/a6d63f1f-4dcb-45f1-8a70-f0cc010d50db
// command took 53.7s and exited with 0
// ***** Done, took 00:01:05 (65.52 sec)   *****
// Found 2 benchmarks:
//   CscBench.CompileTest: Job-FCXLHJ(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:DebugType=portable,-bl:benchmarkdotnet.binlog, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1)
//   CscBench.DatflowTest: Job-FCXLHJ(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:DebugType=portable,-bl:benchmarkdotnet.binlog, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1)

// **************************
// Benchmark: CscBench.CompileTest: Job-FCXLHJ(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:DebugType=portable,-bl:benchmarkdotnet.binlog, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1)
// *** Execute ***
// Launch: 1 / 1
// Execute: dotnet a6d63f1f-4dcb-45f1-8a70-f0cc010d50db.dll --benchmarkName CscBench.CompileTest --job "PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:DebugType=portable,-bl:benchmarkdotnet.binlog, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1" --benchmarkId 0 in /data/xuliangyu/aaa/performance/artifacts/bin/MicroBenchmarks/Release/net6.0/a6d63f1f-4dcb-45f1-8a70-f0cc010d50db/bin/Release/net6.0
...
...
// BeforeAnythingElse

// Benchmark Process Environment Information:
// Runtime=.NET 6.0.3 (42.42.42.42424), LoongArch64 RyuJIT
// GC=Concurrent Workstation
// Job: Job-KGNPHI(PowerPlanMode=00000000-0000-0000-0000-000000000000, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1)

OverheadJitting  1: 1 op, 519381.00 ns, 519.3810 us/op
WorkloadJitting  1: 1 op, 3432740297.00 ns, 3.4327 s/op

OverheadJitting  2: 1 op, 970.00 ns, 970.0000 ns/op
WorkloadJitting  2: 1 op, 1046283088.00 ns, 1.0463 s/op

WorkloadWarmup   1: 1 op, 915047434.00 ns, 915.0474 ms/op

// BeforeActualRun
WorkloadActual   1: 1 op, 1011612854.00 ns, 1.0116 s/op
WorkloadActual   2: 1 op, 1018340870.00 ns, 1.0183 s/op
WorkloadActual   3: 1 op, 870055599.00 ns, 870.0556 ms/op
WorkloadActual   4: 1 op, 966901400.00 ns, 966.9014 ms/op
WorkloadActual   5: 1 op, 826897078.00 ns, 826.8971 ms/op
WorkloadActual   6: 1 op, 827985399.00 ns, 827.9854 ms/op
WorkloadActual   7: 1 op, 826334717.00 ns, 826.3347 ms/op
WorkloadActual   8: 1 op, 1011806290.00 ns, 1.0118 s/op
WorkloadActual   9: 1 op, 1003737802.00 ns, 1.0037 s/op
WorkloadActual  10: 1 op, 1001885599.00 ns, 1.0019 s/op
WorkloadActual  11: 1 op, 999917836.00 ns, 999.9178 ms/op
WorkloadActual  12: 1 op, 1003938340.00 ns, 1.0039 s/op
WorkloadActual  13: 1 op, 846382892.00 ns, 846.3829 ms/op
WorkloadActual  14: 1 op, 822978370.00 ns, 822.9784 ms/op
WorkloadActual  15: 1 op, 1003039938.00 ns, 1.0030 s/op
WorkloadActual  16: 1 op, 821671307.00 ns, 821.6713 ms/op
WorkloadActual  17: 1 op, 981547786.00 ns, 981.5478 ms/op
WorkloadActual  18: 1 op, 987987231.00 ns, 987.9872 ms/op
WorkloadActual  19: 1 op, 982453696.00 ns, 982.4537 ms/op
WorkloadActual  20: 1 op, 820786704.00 ns, 820.7867 ms/op

// AfterActualRun
WorkloadResult   1: 1 op, 1011612854.00 ns, 1.0116 s/op
WorkloadResult   2: 1 op, 1018340870.00 ns, 1.0183 s/op
WorkloadResult   3: 1 op, 870055599.00 ns, 870.0556 ms/op
WorkloadResult   4: 1 op, 966901400.00 ns, 966.9014 ms/op
WorkloadResult   5: 1 op, 826897078.00 ns, 826.8971 ms/op
WorkloadResult   6: 1 op, 827985399.00 ns, 827.9854 ms/op
WorkloadResult   7: 1 op, 826334717.00 ns, 826.3347 ms/op
WorkloadResult   8: 1 op, 1011806290.00 ns, 1.0118 s/op
WorkloadResult   9: 1 op, 1003737802.00 ns, 1.0037 s/op
WorkloadResult  10: 1 op, 1001885599.00 ns, 1.0019 s/op
WorkloadResult  11: 1 op, 999917836.00 ns, 999.9178 ms/op
WorkloadResult  12: 1 op, 1003938340.00 ns, 1.0039 s/op
...
...

adamsitnik · 2022-03-30T13:36:50Z

@HFDude awesome! btw we are most likely going to update BDN version used by the performance repo very soon (so you won't need any local changes)

LuckyXu-HF · 2022-03-31T01:24:55Z

@HFDude awesome! btw we are most likely going to update BDN version used by the performance repo very soon (so you won't need any local changes)

Looking forward it!

adamsitnik · 2022-03-31T07:29:17Z

@HFDude dotnet/performance#2341 (I expect it to get merged today as soon as USA folks start their workday)

Add support for LoongArch64.

43eaf48

adamsitnik reviewed Mar 29, 2022

View reviewed changes

LuckyXu-HF marked this pull request as draft March 29, 2022 15:33

adamsitnik reviewed Mar 30, 2022

View reviewed changes

src/BenchmarkDotNet/Extensions/ConfigurationExtensions.cs Outdated Show resolved Hide resolved

Apply suggestions from code review

faf5296

adamsitnik marked this pull request as ready for review March 30, 2022 10:17

adamsitnik approved these changes Mar 30, 2022

View reviewed changes

adamsitnik merged commit 5f0db4f into dotnet:master Mar 30, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for LoongArch64. #1971

Add support for LoongArch64. #1971

LuckyXu-HF commented Mar 29, 2022

dnfadmin commented Mar 29, 2022 •

edited

Loading

adamsitnik left a comment

LuckyXu-HF commented Mar 30, 2022

adamsitnik commented Mar 30, 2022

adamsitnik commented Mar 30, 2022 •

edited

Loading

LuckyXu-HF commented Mar 30, 2022

adamsitnik commented Mar 30, 2022

LuckyXu-HF commented Mar 30, 2022

adamsitnik commented Mar 30, 2022

LuckyXu-HF commented Mar 30, 2022

adamsitnik left a comment

LuckyXu-HF commented Mar 30, 2022

adamsitnik commented Mar 30, 2022

adamsitnik commented Mar 30, 2022

LuckyXu-HF commented Mar 30, 2022

adamsitnik commented Mar 30, 2022

LuckyXu-HF commented Mar 31, 2022

adamsitnik commented Mar 31, 2022

Add support for LoongArch64. #1971

Add support for LoongArch64. #1971

Conversation

LuckyXu-HF commented Mar 29, 2022

dnfadmin commented Mar 29, 2022 • edited Loading

adamsitnik left a comment

Choose a reason for hiding this comment

LuckyXu-HF commented Mar 30, 2022

adamsitnik commented Mar 30, 2022

adamsitnik commented Mar 30, 2022 • edited Loading

LuckyXu-HF commented Mar 30, 2022

adamsitnik commented Mar 30, 2022

LuckyXu-HF commented Mar 30, 2022

adamsitnik commented Mar 30, 2022

LuckyXu-HF commented Mar 30, 2022

adamsitnik left a comment

Choose a reason for hiding this comment

LuckyXu-HF commented Mar 30, 2022

adamsitnik commented Mar 30, 2022

adamsitnik commented Mar 30, 2022

LuckyXu-HF commented Mar 30, 2022

adamsitnik commented Mar 30, 2022

LuckyXu-HF commented Mar 31, 2022

adamsitnik commented Mar 31, 2022

dnfadmin commented Mar 29, 2022 •

edited

Loading

adamsitnik commented Mar 30, 2022 •

edited

Loading