Significantly worse performance calling endpoint from System.ServiceModel in net8 than framework version

[richardcocks]

I originally raised this in the corewcf repo, but was redirected here as it was found to be a client issue.

I don't have awful performance issues in CoreWCF (Nuget System.ServiceModel), but it is noticably slower in some scenarios.

Using an endpoint which just returns a Random.Shared.Next() with a sequence number, I've been experimenting with different bindings, and with the System.ServiceModel (framework version), and System.ServiceModel (package version in net8):

.NET 8

Starting tests
Testing System.ServiceModel.BasicHttpBinding
Counter:24,489. Successfully filled:24,489 in 5007ms
Testing System.ServiceModel.NetTcpBinding
Counter:31,262. Successfully filled:31,262 in 5003ms
Testing System.ServiceModel.NetNamedPipeBinding
Counter:28,008. Successfully filled:28,008 in 5006ms
Testing System.ServiceModel.BasicHttpBinding
Counter:28,188. Successfully filled:28,188 in 5010ms
Testing System.ServiceModel.NetTcpBinding
Counter:29,188. Successfully filled:29,188 in 5012ms
Testing System.ServiceModel.NetNamedPipeBinding
Counter:28,657. Successfully filled:28,657 in 5007ms

.Net Framework 4.8.1

Starting tests
Testing localServiceEndpointHttps
Counter:19,593. Successfully filled:19,593 in 5000ms
Testing localServiceEndpointTcp
Counter:41,432. Successfully filled:41,432 in 5013ms
Testing localServiceEndpointNamedPipe
Counter:43,183. Successfully filled:43,183 in 5000ms
Testing localServiceEndpointHttps
Counter:19,392. Successfully filled:19,392 in 5000ms
Testing localServiceEndpointTcp
Counter:41,907. Successfully filled:41,907 in 5009ms
Testing localServiceEndpointNamedPipe
Counter:43,181. Successfully filled:43,181 in 5007ms

It's the same endpoint, so this is testing the client only. The HTTPS endpoint is quite a lot faster in net8, but the TCP and Named Pipe bindings are much slower.

Tagged at https://github.com/richardcocks/WcfTest/releases/tag/WcfNet8FrameworkPerformance . Run the service then each of the clients.

I realise this isn't the best way to do this. It doesn't use streamed transfer mode (or indeed streaming the actual numbers), but I wanted to demonstrate the basic performance first and I was surprised at the difference in TCP and named pipe performance between framework and net8 here.

Originally posted by @richardcocks in #395

@richardcocks, as you mentioned, what you observed is a client issue and not CoreWCF. I took a look at your repro and took some profiles. I'm seeing 17.5% of CPU time being from a spinwait caused by the .NET threadpool:

Name                                                                                           	Inc %
 coreclr!ThreadNative::SpinWait                                                                	 17.5
+ System.Private.CoreLib.il!LowLevelLifoSemaphore.Wait                                         	 16.1
|+ system.private.corelib!System.Threading.PortableThreadPool+WorkerThread::WorkerThreadStart()	 16.1
| + coreclr!CallDescrWorkerInternal                                                            	 16.1
|  + coreclr!DispatchCallSimple                                                                	 16.1
|   + coreclr!ThreadNative::KickOffThread_Worker                                               	 16.1
|    + coreclr!ManagedThreadBase_DispatchMiddle                                                	 16.1
|     + coreclr!ManagedThreadBase_DispatchOuter                                                	 16.1
|      + coreclr!ThreadNative::KickOffThread                                                   	 16.1
|       + kernel32!BaseThreadInitThunk                                                         	 16.1
|        + ntdll!RtlUserThreadStart                                                            	 16.1

This looks to be a result of the newer threadpool implementation they introduced in .NET 8.

Originally posted by @mconnew in #395