Explaining default Singleton lifetime and the use of ValueTask

[martinothamar]

This library uses Singleton lifetime configuration by default, and uses ValueTask as part of the abstractions as opposed to Task.
These decisions were performance-related and followed from a philosophy of "pay for what you use" and non-pessimization:

Pessimisation is when there is no BENEFIT to the code being "not optimal"

So in Mediator I made it a goal to make any slower configuration/implementation opt-in. If you just set up an application with a simple AddMediator() call and implement a handler which does no IO or async stuff - it will have as close to zero performance overhead as I could manage. If I had made the default lifetime Transient however, an instance of the handler would be allocated for every message sent for no reason at all. If I had made Task the return type in the abstractions, the handler would have to allocate a Task for no reason at all. This is obviously not a very useful application that I'm using as an example, but I've seen plenty of real world applications that both work fine with Singleton and have sync paths in various message handlers.

Over the years there have been questions and comments surrounding this, e.g.

As far as I understand, in most cases (web development), handlers contain calls to a database or third-party services.

and

It's difficult to imagine the app where all things you inject into handlers are singleton friendly (like ef, http clients, etc).

And my answer here would still be that

1. Great, if you do async stuff or IO, then you specifically pay that async cost at the point when you need it (as opposed to always for everyone). Still, IME a lot of handlers still have possible sync paths in addition to async paths, so there are still opportunities for non-pessimization by keeping early exits in a sync Handle-method and doing the IO in an async local/private method.
2. I actually can't think of anything I couldn't make Singleton-friendly. Below are some examples:

• EF: DbContext -> DbContextFactory
• HTTP client: HttpClient -> IHttpClientFactory (if you use AddHttpClient) or just have a static client
• ASP.NET Core HTTP context: you probably shouldn't put these into handlers, but If you do, use IHttpContextAccessor

In other words - if you chose Scoped because you happen to consume scoped services for no specific reason, your app is going to allocate lots of memory that is completely pointless (i.e. it is pessimized).

If I'm developing a small app I likely don't care (locally) about a little pessimization. But developing larger scale systems and thinking as a library developer as is my capacity in this repo, I do think it is worthwhile to consider these things and advocate for efficient resource utilization.

[goremykin]

The first thing I’d like to say is that I really appreciate the work you’ve done. I hope you won’t take this discussion as a sign of disrespect or ingratitude.

I find your arguments reasonable. However, I can't help but point out that ValueTask comes with certain limitations. Here are a few examples from the documentation where the behavior is undefined:

1. Using .Result or .GetAwaiter().GetResult() when the operation hasn't yet completed, or using them multiple times - If someone migrating from MediatR to Mediator has such code, they might be in for some unpleasant surprises.
2. Awaiting the same instance multiple times - Honestly, it's hard for me to come up with an example, but let's hope no one has such code - otherwise they might face unexpected issues when switching to Mediator.

Additionally, I’d like to note that if you're using (or used to use with MediatR) Task.WhenAll to send multiple concurrent requests, you'll now need to call .AsTask() before, which (I guess) could lead to even more overhead.

Just an idea. Maybe we should consider having two types of handlers:
IRequestHandler that behaves exactly like in MediatR - for those migrating existing code. This way, they won’t have to change anything.
And a second type of handler that can break compatibility to introduce additional benefits.

I agree that you can do a thing Singleton-friendly, and DbContextFactory / IHttpClientFactory are good examples. Additionally this behavior is configurable, so no problem at all.

[martinothamar]

I hope you won’t take this discussion as a sign of disrespect or ingratitude.

Not at all, keep it coming 😄

Using .Result or .GetAwaiter().GetResult() when the operation hasn't yet completed, or using them multiple times

Thread blocking is a bug waiting to happen regardless of Task/ValueTask. I don't think we should let those influence the design at all.

Awaiting the same instance multiple times

Yes I think this and particularly Task.WhenAll requiring AsTask or similar is the main unergonomic part of this. But it does make the Task cost apply only when you actually need it, which is perfectly in line with my goals/principles for the design. For those not familiar with the differences between Task/ValueTask we could perhaps include links to docs in the "6. Differences from MediatR" section in the README?

Maybe we should consider having two types of handlers

I suspect this would complicate and worsen the pipeline setup in the generated code. It would also be a big breaking change the exact way you describe it. Remember that this library was made with performance being one of the key differentiators from MediatR. This proposal would most likely detract from that for the benefit of compatibility with MediatR. IMO that tradeoff is not worth making.

[PhotonicX7]

I don't think it's necessary to be scoped. If you really need to open a scope to control service lifetimes, our little friend IServiceScopeFactory is always there to lend a hand.

My slight disagreement is with ValueTask being used in virtually all instances. Task overhead is typically dwarfed SUBSTANTIALLY by other concerns in most real world code. I believe in many cases this will actually force people to introduce even more allocation than normal in an attempt to use .AsTask on the return, possibly offsetting the performance you want in a majority of systems. I would normally discourage such a micro-optimization.

I guess if you drop it under the policy of "library doesn't fit the needs of those systems" in those instances it's perfectly rational. Those who are willing to adopt the design it naturally produces would end up with better performing code. Given that, I think it's worth linking information on those differences as part of the documentation I think.

Still, I applaud the effort to improve the adoption of better design decisions, as well as the transparency in your thoughts on the matter.

[Herve-M]

If you really need to open a scope to control service lifetimes, our little friend IServiceScopeFactory is always there to lend a hand.

Like any project using BackgroundService/HostedService within API or Worker host?

[martinothamar]

Task overhead is typically dwarfed SUBSTANTIALLY by other concerns in most real world code

Having seen my fair share of "Enterprise Code" I would agree, but then that cost is paid explicitly by the user, if the user wants to (or realistically doesn't care).

Another user can have other design constraints and/or generally would want to avoid alloctions in some case (sync code, IValueTaskSource, whatever) and can do so. So I still think I want the cost of worse perf to be opt-in, not the default (and certainly not unavoidable).

I would normally discourage such a micro-optimization.

I don't view it a as micro-optimization, I view at as non-pessimization 😄

I guess if you drop it under the policy of "library doesn't fit the needs of those systems" in those instances it's perfectly rational

I won't presume to know exactly what you mean by "those systems", but I'm not sure ValueTask over Task would qualify as a "deal-breaker" in choosing a specific library/system/pattern, especially for the group of users whose primary concern is convenience/less code. As a matter of fact I would flip the argument here and say that there could be users who couldn't use a library/system/pattern because the perf is ..... not good. I'm a 100% biased here though, both in terms of my view on software engineering and in terms of past experiences, so I completely understand that this question is raised from time to time