Allowing implementors of UmbracoIntegrationTest etc. to use them as "scoped setup fixtures"

[lars-erik]

It's been an itch for me for a long time that the integration test base classes set up and clean the Umbraco instance in [SetUp] and [TearDown] methods. It's nice that we can have a DB per test, even nicer that we can have it per fixture. But a lot of tests I write just import the schema (settings) from the sibling web project using uSync, and then a subset of the content for representable use-cases.
In those cases it'd be nice to group and organize use-cases by test fixture, but then we'll have to re-run the whole warmup and import steps. I appreciate I solved this a long time ago by instantiating something deriving the test classes, but I'd rather make it even "cleaner" and align with how NUnit works.

During the CanCon hackathon today I managed to create something that "mutates" the base class' discovered [SetUp] and [TearDown] methods into [OneTimeSetUp] and [OneTimeTearDown] such that they effectively become methods for a [SetUpFixture] and members are kept alive for the lifetime of all fixtures in the same namespace hierarchy.

I've got tons of ideas for it, but not enough time ofc. 🙈 In any case I'll bring it into my actual work next week to see if I can't shave off at least 30 seconds of running time for our 1:30min test suite. (At least!)

Would it be interesting to bring this into Umbraco.Integration.Tests? It's a bit of a hack for now, but would help a lot for implementors.
Otherwise, would it be better if the base classes implemented both [OneTimeSetUp] and [SetUp] versions each checking whether the consumer expects "per fixture" or "per test"? Or even an additional "per scope"?

I've noted issues about parallelism being a no-go etc, but I don't think that'd be possible without these changes either. 😅

Here's the repo: https://github.com/lars-erik/NUnitCompositionWithUmbraco

Looking forward to hear anybody's thoughts. 😁

[lars-erik]

I've thought about modifying the UmbracoTestAttribute, UmbracoIntegrationTest base class and possibly the UmbracoTestServerTestBase classes such that the attribute could specify whether to run setup per test or per fixture just like the database option. (Which isn't really useful since we have to control whether we've created a schema anyway)
The caveat is that NUnit will complain about [SetUp] being present anyway.
But by employing some black magic like I did with the [MakeOneTimeLifecycle] attribute in my repo the base classes could get their conditional [SetUp] and [TearDown] "removed" if per fixture is opted in.
I think that could benefit a lot of integration test suites that only need the schema, and can employ transactions to isolate tests.
I managed the latter over the week-end, and it seems awesome to be able to run all tests with possibly only one setup. 🤓