Move from dynamic IL generation to source generators

[marklio]

I built this library back in 2006 as an experiment to see what I could do with a brand new CLR feature called "Lightweight Code Generation", which allowed IL code to be generated at runtime. In the end, it is a tool to keep from having to write a bunch of code for the record parsers and writers (converters and unconverters). The downside is that every process start has to generate those, even though they are basically static content. There's a bit of interaction with the indexing/query rewriting, but there's no data that shows that improves perf.

Anyway, I've used T4 in other projects to generate bunches of code instead of writing it out, but that has some annoyances. C# now has source generators, which is kind of built for this purpose, and I'd like to give it a shot. At first, I'd keep the indexing/query rewriting stuff the same, but I think it's possible to get the indexing stuff to work without query rewriting.

I've played with source generators, but this would be the first time using them for something "real".

One of the big things to decide is how to specify the record layouts. Today, these are done with attributes, which work great, but using them in source generators is tricky because you don't get much help from the attributes, it's just data.

Options:

• Continue using attributes
  • As-is: This doesn't help much. We still need "generator-side" types to help us out
  • As data: basically strip the attributes down to just key-value pairs of data. This is nice because we still get compiler-supported syntax/correctness support, and we'd move functionality to the generator side.
• Create some new way to specify these as data, but still in code. This would be paired with the same generator-side support. The advantage here would be we wouldn't be constrained by what attributes can do, and could have more interesting formats, but it would be easy for it to get out of hand since once you're in C#, you can write anything. These could either be deleted by the generator, or replaced, or whatever.
• Create some DSL that is embedded in comments (or elsewhere in code, but that's harder). This is harder to get compiler support for, but we can still generate analyzer-style errors/warnings, but it means implementing parsing of the DSL

There are clearly permutations of the above, but I'm curious if anyone has other ideas. I think as a start, I'll keep the attributes as data, and move the functionality over to the generators.