Adding indexes to mt_events through StoreOptions

[Mangatome]

Hello, thanks for the nice project.

I'm currently using FeatureSchemaBase to add some views to my event store. I'm also adding indexes to document tables using opt.Schema.For<TDoc>(). Both are working like a charm.

However I would like to also add indexes to the event table (mt_events) but there doesn't seem to be an API to do this. I have workarounds using ISchemaObject in my feature schema, but of course Marten drops and recreates the indexes at every migration.

Is there any way to do this through Marten instead?

[mysticmind]

Unlike user specified documents, mt_events is a special table for event sourcing and I don't see a good reason for you to create an index on it. Can you elaborate for what reason are you wanting to create an index? Most likely, you are using the wrong tool for your use case at hand. You would better of doing some sort of projections from the events and work with it.

Besides, there is no provision to create indexes on mt_events and it is designed that way.

[Mangatome]

My use case is that I am using Marten as a backing store for a mobile app data analytics platform. Events are generated on the client, transmitted to the server and therefore warehoused into Marten. I needed an event store onto Postgres, with facilities for replaying streams and computing projections in the background, as well as the ability to define custom views that can then serve as data source for an analytics dashboard app.

Marten is a good fit for my requirements. For the most part it works well. I am already using projections, async daemon, indexes on document tables, the various Postgres optims, and custom views and materialized views to speed up some querying. What I am not using is direct querying in C# through Marten, because the consumer of my data is not the same application as the one producing it.

As I said, my dashboards app already consumes some views based on document tables. But sometimes one just wants to see an arbitrary visualization computed against all events - in this case, indexing some data fields of the events table is both easy to do and helpful in terms of performance.

I wish there was a built-in way to maintain indexes to mt_events. But short of that, the workarounds I can think of are all kind of bad:

• defining indexes outside of FeatureSchemaBase: they get dropped every time configured changes are applied
• projecting events into custom document table: I would probably need one projected document per source event, multiplying the needed storage space by 2 at least.
• giving up on event store, using document store instead: I'd probably have to rewrite my own facilities for projections and async daemon...

You're probably right that I'm not using the correct solution for my use case. I'm very eager to read your suggestions. But from my perspective Marten is a 95% good fit to what I'm trying to do, with just little limitations preventing it from being a 100% match.

My suggestion would be to increase a tiny bit the customizability of that table. I can think of several options. One could be an API in StoreOptions like opt.Schema.For<TDoc>(), maybe something like opt.Schema.ForEvents(). If that goes against the design, what about this : there could be an option to customize the IgnoredIndexes of the EventTable created by EventGraph so that at least it wouldn't drop custom indexes on migration.