Create JsonEventStore

[cjnolet]

In mango, we recently created a JsonAttributeStore utility class that allows json and generically nested hashmaps to be flattened into attributes to be used in the EventStore and EntityStore implementations.

The problem is, it has partitioned a lot of assumptions that can be made about the underlying events. For instance, if I could assume that all of my events contain attributes that have been flattened from an arbitrarily nested document, I can do things like querying based on nested attributes in Spark SQL. I can also do things with my UI to determine and display the nested layers. I cannot do such things if my only assumption is that I have an attribute which is a simple key/value object.

I propose we create a new Store that is based on the event store, called EventDocumentStore or JsonEventStore. This store should assume any objects are arbitrarily nested and can be marshalled back into json when queried. This will allow us to manage the schema of the entire document (which items are arrays vs objects) and provide that schema to things like Spark SQL.

Methods in the JsonEventStore interface could look like this:

save(Iterable<JsonEvent> jsonDocuments);

JsonEvent query(Node query, Auths auths);

Where a JsonEvent looks like this:

class JsonEvent {
    String type;
    String id;
    Long timestamp;
    Map[String, Object] document;
} 

The reason I want to differentiate this object from Mango's Event object is that I want to separate the internal indexing components of the event from the document itself. An Event's ID, for instance, could possibly align with an ID in the json document itself, but it doesn't need to. This gives it flexibility as the Event's fields are just metadata that inform the store how to index while the documents themselves are completely separate entities.