Experiment Grouping

[Datron]

The problem

Experiments overlap with each other and due to the cascading nature of context aware configs - can cause cases where the end result of a config query is not easily determined by the user as there maybe a lot of experiments interacting with each other in this way.

Note: there are cases where users of superposition would want their experiments to overlap or don't really care if they do.

Proposal

Introduce experiment groups, which is created with a context. The context helps isolate traffic to this experiment, which can also take a number representing how much percentage of traffic you would like to provide to this group. Experiments added to this group will NOT overlap with each other. Experiments outside of the group can overlap with experiments inside the group. The experiments inside a group are isolated only from each other.

Experiment groups table structure

CREATE TABLE IF NOT EXISTS experiment_groups(
    hash TEXT PRIMARY KEY,
    context JSON NOT NULL,
    traffic_percentage integer NOT NULL CONSTRAINT traffic_percentage_range CHECK (traffic_percentage >= 0 AND traffic_percentage <= 100),
    experiments_inside_ids TEXT[] NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL,
    created_by TEXT NOT NULL,
    last_modified_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL,
    last_modified_by TEXT NOT NULL
)