Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions ExperimentFramework.slnx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@
<Folder Name="/src/">
<Project Path="benchmarks/ExperimentFramework.Benchmarks/ExperimentFramework.Benchmarks.csproj" />
<Project Path="src/ExperimentFramework.Generators/ExperimentFramework.Generators.csproj" />
<Project Path="src/ExperimentFramework.Metrics.Exporters/ExperimentFramework.Metrics.Exporters.csproj" />
<Project Path="src/ExperimentFramework.Resilience/ExperimentFramework.Resilience.csproj" />
<Project Path="src/ExperimentFramework/ExperimentFramework.csproj" />
</Folder>
<Folder Name="/tests/">
Expand Down
123 changes: 107 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,31 +1,30 @@
# ExperimentFramework

A .NET framework for runtime-switchable A/B testing, feature flags, trial fallback, and comprehensive observability.
A .NET library for routing service calls through configurable trials based on feature flags, configuration values, or custom routing logic.

**Version 0.1.0** - Production-ready with source-generated zero-overhead proxies
## Features

## Key Features

**Multiple Selection Modes**
**Selection Modes**
- Boolean feature flags (`true`/`false` keys)
- Configuration values (string variants)
- Variant feature flags (IVariantFeatureManager integration)
- Sticky routing (deterministic user/session-based A/B testing)
- Sticky routing (deterministic user/session-based routing)

**Resilience**
- Timeout enforcement with fallback
- Circuit breaker (Polly integration)
- Kill switch for disabling experiments at runtime

**Enterprise Observability**
- OpenTelemetry distributed tracing support
**Observability**
- OpenTelemetry tracing
- Metrics collection (Prometheus, OpenTelemetry)
- Built-in benchmarking and error logging
- Zero overhead when telemetry disabled

**Flexible Configuration**
- Custom naming conventions
**Configuration**
- Error policies with fallback strategies
- Custom naming conventions
- Decorator pipeline for cross-cutting concerns

**Type-Safe & DI-Friendly**
- Composition-root driven registration
- Full dependency injection integration
- Strongly-typed builder API
- Dependency injection integration

## Quick Start

Expand Down Expand Up @@ -230,6 +229,98 @@ Tries ordered list of fallback trials:
// Fine-grained control over fallback strategy
```

## Timeout Enforcement

Prevent slow trials from degrading system performance:

```csharp
var experiments = ExperimentFrameworkBuilder.Create()
.Define<IMyDatabase>(c => c
.UsingFeatureFlag("UseCloudDb")
.AddDefaultTrial<LocalDb>("false")
.AddTrial<CloudDb>("true")
.OnErrorRedirectAndReplayDefault())
.WithTimeout(TimeSpan.FromSeconds(5), TimeoutAction.FallbackToDefault)
.UseDispatchProxy();
```

**Actions:**
- `TimeoutAction.ThrowException` - Throw `TimeoutException` when trial exceeds timeout
- `TimeoutAction.FallbackToDefault` - Automatically fallback to default trial on timeout

See [Timeout Enforcement Guide](docs/user-guide/timeout-enforcement.md) for detailed examples.

## Circuit Breaker

Automatically disable failing trials using Polly:

```bash
dotnet add package ExperimentFramework.Resilience
```

```csharp
var experiments = ExperimentFrameworkBuilder.Create()
.Define<IMyService>(c => c
.UsingFeatureFlag("UseNewService")
.AddDefaultTrial<StableService>("false")
.AddTrial<NewService>("true")
.OnErrorRedirectAndReplayDefault())
.WithCircuitBreaker(options =>
{
options.FailureRatioThreshold = 0.5; // Open after 50% failure rate
options.MinimumThroughput = 10; // Need 10 calls to assess
options.SamplingDuration = TimeSpan.FromSeconds(30);
options.BreakDuration = TimeSpan.FromSeconds(60);
options.OnCircuitOpen = CircuitBreakerAction.FallbackToDefault;
})
.UseDispatchProxy();
```

See [Circuit Breaker Guide](docs/user-guide/circuit-breaker.md) for advanced configuration.

## Metrics Collection

Track experiment performance with Prometheus or OpenTelemetry:

```bash
dotnet add package ExperimentFramework.Metrics.Exporters
```

```csharp
var prometheusMetrics = new PrometheusExperimentMetrics();
var experiments = ExperimentFrameworkBuilder.Create()
.Define<IMyService>(c => c.UsingFeatureFlag("MyFeature")...)
.WithMetrics(prometheusMetrics)
.UseDispatchProxy();

app.MapGet("/metrics", () => prometheusMetrics.GeneratePrometheusOutput());
```

**Collected Metrics:**
- `experiment_invocations_total` (counter) - Total invocations per experiment/trial
- `experiment_duration_seconds` (histogram) - Duration of each invocation

See [Metrics Guide](docs/user-guide/metrics.md) for OpenTelemetry integration and Grafana dashboards.

## Kill Switch

Emergency shutdown for problematic experiments:

```csharp
var killSwitch = new InMemoryKillSwitchProvider();

var experiments = ExperimentFrameworkBuilder.Create()
.Define<IMyDatabase>(c => c.UsingFeatureFlag("UseCloudDb")...)
.WithKillSwitch(killSwitch)
.UseDispatchProxy();

// Emergency disable
killSwitch.DisableExperiment(typeof(IMyDatabase));
killSwitch.DisableTrial(typeof(IMyDatabase), "cloud");
```

See [Kill Switch Guide](docs/user-guide/kill-switch.md) for distributed scenarios with Redis.

## Custom Naming Conventions

Replace default selector naming:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -247,7 +247,7 @@ public string[] Direct_RepeatedSync_10Calls()
using var scope = _directServiceProvider.CreateScope();
var service = scope.ServiceProvider.GetRequiredService<ISimpleService>();
var results = new string[10];
for (int i = 0; i < 10; i++)
for (var i = 0; i < 10; i++)
{
results[i] = service.GetValue();
}
Expand All @@ -260,7 +260,7 @@ public string[] Proxied_Configuration_RepeatedSync_10Calls()
using var scope = _proxiedConfigurationServiceProvider.CreateScope();
var service = scope.ServiceProvider.GetRequiredService<ISimpleService>();
var results = new string[10];
for (int i = 0; i < 10; i++)
for (var i = 0; i < 10; i++)
{
results[i] = service.GetValue();
}
Expand All @@ -273,7 +273,7 @@ public string[] Direct_RepeatedSync_100Calls()
using var scope = _directServiceProvider.CreateScope();
var service = scope.ServiceProvider.GetRequiredService<ISimpleService>();
var results = new string[100];
for (int i = 0; i < 100; i++)
for (var i = 0; i < 100; i++)
{
results[i] = service.GetValue();
}
Expand All @@ -286,7 +286,7 @@ public string[] Proxied_Configuration_RepeatedSync_100Calls()
using var scope = _proxiedConfigurationServiceProvider.CreateScope();
var service = scope.ServiceProvider.GetRequiredService<ISimpleService>();
var results = new string[100];
for (int i = 0; i < 100; i++)
for (var i = 0; i < 100; i++)
{
results[i] = service.GetValue();
}
Expand All @@ -299,7 +299,7 @@ public async Task<string[]> Direct_RepeatedAsync_10Calls()
using var scope = _directServiceProvider.CreateScope();
var service = scope.ServiceProvider.GetRequiredService<ISimpleService>();
var results = new string[10];
for (int i = 0; i < 10; i++)
for (var i = 0; i < 10; i++)
{
results[i] = await service.GetValueAsync();
}
Expand All @@ -312,7 +312,7 @@ public async Task<string[]> Proxied_Configuration_RepeatedAsync_10Calls()
using var scope = _proxiedConfigurationServiceProvider.CreateScope();
var service = scope.ServiceProvider.GetRequiredService<ISimpleService>();
var results = new string[10];
for (int i = 0; i < 10; i++)
for (var i = 0; i < 10; i++)
{
results[i] = await service.GetValueAsync();
}
Expand All @@ -325,7 +325,7 @@ public async Task<string[]> Direct_RepeatedAsync_100Calls()
using var scope = _directServiceProvider.CreateScope();
var service = scope.ServiceProvider.GetRequiredService<ISimpleService>();
var results = new string[100];
for (int i = 0; i < 100; i++)
for (var i = 0; i < 100; i++)
{
results[i] = await service.GetValueAsync();
}
Expand All @@ -338,7 +338,7 @@ public async Task<string[]> Proxied_Configuration_RepeatedAsync_100Calls()
using var scope = _proxiedConfigurationServiceProvider.CreateScope();
var service = scope.ServiceProvider.GetRequiredService<ISimpleService>();
var results = new string[100];
for (int i = 0; i < 100; i++)
for (var i = 0; i < 100; i++)
{
results[i] = await service.GetValueAsync();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,7 @@ public string[] Direct_CPUBound_Repeated_10Calls()
using var scope = _directCacheProvider.CreateScope();
var cache = scope.ServiceProvider.GetRequiredService<ICache>();
var results = new string[10];
for (int i = 0; i < 10; i++)
for (var i = 0; i < 10; i++)
{
results[i] = cache.ComputeHash($"test-data-{i}");
}
Expand All @@ -272,7 +272,7 @@ public string[] Proxied_CPUBound_Repeated_10Calls()
using var scope = _proxiedCacheProvider.CreateScope();
var cache = scope.ServiceProvider.GetRequiredService<ICache>();
var results = new string[10];
for (int i = 0; i < 10; i++)
for (var i = 0; i < 10; i++)
{
results[i] = cache.ComputeHash($"test-data-{i}");
}
Expand All @@ -285,7 +285,7 @@ public string[] Direct_CPUBound_Repeated_100Calls()
using var scope = _directCacheProvider.CreateScope();
var cache = scope.ServiceProvider.GetRequiredService<ICache>();
var results = new string[100];
for (int i = 0; i < 100; i++)
for (var i = 0; i < 100; i++)
{
results[i] = cache.ComputeHash($"test-data-{i}");
}
Expand All @@ -298,7 +298,7 @@ public string[] Proxied_CPUBound_Repeated_100Calls()
using var scope = _proxiedCacheProvider.CreateScope();
var cache = scope.ServiceProvider.GetRequiredService<ICache>();
var results = new string[100];
for (int i = 0; i < 100; i++)
for (var i = 0; i < 100; i++)
{
results[i] = cache.ComputeHash($"test-data-{i}");
}
Expand All @@ -311,7 +311,7 @@ public string[] Proxied_CPUBound_Repeated_100Calls()
using var scope = _directDatabaseProvider.CreateScope();
var db = scope.ServiceProvider.GetRequiredService<IDatabase>();
var results = new Customer?[10];
for (int i = 0; i < 10; i++)
for (var i = 0; i < 10; i++)
{
results[i] = await db.GetCustomerByIdAsync(1);
}
Expand All @@ -324,7 +324,7 @@ public string[] Proxied_CPUBound_Repeated_100Calls()
using var scope = _proxiedDatabaseProvider.CreateScope();
var db = scope.ServiceProvider.GetRequiredService<IDatabase>();
var results = new Customer?[10];
for (int i = 0; i < 10; i++)
for (var i = 0; i < 10; i++)
{
results[i] = await db.GetCustomerByIdAsync(1);
}
Expand All @@ -337,7 +337,7 @@ public string[] Proxied_CPUBound_Repeated_100Calls()
using var scope = _directDatabaseProvider.CreateScope();
var db = scope.ServiceProvider.GetRequiredService<IDatabase>();
var results = new Customer?[100];
for (int i = 0; i < 100; i++)
for (var i = 0; i < 100; i++)
{
results[i] = await db.GetCustomerByIdAsync(1);
}
Expand All @@ -350,7 +350,7 @@ public string[] Proxied_CPUBound_Repeated_100Calls()
using var scope = _proxiedDatabaseProvider.CreateScope();
var db = scope.ServiceProvider.GetRequiredService<IDatabase>();
var results = new Customer?[100];
for (int i = 0; i < 100; i++)
for (var i = 0; i < 100; i++)
{
results[i] = await db.GetCustomerByIdAsync(1);
}
Expand Down
Loading
Loading