Skip to content

Releases: envoyproxy/ai-gateway

v0.1.0

25 Feb 16:46
@github-actions github-actions
v0.1.0
fb26cbe
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.1.0

We're excited to announce the first official release of Envoy AI Gateway v0.1.0! This release marks a significant milestone in our journey to provide a secure, scalable, and efficient way to manage GenAI/LLM traffic using Envoy Proxy.

Overview

Envoy AI Gateway is an open-source project part of the Envoy Ecosystem. It is built on top of Envoy Gateway and Envoy Proxy, designed to simplify how application clients interact with Generative AI services.

It provides a unified layer for routing and managing LLM/AI traffic with built-in security, token based rate limiting, and policy control features.

For installation instructions, see our Getting Started Guide.

Features

Unified API for LLM Providers

  • Standardized Interface: Expose a unified API (currently OpenAI-compatible) to clients while routing to different AI service backends
  • Request/Response Transformation: Automatically transform requests and responses between different provider formats
  • Provider Integration: Initial support for OpenAI and AWS Bedrock

Upstream Authorization

  • API Key Management: Securely manage API keys for upstream AI providers
  • AWS Credentials Support: Built-in support for AWS request signing for Bedrock services
  • OIDC Token Exchange: Support for OIDC-based authentication flows

Token-Based Rate Limiting

  • Token Usage Tracking: Track and limit usage based on tokens rather than just request count
  • Flexible Rate Limit Policies: Configure limits based on input tokens, output tokens, or total tokens
  • Per-User Limiting: Apply rate limits per-client and/or user

Traffic Management

  • Intelligent Routing: Route requests to appropriate AI backends based on model name and other criteria
  • Header-Based Routing: Configure routing rules using HTTP headers and paths
  • Backend Selection: Flexible backend selection with support for weighted traffic distribution

Kubernetes Native

  • Custom Resources: Define your AI Gateway configuration using Kubernetes Custom Resources
  • Helm Installation: Easy deployment using Helm charts
  • Integration with Gateway API: Built on the Kubernetes Gateway API specification

Note that Envoy AI Gateway is built on top of Envoy Gateway and you benefit from all the features of Envoy Gateway and Envoy Proxy. See the Envoy Gateway documentation for more information.

Custom Resources

This release introduces three new Custom Resource Definitions (CRDs):

  1. AIGatewayRoute: Defines the unified API schema and routing rules to AI service backends
  2. AIServiceBackend: Specifies the AI service backend schema and connection details
  3. BackendSecurityPolicy: Configures authentication for upstream AI services

Documentation

Comprehensive documentation is available at aigateway.envoyproxy.io, including:

Quick installation

The helm chart is attached as an asset named ai-gateway-helm-v0.1.0.tgz here. Alternatively, it is also published at Docker Hub, so you can try it out with

helm install aieg oci://registry-1.docker.io/envoyproxy/ai-gateway-helm --version v0.1.0 --namespace envoy-ai-gateway-system --create-namespace

Community

Envoy AI Gateway is a community-driven project.
We welcome your contributions and feedback!

What's Next

We're already working on exciting features for upcoming releases:

  • Google Gemini integration
  • Provider and model fallback logic

Acknowledgements

This release wouldn't be possible without the incredible contributions from our community members across Tetrate, Bloomberg, WSO2, RedHat, Google, and our independent contributors.

Thank you for your dedication and support!

Loading

v0.1.0-rc2

25 Feb 16:07
@github-actions github-actions
v0.1.0-rc2
da18080
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.1.0-rc2 Pre-release
Pre-release

This is a release candidate for v0.1.0. The helm chart is attached as an asset named ai-gateway-helm-v0.1.0-rc2.tgz here. Alternatively, it is also published at Docker Hub, so you can try it out with

helm install aieg oci://registry-1.docker.io/envoyproxy/ai-gateway-helm --version v0.1.0-rc2 --namespace envoy-ai-gateway-system --create-namespace
Loading

v0.1.0-rc1

20 Feb 19:25
@github-actions github-actions
v0.1.0-rc1
176a952
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.1.0-rc1 Pre-release
Pre-release

This is a release candidate for v0.1.0. The helm chart is attached as an asset named ai-gateway-helm-v0.1.0-rc1.tgz here. Alternatively, it is also published at GHCR, so you can try it out with

helm install aieg oci://ghcr.io/envoyproxy/ai-gateway/ai-gateway-helm --version v0.1.0-rc1 --namespace envoy-ai-gateway-system --create-namespace
Loading