-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Description
This issue is particularly problematic in Kubernetes environments where downstream services and Consul are deployed in different locations, but affects any environment with separated Consul and service deployments.
Describe the bug
After upgrading Ocelot from version 23.2.x to 23.3.x or newer (including 24.x), the gateway incorrectly routes traffic to the Consul node address instead of the actual service instance address registered in Consul. This issue is particularly problematic in Kubernetes environments where downstream services and Consul are deployed in different locations.
Steps to reproduce
Steps to reproduce the behavior:
- Set up Ocelot 23.3+ with Consul service discovery in a Kubernetes environment
- Deploy Consul and downstream services in different locations/pods
- Register the microservices with Consul
- Configure Ocelot to use Consul for service discovery
- Make a request through Ocelot to a downstream service
- Observe that Ocelot attempts to connect to the Consul node address instead of the service instance address
Note: This issue does NOT occur in Ocelot 23.2.x and earlier versions.
Expected behavior
Ocelot should route requests to the service instance address provided by Consul (entry.Service.Address
) rather than using the Consul node address (node.Address
).
Actual behavior
Ocelot routes requests to the Consul node address (node.Address
) instead of the service instance address (entry.Service.Address
), causing connection failures when the Consul nodes are not reachable from the gateway or when the service is deployed on different nodes than Consul.
Environment
- Ocelot Version: 23.3.0 and later
- Platform: Kubernetes (any distribution)
- Consul Version: Any version
- .NET Version: Not relevant (affects all .NET versions)
Root Cause Analysis
The regression was introduced in Ocelot 23.3 in the DefaultConsulServiceBuilder.GetDownstreamHost
method. The current implementation incorrectly prioritizes the Consul node address over the service instance address:
// Current broken implementation (Ocelot 23.3+)
protected virtual string GetDownstreamHost(ServiceEntry entry, Node node)
=> node != null ? node.Name : entry.Service.Address;
Proposed Fix
The logic should be reversed to prioritize the service instance address, with node address as fallback only when service address is unavailable:
// Fixed implementation
protected virtual string GetDownstreamHost(ServiceEntry entry, Node node)
=> !string.IsNullOrEmpty(entry?.Service?.Address)
? entry.Service.Address
: node?.Address ?? node?.Name;
This change:
- Restores previous behavior from Ocelot 23.2.x and earlier
- Maintains backward compatibility for environments where service address might be empty
- Fixes Kubernetes deployments where services and Consul run in separate pods
- Preserves fallback behavior for edge cases where service address is unavailable by using node address as backup
Additional context
This issue is particularly critical in modern microservices architectures where:
- Services are containerized and dynamically scheduled across nodes
- Consul runs as a separate cluster or stateful set
- Network segmentation exists between service discovery and application layers
- Service mesh architectures rely on accurate endpoint discovery