Aggregates + RouteKeysConfig where the array of values does not work correctly

[ArtyomZhuravlyov]

Expected Behavior / New Feature

My json

{
  "Routes": [
    {
      "DownstreamPathTemplate": "/comments",
      "DownstreamHostAndPorts": [
        {
          "Host": "localhost",
          "Port": 5002
        }
      ],
      "UpstreamPathTemplate": "/comments",
      "UpstreamHttpMethod": [
        "GET"
      ],
      "Key": "comments"
    },
    {
      "DownstreamPathTemplate": "/users/{id}",
      "DownstreamHostAndPorts": [
        {
          "Host": "localhost",
          "Port": 5001
        }
      ],
      "UpstreamPathTemplate": "/users/{id}",
      "UpstreamHttpMethod": [
        "GET"
      ],
      "Key": "user"
    }
  ],
  "Aggregates": [
    {
      "UpstreamPathTemplate": "/aggregatecommentuser",
      "RouteKeys": [
        "comments",
        "user"
      ],
      "RouteKeysConfig": [
        {
          "RouteKey": "user",
          "JsonPath": "$[*].userId",
          "Parameter": "id"
        }
      ]
    }
  ],
  "GlobalConfiguration": {
    "BaseUrl": "https://localhost:5003"
  }
}

I'm accessing the aggregating url: http://localhost:5003/aggregateСommentUser

The first request occurs for http://localhost:5002/comments
If my first query in the aggregation returns a response

"comments": [
        {
            "id": 1,
            "userId": 1
        },
        {
            "id": 2,
            "userId": 2
        }
    ],

Then when I request information about users, I should get a response.

      "user": [
        {
            "id": 1,
            "name": "Artem"
        },
        {
            "id": 2,
            "name": "Ivan"
        }
    ]

And the requests themselves should look like this:
1. http://localhost:5002/comments
2. http://localhost:5001/users/1
3. http://localhost:5001/users/2

Actual Behavior / Motivation for New Feature

I'm accessing the aggregating url: http://localhost:5003/aggregateСommentUser

The first request occurs for http://localhost:5002/comments
my first query in the aggregation returns a response

 "comments": [
        {
            "id": 1,
            "userId": 1
        },
        {
            "id": 2,
            "userId": 2
        }
    ],

I receive a response from the user service

      "user": [
        {
            "id": 1,
            "name": "Artem"
        },
        {
            "id": 1,
            "name": "Artem"
        }
    ]

Because my requests from Ocelot look like this:
1. http://localhost:5002/comments
2. http://localhost:5001/users/1
3. http://localhost:5001/users/1

Link to the repository: https://github.com/ArtyomZhuravlyov/OcelotAggregate

Specifications

• Version: 23.4.2
• Platform: .NET 8
• Subsystem:

[raman-m]

Hello, Artyom!
Your Users service returns the following response

Yes, you have 2 requests because your downstream service returned 2 user items with the same ID which is 1. Thus, Ocelot makes 2 requests with the same URL which is /user/1.
I am struggling to understand you: why do you report this issue?
Please fix your downstream service and enjoy Ocelot Aggregation feature!
Anything else?

[raman-m]

Artyom,
Please be aware that Complex Aggregation is a pilot feature that may contain bugs, and it could be improved. Additionally, this feature is not well-documented: it was released in version 23.1.0 with draft section for complex aggregating. Therefore, consider it as "under development", although the basic functionality should work.

If you need consultation, our teammate will assist you. However, it would be better if you develop your own aggregator first, without a Complex Aggregation logic.

[ArtyomZhuravlyov]

No, no, you don't understand me.

I have the following specified in json

"RouteKeysConfig": [
     {
       "RouteKey": "user",
       "JsonPath": "$[*].userId",
       "Parameter": "id"
     }
   ]

This means that this request must be executed at the beginning.
http://localhost:5002/comments

The response always comes from it.

[
        {
            "id": 1,
            "userId": 1
        },
        {
            "id": 2,
            "userId": 2
        }
    ]

This is what the controller looks like

[ApiController]
[Route("[controller]")]
public class CommentsController : ControllerBase
{
    [HttpGet]
    public Comment[] GetAll()
    {
        return new Comment[] { new Comment() { Id = 1, UserId = 1 }, new Comment() { Id = 2, UserId = 2 } };
    }
}

So Ocelot should

1. take User Id = 1 and create request http://localhost:5001/users/1
2. take UserId = 2 and create request http://localhost:5001/users/2

Then I will get a response from the first request.

 {
            "id": 1,
            "name": "Artem"
        }

and the response from the second request

```
 {
        "id": 2,
        "name": "Ivan"
    }

[ApiController]
[Route("[controller]")]
public class UsersController : ControllerBase
{
[HttpGet("{id:int}")]
public User? GetById(int id)
{

    var users = new User[] { new User() { Id = 1, Name = "Artem" }, new User() { Id = 2, Name = "Ivan" } };
    return users.SingleOrDefault(x => x.Id == id);
}

}

But Ocelot ignores UserId = 2
And makes two identical requests:
http://localhost:5001/users/1
http://localhost:5001/users/1

        

[raman-m]

🆗 Fair enough!
I have examined the controller in the solution. It could be a potential bug.

Could you please debug the ProcessRouteWithComplexAggregation method to locate issue?
We need to replicate this bug in debug mode to comprehend the problem.

[ArtyomZhuravlyov]

I have debugged, attached screenshots of control values where value(userId) = 1, value(userId) = 2

It is noticeable that HttpContext stores the value value in TemplatePlaceholderNameAndValues at each iteration.
After the first request, the HttpContext stores {[{{userId}}=1]}.
And when the second request is executed, HttpContext already stores {[{{userId}}=1]} and {[{{userId}}=2]} and tPnv, respectively, also stores two values {[{{userId}}=1]} and {[{{userId}}=2]} with a single userId key.

The problem is that tPnv is a reference from HttpContext.items, and when you add Placeholder to tPnv, Placeholder is added to HttpContext, and each time the number of parameters with one userId key increases, although only the last value should be recorded.

Most likely, only the first value (1) with one userId key per url is always sent. http://localhost:5001/users /{id}

Simply addition of ToList() removes the problem because we are already working with another reference object.
var tPnv = httpContext.Items.TemplatePlaceholderNameAndValues().ToList();

After .ToList();