Sorted collections

[marcstern]

For some collections (ARGS, REQUEST_COOKIES), it may be necessary to have them sorted in order to concatenate them in an intelligent way (i.e. concatenated all ARGS named "a" without others in the middle). Think about HTTP parameter pollution...
SORTED_MATCHED_VARS should be sufficient to handle sorting of any collection with an intermediate rule.
The sorting should be alphabetically (case-insensitive for the name) but keep the entries with the same name in the original order.

[zimmerle]

@marcstern can you give a concrete example where such sort will be needed?

[marcstern]

/req?p=cmd&p=/c&p=...
The previous line leads to "p=cmd,/c,..." is ASP (and others) which can be used to perform an attack (in this concrete example, the comma is a valid separator in Windows commands).
We currently can detect this by concatenated all values and adding a comma between these.

However, the following evasion rendres this impossible:
/req?p=cmd&a=xyz***&p=/c&p=...

By having a sorted collection, we can perform the same concatenation to check for patterns.

Note that having the ability to create custom collections would be much more powerful and can be used to created sorted collections if needed. So I'd vote for the latter.

[martinhsv]

Based on the example, it doesn't seem like this proposal actually has anything to do with sorting per se. I.e. there is no processing difference between varying orders of unlike-named arguments.

Your example seems to only be about aggregating like-named arguments.

(For anyone following this who is unfamiliar with the concatentation being referred to, some backends (chiefly IIS) will concatenate like-named parameters. See for more detail (particularly the chart): https://www.acunetix.com/blog/whitepaper-http-parameter-pollution/ )

Or am I missing something?

Assuming I'm correct, yes ModSecurity could implement code to either:

• create a new collection (maybe CONCATENATED_ARGS?) to concatenate like-named arguments
• (if a configuration flag is set) add the concatenated value to the normal collection in addition to the single ones. In other words var1=a&var1=b would result in 3 key-value pairs
  var1 -> a
  var1 -> b
  var1 -> a,b

However before implementing new functionality, we should note that:

• one can already write a rule to check for parameter pollution irrespective of the concatenation issue (multiple like-named parameters may be valid in some requests -- those could be excluded specifically)
• there are already other ways to handle the concatenation issue specifically.
  • write a script
  • use a regular rule to do this (CRS at least used to have something like that. See for example: https://github.com/coreruleset/coreruleset/blob/v2.2/master/experimental_rules/modsecurity_crs_40_http_parameter_pollution.conf -- although the discussion in issue-340 does outline where this becomes challenging

[marcstern]

As I explained, if you try (like in the CRS rules) to do it on unsorted ARGS, you can evade it. This is why we need (also) a sorted collection.
But, as I said, being able to create our own collection would be much better (and allows a lot of other use cases).