Extend filter functionality - Allow filtering by bitmask

[santilland]

In order to be able to extract data more efficiently through the interface we need new types of filtering that are not currently possible through the range filtering.

Allowing filtering by a bitmask.
This is important especially to be able to filter for valid data in L1B mie_bin_quality_flag, rayleigh_bin_quality_flag

A possibility could be to pass an bitmask parameter in filter object instead of min max. In order to allow to filter only specific bits the bitmask object could look like the following:

{
  "bitmask": [[0,1],[2,0]]
}

meaning to filter by bit 0 being 1 (true) and bit 2 being 0 (false)

[pacesm]

Any reason not to use a compact form with just two integer values?

The <mask> select the bits tested against the <expected value>, which leads to the following simple predicate

<mask> & <tested value> == <mask> & <expected value>

For your example, <mask> = 0b0101 = 5 and <expected value> = 0b0001 = 1

5 & <tested value> == 1

[santilland]

That is also a good idea and good approach, i was just thinking how this would be for the user.
So for example the overall validity in the aeolus data is only in the first bit, so a user will probably only want to filter by the first bit. (They also change the most significant bit around a bit which does not make things easier).
Different flags have different lengths in aeolus e.g. 8 bit flags, 12 bit flags (not sure which lengths right now), so a user would need to know how long the flag is and set the complete two integer values accordingly.
So my idea was the user (or us creating the request) would only need to know what is the position one wants to filter by and by which value without the need to know really anything else about how the flags are defined.

[pacesm]

I guess we should make distinction between the UI and the server API.

For the server public API (i.e., WPS XML requests), IMO, we should stick to a format easily handled by the machine (i.e., two integers directly used in the applied predicate). The server API is not the UI and nobody expects users to write they XML requests manually.

There can be many different ways how to present these parameters in the UI (check boxes, list of tuples, binary number, ...etc.) but this can be contained on the client side and the server does not need to know about them.