Skip to content

RFC: Generic Filters for Subscriptions and FETCH #1164

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
+59 −0
Open

RFC: Generic Filters for Subscriptions and FETCH #1164

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
36cedc3
RFC: Generic Filters for Subscriptions and FETCH
afrind Aug 20, 2025
beb7072
Add priority to the filter.
afrind Aug 20, 2025
455b640
Apply suggestions from code review
afrind Aug 21, 2025
5705af4
wrap and newline
afrind Aug 21, 2025
bd66412
More cleanup
afrind Aug 21, 2025
d5c2340
Apply suggestions from code review
afrind Sep 18, 2025
19e15cd
Update draft-ietf-moq-transport.md
afrind Sep 18, 2025
65a7e24
Update draft-ietf-moq-transport.md
afrind Sep 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions draft-ietf-moq-transport.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1633,6 +1633,58 @@ multi-object stream will expire earlier than Objects later in the stream. Once
Objects have expired from cache, their state becomes unknown, and a relay that
handles a downstream request that includes those Objects re-requests them.

#### OBJECT FILTER Parameter

The OBJECT_FILTER parameter(Parameter Type 0x05) MAY appear in SUBSCRIBE,
SUBSCRIBE_UPDATE, PUBLISH_OK, TRACK_STATUS or FETCH message. It is a structure
Copy link
Collaborator

@suhasHere suhasHere Aug 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about adding this to SubscribeNamespace as well ?

Copy link
Collaborator Author

@afrind afrind Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's unclear what adding a filter to SUBSCRIBE_NAMESPACE would do without #1047? Or you have different use cases in mind? Let's keep this focused on subscriptions.

Copy link
Collaborator

@suhasHere suhasHere Aug 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adding it to namespace will enable object filtering across the tracks, where the base case is a single subscription . I was suggesting it would be nice to think the filter more generically

Copy link
Collaborator Author

@afrind afrind Aug 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@suhasHere I am amenable to using the same parameter to filter Tracks in response to SUBSCRIBE_NAMESPACE rather than Objects in SUBSCRIBE, but it requires more semantics (for example, most operands defined here don't make any sense). Can you file an issue explaining the requirements for Track filtering first?

indicating Objects or ranges of Objects that the Publisher will deliver.

~~~
Object Filter {
Type (0x5),
Length (i),
Operand and Flag (i),
Copy link
Collaborator

@suhasHere suhasHere Aug 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Operand and Flag (i),
Operand and Operator (i),

Copy link
Collaborator

@suhasHere suhasHere Aug 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where Operator can be one !-, =, >, < . I think this will cover a good set of use-cases.

Copy link
Collaborator Author

@afrind afrind Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of those can be express via range filters + negation:

=  -> Equal
!= -> Negate + Equal
>  -> Range with no end
<  -> Negate + Range with no end

It's also unclear what < > mean with multiple ranges.

Copy link
Collaborator Author

@afrind afrind Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized < X is also trivially implemented as a range from [0,X]

Values (i) ...
}
~~~

Operand and Flag contains both an enumeration indicating the Operand, with the
least significant bit indicating if the filter is negated.

~~~
Negated = Operand and Flag & 0x1
Operand = Operand and Flag >> 1
~~~

Operand | Meaning
0x0 | No Filter
0x1 | Group ID
0x2 | Subgroup ID
0x3 | Object ID
0x4 | Extension ID

Values is an array of integers which encode the values of interest. When
Operand is Extension ID, the first Value indicates the Extension ID to filter.
This value MUST indicate an Extension with an integer type. The remainder of
the array is a sequence of pairs indicating the start and length of the matching
range. The Start is encoded as a delta from the previous End, or from 0 for the
first element. The length indicates the number of elements including Start to
match. A length of 0 indicates no endpoint. For example [ 10, 3, 1, 1, 20, 0 ]
would match values 10, 11, 12, 14 and 34-max. Only the first (excluding
Extension ID) or last value can be 0. If an endpoint receives a 0 value
anywhere else, it MUST treat the track as Malformed (see {{malformed-tracks}}).

All filters can be removed from a Subscription by sending SUBSCRIBE_UPDATE with
Operand No Filter.

This parameter MAY appear more than once, and filters are cumulative. When
filters are applied, delivery rules regarding Subgroups and FETCH responses are
Copy link
Collaborator

@fluffy fluffy Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't want to deal with an unlimited number of filters operations. How about a max of 4 unless someone can come up with a very good use case for more. Keep in mind I would like to be able to implement this on something like a FPGA or silicon for wire speed filtering.

Copy link
Collaborator Author

@afrind afrind Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A max seems very sensible.

modified. Within a Subgroup, the Publisher is allowed to send an Object on a
Subgroup stream even when it is not the next Object if the expected Objects
did not pass the filter. In a FETCH response, the Subscriber can only infer
Copy link
Collaborator

@fluffy fluffy Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be reading what you are saying here wrong but ...

If you mean you can send stuff that does not pass the filter, then no, I don't think this should be allowed. I think the stream should be ended if this happens on reliable stream and the things should just be filtered if on datagram.

If you mean that if the filter remove object N but object N+1 is the subgroup passes the filter, you can send N+1 even though N is missing ... then I need to think about what this does to caches downstream.

Copy link
Collaborator Author

@afrind afrind Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not saying you should send stuff that doesn't pass the filter. I was thinking the latter. For example f you have a subgroup with objects 0-9, and a filter that says only even objects, then you can put [ 0, 2, 4, 5, 8 ] in a single stream, even though you know you are omitting objects.

That said, I think maybe that is a really bad idea, and we should just use the normal logic and the publisher has to reset the stream and open a new one wherever it skips.

Objects do not exist from their absence if they pass all filters.


## CLIENT_SETUP and SERVER_SETUP {#message-setup}

The `CLIENT_SETUP` and `SERVER_SETUP` messages are the first messages exchanged
Expand Down