New Varint Encoding #1016

afrind · 2025-05-08T22:54:43Z

Fixes: #548
Fixes: #549

Fixes: #548 Fixes: #549

LPardue · 2025-05-08T23:15:29Z

draft-ietf-moq-transport.md

+
+### Variable-Length Integers
+
+RFC9000 variable-length integers use a two-bit length prefix in the first byte


I understand why you put this text here but it seems superfluous to talk about something that is not suitable for MoQT. Consider simply stating MoQT's requirements or abilities and then state how the encoding supports thst on the wire.

draft-ietf-moq-transport.md

LPardue · 2025-05-08T23:30:58Z

For a blanket change, I think you'll need to sanity check all uses. For instance, in SUBSCRIBE_DONE the Stream Count field would allow expressing a larger stream count than the underlying transport can support . That's easily fixable by stating the logical limit of the field and what to do if you receive a bogus value, but it's an example of a new interop issue this change introduces.

draft-ietf-moq-transport.md

Since it's only peer uni streams, the max count is 2^60

draft-ietf-moq-transport.md

Co-authored-by: Suhas Nandakumar <[email protected]>

vasilvv · 2025-05-11T20:18:43Z

draft-ietf-moq-transport.md


+This document redfines the following RFC9000 syntax:
+
+x (i):


Can we change (i) to avoid confusion with QUIC?

I'm not sure it's a real issue to be confused with QUIC, since we define the encoding in this document, and have this comment pointing to the definition.

However, I'm open to suggestions. x (v) for varint? I like x (ï)

I am also fine with x(i) as it is redefinition. If not, x(V) seems fine to me

I posted somewhere already but I can't find it, so I want to repeat it here. I hate the idea of redefinition, it provides no value and only a source of eternal confusion. Please just pick a different identifier like mi or whatever.

In isolation you think you have no issues. When I want to e.g. draw a diagram that renders how a MoQ control message is sent/received on an HTTP/3 stream I'm going to want to draw something like

STREAM Frame { Type (i) = 0x08..0x0f, Stream ID (i), [Offset (i)], [Length (i)], GOAWAY Message { Type (i) = 0x10, Length (16), New Session URI Length (i), New Session URI (..), }

see the ambiguity?

+1 on trying to keep the syntax so it does not clash with quic

I guess I'm sold by @LPardue's example.

+1 ... how about we define x(V64) to be very explicit about the intent of the type ?

@LPardue @afrind @fluffy ??

vasilvv · 2025-05-11T20:28:24Z

draft-ietf-moq-transport.md

+|--------------|----------------|-------------|---------------|
+| 1110         | 8              | 60          | 0-2^60-1      |
+|--------------|----------------|-------------|---------------|
+| 1111XXXX     | 9              | 64          | 0-2^64-1      |


I have no idea what this actually means. What are the XXXX?

Do we expect to revisit this scheme later? If the goal is to make "the range of 1 byte values is as large as possible", it would make more sense to just allocate 0x00~0xf8 values for those, but then we basically don't have two-byte varints.

I have no idea what this actually means. What are the XXXX?

The 9 byte encoding uses 4 bits for length with 4 reserved bits.

Do we expect to revisit this scheme later?

No. If we make the switch let's go from QUIC to MoQ varints once and never look back. Now is the time to propose new schemes.

allocate 0x00~0xf8 values for those, but then we basically don't have two-byte varints.

To clarify, you if the first byte is > 0xf8 the length is value - 0xf8 + 1? Technically you have 2 byte varints but they only encode 7 values that can't be encoded in 1 byte. These things are all about tradeoffs and I'm agnostic if the group prefers 248 1-byte values, at the expense of more bytes for larger values. It might be bad for shortish payload lengths (eg: 300-1200 bytes, though datagrams don't have a length field).

I think the "1111XXXX" should be "11110000" and then we should have
"111110001 to 11111111" as RESERVED

@fluffy sounds good to me. Would you have it be an error to receive 11110001-11111111? I don't want to check those bits in my decoder.

I don't care if the receiver never checks them, I just care that senders set them all zero and that future extensions can negotiate using them.

That said, it really does not seem a big deal to check and it's going to be easier to debug if you get a great reason phrase explaining what is going on when they are not valid.

I pulled up short of making 11110001-11111111 an error and backtracked on reserved, just saying ignored/SHOULD. Since it's a 64 bit value, there's no use for these bits. My thinking was that I didn't want any way processing the first byte could fail - but then the coffee kicked in a bit more and I realized decoding can always fail on some subsequent byte (running out of input) so I don't know that this is a huge advantage.

I'm open to doing whatever here, but want a broader signal from the wg. We should also discuss if we like Victor's idea better.

Still want the change to 1111XXXX to 11110000 and leave the other values as reserved for future extensions.

ianswett

This LG, a few comments. I think reusing 'i' from QUIC as a varint could be very confusing.

draft-ietf-moq-transport.md

afrind · 2025-05-12T15:59:07Z

draft-ietf-moq-transport.md


+This document redfines the following RFC9000 syntax:
+
+x (i):


I'm not sure it's a real issue to be confused with QUIC, since we define the encoding in this document, and have this comment pointing to the definition.

However, I'm open to suggestions. x (v) for varint? I like x (ï)

afrind · 2025-05-12T16:02:58Z

draft-ietf-moq-transport.md

+|--------------|----------------|-------------|---------------|
+| 1110         | 8              | 60          | 0-2^60-1      |
+|--------------|----------------|-------------|---------------|
+| 1111XXXX     | 9              | 64          | 0-2^64-1      |


I have no idea what this actually means. What are the XXXX?

The 9 byte encoding uses 4 bits for length with 4 reserved bits.

Do we expect to revisit this scheme later?

No. If we make the switch let's go from QUIC to MoQ varints once and never look back. Now is the time to propose new schemes.

allocate 0x00~0xf8 values for those, but then we basically don't have two-byte varints.

To clarify, you if the first byte is > 0xf8 the length is value - 0xf8 + 1? Technically you have 2 byte varints but they only encode 7 values that can't be encoded in 1 byte. These things are all about tradeoffs and I'm agnostic if the group prefers 248 1-byte values, at the expense of more bytes for larger values. It might be bad for shortish payload lengths (eg: 300-1200 bytes, though datagrams don't have a length field).

draft-ietf-moq-transport.md

ianswett

Comment Deleted, Lucas convinced me that we should not use (i) *

draft-ietf-moq-transport.md

Co-authored-by: ianswett <[email protected]>

fluffy

Some comments worth considering but the actual core part that defined what we are doing here looks great.

fluffy · 2025-05-14T13:45:25Z

draft-ietf-moq-transport.md


+This document redfines the following RFC9000 syntax:
+
+x (i):


+1 on trying to keep the syntax so it does not clash with quic

fluffy · 2025-05-14T13:48:51Z

draft-ietf-moq-transport.md

+|--------------|----------------|-------------|---------------|
+| 1110         | 8              | 60          | 0-2^60-1      |
+|--------------|----------------|-------------|---------------|
+| 1111XXXX     | 9              | 64          | 0-2^64-1      |


I think the "1111XXXX" should be "11110000" and then we should have
"111110001 to 11111111" as RESERVED

fluffy · 2025-05-14T13:51:41Z

draft-ietf-moq-transport.md

 --- back

+# Sample Variable-Length Integer Encoding and Decoding {#sample-varint}
+


I'm against adding example code like this to the draft - it just makes the draft longer and tends to accumulate errors over time. It gets endless review comments on better ways to write it. People can figure out how to write this.

I'm mirroring RFC 9000 here, but I don't have a strong opinion.

I tend to think it's somewhat useful.

This is in an appendix, correct?

yes, this in an appendix

fluffy

I think this still needs some small changes but agree with big picture

fluffy · 2025-07-24T08:27:51Z

draft-ietf-moq-transport.md

+The WriteVarint function takes two parameters, a 64 bit integer and an
+output buffer with an append operation.
+
+~~~pseudocode


Im very against putting code like this in this draft. I'm fine with another draft about implementation advice but we need to keep this draft as short as possible.

draft-ietf-moq-transport.md

fluffy · 2025-07-24T08:37:26Z

draft-ietf-moq-transport.md

+|--------------|----------------|-------------|---------------|
+| 1110         | 8              | 60          | 0-2^60-1      |
+|--------------|----------------|-------------|---------------|
+| 1111XXXX     | 9              | 64          | 0-2^64-1      |


Still want the change to 1111XXXX to 11110000 and leave the other values as reserved for future extensions.

fluffy · 2025-09-09T21:47:59Z

any update on this one ?

afrind · 2025-09-09T21:58:46Z

any update on this one ?

We can't land this change until ALPN version negotiation (Always use ALPN for version negotiation #499) lands, otherwise this would break version negotiation. That in turn is blocked on Chrome support for WT-AvailableProtocols, which has just reached Chrome canary. I expect we can resolve this blocker by Montreal. Hopefully FireFox and Safari will quickly follow, but I don't think we should wait for them.
There are still some issues to discuss here. First, we've not had a discussion of Victor's alternate design mentioned above or a similar one from MT on the linked issue. Second, there's the other outstanding feedback here - what to do about the trailing 4 bits when using 64-bit varints, and whether we should include pseudocode. We can add time to resolve these in Toronto.

afrind · 2025-09-25T19:58:32Z

Discussed in Toronto:

No one felt strongly one way or the other about having pseudocode in the doc. I'm happy to remove it for now.
We should require the tailing bits of 1111XXXX to be 0000. This will allow future extensibility.
Not a clear signal about whether we should pursue an alternate encoding. There wasn't a specific counterproposal that was easy to digest, and we also lack production data about histograms of the values that matter.
At least one person felt we shouldn't be inventing a new encoding, but should use some existing encoding, although to our knowledge there is not an existing varint encoding with the tradeoffs and properties we want.

afrind · 2025-09-25T20:01:38Z

I believe the counterproposal is:

250 1 byte values, 256 2 byte values, 2^16 3 byte values, etc.

I think this is a worse tradeoff than this PR:

128 1 byte values, 2^14 2 byte values.

For, say, datagram Object IDs in a group where the group is 30s long and there are 50 objects/second, we will more values between 250 and 2^14 than between between 128 and 256.

draft-ietf-moq-transport.md

New Varint Encoding

8b5f6bc

Fixes: #548 Fixes: #549

afrind requested review from ianswett, suhasHere and vasilvv as code owners May 8, 2025 22:54

afrind added Wire Format Related to how messages are serialized and parsed Design Issues or PRs that change how MoQ works including the wire format. labels May 8, 2025

LPardue reviewed May 8, 2025

View reviewed changes

draft-ietf-moq-transport.md Outdated Show resolved Hide resolved

LPardue reviewed May 8, 2025

View reviewed changes

draft-ietf-moq-transport.md Show resolved Hide resolved

afrind commented May 9, 2025

View reviewed changes

afrind and others added 3 commits May 9, 2025 10:24

Apply suggestions from code review

2a2f16d

Update stream count maximum

ae23d21

Since it's only peer uni streams, the max count is 2^60

lint

eac1557

suhasHere reviewed May 11, 2025

View reviewed changes

draft-ietf-moq-transport.md Outdated Show resolved Hide resolved

Update draft-ietf-moq-transport.md

cb4af37

Co-authored-by: Suhas Nandakumar <[email protected]>

vasilvv reviewed May 11, 2025

View reviewed changes

ianswett reviewed May 11, 2025

View reviewed changes

draft-ietf-moq-transport.md Show resolved Hide resolved

draft-ietf-moq-transport.md Outdated Show resolved Hide resolved

afrind commented May 12, 2025

View reviewed changes

draft-ietf-moq-transport.md Outdated Show resolved Hide resolved

Apply suggestions from code review

4527fc9

ianswett approved these changes May 13, 2025

View reviewed changes

draft-ietf-moq-transport.md Outdated Show resolved Hide resolved

Update draft-ietf-moq-transport.md

654a77c

Co-authored-by: ianswett <[email protected]>

suhasHere approved these changes May 14, 2025

View reviewed changes

fluffy approved these changes May 14, 2025

View reviewed changes

Review feedback

a22a952

afrind added the Needs Discussion Tags for issues which need discussion, ideally at an interim or IETF meeting. label May 31, 2025

fluffy requested changes Jul 24, 2025

View reviewed changes

afrind mentioned this pull request Aug 5, 2025

More explicit handling for end-of-group bit #1118

Closed

afrind added 4 commits September 25, 2025 19:49

Merge branch 'main' into varint

7a1bd7a

More i -> vi64

7150fd2

delete pseudocode, reserve bits for future extension

b950764

no longer redefines i

211018d

afrind commented Sep 25, 2025

View reviewed changes

draft-ietf-moq-transport.md Outdated Show resolved Hide resolved

Update draft-ietf-moq-transport.md

086de2c

ianswett approved these changes Sep 30, 2025

View reviewed changes

draft-ietf-moq-transport.md Outdated Show resolved Hide resolved

Make ranges exact with no commas

ca87191

ianswett approved these changes Oct 1, 2025

View reviewed changes

afrind removed the Needs Discussion Tags for issues which need discussion, ideally at an interim or IETF meeting. label Oct 1, 2025

yekuiwang approved these changes Oct 2, 2025

View reviewed changes

afrind mentioned this pull request Nov 7, 2025

Key-Value-Pairs Delta encoding editorial tweaks. #1366

Merged

ianswett added the Merge Ready Has the requisite number of approvals; all comments addressed. label Nov 8, 2025

afrind removed the Merge Ready Has the requisite number of approvals; all comments addressed. label Nov 13, 2025


		### Variable-Length Integers

		RFC9000 variable-length integers use a two-bit length prefix in the first byte

		--- back

		# Sample Variable-Length Integer Encoding and Decoding {#sample-varint}

New Varint Encoding #1016

Are you sure you want to change the base?

New Varint Encoding #1016

Uh oh!

Conversation

afrind commented May 8, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

LPardue commented May 8, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ianswett left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ianswett left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

fluffy left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ianswett left a comment •

edited

Loading