multiline: Specify behavior with respect to formatting.

[progval]

This is the only way to format the fallback for multiline-unaware clients the same way as for multiline-aware clients

[slingamn]

lgtm

[jwheare]

I'm not sure. This puts a significant burden on clients when interpreting user input and composing a multiline message.

[progval]

So currently, if someone sends this:

BATCH +123 draft/multiline #channel
@batch=123 PRIVMSG #channel :not-bold <0x02>bold
@batch=123 PRIVMSG #channel :unclear
BATCH -123

then irccloud and the current weechat master will display it like this, because they don't implement this PR:

not-bold bold
unclear

and assuming servers don't do any reformatting, then every client not supporting multiline will get this:

PRIVMSG #channel :not-bold <0x02>bold
PRIVMSG #channel :unclear

so they will display it like this:

not-bold bold
unclear

I understand this can be annoying to implement, but we need to either:

1. accept this solution
2. propose a different solution
3. decide it's not a problem

Could we resolve this before weechat ships version 4.0 with draft/multiline support?

[progval]

Update: weechat released v4.0 without changing its behavior with respect to formatting.

[progval]

@flashcode As implementer of the multiline spec, could you weigh in on this question?

[JustAnotherArchivist]

This has been stalled for some time.

I agree that this is the only sensible way of doing it. Any other approach has no sane fallback mechanism.

There is however one complication that hasn't been addressed: the current draft says says this about fallback clients:

When delivering multiline batches to clients that have not negotiated the multiline capability, [...]

Servers SHOULD maintain the line composition sent by the client instead of combining to a normalised form before re-splitting. This ensures that steps taken to split long lines appropriately are preserved.

If the servers don't maintain the lines as sent, this would again lead to different formatting between multiline and non-multiline clients. The only sensible approach to me here seems to be to turn that recommendation into a requirement, i.e. change the 'SHOULD' into a 'MUST'.

[JustAnotherArchivist]

Actually, the above would apply not only to fallback clients but also to multiline-capable ones; their representation would also change if the server didn't maintain what the author client sent.

[progval]

And that's fine. Servers have always been allowed to tamper with the text and color codes if they want to.

[JustAnotherArchivist]

I suppose so. Would a recommendation to insert formatting reset codes during such rearrangements (if needed) be reasonable?

[jwheare]

Have any client developers made an attempt at implementing this suggestion and have any thoughts?

[skizzerz]

Alternative server-side implementation (not fully fleshed-out wording wise, just an idea at the moment). I'm interested to hear what ircd devs think of this feasibility-wise, etc.

For each line in a multiline batch, keep track if there's an unterminated format code. When the server processes a new line in the batch, it will synthesize and inject a multline-concat line to clients supporting multiline which terminates those format codes or CTCP. Clients that do not support multiline do not receive any additional fallback behavior, as they are already processing each line independently. (side-note: instead of closing CTCP, it may be better to just reject unclosed CTCP entirely, I'm not sure client-side if the \x01 at the end will cause some clients to process it differently based on whether they support multiline and thus get the extra synthesized line or not).

Example:

Client to server:
BATCH +123 draft/multiline #channel
@batch=123 PRIVMSG #channel :this is not bold, <0x02>but this is,<SPACE>
@batch=123;draft/multiline-concat PRIVMSG #channel :<0x02>this is still bold,<SPACE>
@batch=123;draft/multiline-concat PRIVMSG #channel :but this is not
BATCH -123

Server to client with draft/multiline CAP enabled:
:client!user@host BATCH +123 draft/multiline #channel
@batch=123 :client!user@host PRIVMSG #channel :this is not bold, <0x02>but this is,<SPACE>
@batch=123;draft/multiline-concat :client!user@host PRIVMSG #channel :<0x0F>
@batch=123;draft/multiline-concat :client!user@host PRIVMSG #channel :<0x02>this is still bold,<SPACE>
@batch=123;draft/multiline-concat :client!user@host PRIVMSG #channel :<0x0F>
@batch=123;draft/multiline-concat :client!user@host PRIVMSG #channel :but this is not
BATCH -123

Server to client without draft/multiline CAP:
:client!user@host PRIVMSG #channel :this is not bold, <0x02>but this is,<SPACE>
:client!user@host PRIVMSG #channel :<0x02>this is still bold,<SPACE>
:client!user@host PRIVMSG #channel :but this is not

Repeat example but replace <0x02> with <0x01>ACTION and <0x0F> with <0x01> for CTCP. As a note, these injected lines could cause the batch being sent to clients to exceed the stated max-lines or max-bytes limits.

[progval]

What makes it easier for servers to insert <0x0F> than for receiving clients who implement draft/multiline?

[skizzerz]

What makes it easier for servers to insert <0x0F> than for receiving clients who implement draft/multiline?

imo it's not easier to do this server-side, doing it client-side would be easier because there'd be no concerns about impacting send queue and the like (plus a lot of extra overhead on the wire to add one character). However if this is stalled because clients don't want to go through the effort or we're afraid of client fragmentation with some following spec and others not doing the format reset, then there are a lot fewer servers in active development than clients, so having consistency would involve fewer people to do said implementation.

[progval]

However if this is stalled because clients don't want to go through the effort or we're afraid of client fragmentation with some following spec and others not doing the format reset

Inserting <0x0F> does not fix this issue. If IRCCloud and Weechat don't change their current behavior, then it's still broken, because when they send messages like this:

BATCH +123 draft/multiline #channel
@batch=123 PRIVMSG #channel :this is not bold, <0x02>but this is,<SPACE>
@batch=123;draft/multiline-concat PRIVMSG #channel :<0x02>this is not in bold,<SPACE>
@batch=123;draft/multiline-concat PRIVMSG #channel :and this also is not
BATCH -123

and the server inserts <0x0F> then it would make the message into this:

:client!user@host BATCH +123 draft/multiline #channel
@batch=123 :client!user@host PRIVMSG #channel :this is not bold, <0x02>but this is,<SPACE>
@batch=123;draft/multiline-concat :client!user@host PRIVMSG #channel :<0x0F>
@batch=123;draft/multiline-concat :client!user@host PRIVMSG #channel :<0x02>this is not in bold,<SPACE>
@batch=123;draft/multiline-concat :client!user@host PRIVMSG #channel :<0x0F>
@batch=123;draft/multiline-concat :client!user@host PRIVMSG #channel :and this also is not
BATCH -123

to clients who support batch/multiline, which is wrong because they would bold the second line

and this:

:client!user@host PRIVMSG #channel :this is not bold, <0x02>but this is,<SPACE>
:client!user@host PRIVMSG #channel :<0x02>this is still bold,<SPACE>
:client!user@host PRIVMSG #channel :but this is not

to clients who don't support batch/multiline, which is also wrong for the same reason.

[Steering7253]

There are literally and objectively only 3 options here: either the server adds formatting codes to subsequent lines for fallback clients, or the non-fallback clients treat formatting as ending at the end of each line, or the behavior is different for fallback and non-fallback clients.

If the desire is to have the server handle making sure the formatting is applied to subsequent lines for fallback clients - which I will note is not the one and only objection noted in this thread - then the server will have to wrap lines which become too long as a result of that formatting, which means multiline-concat should have just been clients opting into longer lines in the first place, and then servers could have wrapped lines being sent to fallback clients. (As has been pointed out before, of course, yet multiline has consensus...)

[Celelibi]

Here's a hot take: we need less server-side message tampering, not more.
It hasn't been mentioned yet in this PR, but any server-side tampering wouldn't play well with E2EE in the future if we want such feature. This formatting issue can easily be dealt with in the client. The client can also, at the same time, make sure the message still meets the max-bytes and max-lines limits after adding the additional formatting codes. Doing it on the server would mean rejecting multiline messages that the sender would see as perfectly valid just because the added formatting bytes makes it go over the limit.