When autoplay is disabled in the browser, audio and video don't work in Talk and the user is not told why, or what to do about it

[wknapik]

The relationship between the autoplay permission and being able to hear audio, or see video in a call is not obvious.

If the user disables autoplay globally and joins a Talk call, they'll hear no audio and see no video. There is no explanation anywhere. An icon shows up in the address bar allowing the user to permit autoplay, but their attention is not otherwise drawn there, nor is the relationship explained.

IMO Talk should warn the user, similar to how it warns about the microphone being muted system-wide when joining. Google Meet even warns the user if their camera is enabled, but covered.

I assume it would be detectable that audio/video aren't working. In that case we could tell the user how to fix it. Even if there are multiple possible reasons and we can't distinguish the real cause, we could still tell them about autoplay.

Bonus points if we can narrow it down to autoplay and just present a dialog to enable it, instead of describing where to click.

Slack thread

[bcaller]

I tried to see how to detect autoplay being blocked with the following code:

window.setTimeout("var context = new AudioContext(); console.log(context.state); context.resume().then(function(){console.log(context.state)});", 100)

When run in Brave Talk, the state is running. When run in the context of the 8x8 iframe, its state remains suspended fails.

We are running the iframe with the autoplay permission delegated from talk.brave.com to the 8x8 iframe

A permissions policy allows developers to selectively enable and disable browser features and APIs. Once an origin has received autoplay permission, it can delegate that permission to cross-origin iframes with the permissions policy for autoplay. Note that autoplay is allowed by default on same-origin iframes.

So the autoplay permission can easily be detected by the jitsi iframe. Not sure yet why it's not as simple to retrieve it for the brave parent.

[bcaller]

I think if the 8x8 frame detected autoplay being blocked it could then either: require a click on the page to resume the context (each time Brave Talk is used) or prompt the user to allow the permission.

[bcaller]

We can either ask 8x8 to add a check or we can do the check from talk.brave.com as follows:

• Create an iframe to a cross-origin domain (i.e. not talk.brave.com), needs to be a domain allowed by CSP and a page which doesn't set annoying X-Frame headers.
• Add a postMessage listener to the iframe
• Iframe runs (new AudioContext()).state: if autoplay is blocked, the result is 'suspended' (otherwise it's 'running')
• Iframe needs to communicate this back to the talk.brave.com parent via postMessage
• If the iframe reports that the audio context started in the suspended state, Brave Talk needs to prompt the user to open the page permissions - there is no builtin permissions dialog for this, so we need to show instructions.

[goodov]

Looks like this is solvable with existing APIs available in browser, I don't see anything we should change browser-side.

[mrose17]

Need to discuss with the Jitsi folks.

[jeffemandel]

I am working on Software as a Medical Device that presents the user with an instruction both visually and a a text-to-speech prompt. In most hospitals, users SSO into a virtual machine, so there is no ability to set the user preference to permit autoplay, and no history to drive Media Engagement. So does the user enable audio every time so he can comply with the two factor identification, or ignore it and just go with the visual info? Good guess. This is another example of legitimate use of auditory info that is being blocked to address the need to avoid rickrolls.
It seems to me that any app attempting to do something important will have the user authenticate through something like OAUTH; perhaps adding a claim to the JWT could be a secure way to opt in to features such as this. I know, easy to make the suggestion, hard to implement.

[mattmcalister]

thanks @jeffemandel . it's an interesting suggestion.