Camera not working in PWA after app returns from background in iOS #298

tristansb1 · 2022-04-28T09:21:11Z

Describe the bug
The camera stops working after the PWA returns form background on my 12 mini. The camera also doesn't work at all on an iPhone 13 (see videos below). App works perfectly fine in Safari. PWA also works fine on Android devices.

To Reproduce
Implement this Demo as PWA: https://gruhn.github.io/vue-qrcode-reader/demos/DecodeAll.html#decode-continuously

<template>
  <div class="scanner">
    <p class="error">{{ error }}</p>

    <p class="decode-result">Last result: <b>{{ error }}</b></p>

    <qrcode-stream @decode="onDecode" @init="onInit" />
  </div>
</template>

<script>
import { QrcodeStream } from 'vue-qrcode-reader'
export default {

  components: { QrcodeStream },

  data () {
    return {
      result: '',
      error: ''
    }
  },

  methods: {
    onDecode (result) {
      this.result = result
    },

    async onInit (promise) {
      try {
        await promise
      } catch (error) {
        if (error.name === 'NotAllowedError') {
          this.error = "ERROR: you need to grant camera access permission"
        } else if (error.name === 'NotFoundError') {
          this.error = "ERROR: no camera on this device"
        } else if (error.name === 'NotSupportedError') {
          this.error = "ERROR: secure context required (HTTPS, localhost)"
        } else if (error.name === 'NotReadableError') {
          this.error = "ERROR: is the camera already in use?"
        } else if (error.name === 'OverconstrainedError') {
          this.error = "ERROR: installed cameras are not suitable"
        } else if (error.name === 'StreamApiNotSupportedError') {
          this.error = "ERROR: Stream API is not supported in this browser"
        } else if (error.name === 'InsecureContextError') {
          this.error = 'ERROR: Camera access is only permitted in secure context. Use HTTPS or localhost rather than HTTP.';
        } else {
          this.error = `ERROR: Camera error (${error.name})`;
        }
      }
    }
  }
}
</script>

<style scoped>
.error {
  font-weight: bold;
  color: red;
}
.scanner {
  position: absolute;
  width: 100%;
  height: 100%;
  overflow: hidden;
  left: 0;
  top: 0;
}
</style>

Screenshots
iPhone 12 mini behaviour

ios.mp4

iPhone 13 behaviour

PWA-QR-2.mp4

Smartphone:

Devices: iPhone 12 mini, iPhone 13
OS: iOS 15.4.1
Browser: PWA

The text was updated successfully, but these errors were encountered:

kikan · 2022-05-18T07:00:13Z

It seems there's a dependency to https://cdn.jsdelivr.net/npm/[email protected]/dist/jsQR.min.js
So, when the device if offline, the dependency can't be reached and the QRCode recognition doesn't work.
I think you need to copy jsQR.min.js somewhere and manage to have it fetched instead of the cdn.

0tsu0i0ihajime · 2022-08-29T15:55:32Z

I am using PWA in the same way, but when I use it in the background on iOS, I can't re-use the camera and have no choice but to close it and start over again.
I also used jsQR.min.js but that didn't change it. Why is this?

I have this I am incorporating this into the html

SebastianMueller87 · 2022-08-29T16:04:26Z

It seems there's a dependency to https://cdn.jsdelivr.net/npm/[email protected]/dist/jsQR.min.js So, when the device if offline, the dependency can't be reached and the QRCode recognition doesn't work. I think you need to copy jsQR.min.js somewhere and manage to have it fetched instead of the cdn.

I doubt that it is the reason. The devices are online the whole time. It just occurs when moving the app into background and back forward.
It also behaves differently on different iOS devices (with same iOS-Version), so i would exclude the internet connection and therefore also the cdn as the reason for this.

@0tsu0i0ihajime the error is still not resolved and also the reason has not be found yet.

0tsu0i0ihajime · 2022-08-30T07:16:26Z

If I do this on a non-cellular model iPad in an environment with Wi-Fi turned off, the camera will stop and not come back on when I move it to the background and then undo it when using PWA.
I was wondering, is it possible that Safari is designed to automatically stop the camera when offline?

mrrrk · 2022-10-04T14:00:11Z

On an iPhone 12 I was testing on, the PWA app would get in a state where the video element of the reader would refuse to start at all after going into the background and coming back. My eventual workaround had two parts:

In mounted() of the component showing the qr-reader, find the video element of the qr-reader (using getElementsByTagName or whatever), subscribe to the ended event of the video - which seems to fire when the badness happens. Use $router.back() to close the component whenever this fires (this may or may not be appropriate depending on your implementation of the reader!)

Additionally, in the beforeDestroy hook (Vue 2) of the component with the reader, find the video element and do this:

videoElement.pause();
videoElement.src = "";
videoElement.removeAttribute("src"); // can you smell the desperation?
videoElement.load(); // from https://stackoverflow.com/questions/3258587/how-to-properly-unload-destroy-a-video-element
videoElement?.parentNode?.removeChild(this.videoElement);

I was starting to wonder about brushing off my Swift skills, such as they are, and going native at about this point (which presumably is exactly what Apple wants me to do!)

Jannis033 · 2023-03-28T21:46:26Z

I have the same problem, using iOS 16.4 on an iPhone 13. After reopening the app from the background, the camera remains black. The red camera indicator appears in the top left corner and then immediately disappears again. navigator.mediaDevices.getUserMedia() returns successfully…
Even closing the app in the app switcher does not work. Neither does closing all apps, including Safari and all other PWAs (maybe leaving an instance of safari running leads to the issue). Sometimes it fixes itself after a couple minutes.
Restarting my iPhone always fixes the issue, but that’s not how it should be…
Btw. iPadOS 16.4 also has the issue.
Did you come to a solution?

github-actions · 2023-07-12T21:27:34Z

This issue has been marked as stale. If there is no further activity it will be closed.

mattwhitchurch · 2023-07-17T13:13:39Z

Posting to keep the issue open - same issue on iPhone 14. Same issue exactly as Jannis033.

Apparently, when QrcodeStream is used in an iOS PWA and goes into the background, the stream on the video element is stopped. When the user returns, the stream is not restarted automatically. To fix this we try keep track whether the page/app is in the foreground and stop/start the stream when this state changes. Incidentally, this commit makes the demo page is now PWA installable to make this testable. Closes #298

gruhn · 2023-07-17T18:52:54Z

I deployed a PWA installable version of the demo page (without fix) but I can't reproduce the issue on iOS 16.

https://deploy-preview-351--vue-qrcode-reader.netlify.app/

Can someone confirm whether this is reproducible on the demo page.

mattwhitchurch · 2023-07-18T09:12:17Z

Just tried on iPhone 14 with iOS 16.5.1 and worked initially. However when I closed the app and reopened it, it did not work - as per the issue.

For information it worked fine on Pixel 6 / Chrome.

Apparently, when QrcodeStream is used in an iOS PWA and goes into the background, the stream on the video element is stopped. When the user returns, the stream is not restarted automatically. To fix this we try keep track whether the page/app is in the foreground and stop/start the stream when this state changes. Incidentally, this commit makes the demo page PWA installable to make this fix testable. Closes #298

gruhn · 2023-07-18T10:00:02Z

Ah I see, I can reproduce it too now. You really have to close the app. I thought putting it into background already has this effect.

Also thanks @mrrrk. The hints are very helpful. Sorry for the long wait.

Looks like the issue corresponds to this Webktit bug. In the thread they are saying a fix has been merged literally last week. But I guess that will only benefit (far?) future iOS versions.

On iOS devices in PWA mode, QrcodeStream works initially, but after killing and restarting the PWA, all video elements fail to display camera streams. Looks like this is related to this WebKit issue: https://bugs.webkit.org/show_bug.cgi?id=252465 No workarounds at the time of writing. But as suggested in the thread we can put a timeout on the video elements to start loading. That way we can at least detect when the error occurs and throw an error event. Also this commit: 1. Makes the demo page PWA installable to reproduce this problem. 2. Renames the "DecodeAll" demo to "HandleErrors". The demo already is focused on error handling because it renders the error message instead of just printing to the console, which is particularly useful on mobile devices. Renaming the demo emphasizes that and hints user to use this demo for debugging. See #298

gruhn · 2023-07-18T17:34:37Z

Ok, so until someone finds a workaround, I implemented what they suggested in the WebKit issue thread: throw an error when the problem occurs, so at least the application can respond to it. You should be able to do something like this in v5.1.0:

<qrcode-stream @error="onError" />

const onError = event => {
  if (event.name === "StreamLoadTimeoutError") {
    // handle error
  }
}

Also, check out the corresponding demo.

holgerschillack · 2023-07-27T09:22:13Z

So I added the onError method to show a toast, that the camera initialisation was not successful.

But is there a workaround to restart the camera initialisation or reset it somehow?

I already tried a combination of different approaches, but nothing seems to work. If the user closes the PWA once (with swiping it up from the background app panel), the App is broken and needs to be re-installed.

This is my current try of a workaround:

onError() {
    this.$toast.error('Error opening camera!');
    this.antiBugMethod();
    this.reloadCameraFeed();
    this.requestCameraAccess();
    this.reloadCameraFeed();
    setTimeout(() => {}, 2500);
},
async requestCameraAccess() {
    await navigator.mediaDevices.getUserMedia({ video: true });
},
antiBugMethod() {
    const videoElement = document.querySelector('video.qrcode-stream-camera');
    if (videoElement) {
        videoElement.pause();
        videoElement.src = '';
        videoElement.removeAttribute('src');
        videoElement.load();
        videoElement.parentNode?.removeChild(videoElement);
    }
},
reloadCameraFeed() {
    this.paused = true;
    setTimeout(() => {
        this.paused = false;
        console.log('Restarted camera feed.');
    }, 200);
}

gruhn · 2023-07-28T06:43:17Z

I also tried a lot of things but nothing worked unfortunately. In the webkit issue thread I linked above, also nobody had a workaround.

Jannis033 · 2023-08-09T07:40:00Z

In IOS 17 Beta Public Beta 2 the issue seems to be resolved. I have tried it multiple times with several PWAs and could not reproduce the issue. In IOS 17 Beta 1 the issue was still present for me.
The only thing I noticed was that the camera image is small for some milliseconds and then gets the full size. Only on first load of the PWA. This is since Beta 1...
// edit: I just got the issue again but I stressed the PWA with closing and reopening very often.

Robcodeandcopy · 2023-08-31T09:29:05Z

I also have this issue, so do we need to wait for IOS to update to IOS 17 before this is fixed?

Jannis033 · 2023-08-31T10:19:47Z

I also have this issue, so do we need to wait for IOS to update to IOS 17 before this is fixed?

Yes, I doubt so. But iOS 17 will be released in a few weeks already and if you cannot wait, there are some pretty stable public betas available.

brianoflondon · 2023-09-13T11:18:35Z

Thank you for being friendly with the error messages 😀 (once I made sure to add the code to look at them properly. I'll wait for iOS 17 to hopefully fix.

geri777 · 2023-10-27T13:23:25Z

Since today the scanner exits with the StreamLoadTimeoutError everytime - also when newly started and not activating from the background. I have not changed anything - neither on the web nor on my phone. Same issue on 3 client's phones.
The PWA askes for camera permission every time.

I then updated from 16.4.1 to 16.4.2 - when I started the PWA again for the first time on the new system, the camera access worked. But this was the only time it worked - later tests did not work any more.
When switching off/on the device it works for one single time again.

I also found out that the PWA camera access works when the camera app has just been put to background.

geri777 · 2023-10-27T17:24:29Z

I have tested my PWA on iOS 17.2 and everything works. I hope it will be live soon.

github-actions · 2023-12-27T00:13:57Z

This issue has been marked as stale. If there is no further activity it will be closed.

github-actions · 2024-02-26T00:14:00Z

This issue has been marked as stale. If there is no further activity it will be closed.

denishaskin · 2024-07-03T22:29:44Z

Interesting -- at first I was about to report that I'm still running into this issue, and my iPhone 12 is on 17.5.1.

At first, the Vue app was fine running in Safari on the phone, but the installed PWA would get the StreamLoadTimeoutError every time. I did try a couple of the workarounds here (camera app backgrounded, etc) but so far nothing has worked.

But then I powered the phone off and on, and now it seems to work consistently, no matter what. So far. 🤞

(And the PWA works on the iPad (also running 17.5.1) just fine no matter what. I do get prompted whether to allow the camera each time I launch the app, but that we can live with I think. And interesting I'm not ever getting prompted for camera access on the iPhone)

Asko-Dev · 2024-07-15T18:14:17Z

ios 17.5.1 ... the error StreamLoadTimeoutError gets thrown randomly when switching between cameras - I don't have a big issue with it as it happens very little, but when it happens I have to refresh the web page to get it running again. (I have a web app in vue3) - is there a way to just do a hard-reset when the error gets thrown? unmounting the component doesn't help either, bit confused as how to handle it inside the app without having to force refresh the entire page. Thank you.

gruhn · 2024-07-15T19:25:35Z

ios 17.5.1 ... the error StreamLoadTimeoutError gets thrown randomly

Hmm, if it gets thrown "randomly" maybe it's a race condition with the timeout. I'll increase the timeout a bit and see if that helps.

Asko-Dev · 2024-07-15T20:19:04Z

ios 17.5.1 ... the error StreamLoadTimeoutError gets thrown randomly

Hmm, if it gets thrown "randomly" maybe it's a race condition with the timeout. I'll increase the timeout a bit and see if that helps.

Thank you! Yes it's very "random", I tried to break it on purpose after it appeared the first time ... but suddenly it was fine and I could switch 50 times, then I would refresh and it would throw the error after 3 switches. When it happened I tried to refresh again and follow the exact pattern of switching, suddenly it would work flawlessly. No specific order has proven to be prone to failure.. and I did give it hell in order to give you a better description 😅 but unfortunately, seems random.

@Asko-Dev

@Asko-Dev reports that the `SteamLoadTimeoutError` seems to be thrown "randomly" on iOS 17.5.1. That suggests that there is a race condition with the timeout. Let's try to double the timeout (3s -> 6s) and see if that helps. See: #298

@Asko-Dev

@Asko-Dev reports that the `SteamLoadTimeoutError` seems to be thrown "randomly" on iOS 17.5.1 ( see #298 ). That suggests that there is a race condition with the timeout. Let's try to double the timeout (3s -> 6s) and see if that helps. Side changes: * The ESLint setup is broken somehow and since linting is a pre-commit hook it blocks the committing process. Thus, removing the setup entirely for now. To be fixed later if deemed necessary. * migrate legacy Nix setup to Nix flakes / direnv

gruhn · 2024-07-15T20:42:08Z

@Asko-Dev thanks. I doubled the timeout (3s -> 6s). You should be able to upgrade to v5.5.7 now. Hope that helps.

Asko-Dev · 2024-07-16T10:39:39Z

@Asko-Dev thanks. I doubled the timeout (3s -> 6s). You should be able to upgrade to v5.5.7 now. Hope that helps.

Thank you lots, will report back if it gets thrown again.

meni33a · 2024-08-27T11:34:22Z

Same issue here, still happening in v5.5.7

Info:
PWA on IOS 17.5.1
(When using safari on IOS without WPA it works fine)

Errors:

[Debug] [vue-qrcode-reader] starting camera with constraints: – "{\"facingMode\":\"environment\"}" (index-1Q8lL2HW.js, line 611)

[Debug] [vue-qrcode-reader] calling getUserMedia (index-1Q8lL2HW.js, line 611)

[Error] A MediaStreamTrack ended due to a capture failure

[Debug] [vue-qrcode-reader] waiting for video element to load (index-1Q8lL2HW.js, line 611)

[Error] A MediaStreamTrack ended due to a capture failure

[Debug] [vue-qrcode-reader] starting camera failed with "StreamLoadTimeoutError: Loading camera stream timed out after 6 seconds. If you are on iOS in PWA mode, this is a known issue (see https://github.com/gruhn/vue-qrcode-reader/issues/298)" (index-1Q8lL2HW.js, line 611)

[Error] Unhandled Promise Rejection: AbortError: The operation was aborted.

gruhn · 2024-08-29T11:19:56Z

When using safari on IOS without WPA it works fine

That's still to be expected unfortunately. No known fix available.

CuberHuber · 2024-09-20T11:17:52Z

@tristansb1

Maybe you are using basic HTTP?
I've been having the same problem and then I saw this in the official doc.

Safari also requires HTTPS even on localhost (see [#48](https://github.com/gruhn/vue-qrcode-reader/issues/48)). Support is limited for:
web apps added to home screen (PWA mode): at least iOS 13.4 (see [#76](https://github.com/gruhn/vue-qrcode-reader/issues/76))
iOS browsers other than Safari (Chrome for iOS, Firefox for iOS, ...): at least iOS 14.3 (see [#29](https://github.com/gruhn/vue-qrcode-reader/issues/29))
WkWebView component in native iOS apps: at least iOS 14.3 (see [#29](https://github.com/gruhn/vue-qrcode-reader/issues/29))

I generated SSL cert and added it to the project (I use Nuxt: https://nuxt.com/docs/api/nuxt-config#https).

That solved my problem

furkanipek · 2024-09-22T15:50:51Z

I have this error too, but I don't get this error for a long time after restarting the device. I will run my tests again now.
Device: 15 Pro Max iOS 18

tristansb1 changed the title ~~Camera not working in PWA after app returns from Background in iOS~~ Camera not working in PWA after app returns from background in iOS Apr 28, 2022

futurepaul mentioned this issue May 26, 2023

scanner might stop working on ios MutinyWallet/mutiny-web#174

Open

github-actions bot added the no-issue-activity label Jul 12, 2023

gruhn pinned this issue Jul 17, 2023

gruhn removed the no-issue-activity label Jul 17, 2023

gruhn mentioned this issue Jul 17, 2023

feat(QrcodeStream): throw timeout error when camera won't load #351

Merged

gruhn added the help wanted label Jul 19, 2023

gruhn unpinned this issue Jul 20, 2023

github-actions bot added the no-issue-activity label Dec 27, 2023

gruhn removed the no-issue-activity label Dec 27, 2023

github-actions bot added the no-issue-activity label Feb 26, 2024

github-actions bot closed this as completed Mar 4, 2024

gruhn added dont-mark-stale and removed no-issue-activity labels Jul 15, 2024

gruhn reopened this Jul 15, 2024

gruhn mentioned this issue Jul 15, 2024

Increase steam load timeout #442

Merged

gruhn pinned this issue Jul 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Camera not working in PWA after app returns from background in iOS #298

Camera not working in PWA after app returns from background in iOS #298

tristansb1 commented Apr 28, 2022 •

edited

Loading

kikan commented May 18, 2022

0tsu0i0ihajime commented Aug 29, 2022

SebastianMueller87 commented Aug 29, 2022

0tsu0i0ihajime commented Aug 30, 2022

mrrrk commented Oct 4, 2022 •

edited

Loading

Jannis033 commented Mar 28, 2023

github-actions bot commented Jul 12, 2023

mattwhitchurch commented Jul 17, 2023

gruhn commented Jul 17, 2023

mattwhitchurch commented Jul 18, 2023

gruhn commented Jul 18, 2023 •

edited

Loading

gruhn commented Jul 18, 2023

holgerschillack commented Jul 27, 2023 •

edited

Loading

gruhn commented Jul 28, 2023

Jannis033 commented Aug 9, 2023 •

edited

Loading

Robcodeandcopy commented Aug 31, 2023

Jannis033 commented Aug 31, 2023

brianoflondon commented Sep 13, 2023 •

edited

Loading

geri777 commented Oct 27, 2023 •

edited

Loading

geri777 commented Oct 27, 2023

github-actions bot commented Dec 27, 2023

github-actions bot commented Feb 26, 2024

denishaskin commented Jul 3, 2024

Asko-Dev commented Jul 15, 2024 •

edited

Loading

gruhn commented Jul 15, 2024

Asko-Dev commented Jul 15, 2024

gruhn commented Jul 15, 2024

Asko-Dev commented Jul 16, 2024

meni33a commented Aug 27, 2024 •

edited

Loading

gruhn commented Aug 29, 2024 •

edited

Loading

CuberHuber commented Sep 20, 2024

furkanipek commented Sep 22, 2024

Camera not working in PWA after app returns from background in iOS #298

Camera not working in PWA after app returns from background in iOS #298

Comments

tristansb1 commented Apr 28, 2022 • edited Loading

kikan commented May 18, 2022

0tsu0i0ihajime commented Aug 29, 2022

SebastianMueller87 commented Aug 29, 2022

0tsu0i0ihajime commented Aug 30, 2022

mrrrk commented Oct 4, 2022 • edited Loading

Jannis033 commented Mar 28, 2023

github-actions bot commented Jul 12, 2023

mattwhitchurch commented Jul 17, 2023

gruhn commented Jul 17, 2023

mattwhitchurch commented Jul 18, 2023

gruhn commented Jul 18, 2023 • edited Loading

gruhn commented Jul 18, 2023

holgerschillack commented Jul 27, 2023 • edited Loading

gruhn commented Jul 28, 2023

Jannis033 commented Aug 9, 2023 • edited Loading

Robcodeandcopy commented Aug 31, 2023

Jannis033 commented Aug 31, 2023

brianoflondon commented Sep 13, 2023 • edited Loading

geri777 commented Oct 27, 2023 • edited Loading

geri777 commented Oct 27, 2023

github-actions bot commented Dec 27, 2023

github-actions bot commented Feb 26, 2024

denishaskin commented Jul 3, 2024

Asko-Dev commented Jul 15, 2024 • edited Loading

gruhn commented Jul 15, 2024

Asko-Dev commented Jul 15, 2024

gruhn commented Jul 15, 2024

Asko-Dev commented Jul 16, 2024

meni33a commented Aug 27, 2024 • edited Loading

gruhn commented Aug 29, 2024 • edited Loading

CuberHuber commented Sep 20, 2024

furkanipek commented Sep 22, 2024

tristansb1 commented Apr 28, 2022 •

edited

Loading

mrrrk commented Oct 4, 2022 •

edited

Loading

gruhn commented Jul 18, 2023 •

edited

Loading

holgerschillack commented Jul 27, 2023 •

edited

Loading

Jannis033 commented Aug 9, 2023 •

edited

Loading

brianoflondon commented Sep 13, 2023 •

edited

Loading

geri777 commented Oct 27, 2023 •

edited

Loading

Asko-Dev commented Jul 15, 2024 •

edited

Loading

meni33a commented Aug 27, 2024 •

edited

Loading

gruhn commented Aug 29, 2024 •

edited

Loading