Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HLS live streams just stops no errors #6251

Closed
dbanbahji opened this issue Feb 16, 2024 · 26 comments
Closed

HLS live streams just stops no errors #6251

dbanbahji opened this issue Feb 16, 2024 · 26 comments
Labels
component: HLS The issue involves Apple's HLS manifest format priority: P1 Big impact or workaround impractical; resolve before feature release status: archived Archived and locked; will not be updated type: bug Something isn't working correctly
Milestone

Comments

@dbanbahji
Copy link

dbanbahji commented Feb 16, 2024

Have you read the FAQ and checked for duplicate open issues?
yes

If the problem is related to FairPlay, have you read the tutorial?

not related to fairplay
What version of Shaka Player are you using?

demo page

Can you reproduce the issue with our latest release version?
yes
Can you reproduce the issue with the latest code from main?
yes

Are you using the demo app or your own custom app?
demo

If custom app, can you reproduce the issue using our demo app?

yes
What browser and OS are you using?

chrome latest
For embedded devices (smart TVs, etc.), what model and firmware version are you using?
n/a

What are the manifest and license server URIs?

https://install.perfectgame.tv/speed/broadcast/15/desktop-playlist.m3u8

What configuration are you using? What is the output of player.getConfiguration()?
demo page default

What did you do?
play stream downloads 1-2 segments then stops downloading anymore, manifest continue to download and be updated.
ts stream with no encrtyption.

What did you expect to happen?

playback to continue

@dbanbahji dbanbahji added the type: bug Something isn't working correctly label Feb 16, 2024
@dbanbahji
Copy link
Author

dbanbahji commented Feb 16, 2024

seems to work fine on hlsjs videojs bitmovin etc and safari native. same behavior on chromecast sdk with just shaka. works with the MPL player.

@shaka-bot shaka-bot added this to the v5.0 milestone Feb 16, 2024
@dbanbahji
Copy link
Author

@avelad i checked 4.5.0 and i dont see any issues with the stream freezing.
is it possible transmuxers changed?
but i do see some of these after a discontinuty. seems like that might be seperate issue. is that worth another issue?
https://v4-5-0-dot-shaka-player-demo.appspot.com/demo/#audiolang=en-US;textlang=en-US;uilang=en-US;asset=https://install.perfectgame.tv/speed/broadcast/15/desktop-playlist.m3u8;panel=CUSTOM%20CONTENT;build=uncompiled

@joeyparrish
Copy link
Member

If the transmuxer were to blame, I think we would still see network requests for segments. It stops requesting segments, so I think that is a bug in HlsParser, SegmentIndex, or StreamingEngine.

@joeyparrish joeyparrish added component: HLS The issue involves Apple's HLS manifest format priority: P1 Big impact or workaround impractical; resolve before feature release labels Feb 27, 2024
@joeyparrish
Copy link
Member

joeyparrish commented Feb 27, 2024

I just confirmed this issue with the latest release, v4.7.11.

No configuration required. Quick link to repro:

https://shaka-player-demo.appspot.com/demo/#asset=https://install.perfectgame.tv/speed/broadcast/15/desktop-playlist.m3u8

@joeyparrish
Copy link
Member

Update: while I was writing those last messages, network requests started happening again. It's still not playing, but my previous guesses about the root cause may be invalid.

@joeyparrish
Copy link
Member

In v4.7.0, v4.6.0, v4.5.0, and v4.4.0 it is broken, but with different symptoms:

  • "Possible encoding problem warnings"
  • Repeated segments in buffer
  • Jumping to catch up to seek range

It appears to be working in v4.3.0. I'll bisect through that range to see what changed. Likely the introduction of our transmuxer. :-)

@dbanbahji
Copy link
Author

ive seen the possible encoding problems errors with media tailor streams as well for ad insertions on hls.
that link and this stream doesnt have any ad insertion just a disconintuty and cue markers to put in ads if they went through an ad server. so you might get 2 bugs here for the price of one :-)

@joeyparrish
Copy link
Member

According to my bisection, your stream works fine before 50c9df4, PR #4870, which was meant to fix issue #4589. Between that and v4.7.0, we see the encoding problem logs I mentioned above. There is some other point at which we seem to transition to the "just stops" symptom.

@dbanbahji
Copy link
Author

if you need more content with dynamic ad breaks, discontinuities, and mediatailor streams encoded/packaged by ateme, let me know. i still see those possible possible encoding problems, repeated segements/audio/frames/freezing, Jumping forward to catch up with the seek range with 4.7.11.

i sometimes see also segment_index.js:522 Assertion failed: SegmentReferences are incorrect and log.js:140 Possible encoding problem detected! Unexpected buffered range for reference shaka.media.SegmentReference and a lot of gap jumping but i dont see gaps  in some of these streams/media. most of the time these dont cause too much impact to the stream sync wise or error or playback wise outside of filling up the console.

@joeyparrish
Copy link
Member

All these things are signs of something that is very wrong. They don't always lead to a failed playback, but they point to broken assumptions and/or bad content. (It's hard to tell which is which from the symptoms alone.)

I confirmed your bisection that the "just stops" symptoms start at 776b69d, PR #6059, which is meant to fix issue #5939.

Since reverting that just leads to different problems ("possible encoding problem", repeated segments, etc.), I don't think that is a viable solution.

@dbanbahji
Copy link
Author

dbanbahji commented Feb 27, 2024

i dont think its bad content, ive been testing this on mpl default sdk for the new shaka hls opt in, along with safari native, bitmovin, hlsjs,theo,videojs plays fine on all those. probably just a small bug :-)

@avelad avelad added component: transmuxer The issue involves our built-in transmuxer and removed component: transmuxer The issue involves our built-in transmuxer labels Feb 28, 2024
@avelad
Copy link
Member

avelad commented Feb 28, 2024

Yes, the issue is at 776b69d, and it looks like the same root cause as #6299

@avelad avelad self-assigned this Feb 28, 2024
@dbanbahji
Copy link
Author

yeah that might fix the stopping part, but do you want another issue for the discontinuity ads related errors?

@avelad avelad removed their assignment Mar 25, 2024
@dbanbahji
Copy link
Author

@avelad i do see nightly fixes the just stops behavior. i still see those possible possible encoding problems, repeated segements/audio/frames/freezing,Jumping forward to catch up with the seek range, console errors etc. should i open a new ticket for those?

@avelad avelad modified the milestones: v4.8, v4.9 Apr 26, 2024
@avelad avelad modified the milestones: v4.9, v4.10 May 30, 2024
@avelad avelad modified the milestones: v4.10, v4.11 Jul 1, 2024
@avelad
Copy link
Member

avelad commented Jul 3, 2024

@dbanbahji I've been looking to see what happens with your stream and I think it is poorly generated:

Start of stream (player load)

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:2188154
#EXT-X-DISCONTINUITY-SEQUENCE:28392
#EXT-X-TARGETDURATION:9
#EXT-X-PROGRAM-DATE-TIME:2024-07-03T13:40:35.882500Z
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01011.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01012.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01013.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01014.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01015.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01016.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01017.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01018.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01019.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01020.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01021.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01022.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01023.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01024.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01025.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01026.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01027.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01028.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01029.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01030.ts

1º update

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:2188155
#EXT-X-DISCONTINUITY-SEQUENCE:28392
#EXT-X-TARGETDURATION:9
#EXT-X-PROGRAM-DATE-TIME:2024-07-03T13:40:43.991300Z
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01012.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01013.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01014.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01015.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01016.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01017.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01018.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01019.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01020.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01021.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01022.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01023.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01024.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01025.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01026.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01027.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01028.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01029.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01030.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01031.ts

2º update

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:2188156
#EXT-X-DISCONTINUITY-SEQUENCE:28392
#EXT-X-TARGETDURATION:9
#EXT-X-PROGRAM-DATE-TIME:2024-07-03T13:40:52.098800Z
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01013.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01014.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01015.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01016.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01017.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01018.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01019.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01020.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01021.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01022.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01023.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01024.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01025.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01026.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01027.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01028.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01029.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01030.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01031.ts
#EXTINF:8,
https://pg-15-linear.perfectgame.tv/009000-393466-pg-321_1ev9/file-5928k-01032.ts

The PDTs are:

  • #EXT-X-PROGRAM-DATE-TIME:2024-07-03T13:40:35.882500Z
  • #EXT-X-PROGRAM-DATE-TIME:2024-07-03T13:40:43.991300Z
  • #EXT-X-PROGRAM-DATE-TIME:2024-07-03T13:40:52.098800Z

If the duration of the segments is 8, the time increment should always be 8, and what your stream says is not like that

The expected would be:

  • #EXT-X-PROGRAM-DATE-TIME:2024-07-03T13:40:35.882500Z
  • #EXT-X-PROGRAM-DATE-TIME:2024-07-03T13:40:43.882500Z
  • #EXT-X-PROGRAM-DATE-TIME:2024-07-03T13:40:51.882500Z

You can ignore the PDT by configuration with manifest.hls.ignoreManifestProgramDateTime = true

@avelad avelad added the status: waiting on response Waiting on a response from the reporter(s) of the issue label Jul 3, 2024
@shaka-bot
Copy link
Collaborator

Closing due to inactivity. If this is still an issue for you or if you have further questions, the OP can ask shaka-bot to reopen it by including @shaka-bot reopen in a comment.

@shaka-bot shaka-bot removed the status: waiting on response Waiting on a response from the reporter(s) of the issue label Jul 10, 2024
@dbanbahji
Copy link
Author

dbanbahji commented Jul 17, 2024

@avelad
i was wondering if you could help with some general question.
whats the difference between sequence mode and segements mode? which one should be used/default?
i also see Ignore Manifest Timestamps in Segments Mode and Ignore Program Date Time from manifest options for each of these modes. which is the best combinations of these 3 options for general HLS .ts AES 128 streams like this?
i also see Inaccurate Manifest Tolerance is that related here as well?
@shaka-bot reopen

@dbanbahji
Copy link
Author

@shaka-bot reopen

@avelad
Copy link
Member

avelad commented Jul 17, 2024

Segments mode or sequence mode is how MSE works, in general sequence mode requires less computational power (it is more efficient), but is less tolerant to errors.

Each stream may need something different depending on how it is generated, if it is generated well the default configuration should work correctly.

@shaka-bot shaka-bot reopened this Jul 17, 2024
@dbanbahji
Copy link
Author

whats the difference between sequence mode and segements mode?
if i just want shaka to play the segements/ts to the best of its abiltiy without caring too much about timings/pts/dta/manifest timestamps i assume sequence mode is probably better.
if i have sequence mode which seems to be the default does it make sense to add Ignore Manifest Timestamps in Segments Mode and Ignore Program Date Time from manifest.
what would be the best for inaccurate manifest tolerance?

@avelad
Copy link
Member

avelad commented Jul 18, 2024

It seems that this problem could be mitigated by configuring manifest.hls.sequenceMode to false and manifest.hls. ignoreManifestTimestampsInSegmentsMode to true.

@avelad avelad added the status: waiting on response Waiting on a response from the reporter(s) of the issue label Jul 18, 2024
@brodiddev
Copy link

@avelad Can you explain in detail why setting hls sequenceMode to false in the above issue alleviates the playback issue?

@avelad
Copy link
Member

avelad commented Jul 23, 2024

Because of the way the timestampOffset is calculated to pass it to MSE, we tolerate more errors this way.

@shaka-bot shaka-bot removed the status: waiting on response Waiting on a response from the reporter(s) of the issue label Jul 23, 2024
@brodiddev
Copy link

brodiddev commented Jul 23, 2024

  1. Can you give me a link to the relevant code?
    There is an issue below where hls sync is not correct when buffering is turned off.
    When buffering is turned off, the sync goes out. #7087

By setting sequenceMode to false, the error below that occurs when out of sync does not occur.
2. Is it related to the above option?

The sync problem has been resolved, but the error below occurs repeatedly and the buffer is filled and depleted repeatedly.
shaka-player.compiled.debug.js:852 Assertion failed: SegmentIndex.merge should not change the first reference time!

@avelad
Copy link
Member

avelad commented Jul 24, 2024

We will follow up on this last error at #7087

@avelad avelad added the status: waiting on response Waiting on a response from the reporter(s) of the issue label Jul 24, 2024
@shaka-bot
Copy link
Collaborator

Closing due to inactivity. If this is still an issue for you or if you have further questions, the OP can ask shaka-bot to reopen it by including @shaka-bot reopen in a comment.

@shaka-bot shaka-bot removed the status: waiting on response Waiting on a response from the reporter(s) of the issue label Jul 31, 2024
@shaka-bot shaka-bot added the status: archived Archived and locked; will not be updated label Sep 29, 2024
@shaka-project shaka-project locked as resolved and limited conversation to collaborators Sep 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
component: HLS The issue involves Apple's HLS manifest format priority: P1 Big impact or workaround impractical; resolve before feature release status: archived Archived and locked; will not be updated type: bug Something isn't working correctly
Projects
None yet
Development

No branches or pull requests

5 participants