Audio stuttering when transcode playing ogg vorbis on iOS

[felix920506]

Description
When turning on the "Transcode" option in settings and playing Ogg Vorbis audio, the audio stutters.
When transcoding from other source formats this doesn't happen

Media Information

last
Container: ogg
Path: /Users/user/Music/test_music_lib/Arcaea All Songs/last.ogg
Size: 4 MB

Audio
Title: Last - VORBIS - Stereo
Codec: VORBIS
AVC: No
Layout: stereo
Channels: 2 ch
Bitrate: 192 kbps
Sample rate: 44100 Hz
Video range: Unknown
Video range type: Unknown
Default: No
Forced: No
External: No

Image
Codec: MJPEG
AVC: No
Profile: Baseline
Resolution: 768x768
Bit depth: 8 bit
Video range: Unknown
Video range type: Unknown
Color space: bt470bg
Pixel format: yuvj444p
Ref frames: 1

[Chaphasilor]

Hi, could you take a look at the transcoding logs of your Jellyfin server? I know that transcoding to Vorbis on iOS is problematic, and maybe the server isn't transcoding at all...

I noticed the bitrate of the file is 192 kbps, what do you have your transcoding bitrate set to? There was an issue (#269) where lower-bitrate tracks weren't being transcoded, but that should be fixed already...

[felix920506]

@Chaphasilor I'm not transcoding to Vorbis. I'm transcoding from Vorbis as Vorbis won't direct play.

I have transcoding bitrate set to 320kbps.

Here is a sample ffmpeg log

{"Protocol":0,"Id":"feb56d9b05dab1faabb53c433f74169a","Path":"/Users/user/Music/test_music_lib/Arcaea All Songs/last.ogg","EncoderPath":null,"EncoderProtocol":null,"Type":0,"Container":"ogg","Size":3944557,"Name":"last","IsRemote":false,"ETag":"509a90d84c357b021e3cfd83ae1efa37","RunTimeTicks":1474285710,"ReadAtNativeFramerate":false,"IgnoreDts":false,"IgnoreIndex":false,"GenPtsInput":false,"SupportsTranscoding":true,"SupportsDirectStream":true,"SupportsDirectPlay":true,"IsInfiniteStream":false,"RequiresOpening":false,"OpenToken":null,"RequiresClosing":false,"LiveStreamId":null,"BufferMs":null,"RequiresLooping":false,"SupportsProbing":true,"VideoType":null,"IsoType":null,"Video3DFormat":null,"MediaStreams":[{"Codec":"vorbis","CodecTag":null,"Language":null,"ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Comment":null,"TimeBase":"1/44100","CodecTimeBase":null,"Title":"Last","VideoRange":0,"VideoRangeType":0,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":null,"LocalizedDefault":null,"LocalizedForced":null,"LocalizedExternal":null,"LocalizedHearingImpaired":null,"DisplayTitle":"Last - VORBIS - Stereo","NalLengthSize":null,"IsInterlaced":false,"IsAVC":false,"ChannelLayout":"stereo","BitRate":192000,"BitDepth":null,"RefFrames":null,"PacketLength":null,"Channels":2,"SampleRate":44100,"IsDefault":false,"IsForced":false,"IsHearingImpaired":false,"Height":null,"Width":null,"AverageFrameRate":null,"RealFrameRate":null,"Profile":null,"Type":0,"AspectRatio":null,"Index":0,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":null,"Level":0,"IsAnamorphic":null},{"Codec":"mjpeg","CodecTag":null,"Language":null,"ColorRange":null,"ColorSpace":"bt470bg","ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Comment":"Cover (front)","TimeBase":"1/90000","CodecTimeBase":null,"Title":null,"VideoRange":0,"VideoRangeType":0,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":null,"LocalizedDefault":null,"LocalizedForced":null,"LocalizedExternal":null,"LocalizedHearingImpaired":null,"DisplayTitle":null,"NalLengthSize":null,"IsInterlaced":false,"IsAVC":false,"ChannelLayout":null,"BitRate":null,"BitDepth":8,"RefFrames":1,"PacketLength":null,"Channels":null,"SampleRate":null,"IsDefault":false,"IsForced":false,"IsHearingImpaired":false,"Height":768,"Width":768,"AverageFrameRate":null,"RealFrameRate":90000,"Profile":"Baseline","Type":3,"AspectRatio":"1:1","Index":1,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":"yuvj444p","Level":-99,"IsAnamorphic":false}],"MediaAttachments":[],"Formats":[],"Bitrate":214045,"Timestamp":null,"RequiredHttpHeaders":{},"TranscodingUrl":null,"TranscodingSubProtocol":null,"TranscodingContainer":null,"AnalyzeDurationMs":null,"DefaultAudioStreamIndex":null,"DefaultSubtitleStreamIndex":null}

ffmpeg -analyzeduration 200M -probesize 1G  -i file:"/Users/user/Music/test_music_lib/Arcaea All Songs/last.ogg" -map_metadata -1 -map_chapters -1 -threads 0   -vn -acodec aac_at -ab 256000 -ac 2 -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type mpegts -start_number 0 -hls_segment_filename "/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c%d.ts" -hls_playlist_type vod -hls_list_size 0 -y "/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c.m3u8"


ffmpeg version 6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
  built with Apple clang version 15.0.0 (clang-1500.1.0.2.5)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/6.1.1_3 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags='-Wl,-ld_classic' --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libharfbuzz --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopenvino --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
[ogg @ 0x14b606110] 4602 bytes of comment header remain
Input #0, ogg, from 'file:/Users/user/Music/test_music_lib/Arcaea All Songs/last.ogg':
  Duration: 00:02:27.43, start: 0.000000, bitrate: 214 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 192 kb/s
    Metadata:
      ENCODER         : REAPER
      TITLE           : Last
      ARTIST          : onoken
  Stream #0:1: Video: mjpeg (Baseline), yuvj444p(pc, bt470bg/unknown/unknown), 768x768 [SAR 1:1 DAR 1:1], 90k tbr, 90k tbn (attached pic)
    Metadata:
      comment         : Cover (front)
Stream mapping:
  Stream #0:0 -> #0:0 (vorbis (native) -> aac (aac_at))
Press [q] to stop, [?] for help
[mpegts @ 0x14b6147d0] frame size not set
Output #0, hls, to '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c.m3u8':
  Metadata:
    encoder         : Lavf60.16.100
  Stream #0:0: Audio: aac, 44100 Hz, stereo, s16, 256 kb/s
    Metadata:
      encoder         : Lavc60.31.102 aac_at
size=       0kB time=N/A bitrate=N/A speed=N/A    
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c0.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c1.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c2.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c3.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c4.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c5.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c6.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c7.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c8.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c9.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c10.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c11.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c12.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c13.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c14.ts' for writing
size=N/A time=00:00:00.00 bitrate=N/A speed=   0x    
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c15.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c16.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c17.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c18.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c19.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c20.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c21.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c22.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c23.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c24.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c25.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c26.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c27.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c28.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c29.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c30.ts' for writing
size=N/A time=00:00:49.13 bitrate=N/A speed=49.1x    
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c31.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c32.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c33.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c34.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c35.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c36.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c37.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c38.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c39.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c40.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c41.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c42.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c43.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c44.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c45.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c46.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c47.ts' for writing
size=N/A time=00:01:38.59 bitrate=N/A speed=65.7x    
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c48.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c49.ts' for writing
[out#0/hls @ 0x600003254900] video:0kB audio:4610kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
size=N/A time=00:01:41.07 bitrate=N/A speed=66.1x

[felix920506]

I also noticed a few extra things:

1. the exact same problem happens in Safari
2. Copying out the segments from the transcodes folder and playing the m3u8 file in VLC doesn't have this problem
3. On MacOS Safari the issue seems to goes away when I edit the web source code to use hls.js instead of Safari's native HLS implementation
4. If I disable HLS streaming music through web and make it transcode to a .aac instead, the issue also goes away on MacOS Safari (didn't test iOS Safari on this one).

[Chaphasilor]

I know what you're trying to achieve, I just mentioned the issues in case the transcoding wasn't taking place at all. That doesn't seem to be the case though.

It might be a bug in ffmpeg, that wouldn't be unheard of with regards to HLS transcoding. I'm not surprised VLC can handle it, but sadly that can't be used for playback in our case.

Since I don't have any Apple devices, I'm not able to reproduce the issue myself, and I don't have much experience with transcoding to begin with. Maybe @jmshrv can help out here.

Are there any more details you could share about the media? A demo file would probably help a lot!

[felix920506]

@Chaphasilor just grab some random music file and convert it to ogg Vorbis with ffmpeg will recreate the issue.

More details:

1. If I hack web to say other browsers don't support ogg Vorbis and cause it to transcode in both chrome and Firefox the issue doesn't happen.

2. I've seen some related discussion elsewhere about lower bitrates being unable to properly fill packets when using TS. https://www.wowza.com/community/t/stuttering-audio-on-jwplayer-in-safari-and-ios-using-hls-transport-streams/41395

[felix920506]

dropping by to add more info.
These are recordings of the same source file playing on my phone, one transcoded to HLS AAC in FinAmp and the other directly playing the source file in VLC.
The stutters should be quite obvious.

trim.9651AA2C-9BAB-4EFD-80AF-1D469F6C06F9.MOV

trim.7C714CC4-C6CA-43BC-87D6-7B1D7692066A.MOV

[Chaphasilor]

Okay, another question. What happens when you use the beta and download a transcoded version of that track. Transcoded downloads on iOS are limited to mp3 on 10.8, with support for AAC coming with 10.9.

Since regular transcoding always uses AAC afaik, maybe Vorbis->mp3 works better? 🤔

(Also, it's Finamp, not "FinAmp" ^^ Just like it's not "JellyFin" :P)

[felix920506]

How do I only download one song and not the entire album?

[felix920506]

Jellyfin-web was able to solve this by using fmp4 container for streaming instead of TS jellyfin/jellyfin-web#5378

[felix920506]

any update on this?

[felix920506]

Seems like it should be fixable by adding a parameter here:
"segmentContainer": "mp4"
(maybe a toggleable option for older devices that don't support using fMP4 containers)

finamp/lib/services/music_player_background_task.dart

Lines 738 to 746 in 2b08643

	queryParameters.addAll({
	"audioCodec": "aac",
	// Ideally we'd use 48kHz when the source is, realistically it doesn't
	// matter too much
	"audioSampleRate": "44100",
	"maxAudioBitDepth": "16",
	"audioBitRate":
	FinampSettingsHelper.finampSettings.transcodeBitrate.toString(),
	});

[Chaphasilor]

@felix920506 sorry, issue fell behind the desk 😅

You can download individual tracks on the beta version of Finamp. You should try with that, since it has many improvements. It's available through TestFlight.

I haven't looked into fmp4 yet, but if our audio library (just_audio) supports it, that should indeed be a possibility.

[felix920506]

I haven't looked into fmp4 yet, but if our audio library (just_audio) supports it, that should indeed be a possibility.

Can you add that as a toggleable setting so we can try it out on testflight builds?

Since regular transcoding always uses AAC afaik, maybe Vorbis->mp3 works better?

It should also happens with MP3, I tried it with jellyfin web

[felix920506]

You can download individual tracks on the beta version of Finamp. You should try with that, since it has many improvements. It's available through TestFlight.

I just tried with that and it doesn't happen. It seems like the transcoded download is asking for a complete file rather than segments which doesn't cause the issue.
The issue was narrowed down on jellyfin-web to the very specific scenario of:
Transcoding a Vorbis source to a HLS stream using TS container (both MP3 and AAC have this problem), then playing back said stream on a Apple HLS implementation (eg. Safari, iOS)

[Chaphasilor]

Okay, I just saw that I missed your code example earlier. Many thanks for that, I'll add an option to use fmp4 instead of TS. Sadly I can't give you a version to try out, so you'll have to wait for the next release!
You could try it out by setting up a Flutter environment and building it yourself though, if you're curious or want to help test.

[felix920506]

@Chaphasilor thanks

Do note that "ts" and "mp4" are what you want to pass in for TS and fMP4 respectively

[Chaphasilor]

@felix920506 update is released on Testflight, go give it a try!

[felix920506]

@Chaphasilor tried it on ios17 iPhone 13 mini, fixed the issue indeed