Session Replay video player add a blue line on the side

[krystofwoldrich]

In this replay from Sentry RN Sample app on iOS, the video player adds a blue line on the right side of the video.

Replay Link

The videos here in the web player preview also have the blue line.

Original Video From the Simulator

765112816.802941.mp4

Video Downloded by the Sentry Player

90_5428561_b4e0a63b565240f9baed08234107f6f0_0.video.mp4

[getsantry]

Routing to @getsentry/product-owners-replays for triage ⏲️

[philprime]

I ran ffprobe on the video file and this is the error output:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'output.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 1
    compatible_brands: isommp41mp42
    creation_time   : 2025-03-31T11:20:21.000000Z
  Duration: 00:00:05.00, start: 0.000000, bitrate: 11 kb/s
  Stream #0:0[0x1](und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p(tv, progressive), 393x852 [SAR 1:1 DAR 131:284], 10 kb/s, 1 fps, 1 tbr, 600 tbn (default)
      Metadata:
        creation_time   : 2025-03-31T11:20:21.000000Z
        handler_name    : Core Media Video
        vendor_id       : [0][0][0][0]

We believe the issue is related to the uneven width of 393 px, which is the view resolution on an iPhone 16 (https://useyourloaf.com/blog/iphone-16-screen-sizes/).

The Chrome video player seems to add a blue line to fill up to an even number.
The issue does not exist on Safari.

I believe this should be fixed in the web player rather than the SDK.

[romtsn]

Just noting that we're adjusting to a size divisible by 16 on Android (some codecs requirement), so we're likely not going to face this issue. https://github.com/getsentry/sentry-java/blob/de2136e055052b81ed59a3e3455e446a0f303018/sentry-android-replay/src/main/java/io/sentry/android/replay/ScreenshotRecorder.kt#L270-L281

[philprime]

@romtsn so this means any session replay from Android will be a multiple of 16, i.e.

IIUC this rounding logic applied to 393pt x 852pt would be 393pt / 16pt = 24,5625, so flooring the value to 24 * 16pt = 384pt width. When keeping the aspect ratio it would result in 852pt / 393pt * 384pt = 832,49pt height. Taking the floor value of that too, would be 832pt / 16pt = 52 multiples.

The target resolution would then be 384pt x 832pt but this adds a lot of scaling and rounding, doesn't it? How usable is session replay for pixel quality?

Also looking at the resolution of other screens, they are not alway 9 : 16 but sometimes even 9 : 19.5. I do not know about the android world, but couldn't screens on Android also not be multiples of 16 with variable aspect ratios?

[romtsn]

The target resolution would then be 384pt x 832pt but this adds a lot of scaling and rounding, doesn't it?

Yep it does, but I'm not sure if theres any other way if codecs work on 16x16 blocks. You either scale/round it or draw a blue line to fill the space and make it fit (I guess) :)

How usable is session replay for pixel quality?

Not sure I got this, but I think it's usable and perceived by a human eye just fine. It does not have to be pixel perfect, but good enough.

---

I might be wrong though and this logic can be the reason for mismatching mask but this I'm yet to figure out 😅 so far what I've seen it worked fine (but not pixel perfect obviously)

[ryan953]

I can see the blue line from within quicktime on my computer. It appears during playback; after i click pause it will disappear.

IMG_7589.MOV

[ryan953]

In VLC.app on OSX it doesn't seem to play properly. It starts off as a blank/black screen then seems like it end after 3s.

Quicktime shows that the original mp4 is 6s and plays for 6s. While vlc's media library shows 6s duration, but ends at 3.

IMG_7593.MOV

[ryan953]

I'm testing with the data from https://sentry-sdks.sentry.io/api/0/projects/sentry-sdks/sentry-react-native/replays/b4e0a63b565240f9baed08234107f6f0/videos/0/ which is from the replay linked at the top.

I downloaded it, and renamed the file from *.video to *.mp4
https://github.com/user-attachments/assets/f7aa6630-8532-4363-823b-36bea1cb4fa6

[ryan953]

The issue does not exist on Safari.
I believe this should be fixed in the web player rather than the SDK.

It also renders a blue line in Firefox.

@philprime afaik i don't think there's a way to fix this within the browser/player. Each browser has it's own media playback stack & codecs and we're not able to control that unless we had different encodings of the video (MPEG-4, WebM, etc) which we could generate on the server as an option.

It would be surprising to me if an odd numbered width like 393px was the cause of the problem in either encoding or playback software. I think we would've noticed a limitation like that in video files long ago.

It is suspicious, but not surprising, that iOS and Quicktime.app both work great together: their encoder/decoder are probably based on the same codebase. When software like Chrome, Firefox, and VLC are all failing in the same way, it makes me think there's an encoding quirk (that maybe the codecs these more open platforms share) doesn't handle.

[krystofwoldrich]

@ryan953 I can confirm, when I pause and play the video, I can see the line in QT as well.

[philprime]

@ryan953 I checked the VLC debug log and found a couple of hints:

main warning: picture is too late to be displayed (missing XXXX ms)

• The MP4 file structure is valid (mdat, moov, trak, avc1), but the actual video frames appear to be corrupted, missing, or black frames.
• The decoder (videotoolbox) is active and sees 6 samples (~6 seconds), but the frames can’t be rendered in time, and VLC drops them due to lateness.

mp4 debug: unrecognized major media specification (mp42)

mp42 might be a less common identifier for MPEG-4 files and VLC may have trouble with specific encoder settings used. I'll check if there are other formats available.

mp4 warning: elst box found
...
mp4 debug: elst (0) gives 0ms (movie)-> 0ms (track)

The edit list might be causing misaligned playback, where video is technically there but never displayed due to timing mismatch.

⸻

After some quick research I tried to re-encode the video using ffmpeg

$ ffmpeg -i video.mp4 -c:v libx264 -c:a copy output.mp4

But it failed with this error:

[libx264 @ 0x1378051b0] width not divisible by 2 (393x852)
[vost#0:0/libx264 @ 0x137804ee0] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
[vf#0:0 @ 0x6000028081e0] Error sending frames to consumers: Generic error in an external library
[vf#0:0 @ 0x6000028081e0] Task finished with error code: -542398533 (Generic error in an external library)
[vf#0:0 @ 0x6000028081e0] Terminating thread with return code -542398533 (Generic error in an external library)
[vost#0:0/libx264 @ 0x137804ee0] Could not open encoder before EOF
[vost#0:0/libx264 @ 0x137804ee0] Task finished with error code: -22 (Invalid argument)
[vost#0:0/libx264 @ 0x137804ee0] Terminating thread with return code -22 (Invalid argument)
[out#0/mp4 @ 0x600002c04000] Nothing was written into output file, because at least one of its streams received no packets.

Seems to me like ffmpeg is stricter than the AVFoundation.AVAssetWriter we use in sentry-cocoa to encode and requires an even width of the video.