-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Description
Description
We are seeing reproducible kernel crashes during H.264/H.265 decoding on RK3588 when using the RKVDEC driver.
The issue appears in the decoder cleanup path (mpp_dma_release → mpp_task_finalize → rkvdec_free_task).
Kernel Log
Example trace:
kernel: rk_gmac-dwmac fe1c0000.ethernet ethO: Link is Up - 1Gbps/Full - flow control rx/tx
kernel: IPv6: ADDRCONF(NETDEV_CHANGE): ethO: link becomes ready
kernel: ttyFIQ ttyFIQ0: tty_port_close_start: tty->count = 1 port count = 2
kernel: dwc3 fc000000.usb: request 0000000006337ea4 was not queued to epdout
kernel: android_work: did not send uevent (0 © 0000000000000000)
kernel: of_dma_tequest_slave_channel: dma-names property of node '/serialafeb60000' missing or empty
kernel: dw-apb-uart feb60000.serial: failed to request DMA, use interrupt mode
kernel: vec3v3_lcd0_n: disabling
kernel: vec_1v8_cam_s0: disabling
kernel: vec_2v8_cam_s0: disabling
kernel: Unable to handle kernel paging request at virtual address 0000000000001260
kernel: Mem abort info:
kernel: ESR = 0x96000005
kernel: EC = 0x25: DABT (current EL), IL = 32 bits
kernel: SET = 0, FnV = 0
kernel: EA = 0, S1PTW = 0
kernel: Data abort info:
kernel: ISV = 0, ISS = 0x00000005
kernel: CM = 0, WnR = 0
kernel: user pgtable: 4k pages, 39-bit VAs, pgdp=00000001f9035000
kernel: [0000000000001260] pgd=0000000000000000, p4d=0000000000000000, pud=0000000000000000
kernel: Internal error: Oops: 96000005 [#1] SMP
kernel: Modules Linked in:
kernel: CPU: 1 PID: 149 Comm: queue_work9 Not tainted 5.10.160 #28
kernel: Hardware name: Rockchip RK3588S MARS3500s (DT)
kernel: pstate: 40c00009 (nZcv daif +PAN +UAO -TCO BTYPE=--)
kernel: pe : mutex_Lock+0x28/0x70
kernel: ir : mutex_Lock+0x1c/0x70
kernel: sp : ffffffcodadd3c60
kernel: x29: FFFFFfcOdadd3c60 x28: FFFFFFE1F9752000
kernel: x27: FFFFFfCOOa20cb48 x26: FFFFFFCOVa20c000
kernel: x25: 0000000000000107 x24: FfFfFF81FOC9fcaO
kernel: x23: fFFFFF81f96fa00 x22: FFFFFFE1F9752038
kernel: x21: FfffFf81f975a088 x20: 0000000000001260
kernel: x19: 0000000000001a60 x18: 0000000000000000
kernel: x17: 0000000000000000 x16: 0000000000000000
kernel: x15: 0000007f8904e8c8 x14: 0000000000000000
kernel: x13: 0000000000000000 x12: o000000000000000
kernel: x11: 0000000000000001 x10: 0000000000000990
kernel: x9 : FFfFFFCOO9ICFSE4 x8 : FFTFFFBIfidibfro
kernel: x7 : 0000000000000001 x6 : FFFTFF81F975ae48
kernel: x5 : 0000000000000000 x4 : 0000000000000000
kernel: x3 : 0000000000000000 x2 : 0000000000000000
kernel: x1 : 000000000000000 xo : FFFfFF81fidib600
kernel: Call trace
kernel: mutex_lock+0x28/0x70
kernel: mpp_dma_release+0x28/0xac
kernel: mpp_task_finalize+0x60/Oxb4
kernel: rkvdec2_free_task+0x1c/0:
kernel: mpp_free_task+0x50/0xfO
kernel: rkvdec2_Soft_ccu_worker+0x254/Oxcf4
kernel: kthread_worker_fn+0x12c/0x280
kernel: kthread+0x12c/0x130
kernel: ret_from_fork+0x10/0x24
Suspected Problem Areas
File: drivers/video/rockchip/mpp/mpp_rkvdec.c
1- fill_scaling_list_pps()
pps = map.vaddr + offset;
memcpy(&scaling_fd, pps + base, sizeof(scaling_fd));
No bounds checking on offset / base. If userspace provides a bad PPS buffer or offset, pps + base can point outside the mapped area, leading to memory corruption and later kernel Oops during cleanup.
2- rkvdec_alloc_task() fail path
mpp_task_finalize(session, mpp_task);
kfree(task);
Always calls mpp_task_finalize(), even if the task wasn’t fully initialized. This can trigger invalid releases in mpp_dma_release().
Conditions
- Happens during H.264/H.265 decoding (not using VP8/VP9).
*Triggered when PPS scaling list handling is active (e.g., corrupted PPS buffer or unusual offset). - Platform: RK3588, kernel 5.10.160 (also relevant for 6.1 branch).
Suggested Fix
- Add proper bounds checks in fill_scaling_list_pps() to ensure (offset + base + sizeof(u32)) <= buffer_size.
- Guard mpp_task_finalize() so it only runs when the task is fully initialized.
Impact
This leads to hard kernel crashes during decoding. Fixing it would improve decoder robustness against bad/corrupt input streams.