RPi5 CSI2 stride issue (with IMX335)

[jailuthra]

Describe the bug

When capturing IMX335 binned mode, with resolution 1320x972, the stride is off once the image passes through FE, leading to corrupt looking captures like in [1].

If I perform a raw CSI2-only capture, ignoring FE and BE in the media graph, the capture looks okay with RAW10P and RAW12P memory format, but has an 8-pixel wide column of image data missing (so only 1312x972 is valid) if the CSI2 remaps the data to RAW16 before writing to memory. [2] [3] [4]

I believe this error introduced while remapping is what messes up the image stride completely once it goes through FE, where it is unrecoverable.

The capture looks okay with the same sensor and sensor driver on a different platform (TI AM62A).

A different issue occurs if we configure the sensor to output 1296x972 instead, where the last few pixels in each row are repeated. This happens even in the RAW12P/RAW10P CSI2 only capture, but does not occur with other platforms.

The image looks okay only if the sensor is configured to output 1312x972 as the binned resolution.

Out of all these resolutions, only 1312 % 32 == 0, so am I correct in assuming that RPi5's CSI2 hardware block is unable to handle strides that are not divisible by 32? If not, can this be fixed in the driver?

Steps to reproduce the behaviour

1. Clone and build this branch with IMX335 binned mode: https://github.com/jailuthra/linux/commits/rpi-imx335-binned-1320/
2. Capture images using this libcamera branch: https://git.uk.ideasonboard.com/kbingham/libcamera/src/branch/kbingham/imx238_imx335 or manual raw v4l2 captures.

Device (s)

Raspberry Pi 5

System

Raspberry Pi reference 2024-07-04
Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 0b115f302a8f1e5bd3523614d7f45b9d447434c7, stage4

2025/05/08 15:13:17
Copyright (c) 2012 Broadcom
version 69471177 (release) (embedded)

Linux rpios 6.12.32-v8+ #4 SMP PREEMPT Tue Jun 10 23:57:18 IST 2025 aarch64 GNU/Linux

Logs

No response

Additional context

[1] Broken 1320 capture with FE + BE:

[2][3][4] Raw CSI2 only capture with RAW10P, RAW12P and RAW16 (with black bar):

[6by9]

Out of all these resolutions, only 1312 % 32 == 0, so am I correct in assuming that RPi5's CSI2 hardware block is unable to handle strides that are not divisible by 32? If not, can this be fixed in the driver?

Standard modes with imx219 include :

• 3280x2464 (3280 = 32 * 102.5)
• 1640x1232 (1640 = 32 * 51.25)

Standard modes with imx477 include:

• 4056x304 (4056 = 32 * 126.75)
• 2028x1520 (2028 = 32 * 63.375)

No issues observed with any of those modes, therefore there must be something incorrect in your assumption.

The stride of the output buffer reported as bytesperline will always be aligned to what appears to be 16 by the driver.
https://github.com/raspberrypi/linux/blob/rpi-6.12.y/drivers/media/platform/raspberrypi/rp1_cfe/cfe.c#L78
https://github.com/raspberrypi/linux/blob/rpi-6.12.y/drivers/media/platform/raspberrypi/rp1_cfe/cfe.c#L531-L539

The active width within the buffer with that stride can of course vary.

1. Clone and build this branch with IMX335 binned mode: https://github.com/jailuthra/linux/commits/rpi-imx335-binned-1320/

2. Capture images using this libcamera branch: https://git.uk.ideasonboard.com/kbingham/libcamera/src/branch/kbingham/imx238_imx335 or manual raw v4l2 captures.

Doing so would require an IMX335 module that is compatible with the Pi. I'm not aware of any vendor selling such.

[kbingham]

Doing so would require an IMX335 module that is compatible with the Pi. I'm not aware of any vendor selling such.

I thought you received this as a sample already from Arducam ?

[6by9]

Doing so would require an IMX335 module that is compatible with the Pi. I'm not aware of any vendor selling such.

I thought you received this as a sample already from Arducam ?

Lost track of which modules are which, and Arducam aren't selling it as a CSI module yet so it didn't show up in any searches.
Next job would be to find it, but gut feel is that this isn't a CFE issue as imx219 and imx477 are working without special stride constraints.

[naushir]

libcamera/libpisp will set the stride to be 64-byte aligned for optimal memory accesses for the BE block. I'm guessing camshark assumes width == stride and messes up the image display here?

[jailuthra]

libcamera/libpisp will set the stride to be 64-byte aligned for optimal memory accesses for the BE block. I'm guessing camshark assumes width == stride and messes up the image display here?

@naushir yes, I see 64-byte aligned stride being enforced by libpisp, but the issue happens even without camshark. For example using cam to capture raw frames (CSI2 + FE) as seen in the pixel viewer output below. I have set the stride to 2688, to match what libcamera/pisp used for the buffer.

but gut feel is that this isn't a CFE issue as imx219 and imx477 are working without special stride constraints.

@6by9 Good to know this doesn't happen with other sensors.

While testing more, when I set HNUM = 2*width - 24 it fixed something in the CSI2 output of the sensor, so now the black-bar + artefacts are gone, but instead the same duplication issue from 1296 mode is introduced on the right edge (see below)

diff --git a/drivers/media/i2c/imx335.c b/drivers/media/i2c/imx335.c
index 854753bf4544..86a9671afa7e 100644
--- a/drivers/media/i2c/imx335.c
+++ b/drivers/media/i2c/imx335.c
@@ -358,7 +358,7 @@ static const struct cci_reg_sequence mode_1320x972_regs[] = {
        { IMX335_REG_WINMODE, 0x01 },
        { IMX335_REG_HMAX, 275 },
        { IMX335_REG_HTRIMMING_START, 48 },
-       { IMX335_REG_HNUM, 2618 },
+       { IMX335_REG_HNUM, 2616 },
        { IMX335_REG_Y_OUT_SIZE, 972 },
        { IMX335_REG_AREA2_WIDTH_1, 48 },
        { IMX335_REG_AREA3_WIDTH_1, 3936 },

Duplication on the right edge:

I am working on upstreaming freely configurable cropping/binning support for this sensor, and testing on Pi produced these artefacts. On AM62A the capture looks okay for all resolutions, so I am not sure what about this specific setup is broken.

Maybe something is wrong with the sensor's CSI2 output, and only the Pi's CSI2 hardware block is having problems parsing it.. unfortunately I don't have a CSI analyzer to verify my suspicions.

[naushir]

@jailuthra would you be able to try running rpicam-still -t 5s -o test.jpg and share the output? That will handle any stride changes correctly on the jpg output.

[6by9]

With the duplication, have you tried memsetting the buffer between frames? Is it actually being written by the CFE hardware, or it is lying around from previous use?

[jailuthra]

@jailuthra would you be able to try running rpicam-still -t 5s -o test.jpg and share the output? That will handle any stride changes correctly on the jpg output.

@naushir here's the output of rpicam-still -t 5s --width=1320 --height=972 -o test.jpg with the latest sensor register settings, still see the duplication on the right edge:

here's the jpg with the older register settings, still see the artefacts:

With the duplication, have you tried memsetting the buffer between frames? Is it actually being written by the CFE hardware, or it is lying around from previous use?

@6by9 I see the duplication on right edge even when capturing raw frames with csi2 video device using v4l2-ctl -d0 --stream-mmap (no CFE/BE involved) - I didn't memset the buffers but I believe v4l2-ctl would do it, and in any case the first frame there had the duplication as well.

[naushir]

There's quite a bit of kernel logging in the cfe driver related to format and buffer sizing. Can you enable dynamic debug (off by default on rpi kernels) and post the log here please when running the rpicam-still command?

[naushir]

I see the duplication on right edge even when capturing raw frames with csi2 video device using v4l2-ctl -d0 --stream-mmap (no CFE/BE involved)

The CSI2 block is not able to replicate or add any pixels to the output, it is pretty much a straight DMA transfer. This likely means it originates from the sensor.

[6by9]

videobuf2 will memset the buffers at allocation, but that doesn't stop something having scribbled in there at some point.

Maybe my datasheet is out of date (0.2 2017/06/06), but it lists

Total number of pixels: 2704(H) × 2104(V) = 5.69 M
Number of effective pixels: 2616(H) × 1964(V) = 5.14 M
Number of active pixels: 2608(H) × 1960(V) = 5.11 M
Number of recommended recording pixels: 2592(H) × 1944(V) = 5.04 M

Asking for 2618 pixels would therefore exceed the number of effective pixels of the sensor. My diagram has no information about what is in the additional (2704-2616 =) 88 pixels beyond the listed effective pixels, as it says that there are 0 pixels in the "Ignored area for effective pixel size" region.

I've also just noted in the Window Cropping mode section (page 55 for me)

48 ≦ HTRIMMING_START + HNUM ≦ 2664
HTRIMMING_START = 48 + N×12
312 ≦ HNUM
Set HNUM to a multiple of 24. <<<<<<<<<<<<<<
(N is integer equal or more than 0)

So 2618 is not permitted for HNUM as it isn't a multiple of 24. Where the "312 <= HNUM" comes from I can't immediately work out, as their examples list it as more than that.

Sensors do whacky things if you ask for something outside their pixel array or underflow their FIFOs.

[jailuthra]

Can you enable dynamic debug (off by default on rpi kernels) and post the log here please when running the rpicam-still command?

rp1-cfe-still-logs.txt

The CSI2 block is not able to replicate or add any pixels to the output, it is pretty much a straight DMA transfer. This likely means it originates from the sensor.

Agreed that something is definitely wrong with the CSI2 packet that originates from the sensor.

Asking for 2618 pixels would therefore exceed the number of effective pixels of the sensor. My diagram has no information about what is in the additional (2704-2616 =) 88 pixels beyond the listed effective pixels, as it says that there are 0 pixels in the "Ignored area for effective pixel size" region.
So 2618 is not permitted for HNUM as it isn't a multiple of 24. Where the "312 <= HNUM" comes from I can't immediately work out, as their examples list it as more than that.
Sensors do whacky things if you ask for something outside their pixel array or underflow their FIFOs.

Ah yes, I noticed that in the datasheet as well. 2618 is definitely broken, and the sensor may be putting some invalid data in CSI2 packets. We got that value from a register table that worked for Arducam.

But this whole messing around with the sensor boundaries started precisely because the datasheet suggested settings for 2x2 binned mode i.e. 1296x972 was causing the duplication artefact on the right edge on the Pi.

The datasheet also does not specify any value for the HNUM register when using 2x2 binned 1296x972 mode. And the default value of that register is 0xa78 -> 2616, which leads to frames with width 1320. To get actual 1296x972 working, Umang had modified HNUM = 2568 (i.e 2592-24), and in both cases we see the artefact on the right edge.

The only case where there is no artefact seen on the Pi is when setting HNUM = 2600, and width = 1312. Which is wrong according to the datasheet as it is not divisible by 24.

[naushir]

[  393.491530] rp1-cfe 1f00128000.csi: try_fmt_vid_cap: [fe_image0] 1320x972, V4L2 pix PC1R
[  393.491535] rp1-cfe 1f00128000.csi: cfe_calc_format_size_bpl: PC1R size: 1320x972 bpl:1344 img_size:1306368
[  393.491539] rp1-cfe 1f00128000.csi: cfe_s_fmt_vid_cap: Set 1320x972, V4L2 pix PC1R

Nothing seem strange in the dmesg logs. Looks like you are requesting 1320x972 from the sensor, in pisp Bayer compression mode 1 (PC1R), giving a buffer stride of 1344 (1320 aligned to 64) and buffer size 1344*972 = 1306368 bytes.

[6by9]

Ah yes, I noticed that in the datasheet as well. 2618 is definitely broken, and the sensor may be putting some invalid data in CSI2 packets. We got that value from a register table that worked for Arducam.

But this whole messing around with the sensor boundaries started precisely because the datasheet suggested settings for 2x2 binned mode i.e. 1296x972 was causing the duplication artefact on the right edge on the Pi.

The datasheet also does not specify any value for the HNUM register when using 2x2 binned 1296x972 mode. And the default value of that register is 0xa78 -> 2616, which leads to frames with width 1320. To get actual 1296x972 working, Umang had modified HNUM = 2568 (i.e 2592-24), and in both cases we see the artefact on the right edge.

The only case where there is no artefact seen on the Pi is when setting HNUM = 2600, and width = 1312. Which is wrong according to the datasheet as it is not divisible by 24.

This is the situation where you start to need a CSI2 analyser, a very good FAE at Sony (except they'll just give you a spreadsheet of registers), or a shed-load of patience to just try things.
All my testing with imx327/290/462 and imx415 just seemed to fall into place. It seems like the cropping isn't quite so accommodating :-/

(That said, analysers are fallible too - we have a DSI analyser and I spent a fair amount of time trying to work out why I was getting rubbish in some situations (colour shifts, and significant horizontal shifts), and it appeared to be the analyser at fault. Connected the display back up and it was quite happy with what was generated!)