Setting manual raw_pattern

[mandulaj]

I am loading a Raw image (stored as tiff) however the default Raw pattern RGGB:

array([[0, 1],
       [3, 2]], dtype=uint8)

is not the correct one. I would need GBRG corresponding to something like:

array([[1, 3],
       [0, 2]], dtype=uint8)

However I haven't found a way how to set this manually in rawpy.

[letmaik]

Can you post your complete code and upload an example image?

[mandulaj]

Hey @letmaik, thanks for getting back to me.

Let me show you an example. This is the RAW un-debayerd TIFF (I can also provide the original file if needed). The Sensor has the following bayer pattern order GBRG:

Ignoring the green tint, this is what the correct debayering using https://github.com/cheind/pytorch-debayer and the GBRG bayer pattern results in:

I am using the RawPy library in a very simple way:

import rawpy

with rawpy.imread("image.tif") as raw_img:
    img = raw_img.postprocess()
    plt.imshow(img)
    plt.show()

However this results in the following results:

Clearly rawpy is using a different bayer pattern for the debayering process. This can be seen by accessing the raw_pattern property showing it uses RGGB. Since raw_pattern is read only, I haven't found a way to change it to GBRG.

[mandulaj]

Here is the RAW TIFF:
image.tif.zip

[letmaik]

It's unusual to have RAW images in TIFF format. The issue is that there's not enough metadata in the file I think. The underlying library libraw then doesn't know which Bayer pattern to apply. If I may ask, where do you get this image from? Is it produced by a camera directly?

[mandulaj]

Yes, I know, the TIFFs are in fact written by myself, so I could even add the necessary metadata if I knew what it was.

I don't want to go into details, but the summary is, for performance reasons, TIFF is the fastest format we can write the data to disk. We tried several Debayering and post-processing algorithms, including the standard OpenCV, 5x5 convolutions, but libraw yielded the best results so far (on a different camera model that has the default RGGB pattern).

So let me get this straight, the underlying libraw library is expecting some kind of metadata that indicates the bayer pattern, however, there is no API to set it manually.

Since you are probably more familiar with the Libraw library, would you happen to know which metadata would have to be added? Perhaps if you point me in the right direction, I can try to add this feature to rawpy to set the pattern manually.

[letmaik]

libraw seems to have ways to read from image data without metadata as well, e.g. https://www.libraw.org/docs/API-CXX.html#open_bayer. Maybe that would suit your use case. This is not currently implemented in rawpy but it seems like a useful thing to have. There is normally quite a bit of metadata in RAW camera files that influence the postprocessing, but maybe that's not needed in your case and the Bayer pattern is enough.

[mandulaj]

I will look into it once I have time, but I am not that familiar with making cpython bindings with a C library so not sure if I will be able to do this myself. 😅

[letmaik]

I gave it a try and got it to work. I'll give you something to test soon.

[letmaik]

This is not officially released yet as I'd like you to try it out first. You can manually download Python wheels with this feature from the workflow artifacts (bottom right) from here: https://github.com/letmaik/rawpy/actions/runs/7734024290

This is how you would use it with imageio:

import rawpy
import imageio

arr = imageio.imread("rawbayer.tiff")
with rawpy.RawPy() as raw:
    raw.open_bayer(arr.tobytes(), arr.shape[1], arr.shape[0], "GBRG")
    rgb = raw.postprocess()

And here with pillow:

import rawpy
from PIL import Image

img = Image.open("rawbayer.tiff")
with rawpy.RawPy() as raw:
    raw.open_bayer(img.tobytes(), img.width, img.height, "GBRG")
    rgb = raw.postprocess()

For now, I'm not exposing it under a high-level interface like rawpy.imread since it is quite advanced and I haven't made up my mind yet on how that interface would look like. For your purposes, this doesn't matter though.

Please try it out and let me know if it's working as expected.

[kmilos]

I don't want to go into details, but the summary is, for performance reasons, TIFF is the fastest format we can write the data to disk.

So... just throw in some basic metadata on top (CFA pattern, color matrix, etc.) and you have yourself a DNG. 😉 See e.g. https://www.libraw.org/comment/5771#comment-5771 and https://github.com/schoolpost/PiDNG

[jtomori]

Hi, an alternative solution would be to specify the bayer pattern on the tif as a metadata and then let LibRaw interpret it correctly:

cp image.tif image_modified.tif

exiftool \
    -overwrite_original \
    -IFD0:CFAPattern2="1 2 0 1" \
    image_modified.tif

Then running your snippet will display the modified image like this:

import rawpy
import matplotlib.pyplot as plt

with rawpy.imread("image_modified.tif") as raw_img:
    img = raw_img.postprocess()
    plt.imshow(img)
    plt.show()

Tested with rawpy 0.24.0.