“Image too big” thrown when file does not exist

[Wrzlprmft]

When I enter a wrong filename, I do not get the expected FileNotFoundError, but: “Image too big for processing”:

import rawpy
from pytest import raises

filename = "wrzlprmft.CR2"

with raises(FileNotFoundError):
	open(filename,"rb")

with raises(FileNotFoundError):
	rawpy.imread("non_existing_file.CR2")

This happens to me with RawPy version 0.18.1 and rawpy.libraw_version being (0,21,1).

[letmaik]

The error is coming directly from libraw, looks like a bug. Can you report it over there?

[Wrzlprmft]

Generally yes, but I would have either only be able to relaying everything to this issue here without being able or I would have to spend some time on familiarising myself with LibRaw itself.

[jtomori]

Just tested rawpy 0.24.0 with the following:

with rawpy.imread("non_existing_file.CR2") as 
    pass

Which has raised rawpy._rawpy.LibRawIOError: b'Input/output error'

It looks like it's been fixed in the meantime.

[Wrzlprmft]

@jtomori I cannot confirm this (with the same version of RawPy), which suggests that it depends on the operating system or version of libraw.

[jtomori]

Could you copy what exception you get? This is what I get:

Python 3.11.9 (tags/v3.11.9:de54cf5, Apr  2 2024, 10:12:12) [MSC v.1938 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import rawpy
>>> rawpy.__version__
'0.24.0'
>>> rawpy.libraw_version
(0, 21, 3)
>>> with rawpy.imread("non_existing_file.CR2") as raw:
...     pass
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "c:\[redacted]\rawpy\__init__.py", line 20, in imread
    d.open_file(pathOrFile)
  File "rawpy\\_rawpy.pyx", line 413, in rawpy._rawpy.RawPy.open_file
  File "rawpy\\_rawpy.pyx", line 955, in rawpy._rawpy.RawPy.handle_error
rawpy._rawpy.LibRawIOError: b'Input/output error

[Wrzlprmft]

Okay, I just found something interesting. With pure Python, I get pretty much the same as you do. With IPython, however, I get:

LibRawTooBigError                  Traceback (most recent call last)
File rawpy_bug.py:10
      7 	open(filename,"rb")
      9 with raises(FileNotFoundError):
---> 10 	rawpy.imread("non_existing_file.CR2")

File ~/.local/lib/python3.13/site-packages/rawpy/__init__.py:20, in imread(pathOrFile)
     18     d.open_buffer(pathOrFile)
     19 else:
---> 20     d.open_file(pathOrFile)
     21 return d

File rawpy/_rawpy.pyx:413, in rawpy._rawpy.RawPy.open_file()

File rawpy/_rawpy.pyx:955, in rawpy._rawpy.RawPy.handle_error()

LibRawTooBigError: b'Image too big for processing'

[jtomori]

Interesting, I get the same LibRawIOError: b'Input/output error' even when running the sample above in IPython REPL / running script with ipython (IPython 8.27.0).

[jtomori]

What OS are you using? Today I've repeated the same test on Ubuntu (the same rawpy version) and I got the same error as you: LibRawTooBigError: b'Image too big for processing'. In the previous tests I was using Windows.

When I've tried running sample binaries from LibRaw (same version: 0.21.3) on non-existing files then I got this error: Input/output error implying that probably rawpy might be mapping LibRaw's exceptions incorrectly.

rawpy's error codes are matching LibRaw's. The error mapping on rawpy's end looks fine as well. LibRawIOError's return code is -100009 and LibRawTooBigError's is -100012 🤔

[Wrzlprmft]

What OS are you using?

I am on Arch Linux. But I have also seen this issue on Ubuntu.

[pierotofy]

Can confirm, I've also seen Image too big for processing when the file does not exist on Arch Linux. Mm.

[letmaik]

@pierotofy can you reliably reproduce it? Would be good to write a unit test that reproduces it in CI.

[pierotofy]

I'm going to guess this is a problem upstream with libraw?

INT64 _fsize; is never initialized if the file does not exist https://github.com/LibRaw/LibRaw/blob/master/libraw/libraw_datastream.h#L142 https://github.com/LibRaw/LibRaw/blob/master/src/libraw_datastream.cpp#L161-L186 and some compilers probably initialize that value to 0, but that's not guaranteed and is undefined behavior, so in some builds the value is probably some really large random number and that triggers the file size check later, which throws LIBRAW_TOO_BIG.

Edit: nvm, _fsize in initialized in the constructor.

[jtomori]

It might be worth reporting it to https://www.libraw.org/forum/15. They seem to be pretty quick with replying and fixing issues.

[pierotofy]

I'm just not 100% sure, after looking more carefully, open_file has a stat call which should fail and return the correct error code if the file does not exist. https://github.com/LibRaw/LibRaw/blob/master/src/utils/open.cpp#L32

Edit: the open file logic in libraw is more intricate than it looks; I really don't know what's the cause and would require some in-depth digging/debugging.