Comparing with 1x1 pixel image should fail

[KjellMorgenstern]

When comparing a small image against an empty image, the compare unexpectedly considers them equal.

$ docker run --rm -v .:/tmp odiff /tmp/dfd86070d024eb67.png /tmp/b8aa7a84660a2e94.png -t 0
Success! Images are equal.
No diff output created.

$ file dfd86070d024eb67.png
dfd86070d024eb67.png: PNG image data, 1 x 1, 8-bit/color RGB, non-interlaced

$ file b8aa7a84660a2e94.png
b8aa7a84660a2e94.png: PNG image data, 106 x 106, 8-bit/color RGB, non-interlaced

b8aa7a84660a2e94.png:

If we set fail-on-layout, it also fails as expected:

$ docker run --rm -v .:/tmp odiff /tmp/dfd86070d024eb67.png /tmp/b8aa7a84660a2e94.png -t 0 --fail-on-layout
Failure! Images have different layout.

If we resize the 1x1 image, it starts to fail, as expected.

16x16 still doesn't detect the difference

$ convert dfd86070d024eb67.png -resize 16x16! output16x16.png
$ docker run --rm -v .:/tmp odiff /tmp/output16x16.png /tmp/b8aa7a84660a2e94.png -t 0 /tmp/diff.png
Success! Images are equal.

20x20 starts showing a diff

$ convert dfd86070d024eb67.png -resize 20x20! output20x20.png
$ docker run --rm -v .:/tmp odiff /tmp/output20x20.png /tmp/b8aa7a84660a2e94.png -t 0 /tmp/diff.png
Failure! Images are different.
Different pixels: 1 (0.250000%)

24x24 diff gets bigger

$ convert dfd86070d024eb67.png -resize 24x24! output24x24.png
$ docker run --rm -v .:/tmp odiff /tmp/output24x24.png /tmp/b8aa7a84660a2e94.png -t 0 /tmp/diff.png
Failure! Images are different.
Different pixels: 33 (5.729167%)

Looking at the generated diff for a 32x32 and 64x64 image:

The diff images suggest that the bigger image was just cropped? In that case, the remaining section would be equal.

In standard cases, where a few mostly empty lines were added on one side, or the image was slightly resized, I find it very useful that odiff can still produce sensible diffs.
But in these extreme cases, the comparison doesn't really work anymore.

Idea: A workaround (without changing the internal calculations much) could be to add a --layout-threshold param, that fails e.g. if the layout differs by a certain factor.

[dmtrKovalenko]

we provide a cli option for that called --fail-on-layout-diff that will fail early if the images layouts are different

[KjellMorgenstern]

In main/bin/ODiffBin.ml I can see the "--fail-on-layout" , not a "--fail-on-layout-diff", probably you meant that? I already tried --fail-on-layout, but then it would fail on each layout change, not only on extreme cases like here, right?

What I am doing for now: It isn't difficult to add a wrapper script, and then decide if a diff image to highlight changes is wanted. I have put a vips resize call in front, so I will always get images of comparable sizes. This way I still get diff images from odiff, even if one of the images is tiny.

But until I learned that a 1x1 pixel image can't be directly used for comparing, there went a few tests through, that produced empty images but didn't alert, because odiff considered them equal.

An option to fail on big size differences, but still produce diff images on small size differences, would make sense to me.

[dmtrKovalenko]

This sounds like an extremely custom use case and a very simple logic to write, why not to quickly parse image file for sizes (no need to decode) and manually validate image sizes for differnece

[KjellMorgenstern]

Yes, I solved it with a custom wrapper script.
I don't know how custom the use case is, no strong opinion about it. I could imagine adding such a option to limit the size factor to odiff would be easy to do, but I don't know the language. If it were my project I'd probably add it as a sanity check, enabled by default.