Process Termination in Native AOT Build when ProcessingPipeline Encounters Missing HEIC Codec

[riyasy]

When processing a HEIC/HEIF file on a Windows system that does not have the required WIC codec (e.g., a default Windows 10 installation), pipeline.WriteOutput(ms); causes the entire application to crash and terminate when the application is compiled with Native AOT.

In a standard JIT/.NET build, the same code path correctly throws a manageable exception, which can be caught and handled gracefully. The issue is specific to the Native AOT compilation target, where the process exits immediately, and the try...catch block is bypassed.

try
{
	var settings = new ProcessImageSettings 
	{
		Width = 800, Height = 800, 
		ResizeMode = CropScaleMode.Max, 
		HybridMode = HybridScaleMode.Turbo 
	};
	using var pipeline = MagicImageProcessor.BuildPipeline(inputPath, settings);
	using var ms = new MemoryStream();
	pipeline.WriteOutput(ms);

	// Do other things with ms
}
catch (Exception ex)
{
	Logger.Error(ex);
	// Do cleanup
}

Application type : WinUI3 - C# application
PhotoSauce.MagicScaler - Version=0.15.0
.NET Version: .NET 9.0
Operating System: Windows 10
Target Runtime / Build Configuration: Native AOT (win-x64)

[rickbrew]

What's the exception that you get with the standard JIT build?

[rickbrew]

btw WIC's HEVC/HEIC codec is, as they say, buggy AF, so it would not surprise me at all if this is its fault. If you run under the debugger with mixed-mode enabled ("Enable debugging for managed and native code together"), you can see they throw and catch a lot of their own exceptions on separate threads and sometimes they're very scary looking ("oh let's just ignore Access Violation ...").

[riyasy]

What's the exception that you get with the standard JIT build?

System.Runtime.InteropServices.COMException: No suitable transform was found to encode or decode the content. (0xC00D5212)

Stack Trace -

at System.Runtime.InteropServices.Marshal.ThrowExceptionForHR(Int32 errorCode)
at TerraFX.Interop.Windows.HRESULT.Check(HRESULT hr)
at PhotoSauce.MagicScaler.WicFramePixelSource.CopyPixelsInternal(PixelArea& prc, Int32 cbStride, Int32 cbBufferSize, Byte* pbBuffer)
at PhotoSauce.MagicScaler.PixelSource.CopyPixels(PixelArea& prc, Int32 cbStride, Int32 cbBufferSize, Byte* pbBuffer)
at PhotoSauce.MagicScaler.Transforms.ConversionTransform.copyPixelsBuffered(PixelArea& prc, Int32 cbStride, Byte* pbBuffer)
at PhotoSauce.MagicScaler.Transforms.ConversionTransform.CopyPixelsInternal(PixelArea& prc, Int32 cbStride, Int32 cbBufferSize, Byte* pbBuffer)
at PhotoSauce.MagicScaler.PixelSource.CopyPixels(PixelArea& prc, Int32 cbStride, Int32 cbBufferSize, Byte* pbBuffer)
at PhotoSauce.MagicScaler.Transforms.ConversionTransform.copyPixelsDirect(PixelArea& prc, Int32 cbStride, Byte* pbBuffer)
at PhotoSauce.MagicScaler.Transforms.ConversionTransform.CopyPixelsInternal(PixelArea& prc, Int32 cbStride, Int32 cbBufferSize, Byte* pbBuffer)
at PhotoSauce.MagicScaler.PixelSource.CopyPixels(PixelArea& prc, Int32 cbStride, Int32 cbBufferSize, Byte* pbBuffer)
at PhotoSauce.MagicScaler.Transforms.ConvolutionTransform3.loadBuffer(Int32 first, Int32 lines) at PhotoSauce.MagicScaler.Transforms.ConvolutionTransform3.CopyPixelsInternal(PixelArea& prc, Int32 cbStride, Int32 cbBufferSize, Byte* pbBuffer)
at PhotoSauce.MagicScaler.PixelSource.CopyPixels(PixelArea& prc, Int32 cbStride, Int32 cbBufferSize, Byte* pbBuffer)
at PhotoSauce.MagicScaler.Transforms.ColorMatrixTransformInternal.CopyPixelsInternal(PixelArea& prc, Int32 cbStride, Int32 cbBufferSize, Byte* pbBuffer)
at PhotoSauce.MagicScaler.PixelSource.CopyPixels(PixelArea& prc, Int32 cbStride, Int32 cbBufferSize, Byte* pbBuffer)
at PhotoSauce.MagicScaler.Transforms.ConvolutionTransform3.loadBuffer(Int32 first, Int32 lines) at PhotoSauce.MagicScaler.Transforms.ConvolutionTransform3.CopyPixelsInternal(PixelArea& prc, Int32 cbStride, Int32 cbBufferSize, Byte* pbBuffer)
at PhotoSauce.MagicScaler.PixelSource.CopyPixels(PixelArea& prc, Int32 cbStride, Int32 cbBufferSize, Byte* pbBuffer)
at PhotoSauce.MagicScaler.Transforms.ConversionTransform.copyPixelsBuffered(PixelArea& prc, Int32 cbStride, Byte* pbBuffer)
at PhotoSauce.MagicScaler.Transforms.ConversionTransform.CopyPixelsInternal(PixelArea& prc, Int32 cbStride, Int32 cbBufferSize, Byte* pbBuffer)
at PhotoSauce.Interop.Wic.IWICBitmapSourceImpl.copyPixels(IWICBitmapSource* pinst, WICRect* prc, UInt32 cbStride, UInt32 cbBufferSize, Byte* pbBuffer)
at TerraFX.Interop.Windows.IWICBitmapFrameEncode.WriteSource(IWICBitmapSource* pIBitmapSource, WICRect* prc)
at PhotoSauce.MagicScaler.WicImageEncoder.writeSource(IWICBitmapFrameEncode* frame, PixelSource src, PixelArea area)
at PhotoSauce.MagicScaler.WicImageEncoder.WriteFrame(IPixelSource source, IMetadataSource meta, Rectangle area)
at PhotoSauce.MagicScaler.MagicImageProcessor.WriteOutput(PipelineContext ctx, Stream ostm)
at PhotoSauce.MagicScaler.ProcessingPipeline.WriteOutput(Stream outStream)
at FlyPhotos.Readers.WicReader.<>c__DisplayClass4_0.<g__Action|0>d.MoveNext() in C:\Users\Riyas\Documents\Github\FlyPhotos\Src\FlyPhotos\Readers\WicReader.cs:line 92

[saucecontrol]

It's strange that you get as far as CopyPixels if there's no codec installed, but that's probably unrelated. I'd guess the issue here is a managed exception being thrown while in a callback from native code. CLR on Windows handles unwinding across P/Invoke boundaries fine, but I guess NAOT doesn't? If that's all it is, I have a solution already (it's required for Linux even on CLR).

If you could put together a complete repro project (hopefully without WinUI 😅), that would help me get to this more quickly.

Another thing you could try is simply calling CopyPixels on the pipeline yourself instead of processing into an encoder stream. If my guess is right, it's the native encoder calling back into the managed pipeline (where the exception is thrown) that's surfacing the issue.

[riyasy]

AOT_Test_MagicScaler_HEIF.zip

Please find attached demo app..

Publish using

dotnet publish -c Release -r win-x64

[rickbrew]

It's strange that you get as far as CopyPixels if there's no codec installed

For some of WIC's codecs, like HEVC/HEIC and JPEG XL, there's a stub that's always present. The codec still shows up when you enumerate WIC components, but then it doesn't work when you try to instantiate or use it.

[riyasy]

For some of WIC's codecs, like HEVC/HEIC and JPEG XL, there's a stub that's always present. The codec still shows up when you enumerate WIC components, but then it doesn't work when you try to instantiate or use it.

Yes.. When I enumerate WIC codecs, HEIF and HEIC gets listed (in Windows 10) and my app accepts those file types..

[rickbrew]

This is what you need to do to detect whether HEVC/HEIC is actually available. This is mostly using TerraFX.Interop.Windows, the PDN specific stuff should be easily translatable:

using PaintDotNet.Interop;
using PaintDotNet.Runtime;
using PointerToolkit;
using System;
using System.Runtime.InteropServices;
using TerraFX.Interop.Windows;

namespace PaintDotNet.Imaging;

using static TerraFX.Interop.Windows.MFT;
using static TerraFX.Interop.Windows.Windows;

internal static class HeifCodecInfos
{
    // mfplat.dll isn't part of Windows "N" editions unless you install the "Windows Media Feature Pack"
    // https://forums.getpaint.net/topic/115194-if-installation-fails-first-make-sure-you-are-not-using-windows-10-n/
    private static readonly IntPtr hMFPlatDLL = LoadMFPlatDll();
    private static IntPtr LoadMFPlatDll()
    {
        IntPtr handle;
        bool result = NativeLibrary.TryLoad(
            "mfplat.dll",
            typeof(HeifCodecInfos).Assembly,
            DllImportSearchPath.System32,
            out handle);
        return handle;
    }

    public static MicrosoftStoreCodecInfo Heic { get; } = new MicrosoftStoreCodecInfo(
        OSInfo.IsWin10v1809OrLater,
        [".heic", ".heif", ".hif"],
        [".heic", ".heif", ".hif"],
        () => IsVideoCodecAvailable(CodecCategory.Decoder, MFVideoFormat.MFVideoFormat_HEVC),
        () => IsVideoCodecAvailable(CodecCategory.Encoder, MFVideoFormat.MFVideoFormat_HEVC),
        OSInfo.IsWin10v1809OrLater,
        "https://www.microsoft.com/p/hevc-video-extensions/9nmzlz57r3t7");

    private enum CodecCategory
    {
        Encoder,
        Decoder
    }

    private static unsafe bool IsVideoCodecAvailable(CodecCategory category, in Guid format)
    {
        if (hMFPlatDLL == IntPtr.Zero)
        {
            return false;
        }

        MFT_REGISTER_TYPE_INFO typeStruct;
        typeStruct.guidMajorType = MFMediaType_Video;
        typeStruct.guidSubtype = format;

        MFT_REGISTER_TYPE_INFO* pInputType;
        MFT_REGISTER_TYPE_INFO* pOutputType;
        Guid guidCategory;
        switch (category)
        {
            case CodecCategory.Decoder:
                guidCategory = MFT_CATEGORY_VIDEO_DECODER;
                pInputType = &typeStruct;
                pOutputType = null;
                break;

            case CodecCategory.Encoder:
                guidCategory = MFT_CATEGORY_VIDEO_ENCODER;
                pInputType = null;
                pOutputType = &typeStruct;
                break;

            default:
                throw ExceptionUtil.InvalidEnumArgumentException(category, nameof(category));
        }

        using CoTaskAlloc<Ptr<IMFActivate>> ppActivates = default;
        uint cActivates = 0;
        try
        {
            HRESULT hr = MFTEnumEx(
                guidCategory,
                (uint)_MFT_ENUM_FLAG.MFT_ENUM_FLAG_SYNCMFT,
                pInputType,
                pOutputType,
                (IMFActivate***)&ppActivates,
                &cActivates);

            return hr.SUCCEEDED && (cActivates > 0);
        }
        finally
        {
            if (ppActivates.Get() != null)
            {
                for (long i = 0; i < cActivates; ++i)
                {
                    ppActivates.Get()[i].Get()->Release();
                }
            }
        }
    }
}

The constructor for MicrosoftStoreCodecInfo:

    internal MicrosoftStoreCodecInfo(
        bool isSupported,
        IEnumerable<string> loadExtensions,
        IEnumerable<string> saveExtensions,
        Func<bool> isDecoderAvailableFn,
        Func<bool> isEncoderAvailableFn,
        bool canBeInstalled,
        string installUrl);

[riyasy]

Another thing you could try is simply calling CopyPixels on the pipeline yourself instead of processing into an encoder stream. If my guess is right, it's the native encoder calling back into the managed pipeline (where the exception is thrown) that's surfacing the issue.

I tried CopyPixels and the exception was caught in AOT version of exe. But for non HEIC files like JPEG, the output was mangled. Is there a documented version of CopyPixels which I can use.

[saucecontrol]

Thanks for the repro project and for trying that. It wasn't really intended as a workaround, only a way to test quickly whether replacing the native caller would fix the exception handling.

CopyPixels from the pipeline can be a little tricky to use, because there is currently no way to query the pixel format of the pipeline without materializing it (after which you can't make changes to it). There's a bit of explanation and sample code here: #175 (comment)

Basically, if you force the pixel format to something you are prepared to consume, it will no-op if the pipeline is already that format, otherwise it will convert for you so that you can use the same code for all image sources/formats.

[riyasy]

This is what you need to do to detect whether HEVC/HEIC is actually available. This is mostly using TerraFX.Interop.Windows, the PDN specific stuff should be easily translatable:

@rickbrew Thanks a lot.. I made something out of this code which prevents the crash by skipping MagicScaler in PCs where HEVC is not available. Works good with iphone photos. Haven't checked other sources of heif/heic files.
But I fear there can be other wrong reporting by WIC about unavailable codecs as being available which can crash my app..

[rickbrew]

But I fear there can be other wrong reporting by WIC about unavailable codecs as being available which can crash my app

It could also be worth baking that code into PhotoSauce. WIC also has a codec for JPEG XL that is weird in this way, but I don't use it in PDN for its JPEG XL support so I can't comment on it.