[Feature Request]: Add support for LightOnOCR-2-1B

[Cafelatte1]

Request Description

Sorry, this is a model that came out yesterday, but please add feature LightOnOCR model !
I couldn't convert huggingface repo model to int8 quantized IR model.
Command:
optimum-cli export openvino --model LightOnOCR-2-1B\ --task image-text-to-text --weight-format int8 ./ocr/
(also failed to when task=image-to-text)
Package version:
openvino-genai==2025.4.0
Platform & Device:
Windows 11 / I9-11900K

Feature Use Case

On-Device Document OCR

Issue submission checklist

• The feature request or improvement must be related to OpenVINO

[krgkaushik]

Hi @Cafelatte1

This is a great find. Since LightOnOCR is a very recent release, optimum-cli likely lacks the specific architecture mapping in its registry, causing the export to fail (especially for quantization which requires precise layer tracing).

To help us narrow down the integration path, could you please share:

The specific error trace: Is it failing at the load stage (unsupported architecture) or the quantization stage (NNCF graph tracing error)?

The architectures field: If you look at the config.json in the model repo, what is the value under "architectures"? (e.g., is it Qwen2VLForConditionalGeneration, LlavaForConditionalGeneration, or a custom LightOn... class?)

If it shares a backbone with an existing supported VLM (like Qwen2-VL or Florence-2), we might just need to pass the --trust-remote-code flag or alias the architecture. If it's custom, we'll need to implement a wrapper for the OpenVINO export config.

I can help investigate the config.json and propose a patch for optimum-intel if a new configuration class is needed.

[Cafelatte1]

@krgkaushik Hello!

I tried exporting the model via the CLI as in my previous post, and I can confirm that it failed.

To share the architecture details:
the architectures field in config.json is set to LightOnOCRForConditionalGeneration.

[krgkaushik]

Great catch on the architecture name. Since LightOnOCRForConditionalGeneration is a custom architecture not yet registered in Optimum, the CLI won't know how to handle it.

We need to bypass the CLI and force a manual trace using the Python API. Try running this script—it should get you the OpenVINO XML by using trust_remote_code=True and passing a dummy input:

import openvino as ov
from transformers import AutoModelForConditionalGeneration, AutoProcessor
from PIL import Image

# 1. Load with trust_remote_code
model_id = "lightonai/LightOnOCR-2-1B"
print(f"Loading {model_id}...")
model = AutoModelForConditionalGeneration.from_pretrained(model_id, torch_dtype=torch.float32, trust_remote_code=True)
model.eval()
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)

# 2. Create Dummy Input for Tracing
dummy_image = Image.new('RGB', (1000, 1000), color='white')
inputs = processor(text="Read this.", images=dummy_image, return_tensors="pt")

# 3. Convert and Save
print("Tracing model...")
ov_model = ov.convert_model(model, example_input=dict(inputs))
ov.save_model(ov_model, "lightonocr_f16.xml", compress_to_fp16=True)
print("Export complete!")```