docs/examples/vlm_pipeline_api_model.py does not work after checkout

[yogesh-n]

Question

I tried the example code in - docs/examples/vlm_pipeline_api_model.py
I checked out the repo and ran this file, while having a local instance of 'granite3.2-vision:2b' model running via ollama.

I tried both options of the ollama url - In the function 'ollama_vlm_options()' I tried both the below -
url="http://localhost:11434/v1/chat/completions", # the default Ollama endpoint
url="http://localhost:11434/api/generate",

However on running I get the below error - Can anyone help with regards to why the code is not working as is out of the box. Thanks very much in advance.

2025-11-27 12:35:47,404 - INFO - detected formats: [<InputFormat.PDF: 'pdf'>]
2025-11-27 12:35:47,448 - INFO - Going to convert document batch...
2025-11-27 12:35:47,448 - INFO - Initializing pipeline for VlmPipeline with options hash a42b1a61978945f220a4d714a54545d9
2025-11-27 12:35:47,476 - INFO - Loading plugin 'docling_defaults'
2025-11-27 12:35:47,479 - INFO - Registered picture descriptions: ['vlm', 'api']
2025-11-27 12:35:47,479 - INFO - Processing document 2305.03393v1-pg9.pdf
2025-11-27 12:35:49,811 - WARNING - Encountered an error during conversion of document a07f5c34601ba2c234d898cbfaa9e29a7045996ccd82ccab3012516220a1f3a4:
Traceback (most recent call last):

File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\pipeline\base_pipeline.py", line 237, in _build_document
for p in pipeline_pages: # Must exhaust!

File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\pipeline\base_pipeline.py", line 202, in _apply_on_pages
yield from page_batch

File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\models\api_vlm_model.py", line 87, in call
predictions = list(self.process_images(images, prompts))

File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\models\api_vlm_model.py", line 180, in process_images
yield from executor.map(_process_single_image, zip(images, prompts))

File "c:\Users\ynars\miniconda3\envs\py310\lib\concurrent\futures_base.py", line 621, in result_iterator
yield _result_or_cancel(fs.pop())

File "c:\Users\ynars\miniconda3\envs\py310\lib\concurrent\futures_base.py", line 319, in _result_or_cancel
return fut.result(timeout)

File "c:\Users\ynars\miniconda3\envs\py310\lib\concurrent\futures_base.py", line 458, in result
return self.__get_result()

File "c:\Users\ynars\miniconda3\envs\py310\lib\concurrent\futures_base.py", line 403, in __get_result
raise self._exception

File "c:\Users\ynars\miniconda3\envs\py310\lib\concurrent\futures\thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)

File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\models\api_vlm_model.py", line 161, in _process_single_image
page_tags, num_tokens, stop_reason = api_image_request(

File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\utils\api_image_request.py", line 61, in api_image_request
api_resp = OpenAiApiResponse.model_validate_json(r.text)

File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\pydantic\main.py", line 766, in model_validate_json
return cls.pydantic_validator.validate_json(

pydantic_core._pydantic_core.ValidationError: 4 validation errors for OpenAiApiResponse
id
Field required [type=missing, input_value={'model': 'granite3.2-vis..., 'done_reason': 'load'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.12/v/missing
choices
Field required [type=missing, input_value={'model': 'granite3.2-vis..., 'done_reason': 'load'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.12/v/missing
created
Field required [type=missing, input_value={'model': 'granite3.2-vis..., 'done_reason': 'load'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.12/v/missing
usage
Field required [type=missing, input_value={'model': 'granite3.2-vis..., 'done_reason': 'load'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.12/v/missing

Traceback (most recent call last):
File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\pipeline\base_pipeline.py", line 72, in execute
conv_res = self._build_document(conv_res)
File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\pipeline\base_pipeline.py", line 277, in _build_document
raise e
File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\pipeline\base_pipeline.py", line 237, in _build_document
for p in pipeline_pages: # Must exhaust!
File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\pipeline\base_pipeline.py", line 202, in _apply_on_pages
yield from page_batch
File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\models\api_vlm_model.py", line 87, in call
predictions = list(self.process_images(images, prompts))
File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\models\api_vlm_model.py", line 180, in process_images
yield from executor.map(_process_single_image, zip(images, prompts))
File "c:\Users\ynars\miniconda3\envs\py310\lib\concurrent\futures_base.py", line 621, in result_iterator
yield _result_or_cancel(fs.pop())
File "c:\Users\ynars\miniconda3\envs\py310\lib\concurrent\futures_base.py", line 319, in _result_or_cancel
return fut.result(timeout)
File "c:\Users\ynars\miniconda3\envs\py310\lib\concurrent\futures_base.py", line 458, in result
return self.__get_result()
File "c:\Users\ynars\miniconda3\envs\py310\lib\concurrent\futures_base.py", line 403, in __get_result
raise self._exception
File "c:\Users\ynars\miniconda3\envs\py310\lib\concurrent\futures\thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\models\api_vlm_model.py", line 161, in _process_single_image
page_tags, num_tokens, stop_reason = api_image_request(
File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\utils\api_image_request.py", line 61, in api_image_request
api_resp = OpenAiApiResponse.model_validate_json(r.text)
File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\pydantic\main.py", line 766, in model_validate_json
return cls.pydantic_validator.validate_json(
pydantic_core._pydantic_core.ValidationError: 4 validation errors for OpenAiApiResponse
id
Field required [type=missing, input_value={'model': 'granite3.2-vis..., 'done_reason': 'load'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.12/v/missing
choices
Field required [type=missing, input_value={'model': 'granite3.2-vis..., 'done_reason': 'load'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.12/v/missing
created
Field required [type=missing, input_value={'model': 'granite3.2-vis..., 'done_reason': 'load'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.12/v/missing
usage
Field required [type=missing, input_value={'model': 'granite3.2-vis..., 'done_reason': 'load'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.12/v/missing

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "c:\Users\ynars\miniconda3\envs\py310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\Users\ynars\miniconda3\envs\py310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "c:\Users\ynars.vscode\extensions\ms-python.debugpy-2025.16.0-win32-x64\bundled\libs\debugpy_main.py", line 71, in
cli.main()
File "c:\Users\ynars.vscode\extensions\ms-python.debugpy-2025.16.0-win32-x64\bundled\libs\debugpy/..\debugpy\server\cli.py", line 508, in main
run()
File "c:\Users\ynars.vscode\extensions\ms-python.debugpy-2025.16.0-win32-x64\bundled\libs\debugpy/..\debugpy\server\cli.py", line 358, in run_file
runpy.run_path(target, run_name="main")
File "c:\Users\ynars.vscode\extensions\ms-python.debugpy-2025.16.0-win32-x64\bundled\libs\debugpy_vendored\pydevd_pydevd_bundle\pydevd_runpy.py", line 310, in run_path
return _run_module_code(code, init_globals, run_name, pkg_name=pkg_name, script_name=fname)
File "c:\Users\ynars.vscode\extensions\ms-python.debugpy-2025.16.0-win32-x64\bundled\libs\debugpy_vendored\pydevd_pydevd_bundle\pydevd_runpy.py", line 127, in _run_module_code
_run_code(code, mod_globals, init_globals, mod_name, mod_spec, pkg_name, script_name)
File "c:\Users\ynars.vscode\extensions\ms-python.debugpy-2025.16.0-win32-x64\bundled\libs\debugpy_vendored\pydevd_pydevd_bundle\pydevd_runpy.py", line 118, in _run_code
exec(code, run_globals)
File "c:\Users\ynars\Documents\Professional-Projs\docling_src\docling\docs\examples\vlm_pipeline_api_model.py", line 282, in
main()
File "c:\Users\ynars\Documents\Professional-Projs\docling_src\docling\docs\examples\vlm_pipeline_api_model.py", line 277, in main
result = doc_converter.convert(input_doc_path)
File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\pydantic_internal_validate_call.py", line 39, in wrapper_function
return wrapper(*args, **kwargs)
File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\pydantic_internal_validate_call.py", line 136, in call
res = self.pydantic_validator.validate_python(pydantic_core.ArgsKwargs(args, kwargs))
File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\document_converter.py", line 264, in convert
return next(all_res)
File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\document_converter.py", line 287, in convert_all
for conv_res in conv_res_iter:
File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\document_converter.py", line 363, in _convert
for item in map(
File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\document_converter.py", line 410, in _process_document
conv_res = self._execute_pipeline(in_doc, raises_on_error=raises_on_error)
File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\document_converter.py", line 433, in _execute_pipeline
conv_res = pipeline.execute(in_doc, raises_on_error=raises_on_error)
File "c:\Users\ynars\miniconda3\envs\py310\lib\site-packages\docling\pipeline\base_pipeline.py", line 87, in execute
raise RuntimeError(f"Pipeline {self.class.name} failed") from e
RuntimeError: Pipeline VlmPipeline failed

[dosubot]

Hi @yogesh-n! I'm Dosu and I’m helping the docling team.

The error happens because Ollama's API response doesn't match the OpenAI schema that Docling expects—it's missing required fields like id, choices, created, and usage. Docling's VLM pipeline is tightly coupled to the OpenAI response format and doesn't currently support alternative schemas out of the box, so any backend (like Ollama) that doesn't return these fields will cause a validation error and pipeline failure (source).

To make this work, you would need to patch the api_image_request function to either preprocess Ollama's response into an OpenAI-like dict before validation, or bypass the strict Pydantic validation and extract the fields you need manually. For example, you could check if the response is missing those fields and, if so, parse out the relevant content and token counts directly from Ollama's schema (source).

There is no built-in workaround or config toggle for this—code changes are required. If you want a concrete example of how to patch api_image_request for Ollama, let me know!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}