Query Regarding GPU Acceleration Support

[alexmao1024]

Question

Hi docling Team,

I have a question regarding GPU acceleration in RapidOCR. I've been looking through the documentation but couldn't find explicit details on how to enable GPU support.

My question is: Does RapidOCR support GPU acceleration?

If it does, is the correct procedure to install a GPU-compatible backend like rapidocr_paddle and then configure the AcceleratorOptions as shown below?

# ...

accelerator_options = AcceleratorOptions(
    num_threads=8, 
    device=AcceleratorDevice.GPU
)

I am asking because I don't have a local machine with a GPU to test this configuration myself, but our production environment does have GPU resources. I would like to confirm the correct setup before deploying.

Thank you for your time and for the great work on this project!

[dosubot]

Hi @alexmao1024! I'm Dosu and I’m helping the docling team.

RapidOCR does support GPU acceleration, but enabling it depends on the backend you use—most commonly, rapidocr_paddle. To use GPU acceleration, you need to:

1. Install a GPU-compatible backend (like rapidocr_paddle) that supports GPU.
2. Ensure your environment has the necessary NVIDIA drivers and CUDA libraries.
3. Deploy using a CUDA-enabled Docker image if you're running in containers (for example, ghcr.io/docling-project/docling-serve-cu124), and configure Docker to expose the GPU to the container (deployment details, compose example).

The device selection (CPU/GPU) is handled by the backend and the deployment environment. There is no evidence that AcceleratorOptions or AcceleratorDevice are used in the RapidOCR integration for device selection—GPU usage is determined by the backend and whether the container or environment has GPU access. If the backend supports GPU and the environment is set up correctly, GPU acceleration will be used automatically if available. Your code snippet using AcceleratorOptions is not required for enabling GPU support in this context—just ensure the backend and environment are configured for GPU use.

If you want to verify GPU usage, check that the GPU is visible inside your container or environment using nvidia-smi and that your backend is the GPU-enabled version. For more, see the deployment guide.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[alexmao1024]

@dosu
I attempted to use RapidOCR with only the rapidocr_paddle package installed, but I encountered an error message stating that rapidocr_onnxruntime is required.

The error was:
RapidOCR is not installed. Please install it via 'pip install rapidocr_onnxruntime' to use this OCR engine. Alternatively, Docling has support for other OCR engines. See the documentation.

This suggests that onnxruntime is mandatory, but the problem is that it lacks GPU support.

[dosubot]

You're correct—Docling's RapidOCR integration is currently hardcoded to require the rapidocr_onnxruntime package. Even if you install rapidocr_paddle, Docling will not use it, and you'll see the error you described. There is no built-in way to use rapidocr_paddle (which supports GPU acceleration) as an alternative backend in Docling right now; the code directly imports RapidOCR from rapidocr_onnxruntime and raises an error if it's missing, matching your experience (source).

This means GPU acceleration via rapidocr_paddle is not currently possible in Docling without code changes to add support for that backend. If you need GPU-accelerated OCR, you'll need to use one of the other engines that support it, or wait for/pursue a feature update to add rapidocr_paddle support.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}