Performance considerations regarding PDF backend

[mara004]

Re-reading the code in io/pdf.py, I've got a performance question concerning your implementation.
You're rendering multiple PDF pages using pypdfium2's concurrent PdfDocument.render_topil() method. However, instead of yielding the results incrementally, you accumulate all pixmaps in a list and return that. This has the following disadvantages:

• Increased memory load (all image data is present in memory at the same time). Problematic for long documents or high resolution rendering.
• Time is wasted. You could already start analysing pages as they are released while pypdfium2 is still operating its process pool.

So unless you actually need to have all pages available at the same time, this should be changed to

    with pdfium.PdfDocument(file, password=password) as pdf:
        for img in pdf.render_topil(scale=scale, **kwargs):
            yield np.asarray(img)

and code that uses read_pdf() may need to be adapted to assume an iterator rather than a list. For instance, DocumentFile.read_pdf() would need to do yield from read_pdf(...) instead of return read_pdf(...)

[frgfm]

Hi @mara004 👋

Always a pleasure to see you around here :)
Yes I agree, it would be an improvement. Here are my thoughts about this:

• while training, PDF are prerendered to avoid computation bottlenecks on CPU
• during inference though, before sending the batch through the model, the rendered page will have to be resized, then converted to a PyTorch/TF tensor. The whole process can be improved, and it's a good idea to start at the beginning.
• either way, for other open uses of the PDF rendering feature, I agree that we could investigate this option. If so, I'd argue we should document the difference in memory and latency between the two methods 👍

What do you think? You'd like to open a PR or do you prefer that we take care of this?

Take care!

[mara004]

Thanks for your reply!
I'm not familiar with the doctr codebase and have some more private projects on the way, so I'd prefer if you could take care of this. I merely wondered about the code and wanted to ask.
The first point sounds particularly interesting, but admittedly I don't fully understand yet. Can you explain why pre-rendering the whole file constitutes a performance advantage?
Perhaps you should define some kind of limit or batch size, though? The problem is that there are quite long PDFs around, and if a better resolution is selected, I imagine that memory usage might increase too steeply.

[frgfm]

Yes sure, I'll ping you for reviews 👍 Would you mind opening a dedicated issue? (so that we can track this for next release)

The first point sounds particularly interesting, but admittedly I don't fully understand yet. Can you explain why pre-rendering the whole file constitutes a performance advantage?

Of course: during training, the data reading & preprocessing is performed on CPU in most ML projects. This can be a bottleneck and leave your GPU idle while the data is getting read. The page rendering is part of this preprocessing & reading process. So instead of reading & rendering on the fly, we added an option so that all pages are prerendered and saved locally (to be accessed later on). This considerably boosts the training speed

Perhaps you should define some kind of limit or batch size in any case, though? The problem is that there are quite long PDFs around, and if a better resolution is selected, I imagine that memory usage might increase too steeply.

In which context? training again? I agree but generally, those are not 100 pages PDFs and deep learning hardware is kind of beefy in terms of RAM so this isn't a limitation most of the time. But still, that's why I think your suggestion is quite interesting :)

[mara004]

Would you mind opening a dedicated issue?

Perhaps it would be better if you did that? Then you could directly add some more information on the doctr internals in question. For now, I wouldn't know what else to write apart from what I've already mentioned here (essentially I could only reference this thread).

In which context? training again?

I still don't know enough about your processing pipeline, so maybe I'm subject to some delusion here, but I was thinking about something like doing multiple cycles of render & train with a given batch size – say, 25 pages per cycle, for instance. Could this avoid the bottleneck while keeping a limit on RAM usage?

so that all pages are prerendered and saved locally (to be accessed later on)

Does this mean you are already disk-caching rendered pixmaps somehow? If so, what about saving directly on rendering and then only reading images into RAM on demand?

I agree but generally, those are not 100 pages PDFs and deep learning hardware is kind of beefy in terms of RAM so this isn't a limitation most of the time.

Yeah, if you didn't have any problems so far, then probably this isn't so relevant. Nonetheless, I believe in general it'd be good to write hardware-agnostic code, with smaller RAM in mind.

[frgfm]

Sure about the GH issue 👍

Could this avoid the bottleneck while keeping a limit on RAM usage?

Not sure I fully understand but you mean avoiding to keeping in RAM all rendered pages? (Your previous point about yield)

If so, what about saving directly on rendering and then only reading images into RAM on demand?

That's exactly what we intend to do when the dataset is made of PDFs :)

Nonetheless, I believe in general it'd be good to write hardware-agnostic code, with smaller RAM in mind.

I fully agree 👍

[mara004]

Not sure I fully understand but you mean avoiding to keeping in RAM all rendered pages? (Your previous point about yield)

I meant, if you can't process each page as it arrives because of the bottleneck, maybe it would work to accumulate a certain amount of pages and thus process long documents in multiple batches to achieve a compromise between performance and RAM usage?
But as I said, I hardly looked at any doctr code except io/pdf.py and might have misunderstood something.
I'd say, let's just wait until someone starts a Pull Request – seeing this in code might make matters easier.

[mara004]

FYI, I just received a user report confirming this problem: pypdfium2-team/pypdfium2#141

[felixdittrich92]

Thanks @mara004 for sharing this 👍