Some feedback and requests

[gabriel-wainmann]

Hi Docling team. This is nothing less than wonderful. Thank you.

Running this on a two-page scanned pdf, I get these errors:

1. The first and last lines of the entire PDF are not OCRed at all.
2. Rows merged across columns in a table get repeated for each column.

Please, can you add other OCR options to choose from, such as easyocr and tesseract?

[dolfim-ibm]

Hi @gabriel-wainmann, I think that what you see is the (current) expected behavior. For both your points, I assume you are referring to the markdown output, correct?

1. When Docling detects page headers and footers, those are removed from the markdown, because they are not part of the "natural text flow". The content should anyway be there in the JSON output.
2. Merge columns headers (spanning multiple columns) don't have a native representation in markdown, this is why we, on-purpose, replicate the header for all columns it belongs to. The output is a regular 2d grid which can easily be iterated on.
   • On the other hand, the JSON format contains details about which cells are merged together.
   • We are working on an example which exports the tables in HTML and Pandas Dataframes, where these relationships will be represented correctly.

[aeamaea]

We are working on an example which exports the tables in HTML and Pandas Dataframes, where these relationships will be represented correctly.

@dolfim-ibm That would be amazing! I would love to be able to export tables with each table identifiable by it's header or something so I can iterate on them in the JSON or the intermediate format in the dataframe.

[dolfim-ibm]

@gabriel-wainmann @aeamaea This was just finalized. You can find an example in export_tables.py.

[gabriel-wainmann]

This is wonderful. Thank you so much

…

On Mon, 9 Sept 2024, 16:12 Michele Dolfi, ***@***.***> wrote: Hi @gabriel-wainmann <https://github.com/gabriel-wainmann>, I think that what you see is the (current) expected behavior. For both your points, I assume you are referring to the markdown output, correct? 1. When Docling detects page headers and footers, those are removed from the markdown, because they are not part of the "natural text flow". The content should anyway be there in the JSON output. 2. Merge columns headers (spanning multiple columns) don't have a native representation in markdown, this is why we, on-purpose, replicate the header for all columns it belongs to. The output is a regular 2d grid which can easily be iterated on. - On the other hand, the JSON format contains details about which cells are merged together. - We are working on an example which exports the tables in HTML and Pandas Dataframes, where these relationships will be represented correctly. — Reply to this email directly, view it on GitHub <#64 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOTQV4NTJJVMJGJCT2O7JUTZVU33XAVCNFSM6AAAAABN3N46ASVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANJYGY4TCOA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[gabriel-wainmann]

Hi @dolfim-ibm.
Would you please share Docling code to export to JSON instead of Markup?
Couldn't find it.

[dolfim-ibm]

You could start from this example https://github.com/DS4SD/docling/blob/main/examples/custom_convert.py

[gabriel-wainmann]

Thanks Gabriel Wainmann [image: LinkedIn Professional Profile] <http://www.linkedin.com/in/gabrielwainmann/>

…

On Mon, 9 Sept 2024 at 19:39, Michele Dolfi ***@***.***> wrote: You could start from this example https://github.com/DS4SD/docling/blob/main/examples/custom_convert.py — Reply to this email directly, view it on GitHub <#64 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOTQV4NYU5243XOIR73EDUTZVVUEBAVCNFSM6AAAAABN3N46ASVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANJYHA4TCMA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Sahil-v05]

How to install docling in cpu.Actually after installing docling through pip , it show error docling not found while importing

[dolfim-ibm]

@Sahil-v05 can you please share a few more details about the error you get?

[aeamaea]

How to install docling in cpu.Actually after installing docling through pip , it show error docling not found while importing

Hello @Sahil-v05 , I had this type of error and I realized I was not activating the right conda environment. Mentioning it just in case.

[oedemis]

Idea: Add more Integration to LlamaIndex/Langchain like llamaparse

[dolfim-ibm]

we actually have them ready, and we are about to push them upstream.

the actual object which is needed is the DoclingPDFReader in https://github.com/DS4SD/docling/blob/main/examples/rag_llamaindex.ipynb.

[oedemis]

In llamaparse there is for example tracked the page_label, and extracted via prompt latet. how to adjust this in the example?