feat: Llama index milvus rag

[aevo98765]

Description

This pull request adds a llama-index RAG implementation. This depends upon ollama local model hosting. The instructions of how to setup have been added.

Suggestions for future improvements:

• Get docling to do the chunking.
• Return sources from the search function so the retrieval sources can be displayed on the mcp client.

[mergify]

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

[PeterStaar-IBM]

Hi @aevo98765 , this is a good start. I see that you use the native markdown chunker from RAG. I think we can do better: @vagenas has been working on native chunker on the DoclingDocument and a lot of experience with llama-index. You can find a nice example here: rag_llamaindex.ipynb

1. @vagenas : Can you have a look at export_docling_document_to_vector_db
2. @aevo98765 : I dont see in the tools any ollama call. Why do we need the ollama and the llm if we dont use it in the tools?

[PeterStaar-IBM]

@aevo98765 I dont see where we use the ollama llm. Do we need to define an llm, or can we simply get away with a simple chromadb?

[aevo98765]

@PeterStaar-IBM Changed the code to use the docling native approach. I had to switch to Milvus to achieve this and store this in memory. ChromaDB wouldn't work. Ollama models are used under the hood in the LlamaIndex query engine. It is not explicit at the moment as it is set as a 'SETTING' which reads in like an environment variable for LlamaIndex to use. Hope this makes sense.

[ceberam]

• To frame this new feature within the purpose of docling-mcp: I would document it in README.md more clearly as an optional feature. We could have a separate section at the end (for instance, Applications) where we explain how docling-mcp can be leveraged in Agentic RAG applications (basically, what you wrote in terms of configuration and examples).
• In connection with above, I would move the export_docling_document_to_vector_db and search_documents tools to another module. For instance, rag.py
• In other docling repos, we leverage pydantic settings, since we think they offer some advantages (we can clearly define the options and type-hint them, validate them, leverage aliases, automatically manage nested variables...).

Later on, we can:

• enable other LLM providers (e.g., watsonx.ai)
• add more variables (e.g., chunker type and chunking options)

[codecov]

Codecov Report

Attention: Patch coverage is 20.68966% with 46 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
docling_mcp/tools/applications.py	0.00%	36 Missing ⚠️
docling_mcp/shared.py	63.15%	7 Missing ⚠️
docling_mcp/server.py	0.00%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!

[aevo98765]

Also the tests are failing. This will be because the applications milvus RAG code is conditional. We would need to pass env vars to the test pipeline to fix this. Any suggesstions or advice here?

[ceberam]

Also the tests are failing. This will be because the applications milvus RAG code is conditional. We would need to pass env vars to the test pipeline to fix this. Any suggesstions or advice here?

I have added a conditional import on commit dbde847
Since we will add more applications, I will create a new issue to add test environment variables in the test-package job of the GitHub CI/CD.

[aevo98765]

LGTM