feat: kotaemon as service

[phv2312]

Description

• Provide convinient method to call kotaemon without gradio App

Type of change

• New features (non-breaking change).
• Bug fix (non-breaking change).
• Breaking change (fix or feature that would cause existing functionality not to work as expected).

Checklist

• I have performed a self-review of my code.
• I have added thorough tests if it is a core feature.
• There is a reference to the original bug report and related work.
• I have commented on my code, particularly in hard-to-understand areas.
• The feature is well documented.

[phv2312]

How to use:

from kotaemon.deps.container import Container
from ktem.index.file.pipelines import DocumentRetrievalPipeline, IndexDocumentPipeline

"""
1. Indexing
"""
indexing_pipeline: IndexDocumentPipeline = Container.indexings.get("default").get()
print(indexing_pipeline)
stream = indexing_pipeline.stream(
    ["<your_filepath>"],
    reindex=True
)
for response in stream:
    print(response)


"""
2. Retriever
"""
crud = Container.crud
docids = crud.list_docids()

retriever_pipeline: DocumentRetrievalPipeline = Container.retrievers.get("default").get()
print(retriever_pipeline)

responses= retriever_pipeline.run(
    text="What is the main content?",
    doc_ids=docids
)
for response in responses:
    print(response.id_)
    print(response.text)