Dedicated Redis Backend for Celery Tasks in LLM-Gateway

[damienlaine]

Summary: Integrate a dedicated Redis instance as the backend for managing Celery tasks within the LLM-Gateway. This Redis service should be configured as a sidecar container in Docker Compose deployments and included in the LinTO Deployment Tool. We need to further analyze alternatives (In memory local celery tasks)

Details:

• Dedicated Redis Backend:

Configure LLM-Gateway to utilize its own Redis instance for handling Celery tasks, ensuring efficient task queuing and result storage.

• Sidecar Deployment:

Implement the Redis service as a sidecar container within Docker Compose configurations, promoting modularity and ease of management.

• LinTO Deployment Tool Integration:

Incorporate the new Redis service into the LinTO Deployment Tool to streamline setup and deployment processes.