RAG Service Stuck During Setup

[dannyfuf]

I'm just getting started with Avante and unsure whether the RAG service is already set up or needs configuration. It gets stuck on 'Starting RAG Service,' and I can't tell if it's doing anything. The container logs show no activity.

Here are my options:

opts = {
	provider = "claude",
	claude = {
		model = "claude-3-7-sonnet-20240307",
		timeout = 60000,
		temperature = 0.1,
		max_tokens = 12000,
	},
	web_search_engine = {
		provider = "google",
	},
	rag_service = {
		enabled = true,
		host_mount = "~/code",
		provider = "openai",
		endpoint = "https://api.openai.com/v1",
		llm_model = "",
		embed_model = "",
	},
}

And here are the container logs. They remain the same even after asking the chat some questions:

   FastAPI   Starting production server 🚀
 
             Searching for package file structure from directories with 
             __init__.py files
2025-03-31 05:34:24,117 - INFO - data dir: /data
2025-03-31 05:34:24,177 - INFO - Anonymized telemetry enabled. See                     https://docs.trychroma.com/telemetry for more information.
2025-03-31 05:34:24,373 - INFO - Loading all indices.
2025-03-31 05:34:24,373 - ERROR - Failed to load index from storage: No index in storage context, check if you specified the right persist_dir.
             Importing from /app/src
 
    module   🐍 main.py
 
      code   Importing the FastAPI app object from the module with the following
             code:
 
             from main import app
 
       app   Using import string: main:app
 
    server   Server started at http://0.0.0.0:20250
    server   Documentation at http://0.0.0.0:20250/docs
 
             Logs:
 
      INFO   Uvicorn running on http://0.0.0.0:20250 (Press CTRL+C to quit)
2025-03-31 05:34:24,552 - INFO - Uvicorn running on http://0.0.0.0:20250 (Press CTRL+C to quit)
      INFO   Started parent process [9]
2025-03-31 05:34:24,553 - INFO - Started parent process [9]
2025-03-31 05:34:30,024 - INFO - data dir: /data
2025-03-31 05:34:30,030 - INFO - data dir: /data
2025-03-31 05:34:30,034 - INFO - data dir: /data
2025-03-31 05:34:30,090 - INFO - Anonymized telemetry enabled. See                     https://docs.trychroma.com/telemetry for more information.
2025-03-31 05:34:30,096 - INFO - Anonymized telemetry enabled. See                     https://docs.trychroma.com/telemetry for more information.
2025-03-31 05:34:30,099 - INFO - Anonymized telemetry enabled. See                     https://docs.trychroma.com/telemetry for more information.
2025-03-31 05:34:30,362 - INFO - Loading all indices.
2025-03-31 05:34:30,362 - ERROR - Failed to load index from storage: No index in storage context, check if you specified the right persist_dir.
2025-03-31 05:34:30,363 - INFO - Loading all indices.
2025-03-31 05:34:30,363 - ERROR - Failed to load index from storage: No index in storage context, check if you specified the right persist_dir.
2025-03-31 05:34:30,367 - INFO - Loading all indices.
2025-03-31 05:34:30,367 - ERROR - Failed to load index from storage: No index in storage context, check if you specified the right persist_dir.
      INFO   Started server process [21]
2025-03-31 05:34:30,559 - INFO - Started server process [21]
      INFO   Waiting for application startup.
2025-03-31 05:34:30,562 - INFO - Waiting for application startup.
      INFO   Started server process [22]
2025-03-31 05:34:30,560 - INFO - Started server process [22]
      INFO   Waiting for application startup.
2025-03-31 05:34:30,563 - INFO - Waiting for application startup.
      INFO   Started server process [23]
2025-03-31 05:34:30,562 - INFO - Started server process [23]
2025-03-31 05:34:30,565 - INFO - Starting RAG service as leader (PID: 21)...
      INFO   Waiting for application startup.
2025-03-31 05:34:30,565 - INFO - Waiting for application startup.
      INFO   Application startup complete.
2025-03-31 05:34:30,566 - INFO - Application startup complete.
2025-03-31 05:34:30,567 - INFO - Found 0 active resources to sync
      INFO   Application startup complete.
2025-03-31 05:34:30,568 - INFO - Application startup complete.
      INFO   Application startup complete.
2025-03-31 05:34:30,569 - INFO - Application startup complete.

Any ideas on what's going on?

[jcarlos7121]

Are you on Mac OS?

[shidil]

On Mac OS I can see the same logs as above.

[jamesonBradfield]

I get the same issue on windows,
made some progress by running rustup update, and defining the HOME env var in my shell (this actually mounts the home dir), other than that I'm getting the exact behavior you are.

[edmundhighcock]

Some things to check.

AFAIK the rag service will only index your code if it is a git repo. Are you starting nvim + avante inside a git repo?

Check the mount points for your container:

$ docker inspect avante-rag-service

Here are my mount points:

        "Mounts": [                                                                                                                                                                
            {                                                                                                                                                                      
                "Type": "bind",                                                                                                                                                    
                "Source": "~/.local/share/nvim/avante/rag_service",                                                                                            
                "Destination": "/data",                                                                                                                                            
                "Mode": "",                                                                                                                                                        
                "RW": true,                                                                                                                                                        
                "Propagation": "rprivate"                                                                                                                                          
            },                                                                                                                                                                     
            {                                                                                                                                                                      
                "Type": "bind",                                                                                                                                                    
                "Source": "~/Code",                                                                                                                            
                "Destination": "/host",                                                                                                                                            
                "Mode": "ro",                                                                                                                                                      
                "RW": false,                                                                                                                                                       
                "Propagation": "rprivate"                                                                                                                                          
            }                                                                                                                                                                      
        ], 

Inside your /data mount (the first one) can you see if it has started indexing

$ ls ~/.local/share/nvim/avante/rag_service
chroma_db       leader.lock     logs            rag_config.json sqlite

Also, I don't think your config is correct... compare with the example in the README. You are missing the subgroups llm and embed.

rag_service = { -- RAG Service configuration
    enabled = false, -- Enables the RAG service
    host_mount = os.getenv("HOME"), -- Host mount path for the rag service (Docker will mount this path)
    runner = "docker", -- Runner for the RAG service (can use docker or nix)
    llm = { -- Language Model (LLM) configuration for RAG service
      provider = "openai", -- LLM provider
      endpoint = "https://api.openai.com/v1", -- LLM API endpoint
      api_key = "OPENAI_API_KEY", -- Environment variable name for the LLM API key
      model = "gpt-4o-mini", -- LLM model name
      extra = nil, -- Additional configuration options for LLM
    },
    embed = { -- Embedding model configuration for RAG service
      provider = "openai", -- Embedding provider
      endpoint = "https://api.openai.com/v1", -- Embedding API endpoint
      api_key = "OPENAI_API_KEY", -- Environment variable name for the embedding API key
      model = "text-embedding-3-large", -- Embedding model name
      extra = nil, -- Additional configuration options for the embedding model
    },
    docker_extra_args = "", -- Extra arguments to pass to the docker command
  },