feat: Initial multi-agent scenario (#11)

details: 
- Simple multi-agent workflow.
 - Removed LlamaIndex and switched to Pydantic AI.
 - Added OAuth authentication for the web frontend when exposed using ngrok using a traffic policy.
 - Added live streaming in command line mode.

Author: anirbanbasu

README.md

@@ -19,6 +19,7 @@ However, DQA is experimental with the emphasis on standardising agentic communic
 ## Installation
 
 - Install [`uv` package manager](https://docs.astral.sh/uv/getting-started/installation/).
+- Install self-hosted ([with Docker](https://docs.prefect.io/v3/how-to-guides/self-hosted/server-docker)) a Prefect server by running `docker run -p 4200:4200 -d --restart unless-stopped --name prefect prefecthq/prefect:3-latest -- prefect server start --host 0.0.0.0`. _This is only necessary for [Durable Agents](https://ai.pydantic.dev/durable_execution/overview/) connecting to a self-hosted Prefect server_. Note that Durable Agents support is experimental and may be dropped in the future.
 - Install project dependencies by running `uv sync --all-groups`.
 - Configure Dapr to run [with docker](https://docs.dapr.io/operations/hosting/self-hosted/self-hosted-with-docker/).
 - Run `dapr init` to initialise `daprd` and the relevant containers.
@@ -49,12 +50,15 @@ The following API keys are optional but maybe provided for additional functional
 The following environment variables are all optional.
  - `APP_LOG_LEVEL`: The general log level of the DQA app. Defaults to `INFO`.
  - `DQA_MCP_SERVER_TRANSPORT`, `FASTMCP_HOST` and `FASTMCP_PORT`: These specify the transport type, the host and port for the built-in MCP server of DQA. The default values are `stdio`, `localhost` and `8000` respectively.
+ - `HTTPX_TIMEOUT`: This specifies the timeout value in seconds a HTTP client will wait for the server to respond before raising an error. This applies, for example, to the communication timeout between the web frontend or CLI frontend to the A2A endpoint(s). Default value is 120 (seconds).
  - `LLM_CONFIG_FILE` and `MCP_CONFIG_FILE`: These specify where the LLM and MCP configurations These default to `conf/llm.json` and `conf/mcp.json` respectively.
- - [Gradio environment variables](https://www.gradio.app/guides/environment-variables) to configure the DQA web app. However, MCP server (not to be confused with DQA's built-in MCP server), SSR mode, API and public sharing will be disabled, irrespective of what is specified through the environment variables.
+ - [Gradio environment variables](https://www.gradio.app/guides/environment-variables) to configure the DQA web app. However, MCP server (not to be confused with DQA's built-in MCP server), server-side rendering (SSR) mode, API, Progressive Web App (PWA), analytics and public sharing will be disabled, irrespective of what is specified through the environment variables.
+ - `PREFECT_API_URL`: This can be used to specify the Prefect Cloud API URL (in which case, you must set the `PREFECT_API_KEY`, see details) or the local self-hosted API URL at `http://localhost:4200/api`. The default value is None, which _will turn off Durable Agents_!
  - `BROWSER_STATE_SECRET`: This is the secret used by Gradio to encrypt the browser state data. The default value is `a2a_dapr_bstate_secret`.
  - `BROWSER_STATE_CHAT_HISTORIES`: This is the key in browser state used by Gradio to store the chat histories (local values). The default value is `a2a_dapr_chat_histories`.
  - `APP_DAPR_SVC_HOST` and `APP_DAPR_SVC_PORT`: The host and port at which Dapr actor service will listen on. These default to `127.0.0.1` and `32768`. Should you change these, you must change the corresponding information in `dapr.yaml`.
  - `APP_DAPR_PUBSUB_STALE_MSG_SECS`: This specifies how old a message should be on the Dapr publish-subscribe topic queue before it will be considered too old, and dropped. The default value is 60 seconds.
+ - `APP_DAPR_PUBSUB_MEMORY_STREAM_BUFFER_SIZE`: This specifies the internal memory object stream buffer size that is used to receive messages through the Dapr publish-subscribe subscription and pass it on to the A2A executor. Default value is 65536.
  - `APP_DAPR_ACTOR_RETRY_ATTEMPTS`: This specifies the number of times an agent executor will try to invoke a method on an actor, if it fails to succeed. The default value is 3.
  - `DAPR_PUBSUB_NAME`: The configured name of the publish-subscribe component at `.dapr/components/pubsub.yaml`. Change this environment variable only if you change the corresponding pub-sub component configuration.
  - `APP_A2A_SRV_HOST` and `APP_MHQA_A2A_SRV_PORT`: The host and port at which A2A endpoint will be available. These default to `127.0.0.1` and `32770`. Should you change these, you must change the corresponding information in `dapr.yaml`.
@@ -66,6 +70,10 @@ The following environment variables are all optional.
 - Invoke the A2A agent using JSON-RPC by calling `uv run dqa-cli --help` to learn about the various skills-based A2A endpoint invocations.
 - Or, start the Gradio web app by running `uv run dqa-web-app` and then browse to http://localhost:7860.
 - Once done, stop the dapr sidecars by running `./stop_dapr_multi.sh`.
+- Alternatively, you could also run `./unified_dapr_webapp.sh` to run the Dapr sidecars as well as the web app, which will be available at http://localhost:7860. Press Ctrl+C to abort the server and the unified runner script will also shutdown the Dapr sidecars.
+- Further to exposing the web app on localhost, you could also call `./run_ngrok.sh` (which requires you to have `ngrok` setup, [see instructions](https://ngrok.com/download/)) with an optional parameter `--domain your-ngrok-FQDN` to make your app available through `ngrok` publicly at https://your-ngrok-FQDN/.
+  - Note that a feature of exposing the app over `ngrok` is that the configured traffic policy for `ngrok` will require any user accessing the app to authenticate themselves using an OAuth provider (GitHub). This is necessary to transparently create a user namespace such that two users concurrently each naming a chat ID as `test-chat` (for instance) will _not_ have a chat ID collision because each chat will be effectively handled by a different actor ID in the namespace based on the underlying OAuth provider supplied user ID. Thus, `user1__test-chat` is different from `user2__test-chat`.
+  - Also note that the same OAuth-authenticated user using more than one separate browsers will _not_ have the same list of chat IDs visible on each browser. This is because the chat IDs list is local to each browser. However, the user will be able to load a specific chat ID on one browser in another browser by manually adding that chat ID. Since those chat IDs are in their OAuth-authenticated user namespaces, an authenticated user will only be able to add their own chat ID, not someone else's.
 
 ## Tests and coverage
 

conf/llm.json

@@ -1,8 +1,10 @@
 {
-  "ollama": {
-    "base_url": "http://localhost:11434",
-    "model": "gpt-oss:20b-cloud",
-    "request_timeout": 300,
-    "thinking": false
+  "responder": {
+    "base_url": "http://localhost:11434/v1",
+    "model": "gpt-oss:20b-cloud"
+  },
+  "reviewer": {
+    "base_url": "http://localhost:11434/v1",
+    "model": "gpt-oss:20b-cloud"
   }
 }

conf/mcp.json

@@ -1,14 +1,16 @@
 {
-  "builtin": {
-    "transport": "stdio",
-    "command": "uv",
-    "args": [
-      "run",
-      "dqa-mcp"
-    ],
-    "env": {
-      "APP_LOG_LEVEL": "CRITICAL",
-      "DQA_MCP_SERVER_TRANSPORT": "stdio"
+  "mcpServers": {
+    "builtin": {
+      "transport": "stdio",
+      "command": "uv",
+      "args": [
+        "run",
+        "dqa-mcp"
+      ],
+      "env": {
+        "APP_LOG_LEVEL": "CRITICAL",
+        "DQA_MCP_SERVER_TRANSPORT": "stdio"
+      }
     }
   }
 }

dapr.yaml

@@ -19,12 +19,6 @@ apps:
     maxBodySize: 256Mi
     # appLogDestination: file # (optional), can be file, console or fileAndConsole. default is fileAndConsole.
     # daprdLogDestination: file # (optional), can be file, console or fileAndConsole. default is file.
-  - appID: echo-a2a-srv # optional
-    appDirPath: . # REQUIRED
-    appChannelAddress: 127.0.0.1
-    command: ["uv", "run", "echo-a2a-srv"]
-    readBufferSize: 32Ki
-    maxBodySize: 256Mi
   - appID: mhqa-a2a-srv # optional
     appDirPath: . # REQUIRED
     appChannelAddress: 127.0.0.1

ngrok_traffic_policy.yml

@@ -0,0 +1,14 @@
+# See: https://ngrok.com/blog/block-threats-waf-actions -- this is a paid feature
+on_http_request:
+  - actions:
+      - type: oauth
+        config:
+          provider: github
+          idle_session_timeout: 15m
+          max_session_duration: 1h
+  - actions:
+      - type: add-headers
+        config:
+          headers:
+            x-auth-user-id: "${actions.ngrok.oauth.identity.provider_user_id}"
+            x-auth-user-name: "${actions.ngrok.oauth.identity.name}"

pyproject.toml

@@ -6,7 +6,7 @@ readme = "README.md"
 authors = [
     { name = "Anirban Basu", email = "anirbanbasu@users.noreply.github.com" }
 ]
-requires-python = ">=3.12.0"
+requires-python = ">=3.12.0,<3.14"
 dependencies = [
     "a2a-sdk[http-server]>=0.3.6",
     "anyio>=4.11.0",
@@ -16,22 +16,22 @@ dependencies = [
     "fastmcp>=2.12.4",
     "frankfurtermcp>=0.3.6",
     "gradio>=5.46.1",
-    "llama-index-llms-ollama>=0.7.4",
-    "llama-index-tools-mcp>=0.4.1",
     "ollama>=0.6.0",
+    "pydantic-ai-slim[openai,fastmcp,prefect]>=1.7.0",
+    "pydantic-graph>=1.7.0",
     "typer>=0.19.1",
     "yfmcp>=0.4.8",
 ]
 
 [project.scripts]
 pad = "dqa:main"
 dapr-srv = "dqa.server.dapr:main"
-echo-a2a-srv = "dqa.server.echo_a2a:main"
 mhqa-a2a-srv = "dqa.server.mhqa_a2a:main"
 dqa-mcp = "dqa.mcp.primary:main"
 dqa-cli = "dqa.cli.a2a:main"
 dqa-web-app = "dqa.web.gradio:main"
 demo-client = "dqa.cli.demo:main"
+test-ma = "dqa.agent_workflow.wf_orchestrator:main"
 
 [build-system]
 requires = ["uv_build>=0.8.15,<0.9.0"]

run_ngrok.sh

@@ -0,0 +1,4 @@
+#!/bin/bash
+# Add a domain such as --domain your-custom-domain.ngrok-free.app
+# Add --traffic-policy-file ngrok_traffic_policy.yml if your ngrok plan supports it
+ngrok http 7860 --host-header='localhost:7860' --name 'dqa-ngrok' --log-format 'json' --traffic-policy-file ngrok_traffic_policy.yml $@

run_upgrade.sh

@@ -1,3 +1,3 @@
 #!/bin/bash
 # Upgrade the dependencies including those from Git sources
-uv lock -U
+uv lock -U --prerelease=allow

src/dqa/__init__.py

@@ -1,9 +1,10 @@
 import logging
+import re
 from typing import ClassVar
 from rich.logging import RichHandler
 from environs import Env
 
-from marshmallow.validate import OneOf
+from marshmallow.validate import OneOf, Range, Regexp
 
 try:
     from icecream import ic
@@ -17,46 +18,83 @@
 env.read_env()
 
 
-class ParsedEnvVars:
+class EnvVars:
     APP_LOG_LEVEL: str = env.str("APP_LOG_LEVEL", default="INFO").upper()
+    HTTPX_TIMEOUT: float = env.float(
+        "HTTPX_TIMEOUT",
+        default=120.0,
+        validate=Range(min=5.0, max=600.0),
+    )
     DQA_MCP_SERVER_TRANSPORT: str = env.str(
         "DQA_MCP_SERVER_TRANSPORT",
         default="stdio",
         validate=OneOf(["stdio", "sse", "streamable-http"]),
     )
     FASTMCP_HOST: str = env.str("FASTMCP_HOST", default="localhost")
     FASTMCP_PORT: int = env.int("FASTMCP_PORT", default=8000)
+
     LLM_CONFIG_FILE: str = env.str("LLM_CONFIG_FILE", default="conf/llm.json")
     MCP_CONFIG_FILE: str = env.str("MCP_CONFIG_FILE", default="conf/mcp.json")
     APP_DAPR_SVC_HOST: str = env.str("APP_DAPR_SVC_HOST", default="127.0.0.1")
-    APP_DAPR_SVC_PORT: int = env.int("APP_DAPR_SVC_PORT", default=32768)
+    APP_DAPR_SVC_PORT: int = env.int(
+        "APP_DAPR_SVC_PORT",
+        default=32768,
+        validate=Range(min=4096, max=65535),
+    )
     APP_DAPR_PUBSUB_STALE_MSG_SECS: int = env.int(
-        "APP_DAPR_PUBSUB_STALE_MSG_SECS", default=60
+        "APP_DAPR_PUBSUB_STALE_MSG_SECS",
+        default=60,
+        validate=Range(min=5, max=3600),
     )
     APP_DAPR_ACTOR_RETRY_ATTEMPTS: int = env.int(
-        "APP_DAPR_ACTOR_RETRY_ATTEMPTS", default=3
+        "APP_DAPR_ACTOR_RETRY_ATTEMPTS",
+        default=3,
+        validate=Range(min=1, max=10),
+    )
+    AGENT_RETRY_ATTEMPTS: int = env.int(
+        "AGENT_RETRY_ATTEMPTS",
+        default=3,
+        validate=Range(min=1, max=10),
     )
     APP_A2A_SRV_HOST: str = env.str("APP_A2A_SRV_HOST", default="127.0.0.1")
-    APP_MHQA_A2A_SRV_PORT: int = env.int("APP_MHQA_A2A_SRV_PORT", default=32770)
+    APP_MHQA_A2A_SRV_PORT: int = env.int(
+        "APP_MHQA_A2A_SRV_PORT",
+        default=32770,
+        validate=Range(min=4096, max=65535),
+    )
     APP_MHQA_A2A_REMOTE_URL: str = env.str("APP_MHQA_A2A_REMOTE_URL", default=None)
-    APP_ECHO_A2A_SRV_PORT: int = env.int("APP_ECHO_A2A_SRV_PORT", default=32769)
     DAPR_PUBSUB_NAME: str = env.str("DAPR_PUBSUB_NAME", default="pubsub")
+    APP_DAPR_PUBSUB_MEMORY_STREAM_BUFFER_SIZE: int = env.int(
+        "APP_DAPR_PUBSUB_MEMORY_STREAM_BUFFER_SIZE",
+        default=65536,
+        validate=Range(min=32768, max=(2**31 - 1)),
+    )
     MCP_SERVER_HOST: str = env.str("FASTMCP_HOST", default="localhost")
     MCP_SERVER_PORT: int = env.int("FASTMCP_PORT", default=8000)
     BROWSER_STATE_SECRET: str = env.str(
-        "BROWSER_STATE_SECRET", default="a2a_dapr_bstate_secret"
+        "BROWSER_STATE_SECRET",
+        default="a2a_dapr_bstate_secret",
+        validate=Regexp(
+            r"^[A-Za-z0-9]{1,8}(?:_[A-Za-z0-9]{1,8}){1,4}$", flags=re.IGNORECASE
+        ),
     )
     BROWSER_STATE_CHAT_HISTORIES: str = env.str(
-        "BROWSER_STATE_CHAT_HISTORIES", default="a2a_dapr_chat_histories"
+        "BROWSER_STATE_CHAT_HISTORIES",
+        default="a2a_dapr_chat_histories",
+        validate=Regexp(
+            r"^[A-Za-z0-9]{1,8}(?:_[A-Za-z0-9]{1,8}){1,4}$", flags=re.IGNORECASE
+        ),
     )
 
+    PREFECT_API_URL: str = env.str("PREFECT_API_URL", default=None)
+
     API_KEY_ALPHAVANTAGE: str = env.str("ALPHAVANTAGE_API_KEY", default=None)
     API_KEY_TAVILY: str = env.str("TAVILY_API_KEY", default=None)
     API_KEY_OLLAMA: str = env.str("OLLAMA_API_KEY", default=None)
 
     _instance: ClassVar = None
 
-    def __new__(cls: type["ParsedEnvVars"]) -> "ParsedEnvVars":
+    def __new__(cls: type["EnvVars"]) -> "EnvVars":
         if cls._instance is None:
             # Create instance using super().__new__ to bypass any recursion
             instance = super().__new__(cls)
@@ -65,7 +103,7 @@ def __new__(cls: type["ParsedEnvVars"]) -> "ParsedEnvVars":
 
 
 logging.basicConfig(
-    level=ParsedEnvVars().APP_LOG_LEVEL,
+    level=EnvVars.APP_LOG_LEVEL,
     format="%(message)s",
     datefmt="[%X]",
     handlers=[RichHandler(show_time=True, show_level=True, show_path=True)],