Conversation
WalkthroughThis update introduces a comprehensive overhaul and expansion of HolmesGPT's documentation. It adds new structured documentation files for installation, AI provider configuration, built-in and custom toolsets, development, evaluation, and troubleshooting. The update also introduces a MkDocs configuration for building and deploying the documentation site, replaces local links with hosted URLs, and removes outdated or redundant documentation. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Docs Site (MkDocs)
participant GitHub Actions
participant HolmesGPT
User->>Docs Site (MkDocs): Access documentation via hosted URLs
User->>Docs Site (MkDocs): Follow installation/configuration guides
User->>HolmesGPT: Install/configure HolmesGPT as per docs
GitHub Actions->>Docs Site (MkDocs): Build and deploy docs on master branch push
Docs Site (MkDocs)-->>User: Serve up-to-date documentation
Possibly related PRs
Suggested labels
Suggested reviewers
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 14
🔭 Outside diff range comments (1)
docs/development/evals/writing.md (1)
208-209: YAML key uses hyphen, examples above use underscore – pick oneEarlier examples use
before_test/after_test; this snippet switches tobefore-test/after-test. The test harness reads the underscore version today – the dash form will be silently ignored.Either rename here or document that both are supported.
🧹 Nitpick comments (70)
docs/snippets/enabled_by_default.md (1)
1-2: Minor wording tweak for readability (optional).Breaking the long second sentence makes the admonition easier to scan.
- This toolset is enabled by default. You do not need to configure it. You can disable it if you want to but doing so may negatively impact HolmesGPT's ability to investigate issues. + This toolset is enabled by default – no additional configuration required. + You can disable it if you really need to, but doing so may reduce HolmesGPT's ability to investigate issues.docs/snippets/toolset_configuration_helm.md (1)
1-10: YAML indentation is Helm-friendly, but add a placeholder prefix (optional).Readers may copy-paste; adding
...aboveholmes:prevents Helm from reading an incomplete fragment verbatim.- ```yaml - holmes: + ```yaml + # values.yaml + ... + holmes:docs/snippets/helm_upgrade_command.md (1)
1-4: Include--install& quote the placeholder for safer copy-pasteUsers often copy the snippet verbatim. Adding
--installprevents failures on first-time installs, and wrapping<YOUR_CLUSTER_NAME>in quotes avoids issues with cluster names that contain dots or dashes.-helm upgrade robusta robusta/robusta --values=generated_values.yaml --set clusterName=<YOUR_CLUSTER_NAME> +helm upgrade --install robusta robusta/robusta \ + --values=generated_values.yaml \ + --set clusterName="<YOUR_CLUSTER_NAME>"docs/snippets/disable_default_logging_toolset.md (1)
1-2: Minor wording tweak for clarity“datasource” → “data source” is the spelling used elsewhere in the docs.
- a different datasource for logs + a different data source for logsdocs/snippets/toolset_configuration_cli.md (1)
5-11: Align YAML indentation with 2-space conventionMost examples in the repo use 2-space indents; keeping it consistent improves readability.
- ```yaml - toolsets: - TOOLSET_NAME: - enabled: true - config: - CONFIGURATION_OPTIONS - ``` +```yaml +toolsets: + TOOLSET_NAME: + enabled: true + config: + CONFIGURATION_OPTIONS +``` </blockquote></details> <details> <summary>README.md (5)</summary><blockquote> `65-66`: **Minor wording & link context** > “For extra privacy, **[bring an API key]**” Consider clarifying “bring **your own** API key” – some readers mis-parse *bring an*. No functional change needed. --- `90-93`: **Grammar nit – “follow” → “following”** > “one of the **follow** three methods” ```diff -You can install HolmesGPT in one of the follow three methods: +You can install HolmesGPT in one of the following three methods:
101-131: Repeated hard-coded docs links – extract a variable?A dozen identical
https://robusta-dev.github.io/holmesgpt/installation/cli-installation/strings appear here.
If this URL changes, every instance must be edited. MkDocs supports variables (e.g.{{ base_url }}) or you can use relative paths (/installation/cli-installation/), reducing future churn.
185-221: Alt-text consistencyMany logos now link to the hosted docs but still use
alt="OpenAI"etc. A few (e.g., Azure) include file names. Align allaltvalues with provider names for accessibility-tool parity.
231-237: Sentence reads awkwardly“Also supported: … setup an LLM API key and ask Holmes a question”
“set up” is the verb; “setup” is the noun.
-setup an LLM API key +set up an LLM API keydocs/development/evals/reporting.md (2)
5-6: Prefer link aliases over raw.mdMkDocs generates nice directory URLs. Linking to
index.md/writing.mdbypasses that and emits warnings in--strictmode. Use:- [Evaluations Overview](../) - [Writing Evaluations](writing/)
12-14: Grammar“how Holmes perform over time” → “how Holmes performs over time”.
docs/snippets/toolset_configuration_tabs.md (2)
1-12: Code block placeholders could mislead copy-pastesUsing bare
TOOLSET_NAME/CONFIGURATION_OPTIONSwithout back-ticks makes it easy for users to forget replacement. Consider wrapping them in angle brackets or adding an explicit note.toolsets: <toolset_name>: enabled: true config: <option>: <value>
24-28: Helm upgrade command omits--namespaceMost users install Robusta in a dedicated namespace (docs default to
robusta). Add--namespace robusta(or a variable) to avoid upgrading the wrong release..github/workflows/build-and-deploy-docs.yaml (2)
1-8: Trigger only onmaster– considermainor PR previewsIf the default branch ever switches to
main, docs will silently stop deploying. You can future-proof with:push: branches: [ master, main ]Optionally add
pull_request: { paths: ['docs/**', 'mkdocs.yml'] }to validate builds in PRs.
20-24: Cache key is fine – but pip cache is missingYou cache MkDocs’ tarball downloads, but pip redownloads
mkdocs-materialevery run. Adding:- uses: actions/cache@v4 with: path: ~/.cache/pip key: pip-${{ hashFiles('**/requirements-docs.txt') }}will cut minutes off CI.
docs/development/evals/writing.md (2)
55-58: Fix typos & unclear sentence in expectations section
preentis miss-spelled and the sentence “HolmesGPT's answer is score 0 is any …” is grammatically broken, making the evaluation criteria hard to follow.- This `expected_output` will be compared against HolmesGPT's answer and evaluated by a LLM ('LLM as judge'). The resulting score is called `correctness` and is a binary score with a value of either `0` or `1`. HolmesGPT's answer is score `0` is any of the expected element is not present in the answer, `1` if all expected elements are preent in the answer. + HolmesGPT’s answer is evaluated by an LLM (“LLM-as-judge”) against the `expected_output`. + The result – `correctness` – is binary: + * `1` if **all** expected elements are present in the answer + * `0` if **any** expected element is missing
279-284: Shell snippets: add-sonly when needed & avoid repetition
pytest -v -s …is great for debugging, but-sstreams stdout/stderr which slows CI runs. Maybe add a note: “omit-sin CI for faster output”.Minor, feel free to ignore.
docs/ai-providers/ollama.md (1)
18-27: Duplicate command block – consider removing oneBlocks at lines 18-19 and 25-27 show the exact same invocation. Perhaps keep only one and move the second under a “Quick example” caption to avoid vertical scroll.
docs/ai-providers/openai.md (1)
30-35: Clarify “mini” model caveat“Results are not as accurate” is subjective. Maybe phrase as “may be less accurate for complex queries” or link to OpenAI’s own guidance.
docs/index.md (1)
5-5: Add alt text for accessibilityThe header GIF lacks
alttext. Screen-reader users will miss context.- +docs/data-sources/builtin-toolsets/internet.md (1)
1-1: Remove the check-mark from the heading for consistency across toolset docsOther built-in toolset pages use plain titles like “# Coralogix logs”, “# Slab”, etc. The lone “✓” in “# Internet ✓” introduces an inconsistent naming pattern and can break generated anchors.
Consider dropping the glyph:-# Internet ✓ +# Internetdocs/snippets/toolset_config_template.md (1)
8-14: Clarify placeholder syntax to prevent copy-paste errors
TOOLSET_PATHandCUSTOM_CONFIGare currently bare identifiers. Readers may mistake them for literal keys. Wrapping them in angle brackets or back-ticks improves affordance:- toolsets: - TOOLSET_PATH: + toolsets: + <TOOLSET_PATH>: ... - CUSTOM_CONFIG + <CUSTOM_CONFIG>docs/data-sources/builtin-toolsets/slab.md (1)
41-43: Polish the sampleholmescommandMinor wording + typo: “crashloopbackoff” ➜ “CrashLoopBackOff”, and splitting the sentence improves readability.
-holmes ask "Why is my pod failing, if it's a crashloopbackoff use the runbooks from Slab" +holmes ask "Why is my pod failing? If it's in a CrashLoopBackOff state, use the runbooks from Slab."docs/data-sources/index.md (1)
1-3: Minor phrasing improvement“connects to your monitoring and observability tools” → “connects to monitoring and observability tools” (drop the second your for cleaner docs tone).
docs/data-sources/builtin-toolsets/datetime.md (2)
15-20: YAML sample: indent with 2 spaces for consistencyHolmesGPT docs use 2-space indents everywhere else. The snippet here is 4-space. Consider:
-holmes: - toolsets: - datetime: - enabled: true +holmes: + toolsets: + datetime: + enabled: true
24-26: Table column width spills on mobileLong description text (“Return current time information…”) wraps oddly on narrow screens. Shorten or split for responsiveness (MkDocs Material auto-wrap is limited inside tables).
docs/ai-providers/azure-openai.md (1)
24-25: Avoid passing secrets on the command lineShell history will retain
--api-key. Recommend env-var or stdin prompt instead; at least add a security note.docs/data-sources/builtin-toolsets/opensearch-status.md (2)
18-30: Examples mix list-stylehostsformatsFirst snippet lists plain host strings, later snippets use objects
{ host, port }. Make a note that both are valid per opensearch-py, or standardise to one style to avoid confusion.hosts: - host: host1.com port: 9200
60-68: Highlight secret management best practicesPlain-text
OPENSEARCH_BEARER_TOKENenv var is fine for local dev, but add a warning box recommending Kubernetes secrets or a password manager for production.docs/ai-providers/google-vertex-ai.md (1)
25-27:--api-keyflag may mislead users of service-account JSONVertex AI auth typically relies on a JSON key file, not a short API key string. If HolmesGPT really expects the full JSON via
--api-key, clarify this explicitly (size, quoting, security caveats) or rename the flag (e.g.--credentials-file) to avoid confusion.docs/data-sources/builtin-toolsets/docker.md (1)
6-6: Turn the loose sentence into a clear “Prerequisites” section-Read access to Docker resources. +### Prerequisites + +Ensure the user running HolmesGPT has read-only access to the Docker daemon +(e.g. by being in the `docker` group or using a rootless socket).docs/data-sources/builtin-toolsets/helm.md (1)
1-1: Heading suffix✓breaks some MkDocs slug generatorsA few Markdown renderers strip or URL-encode the check-mark, producing awkward anchor links.
Consider dropping the symbol for portability.docs/development/index.md (1)
44-47: Duplicate link – drop one occurrence
Contributing Guidelinesis linked twice (lines 7–8 and again here). Keeping both adds noise without benefit.-1. **[Contributing Guidelines](https://github.com/robusta-dev/holmesgpt/blob/master/CONTRIBUTING.md){:target="_blank"}** - Understand the development process -2. **[Evaluations Overview](evals/index.md)** - Learn about the evaluation framework +1. **[Evaluations Overview](evals/index.md)** - Learn about the evaluation frameworkdocs/data-sources/builtin-toolsets/opensearch-logs.md (2)
14-30: YAML fences trigger MD046 – switch to indented blocks or disable the rule
markdownlintcomplains because fenced blocks are used instead of indented ones. Either:
- Convert fenced YAML to an indented block, or
- Add a rule override in
.markdownlint.yaml/front-matter to disableMD046.Consistency with other docs is key; pick one style and apply it repo-wide.
42-45: Use neutral placeholders – avoid leaking random hostnames
skdjasid.europe-west1.gcp.cloud.es.iois clearly a throw-away value but can confuse readers.- opensearch_url: https://skdjasid.europe-west1.gcp.cloud.es.io:443 # The URL to your opensearch cluster. + opensearch_url: https://<your-opensearch-cluster>:443 # The URL to your OpenSearch clusterdocs/data-sources/builtin-toolsets/robusta.md (1)
1-1: Header slug contains a Unicode check-mark – breaks predictable anchors
# Robusta ✓produces an anchor likerobusta-✓, which is hard to type and link to. Drop the symbol for cleaner URLs.-# Robusta ✓ +# Robustadocs/data-sources/builtin-toolsets/confluence.md (1)
13-17: Add a secret-based alternative for shell examples.The Bash snippet only shows exporting credentials directly. Including a short comment (as you did in the Helm section) demonstrating how to source these from a secret keeps parity with the Helm example and discourages leaking API keys in shell history.
docs/data-sources/builtin-toolsets/rabbitmq.md (2)
5-9: Minor grammar & clarity fix.
- “detect partition” → “detect partitions”
- “The nodes and partitioning status is obtained” → “The node and partition status are obtained”
-By enabling this toolset, HolmesGPT will be able to detect RabbitMQ partitions, memory alerts, and disk alerts and suggest mitigations. - -This toolset follows a two-step process to detect partition: +By enabling this toolset, HolmesGPT can detect RabbitMQ partitions, memory alerts, and disk alerts, then suggest mitigations. + +This toolset follows a two-step process to detect partitions: @@ -1. The nodes and partitioning status is obtained by fetching information from the configured `management_url`. +1. Node and partition status are obtained by fetching information from the configured `management_url`.
51-60: Consider silencing MD046 or switching to indented code blocks.
markdownlint-cli2flags fenced blocks inside a fenced block list (MD046). If the rule is enforced in CI you can either:
- Disable MD046 for this file (
<!-- markdownlint-disable MD046 -->)- Replace the fenced YAML with an indented block.
No action needed if MD046 isn’t part of required checks.
docs/installation/kubernetes-installation.md (1)
112-124: Shell snippet: quote JSON for portability.Some shells misinterpret single quotes inside the JSON. Safer to use a here-document or escape quotes:
curl -X POST http://localhost:8080/api/chat \ -H 'Content-Type: application/json' \ -d '{"ask":"list pods in namespace default?"}'docs/data-sources/builtin-toolsets/aks-node-health.md (1)
50-64: Avoid duplicating unchanged YAML in the advanced-config exampleLines 55-62 repeat the entire basic configuration and only add two extra keys (
health_check_interval,max_unhealthy_nodes). Re-listing all fields obscures what actually changed and increases maintenance cost if defaults evolve.Diff showing a more concise “patch-style” example:
toolsets: aks/node-health: enabled: true config: - subscription_id: "<your Azure subscription ID>" - resource_group: "<your AKS resource group>" - cluster_name: "<your AKS cluster name>" + # …previous mandatory keys … + health_check_interval: 300 # seconds + max_unhealthy_nodes: 3 # thresholdThat makes the delta crystal-clear and reduces copy-paste churn.
docs/data-sources/builtin-toolsets/index.md (1)
9-36: Consider alphabetical ordering for quicker scanningThe grid currently mixes cloud, observability and misc toolsets in a semi-random order. Sorting the bullet list alphabetically (or grouping by domain) would let users locate a toolset instantly without reading every line. It’s a one-line win for UX.
docs/data-sources/builtin-toolsets/grafanaloki.md (2)
39-56: Markdown-lint MD046 noise – switch to indented code blocks inside tab sets
markdownlint-cli2flags these fenced blocks (```yaml) because they sit inside an indented “tabs” section. Converting them to indented code blocks (four-space prefix, no back-ticks) silences MD046 and matches Material-for-MkDocs’ recommendation:-```yaml -toolsets: - grafana/loki: - enabled: true - config: - api_key: <your grafana API key> - … -``` + toolsets: + grafana/loki: + enabled: true + config: + api_key: <your grafana API key> + …Same applies to the Helm and advanced-config tabs further down.
70-72: Redundant comment – move to central snippetThe “HolmesGPT’s default logging mechanism MUST be disabled” comment is repeated four times in this file. Consider extracting it to a single include (e.g.
snippets/disable_default_logging.md) just like you already do fortoolset_refresh_warning.md.docs/data-sources/builtin-toolsets/github.md (1)
52-64: Same MD046 warning as Loki docThe fenced YAML block inside the advanced-config section causes
markdownlintnoise. Use an indented block or disable MD046 for this repo.-```yaml -toolsets: - git/github: - enabled: true - config: - default_branch: "main" - max_file_size: 1048576 - timeout: 30 -``` + toolsets: + git/github: + enabled: true + config: + default_branch: "main" + max_file_size: 1_048_576 # 1 MB + timeout: 30docs/data-sources/builtin-toolsets/aws.md (1)
43-43: Prefer “who” when referring to people
... along with the users that called themReplace “that” with “who” for grammatical correctness.
-| aws_cloudtrail_event_lookup | Fetches events of a specified type from AWS CloudTrail along with the users that called them | +| aws_cloudtrail_event_lookup | Fetches events of a specified type from AWS CloudTrail along with the users who called them |docs/walkthrough/index.md (1)
56-66: Tighten the wording of the outcome sectionThe bullets mix informal wording (“it's stuck”, “Saved investigation time”) with the otherwise formal tone of the docs. Consider:
- Holmes will analyze the pod, identify that it's stuck in "Pending" state due to an invalid node selector, and suggest specific remediation steps - all without you needing to manually run `kubectl describe`, check events, or dig through logs. + Holmes analyzes the pod, identifies that it is stuck in the “Pending” state because of an invalid node selector, and suggests concrete remediation steps — without any manual `kubectl describe`, event inspection, or log digging.docs/ai-providers/openai-compatible.md (1)
22-24: Port mismatch between quick-start and LocalAI exampleThe quick-start sets
OPENAI_API_BASEtolocalhost:8000, whereas the LocalAI example below runs on port 8080. New users may assume 8000 is always correct and hit a connection error.Either change the quick-start to 8080 or add an explicit note that the port must match the server you run.
docs/data-sources/remote-mcp-servers.md (2)
49-49: Spelling: “supergateway”
supergatwayis missing an “e”.-Check out supergatway docs to find out other useful flags. +Check out Supergateway docs to discover additional useful flags.
54-56: Add alt text for accessibilityThe embedded GIF lacks alt text, which hurts accessibility and violates MD045.
-<img style="max-width:300px;" src="https://cdn.loom.com/sessions/thumbnails/1b290511b79942c7b1d672a2a4cde105-ed4eed3f9d70b125-full-play.gif"> +<img style="max-width:300px;" src="https://cdn.loom.com/sessions/thumbnails/1b290511b79942c7b1d672a2a4cde105-ed4eed3f9d70b125-full-play.gif" alt="Demo of running Dynatrace MCP via Supergateway">docs/data-sources/builtin-toolsets/kubernetes.md (1)
12-17: YAML indentation deviates from other examplesMost docs indent two spaces under
holmes:; this snippet uses four. Standardising indentation aids copy-paste and avoids Helm lint warnings in strict mode.holmes: - toolsets: - kubernetes/core: - enabled: true + toolsets: + kubernetes/core: + enabled: truedocs/installation/ui-installation.md (1)
17-24: Update internal link to be MkDocs-friendlyMkDocs converts “pretty” links only when they start with
/or are relative to the file’s parent path.
[CLI Installation Guide](cli-installation.md)works locally but will break once the docs are served from/installation/ui-installation/because the generated URL becomes/installation/ui-installation/cli-installation/.Add one leading
../so the link resolves to/installation/cli-installation/in both local and published builds.-1. **Install HolmesGPT CLI and set API key** - Follow the [CLI Installation Guide](cli-installation.md) … +1. **Install HolmesGPT CLI and set API key** - Follow the [CLI Installation Guide](../installation/cli-installation.md) …docs/data-sources/builtin-toolsets/datadog.md (1)
61-68: Fix Markdown-lint MD046 violation inside tabbed block
markdownlintflags fenced code-blocks inside tabbed content; indenting them by four spaces satisfies the rule and keeps the rendering identical.-```bash -holmes ask "show me recent application errors" -``` + ```bash + holmes ask "show me recent application errors" + ```docs/installation/cli-installation.md (1)
31-37: Pin pipx install to a release tag instead ofmaster.zipInstalling from a moving HEAD breaks reproducibility and prevents caching by pipx.
-pipx install "https://github.com/robusta-dev/holmesgpt/archive/refs/heads/master.zip" +pipx install "https://github.com/robusta-dev/holmesgpt/archive/refs/tags/v<latest>.zip"Replace
<latest>with the current release tag whenever you cut a new version.docs/data-sources/builtin-toolsets/grafanatempo.md (1)
18-26: Use proper headings instead of bold text
markdownlintflags**Getting Grafana URL**/**Obtaining the datasource UID**as emphasis-as-heading.
Converting to###headings improves accessibility and ToC generation.-**Getting Grafana URL** +### Getting Grafana URLRepeat for the Obtaining the datasource UID section.
docs/README.md (1)
70-81: Add language identifier to directory-tree code blockExplicit language hints silence markdown-lint (MD040) and improve syntax highlighting.
-``` +```text docs/ ├── index.md # Homepage ... └── assets/ # Images and static files -``` +```docs/installation/python-installation.md (1)
66-75: Avoid hard-coding dummy API keys in docsReplacing
your-api-key-herewith an environment-variable example discourages accidental commits of real keys and demonstrates best practice.- api_key = os.getenv("OPENAI_API_KEY", "your-api-key-here") + # TIP: prefer environment variables over literals + api_key = os.getenv("OPENAI_API_KEY")docs/data-sources/builtin-toolsets/prometheus.md (1)
68-76: Use proper sub-headings instead of bold paragraphs
MD036is triggered by using emphasis for headings. Converting Option 1 / Option 2 to####keeps navigation consistent and avoids linter noise.-**Option 1: Simple method (port-forwarding)** +#### Option 1: Simple method (port-forwarding) ... -**Option 2: Advanced method (get full cluster DNS URL)** +#### Option 2: Advanced method (get full cluster DNS URL)docs/data-sources/builtin-toolsets/azure-sql.md (1)
94-108: Add language identifiers to ASCII-diagram code blocksThese fenced blocks miss a language spec (
MD040). Usetextto suppress warnings while preserving formatting.-``` +```text Azure Level (RBAC): ├── Monitoring Reader (subscription) ├── SQL DB Contributor (resource group) ... -``` +``` ```diff -``` +```text Database Level (SQL permissions): ├── CREATE USER [holmes-service-principal] FROM EXTERNAL PROVIDER ... -``` +```docs/reference/troubleshooting.md (1)
25-29: Label error-output code block as textAdding
textremovesMD040and avoids mis-highlighting.-``` +```text pods is forbidden: User "system:serviceaccount:default:holmesgpt" cannot get resource "pods" -``` +```docs/reference/helm-configuration.md (2)
33-46: Remove duplicated “Toolset Configuration” sectionThe Toolset Configuration snippet appears twice (lines 33-46 and again at 130-146). Keeping a single, canonical example avoids bloating the page and eliminates the risk that the two diverge over time.
Also applies to: 130-146
19-24: Grammar / wording nit“Enabled/disable/customize specific toolsets” → “Enable/disable/customize specific toolsets”.
docs/data-sources/builtin-toolsets/mongodb-atlas.md (1)
84-90: Minor naming consistencyAll other built-in toolsets follow
<provider>/<topic>naming conventions (e.g.prometheus/metrics).
Consider renaming tools fromatlas_return_*tomongodb_atlas/…for uniformity.
Not blocking, but consistency helps future discoverability.docs/development/evals/index.md (1)
76-78: Wording tweak for conciseness“A lot of cases” → “many cases” reads better.
- In a lot of cases this can cause the eval to fail + In many cases this can cause the eval to faildocs/data-sources/custom-toolsets.md (1)
345-363: markdownlint MD046 – fenced vs indented code blocksProject markdown elsewhere consistently uses fenced blocks, so ignoring MD046 is acceptable.
If you want a clean lint run, disable rule MD046 or switch these three YAML/Dockerfile examples to indented blocks.Also applies to: 398-412, 416-429
docs/data-sources/builtin-toolsets/newrelic.md (4)
5-6: Verify snippet include path to avoid MkDocs build warningsPast builds raised warnings whenever an
--8<--include pointed at a non-existent file. Double-check thatsnippets/toolsets_that_provide_logging.mdis present indocs/snippets/(or adjust the relative path) so the generated site doesn’t emit “file not found” warnings.
18-30: Avoid hard-coding secrets in examplesStoring the API key directly in
~/.holmes/config.yamlencourages insecure practices. Consider showing an environment-variable based example instead, e.g.:- nr_api_key: "<your New Relic API key>" + nr_api_key: "${NEWRELIC_API_KEY}"…and note that users can
export NEWRELIC_API_KEY=...before running Holmes.
36-47: YAML indentation looks off under the Helm tabBecause the code block is already indented four spaces (inside the tab block), every line in the YAML now has two different indentation levels, which can be pasted incorrectly. Remove the leading four spaces inside the fenced block:
- holmes: - toolsets: - newrelic: +holmes: + toolsets: + newrelic:This keeps the example copy-pasteable without manual fixes.
52-54: Elaborate on input parameters for each capabilityReaders may not know what “duration threshold” means in
newrelic_get_traces. Add a brief column (or footnote) specifying required/optional parameters (e.g.service,min_duration_ms,trace_id). This aligns with other toolset docs.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
docs/reference/helm-configuration.md (1)
8-9: Fix 404 – usemainin GitHub URLThe repo’s default branch is
main, so links that hard-codeblob/master/…404.
Update both occurrences toblob/main/….-https://github.com/robusta-dev/holmesgpt/blob/master/helm/holmes/values.yaml +https://github.com/robusta-dev/holmesgpt/blob/main/helm/holmes/values.yamldocs/reference/http-api.md (1)
21-34: Replace bold text with real headings (MD036)Using
**Example**/**Example Response**as pseudo-headings violates markdownlint rule MD036 and breaks anchor generation. Convert to proper####headings here and for every other occurrence (lines 33, 68, 83, 115, 130, 179, 196, 230, 241, 277, 294, 312, 317).-**Example** +#### Example ... -**Example Response** +#### Example Response
🧹 Nitpick comments (2)
docs/reference/helm-configuration.md (2)
33-41: Minor wording – use the imperative “Enable/disable/customize”Nit: “Enabled/disable/customize specific toolsets” mixes tenses.
Rephrase for clarity.-# Enabled/disable/customize specific toolsets +# Enable/disable/customize specific toolsets
130-146: Duplicate “Toolset Configuration” sectionLines 130-146 repeat the toolset block already shown at 86-98 without adding new information. Consider removing one copy to keep the doc concise.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
README.md(8 hunks)docs/ai-providers/aws-bedrock.md(1 hunks)docs/ai-providers/azure-openai.md(1 hunks)docs/installation/cli-installation.md(1 hunks)docs/reference/helm-configuration.md(1 hunks)docs/reference/http-api.md(18 hunks)
✅ Files skipped from review due to trivial changes (1)
- docs/installation/cli-installation.md
🚧 Files skipped from review as they are similar to previous changes (3)
- docs/ai-providers/azure-openai.md
- README.md
- docs/ai-providers/aws-bedrock.md
🧰 Additional context used
🧠 Learnings (3)
📓 Common learnings
Learnt from: nherment
PR: robusta-dev/holmesgpt#436
File: tests/llm/utils/mock_utils.py:240-249
Timestamp: 2025-06-05T12:23:27.634Z
Learning: The holmesgpt project uses Python >= 3.10 and prefers modern type hint syntax like `list[str]`, `dict[str, int]` over importing equivalent types from the typing module like `List[str]`, `Dict[str, int]`.
docs/reference/helm-configuration.md (2)
Learnt from: Sheeproid
PR: robusta-dev/holmesgpt#586
File: tests/llm/fixtures/test_ask_holmes/03_what_is_the_command_to_port_forward/test_case.yaml:4-4
Timestamp: 2025-07-02T10:27:17.231Z
Learning: In LLM-as-judge test cases for HolmesGPT, expected outputs should be descriptive rather than prescriptive when testing for flexible responses like port numbers. Using specific values in expected outputs can cause unnecessary test failures when the AI generates different but equally valid responses.
Learnt from: nherment
PR: robusta-dev/holmesgpt#610
File: .github/workflows/llm-evaluation.yaml:39-42
Timestamp: 2025-07-08T08:45:41.069Z
Learning: The robusta-dev/holmesgpt codebase has comprehensive existing validation for Azure environment variables (AZURE_API_BASE, AZURE_API_KEY, AZURE_API_VERSION) and MODEL in tests/llm/utils/classifiers.py, tests/llm/conftest.py, and holmes/core/llm.py. Don't suggest adding redundant validation logic.
docs/reference/http-api.md (1)
Learnt from: Sheeproid
PR: robusta-dev/holmesgpt#586
File: tests/llm/fixtures/test_ask_holmes/03_what_is_the_command_to_port_forward/test_case.yaml:4-4
Timestamp: 2025-07-02T10:27:17.231Z
Learning: In LLM-as-judge test cases for HolmesGPT, expected outputs should be descriptive rather than prescriptive when testing for flexible responses like port numbers. Using specific values in expected outputs can cause unnecessary test failures when the AI generates different but equally valid responses.
🪛 LanguageTool
docs/reference/helm-configuration.md
[style] ~48-~48: Consider shortening or rephrasing this to strengthen your wording.
Context: ...bled: true ... ``` > Note: After making changes to your configuration, run `holmes toolset...
(MAKE_CHANGES)
🪛 markdownlint-cli2 (0.17.2)
docs/reference/http-api.md
21-21: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
33-33: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
68-68: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
115-115: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
179-179: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
230-230: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
277-277: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
312-312: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: llm_evals
- GitHub Check: Pre-commit checks
- GitHub Check: Pre-commit checks
github-project-automation
bot
moved this from In progress
to Done
in DEPRECATED - HolmesGPT Roadmap
2 participants
https://robusta-dev.github.io/holmesgpt/ <- See the docs here