Skip to content

ROB-1267 unified tool logs api kubernetes override #432

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: rob-1267_unified_tool_logs_api
Choose a base branch
from

Conversation

nherment
Copy link
Collaborator

@nherment nherment commented May 20, 2025

Summary by CodeRabbit

  • New Features

    • Added a utility to filter out default logging toolsets, improving toolset management.
  • Refactor

    • Improved internal structure by relocating and reorganizing tool and toolset utilities for better maintainability.
  • Tests

    • Introduced new unit and integration tests for toolset filtering and OpenSearch logs functionality, enhancing test coverage.
  • Chores

    • Updated import paths throughout the codebase for consistency and clarity.

Copy link
Contributor

coderabbitai bot commented May 20, 2025

Walkthrough

This change refactors import paths throughout the codebase, moving most tool-related imports to more specific submodules such as holmes.core.tools.tools, holmes.core.tools.tool_executor, and holmes.core.tools.tools_utils. The ToolExecutor class is relocated to its own module, and a new utility function for filtering logging toolsets is introduced. Related tests are updated and expanded accordingly.

Changes

File(s) Change Summary
holmes/core/tools/tools.py Removed the ToolExecutor class and its logic from this module.
holmes/core/tools/tool_executor.py Introduced new ToolExecutor class with methods for tool invocation, tool lookup, and OpenAI-format listing. Handles toolset filtering and logging.
holmes/core/tools/toolset_utils.py Added filter_out_default_logging_toolset utility function to manage enabled logging toolsets, preferring non-default ones.
holmes/core/tools/logging_api.py, holmes/core/tool_calling_llm.py, holmes/config.py, examples/custom_llm.py, holmes/plugins/toolsets/__init__.py, holmes/plugins/toolsets/coralogix/api.py, holmes/plugins/toolsets/coralogix/toolset_coralogix_logs.py, holmes/plugins/toolsets/datadog.py, holmes/plugins/toolsets/datetime.py, holmes/plugins/toolsets/git.py, holmes/plugins/toolsets/grafana/base_grafana_toolset.py, holmes/plugins/toolsets/grafana/common.py, holmes/plugins/toolsets/grafana/toolset_grafana.py, holmes/plugins/toolsets/grafana/toolset_grafana_loki.py, holmes/plugins/toolsets/grafana/toolset_grafana_tempo.py, holmes/plugins/toolsets/grafana/trace_parser.py, holmes/plugins/toolsets/internet/internet.py, holmes/plugins/toolsets/internet/notion.py, holmes/plugins/toolsets/kafka.py, holmes/plugins/toolsets/kubernetes_logs.py, holmes/plugins/toolsets/newrelic.py, holmes/plugins/toolsets/opensearch/opensearch.py, holmes/plugins/toolsets/opensearch/opensearch_logs.py, holmes/plugins/toolsets/opensearch/opensearch_traces.py, holmes/plugins/toolsets/opensearch/opensearch_utils.py, holmes/plugins/toolsets/prometheus/prometheus.py, holmes/plugins/toolsets/rabbitmq/toolset_rabbitmq.py, holmes/plugins/toolsets/robusta/robusta.py, holmes/utils/holmes_sync_toolsets.py Updated import paths for tool and toolset entities to use holmes.core.tools.tools or related submodules for better modularity.
tests/core/tools/test_toolset_utils.py New test file with unit tests for filter_out_default_logging_toolset, using dummy toolset classes to verify filtering logic.
tests/plugins/toolsets/opensearch/test_opensearch_logs_integration.py New integration test file for OpenSearch logs toolset, including fixtures and tests for health check, basic query, search term, and date range filtering.
tests/llm/test_ask_holmes.py, tests/llm/test_investigate.py, tests/llm/test_mocks.py, tests/llm/utils/mock_toolset.py, tests/llm/utils/mock_utils.py, tests/plugins/prompt/test_fetch_logs.py, tests/plugins/prompt/test_generic_ask_conversation.py, tests/plugins/prompt/test_toolsets_instructions.py, tests/plugins/toolsets/grafana/test_grafana_loki.py, tests/plugins/toolsets/grafana/test_grafana_tempo.py, tests/plugins/toolsets/kubernetes/test_kubernetes_logs.py, tests/plugins/toolsets/kubernetes/test_kubernetes_logs_filter_by_timestamp.py, tests/plugins/toolsets/opensearch/test_opensearch_integration.py, tests/plugins/toolsets/test_internet.py, tests/plugins/toolsets/test_notion.py, tests/plugins/toolsets/test_prometheus_integration.py, tests/plugins/toolsets/test_tool_kafka.py, tests/plugins/toolsets/test_toolset_utils.py, tests/test_holmes_sync_toolsets.py, tests/test_structured_toolcall_result.py Updated test imports to match new module structure for tools, toolsets, and related enums/classes.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant ToolExecutor
    participant Toolset(s)
    participant Tool

    User->>ToolExecutor: invoke(tool_name, params)
    ToolExecutor->>Tool: get_tool_by_name(tool_name)
    alt Tool found
        ToolExecutor->>Tool: invoke(params)
        Tool-->>ToolExecutor: StructuredToolResult
        ToolExecutor-->>User: StructuredToolResult
    else Tool not found
        ToolExecutor-->>User: StructuredToolResult (error)
    end
Loading
sequenceDiagram
    participant System
    participant Toolset(s)
    participant ToolsetUtils

    System->>ToolsetUtils: filter_out_default_logging_toolset(toolsets)
    ToolsetUtils->>Toolset(s): check enabled status and names
    ToolsetUtils-->>System: filtered toolsets (at most one logging toolset, prefer non-default)
Loading

Suggested reviewers

  • moshemorad
✨ Finishing Touches
  • 📝 Generate Docstrings

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🔭 Outside diff range comments (1)
holmes/core/tools/toolset_utils.py (1)

5-54: 🛠️ Refactor suggestion

Replace debug print statements with proper logging.

The function contains several print statements that appear to be for debugging purposes. These should be replaced with proper logging using the Python logging module to ensure consistent log levels and formatting.

-        print("NO ENABLED LOGGING TOOLSET")
+        logging.debug("No enabled logging toolsets found")

-        print(f"ONLY ONE ENABLED LOGGING TOOLSET {logging_toolsets[0].name}")
+        logging.debug(f"Only one enabled logging toolset found: {logging_toolsets[0].name}")

-            print(f"Non k8s {non_k8s_logs_candidates[0].name}")
+            logging.debug(f"Selected non-kubernetes/logs toolset: {non_k8s_logs_candidates[0].name}")

-            print(f"k8s {logging_toolsets[0].name}")
+            logging.debug(f"Selected kubernetes/logs toolset: {logging_toolsets[0].name}")

Additionally, add an import for logging at the top of the file:

+import logging
 from holmes.core.tools.logging_api import BasePodLoggingToolset
 from holmes.core.tools.tools import Toolset, ToolsetStatusEnum
🧹 Nitpick comments (12)
holmes/plugins/toolsets/datetime.py (1)

1-1: Import path updated to specific submodule
The ToolsetTag import is correctly updated to holmes.core.tools.tools, aligning with the new module structure.

Consider grouping all imports from holmes.core.tools.tools into a single statement to reduce repetition.

holmes/plugins/toolsets/newrelic.py (1)

12-12: Consolidate imports for clarity
You can merge the two import statements into one block to simplify maintenance.

-from holmes.core.tools.tools import (
-    CallablePrerequisite,
-    Tool,
-    ToolParameter,
-    Toolset,
-    ToolsetTag,
-)
-from holmes.core.tools.tools import StructuredToolResult, ToolResultStatus
+from holmes.core.tools.tools import (
+    CallablePrerequisite,
+    Tool,
+    ToolParameter,
+    Toolset,
+    ToolsetTag,
+    StructuredToolResult,
+    ToolResultStatus,
+)
holmes/plugins/toolsets/datadog.py (1)

12-12: Consolidate imports for clarity
Consider merging these imports into one block to reduce redundancy:

-from holmes.core.tools.tools import (
-    CallablePrerequisite,
-    Tool,
-    ToolParameter,
-    Toolset,
-    ToolsetTag,
-)
-from holmes.core.tools.tools import StructuredToolResult, ToolResultStatus
+from holmes.core.tools.tools import (
+    CallablePrerequisite,
+    Tool,
+    ToolParameter,
+    Toolset,
+    ToolsetTag,
+    StructuredToolResult,
+    ToolResultStatus,
+)
holmes/core/tools/toolset_utils.py (2)

32-33: Add a more explicit comment for the empty condition.

When no logging toolsets are enabled, there's a pass statement without a clear explanation of why no action is needed.

 if not logging_toolsets:
-        print("NO ENABLED LOGGING TOOLSET")
-        pass
+        logging.debug("No enabled logging toolsets found")
+        # No logging toolsets to add, continue with other toolsets

38-40: Consider using a more descriptive variable name.

The variable name non_k8s_logs_candidates is somewhat unclear. Consider a more descriptive name to improve readability.

-        non_k8s_logs_candidates = [
-            ts for ts in logging_toolsets if ts.name != "kubernetes/logs"
-        ]
+        preferred_logging_toolsets = [
+            ts for ts in logging_toolsets if ts.name != "kubernetes/logs"
+        ]

And update subsequent references to this variable accordingly.

tests/plugins/toolsets/opensearch/test_opensearch_logs_integration.py (3)

80-81: Remove or replace print statement with proper logging

Debug print statements like print(result.data) should be replaced with assertions or proper logging before merging to production code.

-    print(result.data)
+    # Consider using debug logging instead
+    # logging.debug(f"Result data: {result.data}")

93-94: Remove or replace print statement with proper logging

Debug print statements like print(result.data) should be replaced with assertions or proper logging before merging to production code.

-    print(result.data)
+    # Consider using debug logging instead
+    # logging.debug(f"Result data: {result.data}")

111-112: Remove or replace print statement with proper logging

Debug print statements like print(result.data) should be replaced with assertions or proper logging before merging to production code.

-    print(result.data)
+    # Consider using debug logging instead
+    # logging.debug(f"Result data: {result.data}")
holmes/core/tools/tool_executor.py (3)

41-42: Avoid accessing private attributes directly

Accessing the private attribute _path directly might lead to maintainability issues. Consider adding a getter method if one doesn't exist, or use the existing get_path() method shown in the relevant code snippets.

-                        f"Overriding existing tool '{tool.name} with new tool from {ts.name} at {ts._path}'!"
+                        f"Overriding existing tool '{tool.name} with new tool from {ts.name} at {ts.get_path()}'!"

26-28: Consider logging filtered toolsets for debugging

Adding logging here would help with debugging toolset filtering issues, especially when multiple logging toolsets are present.

+        logging.debug(f"Filtering enabled toolsets: {[ts.name for ts in enabled_toolsets]}")
         self.enabled_toolsets: list[Toolset] = filter_out_default_logging_toolset(
             enabled_toolsets
         )
+        logging.debug(f"After filtering: {[ts.name for ts in self.enabled_toolsets]}")

59-60: Consider raising a specific error for missing tools

Currently, the method logs a warning and returns None when a tool isn't found. Consider either:

  1. Raising a specific exception to fail fast
  2. Returning a default/dummy tool that returns an error result when invoked

This would make error handling more explicit in the calling code.

tests/core/tools/test_toolset_utils.py (1)

19-20: Remove unused methods in DummyNonLoggingToolset

The _invoke and get_parameterized_one_liner methods in DummyNonLoggingToolset are never called in these tests. Consider removing them to simplify the test class.

-    def _invoke(self, params: dict) -> StructuredToolResult:
-        return StructuredToolResult(status=ToolResultStatus.SUCCESS)
-
-    def get_parameterized_one_liner(self, params: dict) -> str:
-        """Generate a one-line description of this tool invocation"""
-        namespace = params.get("namespace", "unknown-namespace")
-        pod_name = params.get("pod_name", "unknown-pod")
-        return f"Fetching logs for pod {pod_name} in namespace {namespace}"

Also applies to: 22-26

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7399b53 and d564317.

📒 Files selected for processing (54)
  • examples/custom_llm.py (1 hunks)
  • holmes/config.py (2 hunks)
  • holmes/core/tool_calling_llm.py (2 hunks)
  • holmes/core/tools/logging_api.py (1 hunks)
  • holmes/core/tools/tool_executor.py (1 hunks)
  • holmes/core/tools/tools.py (0 hunks)
  • holmes/core/tools/toolset_utils.py (1 hunks)
  • holmes/plugins/toolsets/__init__.py (1 hunks)
  • holmes/plugins/toolsets/coralogix/api.py (1 hunks)
  • holmes/plugins/toolsets/coralogix/toolset_coralogix_logs.py (2 hunks)
  • holmes/plugins/toolsets/datadog.py (1 hunks)
  • holmes/plugins/toolsets/datetime.py (1 hunks)
  • holmes/plugins/toolsets/git.py (1 hunks)
  • holmes/plugins/toolsets/grafana/base_grafana_toolset.py (1 hunks)
  • holmes/plugins/toolsets/grafana/common.py (1 hunks)
  • holmes/plugins/toolsets/grafana/toolset_grafana.py (1 hunks)
  • holmes/plugins/toolsets/grafana/toolset_grafana_loki.py (2 hunks)
  • holmes/plugins/toolsets/grafana/toolset_grafana_tempo.py (3 hunks)
  • holmes/plugins/toolsets/grafana/trace_parser.py (1 hunks)
  • holmes/plugins/toolsets/internet/internet.py (2 hunks)
  • holmes/plugins/toolsets/internet/notion.py (2 hunks)
  • holmes/plugins/toolsets/kafka.py (2 hunks)
  • holmes/plugins/toolsets/kubernetes_logs.py (1 hunks)
  • holmes/plugins/toolsets/newrelic.py (1 hunks)
  • holmes/plugins/toolsets/opensearch/opensearch.py (2 hunks)
  • holmes/plugins/toolsets/opensearch/opensearch_logs.py (2 hunks)
  • holmes/plugins/toolsets/opensearch/opensearch_traces.py (2 hunks)
  • holmes/plugins/toolsets/opensearch/opensearch_utils.py (1 hunks)
  • holmes/plugins/toolsets/prometheus/prometheus.py (2 hunks)
  • holmes/plugins/toolsets/rabbitmq/toolset_rabbitmq.py (1 hunks)
  • holmes/plugins/toolsets/robusta/robusta.py (1 hunks)
  • holmes/utils/holmes_sync_toolsets.py (1 hunks)
  • tests/core/tools/test_toolset_utils.py (1 hunks)
  • tests/llm/test_ask_holmes.py (1 hunks)
  • tests/llm/test_investigate.py (1 hunks)
  • tests/llm/test_mocks.py (1 hunks)
  • tests/llm/utils/mock_toolset.py (1 hunks)
  • tests/llm/utils/mock_utils.py (1 hunks)
  • tests/plugins/prompt/test_fetch_logs.py (1 hunks)
  • tests/plugins/prompt/test_generic_ask_conversation.py (1 hunks)
  • tests/plugins/prompt/test_toolsets_instructions.py (1 hunks)
  • tests/plugins/toolsets/grafana/test_grafana_loki.py (1 hunks)
  • tests/plugins/toolsets/grafana/test_grafana_tempo.py (1 hunks)
  • tests/plugins/toolsets/kubernetes/test_kubernetes_logs.py (1 hunks)
  • tests/plugins/toolsets/kubernetes/test_kubernetes_logs_filter_by_timestamp.py (1 hunks)
  • tests/plugins/toolsets/opensearch/test_opensearch_integration.py (1 hunks)
  • tests/plugins/toolsets/opensearch/test_opensearch_logs_integration.py (1 hunks)
  • tests/plugins/toolsets/test_internet.py (1 hunks)
  • tests/plugins/toolsets/test_notion.py (1 hunks)
  • tests/plugins/toolsets/test_prometheus_integration.py (1 hunks)
  • tests/plugins/toolsets/test_tool_kafka.py (1 hunks)
  • tests/plugins/toolsets/test_toolset_utils.py (1 hunks)
  • tests/test_holmes_sync_toolsets.py (1 hunks)
  • tests/test_structured_toolcall_result.py (1 hunks)
💤 Files with no reviewable changes (1)
  • holmes/core/tools/tools.py
🧰 Additional context used
🧬 Code Graph Analysis (34)
tests/plugins/toolsets/grafana/test_grafana_tempo.py (1)
holmes/core/tools/tools.py (1)
  • ToolsetStatusEnum (107-110)
holmes/plugins/toolsets/internet/notion.py (1)
holmes/core/tools/tools.py (2)
  • StructuredToolResult (34-57)
  • ToolResultStatus (28-31)
tests/plugins/prompt/test_generic_ask_conversation.py (1)
holmes/core/tools/tools.py (1)
  • ToolsetStatusEnum (107-110)
tests/plugins/toolsets/grafana/test_grafana_loki.py (1)
holmes/core/tools/tools.py (1)
  • ToolsetStatusEnum (107-110)
holmes/plugins/toolsets/grafana/common.py (1)
holmes/core/tools/tools.py (2)
  • StructuredToolResult (34-57)
  • ToolResultStatus (28-31)
tests/plugins/prompt/test_fetch_logs.py (1)
holmes/core/tools/tools.py (1)
  • ToolsetStatusEnum (107-110)
tests/plugins/toolsets/test_notion.py (1)
holmes/core/tools/tools.py (1)
  • ToolsetStatusEnum (107-110)
tests/test_structured_toolcall_result.py (1)
holmes/core/tools/tools.py (2)
  • StructuredToolResult (34-57)
  • ToolResultStatus (28-31)
tests/plugins/toolsets/test_tool_kafka.py (1)
holmes/core/tools/tools.py (1)
  • ToolsetStatusEnum (107-110)
holmes/plugins/toolsets/opensearch/opensearch_utils.py (1)
holmes/core/tools/tools.py (1)
  • Toolset (318-467)
tests/plugins/toolsets/test_toolset_utils.py (1)
holmes/core/tools/tools_utils.py (1)
  • process_timestamps_to_rfc3339 (72-84)
tests/llm/test_mocks.py (2)
holmes/core/tools/tools.py (2)
  • StructuredToolResult (34-57)
  • ToolResultStatus (28-31)
holmes/core/tools/tool_executor.py (1)
  • ToolExecutor (15-64)
holmes/plugins/toolsets/grafana/toolset_grafana.py (1)
holmes/core/tools/tools.py (2)
  • Tool (125-155)
  • ToolParameter (119-122)
holmes/plugins/toolsets/newrelic.py (1)
holmes/core/tools/tools.py (7)
  • CallablePrerequisite (305-306)
  • Tool (125-155)
  • ToolParameter (119-122)
  • Toolset (318-467)
  • ToolsetTag (113-116)
  • StructuredToolResult (34-57)
  • ToolResultStatus (28-31)
holmes/plugins/toolsets/datadog.py (1)
holmes/core/tools/tools.py (7)
  • CallablePrerequisite (305-306)
  • Tool (125-155)
  • ToolParameter (119-122)
  • Toolset (318-467)
  • ToolsetTag (113-116)
  • StructuredToolResult (34-57)
  • ToolResultStatus (28-31)
holmes/core/tools/logging_api.py (2)
holmes/core/tools/tools.py (4)
  • StructuredToolResult (34-57)
  • Tool (125-155)
  • ToolParameter (119-122)
  • Toolset (318-467)
holmes/core/tools/tools_utils.py (1)
  • get_param_or_raise (136-140)
holmes/utils/holmes_sync_toolsets.py (1)
holmes/core/tools/tools.py (2)
  • Toolset (318-467)
  • ToolsetDBModel (504-514)
holmes/plugins/toolsets/internet/internet.py (1)
holmes/core/tools/tools.py (2)
  • StructuredToolResult (34-57)
  • ToolResultStatus (28-31)
holmes/plugins/toolsets/opensearch/opensearch_traces.py (1)
holmes/core/tools/tools.py (2)
  • StructuredToolResult (34-57)
  • ToolResultStatus (28-31)
tests/plugins/toolsets/kubernetes/test_kubernetes_logs.py (2)
holmes/core/tools/tools.py (1)
  • ToolResultStatus (28-31)
holmes/core/tools/logging_api.py (1)
  • FetchPodLogsParams (27-33)
holmes/plugins/toolsets/robusta/robusta.py (1)
holmes/core/tools/tools.py (7)
  • StaticPrerequisite (300-302)
  • Tool (125-155)
  • ToolParameter (119-122)
  • Toolset (318-467)
  • ToolsetTag (113-116)
  • StructuredToolResult (34-57)
  • ToolResultStatus (28-31)
holmes/plugins/toolsets/grafana/trace_parser.py (1)
holmes/core/tools/tools_utils.py (1)
  • unix_nano_to_rfc3339 (43-50)
tests/llm/utils/mock_utils.py (1)
holmes/core/tools/tools.py (2)
  • StructuredToolResult (34-57)
  • ToolResultStatus (28-31)
tests/plugins/toolsets/kubernetes/test_kubernetes_logs_filter_by_timestamp.py (1)
holmes/core/tools/tools_utils.py (1)
  • to_unix (38-40)
tests/llm/test_investigate.py (2)
tests/plugins/toolsets/test_prometheus_integration.py (1)
  • tool_executor (20-27)
holmes/core/tools/tool_executor.py (1)
  • ToolExecutor (15-64)
holmes/plugins/toolsets/opensearch/opensearch.py (1)
holmes/core/tools/tools.py (2)
  • StructuredToolResult (34-57)
  • ToolResultStatus (28-31)
holmes/plugins/toolsets/datetime.py (1)
holmes/core/tools/tools.py (5)
  • ToolsetTag (113-116)
  • Tool (125-155)
  • Toolset (318-467)
  • StructuredToolResult (34-57)
  • ToolResultStatus (28-31)
holmes/plugins/toolsets/git.py (1)
holmes/core/tools/tools.py (2)
  • StructuredToolResult (34-57)
  • ToolResultStatus (28-31)
holmes/plugins/toolsets/__init__.py (1)
holmes/core/tools/tools.py (2)
  • Toolset (318-467)
  • YAMLToolset (470-479)
tests/plugins/toolsets/opensearch/test_opensearch_integration.py (1)
holmes/core/tools/tools.py (1)
  • ToolResultStatus (28-31)
holmes/plugins/toolsets/grafana/toolset_grafana_tempo.py (1)
holmes/core/tools/tools.py (2)
  • StructuredToolResult (34-57)
  • ToolResultStatus (28-31)
holmes/plugins/toolsets/prometheus/prometheus.py (3)
holmes/core/tools/tools_utils.py (3)
  • standard_start_datetime_tool_param_description (11-12)
  • get_param_or_raise (136-140)
  • process_timestamps_to_rfc3339 (72-84)
holmes/utils/cache.py (1)
  • TTLCache (33-84)
holmes/core/tools/tools.py (2)
  • StructuredToolResult (34-57)
  • ToolResultStatus (28-31)
holmes/core/tools/toolset_utils.py (3)
holmes/core/tools/logging_api.py (1)
  • BasePodLoggingToolset (36-41)
holmes/core/tools/tools.py (3)
  • Toolset (318-467)
  • ToolsetStatusEnum (107-110)
  • get_status (381-382)
tests/llm/utils/mock_toolset.py (1)
  • get_status (188-189)
holmes/core/tools/tool_executor.py (3)
holmes/core/tools/tools.py (8)
  • StructuredToolResult (34-57)
  • Tool (125-155)
  • ToolResultStatus (28-31)
  • Toolset (318-467)
  • ToolsetStatusEnum (107-110)
  • get_status (381-382)
  • invoke (141-147)
  • get_openai_format (134-139)
holmes/core/tools/toolset_utils.py (1)
  • filter_out_default_logging_toolset (5-53)
tests/llm/utils/mock_toolset.py (1)
  • get_status (188-189)
🪛 GitHub Actions: Build and test HolmesGPT
tests/plugins/toolsets/test_toolset_utils.py

[error] 1-1: Import file mismatch: imported module 'test_toolset_utils' points to a different file path. Remove pycache/.pyc files or use unique basenames for test modules.

tests/llm/test_ask_holmes.py

[error] 1-1: PyInstaller build failed due to missing hidden import 'litellm.llms.tokenizers'.

tests/plugins/toolsets/opensearch/test_opensearch_logs_integration.py

[error] 14-14: ImportError: cannot import name 'OpenSearchLoggingConfig' from 'holmes.plugins.toolsets.opensearch.opensearch_utils'.

🔇 Additional comments (79)
tests/plugins/toolsets/test_notion.py (1)

3-3: Correct import path for ToolsetStatusEnum
The ToolsetStatusEnum enum is now correctly imported from its new module holmes.core.tools.tools, matching the relocation in holmes/core/tools/tools.py.

tests/plugins/toolsets/opensearch/test_opensearch_integration.py (1)

5-5: Update import for ToolResultStatus
The ToolResultStatus enum is correctly imported from holmes.core.tools.tools, aligning with its definition in holmes/core/tools/tools.py.

tests/plugins/toolsets/grafana/test_grafana_loki.py (1)

3-3: Align import for ToolsetStatusEnum
Import path updated to the new holmes.core.tools.tools module, consistent with the enum’s relocated definition.

tests/plugins/prompt/test_generic_ask_conversation.py (1)

1-1: Import path for ToolsetStatusEnum updated
The test now correctly imports ToolsetStatusEnum from holmes.core.tools.tools, matching the refactored module structure.

tests/plugins/prompt/test_fetch_logs.py (1)

4-4: Update import for ToolsetStatusEnum
The import has been adjusted to holmes.core.tools.tools, aligning with the relocation of tool enums in the core module.

tests/test_structured_toolcall_result.py (1)

5-5: Import path updated correctly
The relocation of StructuredToolResult and ToolResultStatus into holmes.core.tools.tools is accurately reflected here, and the classes will be resolved from their new module.

holmes/plugins/toolsets/grafana/trace_parser.py (1)

4-4: Import path for utility function corrected
unix_nano_to_rfc3339 is now imported from holmes.core.tools.tools_utils, matching its new location.

holmes/plugins/toolsets/grafana/common.py (1)

6-6: Import path for core tool classes updated
Updated imports of StructuredToolResult and ToolResultStatus to holmes.core.tools.tools align with the refactored module structure.

holmes/utils/holmes_sync_toolsets.py (1)

10-10: Import path aligned with refactored module layout
Importing Toolset and ToolsetDBModel from holmes.core.tools.tools correctly reflects their new location after the reorganization.

tests/plugins/toolsets/test_toolset_utils.py (1)

3-3: Import path updated correctly
The import of process_timestamps_to_rfc3339 from holmes.core.tools.tools_utils matches the function’s new module.

holmes/plugins/toolsets/opensearch/opensearch_utils.py (1)

8-8: Import path correctly updated to reflect new module structure.

The import for Toolset has been updated to use the new specific path holmes.core.tools.tools which aligns with the overall refactoring where tool-related classes have been moved to more specific submodules.

tests/llm/test_investigate.py (1)

12-12: Import path correctly updated for ToolExecutor.

The import path has been updated to use the new dedicated module holmes.core.tools.tool_executor where the ToolExecutor class has been relocated as part of this refactoring effort.

tests/plugins/toolsets/test_tool_kafka.py (1)

7-7: Import path correctly updated for ToolsetStatusEnum.

The import path has been properly updated to use the new specific module path holmes.core.tools.tools for the ToolsetStatusEnum enum class, consistent with the overall refactoring pattern.

holmes/plugins/toolsets/rabbitmq/toolset_rabbitmq.py (1)

6-14: Import paths updated to use the specific tools submodule.

All tool-related imports have been correctly updated to use the new specific module path holmes.core.tools.tools which aligns with the codebase reorganization. This change maintains consistency with similar updates throughout the codebase.

tests/plugins/toolsets/grafana/test_grafana_tempo.py (1)

5-5: LGTM, clean import refactoring.

The import path for ToolsetStatusEnum has been correctly updated to use the new module structure, consistent with the broader codebase refactoring moving tool-related classes to more specific submodules.

holmes/plugins/toolsets/internet/notion.py (2)

5-9: LGTM, correctly refactored imports.

Import paths have been properly updated to reflect the new module structure where tool-related classes are now in the holmes.core.tools.tools submodule.


14-14: LGTM, correctly refactored imports.

Import path has been properly updated for StructuredToolResult and ToolResultStatus to match the new module structure.

tests/llm/test_ask_holmes.py (1)

10-10: LGTM, correctly refactored import path.

The import path for ToolExecutor has been correctly updated to use its dedicated module holmes.core.tools.tool_executor, which aligns with the architecture refactoring.

holmes/plugins/toolsets/grafana/toolset_grafana.py (1)

3-3: LGTM, correctly refactored imports.

Import path has been properly updated for Tool and ToolParameter to use the new module structure where tool-related classes are now in the holmes.core.tools.tools submodule.

holmes/plugins/toolsets/datetime.py (2)

3-3: Tool and Toolset imports are correct
The imports for Tool and Toolset now reference the refined submodule, matching the refactoring in holmes.core.tools.tools.


5-5: StructuredToolResult and ToolResultStatus imports are correct
Import paths updated in line with the refactoring; both classes are defined in holmes.core.tools.tools as expected.

tests/llm/utils/mock_utils.py (1)

15-15: Align imports with refactored core tools module
The StructuredToolResult and ToolResultStatus imports are updated to their new location in holmes.core.tools.tools, matching the recent module reorganization.

tests/plugins/toolsets/kubernetes/test_kubernetes_logs_filter_by_timestamp.py (1)

6-6: Import to_unix from the new utility module
This change correctly updates the import path to holmes.core.tools.tools_utils.to_unix, aligning with the consolidated utilities.

tests/plugins/prompt/test_toolsets_instructions.py (1)

3-9: Update imports to reflect module reorganization
The imports for StaticPrerequisite, StructuredToolResult, Tool, ToolResultStatus, and Toolset have been correctly moved to holmes.core.tools.tools, consistent with the recent refactoring.

holmes/plugins/toolsets/coralogix/api.py (1)

16-19: Align utility imports with new module layout
The get_param_or_raise and process_timestamps_to_rfc3339 functions are now correctly imported from holmes.core.tools.tools_utils, matching the reorganization of utility functions.

holmes/plugins/toolsets/newrelic.py (1)

4-10: Imports updated to new submodule structure
The tool-related classes (CallablePrerequisite, Tool, ToolParameter, Toolset, ToolsetTag) are now correctly imported from holmes.core.tools.tools following the core refactor.

examples/custom_llm.py (2)

5-5: Updated import for Tool
The Tool class is now correctly imported from holmes.core.tools.tools following the modularization.


6-6: Updated import for ToolExecutor
ToolExecutor is now imported from its dedicated module holmes.core.tools.tool_executor, aligning with the refactoring.

holmes/plugins/toolsets/datadog.py (1)

4-10: Imports updated to new submodule structure
The core tool classes (CallablePrerequisite, Tool, ToolParameter, Toolset, ToolsetTag) have been moved under holmes.core.tools.tools. This import change matches the refactor in other plugin modules.

holmes/plugins/toolsets/__init__.py (1)

30-30: Import path updated to holmes.core.tools.tools
The Toolset and YAMLToolset classes are now correctly imported from the tools submodule.

tests/llm/test_mocks.py (1)

1-2: Imports aligned with core refactor
StructuredToolResult and ToolResultStatus are imported from holmes.core.tools.tools, and ToolExecutor from holmes.core.tools.tool_executor. This matches the new module layout.

holmes/plugins/toolsets/internet/internet.py (2)

7-13: Import paths updated correctly

Import statements have been updated to use the new submodule structure, importing tool-related classes from the more specific holmes.core.tools.tools module instead of directly from holmes.core.tools.


18-18: Import path updated correctly

StructuredToolResult and ToolResultStatus are now imported from the reorganized holmes.core.tools.tools module.

tests/llm/utils/mock_toolset.py (2)

5-10: Import paths updated correctly

The import statements have been updated to use the more specific holmes.core.tools.tools module path for tool-related classes, which aligns with the broader refactoring effort in the codebase.


17-17: Import path updated correctly

StructuredToolResult is now imported from the reorganized holmes.core.tools.tools module.

holmes/plugins/toolsets/opensearch/opensearch_logs.py (2)

8-13: Import paths updated correctly

The import statements have been updated to use the more specific holmes.core.tools.tools module path for tool-related classes, aligning with the broader refactoring in the codebase.


23-23: Import path updated correctly

StructuredToolResult and ToolResultStatus are now imported from the reorganized holmes.core.tools.tools module.

tests/plugins/toolsets/kubernetes/test_kubernetes_logs.py (2)

5-5: Import path updated correctly

ToolResultStatus is now imported from the more specific holmes.core.tools.tools module, maintaining consistency with the import path restructuring.


7-7: Import path updated correctly

FetchPodLogsParams is now imported from holmes.core.tools.logging_api instead of holmes.plugins.toolsets.logging_api, reflecting the reorganization of tool-related components into more specific submodules.

holmes/plugins/toolsets/opensearch/opensearch_traces.py (2)

8-13: Import path refactored correctly.

The imports for tool-related entities have been properly updated to use the more specific submodule holmes.core.tools.tools instead of directly from holmes.core.tools. This change is consistent with the broader refactoring effort to organize tool-related entities into more specific submodules.


22-22: Import path refactored correctly.

The imports for StructuredToolResult and ToolResultStatus have been properly updated to use the more specific submodule holmes.core.tools.tools. This change matches the refactoring pattern applied across the codebase.

tests/plugins/toolsets/test_prometheus_integration.py (1)

6-7: Import paths correctly separated and refactored.

The imports have been properly refactored to match the new code organization:

  1. ToolsetStatusEnum is now imported from holmes.core.tools.tools
  2. ToolExecutor is now imported from holmes.core.tools.tool_executor

This change aligns with the broader refactoring to move the ToolExecutor class into its own dedicated module.

holmes/core/tool_calling_llm.py (2)

35-35: Import path refactored correctly for ToolExecutor.

The import for ToolExecutor has been properly updated to use the more specific submodule holmes.core.tools.tool_executor instead of directly from holmes.core.tools. This change is consistent with the refactoring to move ToolExecutor to its own dedicated module.


44-44: Import path refactored correctly for tool-related entities.

The imports for StructuredToolResult and ToolResultStatus have been properly updated to use the more specific submodule holmes.core.tools.tools instead of directly from holmes.core.tools. This change is consistent with the refactoring pattern applied across the codebase.

holmes/plugins/toolsets/robusta/robusta.py (2)

7-13: Import path refactored correctly for tool-related entities.

The imports for tool-related entities (StaticPrerequisite, Tool, ToolParameter, Toolset, ToolsetTag) have been properly updated to use the more specific submodule holmes.core.tools.tools instead of directly from holmes.core.tools. This change is consistent with the refactoring pattern applied across the codebase.


14-14: Import path refactored correctly for result-related entities.

The imports for StructuredToolResult and ToolResultStatus have been properly updated to use the more specific submodule holmes.core.tools.tools instead of directly from holmes.core.tools. This change matches the refactoring pattern applied across the codebase.

tests/plugins/toolsets/test_internet.py (1)

8-9: Consistent import path updates
Imports of ToolsetStatusEnum and ToolExecutor have been updated to their new modules, matching the reorganized holmes.core.tools package structure.

holmes/plugins/toolsets/grafana/base_grafana_toolset.py (2)

3-8: Approve modularized imports
Core toolset classes are now imported from holmes.core.tools.tools, aligning with the refactoring. No behavior changes detected.


11-11: Approve utility import path update
TOOLSET_CONFIG_MISSING_ERROR is correctly imported from tools_utils, reflecting the new module location.

holmes/core/tools/logging_api.py (2)

8-13: Accept updated import sources
StructuredToolResult, Tool, ToolParameter, and Toolset are now imported from holmes.core.tools.tools, consistent with the refactoring.


14-14: Approve import of get_param_or_raise
Utility function imported from tools_utils matches its new location.

holmes/plugins/toolsets/git.py (2)

7-8: Approve refactored imports for result types
Updated import of StructuredToolResult and ToolResultStatus from the reorganized tools submodule aligns with the new module layout.


9-15: Approve tool and prerequisite import adjustments
Toolset, Tool, ToolParameter, ToolsetTag, and CallablePrerequisite now come from holmes.core.tools.tools, matching the reorganization.

holmes/config.py (3)

15-16: Approve import of orchestrator classes
IssueInvestigator and ToolCallingLLM now correctly imported from holmes.core.tool_calling_llm, and ToolExecutor from holmes.core.tools.tool_executor, aligning with the refactor.


17-24: Approve core toolset imports
Toolset, ToolsetPattern, ToolsetYamlFromConfig, get_matching_toolsets, ToolsetStatusEnum, and ToolsetTag are now sourced from holmes.core.tools.tools, reflecting the modularized package structure.


38-38: Approve YAMLToolset import update
YAMLToolset is accurately imported from the reorganized holmes.core.tools.tools module.

tests/test_holmes_sync_toolsets.py (1)

8-16: Import path refactoring looks good.

The import path has been correctly updated from holmes.core.tools to the more specific holmes.core.tools.tools module, which aligns with the broader refactoring of moving tool-related classes to more specific submodules.

holmes/plugins/toolsets/coralogix/toolset_coralogix_logs.py (2)

2-10: Import path refactoring looks good.

The tool-related classes are now correctly imported from the more specific holmes.core.tools.tools module instead of directly from holmes.core.tools.


25-29: Utility imports correctly relocated.

The utility constants and functions have been properly moved from holmes.plugins.toolsets.utils to holmes.core.tools.tools_utils, which is a more appropriate location for these shared utilities.

holmes/plugins/toolsets/opensearch/opensearch.py (2)

5-11: Tool class imports refactoring looks good.

The import path has been correctly updated to get tool-related classes from the more specific holmes.core.tools.tools module.


14-15: Utility and model imports properly separated.

Good separation of concerns:

  1. Utility constants are now imported from holmes.core.tools.tools_utils
  2. Model classes (StructuredToolResult, ToolResultStatus) are imported from holmes.core.tools.tools

This follows the new module organization pattern consistently.

holmes/plugins/toolsets/kafka.py (3)

5-11: Tool class imports refactoring looks good.

Tool-related classes are now correctly imported from the more specific holmes.core.tools.tools module instead of directly from holmes.core.tools.


12-15: Utilities import path updated correctly.

The utility constants and functions are now properly imported from holmes.core.tools.tools_utils instead of holmes.plugins.toolsets.utils.


32-32: Model imports separated appropriately.

StructuredToolResult and ToolResultStatus are now imported separately from holmes.core.tools.tools, which improves clarity about where these models come from.

holmes/plugins/toolsets/kubernetes_logs.py (3)

8-13: Import path refactoring looks good.

The updates to import StaticPrerequisite, StructuredToolResult, ToolResultStatus, and ToolsetTag from the more specific holmes.core.tools.tools module align with the project-wide refactoring of tool-related imports.


14-19: Import structure improvement is clean.

Moving the logging API classes to their dedicated module improves code organization and maintainability.


20-20: Utility function import path corrected.

Importing utility functions like process_timestamps_to_int and to_unix from the dedicated utilities module follows good separation of concerns.

holmes/plugins/toolsets/grafana/toolset_grafana_loki.py (3)

5-5: Tool classes import path updated properly.

The import path for Tool and ToolParameter has been correctly updated to the new module structure.


13-16: Utility functions import path improved.

The utility functions get_param_or_raise and process_timestamps_to_rfc3339 are now imported from the dedicated utils module, which better organizes the codebase.


22-22: Result classes import path updated.

StructuredToolResult and ToolResultStatus are now correctly imported from their new location.

holmes/plugins/toolsets/grafana/toolset_grafana_tempo.py (3)

7-10: Tool imports refactored correctly.

The import for core tool classes has been updated to the new module structure.


16-20: Utility functions import paths improved.

The utilities and constants are now properly imported from the dedicated utils module.


27-27: Result class imports updated.

StructuredToolResult and ToolResultStatus are now imported from their refactored location.

holmes/plugins/toolsets/prometheus/prometheus.py (3)

12-18: Core tool class imports updated.

The core tool classes are now imported from their new dedicated module.


24-29: Utility functions import paths improved.

Constants and utility functions are now imported from the appropriate tools_utils module.


31-31: Result classes import path updated.

StructuredToolResult and ToolResultStatus are now imported from the new module location.

holmes/core/tools/toolset_utils.py (1)

1-3: Appropriate imports for the utility function.

The imports are correctly set up for the new utility function, importing the necessary types from their respective modules.

tests/plugins/toolsets/opensearch/test_opensearch_logs_integration.py (2)

31-32: Verify test date range validity

These hardcoded future dates (2025) may cause the tests to fail if they assume data exists in that period. Since this is an integration test, consider using relative dates or documenting how test data should be set up.


64-65:

✅ Verification successful

Ensure prerequisites_callable accepts dict input

You're converting OpenSearchLoggingConfig to a dict with model_dump() before passing to prerequisites_callable. Verify that the callable properly handles dict input rather than the config object directly.


🏁 Script executed:

#!/bin/bash
# Check if prerequisites_callable is defined to accept a dict parameter
rg -A 3 "def prerequisites_callable" --glob "**/*.py"

Length of output: 4323


No action needed: prerequisites_callable accepts dict input

Verified that in holmes/plugins/toolsets/opensearch/opensearch.py (and its utility in opensearch_utils.py),

def prerequisites_callable(self, config: dict[str, Any]) -> Tuple[bool, str]:
    …

declares config as a dict[str, Any]. Passing OpenSearchLoggingConfig.model_dump() is therefore appropriate.

Comment on lines 1 to +3
import pytest
from dateutil import parser
from holmes.plugins.toolsets.utils import process_timestamps_to_rfc3339
from holmes.core.tools.tools_utils import process_timestamps_to_rfc3339
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Resolve test module naming conflict
The pipeline failure indicates an Import file mismatch for test_toolset_utils. This is likely due to a conflicting basename or stale __pycache__ entries. Please rename this test file or clear outdated bytecode caches to ensure a unique module name.

🧰 Tools
🪛 GitHub Actions: Build and test HolmesGPT

[error] 1-1: Import file mismatch: imported module 'test_toolset_utils' points to a different file path. Remove pycache/.pyc files or use unique basenames for test modules.

🤖 Prompt for AI Agents
In tests/plugins/toolsets/test_toolset_utils.py at the beginning of the file,
there is a module naming conflict causing an Import file mismatch error in the
pipeline. To fix this, rename the test file to a unique name that does not
conflict with other test modules, or alternatively clear any stale __pycache__
directories and .pyc files related to this test to remove outdated bytecode
caches.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant