Skip to content

Fix Ollama embedding configurator to respect API_BASE environment variable #3100

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

devin-ai-integration[bot]
Copy link
Contributor

@devin-ai-integration devin-ai-integration bot commented Jul 1, 2025

Fix Ollama Embedding Configuration: Support API_BASE Environment Variable

Summary

This PR fixes issue #3099 by implementing proper support for the API_BASE environment variable in Ollama embedding configuration. The implementation provides a flexible, priority-based URL resolution system while addressing GitHub review feedback to use URLs exactly as provided.

Key Changes:

  • Added support for API_BASE environment variable in Ollama embedding configuration
  • Implemented priority-based URL resolution: url config > api_base config > API_BASE env var > default
  • Added URL validation for security (HTTP/HTTPS schemes only)
  • Removed automatic URL construction logic per reviewer feedback - URLs now used as-is
  • Updated comprehensive test suite to match new behavior

Breaking Change Note: The URL handling behavior has changed - URLs are now used exactly as provided without automatic /api/embeddings endpoint appending.

Review & Testing Checklist for Human

Risk Level: 🟡 Medium - Significant behavior changes with potential backward compatibility impact

  • End-to-end testing with actual Ollama instance - Verify the fix works with real Ollama server, not just mocked tests
  • Backward compatibility verification - Test existing configurations to ensure they still work (especially those that might have relied on automatic URL construction)
  • URL validation review - Confirm the HTTP/HTTPS-only restriction is appropriate and doesn't block legitimate use cases
  • Test scenario realism - Review the updated test cases to ensure they reflect realistic usage patterns after extensive modifications

Recommended Test Plan:

  1. Set up local Ollama instance
  2. Test various API_BASE configurations (with/without /api/embeddings endpoint)
  3. Verify priority order works correctly when multiple URL sources are provided
  4. Test edge cases like trailing slashes, different ports, custom domains

Diagram

%%{ init : { "theme" : "default" }}%%
graph TD
    Issue["Issue #3099<br/>Ollama API_BASE not respected"]
    
    EmbedConfig["src/crewai/utilities/<br/>embedding_configurator.py"]:::major-edit
    TestFile["tests/utilities/<br/>test_ollama_embedding_configurator.py"]:::major-edit
    
    OllamaFunc["chromadb.utils.embedding_functions.<br/>OllamaEmbeddingFunction"]:::context
    EnvVars["Environment Variables<br/>(API_BASE)"]:::context
    
    Issue --> EmbedConfig
    EmbedConfig --> OllamaFunc
    EnvVars --> EmbedConfig
    EmbedConfig --> TestFile
    
    subgraph "Key Changes"
        URLValidation["URL Validation<br/>(HTTP/HTTPS only)"]:::minor-edit
        PriorityLogic["Priority Resolution<br/>url > api_base > API_BASE > default"]:::major-edit
        AsIsURLs["URLs Used As-Is<br/>(no auto /api/embeddings)"]:::major-edit
    end
    
    EmbedConfig --> URLValidation
    EmbedConfig --> PriorityLogic  
    EmbedConfig --> AsIsURLs
    
    subgraph Legend
        L1["Major Edit"]:::major-edit
        L2["Minor Edit"]:::minor-edit
        L3["Context/No Edit"]:::context
    end

classDef major-edit fill:#90EE90
classDef minor-edit fill:#87CEEB  
classDef context fill:#FFFFFF
Loading

Notes

  • Session Details: Requested by João ([email protected]) via Slack - Session: https://app.devin.ai/sessions/c4af68dda274403f922b6da76361ff9c
  • GitHub Review Integration: Addressed feedback from lucasgomide to remove URL construction helper and use URLs as-is
  • CI Status: All 10 CI checks passing (Python 3.10-3.13, lint, security, type checking)
  • Documentation: Added inline documentation explaining the new URL configuration approach and priority order
  • Backward Compatibility: While the API remains the same, the URL handling behavior has changed significantly - users who relied on automatic endpoint appending may need to update their configurations

…iable

- Add support for API_BASE environment variable in _configure_ollama method
- Implement priority order: url config > api_base config > API_BASE env var > default
- Handle URL construction with proper /api/embeddings endpoint appending
- Maintain backward compatibility with existing url parameter
- Add comprehensive tests covering all configuration scenarios
- Fixes issue #3099: Ollama integration problems with API_BASE configuration

The fix ensures consistent behavior with other embedding providers and
allows users to configure Ollama embeddings using environment variables
for better deployment flexibility.

Co-Authored-By: João <[email protected]>
Copy link
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@joaomdmoura
Copy link
Collaborator

Disclaimer: This review was made by a crew of AI Agents.

Code Review Comment for Ollama Embedding Configurator Changes

Overview

The recent changes made to the embedding_configurator.py and the accompanying tests in test_ollama_embedding_configurator.py enhance the configurability of the Ollama embedding service by allowing for flexible API base URL configurations. The proposed modifications prioritize URL sourcing from multiple origins, aiding various deployment environments.

Code Quality Findings

1. embedding_configurator.py

Positive Aspects

  • URL Priority Handling: This change clearly defines a priority for URL configuration, which is excellent for ensuring user flexibility:
    1. url from configuration
    2. api_base from configuration
    3. API_BASE environment variable
    4. Default localhost URL
  • Backward Compatibility: The implementation maintains backward compatibility, which is critical for users still using previous configurations.

Recommendations

  • URL Construction Logic: The logic for constructing the URLs can be simplified to enhance robustness. Consider the following implementation that refines URL handling:

    def _construct_embeddings_url(base_url):
        if not base_url:
            return "http://localhost:11434/api/embeddings"
        
        base_url = base_url.rstrip('/')
        return f"{base_url}/api/embeddings" if not base_url.endswith('/api/embeddings') else base_url
  • Error Handling for Invalid URLs: The absence of error validation for malformed URLs can lead to unexpected behavior. Implementing a validation function will ensure that users receive clear error messages when misconfiguring URLs:

    def _validate_url(url):
        from urllib.parse import urlparse
        
        if not url:
            return False
        
        try:
            result = urlparse(url)
            return all([result.scheme, result.netloc])
        except ValueError:
            return False

    Integrate this validation with _configure_ollama to raise meaningful errors on failure.

2. test_ollama_embedding_configurator.py

Positive Aspects

  • Comprehensive Test Coverage: The tests effectively cover all configuration scenarios, verifying that the correct URLs are constructed and integrated.
  • Use of Pytest Fixtures and Mocking: This organization supports clean and readable test code, fulfilling best practices in test-driven development.

Recommendations

  • Organized Test Structure: The test organization could benefit from utilizing pytest markers to group and manage tests based on their functions or categories. For instance:

    import pytest
    
    @pytest.mark.url_configuration
    class TestOllamaUrlConfiguration:
        # existing test methods
  • Adding Edge Case Tests: It is imperative to expand the test coverage to include edge cases, particularly:

    • Handling of invalid URL formats.
    • Scenarios involving network timeouts.
    • Authentication failures.

Example for Testing Invalid URLs

@pytest.mark.error_handling
class TestOllamaErrorHandling:
    @pytest.mark.parametrize("invalid_url", [
        "not-a-url",
        "ftp://invalid-scheme",
        "http://",
    ])
    def test_invalid_url_raises_error(self, invalid_url):
        config = {
            "provider": "ollama",
            "config": {
                "model": "llama2",
                "url": invalid_url
            }
        }

        with pytest.raises(ValueError, match="Invalid Ollama API URL"):
            self.configurator.configure_embedder(config)

Impact Analysis

These contributions to both the configurator and its testing suite represent a significant step forward in making the Ollama embedding service more robust and user-friendly. Nevertheless, the implementation would be substantially fortified with enhanced error handling and comprehensive test coverage.

The pull request is generally constructive and moves towards improved reliability. It is recommended to address the outlined issues and enhancements before final merging. The strengthened error-handling components would notably improve the codebase's readiness for production deployment.

Feel free to merge after addressing the suggested improvements!

devin-ai-integration bot and others added 2 commits July 1, 2025 22:07
… and edge case tests

- Add _validate_url() helper function with proper URL validation using urllib.parse
- Add _construct_embeddings_url() helper function to refactor URL construction logic
- Add comprehensive error handling with ValueError for invalid URLs
- Fix test mocking to use correct chromadb import path
- Add edge case tests for invalid URLs with pytest markers
- Organize tests with @pytest.mark.url_configuration and @pytest.mark.error_handling
- Remove unused imports (pytest, MagicMock) to fix lint issues

This addresses all suggestions from joaomdmoura's AI review while maintaining backward compatibility.

Co-Authored-By: João <[email protected]>
…dings

- Update _validate_url() to restrict schemes to 'http' and 'https' only
- Fixes CI test failures for ftp://invalid-scheme test cases
- Maintains security by preventing non-web protocols

Co-Authored-By: João <[email protected]>
Copy link
Contributor

@lucasgomide lucasgomide left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add docs about that also weel

Comment on lines 112 to 118
def _construct_embeddings_url(base_url):
"""Construct the full embeddings URL from a base URL."""
if not base_url:
return "http://localhost:11434/api/embeddings"

base_url = base_url.rstrip('/')
return f"{base_url}/api/embeddings" if not base_url.endswith('/api/embeddings') else base_url
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you are assuming that all embeddings providers's url has the same format. It might not be totally truth.

I think we don't need this method, please drop it and then use the provided byt config or env var

devin-ai-integration bot and others added 3 commits July 2, 2025 17:00
…d URLs as-is

- Remove _construct_embeddings_url() method as requested by lucasgomide
- Use URLs exactly as provided by config or environment variable
- Add comprehensive documentation about API_BASE environment variable support
- Maintain URL validation for security
- Simplify implementation to avoid assumptions about URL formats across providers

Co-Authored-By: João <[email protected]>
- Update test expectations to provide complete URLs with /api/embeddings endpoint
- Remove assumptions about automatic URL construction per lucasgomide's feedback
- Tests now reflect the new behavior where URLs are used exactly as provided
- Maintains backward compatibility for users who provide complete URLs

Co-Authored-By: João <[email protected]>
- Update test inputs to provide complete URLs with /api/embeddings endpoint
- Remove tests for automatic URL construction that no longer exists
- Tests now reflect the requirement for users to provide complete URLs
- Addresses CI failures after removing _construct_embeddings_url helper

Co-Authored-By: João <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants