Skip to content

Commit 2af958e

Browse files
Feat/llm config (#724)
* feature: Add LlmConfig to easily configure and pass LLM configs to different strategies * pulled in next branch and resolved conflicts * feat: Add gemini and deepseek providers. Make ignore_cache in llm content filter to true by default to avoid confusions * Refactor: Update LlmConfig in LLMExtractionStrategy class and deprecate old params * updated tests, docs and readme
1 parent 3cb2887 commit 2af958e

25 files changed

+421
-241
lines changed

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -407,7 +407,7 @@ if __name__ == "__main__":
407407
```python
408408
import os
409409
import asyncio
410-
from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, CacheMode
410+
from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, CacheMode, LlmConfig
411411
from crawl4ai.extraction_strategy import LLMExtractionStrategy
412412
from pydantic import BaseModel, Field
413413

@@ -423,7 +423,7 @@ async def main():
423423
extraction_strategy=LLMExtractionStrategy(
424424
# Here you can use any provider that Litellm library supports, for instance: ollama/qwen2
425425
# provider="ollama/qwen2", api_token="no-token",
426-
provider="openai/gpt-4o", api_token=os.getenv('OPENAI_API_KEY'),
426+
llmConfig = LlmConfig(provider="openai/gpt-4o", api_token=os.getenv('OPENAI_API_KEY')),
427427
schema=OpenAIModelFee.schema(),
428428
extraction_type="schema",
429429
instruction="""From the crawled content, extract all mentioned model names along with their fees for input and output tokens.

0 commit comments

Comments
 (0)