Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(rust-sdk): Make API key optional for self-hosted instances #991

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions SELF_HOST.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,10 @@ If you’d like to test the crawl endpoint, you can run this:

This section provides solutions to common issues you might encounter while setting up or running your self-hosted instance of Firecrawl.

### API Keys for SDK Usage

**Note:** When using Firecrawl SDKs with a self-hosted instance, API keys are optional. API keys are only required when connecting to the cloud service (api.firecrawl.dev).

### Supabase client is not configured

**Symptom:**
Expand Down
2 changes: 1 addition & 1 deletion apps/rust-sdk/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ use crate::crawl::CrawlStatus;
#[derive(Debug, Deserialize, Serialize, Clone)]
pub struct FirecrawlAPIError {
/// Always false.
success: bool,
pub success: bool,

/// Error message
pub error: String,
Expand Down
19 changes: 17 additions & 2 deletions apps/rust-sdk/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ pub mod map;
pub mod scrape;

pub use error::FirecrawlError;
use error::FirecrawlAPIError;

#[derive(Clone, Debug)]
pub struct FirecrawlApp {
Expand All @@ -18,16 +19,30 @@ pub struct FirecrawlApp {
}

pub(crate) const API_VERSION: &str = "/v1";
const CLOUD_API_URL: &str = "https://api.firecrawl.dev";

impl FirecrawlApp {
pub fn new(api_key: impl AsRef<str>) -> Result<Self, FirecrawlError> {
FirecrawlApp::new_selfhosted("https://api.firecrawl.dev", Some(api_key))
FirecrawlApp::new_selfhosted(CLOUD_API_URL, Some(api_key))
}

pub fn new_selfhosted(api_url: impl AsRef<str>, api_key: Option<impl AsRef<str>>) -> Result<Self, FirecrawlError> {
let url = api_url.as_ref().to_string();

if url == CLOUD_API_URL && api_key.is_none() {
return Err(FirecrawlError::APIError(
"Configuration".to_string(),
FirecrawlAPIError {
success: false,
error: "API key is required for cloud service".to_string(),
details: None,
}
));
}

Ok(FirecrawlApp {
api_key: api_key.map(|x| x.as_ref().to_string()),
api_url: api_url.as_ref().to_string(),
api_url: url,
client: Client::new(),
})
}
Expand Down
28 changes: 27 additions & 1 deletion apps/rust-sdk/tests/e2e_with_auth.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
use assert_matches::assert_matches;
use dotenvy::dotenv;
use firecrawl::scrape::{ExtractOptions, ScrapeFormats, ScrapeOptions};
use firecrawl::FirecrawlApp;
use firecrawl::{FirecrawlApp, FirecrawlError};
use serde_json::json;
use std::env;

Expand Down Expand Up @@ -154,3 +154,29 @@ async fn test_llm_extraction() {
assert!(llm_extraction["supports_sso"].is_boolean());
assert!(llm_extraction["is_open_source"].is_boolean());
}

#[test]
fn test_api_key_requirements() {
dotenv().ok();

let api_url = env::var("API_URL").unwrap_or("http://localhost:3002".to_string());
let api_key = env::var("TEST_API_KEY").ok();

match (api_url.contains("api.firecrawl.dev"), api_key) {
(false, _) => {
let result = FirecrawlApp::new_selfhosted(&api_url, None::<String>);
assert!(result.is_ok(), "Local setup failed: {:?}", result.err().unwrap());
}
(true, None) => {
let result = FirecrawlApp::new_selfhosted(&api_url, None::<String>);
assert!(matches!(
result,
Err(FirecrawlError::APIError(msg, _)) if msg == "Configuration"
));
}
(true, Some(key)) => {
let result = FirecrawlApp::new_selfhosted(&api_url, Some(&key));
assert!(result.is_ok());
}
}
}