Release 0.5.7

Fix bugs from 0.5.5

Release v0.5.5

What's New
✨ Data Exclusion System - Selectively exclude sensitive fields or entire tables
🧪 Comprehensive Integration Tests - Automated testing across all databases and embedding providers
🔧 TEI Provider Improvements - Enhanced stability and dynamic port allocation
🐛 Bug Fixes - Resolved port conflicts and connection issues

Release v0.5.0

+## What’s New (Today’s Commit)
+* 🚚 Removed the old rust-bert embedding backend and fully replaced it with TEI (Text Embeddings Inference v1.7.0).
+* 🔧 Updated all embedding code paths (Ollama, TEI, Google) to work seamlessly with the new TEI client.
+* 📝 Improved logging (via env_logger) so RUST_LOG=debug now shows all internal db2vec events.
+* 📦 Bundled the TEI binaries (tei-metal / tei-onnx) and added automatic extraction if no --tei-binary-path is provided.

Release v0.4.0

Release v0.4.0 - Expanded Embedding Options & CPU Focus

This release significantly expands the embedding capabilities of db2vec, offering more flexibility while maintaining a strong focus on efficient CPU-based processing.

✨ New Features

• Multiple Embedding Providers: You can now choose your embedding engine!
  • rustbert: Utilizes the built-in all-MiniLM-L6-v2 model (384 dimensions) directly on your CPU via the rust-bert library. No external service needed!
  • google: Connect to Google's Generative AI API (e.g., text-embedding-004) using your API key.
  • ollama: Continued support for using any model served locally via Ollama.

🚀 How to Use New Embeddings

Select your provider and model using the following flags:

• --embedding-provider <PROVIDER>: Choose ollama, rustbert, or google.
• --embedding-model <MODEL_NAME>: Specify the model name (e.g., nomic-embed-text for Ollama, text-embedding-004 for Google). This flag is ignored for rustbert as it uses the built-in model.
• --dimension <DIM>: Crucial! Set this to match your chosen model's output dimension (e.g., 384 for rustbert, 768 for text-embedding-004).
• --embedding-api-key <KEY>: Required only when using --embedding-provider google. Can also be set via SECRET env var.
• --embedding-url <URL>: Primarily for Ollama, but can be used to specify a different Google API endpoint if needed.

Example (Rust-Bert):

./db2vec -f dump.sql -t qdrant --embedding-provider rustbert --dimension 384

Release v0.3.0

Release v0.3.0

This release provides pre-compiled binaries for db2vec, a high-performance Rust tool designed to parse database dumps, generate vector embeddings using local Ollama models, and load them efficiently into various vector databases.

Key Features

• Fast Processing: Built in Rust for speed on large datasets.
• Parallel Embeddings: Leverages multiple threads and configurable batching for Ollama requests.
• Supported Dumps: Parses .sql (MySQL, PostgreSQL, MSSQL, SQLite, Oracle) and .surql (SurrealDB) files.
• Vector DB Targets: Supports Chroma, Milvus, Pinecone, Qdrant, Redis Stack, and SurrealDB.
• Automatic Schema: Creates databases, collections/indexes, and configures dimensions/metrics automatically in the target vector DB based on source table names and CLI flags.
• Local Embeddings: Integrates with Ollama for generating embeddings locally.
• Configurable: Extensive CLI options for fine-tuning performance and connections.

Binaries Included

• Linux (Static): db2vec-0.3.0-linux-x86_64 (Built with MUSL, should run on most x86_64 Linux distributions without extra dependencies)
• macOS (Universal): db2vec-0.3.0-macos-universal (Runs on both Intel and Apple Silicon Macs)
• Windows: db2vec-0.3.0-windows-x86_64.exe

Checksum files (.sha256) are provided for verifying downloads.

How to Use

1. Download the appropriate binary for your operating system from the Assets section below.
2. (Optional but Recommended) Download the corresponding .sha256 checksum file.
3. Verify the download (Example for Linux):

   shasum -a 256 -c db2vec-0.3.0-linux-x86_64.sha256
   # Should output: db2vec-0.3.0-linux-x86_64: OK

4. Make the binary executable (Linux/macOS):

   chmod +x db2vec-0.3.0-linux-x86_64
   # or
   chmod +x db2vec-0.3.0-macos-universal

5. Run the tool:
   • Linux:

     ./db2vec-0.3.0-linux-x86_64 --help
     ./db2vec-0.3.0-linux-x86_64 -f your_dump.sql -t redis --host redis://localhost:6379 ...

   • macOS:

     ./db2vec-0.3.0-macos-universal --help
     ./db2vec-0.3.0-macos-universal -f your_dump.sql -t redis --host redis://localhost:6379 ...

   • Windows (Command Prompt or PowerShell):

     .\db2vec-0.3.0-windows-x86_64.exe --help
     .\db2vec-0.3.0-windows-x86_64.exe -f your_dump.sql -t redis --host redis://localhost:6379 ...

For detailed configuration options and examples, please refer to the README.md.