FeatherFlow is a Rust-based alternative to dbt (data build tool) designed to provide robust, statically-checked data transformations with superior performance. This project aims to solve common pain points in the data transformation workflow through static analysis and compiled performance.
Problems with existing tools like dbt:
- Runtime SQL errors that could be caught earlier with static analysis
- Performance limitations of Python-based tooling
- Limited capabilities for deep SQL introspection and optimization
FeatherFlow's approach:
- Parse SQL into Abstract Syntax Trees (ASTs) to perform static analysis
- Catch schema issues, syntax errors, and semantic problems before execution
- Compile transformation pipelines to efficient execution plans
- Provide a robust, type-safe experience for data engineers
- Static SQL Analysis: Catch errors before runtime by analyzing the SQL AST
- Schema Validation: Validate schema references without connecting to a database
- Dependency Management: Automatically track dependencies between models
- Performance: Rust-based implementation for superior speed and memory efficiency
- SQL Transformation: Intelligent handling of SQL queries with schema manipulation
FeatherFlow is in active development. Current focus areas include:
- SQL parsing and AST manipulation
- Schema transformation and validation
- Building core transformation engine components
You can run tests using the following commands:
# Run all tests in the project
make test
# Run only the sql_engine/ast_utils tests
make test-module MODULE=sql_engine::ast_utils
# Run a specific test
make test-single TEST=test_simple_select
# Run tests with output (even for passing tests)
make test-verbose
For tests to be discovered by Cargo:
- Unit tests should be in the same file as the code they test, in a
mod tests
module - Integration tests should be directly in the
tests/
directory (not intests/src/
)
If your tests aren't running, ensure they follow this structure.