Skip to content

Commit f2437d1

Browse files
authored
Disable parquet_encryption by default in datafusion-sqllogictests (#18492)
## Which issue does this PR close? - Closes [#18490](#18490) ## Rationale for this change When our internal project to use datafusion, we found it default to enable encryption even for latest datafusion. Problem Analysis In datafusion/sqllogictest/Cargo.toml: ```rust datafusion = { workspace = true, default-features = true, features = ["avro", "parquet_encryption"] } ``` The Problem: datafusion-sqllogictest depends on datafusion with default-features = true And, sqllogictest is the member of datafusion, it will default to encryption when we use datafusion. ```rust [workspace] members = [ "datafusion/common", "datafusion/common-runtime", "datafusion/catalog", "datafusion/catalog-listing", "datafusion/datasource", "datafusion/datasource-avro", "datafusion/datasource-csv", "datafusion/datasource-json", "datafusion/datasource-parquet", "datafusion/core", "datafusion/expr", "datafusion/expr-common", "datafusion/execution", "datafusion/ffi", "datafusion/functions", "datafusion/functions-aggregate", "datafusion/functions-aggregate-common", "datafusion/functions-table", "datafusion/functions-nested", "datafusion/functions-window", "datafusion/functions-window-common", "datafusion/optimizer", "datafusion/physical-expr", "datafusion/physical-expr-adapter", "datafusion/physical-expr-common", "datafusion/physical-optimizer", "datafusion/pruning", "datafusion/physical-plan", "datafusion/proto", "datafusion/proto/gen", "datafusion/proto-common", "datafusion/proto-common/gen", "datafusion/session", "datafusion/spark", "datafusion/sql", "datafusion/sqllogictest", "datafusion/substrait", "datafusion-cli", "datafusion-examples", "datafusion-examples/examples/ffi/ffi_example_table_provider", "datafusion-examples/examples/ffi/ffi_module_interface", "datafusion-examples/examples/ffi/ffi_module_loader", "test-utils", "benchmarks", "datafusion/macros", "datafusion/doc", ] exclude = ["dev/depcheck"] resolver = "2" ``` ## What changes are included in this PR? Fixed above. ## Are these changes tested? Yes ## Are there any user-facing changes? Make encryption a feature instead of default.
1 parent d679b3b commit f2437d1

File tree

3 files changed

+6
-3
lines changed

3 files changed

+6
-3
lines changed

.github/workflows/extended.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -169,7 +169,7 @@ jobs:
169169
rust-version: stable
170170
- name: Run sqllogictest
171171
run: |
172-
cargo test --features backtrace --profile release-nonlto --test sqllogictests -- --include-sqlite
172+
cargo test --features backtrace,parquet_encryption --profile release-nonlto --test sqllogictests -- --include-sqlite
173173
cargo clean
174174
175175

.github/workflows/rust.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -475,7 +475,7 @@ jobs:
475475
export RUST_MIN_STACK=20971520
476476
export TPCH_DATA=`realpath datafusion/sqllogictest/test_files/tpch/data`
477477
cargo test plan_q --package datafusion-benchmarks --profile ci --features=ci -- --test-threads=1
478-
INCLUDE_TPCH=true cargo test --features backtrace --profile ci --package datafusion-sqllogictest --test sqllogictests
478+
INCLUDE_TPCH=true cargo test --features backtrace,parquet_encryption --profile ci --package datafusion-sqllogictest --test sqllogictests
479479
- name: Verify Working Directory Clean
480480
run: git diff --exit-code
481481

datafusion/sqllogictest/Cargo.toml

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ bigdecimal = { workspace = true }
4343
bytes = { workspace = true, optional = true }
4444
chrono = { workspace = true, optional = true }
4545
clap = { version = "4.5.50", features = ["derive", "env"] }
46-
datafusion = { workspace = true, default-features = true, features = ["avro", "parquet_encryption"] }
46+
datafusion = { workspace = true, default-features = true, features = ["avro"] }
4747
datafusion-spark = { workspace = true, default-features = true }
4848
datafusion-substrait = { workspace = true, default-features = true }
4949
futures = { workspace = true }
@@ -78,6 +78,9 @@ postgres = [
7878
"testcontainers-modules",
7979
"tokio-postgres",
8080
]
81+
parquet_encryption = [
82+
"datafusion/parquet_encryption",
83+
]
8184

8285
[dev-dependencies]
8386
env_logger = { workspace = true }

0 commit comments

Comments
 (0)