Dataflow Templates 2025-06-24-00_RC00

New Contributors

• @gh-cli made their first contribution in #2344

Full Changelog: 2025-06-17-00_RC01...2025-06-24-00_RC00

Dataflow Templates 2025-06-22-00_RC01

Cancel the streaming ITs when using PubSub not drain (#2474)

* Cancel the streaming ITs when using PubSub not drain

* try us-west2

Dataflow Templates 2025-06-17-00_RC01

What's Changed

Improvements

• Update Spanner utility class methods to use the rpcPriority template option when making Spanner RPCs by @tcmcheung in #2423
• Mark Spanner_change_stream_to_shareded_file_sink and Gcs_to_sourcedb templates as Deprecated by @darshan-sj in #2461
• feat: Remove migration_shard_id check from AssignShardIdFn by @manitgupta in #2467
• feat: Handle non-PK tables gracefully by @manitgupta in #2470

Bug fixes

• Bulk migration lt for 10mb row by @asthamohta in #2462
• Upgrade Beam SDK to 2.65.0 by @leongn-google in #2471

New Contributors

• @tcmcheung made their first contribution in #2423
• @leongn-google made their first contribution in #2471

Full Changelog: 2025-06-10-01_RC00...2025-06-17-00_RC01

Dataflow Templates 2025-06-17-00_RC00

Use private IPs in Spanner to Sourcdb template ITs (#2463)

Dataflow Templates 2025-06-10-01_RC00

What's Changed

Improvements

• [PropertyGraph] Explicitly output node reference key columns by @smukil in #2262
• exposing num threads as option by @joy91227 in #2349
• feat: use SLF4J for Neo4j driver logs by @fbiville in #2285

Bug fixes

• Adding Option to enable Uniformization stages by @VardhanThigle in #2381
• Renable CassandraAllDataTypesIT by @VardhanThigle in #2395

New Contributors

• @henrikno made their first contribution in #2343

Full Changelog: 2025-05-27-00_RC00...2025-06-10-01_RC00

Dataflow Templates 2025-05-27-00_RC00

What's Changed

New Templates

• Add new template Cloud_Datastream_MongoDB_to_Firestore to support migration from MongoDb to Firestore by @ninjaAB-5 in #2374

Improvements

• Use exluded semantics in PG update SQL by @dhercher in #2366
• Adding BigQuery to ClickHouse Template by @BentsiLeviav in #2210
• Bulk schema overrides by @Deep1998 in #2376
• Added PG support for vector indexes by @atask-g in #2361
• test: Add end to end forward migration followed by reverse migration by @asthamohta in #2240

Bug fixes

• Temporarily disabling Cassandra.allDataTypesTest() until fix for apache/beam#34863 gets merged into dataflow. by @VardhanThigle in #2386
• Fix the query to exclude UDFs like change_streams by @rajkgupt in #2378

New Contributors

• @BentsiLeviav made their first contribution in #2210
• @ninjaAB-5 made their first contribution in #2374

Full Changelog: 2025-05-13-00_RC01...2025-05-27-00_RC00

Dataflow Templates 2025-05-13-00_RC01

What's Changed

Improvements

• Add UDF support to import/export workflows by @MnkyGns in #2253
• Upgrade to Beam 2.65.0 by @akashorabek in #2369

New Contributors

• @MnkyGns made their first contribution in #2253

Full Changelog: 2025-05-06-00_RC01...2025-05-13-00_RC01

Dataflow Templates 2025-05-06-00_RC01

What's Changed

Improvements

• feat: add interleave in feature to optimise bulk migration by @asthamohta in #2295

Bug fixes

• feat(bigtable): Add support for ignore columns by @ron-gal in #2328
• feat(bigtable): Send invalid elements to non retriable DLQ by @ron-gal in #2340

Full Changelog: 2025-04-29-00_RC00...2025-05-06-00_RC01

Dataflow Templates 2025-05-06-00_RC00

Release week of 2025-05-05

Dataflow Templates 2025-04-29-00_RC00

What's Changed

Improvements

• added beam-sdks-java-extensions-sql-expansion-service by @liferoad in #2337

Bug fixes

• Add rowRestriction when fetching BQ schema by @joy91227 in #2273
• Fixing TextImportPipelineIT integration test by @darshan-sj in #2327
• Update docs for inputFilePattern by @manitgupta in #2339
• Add Seperate VPC for Widerow tests by @taherkl in #2335

New Contributors

• @joy91227 made their first contribution in #2273
• @pawankashyapollion made their first contribution in #2300

Full Changelog: 2025-04-15-00_RC01...2025-04-29-00_RC00