Dataflow Templates 2024-12-03-00_RC00

What's Changed

Improvements

• Allow SourceSchemaReference and DataSource to represent Cassandra (and other similar non-jdbc) values by @VardhanThigle in #1998
• fix: add yaml support for specification parser by @Emrehzl94 in #1999
• Add enable backfill flag to terraform template var files by @Deep1998 in #2018
• Avro format Import/Export support for Spanner NOT ENFORCED Foreign Key by @jennyjiang98 in #2031
• Upgrade to Beam 2.61.0 by @damccorm in #2033
• add KMS key support to BigQueryConverters by @Polber in #2024
• feat: Modularization: Terraform changes for schema overrides by @manitgupta in #1841
• Add support for PropertyGraphs in import/export DDL by @smukil in #2003
• Add DLQ support to JdbcToBigQuery using BQ Storage Write API by @Polber in #2016
• Parameterizing spanner Host in spanner tests by @darshan-sj in #2020

Bug fixes

• Removing unsued Cassandra dependency by @VardhanThigle in #2009
• Update requirements annotation for JdbcToBigQuery templates by @Abacn in #2007
• Use safe deserialization option for Yaml specs by @fbiville in #2030
• Fix node target reference lookup and node match mode by @fbiville in #1961
• Allowing Jdbc FetchSize Configuration for handling large rows by @VardhanThigle in #2028
• Fixing Custom Transforms for BigInt Unsigned and Boolean types. by @VardhanThigle in #2039
• Setting useCursorFetch for MySql if FetchSize is configured. by @VardhanThigle in #2041
• [KafkaToBigQueryFlex Template]: Add support for Avro ENUM type and fix FLOAT type. by @an2x in #2029

New Contributors

• @jennyjiang98 made their first contribution in #2031
• @smukil made their first contribution in #2003

Full Changelog: 2024-11-12-00_RC00...2024-12-03-00_RC00