-
Notifications
You must be signed in to change notification settings - Fork 16.3k
Closed
Labels
area:providerskind:metaHigh-level information important to the communityHigh-level information important to the communitytesting statusStatus of testing releasesStatus of testing releases
Description
Body
I have a kind request for all the contributors to the latest provider packages release.
Could you please help us to test the RC versions of the providers?
The guidelines on how to test providers can be found in
Verify providers by contributors
Let us know in the comments, whether the issue is addressed.
These are providers that require testing as there were some substantial changes introduced:
Provider amazon: 9.4.0rc1
- Add MwaaTriggerDagRunOperator and MwaaHook to Amazon Provider Package (#46579): @ramitkataria
- Adding extra links for EC2 (#46340): @ellisms
- Allow to pass container_name parameter to EcsRunTaskOperator (#46152): @eliskovets
Linked issues: - Adding DataSync links (#46292): @ellisms
- Added extra links for Comprehend operators (#46031): @ellisms
- Add support for timeout to BatchOperator (#45660): @nrobinson-intelycare
- Adding SageMaker Transform extra link (#45677): @ellisms
Linked issues: - Add MessageDeduplicationId support to AWS SqsPublishOperator (#45051): @Prab-27
Linked issues: - Rework the TriggererJobRunner to run triggers in a process without DB access (#46677): @ashb
- Fix schema path in AWS auth manager system test due to restructure (#46625): @vincbeck
- Increase retries in
EmrContainerHook.create_emr_on_eks_cluster(#46562): @vincbeck
Linked issues: - Update
create_emr_on_eks_clustermethod to try when "cluster is not reachable as its connection is currently being updated" (#46497): @vincbeck
Linked issues: - Generate partition aware STS endpoints for EKS Hook (#45725): @o-nikolas
Linked issues: - Sagemaker Operator Character limit fix (#45551): @dirkrkotzeml
Linked issues: - Fix
fetch_access_token_for_clusterin EKS hook (#45469): @vincbeck
Linked issues: - Update system test
example_dms_serverless(#46684): @vincbeck - Update
example_sqsto not uselogical_date(#46696): @vincbeck - Fix 3.12 failing providers tests of amazon emr hook (#46612): @insomnes
- Update AWS auth manager to use Fastapi instead of Flask (#46381): @vincbeck
- Swap CeleryExecutor over to use TaskSDK for execution. (#46265): @ashb
Linked issues: - Make parameter
usermandatory for all methods in the auth manager interface (#45986): @vincbeck - Add
run_job_kwargsas templated field inGlueJobOperator(#45973): @eladkal - Use Protocol for
OutletEventAccessor(#45762): @kaxil - Remove classes from
typing_compatthat can be imported directly (#45589): @kaxil - Move Literal alias into TYPE_CHECKING block (#45345): @uranusjr
- Remove marshmallow version restriction; update deprecated usages (#45499): @sloria
Linked issues: - Remove obsolete pandas specfication for pre-python 3.9 (#45399): @potiuk
- Add option in auth manager interface to define FastAPI api (#45009): @vincbeck
Linked issues:
Provider apache.beam: 6.0.2rc1
- Fix DataflowJobLink for Beam operators in deferrable mode (#45023): @MaksYermak
- Set minimum dependencies for apache-beam deps on Py 3.12+3.13 (#46321): @ashb
- Limit Apache Beam's numpy used (#46286): @potiuk
- Refactor deferrable mode for BeamRunPythonPipelineOperator and BeamRunJavaPipelineOperator (#46678): @MaksYermak
Provider apache.hive: 9.0.2rc1
Provider apache.livy: 4.2.0rc1
Provider celery: 3.10.2rc1
- Fixing log typos in Local & Celery Executors (#46866): @amoghrajesh
- Rework the TriggererJobRunner to run triggers in a process without DB access (#46677): @ashb
- AIP-66: Make DAG callbacks bundle aware (#45860): @ephraimbuddy
- Swap CeleryExecutor over to use TaskSDK for execution. (#46265): @ashb
Linked issues: - Remove 2.8 version check from CeleryExecutor CLI (#46910): @jedcunningham
Provider cncf.kubernetes: 10.3.0rc1
- KubernetesPodOperator new callbacks and allow multiple callbacks (#44357): @johnhoran
- Render a pod spec using the
pod_template_fileoverride, if passed to the executor (#46374): @brouberol
Linked issues: - fix: kpo async kube_config_path (#45571): @raphaelauv
- Allow passing empty labels in the spark kubernetes driver config (#45864): @logarithm
- fix: spark operator label (#45353): @chenkovsky
- Allow internal retries when pending k8s pod is deleted (#45184): @karenbraganz
Linked issues: - fix(cncf-kubernetes): correctly shutdown xcom after job is complete (#45455): @kjuulh
Linked issues: - Fix kubernetes executor watcher kube_client_request_args shadowing (#45528): @insomnes
Linked issues: - Mini-fix: use right operator name in doc string of
@task.kubernetes(#46873): @insomnes - AIP-83 amendment: Add logic for generating run_id when logical date is None. (#46616): @sunank200
- Remove overlooked
pod_generator_deprecated.py(#46587): @insomnes - More meaningful
@task.kubernetespod naming (#46535): @insomnes
Linked issues: - chore(kubernetes): reduce redundant code (#46574): @hussein-awala
- Add dynamic task mapping into TaskSDK runtime (#46032): @ashb
Linked issues: - Expand KPO template_fields, fix Spark k8s operator tests (#46268): @insomnes
Linked issues: - Swap CeleryExecutor over to use TaskSDK for execution. (#46265): @ashb
Linked issues: - Move Pod*Exceptions to separate module to avoid unnecessary slow imports in CLI (#45759): @IKholopov
- Add other instances of Context type hints (#45657): @kaxil
- AIP-72: Swap KubernetesExecutor to use taskSDK for execution (#46860): @amoghrajesh
Linked issues:
Provider cohere: 1.4.2rc1
- Update Cohere to 5.13.4 v2 API (#45267): @okirialbert
Linked issues:
Provider common.compat: 1.5.0rc1
- feat: automatically inject OL transport info into spark jobs (#45326): @kacpermuda
Linked issues: - feat: Add OpenLineage support for some SQL to GCS operators (#45242): @kacpermuda
- fix: OL sql parsing add try-except for sqlalchemy engine (#46366): @kacpermuda
- Remove old lineage stuff (#45260): @jason810496
Linked issues:
Provider common.sql: 1.23.0rc1
- Introduce notion of dialects in DbApiHook (#41327): @dabla
- Fix DbApiHook.insert_rows logging incorrect number of rows commited (#45789): @plutaniano
- Fix escaping of special characters or reserved words as column names in dialects of common sql provider (#45640): @dabla
- Add run_after column to DagRun model (#45732): @uranusjr
- Adding type ignore to sql hook (#46163): @amoghrajesh
Linked issues: - Added ADR document describing why the notion of dialects was introduced in the common sql provider (#45456): @dabla
- Bring back mypy checks for new-structure providers (#45815): @potiuk
- Remove obsolete pandas specfication for pre-python 3.9 (#45399): @potiuk
Provider databricks: 7.2.0rc1
- Accept task_key as an argument in
DatabricksNotebookOperator(#44960): @hardeybisey
Linked issues: - python_named_parameters should be python_named_params (#46299): @lendle
- Remove obsolete pandas specfication for pre-python 3.9 (#45399): @potiuk
Provider dbt.cloud: 4.2.0rc1
- New Optional dbt Cloud Job Operator Params (#45634): @ginone
- Add missing newline on dbt Cloud conn string example (#45603): @joellabes
- Remove classes from
typing_compatthat can be imported directly (#45589): @kaxil
Provider docker: 4.2.0rc1
- Feat: support docker operator arg
labels(#46643): @phi-friday - Fix wrong f-string to satisfy ruff static check (#46813): @insomnes
- Using quotes for file path in find command (#46795): @amoghrajesh
- Update DockerSwarmOperator auto_remove to align with DockerOperator (#45745): @niklasr22
- Using env for file path in find command (#46809): @Vincent550102
- Move Literal alias into TYPE_CHECKING block (#45345): @uranusjr
Provider elasticsearch: 6.2.0rc1
- Implemented cursor for ElasticsearchSQLHook so it can be used through SQLExecuteQueryOperator (#46439): @dabla
- Add write feature to ESTaskHandler (#44973): @Owen-CH-Leung
Linked issues:- Linked Issue #42780: @Owen-CH-Leung
- Start porting mapped task to SDK (#45627): @ashb
- Update docs for passing any valid parameters to Elasticsearch client (#45263): @jason810496
Linked issues:- Linked Issue #44724: @benjamindinh-loreal
Provider exasol: 4.7.2rc1
- Fix rowcount logging in exasol provider (#44022) (#45760): @Briimbo
Linked issues:- Linked Issue #44022: @XtremeBaumer
Provider google: 14.0.0rc1
- Remove deprecated functionality from google provider (#46235): @moiseenkov
- Create operators for working with Clusters for GCP Apache Kafka (#45795): @MaksYermak
- Deprecate BigQueryCreateExternalTableOperator and BigQueryCreateEmptyTableOperator and introduce new BigQueryCreateTableOperator (#46673): @Vlada
- Add new Dataplex Catalog operators for Entry resource (#46647): @VladaZakharova
- Add deferrable mode to the PubSubPullOperator (#45835): @molcay
- Add deferrable mode to google cloud storage transfer sensor and operators (#45754): @tnk-ysk
- Add new Dataplex Catalog Aspect Type operators (#46240): @VladaZakharova
- feat: Add OpenLineage support for BigQueryDataTransferServiceStartTransferRunsOperator (#45801): @kacpermuda
- feat: Add OpenLineage support for CloudSQLExecuteQueryOperator (#45182): @kacpermuda
- Add new Dataplex Catalog Entry Type operators (#45799): @VladaZakharova
- Add new Dataplex Catalog Entry Group operators (#45751): @VladaZakharova
- feat: Add Hook Level Lineage support for BigQueryHook (#45706): @kacpermuda
Linked issues: - feat: Add OpenLineage support for MsSqlHook and MSSQLToGCSOperator (#45637): @kacpermuda
- Adding holiday_region parameter to create_auto_ml_forecasting_training_job in AutoMl hook (#45465): @vinay2242g
- feat: automatically inject OL transport info into spark jobs (#45326): @kacpermuda
Linked issues: - feat: Add OpenLineage support for transfer operators between GCS and SFTP (#45485): @kacpermuda
- Implement AlloyDB create/update/delete user and backups operators (#45378): @moiseenkov
- feat: Add OpenLineage support for some SQL to GCS operators (#45242): @kacpermuda
- feat: Add OpenLineage support for some BQ operators (#45422): @kacpermuda
- feat: Add OpenLineage support for non-query jobs in BigQueryInsertJobOperator (#44996): @kacpermuda
- feat: automatically inject OL info into spark properties in DataprocInstantiateInlineWorkflowTemplateOperator (#44697): @kacpermuda
Linked issues: - feat: Add CLL to OpenLineage in BigQueryInsertJobOperator (#44872): @kacpermuda
- feat: automatically inject OL info into spark properties in DataprocCreateBatchOperator (#44612): @kacpermuda
Linked issues: - Implement AlloyDB create/update/delete instance operators (#45241): @moiseenkov
- [main] Add BigQuery job link (#45020) (#45222): @shahar1
Linked issues:- Linked Issue #45020: @nakamura1878
- Introduce gcp translation (V3) glossaries providers (#45085): @olegkachur-e
- Add dependencies section to gcp cloud build instance response (#46947): @gopidesupavan
- Create operators for working with Topics for GCP Apache Kafka (#46865): @MaksYermak
- Fix BigQueryInsertJobOperator job_id return bug (#46753): @nakamura1878
- fix: Dataproc operators fail to import without OpenLineage (#46561): @kacpermuda
Linked issues: - Add backward compatibility for old Airflow versions for CloudComposerDAGRunSensor (#46011): @MaksYermak
- Improve google credentials error message (#45553): @olegkachur-e
- FIX: add case of aarch64 environments for cloud-sql-proxy binaries (#45488): @mdering
- fix GCSToGCSOperator bug when copy single object with replace to False (#45181): @NatchapolLaow
- Rework the TriggererJobRunner to run triggers in a process without DB access (#46677): @ashb
- Remove Airflow 3 Deprecation Warning in Google Provider, Move to Private Warning (#46599): @jscheffl
- Add evaluation extra to google-cloud-aiplatform (#46270): @potiuk
Linked issues:- Linked Issue #46258: @jscheffl
- Limit
google-cloud-aiplatformto fix issues in CI (#46242): @amoghrajesh - Improve grammar in cloud_composer.rst (#45708): @aaabramov
- Deprecate page_size parameter for Google Ads API v17 and later versions (#45239): @molcay
- Remove obsolete pandas specfication for pre-python 3.9 (#45399): @potiuk
- Deprecation policy for apache-airflow-providers-google package (#44985): @VladaZakharova
- Add lower bound for alloydb (#45180): @potiuk
- Deprecate gcp AutoML module (#44875): @olegkachur-e
Linked issues:- Linked Issue #38636: @shahar1
- Refactor deferrable mode for BeamRunPythonPipelineOperator and BeamRunJavaPipelineOperator (#46678): @MaksYermak
- Bump cloud build to 3.31.0 (#46953): @gopidesupavan
Provider http: 5.2.0rc1
- Deferrable support for HttpOperator (#45228): @TakayukiTanabeSS
- Allow check_response to be defined in extra_options of HTTP connection (#45451): @dabla
Provider microsoft.azure: 12.2.0rc1
- Added support for certificate authentication with MSGraphAsyncOperator (#45935): @dabla
- fix:
KiotaRequestAdapterHookmake sure proxy config parameter is parsed correctly, even if it's a string or json (#46145): @dabla - Fixed retry of PowerBIDatasetRefreshOperator when dataset refresh wasn't directly available (#45513): @Ohashiro
- Fixes compat issue HTTPX proxy configuration in KiotaRequestAdapterHook and fixed retry in MSGraphSensor (#45746): @dabla
- Fixed assignment of proxies in KiotaRequestAdapter for HTTPX 0.28.x compliancy (#45741): @dabla
- Updated instantiation of httpx AsyncClient in KiotaRequestAdapterHook to be httpx 0.28.x compliant (#45464): @dabla
- Fixing return type for azure hook 'run_query' (#45470): @amoghrajesh
- Refactor result_processor and event_handler signatures in MSGraphAsyncOperator (#46637): @dabla
- AIP-83 amendment: Add logic for generating run_id when logical date is None. (#46616): @sunank200
- Add dynamic task mapping into TaskSDK runtime (#46032): @ashb
Linked issues: - Move update subscription from
AzureServiceBusTopicCreateOperatortoAdminClientHook(#45367): @perry2of5 - Move create topic to hook in Azure Service Bus provider (#45297): @perry2of5
- Refactor Azure Service Bus: move create_subscription to hook (#45125): @perry2of5
Provider microsoft.mssql: 4.2.0rc1
- feat: Add OpenLineage support for MsSqlHook and MSSQLToGCSOperator (#45637): @kacpermuda
- Introduce notion of dialects in DbApiHook (#41327): @dabla
- Fixed generated SQL MSSQL dialect when all columns are also primary keys (#46380): @dabla
- Fix escaping of special characters or reserved words as column names in dialects of common sql provider (#45640): @dabla
- Added ADR document describing why the notion of dialects was introduced in the common sql provider (#45456): @dabla
Provider microsoft.winrm: 3.9.0rc1
Provider mongo: 5.0.1rc1
- Fix side effect of not tearing down mongo hook in tests (#46291): @potiuk
- Fix pymongo generated dependencies not updated in pymongo fix (#46220): @potiuk
Linked issues:- Linked Issue #46216: @jscheffl
- Bugfix/prevent pymongo 4.11 (#46216): @jscheffl
- Disallow pymongo: 4.11.1 to fix CI tests (#46636): @amoghrajesh
Provider mysql: 6.1.0rc1
- Introduce notion of dialects in DbApiHook (#41327): @dabla
- Fix escaping of special characters or reserved words as column names in dialects of common sql provider (#45640): @dabla
Provider openlineage: 2.1.0rc1
- AIP-72: Task SDK support for on_task_instance_* listeners, make OpenLineage compatible (#45294): @mobuchowski
- feat: Add ProcessingEngineRunFacet to all OL events (#46283): @kacpermuda
Linked issues:- Linked Issue #43213: @mobuchowski
- feat: automatically inject OL transport info into spark jobs (#45326): @kacpermuda
Linked issues: - feat: Add OpenLineage support for some SQL to GCS operators (#45242): @kacpermuda
- feat: automatically inject OL info into spark properties in DataprocCreateBatchOperator (#44612): @kacpermuda
Linked issues: - Update OpenLineage emmiter to cope with nullable logical_date (#46722): @ashb
Linked issues: - fix: OL sql parsing add try-except for sqlalchemy engine (#46366): @kacpermuda
- include AirflowDagRunFacet in complete/failed OpenLineage events (#45615): @mobuchowski
- Adding uuid6 as a dependency for openlineage (#46653): @amoghrajesh
Linked issues:- Linked Issue #45294: @mobuchowski
- Remove AirflowContextDeprecationWarning as all context should be clean for Airflow 3 (#46601): @jscheffl
Linked issues:- Linked Issue #46600: @jscheffl
- Remove Airflow 3 Deprecation Warning dependency in OTel Provider (#46600): @jscheffl
- Add run_after column to DagRun model (#45732): @uranusjr
- Remove old lineage stuff (#45260): @jason810496
Linked issues: - Start porting mapped task to SDK (#45627): @ashb
- chore: Update docstring for DatabaseInfo in OpenLineage provider (#45638): @kacpermuda
- Remove classes from
typing_compatthat can be imported directly (#45589): @kaxil - [docs openlineage] updated outdated hyperlink to GcsToGcsOperator (#45311): @rahul-madaan
- pass error for on_task_instance_failed in task sdk (#46941): @mobuchowski
Provider opensearch: 1.6.1rc1
Provider pagerduty: 4.0.1rc1
- (bugfix): replace
create_eventwithsend_eventinPagerdutyNotifier(#45630): @bshea5
Linked issues:- Linked Issue #45626: @bshea5
Provider papermill: 3.9.1rc1
- Ignore a semi-random error generated by Mypy (#46123): @potiuk
- Remove old lineage stuff (#45260): @jason810496
Linked issues: - Remove obsolete pandas specfication for pre-python 3.9 (#45399): @potiuk
Provider postgres: 6.1.0rc1
- Introduce notion of dialects in DbApiHook (#41327): @dabla
- Fix escaping of special characters or reserved words as column names in dialects of common sql provider (#45640): @dabla
- Bump psycopg2 to 2.9.9 to possibly avoid crash on Python 3.12 (#46431): @potiuk
- Added ADR document describing why the notion of dialects was introduced in the common sql provider (#45456): @dabla
- Bump minimum version of psycopg2-binary to 2.9.7 (#45635): @potiuk
Provider presto: 5.8.1rc1
Provider redis: 4.0.1rc1
Provider salesforce: 5.10.0rc1
- Implement fetching consumer_secret from Salesforce connection (#45954): @kanagaraj-dhanapal-89
- Remove obsolete pandas specfication for pre-python 3.9 (#45399): @potiuk
Provider sftp: 5.1.0rc1
- Add DELETE operation in SFTPOperator (#46233): @Dawnpool
- Also allow passing buffer instead of path for retrieve_file and store_file methods in SFTPHook (#44247): @dabla
- Add directory transfer support for SFTPOperator (#44126): @Dawnpool
- Make sure the SSHClient is also closed when using connection in context manager from SFTPHook (#46716): @dabla
Provider smtp: 2.0.0rc1
- BREAKING CHANGE: replace Airflow config by conx extras in SMTP provider (#46219): @hussein-awala
- feat(smtp): support html_content and subject templates from SMTP connection (#46212): @hussein-awala
Provider snowflake: 6.1.0rc1
- SnowflakeSqlApiOperator: add snowflake_conn_id to template_fields (#46422): @bbossy
Linked issues:- Linked Issue #46421: @bbossy
- [snowflake] Add ocsp_fail_open setting for SnowflakeHook (#46476): @jcrobak
- AIP-83 amendment: Add logic for generating run_id when logical date is None. (#46616): @sunank200
- Remove obsolete pandas specfication for pre-python 3.9 (#45399): @potiuk
Provider sqlite: 4.0.0rc1
- Remove Provider Deprecations in Sqlite (#44707): @jscheffl
- Add basic asyncio support (#43944): @dstandish
Provider ssh: 4.0.0rc1
- Remove Provider Deprecations in SSH (#44544): @jscheffl
- Add host_proxy_cmd parameter to SSHHook and SFTPHook (#44565): @ajitg25
- Remove XCom pickling (#43905): @kaxil
Provider standard: 0.1.0rc1
- AIP-82 Introduce
BaseEventTriggeras base class for triggers used with event driven scheduling (#46391): @vincbeck - AIP-83 amendment: Add logic for generating run_id when logical date is None. (#46616): @sunank200
- TriggerDagRunOperator by defaults set logical date as null (#46633): @vatsrahul1001
- Use run_id for ExternalDag and TriggerDagRun links (#46546): @uranusjr
- AIP-72: Task SDK support for on_task_instance_* listeners, make OpenLineage compatible (#45294): @mobuchowski
- Remove AirflowContextDeprecationWarning as all context should be clean for Airflow 3 (#46601): @jscheffl
Linked issues:- Linked Issue #46600: @jscheffl
- refactor(utils/decorators): rewrite remove task decorator to use cst (#43383): @josix
Linked issues:- Linked Issue #42766: @jscheffl
- Add dynamic task mapping into TaskSDK runtime (#46032): @ashb
Linked issues: - Moving EmptyOperator to standard provider (#46231): @amoghrajesh
- Add run_after column to DagRun model (#45732): @uranusjr
- Removing feature: send context in venv operators (using
use_airflow_context) (#46306): @amoghrajesh
Linked issues: - Remove import from MySQL provider tests in generic transfer test (#46274): @potiuk
- Fix failures on main related to DagRun validation (#45917): @kaxil
- Start porting mapped task to SDK (#45627): @ashb
- Remove code for deprecation of Context keys (#45585): @kaxil
- Move Literal alias into TYPE_CHECKING block (#45345): @uranusjr
- AIP-72: Add Taskflow API support & template rendering in Task SDK (#45444): @kaxil
- Remove tuple_in_condition helpers (#45201): @uranusjr
Provider trino: 6.0.1rc1
Provider weaviate: 3.0.1rc1
- Move Literal alias into TYPE_CHECKING block (#45345): @uranusjr
- Remove obsolete pandas specfication for pre-python 3.9 (#45399): @potiuk
All users involved in the PRs:
@lendle @darkag @o-nikolas @perry2of5 @amoghrajesh @SuccessMoses @joellabes @MaksYermak @kanagaraj-dhanapal-89 @vatsrahul1001 @plutaniano @kacpermuda @potiuk @chenkovsky @phi-friday @olegkachur-e @kunal
Committer
- I acknowledge that I am a maintainer/committer of the Apache Airflow project.
nrobinson-intelycare and potiuk
Metadata
Metadata
Assignees
Labels
area:providerskind:metaHigh-level information important to the communityHigh-level information important to the communitytesting statusStatus of testing releasesStatus of testing releases