Skip to content

Make the SerializedDAG & SerializedBaseOp be independent of Task SDK's DAG & BaseOP #52141

@kaxil

Description

@kaxil

Currently, models.BaseOperator and models.DAG use the Task SDK classes.

This was done to maintain backwards-compat and to get 3.0 in a timely manner.

However, we don't want to depend on Task SDK in the Scheduler or the API-server (airflow-core package) - at least not a required dependency.

We only need to ensure we have enough information for the Scheduler & API-server for a DAG & the Operator. This is done via the SerializedDAG & SerializedBaseOp.

So this task involves: figuring out how we can consolidate: SerializedDAG, models.DAG by not inheriting Task SDK; and same for SerializedBaseOp and models.BaseOperator

To do (not ordered by dependency):

  • Timetables
  • Asset
  • DagNode
  • NOTSET and ArgNotSet
  • DeadlineAlert (and releted things?)
  • BaseOperatorLink
  • ExpandInput

Metadata

Metadata

Assignees

Type

No type

Projects

Status

In Progress

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions