Skip to content

Conversation

@kacpermuda
Copy link
Contributor

@kacpermuda kacpermuda commented Jan 13, 2026

Documentation improvements for OpenLineage provider

This PR improves the OpenLineage provider documentation with better organization and expanded content:

  • spark.rst: Added separate page for Spark integration, explaining that it's a separate entity from the Airflow provider and requires its own installation and configuration.

  • macros.rst: Expanded to be the central documentation for job hierarchy and macros, consolidating information about cross-job dependencies (TriggerDagRunOperator, API triggers, ExternalTaskSensor, Airflow Assets) and how OpenLineage handles job relationships in each scenario.

  • troubleshooting.rst: Added troubleshooting page with best practices and common errors, where users trying to debug OL problems can find helpful information.

  • supported_classes.rst: Added explanation of what "supported" means, clarifying that all operators emit basic lineage while "supported" operators provide additional operator-specific metadata.

  • guides/structure.rst: Added paragraph explaining the difference between the openlineage-python client package and the apache-airflow-providers-openlineage provider package.

  • guides/developer.rst: Added section about helper functions e.g. emit_openlineage_events_for_databricks_queries

  • provider.yml: Reordered options so that they are alphabetical.

  • guides/user.rst: Moved some basic stuff to structure.rst and dev stuff to developer.rst, nothing was left so this file was removed.

Was generative AI tooling used to co-author this PR?

  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@kacpermuda kacpermuda force-pushed the feat-ol-troubleshooting-doc branch from bd74367 to 32da1cb Compare January 13, 2026 16:31
@kacpermuda kacpermuda marked this pull request as ready for review January 13, 2026 16:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant