Update Databricks integration docs to reflect new OAuth auth method

[rkaneriya]

What does this PR do?

Updates the Databricks integration docs to include the new OAuth auth method (and marks the Personal Access Token method as "legacy"). This achieves parity with the public docs for the DJM <> Databricks integration (documentation markdown here).

Motivation

Now that OAuth has been 100% rolled out for the Databricks integration, we should update the docs to reflect the fact that using Personal Access Tokens is a legacy approach and only available for workspaces created before today. New workspaces must use OAuth. The DJM <> Databricks docs in the "documentation" repo have already been updated. This PR updates the overall Databricks integration docs.

Review checklist (to be filled by reviewers)

• Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
• Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
• If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

[aliciascott]

looks good!

[rkaneriya]

/merge

[dd-devflow]

View all feedbacks in Devflow UI.

2025-07-07 17:09:58 UTC ℹ️ Start processing command /merge

---

2025-07-07 17:10:11 UTC ℹ️ MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

---

2025-07-07 17:31:45 UTC ⚠️ MergeQueue: This merge request was unqueued

[email protected] unqueued this merge request

[rkaneriya]

/merge -c

[dd-devflow]

View all feedbacks in Devflow UI.

2025-07-07 17:31:39 UTC ℹ️ Start processing command /merge -c

[rkaneriya]

/merge

[dd-devflow]

View all feedbacks in Devflow UI.

2025-07-07 19:32:29 UTC ℹ️ Start processing command /merge

---

2025-07-07 19:32:40 UTC ℹ️ MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

---

2025-07-07 20:38:33 UTC ℹ️ MergeQueue: This merge request was already merged

This pull request was merged directly.