Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for clusters in deployment bind/unbind commands #2536

Merged
merged 3 commits into from
Mar 21, 2025
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Add support for clusters in deployment bind/unbind commands
anton-107 committed Mar 20, 2025
commit e357296fe9e1da33dbbee4b49af61456466f34a4
1 change: 1 addition & 0 deletions NEXT_CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -7,5 +7,6 @@
### Bundles

* Processing 'artifacts' section is now done in "bundle validate" (adding defaults, inferring "build", asserting required fields) ([#2526])(https://github.com/databricks/cli/pull/2526))
* Add support for clusters in deployment bind/unbind commands ([#2536](https://github.com/databricks/cli/pull/2536))

### API Changes
4 changes: 4 additions & 0 deletions acceptance/acceptance_test.go
Original file line number Diff line number Diff line change
@@ -290,6 +290,10 @@ func getSkipReason(config *internal.TestConfig, configPath string) string {
return fmt.Sprintf("Disabled via RequiresUnityCatalog setting in %s (TEST_METASTORE_ID=%s)", configPath, os.Getenv("TEST_METASTORE_ID"))
}

if isTruePtr(config.RequiresCluster) && os.Getenv("TEST_DEFAULT_CLUSTER_ID") == "" {
return fmt.Sprintf("Disabled via RequiresCluster setting in %s (TEST_DEFAULT_CLUSTER_ID=%s)", configPath, os.Getenv("TEST_DEFAULT_CLUSTER_ID"))
}

} else {
// Local run
if !isTruePtr(config.Local) {
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
bundle:
name: bind-cluster-test-$BUNDLE_NAME_SUFFIX

resources:
clusters:
cluster1:
cluster_name: "DEFAULT Test Cluster"
spark_version: '16.2.x-scala2.12'

24 changes: 24 additions & 0 deletions acceptance/bundle/deployment/bind/cluster/output.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@

>>> cat databricks.yml
bundle:
name: bind-cluster-test-[UUID]

resources:
clusters:
cluster1:
cluster_name: "DEFAULT Test Cluster"
spark_version: '16.2.x-scala2.12'


>>> [CLI] clusters get [CLUSTER-ID]
{
"cluster_name": "DEFAULT Test Cluster"
}

>>> [CLI] bundle deployment bind cluster1 [CLUSTER-ID] --auto-approve
Updating deployment state...
Successfully bound databricks_cluster with an id '[CLUSTER-ID]'. Run 'bundle deploy' to deploy changes to your workspace

>>> [CLI] bundle deployment unbind cluster1
Updating deployment state...
0
17 changes: 17 additions & 0 deletions acceptance/bundle/deployment/bind/cluster/script
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
BUNDLE_NAME_SUFFIX=$(uuid)
export BUNDLE_NAME_SUFFIX

CLUSTER_ID="0123-456789-cluster0"
if [ -n "${TEST_DEFAULT_CLUSTER_ID:-}" ]; then
CLUSTER_ID=${TEST_DEFAULT_CLUSTER_ID}
fi
export CLUSTER_ID
envsubst < databricks.yml.tmpl > databricks.yml
trace cat databricks.yml

trace $CLI clusters get ${CLUSTER_ID} | jq '{cluster_name}'

trace $CLI bundle deployment bind cluster1 "${CLUSTER_ID}" --auto-approve

trace $CLI bundle deployment unbind cluster1
echo $?
19 changes: 19 additions & 0 deletions acceptance/bundle/deployment/bind/cluster/test.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
Local = true
Cloud = true
RequiresCluster = true

Ignore = [
"databricks.yml",
]

[[Repls]]
Old = "[0-9]{4}-[0-9]{6}-[0-9a-z]{8}"
New = "[CLUSTER-ID]"

[[Server]]
Pattern = "GET /api/2.1/clusters/get"
Response.Body = '''
{
"cluster_name": "DEFAULT Test Cluster"
}
'''
3 changes: 3 additions & 0 deletions acceptance/internal/config.go
Original file line number Diff line number Diff line change
@@ -39,6 +39,9 @@ type TestConfig struct {
// If true and Cloud=true, run this test only if unity catalog is available in the cloud environment
RequiresUnityCatalog *bool

// If true and Cloud=true, run this test only if a default test is available in the cloud environment
RequiresCluster *bool

// List of additional replacements to apply on this test.
// Old is a regexp, New is a replacement expression.
Repls []testdiff.Replacement
6 changes: 6 additions & 0 deletions bundle/config/resources.go
Original file line number Diff line number Diff line change
@@ -124,6 +124,12 @@ func (r *Resources) FindResourceByConfigKey(key string) (ConfigResource, error)
}
}

for k := range r.Clusters {
if k == key {
found = append(found, r.Clusters[k])
}
}

if len(found) == 0 {
return nil, fmt.Errorf("no such resource: %s", key)
}