Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PLAT-1243 - add data studio command stop; refactor to be able to pass in studio and datalinks by name #478

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

georgi-seqera
Copy link
Contributor

@georgi-seqera georgi-seqera commented Jan 27, 2025

Context

Work relates to task: https://seqera.atlassian.net/browse/PLAT-1243

Summary: add the ability to stop a data studio via the CLI.

  • Add ability to select Data Studio by name or id
  • Add ability to select the DataLink to be mounted to data studio by name, resourceRef or id

Testing

Trigger a stop:

$ ./tw studios stop -w data-studios/data-studios -n studio-ef6b

 Data Studio studio-ef6b STOP successfully submitted at [data-studios / data-studios] workspace.

Trigger a start with --mount-data supplied with name:

$ ./tw studios start -w data-studios/data-studios -n studio-ef6b --mount-data adrian-navarro-test

  Data Studio studio-ef6b START successfully submitted at [data-studios / data-studios] workspace.

    https://dev-tower.net/orgs/data-studios/workspaces/data-studios/studios/studio-ef6b/connect
Screenshot 2025-01-28 at 12 45 11

Trigger a start with --mount-data-resource-ref supplied with resourceReference:

$ ./tw studios start -w data-studios/data-studios -n studio-ef6b --mount-data-resource-refs s3://a-test-bucket-eend-us-east-1,s3://adrian-test-london

  Data Studio studio-ef6b START successfully submitted at [data-studios / data-studios] workspace.

    https://dev-tower.net/orgs/data-studios/workspaces/data-studios/studios/studio-ef6b/connect
image

Trigger a start with --mount-data supplied with name where multiple datalinks match throws an error:

Given we have multiple datalinks with same name:
Screenshot 2025-01-28 at 12 38 22

$ ./tw studios start -w data-studios/data-studios -n studio-ef6b --mount-data georgi-hristov-test
Running in Zsh

 ERROR: Multiple DataLink items found for 'georgi-hristov-test' found at workspace '27230932650799'. Found DataLink IDs: [v1-user-5ce468ea39369b770af8c58390464eb0, v1-cloud-YnVja2V0OndzOjI3MjMwOTMyNjUwNzk5OnMzOi8vZ2VvcmdpLWhyaXN0b3YtdGVzdA==]

The above is open to suggestion about how to alternatively handle this scenario - for the same case when trying to launch a pipeline by name and multiple are with same matching name are found a similar error is thrown.

@georgi-seqera georgi-seqera marked this pull request as ready for review January 28, 2025 12:51
public List<String> mountDataIds;

@CommandLine.Option(names = {"--mount-data-resource-refs"}, description = "Optional configuration override for 'mountData' setting (comma separate list of data-link resource refs)", split = ",")
public List<String> mountDataResourceRefs;
Copy link
Contributor Author

@georgi-seqera georgi-seqera Jan 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here I was toying with the idea of just having --mount-data and being able to pass in either the DataLink name or resource ref there and have logic to check if the string contains s3:// or az:// or even just :// and infer from that it is a resourceRef, rather than dataLink name.

It might be nicer but I don't know if wouldn't be a bit brittle, so have kept it separate params

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's better to stick to separate parameters, as you did. We are going to expand the supported datalinks provider in the future, and new ones might have a different shape for the resourceRef.

import io.seqera.tower.model.DataStudioDto;
import io.seqera.tower.model.DataStudioProgressStep;

import static io.seqera.tower.model.DataStudioProgressStepStatus.ERRORED;
import static io.seqera.tower.model.DataStudioProgressStepStatus.IN_PROGRESS;

public class AbstractStudiosCmd extends AbstractApiCmd {
public class AbstractStudiosCmd extends AbstractDataLinkCmd {
Copy link
Contributor Author

@georgi-seqera georgi-seqera Jan 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reads a bit weird, but the inheritance structure makes it a bit awkward to share logic without dumping everything in the AbstractApiCmd which is a pattern I feel like is to be avoided..

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't you extract the functionality that you need from AbstractDataLinkCmd and use composition instead?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment was duplicated somehow :)

import io.seqera.tower.model.DataStudioDto;
import io.seqera.tower.model.DataStudioProgressStep;

import static io.seqera.tower.model.DataStudioProgressStepStatus.ERRORED;
import static io.seqera.tower.model.DataStudioProgressStepStatus.IN_PROGRESS;

public class AbstractStudiosCmd extends AbstractApiCmd {
public class AbstractStudiosCmd extends AbstractDataLinkCmd {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't you extract the functionality that you need from AbstractDataLinkCmd and use composition instead?

@t0randr
Copy link
Contributor

t0randr commented Jan 28, 2025

Does stop command support the ---wait option, so we know that the studio is actually stopped?

Copy link
Member

@endre-seqera endre-seqera left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice solutions, it turned out very user friendly!

Comment on lines 46 to 48
String sessionId = dataStudioRefOptions.dataStudio.sessionId != null
? dataStudioRefOptions.dataStudio.sessionId
: fetchDataStudio(dataStudioRefOptions, wspId).getSessionId();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: this feels like will be a commonly needed functionality, getting the sessionId - so maybe this could be moved to the AbstractStudiosCmd, or inside the DataStudioRefOptions?

@georgi-seqera georgi-seqera force-pushed the PLAT-1243-add-data-studio-command-stop branch from da2a534 to 30e2dc5 Compare January 29, 2025 18:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants