Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

eLabFTW file source from file source templates #19493

Open
wants to merge 1 commit into
base: dev
Choose a base branch
from

Conversation

kysrpex
Copy link
Contributor

@kysrpex kysrpex commented Jan 30, 2025

Support user-defined file sources based on eLabFTWFilesSource. Add a file source template for eLabFTW and the required documentation.

Adding eLabFTW as user-defined files source

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. Run eLabFTW, for example using Docker Compose (or use https://demo.elabftw.net).
    2. Enable the file source template.
    3. Create an account and generate an API Key on eLabFTW (or use the demo account from https://demo.elabftw.net).
    4. Create a new eLabFTW file source on the user preferences page, configuring the endpoint and API Key.
    5. Create an experiment or resource in eLabFTW and attach a file.
    6. Import the file to your history from the eLabFTW file source.
    7. Export your history to the eLabFTW file source.

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

Comment on lines -172 to +193
def get_prefix(self) -> Optional[str]:
return None
def get_prefix(self, user_context: OptionalUserContext = None) -> Optional[str]:
endpoint: ParseResult = self._get_endpoint(user_context=user_context)
return self.id if self.scheme not in {"elabftw", DEFAULT_SCHEME} else (endpoint.netloc or None)
# it would make better sense to return
# `self.id if self.scheme == USER_FILE_SOURCES_SCHEME else (endpoint.netloc or None)`, where
# `USER_FILE_SOURCES_SCHEME` comes from `galaxy.managers.file_source_instances`; however, that would lead to a
# circular import (maybe `USER_FILE_SOURCES_SCHEME` should be moved to a module in a layer deeper than
# `galaxy.managers`)

def get_scheme(self) -> str:
return "elabftw"
return self.scheme if self.scheme and self.scheme != DEFAULT_SCHEME else "elabftw"
# it would make better sense to return `self.scheme if self.scheme == USER_FILE_SOURCES_SCHEME else "elabftw"`,
# but the same circular import issue as above arises
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this ended up being so weird there must be something I am not getting about the scheme... Where are schemes other than "gxfiles" and "gxuserfiles" coming from? The design intended in #19319 with a URI that only depends on the configured endpoint is not how things are supposed to work, right?

File sources reference files via a URI, while eLabFTW uses auto-incrementing positive integers. For more details read #18665 [5]. This leads to the need to declare a mapping between said identifiers and Galaxy URIs.

Those take the form elabftw://demo.elabftw.net/entity_type/entity_id attachment_id, where:

  • entity_type is either 'experiments' or 'resources'
  • entity_id is the id (an integer in string form) of an experiment or resource
  • attachment_id is the id (an integer in string form) of an attachment

For the user-defined file sources use case (when users configure one or more instances of the FilesSource via a file source template), Galaxy URIs have a different scheme and authority, taking the form gxuserfiles://file_source_id/entity_type/entity_id/attachment_id, where:

  • file_source_id is the file source identifier assigned by Galaxy

@kysrpex
Copy link
Contributor Author

kysrpex commented Jan 30, 2025

FYI @bgruening

@kysrpex kysrpex self-assigned this Jan 30, 2025
@kysrpex kysrpex added kind/enhancement kind/refactoring cleanup or refactoring of existing code, no functional changes kind/feature area/API area/backend labels Jan 30, 2025
@kysrpex kysrpex added this to the 25.0 milestone Jan 30, 2025
@@ -804,7 +849,7 @@ def _write_from(
entity_type, entity_id, attachment_id = match.groups()
entity_type = entity_type.replace("items", "resources")

return f"elabftw://{location.netloc}/{entity_type}/{entity_id}/{attachment_id}"
return f"/{entity_type}/{entity_id}/{attachment_id}"
Copy link
Contributor Author

@kysrpex kysrpex Jan 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should've been part of #19319 (bugfix).

@kysrpex kysrpex marked this pull request as draft January 30, 2025 16:22
Support user-defined file sources based on `eLabFTWFilesSource`. Add a file source template for eLabFTW and the required documentation.
@kysrpex kysrpex force-pushed the elabftw_file_source_template branch from cf60ef4 to f0e4866 Compare January 30, 2025 17:02
@kysrpex kysrpex marked this pull request as ready for review January 30, 2025 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/API area/backend kind/bug kind/enhancement kind/feature kind/refactoring cleanup or refactoring of existing code, no functional changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant