Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not block on opening catalogs #27

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

danielballan
Copy link
Member

Closes #25

We were blocking the main thread on accessing sub-Catalogs. We already use create_worker (a nice wrapper around QRunnable that we vendored from napari) to access the items in a Catalog. This PR applies that approach to the initial sub-Catalog access as well, which is when MongoDB client--server connection happens and has the potential to hang for 30 seconds.

I tested this against this configuration file, which points to a nonexistent MongoDB deployment.

# ~/.local/share/intake/test_not_connectable.yml 
sources:
  broken_example:
    driver: bluesky-mongo-normalized-catalog
    args:
      metadatastore_db: mongodb://localhost:55555/WRONG_PORT_CANNOT_CONNECT
      asset_registry_db: mongodb://localhost:55555/WRONG_PORT_CANNOT_CONNECT

When I select this item in the GUI, the combobox is immediately disabled but the GUI remains responsive. After 30 seconds (pymongo's default server selection timeout) the traceback is logged, the combox is re-enabled and reset to a blank value.

It would be nice if we could back out before waiting for the timeout---that is, to give the option to cancel the operation before it succeeds for times out. Cancellation is generally difficult to get right, so I would like to punt that to a future PR.

As is, this at least does not lock up the application and does leave it in a usable state after the failure.

Needs a unit test

This avoids locking up the app if, for example, the mongo client
cannot connect and makes us wait 30 seconds.
@danielballan
Copy link
Member Author

P.S. This work reinforced my view that the Search abstraction, which currently includes sub-Catalog combo boxes and forms/results for searching a Catalog of Runs, needs to be refactored. Not a huge priority, but should make this easier to look at.

@dylanmcreynolds
Copy link

I pull this down, and it seems to have to have fixed the locking problem. Thanks!

@danielballan
Copy link
Member Author

I have let this languish because I have reconsidered this statement above:

It would be nice if we could back out before waiting for the timeout---that is, to give the option to cancel the operation before it succeeds for times out. Cancellation is generally difficult to get right, so I would like to punt that to a future PR.

If we refactor the model to accept absolute sub-catalog paths, rather than "Go down one level" or "Go up one level," it will be perfectly possible to do cancellation. I think it's important to not make the user wait for the timeout.

@danielballan
Copy link
Member Author

This is waiting on #62, in which I can easily implement the proposal in my comment above, and then this will work nicely.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Threading issues with mongo server timeout
2 participants