Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

External storage upload causes app to hang #333

Open
1 of 3 tasks
allaway opened this issue May 23, 2022 · 0 comments
Open
1 of 3 tasks

External storage upload causes app to hang #333

allaway opened this issue May 23, 2022 · 0 comments
Labels
bug Something isn't working mid-term Timeline: 2-4 months minor a low priority item

Comments

@allaway
Copy link
Contributor

allaway commented May 23, 2022

Describe the bug
When annotating a dataset on external S3 storage, often we will be annotatting data where Synapse only has READ access to the bucket. The app/schematic will try to write the synapse_metadata_manifest file to the bucket, but it just sits there forever because we do not have bucket write access. The files are still successfully annotated, so this is a minor problem for us, since we rely on that as our ground truth.

To Reproduce
Steps to reproduce the behavior:

  1. Go an external bucket dataset with READ only access.
  2. Annotate
  3. App will hang (see log file below)

Expected behavior
Check for write access, or alternatively, write synapse_metadata_manifest to local synapse storage, not external bucket.

Priority (select one)

  • Minor ⬇️
  • Major 📢
  • Critical 🆘

Additional context

2022-05-23T22:41:30.365822+00:00 shinyapps[5711947]: ##################################################
2022-05-23T22:41:30.365896+00:00 shinyapps[5711947]:  Uploading file to your external S3 storage 
2022-05-23T22:41:30.365970+00:00 shinyapps[5711947]: ##################################################
2022-05-23T22:41:30.366033+00:00 shinyapps[5711947]: 
2022-05-23T22:41:30.366106+00:00 shinyapps[5711947]: /srv/connect/apps/NF_data_curator-staging/.venv/lib/python3.8/site-packages/synapseclient/table.py:1968: DtypeWarning: Columns (56) have mixed types.Specify dtype option on import or set low_memory=False.
2022-05-23T22:41:30.366183+00:00 shinyapps[5711947]:   return _csv_to_pandas_df(self.filepath,
2022-05-23T22:41:30.366247+00:00 shinyapps[5711947]: INFO: [2022-05-23 22:41:29] synapseclient_default - 
2022-05-23T22:41:30.366304+00:00 shinyapps[5711947]: ##################################################
2022-05-23T22:41:30.366488+00:00 shinyapps[5711947]: 
2022-05-23T22:41:30.366370+00:00 shinyapps[5711947]:  Uploading file to your external S3 storage 
2022-05-23T22:41:30.366427+00:00 shinyapps[5711947]: ##################################################
@allaway allaway added the bug Something isn't working label May 23, 2022
@ychae ychae added minor a low priority item mid-term Timeline: 2-4 months labels Jun 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working mid-term Timeline: 2-4 months minor a low priority item
Projects
None yet
Development

No branches or pull requests

3 participants