Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support scp and rsync with a similar model to how ssh and sftp work #83

Open
fperez opened this issue Oct 1, 2021 · 0 comments
Open
Labels
🏷️ JupyterHub Something related to JupyterHub

Comments

@fperez
Copy link
Collaborator

fperez commented Oct 1, 2021

I've been using the hub for more extensive development, and the need to conveniently synchronize directories continues to reappear. The ssh and sftp support are fantastic, but unfortunately, from my perspective, they don't quite have the immediate convenience of scp for simple transfers and rsync for more complex synchronization. Nothing beats scp foo.tgz hub.jupytearth.org: in terms of simply sending a big file over, with perhaps immediately after doing ssh hub.jupytearth.org to go in and unpack/move it around as needed... Sftp is great for many things, and I'm glad to know protocol-wise it's adopting the sftp tools underneath, but I still think the day-to-day value of this is very high...

And for complex directory synchronization (a very common need when working across machines and hubs), then sftp is simply not the right tool, and I don't know of anything in existence that matches rsync for simultaneous simplicity and performance.

I've been trying quite a bit to make do with the alternatives to see if it was really a matter of adapting my habits, but I'm convinced that robust scp/rsync usage is really a need for at least some very real, legitimate use cases.

I know that the ultimate implementation space is in jupyterhub-ssh and @yuvipanda has already commented in yuvipanda/jupyterhub-ssh#55 opened by @consideRatio, so I'm happy to work there. Just flagging this one as a top-level point for deployment/discussion of that solution in our hub.

BTW - I think it's worth thinking more about generic file transfer ideas as indicated by Erik, and in that context we'll probably want to consider also tools like Globus, which are very relevant in science. But for now, a more pedestrian toolset with these old-school basics (scp and rsync) will already make a big difference (in addition to the sftp support, which I'm grateful for!).

@consideRatio consideRatio added the 🏷️ JupyterHub Something related to JupyterHub label Oct 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ JupyterHub Something related to JupyterHub
Projects
None yet
Development

No branches or pull requests

2 participants