Implement assynchronous SFTP file transfer #641
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
psf-chronographer
bot
added
the
bot:chronographer:provided
|
I applied some CI fixes and now this PR needs to be rebased to pick them up. I'm clicking the rebase button now. |
webknjaz
force-pushed
the
async-sftp
branch
2 times, most recently
from
3c22bba
to
fd42bb2
Compare
|
Signed-off-by: Jakub Jelen <[email protected]>
galorithm
suggested changes
Sep 23, 2024
Signed-off-by: Jakub Jelen <[email protected]>
Signed-off-by: Jakub Jelen <[email protected]>
Signed-off-by: Jakub Jelen <[email protected]>
for more information, see https://pre-commit.ci
|
|
Congratulations! One of the builds has completed. 🍾 You can install the built RPMs by following these steps:
Please note that the RPMs should be used only in a testing environment. |
|
The testing farm build are failing because the libssh 0.11 did not make it to the stable fedora versions. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SUMMARY
The SFTP transfers done the way they work now are very slow especially on connections with large latency because each chunk (now 1024B) needs to travel all the way to remote as well as the confirmation (or request + data for the other transfer), before another chunk is sent. This can be improved by two things:
This depends on #636 (libssh 0.11), but also on other platforms having the updated libssh. It will likely need some conditional compilation based on the libssh version, but I did not find a good solution for cython after it was deprecated (cython/cython#4310) so any help/suggestions welcomed.
This PR also reproduces an issue with the new API that it currently hangs waiting for the second message, while it already happily sits in the queue, which is being worked on upstream.
Therefore marking as a draft to collect initial inputs.
ISSUE TYPE