Normalized interface for automatic mirror selection

[MSevey]

Issue by xloem
Monday Jul 06, 2020 at 13:01 GMT
Originally opened as NebulousLabs/skynet-docs#11

---

I'm interested in coming to agreement on an API for reliable fast transfers: where a backend automatically selects mirrors and retries through network issues.

It could possibly look just like the current api without the portal options object.

I've also been exploring more complex apis that give the user more participation in mirror selection: being able to provide additional mirrors or optionally specify which one to use, with the system recording speed metrics for later selection.

[MSevey]

Comment by m-cat
Tuesday Jul 07, 2020 at 11:37 GMT

---

Thanks for writing the issue!

It could possibly look just like the current api without the portal options object.

If we were to add this to the API we'd still want the other options to be accessible. I would keep the options object.

The upload function itself should not be choosing the mirror, but should stick to a single well-defined responsibility and only accept a single portal and try to upload to it. Here are some possibilities I see for additions to the API:

1. Helper functions that choose mirrors which can then be passed one at a time into separate upload calls by the user. Possibly there can be a helper that queries an API for available portals, though we don't have an official API for this yet I don't think.
2. Functions that try uploading with different supplied mirrors. They call the existing functions but take an array of options. The user can call these after getting portals from (1).
3. A single function that combines (1) and (2), choosing portals for the user automatically and keeping track of metrics. The user can still pass non-portal options into this function like custom_filename and timeout_seconds. Due to the complexity of this option, it is the least likely to be added in the short term, though it is the best for the end user.

I wonder whether these additions belong in the SDKs or if they should live in a separate more high-level library. The SDKs for now only provide low-level calls to the Skynet endpoints with minimal wrapping. There are issues with introducing complexity, such as replicating the functionality across all the SDKs, maintaining the code and ensuring its quality, and not overwhelming the end user with complexity.