Support bulk file operations in Workspace Service

[dandelany]

Background

We plan to implement NASA-AMMOS/aerie-ui#1791 to support selecting multiple files in the Sequence Editor workspace. The context menu items move, copy, delete, and move to workspace must work on multiple selected items at once. Workspace Service is the backend for these operations, but some endpoints are single-path only, so we need bulk file support.

Goals

• Accept multiple paths per request for move, copy, delete, and move-to-workspace
• Support file paths, directory paths, or a mix of both
• Predictable behavior with clear partial-failure reporting.
• Be consistent with current overwrite protection behavior. Don't overwrite files unless a "force" parameter is passed
• Use "best effort" approach in case of partial failures & attempt to operate on as many files as possible
• Report partial failures clearly so the UI can surface them to the user
• Ideally keep old single-path requests working for backward compat.

Partial failure cases

Since we are performing multiple file operations in one request, some of them may fail & others succeed. Ideally (IMO) these would be more atomic operations, and either completely succeed or completely fail, but this is not easy to implement right now & comes with usability challenges. Instead we plan to implement "best effort" operations, eg. move as many files as possible, but also report any failures.

In general, if the entire operation fails, the server should respond with an appropriate error code. If there is a partial failure, it should respond with the status code 207 "Multi-Status Response" & include details about which parts failed and which succeeded. (This code is a new part of the proposal & open to feedback)

Some of these errors may be the result of our rules refusing to overwrite an existing file by the same name - these cases should be clearly communicated, as the UI will present the user with the option to "proceed anyway", which will result in another request to the same endpoint with the "force" parameter set to true.

Concurrent file operations

Some of these bulk file operations may be dangerous if they are performed concurrently with other operations on the same files. We need to consider these cases and have some form of protection against dangerous/inconsistent behavior due to concurrency. This may mean having global or path-based file locks to prevent concurrent operations. Requirements for this need to be fleshed out further. It may be useful to refer to the WebDAV protocol, which has similar goals, for inspiration on handling these cases.

[Mythicaeda]

Can a user provide a mix of operations to the endpoint? Ie, move files A and D to folder B, but copy file C to folder B?
Or can they only provide one operation per request? Ie, files A and D are getting moved to folder B, and in a separate request, copy file C to folder B?

[Mythicaeda]

Similarly, is "force" per request or per item? Ie, can I specify that file A should produce a 409 CONFLICTED on conflict but file B should overwrite? My assumption is it's per request.