Questions on fsspec's API design: why was rb chosen as default mode of fsspec.open(...)?

[vadimkantorov]

In regular Python's open(...) the default mode is r, which is equivalent to rt

So for porting any code just replacing open(...) with fsspec.open(...) is not sufficient and mode must be tweaked

Same question about the context manager design: in regular python we can do python -c 'import sys; print(open(sys.argv[1]).read())', but in fsspec result of fsspec.open(...) does not have method .read(), and so we have to port to use the context manager (which by accident also does not work with one-liners:) ). Why was a decision taken to break the compat here?

Thanks :)

[martindurant]

Answering the second question first: we designed the result of fsspec.open() to be serialisable, so it cannot directly create resources on instantiation. Since the builtins open() is always demonstrated in use with a context, it seemed like a reasonable pattern to follow. For your use case, you can do:

fsspec.open(...).open().read()

As for the default mode, I think this is just historical: we didn't have a text mode originally, and essentially all of the implementations only handle bytes internally.

[vadimkantorov]

If 'rt' is made default, then probably one could do sth like from fsspec import open to override python's default open. Maybe good for porting legacy python I/O code would be to have a number of compat functions from fsspec.compat import open, listdir, exists which mimick open, os.listdir, os.path.exists etc