Protocol of URI without authority not correctly idenfied

[observingClouds]

I am currently developing a fsspec driver for a filesystem protocol that does not need/have an authority, such that:

URI = protocol ":" path

is a valid URI.

However, fsspec's split_protocol function seems to assume that an authority is always given, i.e.:

URI = protocol ":" "//" authority path

filesystem_spec/fsspec/core.py

Lines 548 to 558 in 6b85a47

	def split_protocol(urlpath):
	"""Return protocol, path pair"""
	urlpath = stringify_path(urlpath)
	if "://" in urlpath:
	protocol, path = urlpath.split("://", 1)
	if len(protocol) > 1:
	# excludes Windows paths
	return protocol, path
	if urlpath.startswith("data:"):
	return urlpath.split(":", 1)
	return None, urlpath

The only exceptions are data (see above) and indirectly file (see here) which are handled specifically.

Are there any issues with supporting also protocols without //authority? E.g. could we just change split_protocol to something like:

def split_protocol(urlpath):
    """Return protocol, path pair"""
    urlpath = stringify_path(urlpath)
    if "://" in urlpath:
        protocol, path = urlpath.split("://", 1)
        if len(protocol) > 1:
            # excludes Windows paths
            return protocol, path
    elif ":/" in urlpath:
        protocol, path = urlpath.split(":/", 1)
        if len(protocol) > 1:
            path = "/" + path
            return protocol, path
    if urlpath.startswith("data:"):
        return urlpath.split(":", 1)
    return None, urlpath

[martindurant]

I could imagine that changing split_protocol might have cascading effects elsewhere, so I would rather not consider that if possible. You are right that data: is an exception - we ideally shouldn't have such exceptions either ...

From fsspec's point of view, the URL doesn't need to have HTTP-like parts to it, so "protocol://path" might still be a reasonable solution for you, even if users don't typically use this syntax. You can always override _split_protocol to end up with an implementation-specific path that makes sense to you. In some implementations, you definitely have a server/path split, sometimes bucket/path split (where buckets are a special type of directory/container), but some have a simple path with no special parts (e.g., memory).