Skip to content

Optimisation: S3/GS files shouldn't need to touch disk #192

@JensRantil

Description

@JensRantil

I notice that the S3 and GS virtual file systems both copy the content first to disk. Strictly speaking, given that both APIs support range requests to fetch file content, I don't think they really need to touch disk at all. Instead they could be streamed straight over HTTP while still implementing

gleam/filesystem/vfs.go

Lines 20 to 25 in 53a6956

type VirtualFile interface {
io.ReaderAt
io.ReadCloser
io.Seeker
Size() int64
}
.

I guess temporarily storing to disk is a simpler implementation, though. 🤷‍♂️

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions