Skip to content

closeable-seq may end prematurely after GC? #37

@jerrypnz

Description

@jerrypnz

We have some code that reads an InputStream into chunks of fixed size ByteBuffers using (convert s (seq-of ByteBuffer)). When testing the code, it's found that the returned seq doesn't always have all the data when the input stream is relatively big.

We were able to reproduce it using the following code snippet:

user> (count (convert (io/input-stream (io/file "bigfile")) (seq-of ByteBuffer)))
;; => 26955
user> (count (convert (io/input-stream (io/file "bigfile")) (seq-of ByteBuffer)))
;; => 79920
user> (count (convert (io/input-stream (io/file "bigfile")) (seq-of ByteBuffer)))
;; => 86093
user> (count (convert (io/input-stream (io/file "bigfile")) (seq-of ByteBuffer)))
;; => 27063

As the above snippet shows, the count of the seq returned by convert keeps changing. We suspect it might be cause by the finalize in closeable-seq. We think what's happening is that the head of the seq got GCed when it's halfway through consuming it, which triggered a finalize call that caused the source stream to be closed prematurely.

I'm wondering if closeable-seq should stop overriding finalize and leave it to the user to close the source, like line-seq does.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions