-
Notifications
You must be signed in to change notification settings - Fork 33
Description
We have some code that reads an InputStream
into chunks of fixed size ByteBuffer
s using (convert s (seq-of ByteBuffer))
. When testing the code, it's found that the returned seq doesn't always have all the data when the input stream is relatively big.
We were able to reproduce it using the following code snippet:
user> (count (convert (io/input-stream (io/file "bigfile")) (seq-of ByteBuffer)))
;; => 26955
user> (count (convert (io/input-stream (io/file "bigfile")) (seq-of ByteBuffer)))
;; => 79920
user> (count (convert (io/input-stream (io/file "bigfile")) (seq-of ByteBuffer)))
;; => 86093
user> (count (convert (io/input-stream (io/file "bigfile")) (seq-of ByteBuffer)))
;; => 27063
As the above snippet shows, the count
of the seq returned by convert
keeps changing. We suspect it might be cause by the finalize
in closeable-seq
. We think what's happening is that the head of the seq got GCed when it's halfway through consuming it, which triggered a finalize
call that caused the source stream to be closed prematurely.
I'm wondering if closeable-seq
should stop overriding finalize
and leave it to the user to close the source, like line-seq
does.