lost data from mark/reset?

[stevengj]

I noticed this when rewriting peek in #77. Running peek (which calls mark and reset) sometimes seems to screw up the buffer so that the subsequent read fails:

julia> io = BufferedInputStream(IOBuffer("α∆"), 1)
BufferedInputStream{IOBuffer}(<1 B buffer, 0% filled>)

julia> read(io, Char)
'α': Unicode U+03B1 (category Ll: Letter, lowercase)

julia> peek(io, Char)
'∆': Unicode U+2206 (category Sm: Symbol, math)

julia> read(io, Char)
ERROR: EOFError: read end of file
Stacktrace:
 [1] read
   @ ~/.julia/packages/BufferedStreams/0mu59/src/bufferedinputstream.jl:186 [inlined]
 [2] read(io::BufferedInputStream{IOBuffer}, #unused#::Type{Char})
   @ Base ./io.jl:789
 [3] top-level scope
   @ REPL[72]:1

PR #77 avoids this problem specifically for peek(io, Char) since it adds an optimized implementation of that function, but it would be good to identify and solve the underlying problem.

[stevengj]

Here is some more detailed debugging data for a byte-by-byte read (not using peek), showing that the anchored data from the mark has not actually been preserved — we are only saving the last byte of ∆:

julia> io = BufferedInputStream(IOBuffer("α∆"), 1)
BufferedInputStream{IOBuffer}(<1 B buffer, 0% filled>)

julia> read(io, Char)
'α': Unicode U+03B1 (category Ll: Letter, lowercase)

julia> mark(io)
2

julia> @show io.buffer, io.position, io.available, io.anchor
(io.buffer, io.position, io.available, io.anchor) = (UInt8[0xb1], 2, 1, 2)
(UInt8[0xb1], 2, 1, 2)

julia> read(io, UInt8)
0xe2

julia> @show io.buffer, io.position, io.available, io.anchor
(io.buffer, io.position, io.available, io.anchor) = (UInt8[0xe2], 2, 1, 2)
(UInt8[0xe2], 2, 1, 2)

julia> read(io, UInt8)
0x88

julia> @show io.buffer, io.position, io.available, io.anchor
(io.buffer, io.position, io.available, io.anchor) = (UInt8[0x88], 2, 1, 2)
(UInt8[0x88], 2, 1, 2)

julia> read(io, UInt8)
0x86

julia> @show io.buffer, io.position, io.available, io.anchor
(io.buffer, io.position, io.available, io.anchor) = (UInt8[0x86], 2, 1, 2)
(UInt8[0x86], 2, 1, 2)

julia> reset(io)
2

julia> @show io.buffer, io.position, io.available, io.anchor
(io.buffer, io.position, io.available, io.anchor) = (UInt8[0x86], 2, 1, 0)
(UInt8[0x86], 2, 1, 0)

My understanding was that fillbuffer! is supposed to resize! the buffer as needed to preserve the anchored data?

[stevengj]

Could be related to #63, although I can't reproduce @feanor12's example in that issue.