BufferedStream<File> doesn't maintain logical position in file with mixed reads/writes

[ben0x539]

Here's an example:

use std::io::{println, File, Open, Truncate, Write, ReadWrite, BufferedStream};

fn main() {
    let path = Path::new("test");

    let mut file = File::open_mode(&path, Truncate, Write).unwrap();
    file.write_str("123456789").unwrap();
    drop(file);

    let file = File::open_mode(&path, Open, ReadWrite).unwrap();
    let mut bs = BufferedStream::new(file);
    println(bs.read_exact(2).unwrap().into_ascii().into_string().as_slice());
    bs.write_str("xyz").unwrap();
    bs.flush().unwrap();
    println(bs.read_to_string().unwrap().as_slice());
}

This prints

12
3456789

and afterwards the file contains the string 123456789xyz even though logically the write happens between the two reads and should have moved the file position over the 345 without them being returned from read calls instead of appending to the end (or overwriting the wrong bytes if a tiny buffer size is chosen). This makes the buffering leak through the abstraction where a mostly equivalent C program maintains the illusion:

#include <stdio.h>

int main(void) {
  char buf[10] = { 0 };
  FILE* f;

  f = fopen("test", "w");
  fwrite("123456789", 1, 9, f);
  fclose(f);

  f = fopen("test", "r+");
  fread(buf, 1, 2, f);
  puts(buf);
  fwrite("xyz", 1, 3, f);
  fread(buf, 1, 9, f);
  puts(buf);
  fclose(f);

  return 0;
}

printing

12
6789

and leaving the file to contain 12xyz6789. strace suggests glibc just seems to seek backwards (lseek(3, -7, SEEK_CUR)) before the write presumably for the length of the discarded read buffer or otherwise manually keeping track of the logical file position.

[mahkoh]

This has to be decided before 1.0.

[pnkfelix]

P-backcompat-libs, not 1.0. We already have a separate process for stabilizing the libraries and particular types within them, independent of the 1.0 relaese itself.

[sfackler]

I don't think that BufferedStream itself can handle this case. For many (most?) other Stream types, the input and output parts of the stream aren't connected in any way. For example, if I write to a TCP stream, I don't necessarily expect to be reading that back out.

I think the best solution here is probably a separate BufferedFile type that can properly keep everything in sync, while offering all of the extra methods like seek and truncate that you'd want.

[brson]

Renominating. Are we going to go with this as is?

[pnkfelix]

we need to decide what we were doing it. (We might mark BufferedStream unstable, or we might stick with status quo, or we might steal ideas from other languages...)

I-needs-decision, P-high.

[aturon]

Potentially relevant: a similar abstraction in .NET (https://msdn.microsoft.com/en-us/library/system.io.bufferedstream%28v=vs.110%29.aspx).

[alexcrichton]

We ended up deprecating/removing BufStream, so closing.