Reader API

[marceloboeira]

Right now, you need to know exactly the points you want to read in order to be able to read from a log, otherwise you get an error.

Considering that the commit_log API is pretty much a crate, we should handle all its public methods as public interface, so, it is essential that we improve that by having a wrapper "response" to improve the UX of the reader side…

Acceptance criteria

At the end of this task, we should be able to read messages through:

commit_log = …

let last_record = commit_log.read(Position::Horizon) // Horizon is the first entry available for the log

// The read can receive either an offset of horizon
let last_record = commit_log.read(Position::Offset(1000)).unwrap()!

// last record returns the content of the record as well as a reference to the next one, when available
 
last_record.record() // returns a stream of bytes …
last_record.next() // returns a Result<Record, io::Error> (it could be the last record, then we would return “EOF” error…)

// there could be options, such as:
last_position = last_record.record().position()...
commit_log.read_after(last_position, 10) // to read 10 records after the given position) 

This might envolve creating the Record struct, if it comes to that, let’s make sure we change Writing/Reading from the commit log to support that.

[cuichenli]

Hi @marceloboeira, I would like to help on this issue, and would like to ensure I am not in the wrong direction. Following is my understanding, for this issue, you would like to implement one method read for commit_log, the read method would take in one parameter of Position structure and will return one instance of Record structure. Is that right? Thanks!

[marceloboeira]

@cuichenli Position is an Enum, where you can send Horizon or a specific offset.

pub fn read_at(&self, position, number_of_records) -> Result<Vec<Record>, io::Error>

max_number_of_records -> how many entries from the index/log should be read (max). If there are less than the specified number, return them...

And the commit_log::read(position) is basically just a syntax-sugar for read_at(position, 1).

something like:

pub fn read(&self, position) -> Result<Record, io::Error> {
   self.read_at(position, 1)?.[0]
}

You can start small, open the PR and we discuss further details.

[cuichenli]

Hi, @marceloboeira I have made a simple draft pr #14 , hope I am not too wrong on it. I am new to Rust so I may have made a lot of mistakes as well, please kindly forgive.

[marceloboeira]

I had a quick look, seems great! I’ll double check later today and merge it 👌🏻

[cuichenli]

@marceloboeira What should we do next for this issue? By now we have implemented the Record, Reader, and Position. 🤔

[marceloboeira]

I think we're good for now! 😄