Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix condition in mincore #627

Merged

Conversation

cedlemo
Copy link
Contributor

@cedlemo cedlemo commented Oct 8, 2018

@aantron,

#625

I started to try to understand the behavior with different cases :

when the last condition is : offset > length buffer - (Array.length states * page_size)

case where mem_len is > buffer_len

	buff_len = 6 offset = 0  pages = 1
		offset = 0
		mem_len = 1*4096
		offset > (buff_len - mem_len < 0) -> true -> Error

        buff_len = 4096 + 6 offset = 0  pages = 2
		offset = 0
		mem_len = 2*4096
		offset > (buff_len - mem_len < 0) -> true -> Error

	buff_len = 4096 + 6 offset = 4096 pages = 2
		offset = 4096
		mem_len = 2*4096
		offset > (buff_len - mem_len) < 0 -> true ->  Error

case where mem_len is < buffer_len

	buff_len = 4096 + 6 offset = 0 pages = 1
		offset = 0
		mem_len = 1*4096
		offset > buff_len - mem_len = 6  -> false -> ok

	buff_len = 4096 + 6 offset = 4096 pages = 1
		offset = 4096
		mem_len = 1*4096
		offset > (buff_len - mem_len) = 6 -> true ->  Error

and instead of we need to change the check to allow the last page queried to only partially overlap the buffer. like you said, I thought that we just have to be sure that the offset is not greater than the buffer length.
I have done the tests with the last condition offset > buffer length and it works as expected.

case where mem_len is > buffer_len

	buff_len = 6 offset = 0  pages = 1
		offset = 0
		mem_len = 1*4096
		offset > buff_len -> false -> ok

        buff_len = 4096 + 6 offset = 0  pages = 2
		offset = 0
		mem_len = 2*4096
		offset > buff_len -> false -> ok

	buff_len = 4096 + 6 offset = 4096 pages = 2
		offset = 4096
		mem_len = 2*4096
		offset > buff_len > 0 -> false -> ok

case where mem_len is < buffer_len

	buff_len = 4096 + 6 offset = 0 pages = 1
		offset = 0
		mem_len = 1*4096
		offset > buff_len -> false -> ok

	buff_len = 4096 + 6 offset = 4096 pages = 1
		offset = 4096
		mem_len = 1*4096
		offset > buff_len -> false -> ok

I send this PR first for validation, then I will send you another PR with the tests for all the IO part of Lwt_bytes.

@@ -240,7 +240,7 @@ external stub_mincore : t -> int -> int -> bool array -> unit = "lwt_unix_mincor
let mincore buffer offset states =
if (offset mod page_size <> 0
|| offset < 0
|| offset > length buffer - (Array.length states * page_size)) then
|| offset > length buffer) then
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we want to make sure that the offset is far back enough in the buffer, that for a flag array for N pages, the first N-1 pages are fully in memory, and there is at least one byte for the last page.

So, we want there to be at least (Array.length states - 1) * page_size + 1 bytes in the buffer starting at offset. So, we need something like length buffer - offset >= (Array.length states - 1) * page_size + 1. The only thing to watch out for is that Array.length states - 1 might be negative, if the array is empty. It seems like the check will still give us the right behavior, though, but we should probably leave a comment about that in the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok,
so if we want that there is at least (Array.length states - 1) * page_size + 1 bytes in the buffer starting at offset, the condition that throws an exception for bad parameter is:

length buffer - offset < (Array.length states - 1) * page_size + 1

If I go through several cases like this:

  • buff_len = 6 offset = 0 n_states = 1
6 - 0 < (1 - 1) * page_size + 1 (* -> false *)
  • buff_len = page_size + 6 offset = 0 n_states = 1
page_size + 6 - 0 < (1 - 1) * page_size + 1 (* -> false *)
  • buff_len = page_size + 6 offset = page_size n_states = 1
page_size + 6 - page_size < (1 - 1) * page_size + 1 (* -> false *)
  • buff_len = page_size + 6 offset = page_size n_states = 2
page_size + 6 - page_size < (2 - 1) * page_size + 1 (*-> true (throw exception)*)

Do you want that I do all those tests (I have already almost done them) or do you see another case that are not covered?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say to set the offset always to page_size, then do:

  1. buffer length = page_size * 2, n_states = 1 (should be ok)

  2. buffer length = page_size * 2, n_states = 2 (should fail)

  3. buffer length = page_size * 2 + 1, n_states = 2 (should be ok)

I think that exercises the main corner cases completely and exactly, though something may be missing.

We should also test what happens when n_states = 0.

@@ -240,7 +240,8 @@ external stub_mincore : t -> int -> int -> bool array -> unit = "lwt_unix_mincor
let mincore buffer offset states =
if (offset mod page_size <> 0
|| offset < 0
|| offset > length buffer - (Array.length states * page_size)) then
|| length buffer - offset >= (Array.length states - 1) * page_size + 1)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the condition is inverted, and we need < instead of >=.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

argh you are right, I copy/pasted without checking sorry, the tests in the PR #628 are for length buffer - offset < (Array.length states - 1) * page_size + 1

@aantron aantron merged commit fe5f3cd into ocsigen:master Oct 13, 2018
@aantron
Copy link
Collaborator

aantron commented Oct 13, 2018

Thanks :) It happens, I guess I started talking in inverted terms (when the bounds are ok, rather than when to throw an exception). It was misleading, sorry about that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants