LMDB storage #16274
LMDB storage #16274
Conversation
|
!ci-build-me |
src/lib/lmdb_storage/block.ml
Outdated
| (* increase to 1000000 to reach past mmap size of 256 MiB*) | ||
| Base_quickcheck.quickcheck_generator_char ) ~trials:n | ||
| ~f:(fun _ -> | ||
| let body = Breadcrumb.block breadcrumb |> Mina_block.body in |
There was a problem hiding this comment.
I used already created breadcrumb instead of generated string, is there any way to convert string to ~body ?
…1ae44c067984cdcb20c4274278ced9c2b418"" This reverts commit 10a63d13640e909b02875b10a6504a9bab29e4ef. Revert "Auxiliary commit to revert individual files from 9fb91ae44c067984cdcb20c4274278ced9c2b418" This reverts commit 2038250076ccd685aaaeb17bc691c551d5b5a9d4. Add generic implementation of LMDB storage
This reverts commit 90c88b38dd997fa22ecbfe93523f175276e5f01e.
dkijania
force-pushed
the
dkijania/cache_keys_with_lmdb
branch
from
a779250 to
053a80e
Compare
|
!ci-build-me |
|
!ci-build-me |
There was a problem hiding this comment.
I pushed a Revert changes to block.ml commit to the branch.
It breaks compilation to be sure, but I insist on using that version. We need to update interface of send_add_resource and move Bitsawp_tag module to Mina_net2 (from Staged_ledger_diff).
For a moment I considered whether we want block.ml and header.ml right in this PR. And maybe the answer is "no", but I still think it's ok to have them for purpose of introducing some tests that were written for block.ml (and both files are needed for bitswap work, so they are likely to be cherry-picked soon).
|
!ci-build-me |
| @@ -0,0 +1,43 @@ | |||
| open Core_kernel | |||
|
|
|||
| let uint32_be = | |||
There was a problem hiding this comment.
Here is how int32_be defined in
let int32_be =
{ flags =
if Sys.big_endian && is_int_size 4
then Flags.(integer_key + integer_dup + dup_fixed)
else Flags.(dup_fixed)
; serialise = begin fun alloc x ->
let a = alloc 4 in
Bigstring.set_int32_be a 0 x;
a
end
; deserialise = begin fun a ->
Bigstring.get_int32_be a 0
end
}
Notice the difference in flags. Specification of LMDB prescribes a certain pattern of usage for the flags: http://www.lmdb.tech/doc/group__mdb.html.
It seems like the check for Sys.big_endian && is_int_size 4 might actually be needed to ensure the code behaves well under all circumstances.
src/lib/lmdb_storage/dune
Outdated
| network_peer | ||
| pipe_lib | ||
| timeout_lib | ||
| mina_metrics |
There was a problem hiding this comment.
Some of these dependencies are redundant to be sure
Co-authored-by: George Agapov <[email protected]>
Co-authored-by: George Agapov <[email protected]>
Co-authored-by: George Agapov <[email protected]>
|
!ci-build-me |
src/lib/lmdb_storage/conv.ml
Outdated
| Lmdb.Conv.make | ||
| ~flags: | ||
| Lmdb.Conv.Flags.( | ||
| if Sys.big_endian && Int.equal Sys.int_size 4 then |
There was a problem hiding this comment.
@georgeee I used Sys.int_size instead of is_int_size, since i cannot find it anywhere. I'm not sure if the 4 is right here. Sys.int_size is measuring int size in bits (so it will be 63 on majority of os). Is is_int_size expressed in bits too ? in that case 4 looks very suspicious
There was a problem hiding this comment.
I thought a bit. It's unlikely that integer will be 4 bytes on any modern server. So let's just drop the edge case and have ~flags:Lmdb.Conv.Flags.dup_fixed
|
!ci-build-me |
|
!ci-build-me |
Cherry picked @georgeee commits regarding src/lib/lmdb_storage project.