Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[storage] Merge PartialSerialize and Serialize #363 #364

Merged
merged 2 commits into from
Nov 18, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/agdb/collections/dictionary.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ mod dictionary_data_memory;

use self::dictionary_data_memory::DictionaryDataMemory;
use self::dictionary_impl::DictionaryImpl;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;
use std::marker::PhantomData;

Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/dictionary/dictionary_data.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
use super::dictionary_index::DictionaryIndex;
use super::dictionary_value::DictionaryValue;
use crate::db::db_error::DbError;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;

pub trait DictionaryData<T>
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/dictionary/dictionary_data_memory.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ use super::dictionary_index::DictionaryIndex;
use super::dictionary_value::DictionaryValue;
use crate::collections::multi_map::MultiMap;
use crate::db::db_error::DbError;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;

pub struct DictionaryDataMemory<T>
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/dictionary/dictionary_data_storage.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ use crate::db::db_error::DbError;
use crate::storage::storage_file::StorageFile;
use crate::storage::storage_index::StorageIndex;
use crate::storage::OldStorage;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;
use std::cell::RefCell;
use std::rc::Rc;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use crate::db::db_error::DbError;
use crate::storage::storage_index::StorageIndex;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use std::mem::size_of;

pub(crate) struct DictionaryDataStorageIndexes {
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/dictionary/dictionary_impl.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ use super::dictionary_data::DictionaryData;
use super::dictionary_index::DictionaryIndex;
use super::dictionary_value::DictionaryValue;
use crate::db::db_error::DbError;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;
use std::marker::PhantomData;

Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/dictionary/dictionary_index.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use crate::db::db_error::DbError;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;

#[derive(Clone, Debug, Default, Eq, PartialEq)]
pub struct DictionaryIndex {
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/dictionary/dictionary_value.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use crate::db::db_error::DbError;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;

#[derive(Clone, Default)]
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/map/map_impl.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ use crate::collections::map_common::map_iterator::MapIterator;
use crate::collections::map_common::map_value_state::MapValueState;
use crate::collections::map_common::MapCommon;
use crate::db::db_error::DbError;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;
use std::collections::HashMap;
use std::hash::Hash;
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/map_common.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ use self::map_iterator::MapIterator;
use self::map_value::MapValue;
use self::map_value_state::MapValueState;
use crate::db::db_error::DbError;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;
use std::cmp::max;
use std::hash::Hash;
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/map_common/map_data.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
use super::map_value::MapValue;
use super::map_value_state::MapValueState;
use crate::db::db_error::DbError;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;
use std::hash::Hash;

Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/map_common/map_data_memory.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ use super::map_data::MapData;
use super::map_value::MapValue;
use super::map_value_state::MapValueState;
use crate::db::db_error::DbError;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;
use std::hash::Hash;
use std::mem::take;
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/map_common/map_data_storage.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ use super::map_value_state::MapValueState;
use crate::db::db_error::DbError;
use crate::storage::storage_index::StorageIndex;
use crate::storage::OldStorage;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;
use std::cell::RefCell;
use std::hash::Hash;
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/map_common/map_iterator.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use super::map_data::MapData;
use super::map_value_state::MapValueState;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;
use std::hash::Hash;
use std::marker::PhantomData;
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/map_common/map_value.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use super::map_value_state::MapValueState;
use crate::db::db_error::DbError;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;

#[derive(Clone, Debug, Default, Eq, PartialEq)]
pub struct MapValue<K, T>
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/map_common/map_value_state.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use crate::db::db_error::DbError;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use std::mem::size_of;

#[derive(Clone, Default, Debug, Eq, PartialEq)]
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/multi_map.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ pub mod multi_map_impl;
use self::multi_map_impl::MultiMapImpl;
use super::map_common::map_data_memory::MapDataMemory;
use super::map_common::MapCommon;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;
use std::hash::Hash;

Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/multi_map/multi_map_impl.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ use crate::collections::map_common::map_data::MapData;
use crate::collections::map_common::map_iterator::MapIterator;
use crate::collections::map_common::map_value_state::MapValueState;
use crate::db::db_error::DbError;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;
use std::hash::Hash;

Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/storage_dictionary.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ use crate::db::db_error::DbError;
use crate::storage::storage_file::StorageFile;
use crate::storage::storage_index::StorageIndex;
use crate::storage::OldStorage;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;
use std::cell::RefCell;
use std::marker::PhantomData;
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/storage_map.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ use crate::db::db_error::DbError;
use crate::storage::storage_file::StorageFile;
use crate::storage::storage_index::StorageIndex;
use crate::storage::OldStorage;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;
use std::cell::RefCell;
use std::hash::Hash;
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/storage_multi_map.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ use crate::db::db_error::DbError;
use crate::storage::storage_file::StorageFile;
use crate::storage::storage_index::StorageIndex;
use crate::storage::OldStorage;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;
use std::cell::RefCell;
use std::hash::Hash;
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/storage_vec.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ use crate::db::db_error::DbError;
use crate::storage::storage_file::StorageFile;
use crate::storage::storage_index::StorageIndex;
use crate::storage::OldStorage;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use std::cell::RefCell;
use std::cell::RefMut;
use std::cmp::max;
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/collections/vec/storage_vec_iterator.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use crate::collections::storage_vec::StorageVec;
use crate::storage::OldStorage;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use std::marker::PhantomData;

pub struct StorageVecIterator<'a, T, Data>
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/db/db_float.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;
use std::cmp::Ordering;
use std::hash::Hash;
Expand Down
15 changes: 9 additions & 6 deletions src/agdb/db/db_index.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
use crate::{utilities::serialize::Serialize, DbError};
use crate::utilities::serialize::{Serialize, SerializeFixed};
use crate::DbError;

#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, PartialOrd, Ord)]
pub struct DbIndex {
Expand All @@ -12,11 +13,11 @@ impl DbIndex {
}

pub fn from_bytes(bytes: &[u8]) -> Result<DbIndex, DbError> {
if bytes.len() > DbIndex::serialized_size() {
if bytes.len() > DbIndex::fixed_size() {
return Err(DbError::from(format!(
"DbIndex::from_bytes error: value ({}) too long (>{})",
bytes.len(),
DbIndex::serialized_size()
DbIndex::fixed_size()
)));
}

Expand Down Expand Up @@ -51,7 +52,7 @@ impl DbIndex {
impl Serialize for DbIndex {
fn serialize(&self) -> Vec<u8> {
let mut bytes = Vec::<u8>::new();
bytes.reserve(Self::serialized_size());
bytes.reserve(Self::fixed_size());
bytes.extend(self.value.serialize());
bytes.extend(self.meta.serialize());

Expand All @@ -61,11 +62,13 @@ impl Serialize for DbIndex {
fn deserialize(bytes: &[u8]) -> Result<Self, crate::DbError> {
Ok(DbIndex {
value: u64::deserialize(bytes)?,
meta: u64::deserialize(&bytes[u64::serialized_size()..])?,
meta: u64::deserialize(&bytes[u64::fixed_size()..])?,
})
}
}

impl SerializeFixed for DbIndex {}

impl From<usize> for DbIndex {
fn from(value: usize) -> Self {
Self {
Expand Down Expand Up @@ -216,7 +219,7 @@ mod tests {
original.set_value(1_u64);
original.set_meta(2_u64);

let serialized_size = DbIndex::serialized_size();
let serialized_size = DbIndex::fixed_size();
let mut bytes = original.serialize();

assert_eq!(bytes.len(), serialized_size);
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/db/db_value.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use super::db_float::DbFloat;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;
use crate::DbError;

Expand Down
2 changes: 1 addition & 1 deletion src/agdb/graph/graph_data_storage_indexes.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use crate::db::db_error::DbError;
use crate::storage::storage_index::StorageIndex;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use std::mem::size_of;

pub(crate) struct GraphDataStorageIndexes {
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/storage.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ mod write_ahead_log;
use self::partial_storage::PartialStorage;
use crate::db::db_error::DbError;
use crate::storage::storage_index::StorageIndex;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::serialize::Serialize;

pub trait OldStorage {
Expand Down
22 changes: 7 additions & 15 deletions src/agdb/storage/partial_storage.rs
Original file line number Diff line number Diff line change
@@ -1,33 +1,25 @@
use super::storage_index::StorageIndex;
use crate::utilities::partial_serialize::PartialSerialize;
use crate::utilities::serialize::Serialize;
use crate::DbError;

pub trait PartialStorage {
fn commit(&mut self) -> Result<(), DbError>;
fn insert<T: PartialSerialize>(&mut self, value: &T) -> Result<StorageIndex, DbError>;
fn insert_at<T: PartialSerialize>(
fn insert<T: Serialize>(&mut self, value: &T) -> Result<StorageIndex, DbError>;
fn insert_at<T: Serialize>(
&mut self,
index: &StorageIndex,
offset: usize,
value: &T,
) -> Result<usize, DbError>;
fn move_at<T: PartialSerialize>(
fn move_at<T: Serialize>(
&mut self,
index: &StorageIndex,
offset: usize,
value: &T,
) -> Result<(), DbError>;
fn replace<T: PartialSerialize>(
&mut self,
index: &StorageIndex,
value: &T,
) -> Result<usize, DbError>;
fn value<T: PartialSerialize>(&self, index: &StorageIndex) -> Result<T, DbError>;
fn value_at<T: PartialSerialize>(
&self,
index: &StorageIndex,
offset: usize,
) -> Result<T, DbError>;
fn replace<T: Serialize>(&mut self, index: &StorageIndex, value: &T) -> Result<usize, DbError>;
fn value<T: Serialize>(&self, index: &StorageIndex) -> Result<T, DbError>;
fn value_at<T: Serialize>(&self, index: &StorageIndex, offset: usize) -> Result<T, DbError>;
fn resize_value(&mut self, index: &StorageIndex, new_size: u64) -> Result<(), DbError>;
fn shrink_to_fit(&mut self) -> Result<(), DbError>;
fn size(&mut self) -> Result<u64, DbError>;
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/storage/storage_file.rs
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ mod tests {
use crate::storage::storage_index::StorageIndex;
use crate::storage::OldStorage;
use crate::test_utilities::test_file::TestFile;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use std::fs::metadata;

#[test]
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/storage/storage_impl.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ use crate::storage::storage_index::StorageIndex;
use crate::storage::storage_record::StorageRecord;
use crate::storage::write_ahead_log::WriteAheadLogRecord;
use crate::storage::OldStorage;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use std::cmp::max;
use std::cmp::min;
use std::io::SeekFrom;
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/storage/storage_index.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use crate::db::db_error::DbError;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;

#[derive(Clone, Debug, Default, Eq, Ord, PartialEq, PartialOrd)]
pub struct StorageIndex {
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/storage/storage_record.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use crate::db::db_error::DbError;
use crate::storage::storage_index::StorageIndex;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;

#[derive(Clone, Debug, Default, Eq, Ord, PartialEq, PartialOrd)]
pub struct StorageRecord {
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/storage/write_ahead_log.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use crate::db::db_error::DbError;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use std::fs::File;
use std::fs::OpenOptions;
use std::io::Read;
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/test_utilities/collided_value.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use crate::db::db_error::DbError;
use crate::utilities::serialize::OldSerialize;
use crate::utilities::old_serialize::OldSerialize;
use crate::utilities::stable_hash::StableHash;

#[derive(Clone, Debug, Default, Eq, PartialEq)]
Expand Down
2 changes: 1 addition & 1 deletion src/agdb/utilities.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
pub mod partial_serialize;
pub mod old_serialize;
pub mod serialize;
pub mod stable_hash;
Loading