Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

INDEX: index key and data serialization API #834

Merged
merged 11 commits into from
Aug 18, 2021

Conversation

yiwen-wong
Copy link

No description provided.

Copy link
Contributor

@simone-gaia simone-gaia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nit comments.

@@ -78,6 +79,31 @@ index_key_t index_builder_t::make_key(gaia_id_t index_id, gaia_type_t type_id, c
return k;
}

void index_builder_t::serialize_key(const index_key_t& key, data_write_buffer_t& buffer)
{
for (const auto& k : key.values())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unclear what auto is at this point, maybe vector<data_holder_t> won't hurt.

{
ASSERT_PRECONDITION(index_id != c_invalid_gaia_id, "Invalid gaia id.");

index_key_t k;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: key or index_key.


// These classes provide serialization/deserialization interfaces over flatbuffers type [byte].

class data_write_buffer_t
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the word buffer refers to Faltbuffers or to the fact that the writing is buffered? A class comment would help.

production/db/inc/payload_types/data_buffer.hpp Outdated Show resolved Hide resolved
@@ -91,6 +94,8 @@ struct data_holder_t
int compare(const data_holder_t& other) const;
data_hash_t hash() const;

void serialize(data_write_buffer_t& buffer) const;

// Convenience implicit conversions to native types.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These seems more like explicit conversions...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should update the comment to mentionexplicit.

@@ -68,7 +69,7 @@ index_key_t index_builder_t::make_key(gaia_id_t index_id, gaia_type_t type_id, c
auto schema = table.binary_schema();
auto index_view = index_view_t(id_to_ptr(index_id));

auto& fields = *(index_view.fields());
const auto& fields = *(index_view.fields());
for (auto field_id : fields)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be gaia_id_t.. (same below)

production/db/inc/payload_types/data_holder.hpp Outdated Show resolved Hide resolved
DEPENDS ${PROJECT_SOURCE_DIR}/tests/test_serialization.fbs
DEPENDS ${CMAKE_BINARY_DIR}/flatbuffers/flatc
VERBATIM)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra empty line.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that it still shows - did you miss merging the change?

Copy link
Contributor

@simone-gaia simone-gaia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@LaurentiuCristofor LaurentiuCristofor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please revert the changes to the field names of data_holder struct.
The type conversion method should also be able to handle unexpected types.

using serialization_write_buffer_t = std::vector<serialization_byte_t>;
using serialization_output_t = flatbuffers::Offset<flatbuffers::Vector<serialization_byte_t>>;

// These classes provide serialization/deserialization interfaces over the flatbuffers type [byte]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should be consistent in comments: we have flatbuffers, FlatBuffer and Flatbuffer. I think we should stick with flatbuffers, which is also the folder name.

case common::data_type_t::e_string:
return "e_string";
default:
return "unknown";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can use ASSERT_UNREACHABLE here.

case common::data_type_t::e_string:
return reflection::String;
default:
throw data_type_not_handled_t(type);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can also use ASSERT_UNREACHABLE here. The FDW can catch all exceptions and print them.

Copy link
Contributor

@LaurentiuCristofor LaurentiuCristofor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only some minor cleanup issues remaining now, so I'm making this comment to remove the "change request" and not block this if anyone else approves.

@LaurentiuCristofor LaurentiuCristofor dismissed their stale review August 18, 2021 17:59

To remove the change request, because a simple comment does not do that.

@yiwen-wong yiwen-wong merged commit a72938b into master Aug 18, 2021
@yiwen-wong yiwen-wong deleted the yiwen_key_serialization_pr branch August 20, 2021 17:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants