Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Implement Iceberg values #20

Merged
merged 42 commits into from
Aug 9, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
1a5236e
implement values
JanKaul Aug 2, 2023
1f9de91
improve getters
JanKaul Aug 2, 2023
20f8e2c
fix clippy warnings
JanKaul Aug 2, 2023
5be7a31
fix clippy warnings
JanKaul Aug 2, 2023
2ad06e2
change into bytebuf to from
JanKaul Aug 2, 2023
c3fc2c6
add license header
JanKaul Aug 2, 2023
50e41e0
Use Long instead of LongInt
JanKaul Aug 3, 2023
4775f16
use more general error kind
JanKaul Aug 3, 2023
bb59703
use Naivetime
JanKaul Aug 3, 2023
1713492
use naivedate
JanKaul Aug 3, 2023
53182e4
use naivedatetime
JanKaul Aug 3, 2023
af46ba1
fix clippy warnings
JanKaul Aug 3, 2023
1982d4c
use uuid
JanKaul Aug 3, 2023
7cab82b
use orderedfloat
JanKaul Aug 3, 2023
831f821
fix clippy warnings
JanKaul Aug 3, 2023
522d85f
fix tests
JanKaul Aug 3, 2023
8f91695
use datatime utz
JanKaul Aug 3, 2023
8432e8a
fix docs
JanKaul Aug 3, 2023
abd4327
rename value to literal
JanKaul Aug 4, 2023
7dda613
introduce primitive literal
JanKaul Aug 4, 2023
10d2183
remove length from fixed
JanKaul Aug 4, 2023
c37640b
serialize json via serde_json value
JanKaul Aug 4, 2023
ab583d7
remove derive serialize/deserialize
JanKaul Aug 4, 2023
6baf6db
implement From Literal for ByteBuf
JanKaul Aug 4, 2023
e84f223
implement From Literal for JsonValue
JanKaul Aug 4, 2023
72d4b25
fix From Literal for JsonValue
JanKaul Aug 4, 2023
c6bbaa4
implement struct
JanKaul Aug 4, 2023
61258b5
fix clippy warnings
JanKaul Aug 4, 2023
b484b2e
add avro tests for some types
JanKaul Aug 4, 2023
e0ed4ff
fix clippy warnings
JanKaul Aug 4, 2023
bb379bf
fix nested field
JanKaul Aug 4, 2023
8292d42
fix nested field
JanKaul Aug 4, 2023
7058b93
implement list test
JanKaul Aug 4, 2023
7e032b5
implement map test
JanKaul Aug 4, 2023
9fe7c76
fix error
JanKaul Aug 4, 2023
0bf2a7b
fix clippy warnings
JanKaul Aug 4, 2023
06b5676
change timestamps to int/long
JanKaul Aug 9, 2023
29488dc
convert nulls to None
JanKaul Aug 9, 2023
583e3b1
add tests for null
JanKaul Aug 9, 2023
927fa7f
null test for struct
JanKaul Aug 9, 2023
451f3d4
fix clippy warning
JanKaul Aug 9, 2023
c3fa7d7
convert json null to option
JanKaul Aug 9, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions crates/iceberg/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,11 @@ serde_json = "^1.0"
serde_derive = "^1.0"
anyhow = "1.0.72"
once_cell = "1"
rust_decimal = "1.31.0"
chrono = "0.4"
uuid = "1.4.1"
ordered-float = "3.7.0"
bitvec = "1.0.1"

[dev-dependencies]
pretty_assertions = "1.4.0"
24 changes: 24 additions & 0 deletions crates/iceberg/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,30 @@ define_from_err!(
"handling invalid utf-8 characters"
);

define_from_err!(
std::array::TryFromSliceError,
ErrorKind::DataInvalid,
"failed to convert byte slive to array"
);

define_from_err!(
std::num::TryFromIntError,
ErrorKind::DataInvalid,
"failed to convert integer"
);

define_from_err!(
chrono::ParseError,
ErrorKind::DataInvalid,
"Failed to parse string to date or time"
);

define_from_err!(
uuid::Error,
ErrorKind::DataInvalid,
"Failed to convert between uuid und iceberg value"
);

#[cfg(test)]
mod tests {
use anyhow::anyhow;
Expand Down
5 changes: 5 additions & 0 deletions crates/iceberg/src/spec/datatypes.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ use ::serde::de::{MapAccess, Visitor};
use serde::de::{Error, IntoDeserializer};
use serde::{de, Deserialize, Deserializer, Serialize, Serializer};
use std::cell::OnceCell;
use std::slice::Iter;
use std::{collections::HashMap, fmt, ops::Index};

/// Field name for list type.
Expand Down Expand Up @@ -293,6 +294,10 @@ impl StructType {
.get(&field_id)
.copied()
}
/// Returns an iteratorr over the struct fields
pub fn iter(&self) -> Iter<NestedField> {
self.fields.iter()
}
}

impl PartialEq for StructType {
Expand Down
1 change: 1 addition & 0 deletions crates/iceberg/src/spec/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,4 @@

pub mod datatypes;
pub mod schema;
pub mod values;
Loading