fix: Ensure that Snowflake accounts for number columns that overspeci…

…fy precision (#3306) * fix: Ensure that Snowflake properly accounts for INT32/INT64 columns with default precision 38 type Signed-off-by: Danny Chiao <[email protected]> * fix: Snowflake infer snowflake OBJECT return type for number inference Signed-off-by: miles.adkins <[email protected]> Signed-off-by: Danny Chiao <[email protected]> Signed-off-by: miles.adkins <[email protected]> Co-authored-by: miles.adkins <[email protected]>
feast-dev · Oct 26, 2022 · 0ad0ace · 0ad0ace
1 parent 1d288d3
commit 0ad0ace
Showing 1 changed file with 10 additions and 0 deletions.
diff --git a/sdk/python/feast/infra/offline_stores/snowflake_source.py b/sdk/python/feast/infra/offline_stores/snowflake_source.py
@@ -263,6 +263,16 @@ def get_table_column_names_and_types(
                                 result.dtypes[column].name
                             ]
                         else:
+                            if len(result) > 0:
+                                max_value = result.iloc[0][0]
+                                if max_value is not None and len(str(max_value)) <= 9:
+                                    row["snowflake_type"] = "NUMBER32"
+                                    continue
+                                elif (
+                                    max_value is not None and len(str(max_value)) <= 18
+                                ):
+                                    row["snowflake_type"] = "NUMBER64"
+                                    continue
                             raise NotImplementedError(
                                 "NaNs or Numbers larger than INT64 are not supported"
                             )