You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
`CREATE VIEW '${name}' AS SELECT * FROM parquet_scan('${file.name}')`
Because views are not physically materialized, this can cause many range requests as DuckDB reads the file. Sometimes the overhead of these range requests outweighs the benefits of not loading the entire file.
Using the new file.size property #1608, we could employ a heuristic that switches to CREATE TABLE instead of CREATE VIEW for “small” Parquet files. We might also want an option to provide an explicit hint to use a materialized table instead of a view.
The text was updated successfully, but these errors were encountered:
We currently use
CREATE VIEW
for Parquet files:`CREATE VIEW '${name}' AS SELECT * FROM parquet_scan('${file.name}')`
Because views are not physically materialized, this can cause many range requests as DuckDB reads the file. Sometimes the overhead of these range requests outweighs the benefits of not loading the entire file.
Using the new
file.size
property #1608, we could employ a heuristic that switches toCREATE TABLE
instead ofCREATE VIEW
for “small” Parquet files. We might also want an option to provide an explicit hint to use a materialized table instead of a view.The text was updated successfully, but these errors were encountered: