Skip to content

Commit

Permalink
update spilling setting
Browse files Browse the repository at this point in the history
  • Loading branch information
ray6080 committed Nov 15, 2024
1 parent 83ef55b commit 38ea6a4
Showing 1 changed file with 6 additions and 2 deletions.
8 changes: 6 additions & 2 deletions src/content/post/2024-11-13-kuzu-v-0.7.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,8 +138,12 @@ creating its persistent disk-based structures. This meant that if you had machin
to manually chunk the source data into smaller parts, otherwise the `COPY FROM` pipeline could run out of memory and fail.

In this release, Kùzu automatically spills the records it scans from the source table into
a temporary file on disk during `COPY FROM`.This is in fact done automatically by the buffer manager
as `COPY FROM` demands more and more memory. As a result of this improvement, Kùzu can now ingest very
a temporary file `.tmp` located inside the database directory on disk during `COPY FROM`.
The spilling is in fact done automatically by the buffer manager
as `COPY FROM` demands more and more memory.
It is by default turned on when you open a local Kùzu database under read-write mode, and you can choose to turn if off by `CALL spill_to_disk=false;`.
Note that we don't support spilling under in-memory mode for now.
As a result of this improvement, Kùzu can now ingest very
large sets of relationship records (such as the edges of the 17B Graph500-30 graph in the above experiment)
when it is running on a machine with limited memory.

Expand Down

0 comments on commit 38ea6a4

Please sign in to comment.