Skip to content
/ 1brc Public
forked from tumdum/1brc

Fast 1BRC implementation for Rust

Notifications You must be signed in to change notification settings

p-adema/1brc

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

1BRC Rust implementation

This is an implementation of the 1BRC parsing / aggregation challenge is Rust, using mainly the standard library (and memchr to find newlines).

It runs a single reader thread, which fills up buffers that are read from by parser threads. The parsing process is generally zero-copy: after data is read from the file by the reader thread, the data is only copied once per unique station name (per thread, to fill up the HashMap). This is accomplished by using a wrapper around the standard HashMap that allows for using the Entry API with a reference type.

On my machine (M2 Macbook Air), it runs in read time: it takes ~8s to read in the file without parsing, and the same amount of time to also parse the file. With faster SSD's, your mileage may vary. This implementation runs around twice as fast as the reference implementation (~20s) on my machine.

About

Fast 1BRC implementation for Rust

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Rust 100.0%