Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benches for ZeroMap #853

Closed
6 tasks
sffc opened this issue Jul 13, 2021 · 7 comments · Fixed by #1087
Closed
6 tasks

Add benches for ZeroMap #853

sffc opened this issue Jul 13, 2021 · 7 comments · Fixed by #1087
Assignees
Labels
C-data-infra Component: provider, datagen, fallback, adapters good first issue Good for newcomers S-small Size: One afternoon (small bug fix or enhancement) T-docs-tests Type: Code change outside core library

Comments

@sffc
Copy link
Member

sffc commented Jul 13, 2021

Follow-up from #652

Action items:

@andrewnguyen

@sffc sffc added T-docs-tests Type: Code change outside core library C-data-infra Component: provider, datagen, fallback, adapters S-small Size: One afternoon (small bug fix or enhancement) labels Jul 13, 2021
@sffc sffc added backlog good first issue Good for newcomers help wanted Issue needs an assignee labels Jul 15, 2021
@Manishearth
Copy link
Member

We should probably write the same benches for litemap too.

@Manishearth
Copy link
Member

@andrewnguyen is interested in working on this

@sffc sffc removed the help wanted Issue needs an assignee label Jul 26, 2021
@Manishearth Manishearth added the help wanted Issue needs an assignee label Aug 20, 2021
@Manishearth
Copy link
Member

(Andrew isn't planning on fixing this anymore)

@Manishearth
Copy link
Member

Alan plans to work on this

@sffc sffc removed the help wanted Issue needs an assignee label Sep 16, 2021
@sffc sffc removed their assignment Sep 16, 2021
@poulsbo
Copy link
Contributor

poulsbo commented Sep 17, 2021

It's not clear to me what these benches are doing. I've put together a basic ZeroMap benchmark, and I can add more, but it would help to know our aim here.

I see with ZeroVec we quote benchmark numbers against Vec<T>, proving the speed of building/deserializing the thing, with comparable speeds for operations such as binary search, as expected. Do we want something similar here? We're binary searching the keys; do we want some numbers showing "serialization is a win, but you pay at lookup time"?

@Manishearth
Copy link
Member

@poulsbo it's worth comparing against HashMap and also potentially LiteMap. We absolutely expect that lookup is more expensive, the question to answer is "how much more expensive" :)

utils/litemap already has a couple benchmarks to copy.

The main operations we care about are read operations, but benchmarking some write operations can be good too. Note that borrowed ZeroMaps will switch internal implementations if you try writing to them, it would be interesting to capture the performance impact of that.

@sffc
Copy link
Member Author

sffc commented Sep 19, 2021

it would help to know our aim here.

I would like to have results to allow us to expand this table to include ZeroMap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-data-infra Component: provider, datagen, fallback, adapters good first issue Good for newcomers S-small Size: One afternoon (small bug fix or enhancement) T-docs-tests Type: Code change outside core library
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants