Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added some benchmarking tests for deduplicate filter #2466

Merged
merged 5 commits into from
Jun 11, 2020
Merged

Added some benchmarking tests for deduplicate filter #2466

merged 5 commits into from
Jun 11, 2020

Conversation

yashrsharma44
Copy link
Contributor

@yashrsharma44 yashrsharma44 commented Apr 19, 2020

This WIP PR intends to add benchmarking tests to Thanos for the deduplication filter.
Currently, I have used some ideas from this PR #2118

Fixes #2174

Signed-off-by: Yash [email protected]

  • I added CHANGELOG entry for this change.
  • Change is not relevant to the end user.

Changes

  • Added a benchmarking test in fetcher_test.go.

Verification

Some details of the benchmarking result after I ran them -

goos: linux
goarch: amd64
pkg: github.com/thanos-io/thanos/pkg/block
BenchmarkDeduplicateFilter_Filter/Block-10/#00-4     	   16364	     77193 ns/op	    3544 B/op	      48 allocs/op
BenchmarkDeduplicateFilter_Filter/Block-10/#01-4     	    8421	    155745 ns/op	    9502 B/op	     134 allocs/op
BenchmarkDeduplicateFilter_Filter/Block-100/#00-4    	     126	   8885819 ns/op	   31375 B/op	     356 allocs/op
BenchmarkDeduplicateFilter_Filter/Block-100/#01-4    	      74	  15998391 ns/op	   93495 B/op	    1066 allocs/op
BenchmarkDeduplicateFilter_Filter/Block-100/#02-4    	 1222910	      1051 ns/op	     280 B/op	       4 allocs/op
BenchmarkDeduplicateFilter_Filter/Block-100/#03-4    	 1000000	      1034 ns/op	     280 B/op	       4 allocs/op
BenchmarkDeduplicateFilter_Filter/Block-1000/#00-4   	       1	1777479341 ns/op	  640752 B/op	    6726 allocs/op
BenchmarkDeduplicateFilter_Filter/Block-1000/#01-4   	       1	3027688695 ns/op	 1919616 B/op	   20154 allocs/op
BenchmarkDeduplicateFilter_Filter/Block-1000/#02-4   	 1250241	       874 ns/op	     280 B/op	       4 allocs/op
BenchmarkDeduplicateFilter_Filter/Block-1000/#03-4   	 1343001	       888 ns/op	     280 B/op	       4 allocs/op
BenchmarkDeduplicateFilter_Filter/Block-1000/#04-4   	 1000000	      1016 ns/op	     280 B/op	       4 allocs/op
BenchmarkDeduplicateFilter_Filter/Block-1000/#05-4   	 1278493	       991 ns/op	     280 B/op	       4 allocs/op
BenchmarkDeduplicateFilter_Filter/Block-10000/#00-4  	       1	191091773100 ns/op	 6879488 B/op	   66758 allocs/op
BenchmarkDeduplicateFilter_Filter/Block-10000/#01-4  	       1	354931689492 ns/op	20635776 B/op	  200250 allocs/op
BenchmarkDeduplicateFilter_Filter/Block-10000/#02-4  	 1000000	      1036 ns/op	     280 B/op	       4 allocs/op
BenchmarkDeduplicateFilter_Filter/Block-10000/#03-4  	 1213599	      1014 ns/op	     280 B/op	       4 allocs/op
BenchmarkDeduplicateFilter_Filter/Block-10000/#04-4  	 1000000	      1005 ns/op	     280 B/op	       4 allocs/op
BenchmarkDeduplicateFilter_Filter/Block-10000/#05-4  	 1000000	      1009 ns/op	     280 B/op	       4 allocs/op
BenchmarkDeduplicateFilter_Filter/Block-10000/#06-4  	 1000000	      1003 ns/op	     280 B/op	       4 allocs/op
BenchmarkDeduplicateFilter_Filter/Block-10000/#07-4  	 1000000	      1053 ns/op	     280 B/op	       4 allocs/op
PASS
ok  	github.com/thanos-io/thanos/pkg/block	577.057s

Post-Thoughts

I am currently thinking of various other test cases for the benchmarking, so this PR would be updated as time goes.
I would appreciate suggestions from the maintainers and contributors!

@yashrsharma44
Copy link
Contributor Author

@bwplotka @kakkoyun Would love a round of review for this PR!

@yashrsharma44
Copy link
Contributor Author

I am not sure if the failing tests are related to this PR?

Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM other than CHANGELOG! (:

CHANGELOG.md Outdated Show resolved Hide resolved
@bwplotka bwplotka merged commit 483572d into thanos-io:master Jun 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add micro benchmarks to fetcher.go filters.
2 participants