Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make bloom filters simpler #1021

Merged
merged 1 commit into from
Apr 7, 2015
Merged

Make bloom filters simpler #1021

merged 1 commit into from
Apr 7, 2015

Conversation

krl
Copy link
Contributor

@krl krl commented Apr 6, 2015

These did not work before, and had some unnecessary complexity.

Now the filters use only one hashing function, no bignum arithmetic, and gets the additional bit positions by repeatedly hashing the result of prior hash.

Since we're not concerned about crypto hashing here, this should be a win.

External interfaces unchanged.

@whyrusleeping whyrusleeping added the status/in-progress In progress label Apr 6, 2015
}

// fmt.Println(f)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove as well

@whyrusleeping
Copy link
Member

if we are going to go with a single hash function, i prefer not to use fnv, it doesnt have very good properties for this application (as described here: http://bretmulvey.com/hash/6.html)

@jbenet
Copy link
Member

jbenet commented Apr 6, 2015

there probably are some fast hash function impls in the golang.org/x packages. google's been leading the development of fastest non-crypto hashes.

Anyone recall why we needed multiple hash functions in the first place? @whyrusleeping ( i mean flexibility is always good if it can be afforded, just wondering if there was a specific reason we needed it )

@whyrusleeping
Copy link
Member

my only reason for multiple hash functions was the flexibility to allow the user to use whatever they wanted

@jbenet
Copy link
Member

jbenet commented Apr 6, 2015

yeahh that's nice. :/ your call @whyrusleeping

@whyrusleeping
Copy link
Member

eh, if we want it later its an easy change

@krl krl force-pushed the bloom-filter-fix branch from c311bda to b5ae6d3 Compare April 7, 2015 15:34
@krl
Copy link
Contributor Author

krl commented Apr 7, 2015

Pushed a fixed up version, using jenkins hash for now, as recommended by http://bretmulvey.com/hash/7.html (thx @whyrusleeping)

Also added test coverage for merging filters together.

@krl
Copy link
Contributor Author

krl commented Apr 7, 2015

if you are ok with random-ish github dependencies that is (MIT license), otherwise i could look around.

@krl krl force-pushed the bloom-filter-fix branch from b5ae6d3 to 3d8e96a Compare April 7, 2015 16:02
These did not work before, and had some unnecessary complexity.

Now the filters use only one hashing function, no bignum arithmetic, and gets the additional bit positions by repeatedly hashing the result of prior hash.

Since we're not concerned about crypto hashing here, this should be a win.

External interfaces unchanged.
"ImportPath": "github.com/whyrusleeping/go-metrics",
"Rev": "1cd8009604ec2238b5a71305a0ecd974066e0e16"
},
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be fine

@jbenet
Copy link
Member

jbenet commented Apr 7, 2015

LGTM!

@jbenet
Copy link
Member

jbenet commented Apr 7, 2015

(i'll merge when tests green)

@whyrusleeping
Copy link
Member

@krl 👍 Looks good :)

jbenet added a commit that referenced this pull request Apr 7, 2015
@jbenet jbenet merged commit 096420c into master Apr 7, 2015
@jbenet jbenet removed the status/in-progress In progress label Apr 7, 2015
@jbenet jbenet deleted the bloom-filter-fix branch April 7, 2015 16:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants