Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pytx] Experiment with Querying PDQ Rotations Natively #1730

Open
6 tasks
Dcallies opened this issue Jan 2, 2025 · 0 comments
Open
6 tasks

[pytx] Experiment with Querying PDQ Rotations Natively #1730

Dcallies opened this issue Jan 2, 2025 · 0 comments
Labels
pdq Items related to the pdq libraries or reference implementations python-threatexchange Items related to the threatexchange python tool / library

Comments

@Dcallies
Copy link
Contributor

Dcallies commented Jan 2, 2025

PDQ and the pdqhash binding have a native "generate all rotations" hash method. In previous PRs, we experimented with rotating the image itself, but it is much faster to use the native rotation function.

This change will be involved, as we only need to hash rotations some of the time, and not all the time. For SignalTypes that support it without any overhead, querying by rotations should probably be the default.

Tasks

  • Add new "SignalTypeWithNativeRotations" mixin interface
    • It should have a separate hash_with_rotations(file-like-object IO[bytes])
      • This should return a dict of RotationType -> str hash
  • Adjust the SignalTypeIndex to use rotations by default if the SignalType supports it
    • It needs to at least work for PDQ
    • Adjust the default index match type also include the rotation, which should be included in the native __str__() method for debugging
@Dcallies Dcallies added pdq Items related to the pdq libraries or reference implementations python-threatexchange Items related to the threatexchange python tool / library labels Jan 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pdq Items related to the pdq libraries or reference implementations python-threatexchange Items related to the threatexchange python tool / library
Projects
None yet
Development

No branches or pull requests

1 participant