implement typographic noise function #19
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
stevebachmeier
changed the title
stevebachmeier
commented
Mar 28, 2023
| """Abie's implementation of typographical noising""" | ||
| err = "" | ||
| i = 0 | ||
| while i < len(truth): |
There was a problem hiding this comment.
This is where a followup PR will try and implement numba or otherwise refactor.
stevebachmeier
commented
Mar 28, 2023
stevebachmeier
commented
Mar 28, 2023
stevebachmeier
commented
Mar 28, 2023
| ) | ||
|
|
||
| assert noised_data.equals(noised_data_same_seed) | ||
| assert not noised_data.equals(noised_data_different_seed) | ||
| assert not data.equals(noised_data) | ||
| # TODO: Confirm correct columns exist once the interface functions |
There was a problem hiding this comment.
I copied this deletion from Jim's open PR since we decided this is all too complicated given the number of columns and functions to be implemented.
stevebachmeier
commented
Mar 28, 2023
|
|
||
|
|
||
| # TODO: refactor this into its own test parameterized by noise functions | ||
| def _validate_seed_and_noise_data(func, column, config): |
There was a problem hiding this comment.
A followup PR will move this method out of the existing tests and do these checks as their own tests.
stevebachmeier
requested review from
albrja,
hussain-jafari,
mattkappel,
ramittal and
rmudambi
as code owners
rmudambi
approved these changes
Mar 28, 2023
zmbc
reviewed
Mar 29, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Title: Implement typographnic noising
Description
This is one of probably a few PRs to implement typographic noising.
This one includes the function as well as some unit tests.
Followup PRs may include:
loop over every character in every string of potentially-noised rows)
Testing
This PR includes a (very slow) unit test that is passing
To discuss: how to add an integration test?