Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]: enhance mapper to produce realistic test data #1009

Closed
aphethean1 opened this issue Sep 28, 2022 · 2 comments · Fixed by #2170
Closed

[Feature]: enhance mapper to produce realistic test data #1009

aphethean1 opened this issue Sep 28, 2022 · 2 comments · Fixed by #2170

Comments

@aphethean1
Copy link

Feature scope

Other

Description

As a user, I need to move data from a production database to a test database keeping realistic looking values for names, addresses, and so on, but not share actual personal information.

proposal

Add a dependency on 'faker' library
https://github.com/joke2k/faker/blob/master/setup.py#L73-L77

Enable mapper configuration similar to md5 to mask values during a run from tap to target.

benefits and use cases

As of today, it is possible to use a hashing function like md5() but with this feature, we will have data that "looks like" real data - especially good for test environments!

links

https://app.slack.com/client/TFG99TU9K/C013EKWA2Q1/thread/C013EKWA2Q1-1664297456.961009

@aphethean1 aphethean1 added kind/Feature New feature or request valuestream/SDK labels Sep 28, 2022
@stale
Copy link

stale bot commented Jul 18, 2023

This has been marked as stale because it is unassigned, and has not had recent activity. It will be closed after 21 days if no further activity occurs. If this should never go stale, please add the evergreen label, or request that it be added.

@edgarrmondragon
Copy link
Collaborator

Still open to support this. Faker is a pure-python dependency with an MIT license. We probably want to give users the option to configure locale and seed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants