[Issue #3271] Setup structure of opportunity attachment transformation (minus file logic) #3443
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
chouinar
commented
Jan 7, 2025
| ) | ||
|
|
||
|
|
||
| def test_uploading_attachment_staging(db_session, enable_factory_create, tmp_path): |
There was a problem hiding this comment.
Because these tests use the staging table factories heavily, we don't need the specific tests anymore
mikehgrantsgov
approved these changes
Jan 15, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #3271
Time to review: 10 mins
Changes proposed
Add a subtask to transform the opportunity attachment table
DOES NOT handle the files themselves (next PR will handle that)
Context for reviewers
The transformation code is bulky and has a lot of repeated patterns that needed to be setup. This PR gets all of that setup and working for the opportunity attachment table, except for the logic regarding moving the file from the staging table to our s3 bucket (which has a lot of extra fun complexities that will need to be handled special). Every other column in the table we want is copied over and functionally working with these changes.
The follow-up to this likely will refactor a bit to make the file logic reasonable, but figured it was best to start with a pattern that looks like the other transformations we've done.
Additional information
From a clean database, you can run this locally by doing
make consoleand then using our factories to generate dataI then modified
TransformOracleDataTaskto only run the TransformOpportunity and new TransformOpportunityAttachment classes which successfully copied to my local opportunity_attachment table as expected: