-
Notifications
You must be signed in to change notification settings - Fork 35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
B-21322 TPPS ECS Scheduled Task - INT #14790
Merged
Merged
Changes from all commits
Commits
Show all changes
141 commits
Select commit
Hold shift + click to select a range
ad2c048
cleanup from older tpps feature work
traskowskycaci 0a6a812
cli works to run but needs more
traskowskycaci a4920f7
add step to deploy process-tpps task
traskowskycaci 900a116
additional steps for deploying process-tpps task based on https://git…
traskowskycaci 9893072
cleanup and some fine-tuning
traskowskycaci ce0b61c
rename process-tpps to process_tpps
traskowskycaci c4299cd
Merge branch 'main' into B-21322-MAIN
traskowskycaci b8a21f1
environment variable updates for pickup dir
traskowskycaci 136b023
return out of processing if no valid filepath found
traskowskycaci 0066beb
add temp logging for s3 path
traskowskycaci 5828df2
circle config for demo
traskowskycaci 437f3e3
Merge branch 'main' into B-21322-MAIN
traskowskycaci 3046c07
add demo env process tips
traskowskycaci bc7ac26
release demo env
traskowskycaci 27adca0
deploy to exp
traskowskycaci 0c744e8
release exp
traskowskycaci 9d7e15a
deploy to exp
traskowskycaci d48b52d
release exp
traskowskycaci eb37ded
Merge branch 'main' into B-21322-MAIN
traskowskycaci d4070bb
update filepath and deploy_dp3_tasks
traskowskycaci 3fdad8f
deploy to exp
traskowskycaci 76a9742
release exp
traskowskycaci 87213e7
comment out some things in config for now
traskowskycaci a8af25b
add some temp logging to ecs deploy script for debugging
traskowskycaci 40e70e1
deploy to exp
traskowskycaci a329f8d
add process-edis back in to deploy_dp3_tasks_steps
traskowskycaci 44d0238
remove temp logging
traskowskycaci 07f3d70
release exp
traskowskycaci 28e11d1
init process tpps flags before adding command
traskowskycaci 622b671
add pseudocode plan for processing specific filenames
traskowskycaci 3f5e66d
get the filepath to process in process_tpps.go to pass filepath to Pr…
traskowskycaci 65399c1
add logging to process_tpps.go and update SilenceUsage to false for p…
traskowskycaci 55f7530
deploy to exp
traskowskycaci 7e98df4
release exp
traskowskycaci 4c4b08a
add logging for env var troubleshooting
traskowskycaci 9f3b472
deploy to exp
traskowskycaci 8eff3b4
swap order for db config call
traskowskycaci e00a34c
db init updates
traskowskycaci e69e4e6
updates
traskowskycaci eafa86b
logging updates
traskowskycaci cdec799
init those dang flags again I guess
traskowskycaci e884c23
update logging
traskowskycaci 0a7883e
add db_region to exp.process-tpps.env
traskowskycaci 9e0e391
add dod_ca_package back in
traskowskycaci 649b480
remove second call to initProcessTPPSFlags in case that is causing re…
traskowskycaci 0913e7e
add logging to ecs deploy script
traskowskycaci 85a4a5b
fix filepath to hard-coded value for test
traskowskycaci 194f713
filepath fixes that may work
traskowskycaci 5858f50
release exp
traskowskycaci 0f6528f
s3 filepath doesn't need cleaned up, remove call to that
traskowskycaci be90c57
deploy to exp
traskowskycaci c9ba4bd
make process-tpps.env match process-edis.env to see if task revision …
traskowskycaci f6c544b
remove some logging from ecs deploy
traskowskycaci 0b3621c
config file changes as a test
traskowskycaci 03f00bd
add a bunch of logging to put target and task def
traskowskycaci b964eff
Revert "add a bunch of logging to put target and task def"
traskowskycaci e72fa94
release exp
traskowskycaci d8f9d86
cleanup unneeded logging statements
traskowskycaci 88e6fd9
filepath date improvements
traskowskycaci 2967aea
Merge branch 'main' into B-21322-MAIN
traskowskycaci 8225d79
deploy to exp
traskowskycaci b47efb5
release exp
traskowskycaci 3ca0c7f
deploy to exp
traskowskycaci f4ddf1c
Merge branch 'main' into B-21322-MAIN
traskowskycaci 542a18a
add logging to figure out go time.now issue with weird year
traskowskycaci 81e7935
add clamAV status check and s3 download to tmp file
traskowskycaci 9cded67
add logging to test deploy again
traskowskycaci 2acf786
modify gitlab.yml to deploy to exp
traskowskycaci 9489140
changes for make nonato_deploy_restore
traskowskycaci 2bf777b
update deploy of process tpps to deploy_tasks_dp3 in gitlab-ci.yml
traskowskycaci aaa7865
hard code stuff for testing purposes for now
traskowskycaci df4c9dc
add logging of s3 getObject response
traskowskycaci 80ccc3f
more general logging
traskowskycaci 09655ac
convert to utf-8 encoding and log metadata so we can see
traskowskycaci abde80e
get s3 object tags for av-status, not metadata
traskowskycaci c0546cb
adding a few new env vars for s3 bucket and s3 folder, logging, cleanup
traskowskycaci cb12e78
download file only if scan is clean
traskowskycaci 95de363
mutable tmp volume for tasks_dp3
traskowskycaci 25410a5
renaming func, some cleanup
traskowskycaci dc648fd
try with 0116 file with data in it
traskowskycaci 729c74f
add logging of contents of local file
traskowskycaci adf6091
still trying to log contents of tmp file
traskowskycaci 1fe9bf4
convert the logged output to utf16
traskowskycaci d664e0a
Merge branch 'main' into B-21322-MAIN
traskowskycaci 2ae6dc8
undo deploy to exp
traskowskycaci 07b9d73
changing how we process entries to fail gracefully if no matching pay…
traskowskycaci 464abf4
deploy to exp
traskowskycaci 309014b
cleanup some logs and remove hard-coded s3 key
traskowskycaci 42f4802
make the logged output preview more readable
traskowskycaci dc9b769
more log cleanup
traskowskycaci d7398d0
log count of successful and unsuccessful rows added to DB and log cou…
traskowskycaci ac730cb
Merge branch 'main' into B-21322-MAIN
traskowskycaci 246842d
additions and updates for testing
traskowskycaci 31313bb
removing unusued env var
traskowskycaci 46ccede
test updates
traskowskycaci c942d11
Merge branch 'main' into B-21322-MAIN
traskowskycaci fca8ab1
update timezone from utc to America/Chicago (central time)
traskowskycaci 3d3be68
more tests
traskowskycaci 9e55d69
temp hard coding of 0208 file
traskowskycaci 2f026a5
Revert "more tests"
traskowskycaci c11e466
Revert "test updates"
traskowskycaci ae95a26
refactor parser to use csv reader to handle complex messages
traskowskycaci 71e7806
Reapply "more tests"
traskowskycaci 9c11ebf
Reapply "test updates"
traskowskycaci 4afb196
remove unnecessary logs
traskowskycaci 35489c9
Merge branch 'main' into B-21322-MAIN
traskowskycaci 830020c
Merge branch 'B-21322-MAIN' into B-21322-INT
traskowskycaci ec9d2fd
remove unused arg
traskowskycaci 9308dd2
Merge branch 'B-21322-MAIN' into B-21322-INT
traskowskycaci dbc4c17
undo deploy to exp
traskowskycaci 3b704ef
Merge branch 'B-21322-MAIN' into B-21322-INT
traskowskycaci 6cdb07c
int specific commit - deploying int branch to exp
traskowskycaci e1631c4
Merge branch 'integrationTesting' into B-21322-INT
traskowskycaci c44796f
Merge branch 'main' into B-21322-MAIN
traskowskycaci 856eeef
Merge branch 'B-21322-MAIN' into B-21322-INT
traskowskycaci 26c0cb0
int specific commit - fixing gitlab merge issue for docker_app_image
traskowskycaci de4c3f7
int specific commit - deploy to exp
traskowskycaci 35e4990
test updates
traskowskycaci b5f42e5
Merge branch 'B-21322-MAIN' into B-21322-INT
traskowskycaci 7843d3e
Merge branch 'integrationTesting' into B-21322-INT
traskowskycaci b19737f
int specific commit - deploy integrationTesting to exp
traskowskycaci afee218
update AVStatusUNKNOWN comment
traskowskycaci 4b87765
lowercase err messages
traskowskycaci c7f7d5c
Merge branch 'B-21322-MAIN' into B-21322-INT
traskowskycaci 72dfefc
set processTPPS SilenceUsage to true
traskowskycaci 0bac6d8
add comment explaining AVStatusUNKNOWN usage
traskowskycaci d51739b
Merge branch 'B-21322-MAIN' into B-21322-INT
traskowskycaci 0e5e0da
check tpps flags in config check
traskowskycaci 121aaa4
Merge branch 'B-21322-MAIN' into B-21322-INT
traskowskycaci 3dd4545
int specific commit - deploying latest 21322 int to exp
traskowskycaci ef8db0b
int specific commit - deploy integrationTesting to exp
traskowskycaci c4ace72
Merge branch 'integrationTesting' into B-21322-INT
traskowskycaci 37c6afc
Merge branch 'integrationTesting' into B-21322-INT
traskowskycaci f4f891f
int specific commit - deploy B-21322-int to exp
traskowskycaci b0d4aed
add init flags check
traskowskycaci f98a957
Merge branch 'B-21322-MAIN' into B-21322-INT
traskowskycaci 31dba9b
int specific commit - undo deploy 21322 to exp
traskowskycaci 7c62f49
remove changes from config.yml as it's not used anymore
traskowskycaci 32d1753
Merge branch 'B-21322-MAIN' into B-21322-INT
traskowskycaci 42ec54f
Merge branch 'integrationTesting' into B-21322-INT
traskowskycaci 5cd0ee3
Merge branch 'integrationTesting' into B-21322-INT
traskowskycaci File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Leaving higher level env script updates as commented out/not added yet. Will have a follow up PR when necessary higher level AWS changes are in place to pull in the script updates to this file.