fix(seatool deletes): When a record is deleted in seatool, it should be deleted from our system. #370
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… being processed correctly, due to a return in place of a continue.
| @@ -52,7 +52,8 @@ export const seatool_main = async (event: KafkaEvent) => { | |||
| submissionDate: null, | |||
| subject: null, | |||
| }; | |||
| return; | |||
| console.log(`SEATOOL DELETE EVENT: ${id}`); | |||
There was a problem hiding this comment.
i know this isn't really related, but i think we should have some way of ensuring that these list of fields are pulled from someone what ensures this inclues all the possible field for these records. cause at some point its gonna be forgotten. im not where where to get that list for.... but could be a good follow on ticket. thoughts?
benjaminpaige
approved these changes
Feb 8, 2024
|
🎉 This PR is included in version 1.7.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
|
🎉 This PR is included in version 1.5.0-val.1 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
This adds support for seatool delete events. When a record is deleted in seatool, it is deleted in our system and is no longer visible.
Linked Issues to Close
Closes https://qmacbis.atlassian.net/browse/OY2-27186
Approach
We had deferred the handling of seatool events until we had a better understanding of what was wanted.
This now adds support for seatool delete events.
On delete
It appears that we had already tried to implement this logic, but a 'return' in the place where a 'continue' should be was causing the null record to not be sent to opensearch.
Difference between return and continue here:
Using return:
When a record with no value is encountered, it logs the delete event and then immediately exits the function. This is significant because it means that no further processing will occur for any remaining records within the same recordKey group or any other groups.
This early exit from the function could lead to incomplete processing of data if there are multiple records to be processed.
Using continue:
The continue keyword is used inside the first loop when a record with no value is encountered, indicating a delete event. This causes the loop to skip the current iteration and move on to the next iteration of the loop.
This means that after logging the delete event, the code will continue processing any remaining records within the same recordKey group.
Assorted Notes/Considerations/Learning
None