Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Migration to remove duplicate users #1040

Merged
merged 10 commits into from
Jun 22, 2023
Merged

Conversation

jcbcapps
Copy link
Contributor

@jcbcapps jcbcapps commented Jun 7, 2023

SC-2026

Proposed changes

Adds migration to remove duplicate users. This is accomplished by:

  • Merging the mySpace of each user into one
  • If any of the duplicate users have a dark theme, we ensure that the final user has that value
  • Keep the displayName of the original user
  • Delete all duplicate users
  • Finally, make the userId a unique field

Reviewer notes

To test locally, you'll need to manually create some duplicate users at http://localhost:8888/db/dev/users

  • Make sure that each user has the same userId

Try running yarn migrate up and refresh http://localhost:8888/db/dev/users (if yarn migrate up doesn't work, you may need to run yarn migrate down first):

  • Any duplicate users should be consolidated into one, that contains a mySpace array featuring all collections from the deleted duplicates
  • The remaining user should also only have one FeaturedShortcut and GuardianIdeal
  • If any of the duplicate users has a theme set to dark we give the remaining user that value
  • Try to add another user manually with the same userId and you should see an error thrown by Mongo. An example: Something went wrong: BulkWriteError: E11000 duplicate key error collection: dev.users index: userId_1 dup key: { userId: "[email protected]" }

Setup

Start the system

yarn services:up
yarn dev
cd ../ussf-portal-cms
yarn dev

Login to the portal http://localhost:3000

Start storybook

yarn storybook

Login to storybook http://localhost:6006, though the command above should open it for you


Code review steps

As the original developer, I have

  • Met the acceptance criteria
  • Created new stories in Storybook if applicable
  • Created/modified automated unit tests in Jest
  • Created/modified automated E2E tests
  • Followed guidelines for zero-downtime deploys, if applicable
  • Use ANDI to check for basic a11y issues

As a reviewer, I have

Check out our How to review a pull request document.


Screenshots

@shortcut-integration
Copy link

@jcbcapps jcbcapps changed the title Sc 2026/fix duplicate user bug fix: Migration to remove duplicate users Jun 8, 2023
@jcbcapps jcbcapps marked this pull request as ready for review June 8, 2023 20:52
@jcbcapps jcbcapps requested a review from a team as a code owner June 8, 2023 20:52
Copy link
Contributor

@gidjin gidjin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. I ran this against a sample of production data and had no issues. I also spot checked a handful of records and they appear to have been merged properly.

})

module.exports.down = (next) => {
// Do nothing. We don't need to put the duplicate users back.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thought: Theoretically we could remove the unique index that is added in the down, but I'm not sure we want to do that since likely we should have had it from the start. Long way to say I'm good with the down of this being a no-op.

@jcbcapps jcbcapps merged commit eec8564 into main Jun 22, 2023
@jcbcapps jcbcapps deleted the sc-2026/fix-duplicate-user-bug branch June 22, 2023 20:11
abbyoung pushed a commit that referenced this pull request Jul 10, 2023
* Add migration to remove duplicate users

* Update to handle FeaturedShortcuts and GuardianIdeal

* Make userId a unique field

* Remove comment

* Update test to include custom collection

---------

Co-authored-by: John Gedeon <[email protected]>
jcbcapps added a commit that referenced this pull request Jul 10, 2023
* Add basic dndkit implementation

* Add sorting to drag and drop

* Sort visibleBookmarks to match sorted dnd items array

* Fix removed bookmarks breaking drag and drop

* Update comment

* Add CustomBookmark

* Remove unused activeId and fix Generic Object Injection Sink

* add type to function

* update ui when add/remove and save on order change

* Update tests

* Update visibleBookmarks and removedBookmarks when bookmarks prop changes

* Fix import

* Update test

* Memoize bookmarks

* Fix flicker

* Add mutation

* Move logic back to MySpace

* Add MySpace context

* Move component logic to context

* Move drag-and-drop logic to context

* Remove draggableWidgets in favor of using mySpace

* Add refetchQueries back

* Remove unnecessary function

* Add activationConstraint

* Change cursor when hovering/dragging collection

* Prevent KeyboardSensor for drag-and-drop from firing when typing a collection title

* feat: a11y: creates alternative means to access inaccessible information in latest announcements carousel (#1032)

* hides carousel dots from screen reader

* makes it so user can keyboard skip annoucements carousel

* styles skip links to behave consistently across pages and makes it so they are hidden when not in use

* turns off poorly implemented accessibility features on slider as we are going to provide an accessible version instead

* adds placeholder for intended output of latest announcements

* links carousel to alternative page area where we will list annoucements statically

* updates id on link to match id on page element

* removes additions to news page in favor of creating annoucements page separate

* adds syntax to output latest announcements on the page

* changes name of announcements pags variables

* adjusts markup of annoucement headr to better utilize semantic html

* fixes eslint type error

* add tests for new announcements page

* removes unused class

* ovverides jest axe heading order warning

* removes ununsed import from test

---------

Co-authored-by: Abigail Young <[email protected]>

* chore: pin debian 11 docker image tags (#1038)

pin debian 11 images

* build: update DoD PKI Bundle to v5.12 (#1041)

update DoD PKI Bundle to v5.12

* build: fix the team names (#1042)

Co-authored-by: minh <[email protected]>

* security: remove typescript codeql language (#1033)

remove typescript codeql language

* build: allow design, eng, and owners merge to main 2 (#1043)

remove org from team names

* fix: a11y: adjusts heading order on myspace page (#1034)

* refactors guardian ideal component to utilize header elements better. adds h3 invisible to typical users to match other components in structure list, and converts h1 to h4 to make the heading order match other components

* converts got feedback heading to an h2 to align with the heading level of other elements on the myspace page

* updates test to match component syntax

---------

Co-authored-by: John Gedeon <[email protected]>
Co-authored-by: minh <[email protected]>

* feat: Filter search results client (#1027)

* Implement basic component

* Style updates

* Update to use form

* Add function to build queries

* Disable default dropdown value

* Add labels query and add labels to query building

* Add query for tags

* Remove tags query and form element

* Move SearchFilter component

* Store filter options in an array

* Remove console.log

* Add SearchContext

* Add SearchFilter component back

* Add searchPageFilters to searchContext

* Clear the searchQuery when not on /search

* Update form submission to include searchQuery

* Add filter reset

* Move searchPageFilters from context to component

* Save filter to local storage

* Style updates

* Dark mode styles

* Fix button style

* Add storybook component and set max-width

* Add back missing SearchBanner, update styles and fix tests

* Add tests

* Update test

* Add test to bump coverage

* Add test to bump coverage

* Change it to test

* Update headers

* Accessibility updates

* Add LD flag with tests

* Remove disabled prop

* Fix what I broke

* Updates to label search

* Allow user to select default option

* Move searchPageFilters to context and update param name

* Update form submit

* Wrap category value in double quotes if more than one word

* Trim search query to 200 characters before submitting

* Update test

* Add tests and remove localStorage for storing filter items

* Add tests

* Remove unnecessary form and set searchQuery when component mounts

* Ensure filter items remain selected after search

* Place EPubs component above SearchFilter

* Update tests and add renderWithSearchContext helper

* Remove searchPageFilters array and add filter items directly to searchQuery

* Update test

* Accessibility updates

* Fix multi-word labels

* Remove console.logs

* Remove unused icon

* Remove unnecessary code

---------

Co-authored-by: Shauna Keating <[email protected]>
Co-authored-by: John Gedeon <[email protected]>

* build: error on no-console unless approved method (#1035)

* build: error on no-console unless approved method

* build: remove unnecessary no-console disables

---------

Co-authored-by: Shauna Keating <[email protected]>

* docs: document main branch team merge restriction (#1044)

document gh main branch team restriction

Co-authored-by: John Gedeon <[email protected]>

* chore(deps): update github-actions (#1011)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update devdependencies (#1046)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore: add otel package rule to renovate config (#1049)

add otel packagerule to renovate

* ci: delete unused cache-to-ecr.yml workflow (#1048)

delete unused workflow

* fix(deps): update dependencies (#935)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* feat: a11y: assigns unique page titles to each page in the portal (#1037)

* Override because the design files changes are removing unused variables from Storybook stories. Mallory is OOO, and Shauna created the original PR. Changes have been re-reviewed by @gidjin and team agreed to merge in.

* adds page title to myspace page

* adds space to my space page title

* adds page title to 404 page

* adds page title to 500 page

* adds page title to log in agreement  page

* adds page title to news & announcements  page

* adds page title to news articles  page

* adds page title for search results

* adds page title for settings

* adds page title to sites and apps, and browser update page

* adds page title to documentation page

* adds page title to about pages

* adds page title with dynamic article title to single article page

* moves myspace head info out of root div and into a wrapping react fragment instead

* adds page title for generic error page

* build: allow design, eng, and owners merge to main 2 (#1043)

remove org from team names

* Pass page title as server side prop to all pages except error pages. For those, use Head component.

* update tests

* add test coverage, fix missing pagetitle, remove unnecessary code

* more cleanup

* clean up lint warnings

---------

Co-authored-by: minh <[email protected]>
Co-authored-by: Abigail Young <[email protected]>

* fix: Migration to remove duplicate users (#1040)

* Add migration to remove duplicate users

* Update to handle FeaturedShortcuts and GuardianIdeal

* Make userId a unique field

* Remove comment

* Update test to include custom collection

---------

Co-authored-by: John Gedeon <[email protected]>

* Remove cmsId from query

* Remove unused classes

* Remove DraggableWidgets

* Add renderWithMySpaceContext and update tests

* Add renderWithMySpaceAndModalContext and update tests

* Remove comments

* Update types

* Disable drag-and-drop in certain instances

* Update test

* Add LD flag for drag-and-drop collections

* Ignore type

* Remove comment

* Remove console.log statements

* Add test file for myspaceContext

* Add test

* Add tests

* Add test

* Add test to check for light theme

* Update tests and make AddWidget consume context

* Ignore mock functions for test coverage

* Change it to test

* Update tests and remove unnecessary default params

* Update test to check collection without title

* Add test to check for no username

* Add test to check for an image

* Add test for Featured Shortcuts

* Update test and make props required

* Remove empty string

* Update storybook

* fix: handle removed articles linked in announcements and missing articles for cms users (#1057)

* fix: announcement with CTA to deleted article links to 404

* fix: return 404 if article not found and cms user

* chore(deps): Upgrade USWDS, React USWDS, and Storybook (#1058)

* Signing WIP commit

* Save point for version that compiles. Some missing styles but looking good

* fix a couple compile errors

* add back the postinstall we need to copy vendor files

* case sensitivity in test somehow got out of sync

* fix ordering of stylesheets to correct discrepancies

* cleaning up wip files

* more cleanup for consistency

* remove unused var theme-hero-image

* chore(deps): update storybook monorepo to v7

* fix storybook upgrade

* update storybook again

* disable storybook auto doc gen

* fix strange error in modalContext tests

* split storybook build from happo build

* upgrade to latest storybook

* use shortened path for index.scss file

* update comment

* moved styles that resolve duplicate alert icon issue into storybook only stylesheet

* adjusts css to work on storybook and in production to hide random extra icon

* adds slim attribute to stories for alerts to match what they used to look like

* remove deprecated file

* add missing import statements

* workaround for prefixing css in storybook

* putting back initIcons because we need those!

* remove unneccesary comments

---------

Co-authored-by: John Gedeon <[email protected]>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Shauna Keating <[email protected]>

* fix: a11y: adjusts link names to improve usability on assistive tech (#1055)

* adds open in new window text to feautured shortcuts

* adds opens in a new window note to orbit link in footer

* adds open in a new window tag to footer links

* adds note to screen reader users about functionality of color theme switcher

* adds unique button names for each bookmark in collection

* refectors widget headers and widget settings buttons to give each button a unique settings button label that tells the user what widget its for

* adds context to custom collections controls about what custom collection would be edited or deleted for screenreader users

* labels remove recent news button more specifically

* adds link name to add to myspace option for screenreader users

* updates Add to My Space strings to include bookmark label

* adds data test id to add to myspace button to help the test find it correctly with using the text inside the button no longer being an option

* adds data test id to add to myspace button to help the test find it correctly with using the text inside the button no longer being an option

* updates featured shortcut test and guardian ideal test to reflect name changes on featured shortcut item and widget settings

* undo test changes that did not work

* removes weird space

* removes data test id that didnt work

* moves h3 tag to correct location

* adjusts Guardian Ideals header to match its test query

* makes adjustments to recent news widget test to reflect wdiget component changes

* updates Featured Shortcuts test file

* makes changes to Applications Table test file

* refactors Removable Bookmarks test to account for link name changes

* adjusts implementation of h3 on widget components

* fixes most of sites & applications tests

* adjusts Custom Collection tests

* adjusts widget back so it doesnt break a ton of tests

* bookmark should take a label prop

* pass in label to bookmark

* fix various tests

* fix: guardian ideal is singular

---------

Co-authored-by: John Gedeon <[email protected]>

* Update to use mock myspace context

* Add comment and cleanup

* fixing rebase whoops

---------

Co-authored-by: John Gedeon <[email protected]>
Co-authored-by: Shauna Keating <[email protected]>
Co-authored-by: Abigail Young <[email protected]>
Co-authored-by: minh <[email protected]>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Shauna Keating <[email protected]>
gidjin pushed a commit that referenced this pull request Jul 10, 2023
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: John Gedeon <[email protected]>

## [4.22.0](v4.21.0...v4.22.0) (2023-07-10)


### Features

* a11y: assigns unique page titles to each page in the portal ([#1037](#1037)) ([7b21d50](7b21d50))
* a11y: creates alternative means to access inaccessible information in latest announcements carousel ([#1032](#1032)) ([3f9a2ff](3f9a2ff))
* Drag-and-drop entire collections ([#1056](#1056)) ([f415265](f415265))
* Filter search results client ([#1027](#1027)) ([35f6f90](35f6f90))


### Bug Fixes

* a11y: adjusts heading order on myspace page ([#1034](#1034)) ([4c6d1aa](4c6d1aa))
* a11y: adjusts link names to improve usability on assistive tech ([#1055](#1055)) ([41854ca](41854ca))
* **deps:** update dependencies ([#935](#935)) ([c15e83d](c15e83d))
* handle removed articles linked in announcements and missing articles for cms users ([#1057](#1057)) ([e9a2459](e9a2459))
* Migration to remove duplicate users ([#1040](#1040)) ([eec8564](eec8564))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants