Cleanup deleted token references.

[ravi-sawlani-yral]

Motivation

Tokens belonging to anonymous users have been removed. Consequently, we must delete any references to such tokens.

Tasks

• Refrain from displaying tokens that lack a valid governance canister.
• Remove tokens from the store if they do not point to valid canisters.

[harshita-srivastava-yral]

• Frontend not to fetch the tokens whose data is not available
• IF there is any invalid entry in individual user database, remove that token

[harshita-srivastava-yral]

• Started working on the same

[harshita-srivastava-yral]

• Made a function using grpc and firestore to filter out the tokens
• Removal of tokens needs to happen on server side and calling needs to happen from frontend

[harshita-srivastava-yral]

• We can use canister info module hash
• Where do we store all the function - We will take all token list? This is not required, we will query when the frontend is loaded on pagination.
• PR for detection of all invalid token on frontend

[harshita-srivastava-yral]

• We will raise PR for filtering the invalid tokens from the frontend

[harshita-srivastava-yral]

• Big queries is being used for analytics today. (large scale queries)
• 1. Object store (canister) 2. Realtime capabilities (Firebase) 3. Relation for range and aggregarate queries (cloudfare KV) 4. Something like redis for object query (Bigquery).
• 2. We have these 4 needs as of now
• 3. We need to cleanup from firestore and big query
• 4. We just to cleanup this for 1 time
• 5. Script writing will be closed today
• 6. Before running the script, ensure we have taken the screenshot before deleting the prod data

[harshita-srivastava-yral]

• Blocker (Need canister APIs which @harsh-mahajan-yral needs to resolve. Should be part of onboarding document)

[harshita-srivastava-yral]

• Draft PR is done.
• Need access to big query DB @harsh-mahajan-yral
• Final PR should be raised today

[harshita-srivastava-yral]

• Testing of the code and stuck on the access
• Create an own account for testing and once we have confidence on the code we can merge

[harshita-srivastava-yral]

• Fetching users and canisters to find the tokens they have
• PR expected to be raised today.