Easy Lightweight Data Migrations

[tnaum-ms]

Easy Lightweight Data Migrations

We’re excited to propose Easy Data Migrations for MongoDB clusters, a feature designed to simplify small-scale data migrations by leveraging a user-friendly copy-and-paste experience. This feature is ideal for smaller datasets where all data can be moved through the user's local machine. Here’s the concept, and we’d love your input to refine and improve it further.

This feature is scheduled for development soon, and we’d love to hear your thoughts!

You can track the release it’s planned for, explore the feature’s details and challenges, and stay updated on its progress by clicking here #2488.

Your feedback and votes are invaluable in helping us fine-tune priorities and ensure this extension truly serves everyone’s needs. We think we know our users, but your input can make all the difference - after all, you know your workflows best!

Proposed Feature Overview

This feature provides an intuitive way to migrate collections between databases, servers, or clusters by mimicking the familiar “copy-and-paste” paradigm. It offers flexibility for handling conflicts, tracking progress, and ensuring user control throughout the migration process.

Core Features

1. Copy-and-Paste Collection Workflow

   • Use the context menu to "Copy" a collection.
   • "Paste" the copied collection into:
     • A different database in the same cluster.
     • A database on a different server or cluster.
     • An existing collection, with options for conflict resolution.

2. Conflict Resolution Options

   • Provide choices when conflicts arise during migration:
     • Document Conflict (_id exists): Overwrite, skip, or rename conflicting documents.
     • Schema Validation Issues: Skip invalid documents and log errors, or pause for user intervention.
   • Handle naming conflicts by prompting the user to confirm a new collection name when a collection with the same name exists.

3. Monitoring and Abort Options

   • Display real-time progress during the migration:
     • Total documents copied.
     • Data transferred (in MB).
     • Time elapsed.
     • Number of errors encountered.
   • Allow the user to abort the operation at any point.

4. Error Logging and Review

   • Log all errors encountered during the migration for user review.
   • Provide actionable insights to help address issues, such as invalid schemas or conflicts.