Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'disconnected' mode #17

Closed
dstreev opened this issue Dec 8, 2022 · 1 comment
Closed

'disconnected' mode #17

dstreev opened this issue Dec 8, 2022 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@dstreev
Copy link
Collaborator

dstreev commented Dec 8, 2022

Allow hms-mirror run without a connection to the "RIGHT" cluster.

In this case, it will not be possible to determine the existence of db's/tables on the right. So all output will assume they don't exist (no incrementals).

Use cases:

  • Schema Only Transfers
  • SQL, EXPORT_IMPORT, and HYBRID only when -is or -cs is used. This might be the case when the clusters are secure (kerberized), but don't share a common kerberos domain/user auth. So an intermediate or common storage location will be used to migrate the data.
  • Both clusters (and HS2 endpoints) are Kerberized, but the clusters are NOT the same major hadoop version. In this case, hms-mirror doesn't support connecting to both of these endpoints at the same time. Running in the disconnected mode will help push through with the conversion.

hms-mirror will run as normal, with the exception of examining and running scripts against the right cluster. It will be assumed that the RIGHT cluster elements do NOT exist.

The RIGHT_ 'execution' scripts and distcp commands will need to be run MANUALLY via Beeline on the RIGHT cluster.

@dstreev dstreev added the enhancement New feature or request label Dec 8, 2022
@dstreev dstreev self-assigned this Dec 8, 2022
@dstreev
Copy link
Collaborator Author

dstreev commented Dec 8, 2022

Note: This will be know as the "right-is-disconnected" option. Which means the process should be run from a node that has access to the "left" cluster. This is 'counter' to our general recommendation that the process should be run from the 'right' cluster.

dstreev added a commit to dstreev/hms-mirror that referenced this issue Dec 9, 2022
@dstreev dstreev added this to the 1.5.4.0-SNAPSHOT milestone Dec 9, 2022
@dstreev dstreev closed this as completed Dec 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant