hms-mirror

The Hikari Connection Pool settings are causing intermittent connection failures, cause table transfer failures.

Doc

hms-mirror

Second run in WebUI Fails

dbRegEx not being processed. Throws MISC_ERROR because it can't find any databases.

Legacy DBPROPERTIES are causing ERROR when attempting to set on CDP

hms-mirror

What's New
BETA Iceberg conversion support for the STORAGE_MIGRATION data strategy. See Iceberg Conversion for more details.

Bugs (Fixed)
Add to connection init the ability to set the queue AND trigger an engine resource
https://github.com/cloudera-labs/hms-mirror/issues/168
Validate SQL elements before making changes to the Cluster
Extend the HS2 connection validation with Tez task validation
The reset-to-default-location doesn't seem to be working in v2
Issue with jobs not completing when some schema's were already present fixed.
Address lingering connections after run completes.
Fixed counters for CLI screen output.
Fixed an issue with tables being processed multiple times under some conditions.
Partition discovery for SHADOW table when source is a Managed table shouldn't try to build partitions with ALTER
LEFT side SQL when running 'execute' mode for SQL data strategy isn't being run.
CLI App version fails when attempting to set 'concurrency' option
MSCK for Shadow table not generated when 'metastore_direct' on the LEFT isn't defined

hms-mirror

Added saveWorkingTables to cli and web ui options.

hms-mirror

What's New
BEHAVIOR CHANGE - Drop any shadow table definitions created during migration automatically

hms-mirror

Error event is not logged for table skipped because RIGHT already has the table
Would be great to be able to omit warnings from the end of logs
Be able to reduce spring framework related log entries

hms-mirror

Bug Fixes

• Doubling of mapping locations for partitions in distcp for STORAGE_MIGRATION.
• STORAGE_MIGRATION is setting ACID to ON, regardless.

What's New

• Validate JDBC Jar Files in config.
• Ability to turn-on strict mode for Storage Migration. This will cause distcp to fail when non-standard locations are used. To turn off, use the -sms|--storage-migration-strict flag via the CLI.

Behavior Changes

The default behavior for Storage Migration 'strict' has changed from true to false. The intent behind the strict mode was to ensure distcp would fail when non-standard locations are used. The combination of metastore_direct and knowing the partition location details gives us a better chance on making these mappings work for distcp. When the scenario arises, we do HIGHLY recommend that you validate the plans created. The new default behavior will allow distcp to continue when non-standard locations are encountered, while throwing a warning. This will allow the migration to continue, but you should validate the results.

You will need to reset the `strict` mode flag in you configuration yaml. It will be set to `true` if the configuration was created before this release. You will need to set it to `false` to maintain this new 'preferred' behavior.

transfer:
  storageMigration:
    strict: false

hms-mirror

Bug Fixes

• Doubling of mapping locations for partitions in distcp for STORAGE_MIGRATION.
• STORAGE_MIGRATION is setting ACID to ON, regardless.

What's New

• Validate JDBC Jar Files in config.
• Ability to turn-on strict mode for Storage Migration. This will cause distcp to fail when non-standard locations are used. To turn off, use the -sms|--storage-migration-strict flag via the CLI.

Behavior Changes

The default behavior for Storage Migration 'strict' has changed from true to false. The intent behind the strict mode was to ensure distcp would fail when non-standard locations are used. The combination of metastore_direct and knowing the partition location details gives us a better chance on making these mappings work for distcp. When the scenario arises, we do HIGHLY recommend that you validate the plans created. The new default behavior will allow distcp to continue when non-standard locations are encountered, while throwing a warning. This will allow the migration to continue, but you should validate the results.

You will need to reset the `strict` mode flag in you configuration yaml. It will be set to `true` if the configuration was created before this release. You will need to set it to `false` to maintain this new 'preferred' behavior.

transfer:
  storageMigration:
    strict: false

hms-mirror

Fixed

• Property overrides - po|pol|por - are not honored in v2, were working in 1.6.x: #156
• Partition auto-discovery config value is misconfigured during setup: #155
• Enhancement for hms-mirror setup: #154
• Remove automatic DB location as db_name.db in db location: #153
• Add support for Hive table backup in DISTCP mode: #152

New

• Beta Flag to be used for future beta features.

hms-mirror

BETA RELEASE

Fixed

• Property overrides - po|pol|por - are not honored in v2, were working in 1.6.x: #156
• Partition auto-discovery config value is misconfigured during setup: #155
• Enhancement for hms-mirror setup: #154
• Remove automatic DB location as db_name.db in db location: #153
• Add support for Hive table backup in DISTCP mode: #152

New

• Beta Flag to be used for future beta features.