EAV - Incorrect extended attribute migration with ignoring

[ihor-sviziev]

Pre-Conditions:

1. Source Magento 1.9.2.4 + mailchimp module (https://github.com/ebizmarts/magemonkey)
2. Destination Magento 2.1.6 + data migration tool v2.1.6
3. Added to ignore list in eav-attribute-groups.xml file few attributes from catalog and customer:
   

Actual result:

                                                                                      
  [PDOException]                                                                                 
  SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '245' for key 'PRIMARY'  
                                                                                                 


Exception trace:
 () at /magento2/vendor/magento/data-migration-tool/src/Migration/ResourceModel/Adapter/Mysql.php:169
 PDOStatement->execute() at /magento2/vendor/magento/data-migration-tool/src/Migration/ResourceModel/Adapter/Mysql.php:169
 Migration\ResourceModel\Adapter\Mysql->insertMultiple() at /magento2/vendor/magento/data-migration-tool/src/Migration/ResourceModel/Adapter/Mysql.php:137
 Migration\ResourceModel\Adapter\Mysql->insertRecords() at /magento2/vendor/magento/data-migration-tool/src/Migration/ResourceModel/Destination.php:52
 Migration\ResourceModel\Destination->saveRecords() at /magento2/vendor/magento/data-migration-tool/src/Migration/Step/Eav/Data.php:823
 Migration\Step\Eav\Data->saveRecords() at /magento2/vendor/magento/data-migration-tool/src/Migration/Step/Eav/Data.php:812
 Migration\Step\Eav\Data->migrateAttributesExtended() at /magento2/vendor/magento/data-migration-tool/src/Migration/Step/Eav/Data.php:184
 Migration\Step\Eav\Data->perform() at /magento2/vendor/magento/data-migration-tool/src/Migration/Mode/AbstractMode.php:73
 Migration\Mode\AbstractMode->runStage() at /magento2/vendor/magento/data-migration-tool/src/Migration/Mode/Data.php:116
 Migration\Mode\Data->runData() at /magento2/vendor/magento/data-migration-tool/src/Migration/Mode/Data.php:68
 Migration\Mode\Data->run() at /magento2/vendor/magento/data-migration-tool/src/Migration/Console/MigrateDataCommand.php:51
 Migration\Console\MigrateDataCommand->execute() at /magento2/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:257
 Symfony\Component\Console\Command\Command->run() at /magento2/vendor/symfony/console/Symfony/Component/Console/Application.php:874
 Symfony\Component\Console\Application->doRunCommand() at /magento2/vendor/symfony/console/Symfony/Component/Console/Application.php:195
 Symfony\Component\Console\Application->doRun() at /magento2/vendor/magento/framework/Console/Cli.php:96
 Magento\Framework\Console\Cli->doRun() at /magento2/vendor/symfony/console/Symfony/Component/Console/Application.php:126
 Symfony\Component\Console\Application->run() at /magento2/bin/magento:23

Expected result:
Import should ignore attributes that are in ignore list.

Reason:

• during inserting data to catalog_eav_attribute table it copy all data from source database (including ignored attributes),
• during inserting values that we had in destination magento and it fails because we already have value from step 1

Workaround:
Do not ignore these attributes and remove them after migration

[victor-v-rad]

Thank you @ihor-sviziev for reporting issue with good details. From the first glance it could actually be real. I will check it out.

[victor-v-rad]

@ihor-sviziev
There is a nice "Data Integrity Step" which stands the first among others in config.xml. Did it throw any errors about orphan records in your catalog_eav_attribute table? It could be a key factor of the issue you posted.

[ihor-sviziev]

@victor-v-rad no, it just fails on data migration step. I checked, during "attribute migration" - attributes ignored successfully, but during extended attribute migration - it doesn't

[victor-v-rad]

Got it. Internal ticket MAGETWO-68777 to fix the issue.

[tramov]

The screenshot at the top (list of mailchimp attributes to ignore) also solved a large headache for me in that it showed how me to easily remove the MailChimp module for Magento 1 left overs out of my migrated Magento 2 database. These fields literally crippled the migrated system with endless exceptions.