url_rewrite duplicates

[redboxmarcins]

While migrating from CE 1.7.0.2 to EE 2.0.4 url rewrites are migrated from core_url_rewrite to url_rewrite table, but EE 2.0.4 require (same as 1.13EE and 1.8CE) that url keys must be unique
so saving category or product is not possible after migration

So far using https://gist.github.com/Vinai/5451584 to clear data before doing migration, but I believe that should be handled by migration tool

[ilol]

Thank you, @redboxmarcins, will discuss this with development team @yaronish @victor-v-rad

[victor-v-rad]

url keys must be unique

Yes, but request_path + store_id must be unique in Magento 2.

so saving category or product is not possible after migration

I have not encountered such issue after migration so far. But I will investigate this case more thoroughly.

[redboxmarcins]

@victor-v-rad did you try to migrate data where url keys are not unique - f.e. from version before 1.8.x ?

[ilol]

MAGETWO-52505

[csdougliss]

@ilol Any updates on this issue? This is the last issue preventing me from handing over a decent M1 database -> M2

[csdougliss]

Even with running: https://gist.github.com/Vinai/5451584 I still get:

[PDOException]
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'upright-vacuum-cleaners/swift-upright-22' for key 'URL_REWRITE_REQUEST_PATH_STORE_ID'

[victor-v-rad]

@redboxmarcins
The issue was fixed in the latest release of the tool 2.1.1
Please recheck

[victor-v-rad]

I am closing the issue. Feel free to reopen if it is still actual

[csdougliss]

@victor-v-rad I just tested this with 2.1.1 and I still get it

[2016-09-29 14:59:24][INFO][mode: data][stage: data migration][step: Url Rewrite Step]: Duplicate resolved. Request path was: spares-and-solutions/bottom-hose-guide-1-3-131793-00 Target path was: catalog/product/view/id/2191 Store ID: 3 New request path: spares-and-solutions/bottom-hose-guide-1-3-131793-00


  [PDOException]                                                                                                                                                
  SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'upright-vacuum-cleaners/swift-upright-22' for key 'URL_REWRITE_REQUEST_PATH_STORE_ID'  



migrate:data [-r|--reset] config

``

[victor-v-rad]

The easiest way might be to send us the DB dump (cleaned from passwords). Here https://github.com/victor-v-rad you can find my email.

[victor-v-rad]

internal ticket MAGETWO-59233 to fix the issue

[alessandrozucca]

I'm having similar issues, i'll send you a private email.

[victor-v-rad]

@craigcarnell @alessandrozucca
Guys try out the solution from the commit

[alessandrozucca]

I've found a workaround for this. I'm removing the index URL_REWRITE_REQUEST_PATH_STORE_ID before I run the migration. If you do this all the values get added.

Then, you need to check the duplicates, I've run a query grouping by request_path and store_id where the count(*) was > 1. In my case I've found around 56 duplicates.
Some of the duplicates where NULL request_paths, some where request_path with a length > 255 chars which got truncated as they were too long, and one was a short request_path (10 char) that for some reason was a duplicate and the migration tool failed to resolve.

In my situation I could just remove all duplicates as all this redirect_url entries were belonging to store that the customer doesn't need, then I have re-added the URL_REWRITE_REQUEST_PATH_STORE_ID index and all sorted!

[csdougliss]

@victor-v-rad I tried out your changes and got much, much further but it stopped here:

[2016-10-10 09:22:17][INFO][mode: data][stage: data migration][step: Url Rewrite Step]: started
98% [===========================>] Remaining Time: 1 sec  


  [PDOException]                                                                                                                        
  SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 's2s-1-steamer -3' for key 'URL_REWRITE_REQUEST_PATH_STORE_ID' 

[csdougliss]

@victor-v-rad any update on this?

[victor-v-rad]

I have migrated magento_prod_2016-12-02.sql you send me and not found such error "Integrity constraint violation". With auto_resolve_urlrewrite_duplicates=1 migration completed without errors