Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQL error saving Element (variant) - Integrity constraint violation #4370

Closed
ghost opened this issue Jun 6, 2019 · 6 comments
Closed

SQL error saving Element (variant) - Integrity constraint violation #4370

ghost opened this issue Jun 6, 2019 · 6 comments

Comments

@ghost
Copy link

ghost commented Jun 6, 2019

Description

I was asked to create a new issue for the problem I'm having so here it is.

I have created a custom plugin to import taxCategories and two types of products into our Commerce. It also sends orders to the external CRM and manages a bunch of other things on our website. The import worked perfectly fine until we added a new site (first we had only one) and cleared all products. Our products do not need a multi-site setup so they are all saved into the default site.

The setup is pretty complex but this is pretty much how it works:
First a job is pushed which retrieves all api data from multiple endpoints with paging. After that it pushes the update job with all of the data. The update job first calls the create() function of all registered importers. It then calls all the update() functions and after that calls the delete() functions.

I'm saving the element (variant) from my create function. The variant is then saved to my custom cache array. Its later checked for updates by comparing the api data to the version in the database. If any changes we're made the product and/or variant are saved. I have product and variant save/delete listeners that update the cached objects whenever those events are called.

I don't use any custom DB commands. I mostly use the Elements service to get and save data. I am now getting a SQL error when saving variants using the Elements service.
https://pastebin.com/56Qxvqqs

This code causes the error (saveContent() line 190 inside craft/services/Content)

Craft::$app->getDb()->createCommand()
    ->update($this->contentTable, $values, ['id' => $element->contentId])
    ->execute();

This update is somehow causing a duplicate key. Its an update which means the contentId exists and its using an existing record.

I have been debugging for quite some time and I don't fully understand why it is saving duplicate content records. I hope you know the answer to my problem. Thanks in advance!

Steps to reproduce

See info above.

Additional info

  • Craft version: 3.1.28
  • PHP version: 7.1.6
  • Database driver & version: MySQL 5.5.59
  • Plugins & versions: Custom plugin
@ghost
Copy link
Author

ghost commented Jun 6, 2019

Extra info:

This is how I save the product (which also saves the new variant):

$success = Craft::$app->getElements()->saveElement($product, true, false);

@ghost
Copy link
Author

ghost commented Jun 6, 2019

Update:

I have just changed some of the Craft Commerce code which seems to save fixed my issue:
craft\commerce\elements\Product -> afterSave() line 710

Craft::$app->getElements()->saveElement($variant, false);
to
Craft::$app->getElements()->saveElement($variant, false, false);

I don't want to propagate my products. Is there any way for me to disable propagation? I know this has now become a Commerce related issue.

@brandonkelly
Copy link
Member

I only see one saveElement() call in your stack trace, and it’s for a variant, not a product, and coming directly from /ImportUpdateTask.php(87). Can you show the code you’ve got around there?

@ghost
Copy link
Author

ghost commented Jun 6, 2019

I disabled all propagation by passing false when saving my products and variants. Forgot to update my stacktrace, sorry!

Here it is:

Next yii\db\IntegrityException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '8604-1' for key 'content_elementId_siteId_unq_idx'
The SQL being executed was: UPDATE `content` SET `elementId`=8604, `siteId`=1, `title`='My title', `dateUpdated`='2019-06-06 12:41:55' WHERE `id`='10133' in /Applications/MAMP/htdocs/privateName/vendor/yiisoft/yii2/db/Schema.php:664
Stack trace:
#0 /Applications/MAMP/htdocs/privateName/vendor/yiisoft/yii2/db/Command.php(1295): yii\db\Schema->convertException(Object(PDOException), 'UPDATE `content...')
#1 /Applications/MAMP/htdocs/privateName/vendor/yiisoft/yii2/db/Command.php(1091): yii\db\Command->internalExecute('UPDATE `content...')
#2 /Applications/MAMP/htdocs/privateName/vendor/craftcms/cms/src/services/Content.php(192): yii\db\Command->execute()
#3 /Applications/MAMP/htdocs/privateName/vendor/craftcms/cms/src/services/Elements.php(512): craft\services\Content->saveContent(Object(craft\commerce\elements\Variant))
#4 /Applications/MAMP/htdocs/privateName/vendor/craftcms/commerce/src/elements/Product.php(710): craft\services\Elements->saveElement(Object(craft\commerce\elements\Variant), false)
#5 /Applications/MAMP/htdocs/privateName/vendor/craftcms/cms/src/services/Elements.php(516): craft\commerce\elements\Product->afterSave(false)
#6 /Applications/MAMP/htdocs/privateName/plugins/privateNamecore/src/imports/ProductImporter.php(110): craft\services\Elements->saveElement(Object(craft\commerce\elements\Product), true, false)
#7 /Applications/MAMP/htdocs/privateName/plugins/privateNamecore/src/jobs/ImportUpdateTask.php(62): pixeldeluxe\privateNamecore\imports\ProductImporter->create()
#8 /Applications/MAMP/htdocs/privateName/vendor/yiisoft/yii2-queue/src/Queue.php(214): pixeldeluxe\privateNamecore\jobs\ImportUpdateTask->execute(Object(craft\queue\Queue))
#9 /Applications/MAMP/htdocs/privateName/vendor/yiisoft/yii2-queue/src/cli/Queue.php(147): yii\queue\Queue->handleMessage('19', 'O:46:"pixeldelu...', '300', 1)
#10 /Applications/MAMP/htdocs/privateName/vendor/craftcms/cms/src/queue/Queue.php(96): yii\queue\cli\Queue->handleMessage('19', 'O:46:"pixeldelu...', '300', 1)
#11 /Applications/MAMP/htdocs/privateName/vendor/craftcms/cms/src/controllers/QueueController.php(86): craft\queue\Queue->run()
#12 [internal function]: craft\controllers\QueueController->actionRun()
#13 /Applications/MAMP/htdocs/privateName/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array(Array, Array)
#14 /Applications/MAMP/htdocs/privateName/vendor/yiisoft/yii2/base/Controller.php(157): yii\base\InlineAction->runWithParams(Array)
#15 /Applications/MAMP/htdocs/privateName/vendor/craftcms/cms/src/web/Controller.php(109): yii\base\Controller->runAction('run', Array)
#16 /Applications/MAMP/htdocs/privateName/vendor/yiisoft/yii2/base/Module.php(528): craft\web\Controller->runAction('run', Array)
#17 /Applications/MAMP/htdocs/privateName/vendor/craftcms/cms/src/web/Application.php(297): yii\base\Module->runAction('queue/run', Array)
#18 /Applications/MAMP/htdocs/privateName/vendor/craftcms/cms/src/web/Application.php(565): craft\web\Application->runAction('queue/run', Array)
#19 /Applications/MAMP/htdocs/privateName/vendor/craftcms/cms/src/web/Application.php(281): craft\web\Application->_processActionRequest(Object(craft\web\Request))
#20 /Applications/MAMP/htdocs/privateName/vendor/yiisoft/yii2/base/Application.php(386): craft\web\Application->handleRequest(Object(craft\web\Request))
#21 /Applications/MAMP/htdocs/privateName/public_html/index.php(21): yii\base\Application->run()
#22 {main}
Additional Information:
Array
(
    [0] => 23000
    [1] => 1062
    [2] => Duplicate entry '8604-1' for key 'content_elementId_siteId_unq_idx'
)

@ghost
Copy link
Author

ghost commented Jun 6, 2019

My create function:

public function create() : bool {
    $defaultTaxCategory = CraftCommerce::getInstance()->taxCategories->getDefaultTaxCategory();

    foreach($this->data as $key => $row) {
        $variant = PrivateNameCore::getInstance()->apiDataImport->findIdLink('variant', $row->Id);

        PrivateNameCore::getInstance()->apiLogger->log(LogType::DEBUG, "Checking Variant create ", "ImportId: " . $this->import->id, "Id: " . $row->Id, $variant);

        // Ignore if variant already exists
        if($variant) continue;

        // Find the taxcategory and validate it before we do anything
        $taxCategory = PrivateNameCore::getInstance()->apiDataImport->getTaxCategoryByHandle($row->{self::FIELD_TAX});
        if(!$taxCategory) {
            PrivateNameCore::getInstance()->apiLogger->log(LogType::WARNING, "Tried to create Variant with non-existing taxcategory, finding replacement...", "ImportId: " . $this->import->id, $row);
            $taxCategory = $defaultTaxCategory;
        }

        // Unable to find taxcategory or replacement category
        if(!$taxCategory) {
            PrivateNameCore::getInstance()->apiLogger->log(LogType::ERROR, "Variant had a non-existing taxCategory and no replacement could be found", "ImportId: " . $this->import->id);
            return false;
        }

        // Create variant
        $variant = new Variant();
        $variant->title = $row->{self::FIELD_TITLE};
        $variant->enabled = true;
        $variant->sku = ProductHelper::createSku($row->{self::FIELD_SKU});
        $variant->price = LocalizationHelper::normalizeNumber($row->{self::FIELD_PRICE});
        $variant->stock = $row->{self::FIELD_STOCK};
        $variant->hasUnlimitedStock = false;

        // Find existing product
        $product = null;
        $newProduct = false;

        // If its a bundled product find the bundle
        $bundled = !empty($row->{self::FIELD_BUNDLE_CODE});
        if($bundled) {
            $product = PrivateNameCore::getInstance()->apiDataImport->findIdLink('product', $row->{self::FIELD_BUNDLE_CODE});
        }

        PrivateNameCore::getInstance()->apiLogger->log(LogType::DEBUG, "Processing variant create ", "ImportId: " . $this->import->id, "Bundled: " . ($bundled ? 'yes' : 'no'), $product);

        // Create product if it doesn't exist
        if(!$product) {
            $product = new Product();

            // Set default values, return false if it fails to do so
            if(!$product->setDefaults()) {
                PrivateNameCore::getInstance()->apiLogger->log(LogType::ERROR, "Error while setting product defaults", "ImportId: " . $this->import->id, $product);
                return false;
            }

            $product->typeId = 3;
            $product->taxCategoryId = $taxCategory->id;

            $newProduct = true;
        }

        // Get variants without invalid variants like the default placeholder
        $variants = $product->getValidVariants();

        // Make this the default variant if not a bundle or the SKU matches the bundlecode
        $variant->isDefault = count($variants) <= 1 || ProductHelper::createSku($row->{self::FIELD_SKU}) == $row->{self::FIELD_BUNDLE_CODE};

        // Add variant to product
        $variants[] = $variant;
        $product->setVariants($variants);

        // Set title of the product
        if($bundled) {
            $product->title = $row->{self::FIELD_BUNDLE_TITLE};
        } else {
            $product->title = $row->{self::FIELD_TITLE};
        }

        PrivateNameCore::getInstance()->apiLogger->log(LogType::DEBUG, "Saving product and its variants", "ImportId: " . $this->import->id, $variant);


        // Save the new Product
        $success = Craft::$app->getElements()->saveElement($product, true, false);
        if(!$success) {
            PrivateNameCore::getInstance()->apiLogger->log(LogType::ERROR, "Unable to save product", "ImportId: " . $this->import->id, $product->getErrors());
            return false;
        }

        // Create product id link if its a bundled product
        if($newProduct && $bundled && !PrivateNameCore::getInstance()->apiDataImport->createIdLink('product', $product->id, $row->{self::FIELD_BUNDLE_CODE})) {
            PrivateNameCore::getInstance()->apiLogger->log(LogType::ERROR, "Unable to create product id link", "ImportId: " . $this->import->id);
            return false;
        }

        // Create variant id link
        if(!PrivateNameCore::getInstance()->apiDataImport->createIdLink('variant', $variant->id, $row->Id)) {
            PrivateNameCore::getInstance()->apiLogger->log(LogType::ERROR, "Unable to create variant id link", "ImportId: " . $this->import->id);
            return false;
        }

        PrivateNameCore::getInstance()->apiLogger->log(LogType::DEBUG, "Created variant link with internal id " . $variant->id . " and external id " . $row->Id, "ImportId: " . $this->import->id);

        //Log creation
        $this->import->logs->logCreate('variant', $variant->id);
        $this->import->logs->logCreate('product', $product->id);
    }

    return true;
}

@brandonkelly
Copy link
Member

I’m guessing your issue is that you’re always passing $propagate = false when calling saveElement(), even if the element was brand new, which is not how it was intended to work. Next Craft release is going to start ignoring that, and force-propagating new elements.

If that doesn’t work we can look further into this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant