Product import bugs

[raukaute]

Bug report

1. The csv parser is unable to load files larger than ~15rows
2. Prices result in wrong money amount

Describe the bug

I noticed the following bugs regarding the product import via CSV:

1. The first step when the CSV file is initially parsed more often than not results in an error. One the them being an apparently missing value for product handle. It breaks with the error message The csv file parsing failed due to: No value found for target column "Product Handle" in line 19 of the given csv file, although all cells in the product handle column do have a value. This is even the case when I export the products and, without touching the file, try to import it again. However, when I break up the csv in files with fewer rows (~15) the problem is not present anymore.
   In the same line though, I do also get this error: The csv file parsing failed due to: Unable to treat column __parsed_extra from the csv file. No target column found in the provided schema. Also in this case, breaking up the csv into smaller chunks resolves the problem.

2. Prices are not uploaded correctly. For example, a Price EUR of 3361 (33,61 EUR) will result in a money amount of 336100. As with what I described above, this is also the case when exporting products with "correct" prices and re-importing them unchanged.

3. When setting product manage inventory to true, the import will not create inventory items along with the variants created which results in NaN in stock for 1 variant(s) on the product list in the admin panel.
   --

System information

Medusa version (including plugins): 1.11.0
Node.js version: 1.16.x
Database: Postgres
Operating system: MacOS

Steps to reproduce the behavior

1. Export products / or use any existing CSV
2. Choose file in the import products modal
3. Wait for parser
4. See error

Expected behavior

1. Correct product csv data should cause any errors regardless of the number of rows.
2. Prices should result in correct money_amount
3. Product manage inventory should result in a inventory item for a given variant.

Screenshots

Upload errors:

Prices:

[fPolic]

hey @raukaute, thanks for sharing.

We will investigate the parser issue. Regarding your second issue - prices are imported/exported in a "human-readable" format and not the format in which we store values in the database. So in your case, we expect 33.61 EUR in the import CSV which would be converted and stored as 3361 on import.

[raukaute]

Hey @fPolic

Thanks for your reply. I will take another look into the price imports. Meanwhile, I found another strange thing:

The csv:

The error:

Kind of strange behaviour, I have to say. What's puzzling me is that nobody else seems to be experiencing these problems? Or am I the only importing products? :D

[fPolic]

hey @raukaute, do you have a product with that id already in the database?

Also, which application do you use to edit CSVs? It seems that the product id is not parsed correctly so could you check the formatting or hidden characters that might be added?

[raukaute]

Hey @fPolic ,

after spending another couple of hours on the issue, I am kind of convinced that the issue is on medusa's side. I tested some further csv files that have only values for product id and product handles and it reliably breaks if there is more than 16 lines. This is regardless of the data in those two cells. I tried with google sheets, numbers, some other weird csv tool, used csv validators to validate the files - all nothing.

Then I went streaming the files on my own into papaparse and it nicely (and correctly) parses everything, no matter the number of rows. Given the steady number of rows threshold when it breaks makes me suspect there is something wrong in how the stream is handled. As a matter of fact, logging what is actually parsed looks actually broken. You can also see that in the last example I have given, where the product_id from the csv is 'prod_xxxx' and the error is reporting missing product with id 'od_xxxx'.

I am still wondering how apparently I can be the only person experiencing that issue.

I am using spaces plugin, in case this might be relevant.

[raukaute]

@fPolic

My assumption is correct. This is not possible

Correct me if I'm wrong. In a strategy override I tested the parsing with my own implementation of piping this into papa and everything works.

I would like to create a PR for this (could be my first contribution).

But again, I would really be happy if someone could explain how this has not been an issue in the past 11 month since this piece of code is in there..... I really nobody using this feature?

[fPolic]

@raukaute - thanks for taking the time to dive into this.

I managed to reproduce your issues on my end. It seems that the issue is related to a recent minor release of Papaprse library (mholt/PapaParse#998). When I downgraded the library to "5.3.2", importing a larger CSV worked as expected. So temporary fix will be for us to downgrade the library version to "5.3.2" since this fixes other issues that have been reported as well.
We are happy to accept your PR as well after review.

[raukaute]

@fPolic

Wow - thats crazy! I guess my PR is redundant in this case. Just tested it aswell. Thanks for also getting into this.

Cheers!

[goodpixels]

FYI this is still a problem in the latest version of Medusa (1.12.2), I am unable to export or import any CSV files, even with one product.

[victorraguin]

This is still a problem =/

[kb-0912]

Still problem. I imported 10 files and it worked 8/10 files. 2 last one just got the same error with OP

[NicolasGorga]

I am experiencing a different issue, wondering if it happens for anybody else. The batch-job gets created correctly and i started debugging, and it creates the product, but when it tries to create the product variant it fails, with message:

warn: An error occurred while processing batch.confirmed: Error: Error while processing row with
[(product id: null),
(product handle: coffee-mug-v2),
(variant id: null),
(variant sku: null)]:
Product with handle: coffee-mug-v2 was not found

While debugging, i see the newly created id from within the transaction but my DB doesn't have the product entry, which aligns with the message This makes me wonder if there is a problem with the isolation level configured for the transaction.

Although if that is the case, the feature would not be working for anyone, or is there nobody using the import product feature?