Import products job is stuck at pre-processing

[viicslen]

Bug report

Describe the bug

Import products list job is stuck in pre-processing. When a file is selected on the import product list modal, the file is uploaded and shows preprocessing but never gets processed. The UI doesn't show any signs of exceptions or errors but an error is thrown in the console window in which the medusa develop command is running.

Im using the MinIO file plugin.

info:    Processing batch.created which has 1 subscribers
NoSuchKey: The specified key does not exist.
    at Request.extractError (backend/node_modules/aws-sdk/lib/services/s3.js:711:35)
    at Request.callListeners (backend/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (backend/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (backend/node_modules/aws-sdk/lib/request.js:686:14)
    at Request.transition (backend/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (backend/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at backend/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (backend/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (backend/node_modules/aws-sdk/lib/request.js:688:12)
    at Request.callListeners (backend/node_modules/aws-sdk/lib/sequential_executor.js:116:18) {
  code: 'NoSuchKey',
  region: null,
  time: 2023-05-05T14:17:14.158Z,
  requestId: '175C4501A87F0AF0',
  extendedRequestId: 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
  cfId: undefined,
  statusCode: 404,
  retryable: false,
  retryDelay: 46.983953045236504
}

System information

Medusa version: v1.10.0
Medusa MinIO plugin version: v1.2.0
Node.js version: v19.9.0
Database: PostgreSQL
Operating system: GNU/Linux

Steps to reproduce the behavior

1. Go to the Admin dashboard
2. Click on "Products" in the sidemenu
3. Click on "Import Products" in the top right corner of the products section
4. Browse or drop file to import in the shown modal

Expected behavior

The file should finish preprocessing and the import process should start. After that the imported products should show on the product list.

Screenshots

MinIO file key

Batch Job context in database

Additional context

Plugin Configuration

const plugins = [
	// ...

	{
        resolve: `medusa-file-minio`,
        options: {
            endpoint: process.env.MINIO_ENDPOINT,
            bucket: process.env.MINIO_BUCKET,
            access_key_id: process.env.MINIO_ACCESS_KEY,
            secret_access_key: process.env.MINIO_SECRET_KEY,
            private_bucket: process.env.MINIO_PRIVATE_BUCKET,
        },
    },

	// ...
];

# MinIO
MINIO_ENDPOINT=http://127.0.0.1:9001
MINIO_BUCKET=store-public
MINIO_ACCESS_KEY=******
MINIO_SECRET_KEY=**********
MINIO_PRIVATE_BUCKET=store-private

API responses

POST `/admin/uploads/protected`

{
  "uploads": [
    {
      "url": "http://127.0.0.1:9001/store-private/catalog-1683642074137.csv",
      "key": "catalog-1683642074137.csv"
    }
  ]
}

POST `/admin/batch-jobs`

{
  "batch_job": {
    "dry_run": true,
    "type": "product-import",
    "created_by": "usr_01GYWBJPCNHQQB0DD7BDJH408S",
    "context": {
      "fileKey": "catalog-1683642074137.csv"
    },
    "id": "batch_01H00E6H1N8RT3TDZB61MG99Z5",
    "deleted_at": null,
    "result": null,
    "pre_processed_at": null,
    "processing_at": null,
    "confirmed_at": null,
    "completed_at": null,
    "canceled_at": null,
    "failed_at": null,
    "created_at": "2023-05-09T14:21:14.164Z",
    "updated_at": "2023-05-09T14:21:14.164Z",
    "status": "created"
  }
}

GET `admin/batch-jobs/batch_01H00E6H1N8RT3TDZB61MG99Z5`

{
    "batch_job": {
        "dry_run": true,
        "id": "batch_01H00E6H1N8RT3TDZB61MG99Z5",
        "created_at": "2023-05-09T14:21:14.164Z",
        "updated_at": "2023-05-09T14:22:16.081Z",
        "deleted_at": null,
        "type": "product-import",
        "created_by": "usr_01GYWBJPCNHQQB0DD7BDJH408S",
        "context": {
            "fileKey": "catalog-1683642074137.csv"
        },
        "result": null,
        "pre_processed_at": null,
        "processing_at": null,
        "confirmed_at": null,
        "completed_at": null,
        "canceled_at": "2023-05-09T14:22:16.083Z",
        "failed_at": null,
        "status": "canceled"
    }
}

[omarghader]

I have the same issue.
I set 2 buckets: public bucket and private bucket

Export writes the file to the public bucket
import returns error: NoSuchKey: The specified key does not exist.

Any help is appreciated, thanks

[viicslen]

This bug still exists in the latest version v1.11.0 released

[alessioacella]

I have the same issue with public and private buckets

[zpuckeridge]

{"level":"info","message":"Processing batch.created which has 1 subscribers","timestamp":"2023-08-28 04:21:49"}
NoSuchKey: The specified key does not exist.
at Request.extractError (/app/node_modules/aws-sdk/lib/services/s3.js:711:35)
at Request.callListeners (/app/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
at Request.emit (/app/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
at Request.emit (/app/node_modules/aws-sdk/lib/request.js:686:14)
at Request.transition (/app/node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (/app/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /app/node_modules/aws-sdk/lib/state_machine.js:26:10
at Request.<anonymous> (/app/node_modules/aws-sdk/lib/request.js:38:9)
at Request.<anonymous> (/app/node_modules/aws-sdk/lib/request.js:688:12)
at Request.callListeners (/app/node_modules/aws-sdk/lib/sequential_executor.js:116:18) {
code: 'NoSuchKey',
region: null,
time: 2023-08-28T04:21:49.482Z,
requestId: '177F713FC677FCF0',
extendedRequestId: 'dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8',
cfId: undefined,
statusCode: 404,
retryable: false,
retryDelay: 44.79494170616547
}

Confirming same setup, same issue. This is using the CSV import example in the docs:
https://medusa-doc-files.s3.amazonaws.com/product-import-sales-channels.csv
https://docs.medusajs.com/modules/products/admin/import-products

Also to note, downloading the template in the admin dashboard returns the following inside of the CSV file:
<!doctype html><html lang="en"><head><meta charset="UTF-8"/><link rel="icon" type="image/svg+xml" href="/logo.svg"/><meta name="viewport" content="width=device-width

It does not return useful columns from importing. Exporting the products works correctly however.

[olivermrbl]

@zpuckeridge, we pushed several fixes to the Import-Export domain and file plugins last week. See most recent releases here.

Can I get you to ensure you are using the latest versions?

[zpuckeridge]

G'day @olivermrbl - I missed this! Oops! I can confirm, import is working for me now. Template button also works as expected. Awesome work!