Trying to use deck.gl with Vite does not work (promisify error)

[fdb]

Describe the bug

When trying to use deck.gl, running npm run build fails with an error about promisify.

Reproduction

Here's a demonstration repo: https://github.com/fdb/vite-bug-deck-gl.
I've also made a demonstration repo with webpack, which does not give an error: https://github.com/fdb/webpack-deck-gl

1. Make a new Vite project with npm init @vitejs/app and choose Vanilla. cd into the folder and install dependencies.
2. Install deck.gl with npm install @deck.gl/core @deck.gl/layers
3. Replace the code in main.js with the basic "hello world" example
4. The dev server (npm run dev) works well.
5. Try to build it with npm run build

This gives the following error:

vite v2.0.4 building for production...
✓ 515 modules transformed.
'promisify' is not exported by __vite-browser-external, imported by node_modules/@loaders.gl/core/dist/esm/node/write-file.node.js
file: /Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/@loaders.gl/core/dist/esm/node/write-file.node.js:2:9
1: import fs from 'fs';
2: import { promisify } from 'util';
            ^
3: import { toBuffer } from '@loaders.gl/loader-utils';
4: export function writeFile(filePath, arrayBufferOrString, options) {
error during build:
Error: 'promisify' is not exported by __vite-browser-external, imported by node_modules/@loaders.gl/core/dist/esm/node/write-file.node.js
    at error (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:5279:30)
    at Module.error (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:9996:16)
    at Module.traceVariable (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:10389:29)
    at ModuleScope.findVariable (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:8847:39)
    at FunctionScope.findVariable (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:2641:38)
    at ChildScope.findVariable (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:2641:38)
    at Identifier.bind (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:4017:40)
    at CallExpression.bind (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:2728:23)
    at CallExpression.bind (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:6730:15)
    at CallExpression.bind (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:2728:23)

System Info

• Vite version 2.0.4
• Operating System: macOS Big Sur
• Node version: v14.10.0
• Package manager (npm/yarn/pnpm) and version: npm 7.6.0

Logs (Optional if provided reproduction)

  vite:config no config file found. +0ms
  vite:config using resolved config: {
  vite:config   root: '/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl',
  vite:config   base: '/',
  vite:config   mode: 'production',
  vite:config   configFile: undefined,
  vite:config   logLevel: undefined,
  vite:config   clearScreen: undefined,
  vite:config   build: {
  vite:config     target: [ 'es2019', 'edge16', 'firefox60', 'chrome61', 'safari11' ],
  vite:config     polyfillDynamicImport: true,
  vite:config     outDir: 'dist',
  vite:config     assetsDir: 'assets',
  vite:config     assetsInlineLimit: 4096,
  vite:config     cssCodeSplit: true,
  vite:config     sourcemap: false,
  vite:config     rollupOptions: {},
  vite:config     commonjsOptions: { include: [Array], extensions: [Array] },
  vite:config     minify: 'terser',
  vite:config     terserOptions: {},
  vite:config     cleanCssOptions: {},
  vite:config     write: true,
  vite:config     emptyOutDir: null,
  vite:config     manifest: false,
  vite:config     lib: false,
  vite:config     ssr: false,
  vite:config     ssrManifest: false,
  vite:config     brotliSize: true,
  vite:config     chunkSizeWarningLimit: 500
  vite:config   },
  vite:config   inlineConfig: {
  vite:config     root: undefined,
  vite:config     base: undefined,
  vite:config     mode: undefined,
  vite:config     configFile: undefined,
  vite:config     logLevel: undefined,
  vite:config     clearScreen: undefined,
  vite:config     build: {}
  vite:config   },
  vite:config   resolve: { dedupe: undefined, alias: [ [Object] ] },
  vite:config   publicDir: '/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/public',
  vite:config   command: 'build',
  vite:config   isProduction: true,
  vite:config   optimizeCacheDir: '/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/.vite',
  vite:config   plugins: [
  vite:config     'alias',
  vite:config     'vite:dynamic-import-polyfill',
  vite:config     'vite:resolve',
  vite:config     'vite:html',
  vite:config     'vite:css',
  vite:config     'vite:esbuild',
  vite:config     'vite:json',
  vite:config     'vite:wasm',
  vite:config     'vite:worker',
  vite:config     'vite:asset',
  vite:config     'vite:define',
  vite:config     'vite:css-post',
  vite:config     'vite:build-html',
  vite:config     'commonjs',
  vite:config     'vite:data-uri',
  vite:config     'rollup-plugin-dynamic-import-variables',
  vite:config     'vite:import-analysis',
  vite:config     'vite:esbuild-transpile',
  vite:config     'vite:terser',
  vite:config     'vite:reporter'
  vite:config   ],
  vite:config   server: {},
  vite:config   env: { BASE_URL: '/', MODE: 'production', DEV: false, PROD: true },
  vite:config   assetsInclude: [Function: assetsInclude],
  vite:config   logger: {
  vite:config     hasWarned: false,
  vite:config     info: [Function: info],
  vite:config     warn: [Function: warn],
  vite:config     error: [Function: error],
  vite:config     clearScreen: [Function: clearScreen]
  vite:config   },
  vite:config   createResolver: [Function: createResolver]
  vite:config } +5ms
vite v2.0.4 building for production...
✓ 515 modules transformed.
'promisify' is not exported by __vite-browser-external, imported by node_modules/@loaders.gl/core/dist/esm/node/write-file.node.js
file: /Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/@loaders.gl/core/dist/esm/node/write-file.node.js:2:9
1: import fs from 'fs';
2: import { promisify } from 'util';
            ^
3: import { toBuffer } from '@loaders.gl/loader-utils';
4: export function writeFile(filePath, arrayBufferOrString, options) {
error during build:
Error: 'promisify' is not exported by __vite-browser-external, imported by node_modules/@loaders.gl/core/dist/esm/node/write-file.node.js
    at error (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:5279:30)
    at Module.error (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:9996:16)
    at Module.traceVariable (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:10389:29)
    at ModuleScope.findVariable (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:8847:39)
    at FunctionScope.findVariable (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:2641:38)
    at ChildScope.findVariable (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:2641:38)
    at Identifier.bind (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:4017:40)
    at CallExpression.bind (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:2728:23)
    at CallExpression.bind (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:6730:15)
    at CallExpression.bind (/Users/fdb/Desktop/vite-test-deck-gl/vite-test-deck-gl/node_modules/rollup/dist/shared/rollup.js:2728:23)

[nrayburn-tech]

Pretty sure the library is trying to use nodejs code, which Vite doesn't polyfill.
#2172

[yyx990803]

Expected behavior - the package is not written for the browser and Vite doesn't polyfill Node built-ins.

[yongjun21]

Encountering similar issue when I try to migrate a project I'm working on from Vue CLI to Vite. Realized is because some of Deck.gl's dependencies are coded to be isomorphic.

See this line:
https://github.com/visgl/loaders.gl/blob/master/modules/core/package.json#L31

The browser field in package.json is meant to inform Webpack which modules are "node only" and will never be touched during runtime on browsers.

My understanding is Webpack will replace these modules with an empty object (https://github.com/defunctzombie/package-browser-field-spec#ignore-a-module).

But this doesn't seem to be handled very well at the moment by Vite.

I have tried different combinations of optimizeDeps.exclude and optimizeDeps.include. But getting nowhere. Hope someone with more knowledge of Vite inner working can provide some advice.

@yyx990803 Can we keep this issue open or at least reopen it under a better title? As I believe the issue that is being surfaced out (Vite's handling of package.json's browser field) is worth digging into further. And Deck.gl is indeed a project meant to be run in the browser.

[yongjun21]

This problem affects another Uber project XVIZ which also has loaders.gl as dependency.

[billyc]

@yongjun21 -- I am also experiencing this bug/problem, but I have found a workaround. The problem is exactly as you describe: those objects which are present in Node are causing problems for loaders.gl, and thus deck.gl doesn't load.

I'm no expert on this but I found that if you null out a few objects in vite.config.ts, you can convince loaders.gl into thinking that it's in a browser, and then (at least for me) deck.gl loads properly. Maybe this will work for you as well. If not, try looking at the error messages in your build output and null those, too? Just a guess.

vite.config.ts

// ...
export default defineConfig({
  // plugins: [...]
  define: {
    requireFromFile: null,
    'process.platform': null,
    'process.version': null
  }
})

I still get about a dozen error messages during the vite build step, but the build completes and my site seems to be loading OK.

I'm pasting the errors below so google can point others this this page.

Error messages during build:

'null' is not exported by '__vite-browser-external'
'requireFromFile' is not exported by '__vite-browser-external'
'requireFromString' is not exported by '__vite-browser-external'
'requireFromString' is not exported by '__vite-browser-external'
'toBuffer' is not exported by '__vite-browser-external'
'toBuffer' is not exported by '__vite-browser-external'
'toArrayBuffer' is not exported by '__vite-browser-external'
'toArrayBuffer' is not exported by '__vite-browser-external'
'toArrayBuffer' is not exported by '__vite-browser-external'
'toArrayBuffer' is not exported by '__vite-browser-external'
'readFileSync' is not exported by '__vite-browser-external'
'readFileSync' is not exported by '__vite-browser-external'
'writeFile' is not exported by '__vite-browser-external'
'writeFile' is not exported by '__vite-browser-external'
'writeFileSync' is not exported by '__vite-browser-external'
'writeFileSync' is not exported by '__vite-browser-external'

I hope this helps.