TypeError: Data must be a string or a buffer #1659

cstrangedk · 2017-07-29T18:35:35Z

Gatsby v1.0.0
Node v7.5.0
MacOS v10.12.5

I produce the following error after a gatsby build:

crypto.js:74
  this._handle.update(data, encoding);
               ^

TypeError: Data must be a string or a buffer
    at Hash.update (crypto.js:74:16)
    at ~/Projects/victor/ubby-desktop/node_modules/gatsby/dist/utils/get-hash-fn.js:22:10
    at ~/Projects/victor/ubby-desktop/node_modules/gatsby/dist/utils/hashed-chunk-ids-plugin.js:20:22
    at Array.forEach (native)
    at Compilation.<anonymous> (~/Projects/victor/ubby-desktop/node_modules/gatsby/dist/utils/hashed-chunk-ids-plugin.js:18:14)
    at Compilation.applyPlugins (~/Projects/victor/ubby-desktop/node_modules/tapable/lib/Tapable.js:26:37)
    at Compilation.<anonymous> (~/Projects/victor/ubby-desktop/node_modules/webpack/lib/Compilation.js:545:8)
    at Compilation.applyPluginsAsync (~/Projects/victor/ubby-desktop/node_modules/tapable/lib/Tapable.js:60:69)
    at Compilation.seal (~/Projects/victor/ubby-desktop/node_modules/webpack/lib/Compilation.js:525:7)
    at Compiler.<anonymous> (~/Projects/victor/ubby-desktop/node_modules/webpack/lib/Compiler.js:397:15)
    at ~/Projects/victor/ubby-desktop/node_modules/tapable/lib/Tapable.js:103:11
    at Compilation.<anonymous> (~/Projects/victor/ubby-desktop/node_modules/webpack/lib/Compilation.js:445:10)
    at ~/Projects/victor/ubby-desktop/node_modules/webpack/lib/Compilation.js:417:12
    at ~/Projects/victor/ubby-desktop/node_modules/webpack/lib/Compilation.js:332:10
    at ~/Projects/victor/ubby-desktop/node_modules/webpack/node_modules/async/lib/async.js:52:16
    at done (~/Projects/victor/ubby-desktop/node_modules/webpack/node_modules/async/lib/async.js:246:17)
    at ~/Projects/victor/ubby-desktop/node_modules/webpack/node_modules/async/lib/async.js:44:16
    at _combinedTickCallback (internal/process/next_tick.js:67:7)
    at process._tickCallback (internal/process/next_tick.js:98:9)

My package.json is:

{
  "name": "gatsby-starter-default",
  "description": "Gatsby default starter",
  "version": "1.0.0",
  "author": "Kyle Mathews <[email protected]>",
  "dependencies": {
    "fs": "0.0.1-security",
    "gatsby": "^1.4.1",
    "gatsby-link": "^1.4.1",
    "gatsby-plugin-sharp": "^1.0.0-beta.7",
    "gatsby-remark-copy-linked-files": "^1.0.0-beta.6",
    "gatsby-remark-images": "^1.5.0",
    "gatsby-remark-prismjs": "^1.0.0-beta.6",
    "gatsby-remark-responsive-iframe": "^1.0.0-beta.6",
    "gatsby-remark-smartypants": "^1.0.0-beta.6",
    "gatsby-source-filesystem": "^1.0.0-beta.6",
    "gatsby-transformer-remark": "^1.0.0-beta.6",
    "lodash": "^4.17.4",
    "object-assign": "^4.1.1",
    "react-helmet": "^5.0.3",
    "react-materialize": "^1.0.1",
    "react-typography": "^0.15.0",
    "typography": "^0.15.8",
    "typography-breakpoint-constants": "^0.14.0",
    "typography-plugin-code": "^0.15.9",
    "underscore.string": "^3.3.4"
  },
  "devDependencies": {
    "gh-pages": "^0.12.0"
  },
  "keywords": [
    "gatsby"
  ],
  "license": "MIT",
  "main": "n/a",
  "scripts": {
    "build": "gatsby build",
    "deploy": "gatsby build --prefix-links && gh-pages -d public",
    "develop": "gatsby develop",
    "format": "prettier --trailing-comma es5 --no-semi --single-quote --write \"pages/*.js\" \"utils/*.js\" \"wrappers/*.js\" \"html.js\"",
    "test": "echo \"Error: no test specified\" && exit 1"
  }
}

The text was updated successfully, but these errors were encountered:

cr101 · 2017-07-30T05:54:23Z

@cstrangedk Off topic, you might want to use Material-UI v.1 which now provides a grid system instead of using react-materialize

0xc0dec · 2017-07-30T08:37:24Z

Same issue here. I've found that when several components from /src/pages/ import some shared component X

import X from './../shared/x';

and if the X component imports its css module

import styles from './x.module.css';

then this error occurs.

Also what's really weird is that this happens in my project only when there are more than 3 such page components (importing X). For <= 3 it builds fine.

EDIT: Looks like this is indeed a problem with style imports. When a component imports styles and is itself being imported (even indirectly) several times this error occurs. In the end I had to switch to JSS.

cstrangedk · 2017-08-01T18:29:28Z

Thanks @0xc0dec , I was able to workaround this issue and consolidate how my components import css modules.

Crashes with an error during `gatsby build`, as documented on gatsbyjs/gatsby#1659

maetl · 2017-08-06T13:21:10Z

FYI @KyleAMathews and other @gatsbyjs maintainers, I’ve run into multiple problems relating to this issue as well, which I’ve documented in a test case repo here: https://github.com/digivizer/gatsby-1659

I’ve added notes to the README about how to reproduce the bug and differences in how it manifests between Gatsby 1.2 and 1.3+.

In the test case repo, there are two separate branches with different src/component trees and package.json dependencies that illustrate what’s happening:

Interestingly, the component tree in the 1.3 branch builds successfully with the 1.2 yarn.lock. However adding several more components with imported .module.sass files causes it to break in a related way.

These problems are not visible in gatsby develop. They only manifest when trying to build for production.

It’s also not immediately obvious whether this is a problem in Gatsby itself, or whether it is something to do with the underlying libraries and tools.

bskimball · 2017-08-08T11:11:13Z

I'm also seeing this issue, and it looks to have something to do with importing css modules. It's strange because some css modules are importing fine. As an example I have a contact form which imports a button and it's relevant css file and it builds fine. I can create a new page and import that same button component and the build fails. If I don't import the css file on the newly created page, the build is successful.

bskimball · 2017-08-08T14:24:52Z

Looks like the issue is when generating the chunk, the chunk name is undefined. This causes crypto.js updateHash to fail because it expects a string or buffer. Now the question is why is the chunk name undefined

bskimball · 2017-08-09T00:13:00Z

This looks to be fixed in version 1.7.1
looks like the generation of the chunk names was altered in (#1503)[https://github.com//pull/1503].

edit: Nevermind this still is an issue. I added another page and started getting the error again.

maetl · 2017-08-09T02:29:02Z

Can confirm, 1.7.1 is still failing with the same error.

KyleAMathews · 2017-08-09T02:42:16Z

The error is in utils/get-hash-fn.js. Is there other names for chunks we could use there?

bskimball · 2017-08-09T02:48:54Z

From what I can see hashed-chunk-ids-plugin passes chunk.name to get-hash-fn. On some css modules chunk.name returns as undefined. If I omit typeof chunk.name !== undefined in the plugin it looks like latest css module is there but the previous was undefined. It look like a webpack issue, as hashed-chunk-plugins-id is expecting to get the chunk from webpack.

I literally console.log the chunks to see what it was spitting out.

also .cache/pages.json has the componentChunkName listed in the file

maetl · 2017-08-09T02:54:16Z

I would be happy to add a PR with a regression test case for this, but am unsure how to fix it without a bit more help and guidance.

KyleAMathews · 2017-08-09T02:56:31Z

We just need something to uniquely identify the chunk. On chunks without names, is there anything else that can be used to identify the chunks?

bskimball · 2017-08-09T03:03:42Z

Maybe the resource? But that might not be unique. I'm still looking through the chunks. Math.random() if chunk.name == undefined?

KyleAMathews · 2017-08-09T03:13:07Z

It can't be random as it has to ID the chunk

bskimball · 2017-08-09T03:23:16Z

I don't see anything unique. The chunk.context shows the path to the file but that's not necessarily going to be unique. Do you know why the id returns null?

bskimball · 2017-08-09T03:30:09Z

So, I changed the plugin and the html build completes. Although I know the same css was already imported earlier in the page. I don't know if it fixes anything or just prevents the hashFn failure. I'm going to keep testing.

var getHashFn = require("./get-hash-fn");

function HashedChunkIdsPlugin(options) {
  this.options = options || {};
}

HashedChunkIdsPlugin.prototype.apply = function apply(compiler) {
  var hashFn = getHashFn(this.options);
  compiler.plugin("compilation", function (compilation) {
    compilation.plugin("before-chunk-ids", function (chunks) {
      chunks.forEach(function (chunk) {
        if (chunk.id === null) {
          if (typeof chunk.name !== "undefined") {
            chunk.id = hashFn(chunk.name);
          } else {
            console.log(chunk);
          }
        }
      });
    });
  });
};

module.exports = HashedChunkIdsPlugin;

maetl · 2017-08-09T03:50:24Z

The HashedChunkIdsPlugin assumes that when chunk.id is null, chunk.name can be used. This assumption doesn’t hold in all situations.

For example, the issue documented here occurs when the following is passed through:

Chunk {
  id: null,
  ids: null,
  name: undefined
}

This may also indicate a problem with CSS Modules or Webpack (or an interaction between the two).

Was able to get the site to build just now by using the path string provided in chunk.modules[0].rawRequest:

if (chunk.id === null) {
  if (chunk.name !== undefined) {
    chunk.id = hashFn(chunk.name);
  } else {
    chunk.id = hashFn(chunk.modules[0].rawRequest)
  }
}

bskimball · 2017-08-09T03:55:59Z

rawRequest may not be unique, and I'm not sure that matters. So far, I have just been able to skip the chunks without names all together without issue. Right now, I'm assuming the reason the chunk doesn't have a name is because it was imported by a previous chunk.

maetl · 2017-08-09T04:01:21Z

Thanks for looking at that. I’ll run a build skipping all chunks with undefined names and see if it behaves correctly and that all the CSS is present when it loads in the browser.

This assumption doesn’t hold in all situations.

I don’t really know what Webpack’s standard is for error messages/internal exception handling, but at the very least, would be more robust to have a guard clause that checks that chunk.name is valid before trying to use it.

bskimball · 2017-08-09T04:04:24Z

I think they may have fixed it in v 2.3 which is also what Gatsby is going to in v2
#4072

maetl · 2017-08-09T04:05:37Z

Yeah, so skipping the chunk as described above simply moves the error from compile time to run time, as documented here.

Edit: This error happens in the same way, irrespective of whether the chunk is skipped, or whether the id gets filled in like chunk.id = hashFn(chunk.modules[0].rawRequest).

bskimball · 2017-08-09T16:16:56Z

I'm not getting that error. I tested on my environment throughout the night. I am currently running the following code so I can analyze which modules do not have chunk.id or chunk.name. I have nested components and multiple pages running.

var getHashFn = require("./get-hash-fn");

function HashedChunkIdsPlugin(options) {
  this.options = options || {};
}

HashedChunkIdsPlugin.prototype.apply = function apply(compiler) {
  var hashFn = getHashFn(this.options);
  compiler.plugin("compilation", function (compilation) {
    compilation.plugin("before-chunk-ids", function (chunks) {
      chunks.forEach(function (chunk) {
        var c = {
          path: chunk.modules[0].context,
          id: chunk.id,
          name: chunk.name
        }
        console.log(c);
        if (chunk.id === null) {
          if (typeof chunk.name !== "undefined") {
            chunk.id = hashFn(chunk.name);
          } else {
            console.log(chunk.modules[0].context + ' was not run through the hash function');
          }
        }
      });
    });
  });
};

module.exports = HashedChunkIdsPlugin;

In my case. I am using Ant Design. It doesn't seem that it is failing on the actual css modules. The css modules are being extracted higher up in the webpack config. The HashedChunkIdsPlugin runs in the js build process. The reason it looked like it failed on css modules is because of the way Ant Design imports it's css modules. For example Button.js will import style.js and style.js will import style.css and button.css. It's actually style.js that returns chunk.id === null and chunk.name === undefined and fails the hash function. By this point the css is already extracted and Button is included in the parent page chunk, so skipping the style.js files works fine. I don't know why style.js is being separated to it's own chunk.

Bouncey · 2017-08-21T06:33:19Z

Is this likely to be fixed in v1?

okcoker · 2017-09-08T07:36:10Z

So after many hours of debugging this for my site (which seemed to break out of nowhere, I assume because of dependencies that got updated after re-running yarn), I decided to start fresh from the blog starter site. I slowly added my site's files into the newly created folder, building after each addition.

Turns out my specific issue was the use of require.ensure.

I'm not sure of the fix yet, but if that's not everyone else's issue, I'd recommend starting from scratch or something similar. I'll report back here when I find a solution to this. I'd still like to lazy load modules if possible.

just-boris · 2017-09-08T14:49:21Z

I have got the same issue, solved via third argument to require.ensure

require.ensure([], require => {
   require('./something');
}, 'name-of-my-chunk')

name-of-my-chunk will be explicitly assigned to dynamically loaded chunk.

However, of course it would be nice to make this working without this additional friction.

stale · 2018-03-07T15:39:02Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

just-boris · 2018-03-07T16:47:46Z

@Stale the issue is still actual.

KyleAMathews · 2018-09-08T04:17:58Z

Due to the high volume of issues, we're closing out older ones without recent activity. Please open a new issue if you need help!

maetl added a commit to digivizer/gatsby-1659 that referenced this issue Aug 6, 2017

Demonstrating CSS Module import bug in Gatsby 1.3

a2f53e6

Crashes with an error during `gatsby build`, as documented on gatsbyjs/gatsby#1659

bskimball mentioned this issue Aug 9, 2017

Update hashed-chunk-ids-plugin.js #1753

Merged

pouretrebelle mentioned this issue Aug 13, 2017

GraphQL not sending data to component #1786

Closed

smashou mentioned this issue Aug 17, 2017

Error: Cannot read property 'pathname' of undefined Vagr9K/gatsby-advanced-starter#12

Closed

okcoker mentioned this issue Aug 18, 2017

Make sure to always return array within reduce function #1851

Merged

Bouncey mentioned this issue Aug 21, 2017

Refactor CSS Modules freeCodeCamp/guide#183

Closed

andresgutgon mentioned this issue Jan 21, 2018

Update Gatsby version for not getting TypeError: Data must be a string or a buffer datocms/gatsby-portfolio#1

Closed

stale bot added the stale? Issue that may be closed soon due to the original author not responding any more. label Mar 7, 2018

stale bot removed the stale? Issue that may be closed soon due to the original author not responding any more. label Mar 7, 2018

KyleAMathews closed this as completed Sep 8, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TypeError: Data must be a string or a buffer #1659

TypeError: Data must be a string or a buffer #1659

cstrangedk commented Jul 29, 2017

cr101 commented Jul 30, 2017

0xc0dec commented Jul 30, 2017 •

edited

Loading

cstrangedk commented Aug 1, 2017

maetl commented Aug 6, 2017

bskimball commented Aug 8, 2017 •

edited

Loading

bskimball commented Aug 8, 2017

bskimball commented Aug 9, 2017 •

edited

Loading

maetl commented Aug 9, 2017

KyleAMathews commented Aug 9, 2017

bskimball commented Aug 9, 2017 •

edited

Loading

maetl commented Aug 9, 2017

KyleAMathews commented Aug 9, 2017

bskimball commented Aug 9, 2017

KyleAMathews commented Aug 9, 2017

bskimball commented Aug 9, 2017 •

edited

Loading

bskimball commented Aug 9, 2017 •

edited

Loading

maetl commented Aug 9, 2017 •

edited

Loading

bskimball commented Aug 9, 2017

maetl commented Aug 9, 2017

bskimball commented Aug 9, 2017 •

edited

Loading

maetl commented Aug 9, 2017 •

edited

Loading

bskimball commented Aug 9, 2017 •

edited

Loading

Bouncey commented Aug 21, 2017

okcoker commented Sep 8, 2017

just-boris commented Sep 8, 2017 •

edited

Loading

stale bot commented Mar 7, 2018

just-boris commented Mar 7, 2018

KyleAMathews commented Sep 8, 2018

TypeError: Data must be a string or a buffer #1659

TypeError: Data must be a string or a buffer #1659

Comments

cstrangedk commented Jul 29, 2017

cr101 commented Jul 30, 2017

0xc0dec commented Jul 30, 2017 • edited Loading

cstrangedk commented Aug 1, 2017

maetl commented Aug 6, 2017

bskimball commented Aug 8, 2017 • edited Loading

bskimball commented Aug 8, 2017

bskimball commented Aug 9, 2017 • edited Loading

maetl commented Aug 9, 2017

KyleAMathews commented Aug 9, 2017

bskimball commented Aug 9, 2017 • edited Loading

maetl commented Aug 9, 2017

KyleAMathews commented Aug 9, 2017

bskimball commented Aug 9, 2017

KyleAMathews commented Aug 9, 2017

bskimball commented Aug 9, 2017 • edited Loading

bskimball commented Aug 9, 2017 • edited Loading

maetl commented Aug 9, 2017 • edited Loading

bskimball commented Aug 9, 2017

maetl commented Aug 9, 2017

bskimball commented Aug 9, 2017 • edited Loading

maetl commented Aug 9, 2017 • edited Loading

bskimball commented Aug 9, 2017 • edited Loading

Bouncey commented Aug 21, 2017

okcoker commented Sep 8, 2017

just-boris commented Sep 8, 2017 • edited Loading

stale bot commented Mar 7, 2018

just-boris commented Mar 7, 2018

KyleAMathews commented Sep 8, 2018

0xc0dec commented Jul 30, 2017 •

edited

Loading

bskimball commented Aug 8, 2017 •

edited

Loading

bskimball commented Aug 9, 2017 •

edited

Loading

bskimball commented Aug 9, 2017 •

edited

Loading

bskimball commented Aug 9, 2017 •

edited

Loading

bskimball commented Aug 9, 2017 •

edited

Loading

maetl commented Aug 9, 2017 •

edited

Loading

bskimball commented Aug 9, 2017 •

edited

Loading

maetl commented Aug 9, 2017 •

edited

Loading

bskimball commented Aug 9, 2017 •

edited

Loading

just-boris commented Sep 8, 2017 •

edited

Loading