Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Weird behaviour in buffer on node v4.0.0 #3982

Closed
Junkern opened this issue Nov 23, 2015 · 2 comments
Closed

Weird behaviour in buffer on node v4.0.0 #3982

Junkern opened this issue Nov 23, 2015 · 2 comments
Labels
question Issues that look for answers.

Comments

@Junkern
Copy link

Junkern commented Nov 23, 2015

var length = 256;
var buf = new Buffer(length);
for (var i = 0; i < length; ++i) {
        buf[i] = i;
}
var string = buf.toString();
for(var i = 0; i < string.length; i++) {
    console.log("i",i, string[i], string.charCodeAt(i));
}

results in the following output:

i 123 { 123
i 124 | 124
i 125 } 125
i 126 ~ 126
i 127 ⌂ 127
i 128 ? 65533
i 129 ? 65533
i 130 ? 65533
i 131 ? 65533
i 132 ? 65533

I cannot quite get my head around the fact, that the charCodeAt() suddenly changes from 127 to 65533. I also took a look at the source code of buffer.js and node_buffer.cc, but could not find code that explains this behaviour. Is this behavior intended? If so, could someone explain why?

@fatfisz
Copy link

fatfisz commented Nov 23, 2015

These are invalid characters and they are substituted with 0xFFFD (see: http://www.fileformat.info/info/unicode/char/0fffd/index.htm).
That happens because the default encoding in Buffer.toString is UTF-8.

@targos targos added the question Issues that look for answers. label Nov 23, 2015
@targos
Copy link
Member

targos commented Nov 23, 2015

What @fatfisz said and the fact that Buffer#toString defaults to UTF-8 encoding.
See also https://en.wikipedia.org/wiki/UTF-8

I'm closing this because it's not a technical issue with node.
If you have more questions, you are welcome to ask them in https://github.com/nodejs/help/issues

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Issues that look for answers.
Projects
None yet
Development

No branches or pull requests

3 participants