Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dump(): Non-conforming with JSON-spec escape of strings? #4088

Closed
1 of 2 tasks
theIDinside opened this issue Jul 28, 2023 · 3 comments
Closed
1 of 2 tasks

dump(): Non-conforming with JSON-spec escape of strings? #4088

theIDinside opened this issue Jul 28, 2023 · 3 comments

Comments

@theIDinside
Copy link

theIDinside commented Jul 28, 2023

Description

This library seems to not be in compliance/non-conformant with the JSON spec, though I'm not 100% on that. None the less, it shows serious differences from almost any other library or "first-class citizen" of languages with built-in support of JSON (as is the case in Javascript, or Python for instance).

Reproduction steps

json obj;
obj["foo"] = 10;
obj["str"] = "\nhello world!\n";
obj["baz"] = 30;

const auto dump = obj.dump();

In javascript, if we create the following object and serialize it using JSON.stringify(obj) we get the following:

let obj = { a: 10, foo: 20, str: `
hello world!
`, baz: 30};

JSON.stringify(obj);

// we can also do 

let obj = { a: 10, foo: 20, str: "\nhello world!\n", baz: 30};

Python example:

obj = {}
obj["a"] = 10
obj["str"] = "\nhello world!\n"

import json
json.dumps(obj)

Expected vs. actual results

The outputs from the library is

// output
// "{ "foo": 10, "str": "\nhello world!\n", "baz": 30 }"

Where as Javascript produces

// output
// '{"a":10,"foo":20,"str":"\\nhello world!\\n","baz":30}'

and Python produces

'{"a": 10, "str": "\\nhello world!\\n"}'

Notice the correct escaping of the string. This library seems to only actually do the actual escape and not the "doubly" escape, which as far as I am aware, is not JSON-compliant?

I came across this particular bug (if it is a bug - it sort of feels like a bug at least?), because my application communicates with javascript over the "wire" (whether that is a socket, pipe, stdio, doesn't matter) - and fails, because nlohmann doesn't produce proper JSON string output that Javascript's JSON-functionality can parse.

I've used many languages before, using JSON as a communication protocol, and they all exhibit the same behavior, so I was under the assumption that this library would also be able to function in that manner. Perhaps there is some other feature in this library that produces the behavior seen above? (Python's json.dumps() for instance ,works in this way).

JSON spec says something like:

A string is a sequence of Unicode code points wrapped with quotation marks (U+0022). All characters may be placed within the quotation marks except for the characters that must be escaped: quotation mark (U+0022), reverse solidus (U+005C), and the control characters U+0000 to U+001F. There are two-character escape sequence representations of some characters.

Here's a stack overflow post discussing this issue, as well:

How do I handle newlines in JSON?

Minimal code example

No response

Error messages

No response

Compiler and operating system

clang++-15

Library version

v3.11.2

Validation

@theIDinside theIDinside changed the title Non-conforming escape of strings? Non-conforming with JSON-spec escape of strings? Jul 28, 2023
@theIDinside theIDinside changed the title Non-conforming with JSON-spec escape of strings? dump(): Non-conforming with JSON-spec escape of strings? Jul 28, 2023
@nlohmann
Copy link
Owner

I am confident the escaping of this library is correct. There are thousands of tests and compliance checks, and also many projects where the library is used together with other libraries. Please check again if there is some error in your setup.

@theIDinside
Copy link
Author

theIDinside commented Jul 28, 2023

I'll check again. Could you please inform me where the tests are (if there are any) where .dump is verified to produce escaped escape-sequences (as per JSON spec and as Javascript & Python examples above), perhaps that will guide me in what I'm doing wrong.

Edit:
It could potentially be that another library is simply eating the \\ when doing it's formatting (thus producing the incorrect \ instead. So if that test exists, it would be greatly helpful as this would conclude this issue entirely).

@theIDinside
Copy link
Author

Yeah, I'm closing this. Another library was indeed eating that extra escape character.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants