JSON - loading null as undefined, so saved struct/array would have same values as when saving

[gnysek]

Is your feature request related to a problem?

Gamemaker have inconsistency in json_parse() and json_stringify() functions.

• json_stringify() saves GML undefined as JSON null
• json_parse() loads JSON null as pointer_null

This means, that when we're saving struct in which some values are undefined because they are either not set, or developer marks some variable as "not set" in this way, on loading it back, we need to pass it to convert all pointer_nulls back to undefineds.
This is not that easy and time consuming in case of nested structures and arrays.

I understand that GML saves undefined as null because JSON doesn't offer undefined (while JS does), so there's no other value, but I don't understand why it reads it back as different value (indeed pointer_null is maybe more close to JSON null, but generally pointer_null is rather not often used for storing empty values in structs/lists/maps/arrays in favour of undefined).

It's rather usual, that GM developers are using undefined in GML to mark "not set" things, like:

data = {
  x: 5,
  y: 10,
  last_clicked: undefined,
}

and it's rather expected, that after saving and loading such struct, last_clicked would be still undefined not pointer_null.

Describe the solution you'd like

As changing json_parse() so it gives undefined instead pointer_null could break existing code, my proposal is to add additional argument, how null should be treated:

json_parse(string [, undefined_instead_of_pointer = false])

In that case:

json_parse("{\"x\": 5,\"y\": 10,\"last_clicked\":null}");
json_parse("{\"x\": 5,\"y\": 10,\"last_clicked\":null}", false);

would result in:

{
  x: 5,
  y: 10,
  last_clicked: pointer_null,
}

while:

json_parse("{\"x\": 5,\"y\": 10,\"last_clicked\":null}", true);

would result in:

{
  x: 5,
  y: 10,
  last_clicked: undefined,
}

Describe alternatives you've considered

We already have special prefixes/suffixes for some values for json_stringify:

• int64 is saved as @i64@<NUMBER>$i64$
• infinity is saved as @@infinity$$ / @@-infinity$$
• NaN is saved as @@nan$$

So, undefined could be saved as:

• @@undefined$$

However, this again could break existing code and existing functionality, so that would require:

json_stringify(struct_array [, pretty_print = false, undefined_gml_compatible = false])

This way, saving data which contains undefined that can be read back as undefined would look this way:

json_stringify(data, , true); // skipping pretty_print arg
json_stringify(data, true, true); // in case we want pretty print

Additional context

No response

[gnysek]

Another idea (by @Alphish ) is to add:

var _json = json_parse_ext(string, function(_value) {
   ... // make any additional processing you wish
   return _value;
});

[Alphish]

More complete example, with both stringify and parse sides covered:

var _json = json_stringify(room_data, /* pretty print */ false, function(_value) {
    if (!is_handle(_value))
        return _value;
    if (sprite_exists(_value))
        return "#ASSET:" + sprite_get_name(_value);
    else if (object_exists(_value))
        return "#ASSET:" + object_get_name(_value);
    else if (room_exists(_value))
        return "#ASSET:" + room_get_name(_value);
    else
        return _value;
});

And the parse side:

var _struct = json_parse_ext(room_json, function(_value) {
    if (_value == pointer_null)
        return undefined; // this handles the original point of parsing JSON nulls as undefined values
    else if (is_string(_value) && string_starts_with(_value, "#ASSET:"))
        return asset_get_index(string_delete(_value, 1, string_length("#ASSET:")));
    else
        return _value;
});

(note: apparently in 2024.2 assets will be magically handled by JSON functions, but there are other potential uses too: e.g. making structs store their constructor type and static_setting that type on load etc.)

[gnysek]

mentioned 2024.2 change: #2808

[gnysek]

Also, worth mentioning, as both this FR and #2808 would greatly extend flexibility if #2891 would be also added at some point.

[backYard321]

Definitely would be nice functionality to have! Personally, I would prefer it just to be handled like assets in #2808 (where undefined would presumably be stringified/parsed differently from that point on, in a way that doesn't break existing functionality), rather than needing additional arguments in json_parse()/json_stringify() or to pass in a custom function. That said, the latter would also be cool to have too!

[gnysek]

Russell already confirmed he will work something out, so we need to wait which solution would be taken. Keep fingers crossed for 2024.2 ;)

[rwkay]

Feature added in 2024.2

• Change default behaviour with JSON null becoming undefined when read from JSON.
• This means GML -> JSON -> GML preserves undefined
• Add optional filter functions to json_parse and json_stringify
• functions take a key, value and return a value (this is the value output)

var data = {
  x: 5,
  y: 10,
  last_clicked: undefined,
  haha :  [ 20, 30, undefined, { blah : 10, foo : undefined } ]
}

var jsonData = json_stringify( data, true, function(k,a) { show_debug_message($"key={k}, value={a}"); return a; } );
show_debug_message( jsonData );

var jsonRoundtrip = json_parse( jsonData, function(k,a) { show_debug_message($"key={k}, value={a}");  return a; } );
show_debug_message( jsonRoundtrip 

[sihammill]

Feature verified in IDE v2024.200.0.472 Runtime v2024.200.0.488