Non-string data keys

[davishmcclurg]

Non-string data keys cause errors when generating pointers:

?> schema = {
?>   'type' => 'object'
>> }
=> {"type"=>"object"}
>>
?> data = {
?>   true => 'value'
>> }
=> {true=>"value"}
>>
>> JSONSchemer.schema(schema).valid?(data)
/Users/dharsha/repos/json_schemer/lib/json_schemer/schema/base.rb:625:in `escape_json_pointer_token': undefined method `gsub' for true:TrueClass (NoMethodError)

        token.gsub(JSON_POINTER_TOKEN_ESCAPE_REGEXP, JSON_POINTER_TOKEN_ESCAPE_CHARS)
             ^^^^^
	from /Users/dharsha/repos/json_schemer/lib/json_schemer/schema/base.rb:552:in `block in validate_object'
	from /Users/dharsha/repos/json_schemer/lib/json_schemer/schema/base.rb:551:in `each'
	from /Users/dharsha/repos/json_schemer/lib/json_schemer/schema/base.rb:551:in `validate_object'
	from /Users/dharsha/repos/json_schemer/lib/json_schemer/schema/base.rb:299:in `validate_type'
	from /Users/dharsha/repos/json_schemer/lib/json_schemer/schema/base.rb:211:in `validate_instance'
	from /Users/dharsha/repos/json_schemer/lib/json_schemer/schema/base.rb:91:in `each'
	from /Users/dharsha/repos/json_schemer/lib/json_schemer/schema/base.rb:91:in `none?'
	from /Users/dharsha/repos/json_schemer/lib/json_schemer/schema/base.rb:91:in `valid_instance?'
	from /Users/dharsha/repos/json_schemer/lib/json_schemer/schema/base.rb:81:in `valid?'
	from (irb):9:in `<main>'
	from bin/console:14:in `<main>'

Calling to_s in escape_json_pointer_token solves the issue, but I'm not sure if it makes sense since non-string keys aren't JSON and can produce conflicts with other valid keys.

For now, I'm opening this for discussion and to see if more people run into these errors.

Introduced: #122

[davishmcclurg]

There's a little more info here: #122 (comment)

[legendetm]

Having similar problem, when key is Integer

schemer = JSONSchemer.schema(JSON.parse(<<~JSON))
  {
    "$schema": "http://json-schema.org/draft-07/schema#",
    "$id": "analytics",
    "title": "Analytics",
    "description": "Analytics",
    "type": "object",
    "properties": {
      "superscripts": {
        "type": "object",
        "propertyNames": {
          "type": "integer"
        },
        "additionalProperties": {
          "type": "string"
        }
      }
    }
  }
JSON
schemer.valid?({ 'superscripts' => { 1 => 'Note' } })
# NoMethodError: undefined method `gsub' for 1:Integer
# from json_schemer-0.2.24/lib/json_schemer/schema/base.rb:626:in `escape_json_pointer_token'

Needed to revert back to version 0.2.21.

[pmusaraj]

I am running into this as well. Would it make sense for a non-string key to simply raise a validation error? Something like this:

# on line 553 of schema/base.rb (for example)
if !key.kind_of?(String)
  yield error(instance, 'invalid_keys')
end
escaped_key = escape_json_pointer_token(key)

[legendetm]

Hi pmusaraj, the schema allows me to define, that propertyName type is integer and I would expect this to work, not to raise an error.

[pmusaraj]

@legendetm hmm, as @davishmcclurg pointed out above, only string keys are valid JSON.

[legendetm]

When I read the core specification, it says the "name that the schema is testing will always be a string" what also may mean, that the key is stringified for testing. This does not say, that the key must be string.

propertyNames
The value of "propertyNames" MUST be a valid JSON Schema.

If the instance is an object, this keyword validates if every property name in the instance validates against the provided schema. Note the property name that the schema is testing will always be a string.

Omitting this keyword has the same behavior as an empty schema.

But when viewing understanding object reference, then this states that object keys must always be strings.

Since object keys must always be strings anyway, it is implied that the schema given to propertyNames is always at least: { "type": "string" }

In all, this can be confusing and you can find multiple examples in wild where the type for propertyNames is set to another type. It would be better, if the schema would be also considered invalid when non string type is specified. As currently the schema is considered valid, but cannot be validated against JSON.

Considering the above, I will update my schema to use "pattern": "[0-9]+" instead of "type": "integer".

I checked how different online JSON schema validators validate, and this seems to be handled in multiple different ways:

1. "type": "integer" was mostly invalid.
2. "pattern": "[0-9]+" and actual key was number 1, was mostly working (key in JSON was stringified)
3. "pattern": "[0-9]+" and actual key was string "1", was working everywhere.

[davishmcclurg]

I am running into this as well. Would it make sense for a non-string key to simply raise a validation error?

@pmusaraj that's an interesting idea. I'm concerned about conflating schema validation errors and input errors, though. I think it makes sense to only yield errors for actual validation errors and not for bad input. Maybe it would be better to raise an error similar to InvalidSymbolKey:

json_schemer/lib/json_schemer/schema/base.rb

Line 61 in a7a2218

raise InvalidSymbolKey, 'schemas must use string keys' if schema.is_a?(Hash) && !schema.empty? && !schema.first.first.is_a?(String)

---

When I read the core specification, it says the "name that the schema is testing will always be a string" what also may mean, that the key is stringified for testing. This does not say, that the key must be string.

@legendetm I think the "key must be string" portion is implied because JSON schemas operate on JSON, which only allows string keys:

>> JSON.parse('{1:"x"}')
/Users/dharsha/.asdf/installs/ruby/3.2.0/lib/ruby/3.2.0/json/common.rb:216:in `parse': unexpected token at '{1:"x"}' (JSON::ParserError)
        from /Users/dharsha/.asdf/installs/ruby/3.2.0/lib/ruby/3.2.0/json/common.rb:216:in `parse'
        from (irb):1:in `<main>'
        from /Users/dharsha/.asdf/installs/ruby/3.2.0/lib/ruby/gems/3.2.0/gems/irb-1.6.2/exe/irb:11:in `<top (required)>'
        from /Users/dharsha/.asdf/installs/ruby/3.2.0/bin/irb:25:in `load'
        from /Users/dharsha/.asdf/installs/ruby/3.2.0/bin/irb:25:in `<main>'
>> JSON.parse('{"1":"x"}')
=> {"1"=>"x"}

At the level the JSON schema specification operates on, there's no such thing as a non-string key.

Considering the above, I will update my schema to use "pattern": "[0-9]+" instead of "type": "integer".

That seems like the right approach 👍

[myxoh]

Updating from 0.2.22 to 0.2.24 is breaking some of our specs that were generated using symbols as keys so:
An example minimul reproduceable spec:

This worked in 0.2.23 but stops working on 0.2.24

  it 'tests a schema' do
    schema = JSONSchemer.schema(
      { title: 'A titled schema', additional_properties: false, type: 'object',
        properties: { title: { type: 'string ' } } }.stringify_keys, insert_property_defaults: true
    )
    expect(schema).to be_valid({ title: 'some title' })
  end

While I think the new behaviour is understandable - I don't think this should have been a point change - it completely breaks the contract!

I think it makes sense to have more clear errors (Although I do think there's certainly value in supporting symbol keys when it comes to ease of typing fixtures in Ruby).

But ideally the library should avoid releasing point versions with breaking changes like this - this should definetely be added as a spec to make sure future changes can't break the behaviour.

I'd be happy to contribute with a spec

[davishmcclurg]

Thanks for the comment @myxoh! It seems like this is causing people enough pain to revert to the previous behavior. It's unfortunate because symbol keys are not well supported, ie it only works if both the schema and data use the same type of keys.

I think it makes sense to have more clear errors (Although I do think there's certainly value in supporting symbol keys when it comes to ease of typing fixtures in Ruby).

I agree it would be nice to support symbols properly. Though I'm not sure exactly what that looks like. I've been trying to avoid HashWithIndifferentAccess but maybe that would make the most sense.

I'd be happy to contribute with a spec

That would be great. I believe the fix (if you don't mind contributing that as well) is to add a to_s in escape_json_pointer_token:

json_schemer/lib/json_schemer/schema/base.rb

Line 625 in 0861dd2

token.gsub(JSON_POINTER_TOKEN_ESCAPE_REGEXP, JSON_POINTER_TOKEN_ESCAPE_CHARS)

[davishmcclurg]

Released in 2.0.0. Data keys are deep-stringified to support symbols. Symbol values are not stringified.