Keys with special characters can't be referenced in filters

[lesderid]

If an object key contains special characters (e.g. - or .), it can't be filtered on even though it's a valid JSON object key.

For example, ?has_stuff.foo-bar=true can't be used to check if "foo-bar" is a field of "stuff". There doesn't seem to be any way to escape special characters either.

[leplatrem]

Indeed!

How do you think we could solve this? Using quotes I guess? ?has_stuff."foo-bar"=true?

[lesderid]

How do you think we could solve this? Using quotes I guess? ?has_stuff."foo-bar"=true?

There seem to be a few options:

Syntax	jq	JSONPath
.stuff."foo-bar"	✅	❌
.stuff.["foo-bar"]	✅	❌
.stuff["foo-bar"]	✅1	✅
.stuff['foo-bar']	❌	✅

Edit: For what it's worth, I don't have a particularly strong preference. I think the first one is probably the most intuitive though.

Footnotes

1. While this works, I can't find it specified in the docs. ↩

[leplatrem]

Thanks for the reference 👌

If we can't have both, I think I would prefer if we would follow a IETF standard.
We could replace the jsonpatch lib we currently use with this one that seems to support both JSONPath and JSONPatch.

The code that handles the querystring filters is here:

kinto/kinto/core/resource/__init__.py

Line 1108 in 80f4315

operator = getattr(COMPARISON, keyword.upper())

And filters are then implemented in each storage down the line:

kinto/kinto/core/storage/memory.py

Line 511 in 5ea7d60

if f.operator == COMPARISON.HAS:

kinto/kinto/core/storage/postgresql/__init__.py

Lines 785 to 787 in 1c23559

	# Subfields: ``person.name`` becomes ``data->person->>name``
	subfields = filtr.field.split(".")
	for j, subfield in enumerate(subfields):

With tests for both:

kinto/tests/core/resource/test_filter.py

Lines 251 to 261 in bf94438

	def test_has_values(self):
	self.validated["querystring"] = {"has_sometimes": True}
	result = self.resource.plural_get()
	values = [item["sometimes"] for item in result["data"]]
	assert None in values
	self.assertEqual(sorted([v for v in values if v]), ["available", "present"])
	def test_has_values_false(self):
	self.validated["querystring"] = {"has_sometimes": False}
	result = self.resource.plural_get()
	values = ["sometimes" in item for item in result["data"]]

kinto/kinto/core/storage/testing.py

Lines 710 to 722 in 1c23559

	self.create_object({"flavor": "pineapple", "author": "null"})
	filters = [Filter("author", True, utils.COMPARISON.HAS)]
	objects = self.storage.list_all(filters=filters, **self.storage_kw)
	self.assertEqual(len(objects), 4)
	self.assertEqual(
	sorted([r["flavor"] for r in objects]),
	["blueberry", "pineapple", "raspberry", "watermelon"],
	)
	filters = [Filter("author", False, utils.COMPARISON.HAS)]
	objects = self.storage.list_all(filters=filters, **self.storage_kw)
	self.assertEqual(len(objects), 1)
	self.assertEqual(objects[0]["flavor"], "strawberry")

Would you like to try a contribution?