Consider standardizing the exact behaviour of sub- and pipe-expressions

[springcomp]

Ambiguities in the specification

Sub-Expressions

The specification outlines the behaviour of sub-expressions like this:

left-evaluation = search(left-expression, original-json-document)
result = search(right-expression, left-evaluation)

However, compliance tests disagree and evaluate search( `null`.[@], {} ) -> null.

So the real pseudo code would look like so:

left-evaluation = search(left-expression, original-json-document)
if left-evaluation is `null` then result = `null`
else result = search(right-expression, left-evaluation)```

The canonical JavaScript and Python implementations behave accordingly.

Pipe Expressions

The specification also touches on the fact that a pipe-expression is similar 🤔 to sub-expression. The only difference being in its handling of the projections and the kinds of expressions allowed on its right-hand-side.

My belief is that a pipe-expression should behave a bit differently, however. In fact, I think it should behave as the original pseudo code for sub-expression was written:

left-evaluation = search(left-expression, original-json-document)
result = search(right-expression, left-evaluation)

I believe JMESPath should evaluate search( `null` | [@], {} ) -> [ null ]

However, the JavaScript and Python implementations disagree and behave exactly as evaluating a sub-expression.

Alternative and Standard Behaviour

The goal of JMESPath Community is to improve on JMESPath. For this to gain traction, I think we should strive for compatibility until we have a good track record of improvements, governance and stewardship of the language.

However, I think we have a small window of opportunity.
In fact the compliance tests do not cover the `null` | [@] case so maybe we can standardize on any behaviour we want.

My question to the community is to garner feedback and land on a consensus for how pipe-expression should be standardized.

[springcomp]

See: #114
See: jmespath-community/jmespath.test#5
See: jmespath/jmespath.py#292

[springcomp]

✅

[springcomp]

#156

[springcomp]

Evaluation of Pipe Expressions

JEP	19
Author	Maxime Labelle
Created	29-October-2022
SemVer	MINOR
Status	Draft
[Discussion]	#113

Abstract

This JEP introduces changes that outline the exact intended behaviour of evaluating a pipe-expression. It also clarifies an ambiguity when evaluating a sub-expression where the left-hand-side evaluates to null.

Motivation

Sub Expression Evaluation

The specification for sub-expression outlines how it should be evaluated using pseudocode:

left-evaluation = search(left-expression, original-json-document)
result = search(right-expression, left-evaluation)

This is slightly ambiguous, as many compliance tests expect the result to be null when the left-hand-side evaluates to  null. So, the real pseudocode shoud in fact be:

left-evaluation = search(left-expression, original-json-document)
if left-evaluation is `null` then result = `null`
else result = search(right-expression, left-evaluation)

Pipe Expression Evaluation

In contrast, however, it seems intuitive for pipe-expression to evaluate as was originally outlined by the first pseudocode fragment referred to a above.

left-evaluation = search(left-expression, original-json-document)
result = search(right-expression, left-evaluation)

Which means that the evaluation should still happens if the left-hand-side is null.

Specification

Sub Expressions

The paragraph on sub expressions in the specification will be updated as follows, with changes outlined in bold:

A subexpression is evaluted as follows:

• Evaluate the expression on the left with the original JSON document.
• If the result of the left expression evaluation is null, return null.
• Otherwise, eEvaluate the expression on the right with the result of the left expression evaluation.

In pseudocode:

left-evaluation = search(left-expression, original-json-document)
If result is `null` the result = `null`
else result = search(right-expression, left-evaluation)

Pipe Expressions

Likewise the paragraph on pipe expressions in the specification will be updated as follows:

A pipe expression combines two expressions, separated by the | character. It is similar to a sub-expression with twothe following important distinctions:

1. Any expression can be used on the right hand side. A sub-expression restricts the type of expression that can be used on the right hand side.
2. A pipe-expression stops projections on the left hand side for from propagating to the right hand side. If the left expression creates a projection, it does not apply to the right hand side.
3. A pipe-expression evaluates the right expression unconditionally. A sub-expression shortcuts evaluation if the result of the left expression evaluation is null

In pseudocode:

left-evaluation = search(left-expression, original-json-document)
result = search(right-expression, left-evaluation)

Example

Expression	Result
`null` . [@]	null
`` null `	[@] ``

Compliance

The pipe.json compliance test file will be updated with the example from this JEP.