Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplified standard library; marked redundant filters as duplicated #426

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 15 additions & 8 deletions builtin.c
Original file line number Diff line number Diff line change
Expand Up @@ -693,19 +693,24 @@ static block bind_bytecoded_builtins(block b) {
static const char* const jq_builtins[] = {
"def map(f): [.[] | f];",
"def select(f): if f then . else empty end;",
"def sort_by(f): _sort_by_impl(map([f]));",
"def group_by(f): _group_by_impl(map([f]));",
"def unique: group_by(.) | map(.[0]);",
"def unique_by(f): group_by(f) | map(.[0]);",
"def max_by(f): _max_by_impl(map([f]));",
"def min_by(f): _min_by_impl(map([f]));",
"def sort(f): _sort_by_impl(map([f]));",
"def sort_by(f): sort(f);",
"def group(f): _group_by_impl(map([f]));",
"def group_by(f): group(f);",
"def unique: group(.) | map(.[0]);",
"def unique(f): group(f) | map(.[0]);",
"def unique_by(f): unique(f);",
"def max(f): _max_by_impl(map([f]));",
"def min(f): _min_by_impl(map([f]));",
"def max_by(f): max(f);",
"def min_by(f): min(f);",
#include "libm.h"
"def add: reduce .[] as $x (null; . + $x);",
"def del(f): delpaths([path(f)]);",
"def _assign(paths; value): value as $v | reduce path(paths) as $p (.; setpath($p; $v));",
"def _modify(paths; update): reduce path(paths) as $p (.; setpath($p; getpath($p) | update));",
"def recurse(f): ., (f | select(. != null) | recurse(f));",
"def recurse_down: recurse(.[]?);",
"def recurse: recurse(.[]?);",
"def to_entries: [keys[] as $k | {key: $k, value: .[$k]}];",
"def from_entries: map({(.key): .value}) | add | .//={};",
"def with_entries(f): to_entries | map(f) | from_entries;",
Expand All @@ -714,7 +719,7 @@ static const char* const jq_builtins[] = {
"def index(i): if type == \"array\" and (i|type) == \"array\" then .[i] elif type == \"array\" then .[[i]] else .[i] end | .[0];",
"def rindex(i): if type == \"array\" and (i|type) == \"array\" then .[i] elif type == \"array\" then .[[i]] else .[i] end | .[-1:][0];",
"def paths: path(recurse(if (type|. == \"array\" or . == \"object\") then .[] else empty end))|select(length > 0);",
"def leaf_paths: . as $dot|paths|select(. as $p|$dot|getpath($p)|type|. != \"array\" and . != \"object\");",
"def paths(node_filter): . as $dot|paths|select(. as $p|$dot|getpath($p)|node_filter);",
"def any: reduce .[] as $i (false; . or $i);",
"def all: reduce .[] as $i (true; . and $i);",
"def arrays: select(type == \"array\");",
Expand All @@ -726,9 +731,11 @@ static const char* const jq_builtins[] = {
"def nulls: select(type == \"null\");",
"def values: arrays, objects, booleans, numbers, strings;",
"def scalars: select(. == null or . == true or . == false or type == \"number\" or type == \"string\");",
"def leaf_paths: paths(scalars);",
"def join(x): reduce .[] as $i (\"\"; . + (if . == \"\" then $i else x + $i end));",
"def flatten: reduce .[] as $i ([]; if $i | type == \"array\" then . + ($i | flatten) else . + [$i] end);",
"def flatten(x): reduce .[] as $i ([]; if $i | type == \"array\" and x > 0 then . + ($i | flatten(x-1)) else . + [$i] end);",
"def range(x): range(0;x);",
};
#undef LIBM_DD

Expand Down
109 changes: 59 additions & 50 deletions docs/content/3.manual/manual.yml
Original file line number Diff line number Diff line change
Expand Up @@ -708,29 +708,25 @@ sections:
input: '[1,2,3]'
output: ['[2,3,4]']

- title: "`paths`"
- title: "`paths`, `leaf_paths"
body: |

Outputs the paths to all the elements in its input (except it
does not output the empty list, representing . itself).

`paths` is equivalent to

def paths: path(recurse(if (type|. == "array" or . == "object") then .[] else empty end))|select(length > 0);


It accepts an optional argument consisting of a filter for the paths
to be outputted. That is, `paths(numbers)` outputs the paths to
all numeric values.

For legacy reasons, `leaf_paths` exists as an alias for
`paths(scalars)`. It is considered deprecated and will be
dropped by the next major release.

examples:
- program: '[paths]'
input: '[1,[[],{"a":2}]]'
output: ['[[0],[1],[1,0],[1,1],[1,1,"a"]]']

- title: "`leaf_paths`"
body: |

Outputs the paths to all the leaves (non-array, non-object
elements) in its input.

examples:
- program: '[leaf_paths]'
- program: '[paths(scalars)]'
input: '[1,[[],{"a":2}]]'
output: ['[[0],[1,1,"a"]]']

Expand Down Expand Up @@ -825,6 +821,8 @@ sections:
produces 6 numbers, from 4 (inclusive) to 10 (exclusive). The numbers
are produced as separate outputs. Use `[range(4;10)]` to get a range as
an array.

Its first argument can be omitted; it defaults to zero.

examples:
- program: 'range(2;4)'
Expand All @@ -833,6 +831,9 @@ sections:
- program: '[range(2;4)]'
input: 'null'
output: ['[2,3]']
- program: '[range(4)]'
input: 'null'
output: ['[0,1,2,3]']

- title: "`floor`"
body: |
Expand Down Expand Up @@ -909,82 +910,90 @@ sections:
sorted order), and if their keys are equal then the values
are compared key by key.

`sort_by` may be used to sort by a particular field of an
object, or by applying any jq filter. `sort_by(foo)`
`sort` may be used to sort by a particular field of an
object, or by applying any jq filter. `sort(foo)`
compares two elements by comparing the result of `foo` on
each element.

For legacy reasons, `sort_by(foo)` exists as an alias for `sort(foo)`.
This alias is considered deprecated and will be dropped
in the next major release.

examples:
- program: 'sort'
input: '[8,3,null,6]'
output: ['[null,3,6,8]']
- program: 'sort_by(.foo)'
- program: 'sort(.foo)'
input: '[{"foo":4, "bar":10}, {"foo":3, "bar":100}, {"foo":2, "bar":1}]'
output: ['[{"foo":2, "bar":1}, {"foo":3, "bar":100}, {"foo":4, "bar":10}]']

- title: "`group_by`"
- title: "`group`, `group_by`"
body: |

`group_by(.foo)` takes as input an array, groups the
`group(.foo)` takes as input an array, groups the
elements having the same `.foo` field into separate arrays,
and produces all of these arrays as elements of a larger
array, sorted by the value of the `.foo` field.

Any jq expression, not just a field access, may be used in
place of `.foo`. The sorting order is the same as described
in the `sort` function above.

For legacy reasons, `group_by(.foo)` exists as an alias for `group(.foo)`.
This alias is considered deprecated and will be dropped in the
next major release.

examples:
- program: 'group_by(.foo)'
- program: 'group(.foo)'
input: '[{"foo":1, "bar":10}, {"foo":3, "bar":100}, {"foo":1, "bar":1}]'
output: ['[[{"foo":1, "bar":10}, {"foo":1, "bar":1}], [{"foo":3, "bar":100}]]']

- title: "`min`, `max`, `min_by`, `max_by`"
body: |

Find the minimum or maximum element of the input array. The
`_by` versions allow you to specify a particular field or
property to examine, e.g. `min_by(.foo)` finds the object
Find the minimum or maximum element of the input array.
This filter accepts an optional argument that
allows you to specify a particular field or
property to examine, e.g. `min(.foo)` finds the object
with the smallest `foo` field.

For legacy reasons, `min_by(.foo)` and `max_by(.foo)` exist as aliases
for `min(.foo)` and `max(.foo)`. These aliases are considered deprecated
and will be dropped in the next major release.

examples:
- program: 'min'
input: '[5,4,2,7]'
output: ['2']
- program: 'max_by(.foo)'
- program: 'max(.foo)'
input: '[{"foo":1, "bar":14}, {"foo":2, "bar":3}]'
output: ['{"foo":2, "bar":3}']

- title: "`unique`"
- title: "`unique`, `unique_by`"
body: |

The `unique` function takes as input an array and produces
an array of the same elements, in sorted order, with
duplicates removed.
duplicates removed. If an optional argument is passed, it
will keep only one element for each value obtained by applying
the argument. Think of it as making an array by taking one
element out of every group produced by `group`.

For legacy reasons, `unique_by(.foo)` exists as an alias for
`unique(.foo)`. This alias is considered deprecated and will be
dropped in the next major release.

examples:
- program: 'unique'
input: '[1,2,5,3,5,3,1,3]'
output: ['[1,2,3,5]']

- title: "`unique_by`"
body: |

The `unique_by(.foo)` function takes as input an array and produces
an array of the same elements, in sorted order, with
elqements with a duplicate `.foo` field removed. Think of it as making
an array by taking one element out of every group produced by
`group_by`.

examples:
- program: 'unique_by(.foo)'
- program: 'unique(.foo)'
input: '[{"foo": 1, "bar": 2}, {"foo": 1, "bar": 3}, {"foo": 4, "bar": 5}]'
output: ['[{"foo": 1, "bar": 2}, {"foo": 4, "bar": 5}]']
- program: 'unique_by(length)'
- program: 'unique(length)'
input: '["chunky", "bacon", "kitten", "cicada", "asparagus"]'
output: ['["chunky", "bacon", "asparagus"]']


- title: "`reverse`"
body: |

Expand Down Expand Up @@ -1144,7 +1153,7 @@ sections:
output: ['"a, b,c,d, e"']


- title: "`recurse`"
- title: "`recurse`, `recurse_down`"
body: |

The `recurse` function allows you to search through a
Expand All @@ -1165,6 +1174,13 @@ sections:
with:

recurse(.children[]) | .name

When called without arguments, `recurse` is equivalent to
`recurse(.[]?)`.

For legacy reasons, `recurse_down` exists as an alias to
calling `recurse` without arguments. This alias is considered
deprecated and will be dropped in the next major release.

examples:
- program: 'recurse(.foo[])'
Expand All @@ -1175,17 +1191,10 @@ sections:
- '{"foo":[{"foo":[]}]}'
- '{"foo":[]}'

- title: "`recurse_down`"
body: |

A quieter version of `recurse(.[])`, equivalent to:

def recurse_down: recurse(.[]?);

- title: "`..`"
body: |

Short-hand for `recurse_down`. This is intended to resemble
Short-hand for `recurse` without arguments. This is intended to resemble
the XPath `//` operator. Note that `..a` does not work; use
`..|a` instead.

Expand Down
2 changes: 1 addition & 1 deletion parser.y
Original file line number Diff line number Diff line change
Expand Up @@ -477,7 +477,7 @@ Term:
$$ = gen_noop();
} |
REC {
$$ = gen_call("recurse_down", gen_noop());
$$ = gen_call("recurse", gen_noop());
} |
Term FIELD '?' {
$$ = gen_index_opt($1, gen_const($2));
Expand Down