From 9b19d3b42db20c8f51d7043eb91d7a93cfd2e86b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edgar=20Ram=C3=ADrez=20Mondrag=C3=B3n?= Date: Wed, 11 Oct 2023 14:25:53 -0600 Subject: [PATCH 1/2] docs: Document support for comprehenions in stream maps --- docs/stream_maps.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/docs/stream_maps.md b/docs/stream_maps.md index e4119d640..4634e3061 100644 --- a/docs/stream_maps.md +++ b/docs/stream_maps.md @@ -155,6 +155,34 @@ Expressions are defined and parsed using the accepts most native python expressions and is extended by custom functions which have been declared within the SDK. +#### Compound Expressions + +Starting in version 0.33.0, the SDK supports the use of simple comprehensions, e.g. `[x + 1 for x in [1,2,3]]`. This is a powerful feature which allows you to perform complex transformations on lists of values. For example, you can use comprehensions to filter out values in an array: + +````{tab} meltano.yml +```yaml +stream_maps: + users: + id: id + fields: "[f for f in fields if f['key'] != 'age']" +``` +```` + +````{tab} JSON +```json +{ + "stream_maps": { + "users": { + "id": "id", + "fields": "[f for f in fields if f['key'] != 'age']" + } + } +} +``` +```` + +2007-docs-document-using-comprehensions-in-stream-maps-expressions + ### Accessing Stream Properties within Mapping Expressions By default, all stream properties are made available via the property's given name. For From 5ab2615740e240926472f7291502899001bb2c14 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edgar=20Ram=C3=ADrez=20Mondrag=C3=B3n?= Date: Wed, 11 Oct 2023 14:37:10 -0600 Subject: [PATCH 2/2] Clean up --- docs/stream_maps.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/stream_maps.md b/docs/stream_maps.md index 4634e3061..c8d1e8cc9 100644 --- a/docs/stream_maps.md +++ b/docs/stream_maps.md @@ -181,8 +181,6 @@ stream_maps: ``` ```` -2007-docs-document-using-comprehensions-in-stream-maps-expressions - ### Accessing Stream Properties within Mapping Expressions By default, all stream properties are made available via the property's given name. For