Skip to content

Commit

Permalink
test: Test json in stream maps (#283)
Browse files Browse the repository at this point in the history
  • Loading branch information
edgarrmondragon authored Jul 30, 2024
1 parent ea0dfc1 commit 9dd0442
Show file tree
Hide file tree
Showing 4 changed files with 224 additions and 351 deletions.
4 changes: 4 additions & 0 deletions .github/workflows/meltano-run.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ jobs:
mapping: whitelist
output_db: tap_smoke_test.db
query: "select * from animals limit 20"
- tap: tap-smoke-test
mapping: json_field
output_db: tap_smoke_test.db
query: "select * from animals limit 20"
- tap: people
mapping: flatten
output_db: people.db
Expand Down
9 changes: 9 additions & 0 deletions meltano.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,15 @@ plugins:
id: id
description: description
__else__: __NULL__
# Put some fields into a JSON object
- name: json_field
config:
stream_maps:
# JSON-ify the some fields
animals:
id: id
data: "json.dumps({'description': description, 'verified': verified, 'views': views, 'created_at': created_at})"
__else__: null
- name: flatten
config:
stream_maps: {}
Expand Down
Loading

0 comments on commit 9dd0442

Please sign in to comment.