templating: Add `dict unpacking` section. #3730

daavoo · 2022-07-05T08:34:06Z

config: Add parsing section.

config: Add `parsing` section. Per iterative/dvc#7907

daavoo · 2022-07-05T08:35:55Z

content/docs/user-guide/project-structure/pipelines-files.md

@@ -156,6 +160,49 @@ stages:
 DVC will track simple param values (numbers, strings, etc.) used in `${}` (they
 will be listed by `dvc params diff`).

+### Dict Unpacking


This file was already too large but I think the extraction (#3670) should be addressed in a separate P.R.

dberenbaum · 2022-07-05T18:29:34Z

content/docs/user-guide/project-structure/pipelines-files.md

@@ -156,6 +160,49 @@ stages:
 DVC will track simple param values (numbers, strings, etc.) used in `${}` (they
 will be listed by `dvc params diff`).

+### Dict Unpacking
+
+Only inside the `cmd` entries, you can also reference a dictionary inside `${}`


I forget what happens if this is tried outside of cmd?

Something like:

ResolveError("failed to parse 'stages.build.outs' in 'dvc.yaml':\nCannot interpolate data of type 'dict'")

jorgeorpinel

This file was already too large but I think the extraction (#3670) should be addressed in a separate P.R.

Agree, still this probably doesn't need an H3 sub-section, we could summarize more.

Some edits base on the existing text for now:

content/docs/user-guide/project-structure/pipelines-files.md

jorgeorpinel

On the config parsing ref...

p.s. #340 becomes even more urgent...

jorgeorpinel · 2022-07-09T05:13:50Z

content/docs/command-reference/config.md

+- `parsing.bool` - Controls the templating syntax for boolean values when used
+  in
+  [dict unpacking](/doc/user-guide/project-structure/pipelines-files#dict-unpacking).


It's also for lists though, right @daavoo ? Any other types?

content/docs/command-reference/config.md

jorgeorpinel · 2022-07-09T05:18:15Z

content/docs/command-reference/config.md

+  Given the following `params.yaml`:
+
+  ```yaml
+  dict:
+    bool-true: true


Everything else should probably be a separate Example, not part of the parsing.bool bullet.

jorgeorpinel · 2022-07-09T05:18:46Z

content/docs/command-reference/config.md

+- `parsing.list` - Controls the templating syntax for list values when used in
+  [dict unpacking](/doc/user-guide/project-structure/pipelines-files#dict-unpacking).
+
+  Valid values are `"nargs"` (default) and `"append"`, named after
+  [Python argparse actions](https://docs.python.org/3/library/argparse.html#action).
+
+  Given the following `params.yaml`:
+
+  ```yaml


Same comments for the parsing.list bullet

dberenbaum · 2022-07-21T14:35:23Z

Ping @daavoo . Could you take a look and address the comments from @jorgeorpinel in a new PR?

jorgeorpinel · 2022-07-21T15:30:15Z

Oh thanks for the follow-up @dberenbaum . @daavoo note that I'm touching the Dictionary unpacking section in #3789 (feel free to contribute there or wait after it's merged).

rel. #3730 (review)

per #3730 (review)

jorgeorpinel · 2022-08-01T18:40:42Z

@daavoo I started addressing my comments above in #3789, including the ones on the cmd ref before I realized that part should be a different PR. So I started #3858 for that and assigned you. Thanks 🙂

* initial plan and some content * added content about stages * title and restyle fixes * added dag section * depend to -> depend on * added dependencies section * Update content/docs/user-guide/pipelines/index.md * Update content/docs/user-guide/pipelines/index.md * Restyled by prettier (#3532) Co-authored-by: Restyled.io <[email protected]> * Update content/docs/user-guide/pipelines/index.md Co-authored-by: Jorge Orpinel <[email protected]> * added pipelines to sidebar * updated the title * fixed formatting * updating for dvc.yaml first * fed -> used * dvc.yaml-first * editing to tell dvc.yaml first * minor fix * url dependency * dvc lock example * section titles for deps * section titles for outputs * reproduction -> running * adding hyperparameters section * added experiments section * adding url dependencies * added outputs section content * minor * added running pipelines content * moved outputs below running * removed plots section header * guide: Defining Data Pipelines * guide: split up Data Pipelines section * Update content/docs/command-reference/plots/templates.md * guide: Data Pipes -> ML Pipes per #3414 (review) * guide: oops, remove op-pipes file * guide: remoge ML Pipes intro per #3414 (review) * guide: mention both imports in Def ML Pipes rel #3414 (review) * guide: move DAG info from cmd ref per #3414 (review) * guide: move all info and links about DAG to ML Pipes section about Dependency Graphs * guide: point from some Stage links to ML Pipes section on Stages (Defining Pipes) * guide: delete Running ML Pipes (for now) * nav: remove future ML Pipes guides * guide: remove ML Pipes/ Experimental Pipes * roll back unrelated changes... * guide: roll back dvc.yaml page changes per #3414 (review) * guide: link ML Pipes/ Defining Stages to dvc.yaml/stages spec * guide: some changes to start improving the dvc.yaml guide * guide: edit Stages section (dvc.yaml guide) and and move Stage entry spec to right after that. * guide: link deps and params tooltip to ML Pipes/ Stages guide sections * guide: links from dvc.yaml doc to ML Pipes/ Stages * guide: update dvc.yaml Templating spec * guide: a couple more admons for dvc.yaml doc * guide: more links * guide: oops, remove unused files * remove unrelated change * ref: update stage add/ run Descs * guide: move stage definition details to ML Pipes from cmd ref * guide: move stage definition details to ML Pipes from cmd ref * guide: move stage command details into ML Pipes re-link from existing places * guide; update Stage entries spec descs. * guide: admons for dvc.yaml page * guide: roll back wrong change * edits to dvc.yaml doc per #3730 (review) * ref: roll back unrelated changes * . * ref: few more links to dependency graph in guide * ref: refactor run, stage add, and repro a little * guide: drop old pipelines guide * ref: revert files one change moved to #4024 * pipelines: revert a bunch of files (for now) per #3789 (comment) * proper admons Co-authored-by: iterative <[email protected]> Co-authored-by: Emre Şahin <[email protected]> Co-authored-by: restyled-io[bot] <32688539+restyled-io[bot]@users.noreply.github.com> Co-authored-by: Restyled.io <[email protected]>

templating: Add dict unpacking section.

a795da5

config: Add `parsing` section. Per iterative/dvc#7907

shcheklein temporarily deployed to dvc-org-dict-unpacking-wbpep68 July 5, 2022 08:34 Inactive

daavoo self-assigned this Jul 5, 2022

daavoo requested review from a team and dberenbaum July 5, 2022 08:34

daavoo added the C: guide Content of /doc/user-guide label Jul 5, 2022

daavoo commented Jul 5, 2022

View reviewed changes

dberenbaum reviewed Jul 5, 2022

View reviewed changes

dberenbaum approved these changes Jul 5, 2022

View reviewed changes

daavoo merged commit aee09a8 into main Jul 6, 2022

daavoo deleted the dict-unpacking branch July 6, 2022 13:20

jorgeorpinel reviewed Jul 9, 2022

View reviewed changes

content/docs/user-guide/project-structure/pipelines-files.md Show resolved Hide resolved

jorgeorpinel reviewed Jul 9, 2022

View reviewed changes

jorgeorpinel added A: docs Area: user documentation (gatsby-theme-iterative) C: ref Content of /doc/*-reference labels Jul 9, 2022

jorgeorpinel added a commit that referenced this pull request Aug 1, 2022

ref: config edits

5f60436

rel. #3730 (review)

jorgeorpinel mentioned this pull request Aug 1, 2022

ref: config edits (around dictionary unpacking) #3858

Closed

1 task

jorgeorpinel added a commit that referenced this pull request Aug 1, 2022

edits to dvc.yaml doc

fde5997

per #3730 (review)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

templating: Add `dict unpacking` section. #3730

templating: Add `dict unpacking` section. #3730

daavoo commented Jul 5, 2022 •

edited

Loading

daavoo Jul 5, 2022

dberenbaum Jul 5, 2022

daavoo Jul 6, 2022 •

edited

Loading

jorgeorpinel left a comment •

edited

Loading

jorgeorpinel left a comment •

edited

Loading

This comment was marked as resolved.

jorgeorpinel Jul 9, 2022 •

edited

Loading

jorgeorpinel Jul 9, 2022

jorgeorpinel Jul 9, 2022

dberenbaum commented Jul 21, 2022

jorgeorpinel commented Jul 21, 2022 •

edited

Loading

jorgeorpinel commented Aug 1, 2022

templating: Add dict unpacking section. #3730

templating: Add dict unpacking section. #3730

Conversation

daavoo commented Jul 5, 2022 • edited Loading

daavoo Jul 5, 2022

Choose a reason for hiding this comment

dberenbaum Jul 5, 2022

Choose a reason for hiding this comment

daavoo Jul 6, 2022 • edited Loading

Choose a reason for hiding this comment

jorgeorpinel left a comment • edited Loading

Choose a reason for hiding this comment

jorgeorpinel left a comment • edited Loading

Choose a reason for hiding this comment

This comment was marked as resolved.

jorgeorpinel Jul 9, 2022 • edited Loading

Choose a reason for hiding this comment

jorgeorpinel Jul 9, 2022

Choose a reason for hiding this comment

jorgeorpinel Jul 9, 2022

Choose a reason for hiding this comment

dberenbaum commented Jul 21, 2022

jorgeorpinel commented Jul 21, 2022 • edited Loading

jorgeorpinel commented Aug 1, 2022

templating: Add `dict unpacking` section. #3730

templating: Add `dict unpacking` section. #3730

daavoo commented Jul 5, 2022 •

edited

Loading

daavoo Jul 6, 2022 •

edited

Loading

jorgeorpinel left a comment •

edited

Loading

jorgeorpinel left a comment •

edited

Loading

jorgeorpinel Jul 9, 2022 •

edited

Loading

jorgeorpinel commented Jul 21, 2022 •

edited

Loading