Dvclive api reference (#2632)

* Add api-reference index and init

* Update installation instructions

* Added log

* Update log signature

* Add info about slashes

* Added next_step

* Fixed links

* Add links to dvclive reference

* Update content/docs/dvclive/api-reference/init.md

Co-authored-by: Dave Berenbaum <[email protected]>

* Update content/docs/dvclive/api-reference/next_step.md

Co-authored-by: Dave Berenbaum <[email protected]>

* Rename *plot metrics* -> *metrics logs* and *metrics summary* to

* Rename *plot metrics* -> *metrics logs*

* Remove return type

* Update next_step

* Add link from `step` to `log`

* Applied suggestions

* Add links to better explain *metrics logs*, *summary* and other concepts

Co-authored-by: Dave Berenbaum <[email protected]>

content/docs/dvclive/api-reference/index.md

@@ -0,0 +1,14 @@
+# API Reference
+
+[DVCLive](https://github.com/iterative/dvclive) is a Python library, which can
+be installed with: `pip install dvclive`.
+
+This reference provides the details about the functions in the API module
+`dvclive`, which can be imported any regular way, for example:
+
+```py
+import dvclive
+```
+
+Please choose a function from the navigation sidebar to the left, or click the
+Next button below to jump into the first one ↘

content/docs/dvclive/api-reference/init.md

@@ -0,0 +1,63 @@
+# dvclive.init()
+
+Initializes a DVCLive logger.
+
+```py
+def init(
+    path: str = None,
+    resume: bool = False,
+    step: int = 0,
+    summary: bool = True,
+    html: bool = True,
+)
+```
+
+#### Usage:
+
+```py
+import dvclive
+
+dvclive.init()
+```
+
+## Description
+
+It's usage is optional and focused on configuring the behavior of subsequent
+calls to [`dvclive.log`](/doc/dvclive/api-reference/log) and
+[`dvclive.next_step`](/doc/dvclive/api-reference/next_step).
+
+⚠️ If `path` already exists when this functions is called, it's contents will be
+removed.
+
+## Parameters
+
+- `path` (`dvclive` by default) - The _metrics logs_ (generated by
+  [`dvclive.log`](/doc/dvclive/api-reference/log) and usable by `dvc plots`)
+  will be saved in separated `.tsv` files under `path` and the _metrics summary_
+  (generated by [`dvclive.next_step`](/doc/dvclive/api-reference/next_step) and
+  usable by `dvc metrics`) will be saved in a single `{path}.json` file.
+
+- `step` (`0` by default) - the `step` values in _metrics logs_ files will start
+  incrementing from this value. Check
+  [`dvclive.log` description](/doc/dvclive/api-reference/log#description) for
+  more details.
+
+- `resume` - (`False` by default) - if `True`, DVCLive will try to read the
+  previous `step` from the `path` directory and start from that point (unless a
+  `step` is passed explicitly).
+  [`dvclive.next_step`](/doc/dvclive/api-reference/next_step) calls will
+  increment the `step`.
+
+- `summary` (`True` by default) - if `True`, upon each
+  [`dvclive.next_step`](/doc/dvclive/api-reference/next_step) call, DVCLive will
+  generate a _metrics summary_ (usable by `dvc metrics`). The _summary_ will be
+  located at `{path}.json`.
+
+- `html` (`True` by default) - works only when DVCLive is used alongside DVC. If
+  `True`, upon each [`dvclive.next_step`](/doc/dvclive/api-reference/next_step)
+  call, DVC will prepare an _html report_ with all the _metrics logs_ logged in
+  `path`. The _html report_ will be located at `{path}.html`.
+
+## Exceptions
+
+- `dvclive.error.DvcLiveError` - If the directory `path` can't be created.

content/docs/dvclive/api-reference/log.md

@@ -0,0 +1,55 @@
+# dvclive.log()
+
+Generates _metrics logs_ (usable by `dvc plots`) by saving the given `name`:
+`val` pair to a `.tsv` file.
+
+```py
+ def log(name: str, val: float, step: int = None):
+```
+
+#### Usage:
+
+```py
+import dvclive
+
+dvclive.log("loss", 0.9)
+```
+
+## Description
+
+The first call to `dvclive.log(name, val)` will create a new file in
+`{path}/{name}.tsv` including the header and first row.
+
+For example `dvclive.log("loss", 0.9)` will create `{path}/loss.tsv`:
+
+```
+timestamp step  loss
+1623671484747 0 0.9
+```
+
+Each subsequent call to `dvclive.log(name, val)` will add a new row to
+`{path}/{name}.tsv`.
+
+The created file `{path}/{name}.tsv` is usable by `dvc plots`.
+
+💡 If `name` contains slashes (i.e. `train/loss`), the required subfolders will
+be created and the file will be saved inside the last subfolder (i.e.
+`{path}/train/loss.tsv`).
+
+💡 If you call `dvclive.log` without calling
+[`dvclive.init`](/doc/dvclive/api-reference/init) first, `dvclive` will
+automatically initialize itself using either default values or environment
+variables (when used alongside `DVC`).
+
+## Parameters
+
+- `name` - The _metrics logs_ will be saved in `{path}/{name}.tsv`.
+
+- `val` - The value to be added in the `name` column of a new row.
+
+- `step` (`None` by default) - The value to be added in the `step` column of a
+  new row. If `None`, the value of the internal `_step` property will be used.
+
+## Exceptions
+
+- `dvclive.error.DvcLiveError` - If the provided `val` has not supported type.

content/docs/dvclive/api-reference/next_step.md

@@ -0,0 +1,52 @@
+# dvclive.next_step()
+
+Signals that the current step has ended. Check the
+[init parameters](/doc/dvclive/api-reference/init#parameters) for configuring
+the behavior.
+
+```py
+def next_step()
+```
+
+#### Usage:
+
+```py
+import dvclive
+
+for step in range(3):
+    dvclive.log("metric", 0.9)
+    dvclive.next_step()
+```
+
+## Description
+
+Each call to `dvclive.next_step` will behave depending on the selected
+[init parameters](/doc/dvclive/api-reference/init#parameters) and whether `DVC`
+is available or not.
+
+If `summary` is True, on each `dvclive.next_step()` call, `dvclive` will
+generate a _summary_ of the metrics previously logged with `dvclive.log` and
+increase the `_step` count.
+
+The _metrics summary_ (usable by `dvc metrics`) will be saved to `{path}.json`,
+being `path` the one defined in [dvclive.init](/doc/dvclive/api-reference/init).
+
+The resulting _summary_ of the above code block would be:
+
+```json
+{
+  "step": 2,
+  "metric": 0.9
+}
+```
+
+If `dvclive` is used alongside `DVC`, on each `dvclive.next_step()`, `dvclive`
+will create a [checkpoint](/doc/user-guide/experiment-management/checkpoints).
+In addition, if `html` is True, on each `dvclive.next_step()` call, `DVC` will
+prepare an _html report_ with all the _metrics logs_ logged in `path`.
+
+## Exceptions
+
+- `dvclive.error.InitializationError` - If `dvclive` has not been properly
+  initialized (i.e. by calling [dvclive.init](/doc/dvclive/api-reference/init)
+  or [dvclive.log](/doc/dvclive/api-reference/log)).

content/docs/sidebar.json

@@ -582,6 +582,25 @@
       {
         "label": "Dvclive with DVC",
         "slug": "dvclive-with-dvc"
+      },
+      {
+        "label": "API Reference",
+        "slug": "api-reference",
+        "source": "api-reference/index.md",
+        "children": [
+          {
+            "slug": "init",
+            "label": "init()"
+          },
+          {
+            "slug": "log",
+            "label": "log()"
+          },
+          {
+            "slug": "next_step",
+            "label": "next_step()"
+          }
+        ]
       }
     ]
   }

content/docs/user-guide/experiment-management/checkpoints.md

@@ -185,13 +185,14 @@ for i in range(1, EPOCHS+1):
 The line `torch.save(model.state_dict(), "model.pt")` updates the checkpoint
 file.
 
-The `dvclive.log(k, v)` line stores the metric _k_ with a value _v_ in plain
-text files in the _dvclive_ directory by default.
-
-The `dvclive.next_step()` line tells DVC that it can take a snapshot of the
-entire workspace and version it with Git. It's important that with this approach
-only code with metadata is versioned in Git (as an ephemeral commit), while the
-actual model weight file will be stored in the DVC data cache.
+You can read about what the line `dvclive.log(k, v)` does in the
+[`dvclive.log`](/doc/dvclive/api-reference/log) reference.
+
+The [`dvclive.next_step()`](/doc/dvclive/api-reference/next_step) line tells DVC
+that it can take a snapshot of the entire workspace and version it with Git.
+It's important that with this approach only code with metadata is versioned in
+Git (as an ephemeral commit), while the actual model weight file will be stored
+in the DVC data cache.
 
 ## Running experiments