Skip to content
This repository has been archived by the owner on Oct 16, 2024. It is now read-only.

ref: doctor + #259

Merged
merged 7 commits into from
Dec 22, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions content/docs/gto/command-reference/annotate.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,17 @@ For example,
$ gto annotate awesome-model \
--type model \
--path models/neural_network.h5 \
--label ml
--label cool
--description "This model is very cool"
--label ml \
--label cool \
--description "This model is very cool" \
--custom "{'foo': 'bar'}"
```

will create

```yaml
awesome-model:
custom: "{'foo': 'bar'}"
description: This model is very cool
labels:
- ml
Expand All @@ -44,6 +46,11 @@ awesome-model:

This information can be later retrieved by running `gto describe` command.

`--custom` option allows you to add custom metadata. Within `gto annotate` GTO
treats input as string, but you can edit artifacts.yaml to add arbitrary
information that fits yaml format, such as dict or list, or more complex
structures.
Comment on lines +49 to +52
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't seem important enough to be the only option explained directly in the Description. Probably adding to the Options section is enough?


<admon type="tip">

Don't forget to commit `artifacts.yaml` with Git to associate it with the latest
Expand Down Expand Up @@ -79,6 +86,7 @@ GTO the artifact file is committed to Git.
- `-e`, `--must-exist` - Verify artifact is committed to Git
- `--label <text>` - Labels to add to artifact
- `-d <text>`, `--description <text>` - Artifact description
- `-c <text>`, `--custom <text>` - Custom metadata to add to artifact
- `--commit` - Automatically commit changes due to this command (experimental)
- `--push` - Push created commit automatically (experimental) - will set
commit=True
Expand Down
16 changes: 15 additions & 1 deletion content/docs/gto/command-reference/describe.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,21 @@ $ gto describe churn -r https://github.com/iterative/example-gto
}
```

The output is in JSON format for ease of parsing programatically.
The output is in JSON format for ease of parsing programmatically.

Note, that for local repos the `artifacts.yaml` is read from the workspace
without Git, so if you have uncommitted changes, they will be reflected in the
output. If you want to read from specific commit, you need to specify `--rev`
option.

You can also get annotation for specific versions (these are the same shortcuts
Comment on lines +31 to +36
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first part is expected, I think. We can simplify this

Suggested change
Note, that for local repos the `artifacts.yaml` is read from the workspace
without Git, so if you have uncommitted changes, they will be reflected in the
output. If you want to read from specific commit, you need to specify `--rev`
option.
You can also get annotation for specific versions (these are the same shortcuts
If you want to read information from previous commits, specify a `--rev`.
You can also get annotation for specific artifact versions (same shortcuts

as in `gto show`):

```cli
$ gto describe churn@latest # highest version by SemVer
$ gto describe churn#dev # version in stage `dev`
$ gto describe [email protected] # version `v3.0.0`
```

## Options

Expand Down
36 changes: 36 additions & 0 deletions content/docs/gto/command-reference/doctor.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# describe

Display GTO version and check the registry for problems.

## Synopsis

```usage
usage: gto doctor [-r <text>] [-A] [-h]
```

## Description

This will check the registry and print all the issues if found:
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved

```cli
$ gto doctor
🪴 GTO Version: 0.2.5
---------------------------------
INDEX='artifacts.yaml'
TYPES=None
STAGES=None
LOG_LEVEL='INFO'
DEBUG=False
ENRICHMENTS=[]
AUTOLOAD_ENRICHMENTS=True
CONFIG_FILE_NAME='.gto'
EMOJIS=True
---------------------------------
✅ No issues found
```

## Options

- `-r <text>`, `--repo <text>` - Local or remote repository [default: .]
- `-A`, `--all-commits` - Read all commits
- `-h`, `--help` - Show this message and exit.
7 changes: 7 additions & 0 deletions content/docs/gto/command-reference/show.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,13 @@ $ gto show churn#prod
╞════════════╪═══════════╪═════════╪═════════════════════╪══════════════╡
│ churn │ v3.0.0 │ prod │ 2022-10-28 23:53:53 │ [email protected]
╘════════════╧═══════════╧═════════╧═════════════════════╧══════════════╛

$ gto show [email protected]
╒════════════╤═══════════╤═════════╤═════════════════════╤══════════════╕
│ artifact │ version │ stage │ created_at │ ref │
╞════════════╪═══════════╪═════════╪═════════════════════╪══════════════╡
│ churn │ v3.0.0 │ prod │ 2022-10-28 23:53:53 │ [email protected]
╘════════════╧═══════════╧═════════╧═════════════════════╧══════════════╛
```

By default, GTO is configured to adhere to what we call "Environments"
Expand Down
6 changes: 3 additions & 3 deletions content/docs/gto/user-guide/dvc.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,9 +87,9 @@ learn how to find out `ARTIFACT_PATH` and `REVISION`.
If you need to download the latest version of `model`, that would be:

```cli
$ REVISION=$(gto show --repo $REPO model@greatest --ref)
$ ARTIFACT_PATH=$(gto describe --repo $REPO $ARTIFACT --rev $REVISION --path)
$ dvc get $REPO $ARTIFACT --rev $REVISION -o $ARTIFACT_PATH
$ ARTIFACT_PATH=$(gto describe --repo $REPO model@latest --path)
$ REVISION=$(gto show --repo $REPO model@latest --ref)
$ dvc get $REPO $ARTIFACT_PATH --rev $REVISION -o $ARTIFACT_PATH
```

</details>
Expand Down
2 changes: 1 addition & 1 deletion content/docs/gto/user-guide/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ You may need the artifact's file path. If
`gto describe`:

```cli
$ gto describe churn --rev churn@v3.0.0 --path
$ gto describe [email protected] --path
models/churn.pkl
```

Expand Down
12 changes: 8 additions & 4 deletions content/docs/gto/user-guide/mlem.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,12 @@ jobs:
name: Build a Docker image for new model versions
runs-on: ubuntu-latest
steps:
- name: Login to Docker Hub
uses: docker/login-action@v2
# set credentials to login to DockerHub
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- uses: actions/checkout@v3
- id: gto
uses: iterative/gto-action@v1
Expand All @@ -87,13 +93,11 @@ jobs:
pip install -r requirements.txt
- if: steps.gto.outputs.event == 'registration'
run: |
# TODO: check this works
# What credentials we need to specify to publish image somewhere?
mlem build docker \
--model '${{ steps.gto.outputs.path }}' \
--image.name ${{ steps.gto.outputs.name }} \
--image.tag '${{ steps.gto.outputs.version }}' \
--env.registry docker_io
--image.registry docker_io
```

Note that builder can be
Expand All @@ -112,7 +116,7 @@ on:
tags:
- '*'

# specify credentials needed to run deployment and keep the deployment state
# set credentials to run deployment and save its state to s3
env:
HEROKU_API_KEY: ${{ secrets.HEROKU_API_KEY }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
Expand Down
10 changes: 10 additions & 0 deletions content/docs/sidebar.json
Original file line number Diff line number Diff line change
Expand Up @@ -623,6 +623,11 @@
"slug": "dvc",
"label": "GTO with DVC",
"source": "dvc.md"
},
{
"slug": "mlem",
"label": "GTO with MLEM",
"source": "mlem.md"
}
]
},
Expand Down Expand Up @@ -661,6 +666,11 @@
"source": "describe.md",
"label": "describe"
},
{
"slug": "doctor",
"source": "doctor.md",
"label": "doctor"
Comment on lines +669 to +672
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cmd ref changes are completely unrelated, right? Is the generator script running automatically whenever you change anything? Otherwise 🙏🏼 let's make separate PRs for ease of review in the future. Thanks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, they are unrelated. It doesn't run automatically, I do that manually. But, I can do that in separate PRs in future, sure.

Copy link
Contributor

@jorgeorpinel jorgeorpinel Dec 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Actually this PR was mostly about that now that I see haha. So I just changed the title 😬 and labels

},
{
"slug": "history",
"source": "history.md",
Expand Down