Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

codecov is broken #113

Open
ed1d1a8d opened this issue Apr 15, 2023 · 0 comments
Open

codecov is broken #113

ed1d1a8d opened this issue Apr 15, 2023 · 0 comments

Comments

@ed1d1a8d
Copy link
Collaborator

See https://about.codecov.io/blog/message-regarding-the-pypi-package/

ed1d1a8d added a commit that referenced this issue Apr 19, 2023

Verified

This commit was signed with the committer’s verified signature.
mapno Mario
1. Adds a new script `scripts/dl_katago_networks.py` which downloads all the KataGo networks to `/nas/ucb/k8/go-attack/katago-networks`. The script will detect if a network has already been downloaded and skips downloading in this case.
2. Add a new eval to `scripts/generate_paper_evaluations.py` which runs the an adversary against the latest KataGo networks. The adversary as well as the earliest KataGo network to evaluate against in configurable in `scripts/paper_evaluations.yml`. The new eval can be configured to either run on a CHAI machine or Hofvarpnir.
3. Cleans up the `scripts/generate_paper_evaluations.py` file to remove the need to launch a devbox to generate configs. This is slow and also unnecessary (one can instead generate configs on either a CHAI machine or Hofvarpnir by setting up a python environment using something like micromamba).
4. Remove the `codecov` dependency to make the CI work. See #113.

See b15a92b for an example of the actual eval configs that were generated.

See https://www.notion.so/chaiberkeley/Evals-against-latest-KataGo-checkpoints-1d11bd9115c347c49e21939eda833ba2?pvs=4 for experiments that use this new eval script.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant