diff --git a/notebooks/bonus-unit1/bonus-unit1.ipynb b/notebooks/bonus-unit1/bonus-unit1.ipynb index 8a95f989..93db85ad 100644 --- a/notebooks/bonus-unit1/bonus-unit1.ipynb +++ b/notebooks/bonus-unit1/bonus-unit1.ipynb @@ -3,8 +3,8 @@ { "cell_type": "markdown", "metadata": { - "id": "view-in-github", - "colab_type": "text" + "colab_type": "text", + "id": "view-in-github" }, "source": [ "\"Open" @@ -21,31 +21,34 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "FMYrDriDujzX" + }, "source": [ "\"Bonus\n", "\n", "In this notebook, we'll reinforce what we learned in the first Unit by **teaching Huggy the Dog to fetch the stick and then play with it directly in your browser**\n", "\n", "⬇️ Here is an example of what **you will achieve at the end of the unit.** ⬇️ (launch ▶ to see)" - ], - "metadata": { - "id": "FMYrDriDujzX" - } + ] }, { "cell_type": "code", - "source": [ - "%%html\n", - "" - ], + "execution_count": null, "metadata": { "id": "PnVhs1yYNyUF" }, - "execution_count": null, - "outputs": [] + "outputs": [], + "source": [ + "%%html\n", + "" + ] }, { "cell_type": "markdown", + "metadata": { + "id": "x7oR6R-ZIbeS" + }, "source": [ "### The environment 🎮\n", "\n", @@ -54,22 +57,22 @@ "### The library used 📚\n", "\n", "- [MLAgents](https://github.com/Unity-Technologies/ml-agents)" - ], - "metadata": { - "id": "x7oR6R-ZIbeS" - } + ] }, { "cell_type": "markdown", - "source": [ - "We're constantly trying to improve our tutorials, so **if you find some issues in this notebook**, please [open an issue on the Github Repo](https://github.com/huggingface/deep-rl-class/issues)." - ], "metadata": { "id": "60yACvZwO0Cy" - } + }, + "source": [ + "We're constantly trying to improve our tutorials, so **if you find some issues in this notebook**, please [open an issue on the Github Repo](https://github.com/huggingface/deep-rl-class/issues)." + ] }, { "cell_type": "markdown", + "metadata": { + "id": "Oks-ETYdO2Dc" + }, "source": [ "## Objectives of this notebook 🏆\n", "\n", @@ -80,23 +83,23 @@ "- Be able to play **with your trained Huggy directly in your browser**.\n", "\n", "\n" - ], - "metadata": { - "id": "Oks-ETYdO2Dc" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "mUlVrqnBv2o1" + }, "source": [ "## This notebook is from Deep Reinforcement Learning Course\n", "\"Deep" - ], - "metadata": { - "id": "mUlVrqnBv2o1" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "pAMjaQpHwB_s" + }, "source": [ "In this free course, you will:\n", "\n", @@ -110,13 +113,13 @@ "\n", "\n", "The best way to keep in touch is to join our discord server to exchange with the community and with us 👉🏻 https://discord.gg/ydHrjt3WP5" - ], - "metadata": { - "id": "pAMjaQpHwB_s" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "6r7Hl0uywFSO" + }, "source": [ "## Prerequisites 🏗️\n", "\n", @@ -125,33 +128,30 @@ "🔲 📚 **Develop an understanding of the foundations of Reinforcement learning** (MC, TD, Rewards hypothesis...) by doing Unit 1\n", "\n", "🔲 📚 **Read the introduction to Huggy** by doing Bonus Unit 1" - ], - "metadata": { - "id": "6r7Hl0uywFSO" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "DssdIjk_8vZE" + }, "source": [ "## Set the GPU 💪\n", "- To **accelerate the agent's training, we'll use a GPU**. To do that, go to `Runtime > Change Runtime type`\n", "\n", "\"GPU" - ], - "metadata": { - "id": "DssdIjk_8vZE" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "sTfCXHy68xBv" + }, "source": [ "- `Hardware Accelerator > GPU`\n", "\n", "\"GPU" - ], - "metadata": { - "id": "sTfCXHy68xBv" - } + ] }, { "cell_type": "markdown", @@ -159,7 +159,7 @@ "id": "an3ByrXYQ4iK" }, "source": [ - "## Clone the repository and install the dependencies 🔽\n", + "## Clone the repository 🔽\n", "\n", "- We need to clone the repository, that contains **ML-Agents.**" ] @@ -177,6 +177,77 @@ "!git clone --depth 1 https://github.com/Unity-Technologies/ml-agents" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setup the Virtual Environment 🔽\n", + "- In order for the **ML-Agents** to run successfully in Colab, Colab's Python version must meet the library's Python requirements.\n", + "\n", + "- We can check for the supported Python version under the `python_requires` parameter in the `setup.py` files. These files are required to set up the **ML-Agents** library for use and can be found in the following locations:\n", + " - `/content/ml-agents/ml-agents/setup.py`\n", + " - `/content/ml-agents/ml-agents-envs/setup.py`\n", + "\n", + "- Colab's Current Python version(can be checked using `!python --version`) doesn't match the library's `python_requires` parameter, as a result installation may silently fail and lead to errors like these, when executing the same commands later:\n", + " - `/bin/bash: line 1: mlagents-learn: command not found`\n", + " - `/bin/bash: line 1: mlagents-push-to-hf: command not found`\n", + "\n", + "- To resolve this, we'll create a virtual environment with a Python version compatible with the **ML-Agents** library.\n", + "\n", + "`Note:` *For future compatibility, always check the `python_requires` parameter in the installation files and set your virtual environment to the maximum supported Python version in the given below script if the Colab's Python version is not compatible*" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Colab's Current Python Version (Incompatible with ML-Agents)\n", + "!python --version" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Install virtualenv and create a virtual environment\n", + "!pip install virtualenv\n", + "!virtualenv myenv\n", + "\n", + "# Download and install Miniconda\n", + "!wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh\n", + "!chmod +x Miniconda3-latest-Linux-x86_64.sh\n", + "!./Miniconda3-latest-Linux-x86_64.sh -b -f -p /usr/local\n", + "\n", + "# Activate Miniconda and install Python ver 3.10.12\n", + "!source /usr/local/bin/activate\n", + "!conda install -q -y --prefix /usr/local python=3.10.12 ujson # Specify the version here\n", + "\n", + "# Set environment variables for Python and conda paths\n", + "!export PYTHONPATH=/usr/local/lib/python3.10/site-packages/\n", + "!export CONDA_PREFIX=/usr/local/envs/myenv" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Python Version in New Virtual Environment (Compatible with ML-Agents)\n", + "!python --version" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Installing the dependencies 🔽" + ] + }, { "cell_type": "code", "execution_count": null, @@ -218,23 +289,23 @@ }, { "cell_type": "markdown", - "source": [ - "We downloaded the file Huggy.zip from https://github.com/huggingface/Huggy using `wget`" - ], "metadata": { "id": "IHh_LXsRrrbM" - } + }, + "source": [ + "We downloaded the file Huggy.zip from https://github.com/huggingface/Huggy using `wget`" + ] }, { "cell_type": "code", - "source": [ - "!wget \"https://github.com/huggingface/Huggy/raw/main/Huggy.zip\" -O ./trained-envs-executables/linux/Huggy.zip" - ], + "execution_count": null, "metadata": { "id": "8xNAD1tRpy0_" }, - "execution_count": null, - "outputs": [] + "outputs": [], + "source": [ + "!wget \"https://github.com/huggingface/Huggy/raw/main/Huggy.zip\" -O ./trained-envs-executables/linux/Huggy.zip" + ] }, { "cell_type": "code", @@ -270,6 +341,9 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "dYKVj8yUvj55" + }, "source": [ "## Let's recap how this environment works\n", "\n", @@ -307,13 +381,13 @@ "- *Time penalty*: a fixed-time penalty given at every action to **force him to get to the stick as fast as possible**.\n", "- *Rotation penalty*: we penalize Huggy if **he spins too much and turns too quickly**.\n", "- *Getting to the target reward*: we reward Huggy for **reaching the target**." - ], - "metadata": { - "id": "dYKVj8yUvj55" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "NAuEq32Mwvtz" + }, "source": [ "## Create the Huggy config file\n", "\n", @@ -333,13 +407,15 @@ " \"Create\n", "\n", "- Copy and paste the content below 🔽" - ], - "metadata": { - "id": "NAuEq32Mwvtz" - } + ] }, { "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "loQ0N5jhXW71" + }, + "outputs": [], "source": [ "behaviors:\n", " Huggy:\n", @@ -367,24 +443,22 @@ " max_steps: 2e6\n", " time_horizon: 1000\n", " summary_freq: 50000" - ], - "metadata": { - "id": "loQ0N5jhXW71" - }, - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", - "source": [ - "- Don't forget to save the file!" - ], "metadata": { "id": "oakN7UHwXdCX" - } + }, + "source": [ + "- Don't forget to save the file!" + ] }, { "cell_type": "markdown", + "metadata": { + "id": "r9wv5NYGw-05" + }, "source": [ "- **In the case you want to modify the hyperparameters**, in Google Colab notebook, you can click here to open the config.yaml: `/content/ml-agents/config/ppo/Huggy.yaml`\n", "\n", @@ -394,10 +468,7 @@ "\n", "=> Just keep in mind that **decreasing the `checkpoint_interval` means more models to upload to the Hub and so a longer uploading time**\n", "We’re now ready to train our agent 🔥." - ], - "metadata": { - "id": "r9wv5NYGw-05" - } + ] }, { "cell_type": "markdown", @@ -426,12 +497,12 @@ }, { "cell_type": "markdown", - "source": [ - "The training will take 30 to 45min depending on your machine (don't forget to **set up a GPU**), go take a ☕️you deserve it 🤗." - ], "metadata": { "id": "lN32oWF8zPjs" - } + }, + "source": [ + "The training will take 30 to 45min depending on your machine (don't forget to **set up a GPU**), go take a ☕️you deserve it 🤗." + ] }, { "cell_type": "code", @@ -457,6 +528,9 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "izT6FpgNzZ6R" + }, "source": [ "To be able to share your model with the community there are three more steps to follow:\n", "\n", @@ -469,10 +543,7 @@ "\n", "- Copy the token\n", "- Run the cell below and paste the token" - ], - "metadata": { - "id": "izT6FpgNzZ6R" - } + ] }, { "cell_type": "code", @@ -488,12 +559,12 @@ }, { "cell_type": "markdown", - "source": [ - "If you don't want to use a Google Colab or a Jupyter Notebook, you need to use this command instead: `huggingface-cli login`" - ], "metadata": { "id": "ew59mK19zjtN" - } + }, + "source": [ + "If you don't want to use a Google Colab or a Jupyter Notebook, you need to use this command instead: `huggingface-cli login`" + ] }, { "cell_type": "markdown", @@ -508,6 +579,9 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "KK4fPfnczunT" + }, "source": [ "And we define 4 parameters:\n", "\n", @@ -516,10 +590,7 @@ "3. `--repo-id`: the name of the Hugging Face repo you want to create or update. It’s always /\n", "If the repo does not exist **it will be created automatically**\n", "4. `--commit-message`: since HF repos are git repository you need to define a commit message." - ], - "metadata": { - "id": "KK4fPfnczunT" - } + ] }, { "cell_type": "code", @@ -534,6 +605,9 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "yborB0850FTM" + }, "source": [ "Else, if everything worked you should have this at the end of the process(but with a different url 😆) :\n", "\n", @@ -546,22 +620,22 @@ "It’s the link to your model repository. The repository contains a model card that explains how to use the model, your Tensorboard logs and your config file. **What’s awesome is that it’s a git repository, which means you can have different commits, update your repository with a new push, open Pull Requests, etc.**\n", "\n", "\"ml" - ], - "metadata": { - "id": "yborB0850FTM" - } + ] }, { "cell_type": "markdown", - "source": [ - "But now comes the best: **being able to play with Huggy online 👀.**" - ], "metadata": { "id": "5Uaon2cg0NrL" - } + }, + "source": [ + "But now comes the best: **being able to play with Huggy online 👀.**" + ] }, { "cell_type": "markdown", + "metadata": { + "id": "VMc4oOsE0QiZ" + }, "source": [ "## Play with your Huggy 🐕\n", "\n", @@ -572,30 +646,30 @@ "- Click on Play with my Huggy model\n", "\n", "\"load-huggy\"" - ], - "metadata": { - "id": "VMc4oOsE0QiZ" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "Djs8c5rR0Z8a" + }, "source": [ - "1. In step 1, type your username (your username is case sensitive: for instance, my username is ThomasSimonini not thomassimonini or ThOmasImoNInI) and click on the search button.", + "1. In step 1, type your username (your username is case sensitive: for instance, my username is ThomasSimonini not thomassimonini or ThOmasImoNInI) and click on the search button.\n", "\n", - "2. In step 2, select your model repository.", + "2. In step 2, select your model repository.\n", "\n", - "3. In step 3, **choose which model you want to replay**:", + "3. In step 3, **choose which model you want to replay**:\n", " - I have multiple ones, since we saved a model every 500000 timesteps.\n", " - But since I want the more recent, I choose `Huggy.onnx`\n", "\n", "👉 What’s nice **is to try with different models steps to see the improvement of the agent.**" - ], - "metadata": { - "id": "Djs8c5rR0Z8a" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "PI6dPWmh064H" + }, "source": [ "Congrats on finishing this bonus unit!\n", "\n", @@ -605,18 +679,15 @@ "\n", "\n", "## Keep Learning, Stay awesome 🤗" - ], - "metadata": { - "id": "PI6dPWmh064H" - } + ] } ], "metadata": { "accelerator": "GPU", "colab": { - "provenance": [], + "include_colab_link": true, "private_outputs": true, - "include_colab_link": true + "provenance": [] }, "gpuClass": "standard", "kernelspec": { diff --git a/notebooks/bonus-unit1/bonus_unit1.ipynb b/notebooks/bonus-unit1/bonus_unit1.ipynb index 2b436317..a85452b3 100644 --- a/notebooks/bonus-unit1/bonus_unit1.ipynb +++ b/notebooks/bonus-unit1/bonus_unit1.ipynb @@ -3,8 +3,8 @@ { "cell_type": "markdown", "metadata": { - "id": "view-in-github", - "colab_type": "text" + "colab_type": "text", + "id": "view-in-github" }, "source": [ "\"Open" @@ -21,31 +21,34 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "FMYrDriDujzX" + }, "source": [ "\"Bonus\n", "\n", "In this notebook, we'll reinforce what we learned in the first Unit by **teaching Huggy the Dog to fetch the stick and then play with it directly in your browser**\n", "\n", "⬇️ Here is an example of what **you will achieve at the end of the unit.** ⬇️ (launch ▶ to see)" - ], - "metadata": { - "id": "FMYrDriDujzX" - } + ] }, { "cell_type": "code", - "source": [ - "%%html\n", - "" - ], + "execution_count": null, "metadata": { "id": "PnVhs1yYNyUF" }, - "execution_count": null, - "outputs": [] + "outputs": [], + "source": [ + "%%html\n", + "" + ] }, { "cell_type": "markdown", + "metadata": { + "id": "x7oR6R-ZIbeS" + }, "source": [ "### The environment 🎮\n", "\n", @@ -54,22 +57,22 @@ "### The library used 📚\n", "\n", "- [MLAgents](https://github.com/Unity-Technologies/ml-agents)" - ], - "metadata": { - "id": "x7oR6R-ZIbeS" - } + ] }, { "cell_type": "markdown", - "source": [ - "We're constantly trying to improve our tutorials, so **if you find some issues in this notebook**, please [open an issue on the Github Repo](https://github.com/huggingface/deep-rl-class/issues)." - ], "metadata": { "id": "60yACvZwO0Cy" - } + }, + "source": [ + "We're constantly trying to improve our tutorials, so **if you find some issues in this notebook**, please [open an issue on the Github Repo](https://github.com/huggingface/deep-rl-class/issues)." + ] }, { "cell_type": "markdown", + "metadata": { + "id": "Oks-ETYdO2Dc" + }, "source": [ "## Objectives of this notebook 🏆\n", "\n", @@ -80,23 +83,23 @@ "- Be able to play **with your trained Huggy directly in your browser**.\n", "\n", "\n" - ], - "metadata": { - "id": "Oks-ETYdO2Dc" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "mUlVrqnBv2o1" + }, "source": [ "## This notebook is from Deep Reinforcement Learning Course\n", "\"Deep" - ], - "metadata": { - "id": "mUlVrqnBv2o1" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "pAMjaQpHwB_s" + }, "source": [ "In this free course, you will:\n", "\n", @@ -110,13 +113,13 @@ "\n", "\n", "The best way to keep in touch is to join our discord server to exchange with the community and with us 👉🏻 https://discord.gg/ydHrjt3WP5" - ], - "metadata": { - "id": "pAMjaQpHwB_s" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "6r7Hl0uywFSO" + }, "source": [ "## Prerequisites 🏗️\n", "\n", @@ -125,41 +128,36 @@ "🔲 📚 **Develop an understanding of the foundations of Reinforcement learning** (MC, TD, Rewards hypothesis...) by doing Unit 1\n", "\n", "🔲 📚 **Read the introduction to Huggy** by doing Bonus Unit 1" - ], - "metadata": { - "id": "6r7Hl0uywFSO" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "DssdIjk_8vZE" + }, "source": [ "## Set the GPU 💪\n", "- To **accelerate the agent's training, we'll use a GPU**. To do that, go to `Runtime > Change Runtime type`\n", "\n", "\"GPU" - ], - "metadata": { - "id": "DssdIjk_8vZE" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "sTfCXHy68xBv" + }, "source": [ "- `Hardware Accelerator > GPU`\n", "\n", "\"GPU" - ], - "metadata": { - "id": "sTfCXHy68xBv" - } + ] }, { "cell_type": "markdown", - "metadata": { - "id": "an3ByrXYQ4iK" - }, + "metadata": {}, "source": [ - "## Clone the repository and install the dependencies 🔽\n", + "## Clone the repository 🔽\n", "\n", "- We need to clone the repository, that contains **ML-Agents.**" ] @@ -167,9 +165,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "id": "6WNoL04M7rTa" - }, + "metadata": {}, "outputs": [], "source": [ "%%capture\n", @@ -177,12 +173,81 @@ "!git clone --depth 1 https://github.com/Unity-Technologies/ml-agents" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setup the Virtual Environment 🔽\n", + "- In order for the **ML-Agents** to run successfully in Colab, Colab's Python version must meet the library's Python requirements.\n", + "\n", + "- We can check for the supported Python version under the `python_requires` parameter in the `setup.py` files. These files are required to set up the **ML-Agents** library for use and can be found in the following locations:\n", + " - `/content/ml-agents/ml-agents/setup.py`\n", + " - `/content/ml-agents/ml-agents-envs/setup.py`\n", + "\n", + "- Colab's Current Python version(can be checked using `!python --version`) doesn't match the library's `python_requires` parameter, as a result installation may silently fail and lead to errors like these, when executing the same commands later:\n", + " - `/bin/bash: line 1: mlagents-learn: command not found`\n", + " - `/bin/bash: line 1: mlagents-push-to-hf: command not found`\n", + "\n", + "- To resolve this, we'll create a virtual environment with a Python version compatible with the **ML-Agents** library.\n", + "\n", + "`Note:` *For future compatibility, always check the `python_requires` parameter in the installation files and set your virtual environment to the maximum supported Python version in the given below script if the Colab's Python version is not compatible*" + ] + }, { "cell_type": "code", "execution_count": null, - "metadata": { - "id": "d8wmVcMk7xKo" - }, + "metadata": {}, + "outputs": [], + "source": [ + "# Colab's Current Python Version (Incompatible with ML-Agents)\n", + "!python --version" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Install virtualenv and create a virtual environment\n", + "!pip install virtualenv\n", + "!virtualenv myenv\n", + "\n", + "# Download and install Miniconda\n", + "!wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh\n", + "!chmod +x Miniconda3-latest-Linux-x86_64.sh\n", + "!./Miniconda3-latest-Linux-x86_64.sh -b -f -p /usr/local\n", + "\n", + "# Activate Miniconda and install Python ver 3.10.12\n", + "!source /usr/local/bin/activate\n", + "!conda install -q -y --prefix /usr/local python=3.10.12 ujson # Specify the version here\n", + "\n", + "# Set environment variables for Python and conda paths\n", + "!export PYTHONPATH=/usr/local/lib/python3.10/site-packages/\n", + "!export CONDA_PREFIX=/usr/local/envs/myenv" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Python Version in New Virtual Environment (Compatible with ML-Agents)\n", + "!python --version" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Installing the dependencies 🔽" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, "outputs": [], "source": [ "%%capture\n", @@ -218,23 +283,23 @@ }, { "cell_type": "markdown", - "source": [ - "We downloaded the file Huggy.zip from https://github.com/huggingface/Huggy using `wget`" - ], "metadata": { "id": "IHh_LXsRrrbM" - } + }, + "source": [ + "We downloaded the file Huggy.zip from https://github.com/huggingface/Huggy using `wget`" + ] }, { "cell_type": "code", - "source": [ - "!wget \"https://github.com/huggingface/Huggy/raw/main/Huggy.zip\" -O ./trained-envs-executables/linux/Huggy.zip" - ], + "execution_count": null, "metadata": { "id": "8xNAD1tRpy0_" }, - "execution_count": null, - "outputs": [] + "outputs": [], + "source": [ + "!wget \"https://github.com/huggingface/Huggy/raw/main/Huggy.zip\" -O ./trained-envs-executables/linux/Huggy.zip" + ] }, { "cell_type": "code", @@ -270,6 +335,9 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "dYKVj8yUvj55" + }, "source": [ "## Let's recap how this environment works\n", "\n", @@ -307,13 +375,13 @@ "- *Time penalty*: a fixed-time penalty given at every action to **force him to get to the stick as fast as possible**.\n", "- *Rotation penalty*: we penalize Huggy if **he spins too much and turns too quickly**.\n", "- *Getting to the target reward*: we reward Huggy for **reaching the target**." - ], - "metadata": { - "id": "dYKVj8yUvj55" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "NAuEq32Mwvtz" + }, "source": [ "## Create the Huggy config file\n", "\n", @@ -333,13 +401,15 @@ " \"Create\n", "\n", "- Copy and paste the content below 🔽" - ], - "metadata": { - "id": "NAuEq32Mwvtz" - } + ] }, { "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "loQ0N5jhXW71" + }, + "outputs": [], "source": [ "behaviors:\n", " Huggy:\n", @@ -367,24 +437,22 @@ " max_steps: 2e6\n", " time_horizon: 1000\n", " summary_freq: 50000" - ], - "metadata": { - "id": "loQ0N5jhXW71" - }, - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", - "source": [ - "- Don't forget to save the file!" - ], "metadata": { "id": "oakN7UHwXdCX" - } + }, + "source": [ + "- Don't forget to save the file!" + ] }, { "cell_type": "markdown", + "metadata": { + "id": "r9wv5NYGw-05" + }, "source": [ "- **In the case you want to modify the hyperparameters**, in Google Colab notebook, you can click here to open the config.yaml: `/content/ml-agents/config/ppo/Huggy.yaml`\n", "\n", @@ -394,10 +462,7 @@ "\n", "=> Just keep in mind that **decreasing the `checkpoint_interval` means more models to upload to the Hub and so a longer uploading time**\n", "We’re now ready to train our agent 🔥." - ], - "metadata": { - "id": "r9wv5NYGw-05" - } + ] }, { "cell_type": "markdown", @@ -426,12 +491,12 @@ }, { "cell_type": "markdown", - "source": [ - "The training will take 30 to 45min depending on your machine (don't forget to **set up a GPU**), go take a ☕️you deserve it 🤗." - ], "metadata": { "id": "lN32oWF8zPjs" - } + }, + "source": [ + "The training will take 30 to 45min depending on your machine (don't forget to **set up a GPU**), go take a ☕️you deserve it 🤗." + ] }, { "cell_type": "code", @@ -457,6 +522,9 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "izT6FpgNzZ6R" + }, "source": [ "To be able to share your model with the community there are three more steps to follow:\n", "\n", @@ -469,10 +537,7 @@ "\n", "- Copy the token\n", "- Run the cell below and paste the token" - ], - "metadata": { - "id": "izT6FpgNzZ6R" - } + ] }, { "cell_type": "code", @@ -488,12 +553,12 @@ }, { "cell_type": "markdown", - "source": [ - "If you don't want to use a Google Colab or a Jupyter Notebook, you need to use this command instead: `huggingface-cli login`" - ], "metadata": { "id": "ew59mK19zjtN" - } + }, + "source": [ + "If you don't want to use a Google Colab or a Jupyter Notebook, you need to use this command instead: `huggingface-cli login`" + ] }, { "cell_type": "markdown", @@ -508,6 +573,9 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "KK4fPfnczunT" + }, "source": [ "And we define 4 parameters:\n", "\n", @@ -516,10 +584,7 @@ "3. `--repo-id`: the name of the Hugging Face repo you want to create or update. It’s always /\n", "If the repo does not exist **it will be created automatically**\n", "4. `--commit-message`: since HF repos are git repository you need to define a commit message." - ], - "metadata": { - "id": "KK4fPfnczunT" - } + ] }, { "cell_type": "code", @@ -534,6 +599,9 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "yborB0850FTM" + }, "source": [ "Else, if everything worked you should have this at the end of the process(but with a different url 😆) :\n", "\n", @@ -546,22 +614,22 @@ "It’s the link to your model repository. The repository contains a model card that explains how to use the model, your Tensorboard logs and your config file. **What’s awesome is that it’s a git repository, which means you can have different commits, update your repository with a new push, open Pull Requests, etc.**\n", "\n", "\"ml" - ], - "metadata": { - "id": "yborB0850FTM" - } + ] }, { "cell_type": "markdown", - "source": [ - "But now comes the best: **being able to play with Huggy online 👀.**" - ], "metadata": { "id": "5Uaon2cg0NrL" - } + }, + "source": [ + "But now comes the best: **being able to play with Huggy online 👀.**" + ] }, { "cell_type": "markdown", + "metadata": { + "id": "VMc4oOsE0QiZ" + }, "source": [ "## Play with your Huggy 🐕\n", "\n", @@ -572,13 +640,13 @@ "- Click on Play with my Huggy model\n", "\n", "\"load-huggy\"" - ], - "metadata": { - "id": "VMc4oOsE0QiZ" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "Djs8c5rR0Z8a" + }, "source": [ "1. In step 1, choose your model repository which is the model id (in my case ThomasSimonini/ppo-Huggy).\n", "\n", @@ -587,13 +655,13 @@ " - But since I want the more recent, I choose `Huggy.onnx`\n", "\n", "👉 What’s nice **is to try with different models steps to see the improvement of the agent.**" - ], - "metadata": { - "id": "Djs8c5rR0Z8a" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "PI6dPWmh064H" + }, "source": [ "Congrats on finishing this bonus unit!\n", "\n", @@ -603,18 +671,15 @@ "\n", "\n", "## Keep Learning, Stay awesome 🤗" - ], - "metadata": { - "id": "PI6dPWmh064H" - } + ] } ], "metadata": { "accelerator": "GPU", "colab": { - "provenance": [], + "include_colab_link": true, "private_outputs": true, - "include_colab_link": true + "provenance": [] }, "gpuClass": "standard", "kernelspec": { diff --git a/notebooks/unit5/unit5.ipynb b/notebooks/unit5/unit5.ipynb index e0220c34..cb9ec8b9 100644 --- a/notebooks/unit5/unit5.ipynb +++ b/notebooks/unit5/unit5.ipynb @@ -3,8 +3,8 @@ { "cell_type": "markdown", "metadata": { - "id": "view-in-github", - "colab_type": "text" + "colab_type": "text", + "id": "view-in-github" }, "source": [ "\"Open" @@ -22,6 +22,9 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "97ZiytXEgqIz" + }, "source": [ "\"Thumbnail\"/\n", "\n", @@ -33,33 +36,33 @@ "After that, you'll be able **to watch your agents playing directly on your browser**.\n", "\n", "For more information about the certification process, check this section 👉 https://huggingface.co/deep-rl-course/en/unit0/introduction#certification-process" - ], - "metadata": { - "id": "97ZiytXEgqIz" - } + ] }, { "cell_type": "markdown", - "source": [ - "⬇️ Here is an example of what **you will achieve at the end of this unit.** ⬇️\n" - ], "metadata": { "id": "FMYrDriDujzX" - } + }, + "source": [ + "⬇️ Here is an example of what **you will achieve at the end of this unit.** ⬇️\n" + ] }, { "cell_type": "markdown", + "metadata": { + "id": "cBmFlh8suma-" + }, "source": [ "\"Pyramids\"/\n", "\n", "\"SnowballTarget\"/" - ], - "metadata": { - "id": "cBmFlh8suma-" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "A-cYE0K5iL-w" + }, "source": [ "### 🎮 Environments:\n", "\n", @@ -69,22 +72,22 @@ "### 📚 RL-Library:\n", "\n", "- [ML-Agents](https://github.com/Unity-Technologies/ml-agents)\n" - ], - "metadata": { - "id": "A-cYE0K5iL-w" - } + ] }, { "cell_type": "markdown", - "source": [ - "We're constantly trying to improve our tutorials, so **if you find some issues in this notebook**, please [open an issue on the GitHub Repo](https://github.com/huggingface/deep-rl-class/issues)." - ], "metadata": { "id": "qEhtaFh9i31S" - } + }, + "source": [ + "We're constantly trying to improve our tutorials, so **if you find some issues in this notebook**, please [open an issue on the GitHub Repo](https://github.com/huggingface/deep-rl-class/issues)." + ] }, { "cell_type": "markdown", + "metadata": { + "id": "j7f63r3Yi5vE" + }, "source": [ "## Objectives of this notebook 🏆\n", "\n", @@ -92,20 +95,17 @@ "\n", "- Understand how works **ML-Agents**, the environment library.\n", "- Be able to **train agents in Unity Environments**.\n" - ], - "metadata": { - "id": "j7f63r3Yi5vE" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "viNzVbVaYvY3" + }, "source": [ "## This notebook is from the Deep Reinforcement Learning Course\n", "\"Deep" - ], - "metadata": { - "id": "viNzVbVaYvY3" - } + ] }, { "cell_type": "markdown", @@ -141,6 +141,9 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "xYO1uD5Ujgdh" + }, "source": [ "# Let's train our agents 🚀\n", "\n", @@ -148,66 +151,130 @@ "\n", "- For `Pyramids` : Mean Reward = 1.75\n", "- For `SnowballTarget` : Mean Reward = 15 or 30 targets hit in an episode.\n" - ], - "metadata": { - "id": "xYO1uD5Ujgdh" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "DssdIjk_8vZE" + }, "source": [ "## Set the GPU 💪\n", "- To **accelerate the agent's training, we'll use a GPU**. To do that, go to `Runtime > Change Runtime type`\n", "\n", "\"GPU" - ], - "metadata": { - "id": "DssdIjk_8vZE" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "sTfCXHy68xBv" + }, "source": [ "- `Hardware Accelerator > GPU`\n", "\n", "\"GPU" - ], - "metadata": { - "id": "sTfCXHy68xBv" - } + ] }, { "cell_type": "markdown", - "metadata": { - "id": "an3ByrXYQ4iK" - }, + "metadata": {}, "source": [ - "## Clone the repository and install the dependencies 🔽\n" + "## Clone the repository 🔽\n", + "\n", + "- We need to clone the repository, that contains **ML-Agents.**" ] }, { "cell_type": "code", "execution_count": null, - "metadata": { - "id": "6WNoL04M7rTa" - }, + "metadata": {}, "outputs": [], "source": [ "%%capture\n", - "# Clone the repository\n", + "# Clone the repository (can take 3min)\n", "!git clone --depth 1 https://github.com/Unity-Technologies/ml-agents" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setup the Virtual Environment 🔽\n", + "- In order for the **ML-Agents** to run successfully in Colab, Colab's Python version must meet the library's Python requirements.\n", + "\n", + "- We can check for the supported Python version under the `python_requires` parameter in the `setup.py` files. These files are required to set up the **ML-Agents** library for use and can be found in the following locations:\n", + " - `/content/ml-agents/ml-agents/setup.py`\n", + " - `/content/ml-agents/ml-agents-envs/setup.py`\n", + "\n", + "- Colab's Current Python version(can be checked using `!python --version`) doesn't match the library's `python_requires` parameter, as a result installation may silently fail and lead to errors like these, when executing the same commands later:\n", + " - `/bin/bash: line 1: mlagents-learn: command not found`\n", + " - `/bin/bash: line 1: mlagents-push-to-hf: command not found`\n", + "\n", + "- To resolve this, we'll create a virtual environment with a Python version compatible with the **ML-Agents** library.\n", + "\n", + "`Note:` *For future compatibility, always check the `python_requires` parameter in the installation files and set your virtual environment to the maximum supported Python version in the given below script if the Colab's Python version is not compatible*" + ] + }, { "cell_type": "code", "execution_count": null, - "metadata": { - "id": "d8wmVcMk7xKo" - }, + "metadata": {}, + "outputs": [], + "source": [ + "# Colab's Current Python Version (Incompatible with ML-Agents)\n", + "!python --version" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, "outputs": [], "source": [ + "# Install virtualenv and create a virtual environment\n", + "!pip install virtualenv\n", + "!virtualenv myenv\n", + "\n", + "# Download and install Miniconda\n", + "!wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh\n", + "!chmod +x Miniconda3-latest-Linux-x86_64.sh\n", + "!./Miniconda3-latest-Linux-x86_64.sh -b -f -p /usr/local\n", "\n", - "# Go inside the repository and install the package\n", + "# Activate Miniconda and install Python ver 3.10.12\n", + "!source /usr/local/bin/activate\n", + "!conda install -q -y --prefix /usr/local python=3.10.12 ujson # Specify the version here\n", + "\n", + "# Set environment variables for Python and conda paths\n", + "!export PYTHONPATH=/usr/local/lib/python3.10/site-packages/\n", + "!export CONDA_PREFIX=/usr/local/envs/myenv" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Python Version in New Virtual Environment (Compatible with ML-Agents)\n", + "!python --version" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Installing the dependencies 🔽" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%%capture\n", + "# Go inside the repository and install the package (can take 3min)\n", "%cd ml-agents\n", "!pip3 install -e ./ml-agents-envs\n", "!pip3 install -e ./ml-agents" @@ -215,15 +282,15 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "R5_7Ptd_kEcG" + }, "source": [ "## SnowballTarget ⛄\n", "\n", "If you need a refresher on how this environments work check this section 👉\n", "https://huggingface.co/deep-rl-course/unit5/snowball-target" - ], - "metadata": { - "id": "R5_7Ptd_kEcG" - } + ] }, { "cell_type": "markdown", @@ -252,32 +319,32 @@ }, { "cell_type": "markdown", - "source": [ - "We downloaded the file SnowballTarget.zip from https://github.com/huggingface/Snowball-Target using `wget`" - ], "metadata": { "id": "ekSh8LWawkB5" - } + }, + "source": [ + "We downloaded the file SnowballTarget.zip from https://github.com/huggingface/Snowball-Target using `wget`" + ] }, { "cell_type": "code", - "source": [ - "!wget \"https://github.com/huggingface/Snowball-Target/raw/main/SnowballTarget.zip\" -O ./training-envs-executables/linux/SnowballTarget.zip" - ], + "execution_count": null, "metadata": { "id": "6LosWO50wa77" }, - "execution_count": null, - "outputs": [] + "outputs": [], + "source": [ + "!wget \"https://github.com/huggingface/Snowball-Target/raw/main/SnowballTarget.zip\" -O ./training-envs-executables/linux/SnowballTarget.zip" + ] }, { "cell_type": "markdown", - "source": [ - "We unzip the executable.zip file" - ], "metadata": { "id": "_LLVaEEK3ayi" - } + }, + "source": [ + "We unzip the executable.zip file" + ] }, { "cell_type": "code", @@ -313,6 +380,9 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "NAuEq32Mwvtz" + }, "source": [ "### Define the SnowballTarget config file\n", "- In ML-Agents, you define the **training hyperparameters into config.yaml files.**\n", @@ -333,7 +403,7 @@ " checkpoint_interval: 50000\n", " max_steps: 200000\n", " time_horizon: 64\n", - " threaded: true\n", + " threaded: false\n", " hyperparameters:\n", " learning_rate: 0.0003\n", " learning_rate_schedule: linear\n", @@ -353,31 +423,28 @@ " gamma: 0.99\n", " strength: 1.0\n", "```" - ], - "metadata": { - "id": "NAuEq32Mwvtz" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "4U3sRH4N4h_l" + }, "source": [ "\"Config\n", "\"Config" - ], - "metadata": { - "id": "4U3sRH4N4h_l" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "JJJdo_5AyoGo" + }, "source": [ "As an experimentation, you should also try to modify some other hyperparameters. Unity provides very [good documentation explaining each of them here](https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Training-Configuration-File.md).\n", "\n", "Now that you've created the config file and understand what most hyperparameters do, we're ready to train our agent 🔥." - ], - "metadata": { - "id": "JJJdo_5AyoGo" - } + ] }, { "cell_type": "markdown", @@ -406,12 +473,12 @@ }, { "cell_type": "markdown", - "source": [ - "The training will take 10 to 35min depending on your config, go take a ☕️you deserve it 🤗." - ], "metadata": { "id": "lN32oWF8zPjs" - } + }, + "source": [ + "The training will take 10 to 35min depending on your config, go take a ☕️you deserve it 🤗." + ] }, { "cell_type": "code", @@ -437,6 +504,9 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "izT6FpgNzZ6R" + }, "source": [ "To be able to share your model with the community there are three more steps to follow:\n", "\n", @@ -449,10 +519,7 @@ "\n", "- Copy the token\n", "- Run the cell below and paste the token" - ], - "metadata": { - "id": "izT6FpgNzZ6R" - } + ] }, { "cell_type": "code", @@ -468,15 +535,18 @@ }, { "cell_type": "markdown", - "source": [ - "If you don't want to use a Google Colab or a Jupyter Notebook, you need to use this command instead: `huggingface-cli login`" - ], "metadata": { "id": "aSU9qD9_6dem" - } + }, + "source": [ + "If you don't want to use a Google Colab or a Jupyter Notebook, you need to use this command instead: `huggingface-cli login`" + ] }, { "cell_type": "markdown", + "metadata": { + "id": "KK4fPfnczunT" + }, "source": [ "Then, we simply need to run `mlagents-push-to-hf`.\n", "\n", @@ -493,21 +563,18 @@ "For instance:\n", "\n", "`!mlagents-push-to-hf --run-id=\"SnowballTarget1\" --local-dir=\"./results/SnowballTarget1\" --repo-id=\"ThomasSimonini/ppo-SnowballTarget\" --commit-message=\"First Push\"`" - ], - "metadata": { - "id": "KK4fPfnczunT" - } + ] }, { "cell_type": "code", - "source": [ - "!mlagents-push-to-hf --run-id=\"SnowballTarget1\" --local-dir=\"./results/SnowballTarget1\" --repo-id=\"ThomasSimonini/ppo-SnowballTarget\" --commit-message=\"First Push\"" - ], + "execution_count": null, "metadata": { "id": "kAFzVB7OYj_H" }, - "execution_count": null, - "outputs": [] + "outputs": [], + "source": [ + "!mlagents-push-to-hf --run-id=\"SnowballTarget1\" --local-dir=\"./results/SnowballTarget1\" --repo-id=\"ThomasSimonini/ppo-SnowballTarget\" --commit-message=\"First Push\"" + ] }, { "cell_type": "code", @@ -522,6 +589,9 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "yborB0850FTM" + }, "source": [ "Else, if everything worked you should have this at the end of the process(but with a different url 😆) :\n", "\n", @@ -532,22 +602,22 @@ "```\n", "\n", "It’s the link to your model, it contains a model card that explains how to use it, your Tensorboard and your config file. **What’s awesome is that it’s a git repository, that means you can have different commits, update your repository with a new push etc.**" - ], - "metadata": { - "id": "yborB0850FTM" - } + ] }, { "cell_type": "markdown", - "source": [ - "But now comes the best: **being able to visualize your agent online 👀.**" - ], "metadata": { "id": "5Uaon2cg0NrL" - } + }, + "source": [ + "But now comes the best: **being able to visualize your agent online 👀.**" + ] }, { "cell_type": "markdown", + "metadata": { + "id": "VMc4oOsE0QiZ" + }, "source": [ "### Watch your agent playing 👀\n", "\n", @@ -558,19 +628,19 @@ "2. Launch the game and put it in full screen by clicking on the bottom right button\n", "\n", "\"Snowballtarget" - ], - "metadata": { - "id": "VMc4oOsE0QiZ" - } + ] }, - { + { "cell_type": "markdown", + "metadata": { + "id": "Djs8c5rR0Z8a" + }, "source": [ - "1. In step 1, type your username (your username is case sensitive: for instance, my username is ThomasSimonini not thomassimonini or ThOmasImoNInI) and click on the search button.", + "1. In step 1, type your username (your username is case sensitive: for instance, my username is ThomasSimonini not thomassimonini or ThOmasImoNInI) and click on the search button.\n", "\n", - "2. In step 2, select your model repository.", + "2. In step 2, select your model repository.\n", "\n", - "3. In step 3, **choose which model you want to replay**:", + "3. In step 3, **choose which model you want to replay**:\n", " - I have multiple ones, since we saved a model every 500000 timesteps.\n", " - But since I want the more recent, I choose `SnowballTarget.onnx`\n", "\n", @@ -579,13 +649,13 @@ "And don't hesitate to share the best score your agent gets on discord in #rl-i-made-this channel 🔥\n", "\n", "Let's now try a harder environment called Pyramids..." - ], - "metadata": { - "id": "Djs8c5rR0Z8a" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "rVMwRi4y_tmx" + }, "source": [ "## Pyramids 🏆\n", "\n", @@ -593,39 +663,36 @@ "- Our environment executable is in a zip file.\n", "- We need to download it and place it to `./training-envs-executables/linux/`\n", "- We use a linux executable because we use colab, and colab machines OS is Ubuntu (linux)" - ], - "metadata": { - "id": "rVMwRi4y_tmx" - } + ] }, { "cell_type": "markdown", - "source": [ - "We downloaded the file Pyramids.zip from from https://huggingface.co/spaces/unity/ML-Agents-Pyramids/resolve/main/Pyramids.zip using `wget`" - ], "metadata": { "id": "x2C48SGZjZYw" - } + }, + "source": [ + "We downloaded the file Pyramids.zip from from https://huggingface.co/spaces/unity/ML-Agents-Pyramids/resolve/main/Pyramids.zip using `wget`" + ] }, { "cell_type": "code", - "source": [ - "!wget \"https://huggingface.co/spaces/unity/ML-Agents-Pyramids/resolve/main/Pyramids.zip\" -O ./training-envs-executables/linux/Pyramids.zip" - ], + "execution_count": null, "metadata": { "id": "eWh8Pl3sjZY2" }, - "execution_count": null, - "outputs": [] + "outputs": [], + "source": [ + "!wget \"https://huggingface.co/spaces/unity/ML-Agents-Pyramids/resolve/main/Pyramids.zip\" -O ./training-envs-executables/linux/Pyramids.zip" + ] }, { "cell_type": "markdown", - "source": [ - "We unzip the executable.zip file" - ], "metadata": { "id": "V5LXPOPujZY3" - } + }, + "source": [ + "We unzip the executable.zip file" + ] }, { "cell_type": "code", @@ -661,6 +728,9 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "fqceIATXAgih" + }, "source": [ "### Modify the PyramidsRND config file\n", "- Contrary to the first environment which was a custom one, **Pyramids was made by the Unity team**.\n", @@ -672,32 +742,29 @@ "👉 To do that, we go to config/ppo/PyramidsRND.yaml,**and modify these to max_steps to 1000000.**\n", "\n", "\"Pyramids" - ], - "metadata": { - "id": "fqceIATXAgih" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "RI-5aPL7BWVk" + }, "source": [ "As an experimentation, you should also try to modify some other hyperparameters, Unity provides a very [good documentation explaining each of them here](https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Training-Configuration-File.md).\n", "\n", "We’re now ready to train our agent 🔥." - ], - "metadata": { - "id": "RI-5aPL7BWVk" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "s5hr1rvIBdZH" + }, "source": [ "### Train the agent\n", "\n", "The training will take 30 to 45min depending on your machine, go take a ☕️you deserve it 🤗." - ], - "metadata": { - "id": "s5hr1rvIBdZH" - } + ] }, { "cell_type": "code", @@ -723,50 +790,53 @@ }, { "cell_type": "code", - "source": [ - "!mlagents-push-to-hf --run-id= # Add your run id --local-dir= # Your local dir --repo-id= # Your repo id --commit-message= # Your commit message" - ], + "execution_count": null, "metadata": { "id": "yiEQbv7rB4mU" }, - "execution_count": null, - "outputs": [] + "outputs": [], + "source": [ + "!mlagents-push-to-hf --run-id= # Add your run id --local-dir= # Your local dir --repo-id= # Your repo id --commit-message= # Your commit message" + ] }, { "cell_type": "markdown", + "metadata": { + "id": "7aZfgxo-CDeQ" + }, "source": [ "### Watch your agent playing 👀\n", "\n", "👉 https://huggingface.co/spaces/unity/ML-Agents-Pyramids" - ], - "metadata": { - "id": "7aZfgxo-CDeQ" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "hGG_oq2n0wjB" + }, "source": [ "### 🎁 Bonus: Why not train on another environment?\n", "Now that you know how to train an agent using MLAgents, **why not try another environment?**\n", "\n", "MLAgents provides 17 different and we’re building some custom ones. The best way to learn is to try things of your own, have fun.\n", "\n" - ], - "metadata": { - "id": "hGG_oq2n0wjB" - } + ] }, { "cell_type": "markdown", - "source": [ - "![cover](https://miro.medium.com/max/1400/0*xERdThTRRM2k_U9f.png)" - ], "metadata": { "id": "KSAkJxSr0z6-" - } + }, + "source": [ + "![cover](https://miro.medium.com/max/1400/0*xERdThTRRM2k_U9f.png)" + ] }, { "cell_type": "markdown", + "metadata": { + "id": "YiyF4FX-04JB" + }, "source": [ "You have the full list of the Unity official environments here 👉 https://github.com/Unity-Technologies/ml-agents/blob/develop/docs/Learning-Environment-Examples.md\n", "\n", @@ -775,13 +845,13 @@ "For now we have integrated:\n", "- [Worm](https://huggingface.co/spaces/unity/ML-Agents-Worm) demo where you teach a **worm to crawl**.\n", "- [Walker](https://huggingface.co/spaces/unity/ML-Agents-Walker) demo where you teach an agent **to walk towards a goal**." - ], - "metadata": { - "id": "YiyF4FX-04JB" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "PI6dPWmh064H" + }, "source": [ "That’s all for today. Congrats on finishing this tutorial!\n", "\n", @@ -790,18 +860,15 @@ "See you on Unit 6 🔥,\n", "\n", "## Keep Learning, Stay awesome 🤗" - ], - "metadata": { - "id": "PI6dPWmh064H" - } + ] } ], "metadata": { "accelerator": "GPU", "colab": { - "provenance": [], + "include_colab_link": true, "private_outputs": true, - "include_colab_link": true + "provenance": [] }, "gpuClass": "standard", "kernelspec": { diff --git a/units/en/unit5/hands-on.mdx b/units/en/unit5/hands-on.mdx index 37dd19fc..e7fec94d 100644 --- a/units/en/unit5/hands-on.mdx +++ b/units/en/unit5/hands-on.mdx @@ -81,19 +81,67 @@ Before diving into the notebook, you need to: GPU Step 2 -## Clone the repository and install the dependencies 🔽 -- We need to clone the repository that **contains the experimental version of the library that allows you to push your trained agent to the Hub.** +## Clone the repository 🔽 + +- We need to clone the repository, that contains **ML-Agents.** ```bash -# Clone the repository +# Clone the repository (can take 3min) git clone --depth 1 https://github.com/Unity-Technologies/ml-agents ``` +## Setup the Virtual Environment 🔽 + +- In order for the **ML-Agents** to run successfully in Colab, Colab's Python version must meet the library's Python requirements. + +- We can check for the supported Python version under the `python_requires` parameter in the `setup.py` files. These files are required to set up the **ML-Agents** library for use and can be found in the following locations: + - `/content/ml-agents/ml-agents/setup.py` + - `/content/ml-agents/ml-agents-envs/setup.py` + +- Colab's Current Python version(can be checked using `!python --version`) doesn't match the library's `python_requires` parameter, as a result installation may silently fail and lead to errors like these, when executing the same commands later: + - `/bin/bash: line 1: mlagents-learn: command not found` + - `/bin/bash: line 1: mlagents-push-to-hf: command not found` + +- To resolve this, we'll create a virtual environment with a Python version compatible with the **ML-Agents** library. + +`Note:` *For future compatibility, always check the `python_requires` parameter in the installation files and set your virtual environment to the maximum supported Python version in the given below script if the Colab's Python version is not compatible* + +```bash +# Colab's Current Python Version (Incompatible with ML-Agents) +!python --version +``` + +```bash +# Install virtualenv and create a virtual environment +!pip install virtualenv +!virtualenv myenv + +# Download and install Miniconda +!wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh +!chmod +x Miniconda3-latest-Linux-x86_64.sh +!./Miniconda3-latest-Linux-x86_64.sh -b -f -p /usr/local + +# Activate Miniconda and install Python ver 3.10.12 +!source /usr/local/bin/activate +!conda install -q -y --prefix /usr/local python=3.10.12 ujson # Specify the version here + +# Set environment variables for Python and conda paths +!export PYTHONPATH=/usr/local/lib/python3.10/site-packages/ +!export CONDA_PREFIX=/usr/local/envs/myenv +``` + +```bash +# Python Version in New Virtual Environment (Compatible with ML-Agents) +!python --version +``` + +## Installing the dependencies 🔽 + ```bash -# Go inside the repository and install the package -cd ml-agents -pip install -e ./ml-agents-envs -pip install -e ./ml-agents +# Go inside the repository and install the package (can take 3min) +%cd ml-agents +pip3 install -e ./ml-agents-envs +pip3 install -e ./ml-agents ``` ## SnowballTarget ⛄ diff --git a/units/en/unitbonus1/train.mdx b/units/en/unitbonus1/train.mdx index 4b911b70..cbc5a2f8 100644 --- a/units/en/unitbonus1/train.mdx +++ b/units/en/unitbonus1/train.mdx @@ -64,15 +64,62 @@ Before diving into the notebook, you need to: GPU Step 2 -## Clone the repository and install the dependencies 🔽 +## Clone the repository 🔽 -- We need to clone the repository, that contains ML-Agents. +- We need to clone the repository, that contains **ML-Agents.** ```bash # Clone the repository (can take 3min) git clone --depth 1 https://github.com/Unity-Technologies/ml-agents ``` +## Setup the Virtual Environment 🔽 + +- In order for the **ML-Agents** to run successfully in Colab, Colab's Python version must meet the library's Python requirements. + +- We can check for the supported Python version under the `python_requires` parameter in the `setup.py` files. These files are required to set up the **ML-Agents** library for use and can be found in the following locations: + - `/content/ml-agents/ml-agents/setup.py` + - `/content/ml-agents/ml-agents-envs/setup.py` + +- Colab's Current Python version(can be checked using `!python --version`) doesn't match the library's `python_requires` parameter, as a result installation may silently fail and lead to errors like these, when executing the same commands later: + - `/bin/bash: line 1: mlagents-learn: command not found` + - `/bin/bash: line 1: mlagents-push-to-hf: command not found` + +- To resolve this, we'll create a virtual environment with a Python version compatible with the **ML-Agents** library. + +`Note:` *For future compatibility, always check the `python_requires` parameter in the installation files and set your virtual environment to the maximum supported Python version in the given below script if the Colab's Python version is not compatible* + +```bash +# Colab's Current Python Version (Incompatible with ML-Agents) +!python --version +``` + +```bash +# Install virtualenv and create a virtual environment +!pip install virtualenv +!virtualenv myenv + +# Download and install Miniconda +!wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh +!chmod +x Miniconda3-latest-Linux-x86_64.sh +!./Miniconda3-latest-Linux-x86_64.sh -b -f -p /usr/local + +# Activate Miniconda and install Python ver 3.10.12 +!source /usr/local/bin/activate +!conda install -q -y --prefix /usr/local python=3.10.12 ujson # Specify the version here + +# Set environment variables for Python and conda paths +!export PYTHONPATH=/usr/local/lib/python3.10/site-packages/ +!export CONDA_PREFIX=/usr/local/envs/myenv +``` + +```bash +# Python Version in New Virtual Environment (Compatible with ML-Agents) +!python --version +``` + +## Installing the dependencies 🔽 + ```bash # Go inside the repository and install the package (can take 3min) %cd ml-agents