Update experiment-management/tensorflow (#6)

* Add requirements.txt * Update keras integration nb
VertaAI · Jul 22, 2021 · 1083908 · 1083908
1 parent d1aa3f6
commit 1083908
Show file tree

Hide file tree

Showing 2 changed files with 71 additions and 33 deletions.
diff --git a/experiment-management/tensorflow/keras-integration.ipynb b/experiment-management/tensorflow/keras-integration.ipynb
@@ -4,14 +4,34 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# Client  Integration (Keras)"
+    "# Tracking Keras models in Verta using Autologging\n",
+    "\n",
+    "Verta's experiment management system enables data scientists to track rich information about their modeling experiments including data such as metrics, hyperparameters, confusion matrices, examples of input and output data, and many others.\n",
+    "\n",
+    "This notebook shows how to use Verta's experiment management system with models developed in Keras with a TensorFlow backend. See Verta [documentation](https://docs.verta.ai/verta/experiment-management) for full details on Verta's experiment management capabilities.\n",
+    "\n",
+    "Updated for Verta version: 0.18.2"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This example takes Keras's [MNIST MLP example](https://github.com/keras-team/keras/blob/master/examples/mnist_mlp.py) and incorportates [Verta's client integration](https://verta.readthedocs.io/en/master/_autogen/verta.integrations.keras.VertaCallback.html)."
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "This example takes Keras's [MNIST MLP example](https://github.com/keras-team/keras/blob/master/examples/mnist_mlp.py) and incorportates Verta's Client integration."
+    "<a href=\"https://colab.research.google.com/github/VertaAI/examples/blob/main/experiment-management/torch/keras-integration.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 0. Imports"
    ]
   },
   {
@@ -20,56 +40,60 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "HOST = \"app.verta.ai\"\n",
+    "from __future__ import print_function\n",
     "\n",
-    "PROJECT_NAME = \"MNIST Multiclassification\"\n",
-    "EXPERIMENT_NAME = \"FC-NN\""
+    "from tensorflow import keras\n",
+    "from tensorflow.keras.datasets import mnist\n",
+    "from tensorflow.keras.models import Sequential\n",
+    "from tensorflow.keras.layers import Dense, Dropout\n",
+    "from tensorflow.keras.optimizers import RMSprop"
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": 2,
+   "cell_type": "markdown",
    "metadata": {},
-   "outputs": [],
    "source": [
-    "# import os\n",
-    "# os.environ['VERTA_EMAIL'] = \n",
-    "# os.environ['VERTA_DEV_KEY'] = "
+    "### 0.1 Verta import and setup"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 2,
    "metadata": {},
    "outputs": [],
    "source": [
-    "from verta import Client\n",
-    "\n",
-    "client = Client(HOST)\n",
-    "proj = client.set_project(PROJECT_NAME)\n",
-    "expt = client.set_experiment(EXPERIMENT_NAME)"
+    "# restart your notebook if prompted on Colab\n",
+    "try:\n",
+    "    import verta\n",
+    "except ImportError:\n",
+    "    !pip install verta"
    ]
   },
   {
-   "cell_type": "markdown",
+   "cell_type": "code",
+   "execution_count": 3,
    "metadata": {},
+   "outputs": [],
    "source": [
-    "## Imports"
+    "# import os\n",
+    "# os.environ['VERTA_EMAIL'] = \n",
+    "# os.environ['VERTA_DEV_KEY'] = \n",
+    "# os.environ['VERTA_HOST'] ="
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 4,
-   "metadata": {},
+   "metadata": {
+    "scrolled": true
+   },
    "outputs": [],
    "source": [
-    "from __future__ import print_function\n",
+    "import os\n",
+    "from verta import Client\n",
+    "from verta.utils import ModelAPI\n",
     "\n",
-    "from tensorflow import keras\n",
-    "from tensorflow.keras.datasets import mnist\n",
-    "from tensorflow.keras.models import Sequential\n",
-    "from tensorflow.keras.layers import Dense, Dropout\n",
-    "from tensorflow.keras.optimizers import RMSprop"
+    "client = Client(os.environ['VERTA_HOST'])"
    ]
   },
   {
@@ -83,14 +107,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# Log Workflow"
+    "## 1. Model Training"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Prepare Data"
+    "### 1.1 Prepare data"
    ]
   },
   {
@@ -131,7 +155,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Define Model"
+    "### 1.2 Define model"
    ]
   },
   {
@@ -160,7 +184,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Run and Log Training"
+    "### 1.3 Train model"
    ]
   },
   {
@@ -169,6 +193,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "proj = client.set_project(\"MNIST Multiclassification\")\n",
+    "expt = client.set_experiment(\"FC-NN\")\n",
     "run = client.set_experiment_run()"
    ]
   },
@@ -180,7 +206,6 @@
    "source": [
     "from verta.integrations.keras import VertaCallback\n",
     "\n",
-    "\n",
     "history = model.fit(\n",
     "    x_train, y_train,\n",
     "    batch_size=batch_size,\n",
@@ -201,9 +226,18 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "# that's it! check your run below\n",
     "run"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Additional metadata for the run can be stored using regular API calls as documented here:  \n",
+    "https://verta.readthedocs.io/en/master/_autogen/verta.tracking.entities.ExperimentRun.html"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -214,7 +248,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -228,7 +262,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.5"
+   "version": "3.9.2"
   }
  },
  "nbformat": 4,

diff --git a/experiment-management/tensorflow/requirements.txt b/experiment-management/tensorflow/requirements.txt
@@ -0,0 +1,4 @@
+jupyter
+tensorflow
+
+verta