3 - ResNet.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Convolutional Neural Networks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Abdullah Mobeen\\Anaconda3\\lib\\site-packages\\h5py\\__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n",
      "'matplotlib' is not recognized as an internal or external command,\n",
      "operable program or batch file.\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "!matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Since the aim of this series is to provide tutorials on the necessary components of AlphaZero, we will assume basic familiarity with the Neural Networks and focus more on Convolutional Neural Networks.\n",
    "\n",
    "\n",
    "## What is a Convolutional Neural Network?\n",
    "\n",
    "As stated in Deep Learning (Goodfellow et al.), __Convolutional Networks are simply neural networks that use convolution in place of general matrix multiplication in at least one of their layers__. Before we discuss the key features of a Convolutional Neural Network and the architecture used for AlphaZero, let's just briefly discuss what does __convolution__ mean. \n",
    "\n",
    "## Convolution:\n",
    "\n",
    "### 1D Convolution:\n",
    "\n",
    "Say you have as input a one dimensional array of 10 items, called X. \n",
    "\n",
    "![title](images/x_1d.PNG)\n",
    "\n",
    "Since convolution is an operation where you convolve one filter over an array (matrix in 2D and tensor in high dimension), we also need a filter for this example. Say we take a filter of length 3, called K. \n",
    "\n",
    "![title](images/filter_1d.PNG)\n",
    "\n",
    "To convolve the filter (K) over the input array (X), we would slide the filter over the array. So at the beginning, the filter would be at the top of the first three items in X. We multiply the corresponding items and sum the products -- $x_{1}.k_{1} + x_{2}.k_{2} + x_{3}.k_{3}$. Let's call this sum $y_{1}$. We put this sum into a new array, let's call it Y, which represents the results of convoling K over X. Each item in Y is a result of sliding the filter over the array and doing the arithematic.\n",
    "\n",
    "The discrete formulae is as follows: \n",
    "\n",
    "<img src=\"images/form.PNG\" alt=\"Drawing\" style=\"width: 300px;\"/>\n",
    "\n",
    "The whole operation would like look like this:\n",
    "\n",
    "![title](images/1dconv.PNG)\n",
    "\n",
    "Remember, the choice of arithematic in the convolution is subjective and depends on the type of application you're CNN is built for. For example, instead of just storing the sum of products, we can go a step further and divide the sum by the size of the filter, we would have an __averaging kernel__, which results in output equal to local average of input. Another example is the __identity kernel__, where the output is equal to the input. \n",
    "\n",
    "Also, notice that convolution outputs arrays (some call the outputs map too) that are shorter than the input array. In some applications, where you wish to have the output of the same size as the input, you can use a technique called __padding__. In padding, you add extra zeroes at the end of the input array so that the output comes out to be the same size as the original input. Notice in the above diagram that convolving a filter with __stride__ 1 (how many steps you move your filter forward), shrinks the size of the output by 1. So you can add extra zeroes at the end of the input array to obtain a new output of the same size. Here is what it looks like: \n",
    "\n",
    "![title](images/padding.PNG)\n",
    "\n",
    "Here is a sample code for 1D convolution using Tensorflow:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2. 3. 3. 3. 2.]\n"
     ]
    }
   ],
   "source": [
    "sess = tf.Session()\n",
    "\n",
    "ones_1d = np.ones(5)\n",
    "weight_1d = np.ones(3)\n",
    "strides_1d = 1\n",
    "\n",
    "in_1d = tf.constant(ones_1d, dtype=tf.float32)\n",
    "filter_1d = tf.constant(weight_1d, dtype=tf.float32)\n",
    "\n",
    "in_width = int(in_1d.shape[0])\n",
    "filter_width = int(filter_1d.shape[0])\n",
    "\n",
    "input_1d   = tf.reshape(in_1d, [1, in_width, 1])\n",
    "kernel_1d = tf.reshape(filter_1d, [filter_width, 1, 1])\n",
    "output_1d = tf.squeeze(tf.nn.conv1d(input_1d, kernel_1d, strides_1d, padding='SAME'))\n",
    "\n",
    "print(sess.run(output_1d))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Let's now discuss __2D Convolution__.\n",
    "\n",
    "### 2D Convolution:\n",
    "\n",
    "I hope that by this time, the basics of convolution operation are clear. 2D convolution has same operations as the 1D convolution, the only thing that changes is the dimensions of the input array (which we now call a matrix). Before I explain it diagramatically, let me present the discrete formula for 2D convolution:\n",
    "\n",
    "<img src=\"images/2dconv.PNG\" alt=\"Drawing\" style=\"width: 300px;\"/>\n",
    "\n",
    "Now, consider an input matrix of size (7 x 7), let's call it X. To convolve, we would need a filter. Let's pick a filter of size (3 x 3) and call it K. This is what the input matrix looks like:\n",
    "\n",
    "<img src=\"images/2dx.PNG\" alt=\"Drawing\" style=\"width: 200px;\"/>\n",
    "\n",
    "This is what the filter would look like:\n",
    "\n",
    "<img src=\"images/2dfilter.PNG\" alt=\"Drawing\" style=\"width: 100px;\"/>\n",
    "\n",
    "To convolve this filter, now you need to decide the size of the stride too, which is hoe much you wish to slide the filter horizontally and vertically. This operation has a cool illustration from the web (by deeplearning.ai), using the stride of (1 x 1):\n",
    "\n",
    "<img src=\"images/2dconvm.gif\" alt=\"Drawing\" style=\"width: 300px;\"/>\n",
    "\n",
    "In this illustration, padding is used on the original matrix to produce a matrix of the same size the original size of the input. To make the representation of output more consistent with our representation of the input and the filter, let me show how the output matrix, Y, would look like after the convolution.\n",
    "\n",
    "<img src=\"images/2dconvY.PNG\" alt=\"Drawing\" style=\"width: 200px;\"/>\n",
    "\n",
    "Here's a sample code for 2D convolution in Tensorflow:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[4. 6. 6. 6. 4.]\n",
      " [6. 9. 9. 9. 6.]\n",
      " [6. 9. 9. 9. 6.]\n",
      " [6. 9. 9. 9. 6.]\n",
      " [4. 6. 6. 6. 4.]]\n"
     ]
    }
   ],
   "source": [
    "ones_2d = np.ones((5,5))\n",
    "weight_2d = np.ones((3,3))\n",
    "strides_2d = [1, 1, 1, 1]\n",
    "\n",
    "in_2d = tf.constant(ones_2d, dtype=tf.float32)\n",
    "filter_2d = tf.constant(weight_2d, dtype=tf.float32)\n",
    "\n",
    "in_width = int(in_2d.shape[0])\n",
    "in_height = int(in_2d.shape[1])\n",
    "\n",
    "filter_width = int(filter_2d.shape[0])\n",
    "filter_height = int(filter_2d.shape[1])\n",
    "\n",
    "input_2d   = tf.reshape(in_2d, [1, in_height, in_width, 1])\n",
    "kernel_2d = tf.reshape(filter_2d, [filter_height, filter_width, 1, 1])\n",
    "\n",
    "output_2d = tf.squeeze(tf.nn.conv2d(input_2d, kernel_2d, strides=strides_2d, padding='SAME'))\n",
    "print(sess.run(output_2d))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "There is also a __3D Convolution__, but we'll not discuss it here as the user can extend the convolution operation explained here to high dimensions. \n",
    "\n",
    "For the rest of our discussion, we will assume 2D convolution operation. \n",
    "\n",
    "\n",
    "Notice, that convolving one filter on the input matrix produces one output matrix. So if we convolve many filters on the input matrix, we will end up with a stack of output matrices where each matrix corresponds to one particular feature. To understand it better, think of the famous object recognition problem. Here, the input matrix is of image pixels and the filters correspond to different shapes in the image. For example, consider an image of Albert Einstein\n",
    "\n",
    "<img src=\"images/eins.jpg\" alt=\"Drawing\" style=\"width: 300px;\"/>\n",
    "\n",
    "If we need to detect a human being in this image, we would have filters corresponding to different features in a human being - eyes, lips, nose, ears etc. Convolving the eyes filter over the image would produce an output image that is _activated_ in the regions where eyes are present. Same happens for each filter and we end up with a stack of output images. What I described is a simplification of the whole process - assuming basic familiarity with Neural Networks, the readers should know that a network starts with randomly initialized set of weights (e.g. Xavier Initialization) and the learns these features through __back-propagation__. Essentially, the idea is to have multiple filters at each __convolution layer__ that will learn low level as well as high level features in the data given and hope that the learned filters would detect similar objects in the unseen data. \n",
    "\n",
    "The layer in Convolutional Neural Networks where the convolution operation happens is called the __Convolution Layer__. To summarize, the convolution layer:\n",
    "\n",
    "> - Accepts an input of size Width_1 x Height_1 x Depth_1 (If input is an image in RGB color channel for example).\n",
    "> - Requires four hyperparameters:\n",
    ">> - Number of filters K\n",
    ">> - Filter's dimensions F,\n",
    ">> - The stride S,\n",
    ">> - The amount of zero padding P (if applicable).\n",
    "> - Produces an output of size Width_2 x Height_2 x Depth_2 where:\n",
    ">> - Width_2 = (Width_1 - F + 2P)/S + 1,\n",
    ">> - Height_2 = (Height_1 - F + 2P)/S + 1,\n",
    ">> - Depth_2 = K (number of filters, because remember the output is a stack of matrices correspong to each of K filters)\n",
    "> - With parameter sharing, it introduces F x F x D$_{1}$ weights per filter, for a total of (F x F x D$_{1}$) x K weights and K biases.\n",
    "\n",
    "We have a few more basic techniques to cover before we dive into the fun world of CNN architectures (yayy!). Ok, so we know that performing convolution results in a stack of output matrices. Sometimes, for computation purposes, we need to reduce the size of the stack. Note, we don't reduce the number of output matrices in the stack but the size of each output matrix in the stack. To achieve so, we perform an operation called __Pooling__. Let's discuss what pooling is.\n",
    "\n",
    "## Pooling:\n",
    "\n",
    "Pooling is similar to Convolution in a way that you have a window (similar to filter but there are no weights to learn) and you slide it over the output matrix. There are 4 key steps in pooling:\n",
    "\n",
    "> - Pick a window size, e.g. (2 x 2),\n",
    "> - Pick a stride, e.g (1 x 1),\n",
    "> - Walk your window across your output matrices (in the output stack),\n",
    "> - During sliding a window over the output matrix, pick the maximum value (this is called Max-Pooling).\n",
    "\n",
    "This is what max-pooling would look like using a window of size (2 x 2) and stride (2 x 2), if towards the corner of the matrix, we only have one column/row, we just consider that column/row to pick the maximum (notice the top right element in the output): \n",
    "\n",
    "<img src=\"images/maxj.PNG\" alt=\"Drawing\" style=\"width: 500px;\"/>\n",
    "\n",
    "The layer in a CNN where pooling operation occurs is called the __Pooling Layer__. To summarize, a pooling layer:\n",
    "\n",
    "> - Accepts a volume of size Width_1 x Height_1 x Depth_1,\n",
    "> - Requires two hyperparameters:\n",
    ">> - Dimension of the window F,\n",
    ">> - The stride size S,\n",
    "> - Produces an ouput of size Width_2 x Height_2 x Depth_2 where:\n",
    ">> - Width_2 = (Width_1 - F)/S + 1\n",
    ">> - Height_2 = (Height_1 - F)/S + 1\n",
    ">> - Depth_2 = Depth_1\n",
    "\n",
    "## Normalization:\n",
    "\n",
    "Once, the output stack is shrinked using pooling, we perform another operation for the sake computation convenience. It is called __Normalization__. In this operation, we change all the negative values in the output matrix to 0. This operation is performed by an activation function called __ReLU__ (Rectified Linear Unit). ReLU is just:\n",
    "\n",
    "> $f(x) = max(x,0)$\n",
    "\n",
    "The layer in a convolution neural network where this operation takes place is called the __Normalization Layer__. \n",
    "\n",
    "## Fully Connected Operation:\n",
    "\n",
    "Based on the type of application, a convolutional neural network will have a __Fully Connected Layer__ at the end of the architecture. Neurons in the fully connected layer have fullconnections to all activations in the preceeding layer. For example, if we have an image classification problem for 10 classes, the fully connected layer will have 10 output neurons connected to all the activations in the previous layer. \n",
    "\n",
    "## Summary:\n",
    "\n",
    "Now that I have defined the basic operations in a CNN, let me put these operations/layers together to summarize the steps involved in learning via CNN. The layers are arranged as follows:\n",
    "\n",
    "> - __INPUT LAYER__ ~ Raw representation of the matrix we're working on. For example, if the input is the game-board, it could be represented as a matrix with size Width x Height.\n",
    "> - __CONVOLUTIONAL LAYER__ ~ This layer will compute the output of neurons that are connected to local regions in the input by performing the convolution operation. The output may result in Width x Height x K (if we use K filters).\n",
    "> - __NORMALIZATION LAYER__ ~ This layer will apply element-wise activation function such as ReLU (which is $max(x,0)$). This layer does not affect the volume of its input.\n",
    "> - __POOLING LAYER__ ~ This layer will perform a downsampling operation along the spatial dimensions (width and height), shrinking the volume only in width and height, but maintaining the depth of the stack. \n",
    "> - __FULLY CONNECTED LAYER__ - This layer will compute scores according to the application."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "# CNN Architectures"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Starting from a vanilla deep neural network with just a bunch of fully connected layers stacked on top of each other (for object recognition), we saw that there were two main issues:\n",
    "> - The network learnt the features in the input image but did not learn the spatial relation between the features (so in case of Digit Recognition, you can essentially input a noisy non-sense image and the network will classify it as a digit with high confidence increasing the number of false positives).\n",
    "> - The gradient signal loss increased during backpropagation as we increased the number of hidden layers (__vanishing gradient__).\n",
    "\n",
    "For the first issue, we developed Convolutional Neural Networks, which made use of filters and weight sharing to learn spatial relations in the input image. However, CNN alone does not solve the second issue. If you keep on increasing the number of layers in a CNN, it will also encounter the issue of vanishing gradient. This is how the number of layers kept increasing for the ImageNet winner each year:\n",
    "\n",
    "<img src=\"images/netdepth.png\" alt=\"Drawing\" style=\"width: 500px;\"/>\n",
    "\n",
    "\n",
    "So now the major issue is to design a network in which the gradient can reach all layers (might be a few hundreds) easily during backpropagation. Many famous CNN architectures faced the issue of vanishing gradients, e.g. AlexNet, ZF Net, GoogLeNet, VGGNet etc. Recently, a few architectures were discussed that found a way aroud vanishig gradients, e.g. __Residual Network, DenseNet, Highway Net__ etc. In this tutorial, we will only discuss ResNet, due to its use in AlphaZero program. \n",
    "\n",
    "## ResNet:\n",
    "\n",
    "ResNet solves the issue of vanishing gradients by a simple trick. Remember that a Neural Network is essentially a function approximator: \n",
    "\n",
    "> $y = f(x)$ , where $f(x)$ consists of operations like convolution, activations, batch normalization etc.\n",
    "\n",
    "During backpropagation, a signal has to go back through this $f(x)$. Since $f(x)$ consists of non-linearities like ReLU, where we set the negative values to 0, the gradient might disappear through multiplication with 0. So if a network is having trouble sending the gradient signal back to all layers, why don't we create a shortcut from each layer to pass the signal directly without going through the $f(x)$. This is essentially what ResNet does - it creates shortcuts from layers, called skip connections, to facilitate signal passing without loss. Mathematically, the operation becomes:\n",
    "\n",
    "> $y = f(x) + x$\n",
    "\n",
    "The addition of x allows the gradient to skip the intermediate layers and reach the bottom without being diminished. \n",
    "\n",
    "A rough sketch of a ResNet architecture is as follows:\n",
    "\n",
    "<img src=\"images/resnet.PNG\" alt=\"Drawing\" style=\"width: 500px;\"/>\n",
    "\n",
    "AlphaZero uses a Residual Neural Network. The policy $\\pi(a|s)$, which is the probability of taking an action $a$ given a board state $s$ is represented by an $8 * 8 * 73$ volume (chess). There are $8 * 8$ positions to pick a move from. In chess, there are 56 queen moves, which are in 8 possible directions times 7 maximum steps, 8 knight moves, and 9 under promotions, for a total of 73 possibilities. Here is a 2D chess move representation in AlphaZero:\n",
    "\n",
    "<img src=\"images/chessmove.PNG\" alt=\"Drawing\" style=\"width: 300px;\"/>\n",
    "\n",
    "And the whole input dimensions are represented as follows:\n",
    "\n",
    "<img src=\"images/chessin.PNG\" alt=\"Drawing\" style=\"width: 300px;\"/>\n",
    "\n",
    "Below is the implementation of ResNet in tensorflow:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "def normal_residual_unit(input_layer, is_training, filters,\n",
    "                         kernel_size, initializer, num_unit):\n",
    "    \n",
    "    with tf.variable_scope('Residual_unit_%d' % num_unit):\n",
    "        first_batch = tf.layers.batch_normalization(input_layer, training=is_training, name='First_batchnorm')\n",
    "        first_relu = tf.nn.relu(first_batch, name='First_relu')\n",
    "        first_conv = tf.layers.conv2d(first_relu, filters, kernel_size, padding='same', \n",
    "                                          activation=None, kernel_initializer=initializer,\n",
    "                                          name='First_convolution')\n",
    "        \n",
    "        sec_batch = tf.layers.batch_normalization(first_conv, training=is_training, name='Second_batchnorm')\n",
    "        sec_relu = tf.nn.relu(sec_batch, name='Second_relu')\n",
    "        sec_conv = tf.layers.conv2d(sec_relu, filters, kernel_size, padding='same',\n",
    "                                    activation=None, kernel_initializer=initializer, name='Second_convolution') \n",
    "        \n",
    "        addition = input_layer+sec_conv\n",
    "        \n",
    "    return addition\n",
    "\n",
    "def reshaped_residual_unit(input_layer, is_training, filters,\n",
    "                           kernel_size, initializer, num_unit):\n",
    "\n",
    "    with tf.variable_scope('Residual_unit_%d' % num_unit):\n",
    "        first_batch = tf.layers.batch_normalization(input_layer, training=is_training, name='First_batchnorm')\n",
    "        first_relu = tf.nn.relu(first_batch, name='First_relu')\n",
    "        first_conv = tf.layers.conv2d(first_relu, filters, kernel_size, strides=2,\n",
    "                                      padding='same', activation=None, \n",
    "                                      kernel_initializer=initializer,\n",
    "                                      name='First_convolution')\n",
    "        \n",
    "        sec_batch = tf.layers.batch_normalization(first_conv, training=is_training, name='Second_batchnorm')\n",
    "        sec_relu = tf.nn.relu(sec_batch, name='Second_relu')\n",
    "        sec_conv = tf.layers.conv2d(sec_relu, filters, kernel_size, padding='same',\n",
    "                                    activation=None, kernel_initializer=initializer,\n",
    "                                    name='Second_convolution')  \n",
    "        \n",
    "        reshaped_inp = tf.layers.conv2d(input_layer, filters, 1, strides=2, \n",
    "                                       activation=None, kernel_initializer=initializer,\n",
    "                                       name='Reshaped_input')\n",
    "        \n",
    "        addition = reshaped_inp + sec_conv\n",
    "        \n",
    "    return addition\n",
    "\n",
    "def residual_block(input_layer, is_training, filters, kernel_size, \n",
    "                   num_units, num_block, initializer, reduce_size=True):\n",
    "    \n",
    "    res_units = {}\n",
    "    with tf.variable_scope('Residual_block_%d' % num_block):\n",
    "        if reduce_size:\n",
    "            res_units[0] = reshaped_residual_unit(input_layer, is_training, filters, kernel_size, initializer, 0) #reshape \n",
    "        else:\n",
    "            res_units[0] = normal_residual_unit(input_layer, is_training, filters, kernel_size, initializer, 0) #retain size\n",
    "            \n",
    "        for i in range(1, num_units):\n",
    "            res_units[i] = normal_residual_unit(res_units[i-1], is_training, filters, kernel_size, initializer, i)\n",
    "            \n",
    "    return res_units[num_units-1]\n",
    "\n",
    "def my_model(X, y, is_training):\n",
    "    \n",
    "    initializer = tf.contrib.layers.xavier_initializer() #Xavier Initialization for the weights\n",
    "    \n",
    "    res_block = {} #Dictionary to keep record of the residual blocks\n",
    "    \n",
    "    filters = [32, 64, 128, 256, 512] #filters in each residual block\n",
    "    \n",
    "    # First convolution of the network\n",
    "    first_conv = tf.layers.conv2d(X, 32, 3, padding='same', activation=None, \n",
    "                                  kernel_initializer=initializer,\n",
    "                                  name='First_convolution')\n",
    "    batchnorm_layer = tf.layers.batch_normalization(first_conv, training=is_training,\n",
    "                                                   name='First_batchnorm')\n",
    "    relu_layer = tf.nn.relu(batchnorm_layer, name='First_relu')\n",
    "    \n",
    "    for i in range(5): #assuming 5 residual blocks\n",
    "        res_block[i] = residual_block(relu_layer, is_training, filters[i], 3, 5, i,\n",
    "                                      initializer, reduce_size=False)\n",
    "        if i != 4:\n",
    "            strided_conv = tf.layers.conv2d(res_block[i], filters[i+1], 3, strides=2, \n",
    "                                            padding='same', activation=None, \n",
    "                                            kernel_initializer=initializer,\n",
    "                                            name='Strided_convolution_%d' % i)\n",
    "            \n",
    "            batchnorm_strided = tf.layers.batch_normalization(strided_conv, \n",
    "                                                              training=is_training,\n",
    "                                                              name='Batchnorm_strided_%d' % i)\n",
    "            \n",
    "            relu_layer = tf.nn.relu(batchnorm_strided, name='Relu_%d' % i)\n",
    "    \n",
    "    last_conv = tf.layers.conv2d(res_block[4], 10, 2, padding='valid', \n",
    "                                 activation=None, kernel_initializer=initializer,\n",
    "                                 name='Last_convolution')\n",
    "    \n",
    "    last_batchnorm = tf.layers.batch_normalization(last_conv, training=is_training,\n",
    "                                                   name='Last_batchnorm')\n",
    "    preds = tf.reshape(last_batchnorm, [-1,10], name='Predictions') #subjective, change the prediction dimensions for games\n",
    "    \n",
    "    return preds\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The aim of this post was to highlight the key features of a Convolutional Neural Network and the Architecture that AlphaZero uses. Now the main question is, how does AlphaZero achieve training by combining Monte Carlo Tree Search with Residual Network? This will be explained in the next notebook. "
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [Anaconda3]",
   "language": "python",
   "name": "Python [Anaconda3]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}