Pytorch_vs_Tensorflow.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Installation\n",
    "\n",
    "- 파이썬 버젼 체크 (Python version Check)\n",
    "- 파이토치 설치 (PyTorch Installation)\n",
    "- 쿠다 및 CuDNN 체크 (Cuda & CuDNN Check)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Python Version Check\n",
    "파이썬 버젼 체크"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.6.8 (default, Jan 14 2019, 11:02:34) \n",
      "[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]]\n"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "print(sys.version)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. PyTorch Installation\n",
    "- 구글 콜라브 버젼에 따라 파이토치가 설치되어 있을수도 있고 아닐 수도 있습니다.\n",
    "- 설치가 안되어 있을 경우 아래와 같은 명령어로 설치하면 됩니다.\n",
    "- !pip3 install torch torchvision"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting torch\n",
      "  Using cached https://files.pythonhosted.org/packages/30/57/d5cceb0799c06733eefce80c395459f28970ebb9e896846ce96ab579a3f1/torch-1.2.0-cp36-cp36m-manylinux1_x86_64.whl\n",
      "Collecting torchvision\n",
      "  Using cached https://files.pythonhosted.org/packages/06/e6/a564eba563f7ff53aa7318ff6aaa5bd8385cbda39ed55ba471e95af27d19/torchvision-0.4.0-cp36-cp36m-manylinux1_x86_64.whl\n",
      "Collecting numpy (from torch)\n",
      "  Using cached https://files.pythonhosted.org/packages/e5/e6/c3fdc53aed9fa19d6ff3abf97dfad768ae3afce1b7431f7500000816bda5/numpy-1.17.2-cp36-cp36m-manylinux1_x86_64.whl\n",
      "Collecting six (from torchvision)\n",
      "  Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl\n",
      "Collecting pillow>=4.1.1 (from torchvision)\n",
      "  Using cached https://files.pythonhosted.org/packages/14/41/db6dec65ddbc176a59b89485e8cc136a433ed9c6397b6bfe2cd38412051e/Pillow-6.1.0-cp36-cp36m-manylinux1_x86_64.whl\n",
      "Installing collected packages: numpy, torch, six, pillow, torchvision\n",
      "Successfully installed numpy-1.17.2 pillow-6.1.0 six-1.12.0 torch-1.2.0 torchvision-0.4.0\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "!pip3 install torch torchvision"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Cuda & cudnn Version Check\n",
    "- 파이토치를 통해 각각 몇 버젼이 설치 되어있는지 확인해줍니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Torch version:1.2.0\n",
      "cuda version: 10.0.130\n",
      "cudnn version:7602\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "\n",
    "print(\"Torch version:{}\".format(torch.__version__))\n",
    "print(\"cuda version: {}\".format(torch.version.cuda))\n",
    "print(\"cudnn version:{}\".format(torch.backends.cudnn.version()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. PyTorch CPU & GPU Tensor Check\n",
    "- 파이토치 텐서를 생성해봄으로써 제대로 설치 되었는지, 잘 동작하는지 확인해줍니다.\n",
    "\n",
    "### 4-1 Create CPU tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0., 0., 0.],\n",
      "        [0., 0., 0.]])\n"
     ]
    }
   ],
   "source": [
    "# 0으로 차있는 2x3 형태의 텐서를 생성합니다.\n",
    "cpu_tensor = torch.zeros(2,3)\n",
    "print(cpu_tensor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4-2 Allocate tensor on GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0., 0., 0.],\n",
      "        [0., 0., 0.]], device='cuda:0')\n"
     ]
    }
   ],
   "source": [
    "# 어느 장치(cpu 혹은 gpu)에 텐서를 올릴지 지정합니다.\n",
    "# 아래는 torch.device라는 함수를 사용해 gpu로 장치를 지정합니다. \n",
    "device = torch.device('cuda')\n",
    "\n",
    "# gpu가 사용 가능한지 확인해줍니다.\n",
    "if torch.cuda.is_available():\n",
    "  \n",
    "  # https://pytorch.org/docs/stable/tensors.html?highlight=#torch.Tensor.to\n",
    "  # cpu에 있었던 텐서를 to 함수를 이용해 지정해놓은 장치(여기서는 gpu)로 올려줍니다.\n",
    "  gpu_tensor = cpu_tensor.to(device)\n",
    "  print(gpu_tensor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4-3 Reallocate tensor back on CPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0., 0., 0.],\n",
       "        [0., 0., 0.]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# device 함수와 to 함수를 이용해 gpu에 있던 텐서를 다시 cpu로 옮겨올 수 있습니다.\n",
    "cpu_tensor_back = gpu_tensor.to(torch.device('cpu'))\n",
    "cpu_tensor_back"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "yF1VIGNP-Iu1"
   },
   "source": [
    "# Framework Comparison\n",
    "\n",
    "- Numpy vs Tensorflow vs PyTorch\n",
    "- 같은 연산이 각각 어떻게 구동이 되는지 알아보고 속도 역시 비교해보도록 하겠습니다.\n",
    "- x * y + z"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 187
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1527,
     "status": "ok",
     "timestamp": 1566734161893,
     "user": {
      "displayName": "Gunho Choi",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AAuE7mBSqM0cFU5ZceEDo5gH8hXfSGh8z5BVTu5HgREceg=s64",
      "userId": "04388737836176863066"
     },
     "user_tz": -540
    },
    "id": "P9eZ5Tev-WSP",
    "outputId": "d447ae4e-124a-4293-9275-a1b0bb70b0be"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1.76405235  0.40015721  0.97873798  2.2408932 ]\n",
      " [ 1.86755799 -0.97727788  0.95008842 -0.15135721]\n",
      " [-0.10321885  0.4105985   0.14404357  1.45427351]]\n",
      "[[ 0.76103773  0.12167502  0.44386323  0.33367433]\n",
      " [ 1.49407907 -0.20515826  0.3130677  -0.85409574]\n",
      " [-2.55298982  0.6536186   0.8644362  -0.74216502]]\n",
      "[[1. 1. 1. 1.]\n",
      " [1. 1. 1. 1.]\n",
      " [1. 1. 1. 1.]]\n",
      "0:00:00.001167\n"
     ]
    }
   ],
   "source": [
    "# 연산에 필요한 numpy, 시간을 측정하기 위해 datetime을 불러옵니다.\n",
    "import numpy as np \n",
    "from datetime import datetime\n",
    "start = datetime.now()\n",
    "\n",
    "# 랜덤하게 3x4 형태의 변수 x,y,z를 설정해줍니다.\n",
    "np.random.seed(0)\n",
    "\n",
    "N,D = 3,4\n",
    "\n",
    "x = np.random.randn(N,D)\n",
    "y = np.random.randn(N,D)\n",
    "z = np.random.randn(N,D)\n",
    "\n",
    "# x,y,z를 이용해 x*y+z를 계산해줍니다.\n",
    "a = x * y\n",
    "b = a + z\n",
    "c = np.sum(b)\n",
    "\n",
    "# 기울기(gradient)가 1이라고 가정하고 역전파를 해줍니다. 역전파에 대한 내용은 4장에서 자세히 다룹니다.\n",
    "grad_c = 1.0\n",
    "grad_b = grad_c * np.ones((N,D))\n",
    "grad_a = grad_b.copy()\n",
    "grad_z = grad_b.copy()\n",
    "grad_y = grad_a * y\n",
    "grad_x = grad_a * x\n",
    "\n",
    "# 각각의 기울기가 몇인지 걸린 시간은 얼마인지 확인해봅니다.\n",
    "print(grad_x)\n",
    "print(grad_y)\n",
    "print(grad_z)\n",
    "print(datetime.now()-start)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tensoflow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 187
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 670,
     "status": "ok",
     "timestamp": 1566734233074,
     "user": {
      "displayName": "Gunho Choi",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AAuE7mBSqM0cFU5ZceEDo5gH8hXfSGh8z5BVTu5HgREceg=s64",
      "userId": "04388737836176863066"
     },
     "user_tz": -540
    },
    "id": "Ieqmt7ZG-YQt",
    "outputId": "20eac205-c097-4221-bc70-618c8b7f92d8"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n",
      "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n",
      "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n",
      "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n",
      "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n",
      "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-1.6138978  -0.21274029 -0.89546657  0.3869025 ]\n",
      " [-0.51080513 -1.1806322  -0.02818223  0.42833188]\n",
      " [ 0.06651722  0.3024719  -0.6343221  -0.36274117]]\n",
      "[[ 1.2302907   1.2023798  -0.3873268  -0.30230275]\n",
      " [-1.048553   -1.420018   -1.7062702   1.9507754 ]\n",
      " [-0.5096522  -0.4380743  -1.2527953   0.7774904 ]]\n",
      "[[1. 1. 1. 1.]\n",
      " [1. 1. 1. 1.]\n",
      " [1. 1. 1. 1.]]\n",
      "0:00:00.061126\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n",
      "/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n",
      "/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n",
      "/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n",
      "/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n",
      "/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n"
     ]
    }
   ],
   "source": [
    "# 이번에는 텐서플로 프레임워크를 이용해 같은 연산을 해보도록 하겠습니다.\n",
    "from __future__ import absolute_import\n",
    "from __future__ import division\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "from datetime import datetime\n",
    "start = datetime.now()\n",
    "\n",
    "# 텐서플로는 연산 그래프를 먼저 정의하고 추후에 여기에 값을 전달하는 방식입니다. 여기서는 비어있는 그래프만 정의해줍니다.\n",
    "# Define Graph on GPU\n",
    "x = tf.placeholder(tf.float32)     # 비어있는 노드인 placeholder를 정의하고 여기에 들어가는 데이터타입을 명시 해놓습니다.\n",
    "y = tf.placeholder(tf.float32)\n",
    "z = tf.placeholder(tf.float32)\n",
    "\n",
    "a = x * y                          # 연산 과정 또한 정의해줍니다.\n",
    "b = a + z\n",
    "c = tf.reduce_sum(b)\n",
    "    \n",
    "grad_x, grad_y, grad_z = tf.gradients(c,[x,y,z])  # c에 대한 x,y,z의 기울기(gradient)를 구하고 이를 각각 grad_x, grad_y, grad_z에 저장하도록 지정해놓습니다.\n",
    "\n",
    "# 실제적인 계산이 이루어지는 부분. 텐서플로에서는 이를 세션이라고 합니다.\n",
    "with tf.Session() as sess:\n",
    "    values = {\n",
    "        x: np.random.randn(N,D),     # 여기서 실제 값들이 생성됩니다.\n",
    "        y: np.random.randn(N,D),\n",
    "        z: np.random.randn(N,D)           \n",
    "    }\n",
    "    out = sess.run([c,grad_x,grad_y,grad_z],feed_dict = values)  # 세션에서 실제로 값을 계산하는 부분입니다. feed_dict를 통해서 값들을 전달합니다.\n",
    "    c_val, grad_x_val, grad_y_val, grad_z_val = out\n",
    "\n",
    "# 값들을 확인하고 걸린 시간을 측정합니다.\n",
    "print(grad_x_val)\n",
    "print(grad_y_val)\n",
    "print(grad_z_val)\n",
    "print(datetime.now()-start)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pytorch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 187
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 895,
     "status": "ok",
     "timestamp": 1566734278629,
     "user": {
      "displayName": "Gunho Choi",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AAuE7mBSqM0cFU5ZceEDo5gH8hXfSGh8z5BVTu5HgREceg=s64",
      "userId": "04388737836176863066"
     },
     "user_tz": -540
    },
    "id": "5NsV7f5A_IHb",
    "outputId": "47739337-b3f6-4933-d18a-22247a5c73bb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 8.2338e-01, -5.6320e-01, -2.3355e-03,  3.7213e-01],\n",
      "        [ 1.0584e-01, -9.1745e-01,  1.8612e+00,  8.3635e-01],\n",
      "        [-1.9306e+00,  1.1427e+00,  7.5820e-01, -3.4195e-04]], device='cuda:0')\n",
      "tensor([[-2.4330, -0.3690, -0.5224,  0.4084],\n",
      "        [-0.3373,  0.7050,  0.7853,  0.2445],\n",
      "        [-0.5182, -1.0441,  0.5183, -0.5510]], device='cuda:0')\n",
      "tensor([[1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1.]], device='cuda:0')\n",
      "0:00:00.003761\n"
     ]
    }
   ],
   "source": [
    "# 이번에는 파이토치를 이용해 같은 연산을 진행해보도록 하겠습니다.\n",
    "import torch\n",
    "from datetime import datetime\n",
    "start = datetime.now()\n",
    "\n",
    "N,D = 3,4\n",
    "\n",
    "# x,y,z를 랜덤하게 초기화 해줍니다. \n",
    "# https://pytorch.org/docs/stable/torch.html?highlight=randn#torch.randn\n",
    "x = torch.randn(N,D,device=torch.device('cuda'), requires_grad=True)\n",
    "y = torch.randn(N,D,device=torch.device('cuda'), requires_grad=True)\n",
    "z = torch.randn(N,D,device=torch.device('cuda'), requires_grad=True)\n",
    "\n",
    "# 연산 그래프는 정의됨과 동시에 연산됩니다.\n",
    "a = x * y\n",
    "b = a + z\n",
    "c = torch.sum(b)\n",
    "\n",
    "# 기울기(gradient)가 1.0라고 가정하고 최종 값인 c에서 backward를 통해 역전파를 해줍니다.\n",
    "# 넘파이와 비교했을때 이 과정이 자동적으로 게산되는 것을 확인할 수 있습니다.\n",
    "c.backward(gradient=torch.cuda.FloatTensor([1.0]))\n",
    "\n",
    "# 각각의 기울기와 걸린 시간을 출력합니다.\n",
    "print(x.grad)\n",
    "print(y.grad)\n",
    "print(z.grad)\n",
    "print(datetime.now()-start)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pytorch 기본 동작 방법"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "X라는 변수에 파이토치 텐서를 하나 생성해서 지정, shape = (2,3)  \n",
    "텐서에는 임의의 난수가 들어간다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-3.4702e-01,  4.5702e-41, -3.4702e-01],\n",
      "        [ 4.5702e-41,  1.3563e-19,  4.5071e+16]])\n"
     ]
    }
   ],
   "source": [
    "X = torch.Tensor(2,3)\n",
    "print(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "텐서를 생성하면서 원하는 값으로 초기화 하려면 인수로 배열을 전달  \n",
    "\n",
    "torch.tensor함수는 인수로 data, dtype, device, requires_grad를 전달\n",
    "- data: tensor에 넣을 data => 배열로 전달\n",
    "- dtype: data의 타입 결정\n",
    "- requres_grad: 텐서에 대한 기울기를 저장할 지 여부 지정\n",
    "- device: 텐서를 어느 기기에 올릴 것인지 명시\n",
    "\n",
    "**dtype 종류**\n",
    "<table class=\"docutils align-default\">\n",
    "<colgroup>\n",
    "<col style=\"width: 19%\">\n",
    "<col style=\"width: 34%\">\n",
    "<col style=\"width: 21%\">\n",
    "<col style=\"width: 25%\">\n",
    "</colgroup>\n",
    "<thead>\n",
    "<tr class=\"row-odd\"><th class=\"head\"><p>Data type</p></th>\n",
    "<th class=\"head\"><p>dtype</p></th>\n",
    "<th class=\"head\"><p>CPU tensor</p></th>\n",
    "<th class=\"head\"><p>GPU tensor</p></th>\n",
    "</tr>\n",
    "</thead>\n",
    "<tbody>\n",
    "<tr class=\"row-even\"><td><p>32-bit floating point</p></td>\n",
    "<td><p><code class=\"docutils literal notranslate\"><span class=\"pre\">torch.float32</span></code> or <code class=\"docutils literal notranslate\"><span class=\"pre\">torch.float</span></code></p></td>\n",
    "<td><p><code class=\"xref py py-class docutils literal notranslate\"><span class=\"pre\">torch.FloatTensor</span></code></p></td>\n",
    "<td><p><code class=\"xref py py-class docutils literal notranslate\"><span class=\"pre\">torch.cuda.FloatTensor</span></code></p></td>\n",
    "</tr>\n",
    "<tr class=\"row-odd\"><td><p>64-bit floating point</p></td>\n",
    "<td><p><code class=\"docutils literal notranslate\"><span class=\"pre\">torch.float64</span></code> or <code class=\"docutils literal notranslate\"><span class=\"pre\">torch.double</span></code></p></td>\n",
    "<td><p><code class=\"xref py py-class docutils literal notranslate\"><span class=\"pre\">torch.DoubleTensor</span></code></p></td>\n",
    "<td><p><code class=\"xref py py-class docutils literal notranslate\"><span class=\"pre\">torch.cuda.DoubleTensor</span></code></p></td>\n",
    "</tr>\n",
    "<tr class=\"row-even\"><td><p>16-bit floating point</p></td>\n",
    "<td><p><code class=\"docutils literal notranslate\"><span class=\"pre\">torch.float16</span></code> or <code class=\"docutils literal notranslate\"><span class=\"pre\">torch.half</span></code></p></td>\n",
    "<td><p><code class=\"xref py py-class docutils literal notranslate\"><span class=\"pre\">torch.HalfTensor</span></code></p></td>\n",
    "<td><p><code class=\"xref py py-class docutils literal notranslate\"><span class=\"pre\">torch.cuda.HalfTensor</span></code></p></td>\n",
    "</tr>\n",
    "<tr class=\"row-odd\"><td><p>8-bit integer (unsigned)</p></td>\n",
    "<td><p><code class=\"docutils literal notranslate\"><span class=\"pre\">torch.uint8</span></code></p></td>\n",
    "<td><p><code class=\"xref py py-class docutils literal notranslate\"><span class=\"pre\">torch.ByteTensor</span></code></p></td>\n",
    "<td><p><code class=\"xref py py-class docutils literal notranslate\"><span class=\"pre\">torch.cuda.ByteTensor</span></code></p></td>\n",
    "</tr>\n",
    "<tr class=\"row-even\"><td><p>8-bit integer (signed)</p></td>\n",
    "<td><p><code class=\"docutils literal notranslate\"><span class=\"pre\">torch.int8</span></code></p></td>\n",
    "<td><p><code class=\"xref py py-class docutils literal notranslate\"><span class=\"pre\">torch.CharTensor</span></code></p></td>\n",
    "<td><p><code class=\"xref py py-class docutils literal notranslate\"><span class=\"pre\">torch.cuda.CharTensor</span></code></p></td>\n",
    "</tr>\n",
    "<tr class=\"row-odd\"><td><p>16-bit integer (signed)</p></td>\n",
    "<td><p><code class=\"docutils literal notranslate\"><span class=\"pre\">torch.int16</span></code> or <code class=\"docutils literal notranslate\"><span class=\"pre\">torch.short</span></code></p></td>\n",
    "<td><p><code class=\"xref py py-class docutils literal notranslate\"><span class=\"pre\">torch.ShortTensor</span></code></p></td>\n",
    "<td><p><code class=\"xref py py-class docutils literal notranslate\"><span class=\"pre\">torch.cuda.ShortTensor</span></code></p></td>\n",
    "</tr>\n",
    "<tr class=\"row-even\"><td><p>32-bit integer (signed)</p></td>\n",
    "<td><p><code class=\"docutils literal notranslate\"><span class=\"pre\">torch.int32</span></code> or <code class=\"docutils literal notranslate\"><span class=\"pre\">torch.int</span></code></p></td>\n",
    "<td><p><code class=\"xref py py-class docutils literal notranslate\"><span class=\"pre\">torch.IntTensor</span></code></p></td>\n",
    "<td><p><code class=\"xref py py-class docutils literal notranslate\"><span class=\"pre\">torch.cuda.IntTensor</span></code></p></td>\n",
    "</tr>\n",
    "<tr class=\"row-odd\"><td><p>64-bit integer (signed)</p></td>\n",
    "<td><p><code class=\"docutils literal notranslate\"><span class=\"pre\">torch.int64</span></code> or <code class=\"docutils literal notranslate\"><span class=\"pre\">torch.long</span></code></p></td>\n",
    "<td><p><code class=\"xref py py-class docutils literal notranslate\"><span class=\"pre\">torch.LongTensor</span></code></p></td>\n",
    "<td><p><code class=\"xref py py-class docutils literal notranslate\"><span class=\"pre\">torch.cuda.LongTensor</span></code></p></td>\n",
    "</tr>\n",
    "<tr class=\"row-even\"><td><p>Boolean</p></td>\n",
    "<td><p><code class=\"docutils literal notranslate\"><span class=\"pre\">torch.bool</span></code></p></td>\n",
    "<td><p><a class=\"reference internal has-code\" href=\"#torch.BoolTensor\" title=\"torch.BoolTensor\"><code class=\"xref py py-class docutils literal notranslate\"><span class=\"pre\">torch.BoolTensor</span></code></a></p></td>\n",
    "<td><p><code class=\"xref py py-class docutils literal notranslate\"><span class=\"pre\">torch.cuda.BoolTensor</span></code></p></td>\n",
    "</tr>\n",
    "</tbody>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1, 2, 3],\n",
      "        [4, 5, 6]])\n"
     ]
    }
   ],
   "source": [
    "# 텐서에 직접 값 대입\n",
    "X = torch.tensor([[1,2,3],[4,5,6]])\n",
    "print(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "연산 그래프 생성후 Gradient Descent 로 기울기 확인  \n",
    "x tensor에만 requires_grad = True로 설정하여 값 확인 가능"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([ 8., 12.]) None None\n"
     ]
    }
   ],
   "source": [
    "x = torch.tensor(data=[2.0,3.0],requires_grad=True)\n",
    "y = x**2\n",
    "z = 2*y + 3\n",
    "\n",
    "target = torch.tensor([3.0, 4.0])\n",
    "loss = torch.sum(torch.abs(z-target))\n",
    "loss.backward()\n",
    "\n",
    "print(x.grad, y.grad, z.grad)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "[참고] 파이토치와 텐서플로 비교.ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}