TF_Swift.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "TF Swift.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Swift",
      "language": "swift",
      "name": "swift"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/zaidalyafeai/Notebooks/blob/master/TF_Swift.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "metadata": {
        "id": "yaRlEIvpBamB",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "<center>![alt text](https://pbs.twimg.com/media/D1AKlRnWsAAnyP7.png)</center>\n",
        "\n",
        "Swift for TensorFlow is a next generation platform for deep learning and differentiable programming.\n",
        "\n",
        "In general Swift for TF tries to resolve the drawbacks of Python\n",
        "\n",
        "*   Performance\n",
        "*   Concurrency\n",
        "*   Deployment\n",
        "*   Custom Ops\n",
        "\n",
        "Swift has many advantages \n",
        "\n",
        "*   First-class autodiff\n",
        "*   Next-generation APIs\n",
        "*   High-quality tooling\n",
        "*   Python Integerability\n",
        "*   Performance of graphs + flexibility  of Eager execution\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "IRoey45xwv0w",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# SWIFT"
      ]
    },
    {
      "metadata": {
        "id": "OihtZA6QhEZS",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Constants and Variables"
      ]
    },
    {
      "metadata": {
        "id": "p_pjyevTdE18",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "let x = 2.0 //constant\n",
        "var y = 2   //variable\n",
        "\n",
        "\n",
        "//define types inline \n",
        "let w:Double = 2.0\n",
        "var z:Float = 3.0"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "j-W3HyT9ZBOF",
        "colab_type": "code",
        "outputId": "b156336f-f610-436c-b88d-333870b30084",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "\"I have \\(y) apples.\""
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\"I have 2 apples.\"\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 2
        }
      ]
    },
    {
      "metadata": {
        "id": "yTka2hG9lM_D",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Arrays and Dictionaries \n",
        "\n",
        "Initialize arrays and dictionaries"
      ]
    },
    {
      "metadata": {
        "id": "BI46PGfQdE2I",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "var numbersArry = [1, 2]\n",
        "var numbersDict = [\"one\":1, \"two\":2]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "4l-kIpID_x9N",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "You can also define empty arrays and dictionaries"
      ]
    },
    {
      "metadata": {
        "id": "pZ0hFNzXdE2M",
        "colab_type": "code",
        "outputId": "b4f4990b-4078-47f7-ad33-603c6914f49a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "cell_type": "code",
      "source": [
        "var langs = [String]()\n",
        "\n",
        "langs.append(\"Python\")\n",
        "langs.append(\"Csharp\")\n",
        "langs.append(\"Buisct\") //not sure if that exists \n",
        "langs"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "▿ 3 elements\n",
              "  - 0 : \"Python\"\n",
              "  - 1 : \"Csharp\"\n",
              "  - 2 : \"Buisct\"\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 64
        }
      ]
    },
    {
      "metadata": {
        "id": "5anS2uUesaiV",
        "colab_type": "code",
        "outputId": "e43b8398-8586-4e3b-d280-91f8173922c4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 136
        }
      },
      "cell_type": "code",
      "source": [
        "var gpaStudent  = [String:Float]()\n",
        "\n",
        "gpaStudent[\"Zaid\"] = 3.8\n",
        "gpaStudent[\"Zyan\"] = 3.9\n",
        "gpaStudent"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "▿ 2 elements\n",
              "  ▿ 0 : 2 elements\n",
              "    - key : \"Zaid\"\n",
              "    - value : 3.8\n",
              "  ▿ 1 : 2 elements\n",
              "    - key : \"Zyan\"\n",
              "    - value : 3.9\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 65
        }
      ]
    },
    {
      "metadata": {
        "id": "IcyJIWOqdE2T",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Loops"
      ]
    },
    {
      "metadata": {
        "id": "QhcmCbQydE2U",
        "colab_type": "code",
        "outputId": "2801dd46-e8f2-402c-86e3-66556492b0bd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "let numbers = [1, 2, 3, 4]\n",
        "\n",
        "for x in numbers {\n",
        "  if x > 2{\n",
        "    print(\"\\(x) > 2\")\n",
        "  }\n",
        "}"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "3 > 2\r\n",
            "4 > 2\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "tt54-NuJdE2m",
        "colab_type": "code",
        "outputId": "799c1086-374f-4157-8ef4-7911428d06e0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "var n = 2\n",
        "while n < 100 {\n",
        "    n = n * 2\n",
        "}\n",
        "\n",
        "n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "128\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 67
        }
      ]
    },
    {
      "metadata": {
        "id": "LKkJ4co_dE2r",
        "colab_type": "code",
        "outputId": "b1c8bdb9-8e3a-40f1-e728-4387362bdfb4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "var total = 0\n",
        "for i in 0...4 {\n",
        "    total += i\n",
        "}\n",
        "\n",
        "total"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "10\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 68
        }
      ]
    },
    {
      "metadata": {
        "id": "W7H3ptGaiEcB",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Functions\n",
        "\n",
        "Here we define a function that adds two integers. By using _ we don't have to use the variable names for function calls "
      ]
    },
    {
      "metadata": {
        "id": "sgYex6gigH87",
        "colab_type": "code",
        "outputId": "53817cce-297f-4b89-abd7-a6a6d48e6b22",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "func add(_ x: Int , _ y: Int) -> Int{\n",
        "  return x + y\n",
        "}\n",
        "print(add(2, 3))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "5\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "aAdSAMNmHGP-",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "You can also add labels to add make the function call more expressive "
      ]
    },
    {
      "metadata": {
        "id": "_GmR2pwxHNGq",
        "colab_type": "code",
        "outputId": "80e771d3-004d-4f54-ba87-1401c19d64da",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "cell_type": "code",
      "source": [
        "func range(from start:Int , to end:Int) -> [Int]{\n",
        "  var out = [Int]()\n",
        "  \n",
        "  for num in start...end{\n",
        "    out.append(num)\n",
        "  }\n",
        "  return out\n",
        "}\n",
        "\n",
        "range(from:2, to:5)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "▿ 4 elements\n",
              "  - 0 : 2\n",
              "  - 1 : 3\n",
              "  - 2 : 4\n",
              "  - 3 : 5\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 62
        }
      ]
    },
    {
      "metadata": {
        "id": "pifd8IZzH4mg",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "You can also return many multiple values from a function"
      ]
    },
    {
      "metadata": {
        "id": "wuDe6FtwXP58",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "func mean_variance(scores: [Double]) -> (mean: Double, variance: Double){\n",
        "  \n",
        "  var mean: Double = 0\n",
        "  var variance: Double = 0\n",
        "  let count: Double = Double(scores.count)\n",
        "  \n",
        "  for x in scores{\n",
        "    mean += (x/count)\n",
        "  }\n",
        "  \n",
        "  for x in scores{\n",
        "    variance += (x - mean)*(x - mean)/count\n",
        "  }\n",
        "  \n",
        "  return (mean, variance)\n",
        "}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "cgCxSLrBaBtV",
        "colab_type": "code",
        "outputId": "7bb6eecf-447a-4b66-91ed-349f9a3b1b44",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "cell_type": "code",
      "source": [
        "mean_variance(scores: [2.0 , 3.0 , 4.0])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "▿ 2 elements\n",
              "  - mean : 3.0\n",
              "  - variance : 0.6666666666666666\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 71
        }
      ]
    },
    {
      "metadata": {
        "id": "lLPjckUyOcFk",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Use `inout` to update the value of a variable and use `&` to pass by reference "
      ]
    },
    {
      "metadata": {
        "id": "jF9jqUjxObp7",
        "colab_type": "code",
        "outputId": "68718fc1-4eba-4c34-89d1-69b16e4a9902",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "func increment(_ x:inout Int){\n",
        "  x = x + 1\n",
        "}\n",
        "\n",
        "var x:Int = 0\n",
        "increment(&x)\n",
        "\n",
        "print(x)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "BfV1d5HpdE3K",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Classes\n"
      ]
    },
    {
      "metadata": {
        "id": "mzyk2iV7atrl",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class Student {\n",
        "  var name: String = \"\"\n",
        "  var age : Int = 0 \n",
        "  \n",
        "  init(name: String, age: Int){\n",
        "    self.name = name\n",
        "    self.age = age \n",
        "  }\n",
        "}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "0N8uUPD1bMBo",
        "colab_type": "code",
        "outputId": "693733d5-d5da-4b9e-f756-e151fdca06b7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "let person = Student(name:\"Zaid\", age: 3)\n",
        "print(person.name)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Zaid\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "2dHlqpeQSfvX",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Protocols\n",
        "\n",
        "Protocols are just like interfaces where you can extend them using a class"
      ]
    },
    {
      "metadata": {
        "id": "T419e6qcTpYZ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "protocol Person{\n",
        "  func getWage()-> Int\n",
        "}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "ItBzguGBUpv1",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Any function that extends the protcol must implement the rules of the protocol"
      ]
    },
    {
      "metadata": {
        "id": "Acn4N3MAT6Se",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class Worker:Person {\n",
        "  var name: String = \"\"\n",
        "  var age : Int = 0 \n",
        "  \n",
        "  init(name: String, age: Int){\n",
        "    self.name = name\n",
        "    self.age = age \n",
        "  }\n",
        "  \n",
        "  func getWage() -> Int{\n",
        "    return self.age * 10\n",
        "  }\n",
        "}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Sb_SZiplUO6N",
        "colab_type": "code",
        "outputId": "f8a7fa25-3695-4c04-d017-06377cf30f49",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "let p = Worker(name:\"Zaid\", age: 3)\n",
        "p.getWage()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "30\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 33
        }
      ]
    },
    {
      "metadata": {
        "id": "OuMb_W7SrPXT",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Structs and Extensions "
      ]
    },
    {
      "metadata": {
        "id": "0fPJQwSArYCI",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "struct Complex{\n",
        "  var real:Float\n",
        "  var imag:Float\n",
        "}\n",
        "\n",
        "Complex(real:3 , imag: 2)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "IAEOl7xkvVc-",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "You can also extend struct properties"
      ]
    },
    {
      "metadata": {
        "id": "ZvztduYtrrcG",
        "colab_type": "code",
        "outputId": "b50b6f64-48aa-4153-bbe7-1ff5c5e550c6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "extension Complex{\n",
        "  func getReal() -> Float{\n",
        "    return real\n",
        "  }\n",
        "}\n",
        "\n",
        "var a = Complex(real:3 , imag: 2)\n",
        "print(a.getReal())"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "3.0\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "2trelwkSU5j3",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Closures \n",
        "\n",
        "According to Swift documentation \"*Closures are self-contained blocks of functionality that can be passed around and used in your code*\". \n",
        "Let us create a closure where we square an integer input"
      ]
    },
    {
      "metadata": {
        "id": "3WIjqV_HYWlw",
        "colab_type": "code",
        "outputId": "34824edb-c0b3-4a8c-a80f-39c74c7afa76",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "var squareClosure  = {\n",
        "  (num:Int) -> Int in \n",
        "  return num*num\n",
        "}\n",
        "\n",
        "squareClosure(2)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "4\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 41
        }
      ]
    },
    {
      "metadata": {
        "id": "BrqDWuebavjX",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Closures can be used as inputs to functions"
      ]
    },
    {
      "metadata": {
        "id": "kwGSDYAHayUY",
        "colab_type": "code",
        "outputId": "44a623ee-e8c1-4715-bcb9-1306d9c20621",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "cell_type": "code",
      "source": [
        "func squareNumbers(numbers:[Int], closure:(_ num:Int) -> Int) -> [Int]{\n",
        "  \n",
        "  var result = [Int]()\n",
        "  \n",
        "  for num in numbers{\n",
        "    result.append(closure(num))\n",
        "  }\n",
        "  \n",
        "  return result\n",
        "}\n",
        "\n",
        "squareNumbers(numbers:[2, 3, 4], closure:squareClosure)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "▿ 3 elements\n",
              "  - 0 : 4\n",
              "  - 1 : 9\n",
              "  - 2 : 16\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 44
        }
      ]
    },
    {
      "metadata": {
        "id": "TwVeTx3pL1y5",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "`map` accepts closures and applies the closure to a sequence of elements "
      ]
    },
    {
      "metadata": {
        "id": "llJzEDAcLXRN",
        "colab_type": "code",
        "outputId": "b72d371b-aad0-4923-9633-74069dd6761e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "cell_type": "code",
      "source": [
        "var numbers = [1, 2, 3, 4]\n",
        "\n",
        "numbers.map({(num:Int) in \n",
        "  return num * num            \n",
        "})"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "▿ 4 elements\n",
              "  - 0 : 1\n",
              "  - 1 : 4\n",
              "  - 2 : 9\n",
              "  - 3 : 16\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 83
        }
      ]
    },
    {
      "metadata": {
        "id": "2dXDTcVgM0aj",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Operator Overloading"
      ]
    },
    {
      "metadata": {
        "id": "3r_t9dTRM3WA",
        "colab_type": "code",
        "outputId": "8a8beaae-170f-4e0f-bc90-d3f96bfa4439",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "struct Vector2D {\n",
        "    var x = 0.0, y = 0.0\n",
        "}\n",
        "\n",
        "extension Vector2D {\n",
        "    static func +(left: Vector2D, right: Vector2D) -> Vector2D {\n",
        "        return Vector2D(x: left.x + right.x, y: left.y + right.y)\n",
        "    }\n",
        "  \n",
        "    static prefix func -(arg: Vector2D) -> Vector2D{\n",
        "      return Vector2D(x: -arg.x, y: -arg.y)\n",
        "    }\n",
        "}\n",
        "\n",
        "var w = Vector2D(x:1.0, y:1.5)\n",
        "var z = Vector2D(x:1.2, y:4.0)\n",
        "\n",
        "print(-w)\n",
        "print(w + z)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Vector2D(x: -1.0, y: -1.5)\r\n",
            "Vector2D(x: 2.2, y: 5.5)\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "BCpWt8IYJbPi",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Generics\n",
        "\n",
        "Generics are used to construct generic types. Use `<>` to represent a generic type "
      ]
    },
    {
      "metadata": {
        "id": "hzqlVzFeJizO",
        "colab_type": "code",
        "outputId": "032f3ada-c266-4e05-8490-c7c5bbe6f3f2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 187
        }
      },
      "cell_type": "code",
      "source": [
        "//convert a list of generic type to a dictionary of generic type\n",
        "func makeDict<T>(_ array: [T]) -> [T:T] {\n",
        "  var out = [T:T]()\n",
        "  \n",
        "  \n",
        "  for a in array{\n",
        "    out[a] = a\n",
        "  }\n",
        "  return out\n",
        "}\n",
        "\n",
        "makeDict([2, 3, 4])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "▿ 3 elements\n",
              "  ▿ 0 : 2 elements\n",
              "    - key : 3\n",
              "    - value : 3\n",
              "  ▿ 1 : 2 elements\n",
              "    - key : 4\n",
              "    - value : 4\n",
              "  ▿ 2 : 2 elements\n",
              "    - key : 2\n",
              "    - value : 2\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 74
        }
      ]
    },
    {
      "metadata": {
        "id": "ifoYXlSLc9np",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Python Integeration\n",
        "\n",
        "It is easy to call python modules in Swift"
      ]
    },
    {
      "metadata": {
        "id": "GKS7nCmcb1Bo",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import Python\n",
        "let np = Python.import(\"numpy\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "8RHA4_u6dc9S",
        "colab_type": "code",
        "outputId": "1812af7e-6621-437d-fb9c-06f007e1a622",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "np.exp(0)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.0\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 76
        }
      ]
    },
    {
      "metadata": {
        "id": "WK9y9eKfaab6",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Differentiation\n",
        "\n",
        "We can easily define a differentiable function by using `@differentiable`"
      ]
    },
    {
      "metadata": {
        "id": "N1zx_B_VaeRd",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "@differentiable\n",
        "func frac(_ x:Double) -> Double{\n",
        "  return 1/x\n",
        "}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "E4jvO7Rratfh",
        "colab_type": "code",
        "outputId": "a06208fd-f645-4e96-97e6-8249eb4444d7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "gradient(at:0.5) {x in frac(x)}"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "-4.0\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 78
        }
      ]
    },
    {
      "metadata": {
        "id": "DIg0cI_yvaCJ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# TenosrFlow"
      ]
    },
    {
      "metadata": {
        "id": "YmV1Nk_Nvd2g",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import TensorFlow"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "y_n538Im0_bY",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Neural Network (NN)\n",
        "\n",
        "We create a simple XOR classifier example using NN"
      ]
    },
    {
      "metadata": {
        "id": "p8GpL0gRlZUY",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Tensors"
      ]
    },
    {
      "metadata": {
        "id": "w7Nq3qUzmLXc",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "var X: Tensor<Float> = [[0.0, 0.0], [1.0, 0.0], [0.0, 1.0], [1.0, 1.0]]\n",
        "var y: Tensor<Int32> = [0, 1, 1, 0]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "W2sICW4PlbsG",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Model\n",
        "\n",
        "Create a nerual network with one hidden layer"
      ]
    },
    {
      "metadata": {
        "id": "Wk4fHfPun9MS",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "let hiddenSize: Int = 10\n",
        "\n",
        "struct NN: Layer {\n",
        "    var layer1 = Dense<Float>(inputSize: 2, outputSize: hiddenSize, activation: relu)\n",
        "    var layer2 = Dense<Float>(inputSize: hiddenSize, outputSize: 2)\n",
        "    \n",
        "    @differentiable\n",
        "    func applied(to input: Tensor<Float>, in context: Context) -> Tensor<Float> {\n",
        "        return input.sequenced(in: context, through: layer1, layer2)\n",
        "    }\n",
        "}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Qx5fu_JfpPpz",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "let optimizer = SGD<NN, Float>(learningRate: 0.01)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "4t1JAQf8ldme",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Training"
      ]
    },
    {
      "metadata": {
        "id": "e9zlOYUzpBt6",
        "colab_type": "code",
        "outputId": "e5361e5d-1ca0-4b32-8ba8-c51a0f7e36e3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "cell_type": "code",
      "source": [
        "var model = NN()\n",
        "\n",
        "let trainingContext = Context(learningPhase: .training)\n",
        "\n",
        "for i in 0...1000 {\n",
        "  \n",
        "  //calculate the loss and gradient\n",
        "  let (loss, grads) = model.valueWithGradient { model -> Tensor<Float> in\n",
        "      let logits = model.applied(to: X, in: trainingContext)\n",
        "      return softmaxCrossEntropy(logits: logits, labels: y)\n",
        "  }\n",
        "\n",
        "  //make an optimizer step \n",
        "  optimizer.update(&model.allDifferentiableVariables, along: grads)\n",
        "\n",
        "  if i % 100 == 0 {\n",
        "    print(\"epoch \\(i) train_loss: \\(loss)\")\n",
        "  }\n",
        "  \n",
        "}"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "epoch 0 train_loss: 0.69633675\n",
            "epoch 100 train_loss: 0.626324\n",
            "epoch 200 train_loss: 0.595307\n",
            "epoch 300 train_loss: 0.56721175\n",
            "epoch 400 train_loss: 0.5392669\n",
            "epoch 500 train_loss: 0.5136224\n",
            "epoch 600 train_loss: 0.49167603\n",
            "epoch 700 train_loss: 0.47081134\n",
            "epoch 800 train_loss: 0.45074087\n",
            "epoch 900 train_loss: 0.43119302\n",
            "epoch 1000 train_loss: 0.41217735\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "1M0r-jWelg-4",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Inference"
      ]
    },
    {
      "metadata": {
        "id": "uuOb9lVrtS-K",
        "colab_type": "code",
        "outputId": "fb9a19be-8519-480c-c15b-c0685860cd63",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "let inferenceContext = Context(learningPhase: .inference)\n",
        "\n",
        "// Apply the model to a batch of features.\n",
        "let pred = model.applied(to: X, in: inferenceContext)\n",
        "\n",
        "print(pred.argmax(squeezingAxis: 1))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[0, 1, 1, 0]\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "Wb8UGSnx1Ggi",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Convolutional Neural Network (CNN)\n",
        "\n",
        "We create a simple classifier using a CNN "
      ]
    },
    {
      "metadata": {
        "id": "DgBXEcll9f6D",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Tensors"
      ]
    },
    {
      "metadata": {
        "id": "lb6tS1VB2UYJ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "let X = Tensor<Float>(randomNormal: [10, 16, 16, 3])\n",
        "let y = Tensor<Int32>(rangeFrom: 0, to: 10, stride: 1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "PHWlUQR_9iH0",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Model"
      ]
    },
    {
      "metadata": {
        "id": "06xIZqC2yW6j",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "struct CNN: Layer {\n",
        "    var conv = Conv2D<Float>(filterShape: (5, 5, 3, 16)) //(w,h,c,out)\n",
        "    var maxPool = MaxPool2D<Float>(poolSize: (2, 2), strides: (2, 2))\n",
        "    var flatten = Flatten<Float>()\n",
        "    var dense = Dense<Float>(inputSize: flatten., outputSize: 10)\n",
        "\n",
        "    @differentiable\n",
        "    func applied(to input: Tensor<Float>, in context: Context) -> Tensor<Float> {\n",
        "        return input.sequenced(in: context, through: conv, maxPool, flatten, dense)\n",
        "    }\n",
        "}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "G_SdQty57RVT",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "let optimizer = SGD<CNN, Float>(learningRate: 0.01)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "kQkgRe-S9kqF",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Training"
      ]
    },
    {
      "metadata": {
        "id": "16RtMsFJ34-E",
        "colab_type": "code",
        "outputId": "de67b138-7e8f-45f3-f81c-1b068f1a0a6e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "cell_type": "code",
      "source": [
        "var model = CNN()\n",
        "\n",
        "let trainingContext = Context(learningPhase: .training)\n",
        "\n",
        "for i in 0...100 {\n",
        "  \n",
        "  //calculate the loss and gradient\n",
        "  let (loss, grads) = model.valueWithGradient { model -> Tensor<Float> in\n",
        "      let logits = model.applied(to: X, in: trainingContext)\n",
        "      return softmaxCrossEntropy(logits: logits, labels: y)\n",
        "  }\n",
        "\n",
        "  //make an optimizer step \n",
        "  optimizer.update(&model.allDifferentiableVariables, along: grads)\n",
        "\n",
        "  if i % 10 == 0 {\n",
        "    print(\"epoch \\(i) train_loss: \\(loss)\")\n",
        "  }\n",
        "  \n",
        "}"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "epoch 0 train_loss: 2.862069\n",
            "epoch 10 train_loss: 0.9227737\n",
            "epoch 20 train_loss: 0.3801387\n",
            "epoch 30 train_loss: 0.21386012\n",
            "epoch 40 train_loss: 0.14373973\n",
            "epoch 50 train_loss: 0.10649824\n",
            "epoch 60 train_loss: 0.08388945\n",
            "epoch 70 train_loss: 0.068893135\n",
            "epoch 80 train_loss: 0.05833138\n",
            "epoch 90 train_loss: 0.050462723\n",
            "epoch 100 train_loss: 0.044379223\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "Snub2EZdCKKb",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# References\n",
        "\n",
        "\n",
        "\n",
        "*   https://github.com/tensorflow/swift/blob/master/docs/WhySwiftForTensorFlow.md\n",
        "*   https://www.tensorflow.org/swift\n",
        "*   https://www.tensorflow.org/swift/tutorials/model_training_walkthrough\n",
        "\n"
      ]
    }
  ]
}