diff --git a/In_Db2_Machine_Learning/Deploying External Models with Python UDF/Db2 RESTful External Py UDF.ipynb b/In_Db2_Machine_Learning/Deploying External Models with Python UDF/Db2 RESTful External Py UDF.ipynb
new file mode 100644
index 0000000..c2a34e5
--- /dev/null
+++ b/In_Db2_Machine_Learning/Deploying External Models with Python UDF/Db2 RESTful External Py UDF.ipynb
@@ -0,0 +1,1103 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Create Db2 RESTful Services for a Db2 Hosted ML Model"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Contents:\n",
+ "* [1. Introduction](#Introduction)\n",
+ "* [2. Prerequisites ](#Prerequisites )\n",
+ "* [3. Finding the Db2 RESTful Endpoint Service API Documentation](#Endpoint-doc)\n",
+ "* [4. Import the required programming libraries](#libraries)\n",
+ "* [5. RESTful Host](#RESTfulHost)\n",
+ "* [6. Define grant_auth Function](#grant-auth)\n",
+ "* [7. Get Tokens for Creating and Execluting Services](#get-tokens)\n",
+ "* [8. Create Service Accepting Model Features from App as JSON](#Create-App-Features)\n",
+ "* [9. Execute Service Accepting Model Features from App as JSON](#Exec-App-Features)\n",
+ "* [10. Create Service Accepting Model Features from Db2 Table](#Create-Db2-Features)\n",
+ "* [11. Execute Service Accepting Model Features from Db2 Table](#Exec-Db2-Features)\n",
+ "* [12. Service Utility Examples](#Util-Examples)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# 1. Introduction "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "This notebook provides examples of creating REST services that allow machine learning models hosted in Db2 as external User Defined Functions to be called as REST services. Frequently developers will prefer to call other services with a REST API rather than importing Db2 libraries and writing SQL. These examples are for applications that need to get an on line prediction for a single record. This is different than the example on which this notebook is based as that example does a batch of predicitons on a set of inputs. \n",
+ "\n",
+ "This notebook addresses two use cases:\n",
+ "- The application has the set of features to be evaluated\n",
+ "- The features to be evaluated are in a table in Db2\n",
+ "\n",
+ "The example for the first use case allows the applicaiton to pass the many features as one input parameter in the form of a JSON document. It uses the Db2 JSON_Table function to extract the column values to input to the UDF. This is much easier than having a bunch of input parameters. \n",
+ "\n",
+ "The example for the second use case has the service getting the features from a row in a table already in the database. \n",
+ "\n",
+ "After each service is created, it is executed using he authorities of the service_user role. \n",
+ "\n",
+ "The \"Db2 RESTful Endpoint Get Token Notebook\" notebook is called to provide authentication tokens for the \" service admin\" and \"service user\" users. The notebook expects the \"usertype\" variable to be set and returns the \"token\" string containing the authentication token. \n",
+ "\n",
+ "Each endpoint is associated with a single UDF call. Authenticated users of web, mobile, or cloud applications can use these REST endpoints from any REST HTTP client without having to install any Db2 drivers.\n",
+ "\n",
+ "This notebook is used as example for the db2Dean article for \n",
+ "\n",
+ "You can find more information about this service at: https://www.ibm.com/support/producthub/db2/docs/content/SSEPGG_11.5.0/com.ibm.db2.luw.admin.rest.doc/doc/c_rest.html."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# 2. Prerequisites "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "#### You need to do the following before this notebook will work:\n",
+ "\n",
+ "- Create the UDF and other objects shown in the Deploying External Models with Python UDF Repo. The table described in step 4 or the repo readme is defined in the Db2 RESTful Prep External Py UDF.ipynb.ipynb notebook. https://github.com/IBM/db2-samples/tree/master/In_Db2_Machine_Learning/Deploying%20External%20Models%20with%20Python%20UDF\n",
+ "- Create those objects in the EXTPY schema\n",
+ "- Create the table described in step 4 as extpy.person_features (DDL is also in the Db2 RESTful Prep External Py UDF.ipynb notebook)\n",
+ "- Create users called service_user1 and service_admin1 on the database server.\n",
+ "- Execute the grants and other steps in the Db2 RESTful Prep External Py UDF.ipynb.ipynb notebook. Make sure to change the database connection and credentials to those of your database.\n",
+ "- Put the Db2 RESTful Get Token for External Py UDF.ipynb in the same directory as this notebook. It will be used to get tokes for the two user ids.\n",
+ "- Change the database connection information user ids and passwords in the Db2 RESTful Get Token for External Py UDF.ipynb notebook to those of your database before executing this notebook.\n",
+ "- Create a running Db2 REST Endpoint: https://www.ibm.com/docs/en/db2/11.5?topic=applications-rest-endpoints"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# 3. Finding the Db2 RESTful Endpoint Service API Documentation "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The APIs used in this notebook are documented in the container for the endpoint. If you are running a browser on the host containing the container, you can view the documentation using \"localhost\" host name. If that is your case then you can view the documentaiton by pasting this URL into your browser: https://localhost:50050/docs Otherwise, you would substitute the remote IP or host name if the container is on another host. You would also change https to http if you are running the service in http mode."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# 4. Import the required programming libraries "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The requests library is the minimum required by Python to construct RESTful service calls. The Pandas library is used to format and manipulate JSON result sets as tables. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import requests\n",
+ "import pandas as pd"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# 5. RESTful Host "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The next part defines where the request is sent to. It provides the location of the RESTful service for our calls. In my case I was running this notebook on the same machine as the REST Endpoint container was running. If you are on a different host you would need to replace \"localhost\" with the actual host name or IP. Also if you are running your service as https, you would need to change http to https.\n",
+ "\n",
+ "#### Change the value to the the URL of your Db2 REST Endpoint"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "Db2RESTful = \"http://localhost:50050\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# 6. Define grant_auth Function "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Define \"grant_auth\" Function to authroize the \"Service User\" Role to execute a service\n",
+ "In this case we are giving access to the database role \"SERVICE_USER\" so all users in that role can execute the service. The function has one input value that tells the function which service is being authorized."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def grant_auth(service_version):\n",
+ " API_grant = \"/v1/services/grant/\"+service_version\n",
+ " body = {\n",
+ " \"roles\": {\n",
+ " \"withGrantOption\": False, \n",
+ " \"names\": [\"SERVICE_USER\"]\n",
+ " }\n",
+ " }\n",
+ " \n",
+ " try:\n",
+ " response = requests.put(\"{}{}\".format(Db2RESTful,API_grant), headers=admin_headers, json=body)\n",
+ " except Exception as e:\n",
+ " print(\"Unable to authorize service. Error={}\".format(repr(e)))\n",
+ " \n",
+ " print(\"Response of 200 Means it works :\", response)\n",
+ " \n",
+ " return(response)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# 7. Get Tokens for Creating and Execluting Services "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Get Tokens for Creating and Execluting Services\n",
+ "Get tokens for both the Service Administrator and Service user roles. They will be used in creating and executing the services later. Calling the notebook that requests the tokens simulates a service that would authenticate a user and provide a token for the user's level of authority. The following syntax will look for the notebook in the same directory as this notebook"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Creating token for the service administrator.\n",
+ "Status of token request. 200=Success\n",
+ "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJhZG1pbiI6ZmFsc2UsImNsaWVudF9pZCI6IjIxZTAwZWNiLWY3NTEtNGMyMi04ZTlmLWFkNGIwNDI5NGZhYiIsImV4cCI6MTY3MDYyMjc2NCwiaXNzIjoic2VydmljZV9hZG1pbjEifQ.QK4Ca8dXtfbnmsBihtWco7OUFYsIYvw2iVWxh9pDNCzPirdmiGsi9VMwc1ornozC43euQkUkx7Pk6Y05ZvKuAIDxfOOpPc5BJChEVdASZEgWXTMYfyxRftVPcrJq1FvFFRhd7KFkTckf2zcKOyocFZ9DkXZLafvuEDbTEmVsrSitRl2XDYLQmLOSNfB7LjcC8chQSP8hiRUCbyapN6_KeP45nEoZf7nRG9F1Cqx9Wt22HZeVIqWNkyuFHCVqqb7m-23lWTV_veLMY0NE5KPn7jPzA8dhtLqjHUCAqrpbdy77wGy_5HlMVq7fwmVzMHh1OhAJLAH7_vz30JT8IoA3vQ\n"
+ ]
+ }
+ ],
+ "source": [
+ "usertype=\"admin\"\n",
+ "%run \"Db2 RESTful Get Token for External Py UDF.ipynb\"\n",
+ "\n",
+ "admin_headers = {\n",
+ " \"authorization\": token, \n",
+ " \"content-type\": \"application/json\"\n",
+ "}\n",
+ "\n",
+ "#print(admin_headers)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Creating token for the service user.\n",
+ "Status of token request. 200=Success\n",
+ "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJhZG1pbiI6ZmFsc2UsImNsaWVudF9pZCI6IjhiMWYzMTIwLTFmNWQtNDJiNC1iZDQ3LTM5M2RkMjJlZjE5ZSIsImV4cCI6MTY3MDYyMjc2NSwiaXNzIjoic2VydmljZV91c2VyMSJ9.MsnWl-vSMCE2VdEdu3UZuhIFUNenqTVkaBsA7bkloaTEwUQ_3UTwxrdOPb99xJ9MRAmQb6JRe2rWryZYXEaqNNotnNXDAHplCxCQ6UkOO6sUW5FycYTBtDGT-jCjSEPFYa7MGo-eB80_jIGFetRCaX_VOyXKnQEk5j1IfwNgkS3IZtkrG8GhJY_2sKeiGRAcrehUCla6-ZGMC-TrDq8YydGFNinssuidKgFuihyNAeiJU25PLQyHr3QMDlC71hzkV4wQk3vHMNrkHXZyqZWxiQv_OVjUmFDqq5EAniubpkDdIIEdDzm1zevKu3V5KjR0LTgITFdobQghU7TUnMpGgg\n"
+ ]
+ }
+ ],
+ "source": [
+ "usertype=\"user\"\n",
+ "%run \"Db2 RESTful Get Token for External Py UDF.ipynb\"\n",
+ "\n",
+ "service_headers = {\n",
+ " \"authorization\": token, \n",
+ " \"content-type\": \"application/json\"\n",
+ "}\n",
+ "\n",
+ "#print(admin_headers)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# * * * * Application Sends Features to the Service * * * *"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# 8. Create Service Accepting Model Features from App as JSON "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Create service to accept features from from JSON doc\n",
+ "For calling models with lots of features, it may be quite tedious to create an input parameter in the service for every feature that the service might need. One way to make this easier is to construct a Python Dictionary data type (JSON document) and pass it as a single parameter to the insert service. Db2 can use its JSON_TABLE function to extract the values from the document into the columns as shown in the following example. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In the next set of cells I show a way of creating the query that uses the JSON_TABLE function to extract the values. To get all of the quoting and special characters put into the string I construct, I build the final body over a few different cells. A more advance Python developer could probably do this in fewer steps, but at least this works. His a link to the Db2 documentation for the function: https://www.ibm.com/docs/en/db2/11.5?topic=functions-json-table"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Create query using the Service \"@FEATURESJSON\" input variable\n",
+ "strict= \"'strict $'\"\n",
+ "query= 'select EXTPY.predict_price( \\\n",
+ " U.\"female\" \\\n",
+ " , U.\"male\" \\\n",
+ " , U.\"married\" \\\n",
+ " , U.\"single\" \\\n",
+ " , U.\"unspecified\" \\\n",
+ " , U.\"executive\" \\\n",
+ " , U.\"hospitality\" \\\n",
+ " , U.\"other\" \\\n",
+ " , U.\"professional\" \\\n",
+ " , U.\"retail\" \\\n",
+ " , U.\"retured\" \\\n",
+ " , U.\"sales\" \\\n",
+ " , U.\"student\" \\\n",
+ " , U.\"trades\" \\\n",
+ " , U.\"camping_equipment\" \\\n",
+ " , U.\"golf_equipment\" \\\n",
+ " , U.\"mountaineering_equipment\" \\\n",
+ " , U.\"outdoor_protection\" \\\n",
+ " , U.\"personal_accessories\" \\\n",
+ " , U.\"age\" \\\n",
+ " , U.\"is_tent\") as predicted_value\\\n",
+ " FROM sysibm.sysdummy1 as E , JSON_TABLE(@FEATURESJSON,' + strict + ' COLUMNS ( \\\n",
+ " \"female\" float \\\n",
+ " , \"male\" float \\\n",
+ " , \"married\" float \\\n",
+ " , \"single\" float \\\n",
+ " , \"unspecified\" float \\\n",
+ " , \"executive\" float \\\n",
+ " , \"hospitality\" float \\\n",
+ " , \"other\" float \\\n",
+ " , \"professional\" float \\\n",
+ " , \"retail\" float \\\n",
+ " , \"retured\" float \\\n",
+ " , \"sales\" float \\\n",
+ " , \"student\" float \\\n",
+ " , \"trades\" float \\\n",
+ " , \"camping_equipment\" float \\\n",
+ " , \"golf_equipment\" float \\\n",
+ " , \"mountaineering_equipment\" float \\\n",
+ " , \"outdoor_protection\" float \\\n",
+ " , \"personal_accessories\" float \\\n",
+ " , \"age\" float \\\n",
+ " , \"is_tent\" float) \\\n",
+ " ERROR ON ERROR) AS U'\n",
+ "#print(query)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Create the \"predict_lr_ext_json\" service"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "{'isQuery': True, 'parameters': [{'datatype': 'CLOB', 'name': '@FEATURESJSON'}], 'schema': 'REST_SERVICES', 'serviceDescription': 'Call Python UDF for Linear Prediction', 'serviceName': 'predict_lr_ext_json', 'sqlStatement': 'select EXTPY.predict_price( U.\"female\" , U.\"male\" , U.\"married\" , U.\"single\" , U.\"unspecified\" , U.\"executive\" , U.\"hospitality\" , U.\"other\" , U.\"professional\" , U.\"retail\" , U.\"retured\" , U.\"sales\" , U.\"student\" , U.\"trades\" , U.\"camping_equipment\" , U.\"golf_equipment\" , U.\"mountaineering_equipment\" , U.\"outdoor_protection\" , U.\"personal_accessories\" , U.\"age\" , U.\"is_tent\") as predicted_value FROM sysibm.sysdummy1 as E , JSON_TABLE(@FEATURESJSON,\\'strict $\\' COLUMNS ( \"female\" float , \"male\" float , \"married\" float , \"single\" float , \"unspecified\" float , \"executive\" float , \"hospitality\" float , \"other\" float , \"professional\" float , \"retail\" float , \"retured\" float , \"sales\" float , \"student\" float , \"trades\" float , \"camping_equipment\" float , \"golf_equipment\" float , \"mountaineering_equipment\" float , \"outdoor_protection\" float , \"personal_accessories\" float , \"age\" float , \"is_tent\" float) ERROR ON ERROR) AS U', 'version': '1.0'}\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Notice that isQuery is set to true because a row will be returned from the service. \n",
+ "body = {\"isQuery\": True,\n",
+ " \"parameters\": [\n",
+ " {\n",
+ " \"datatype\": \"CLOB\",\n",
+ " \"name\": \"@FEATURESJSON\"\n",
+ " }\n",
+ " ],\n",
+ " \"schema\": \"REST_SERVICES\",\n",
+ " \"serviceDescription\": \"Call Python UDF for Linear Prediction\",\n",
+ " \"serviceName\": \"predict_lr_ext_json\",\n",
+ " \"sqlStatement\": query,\n",
+ " \"version\": \"1.0\"\n",
+ "}\n",
+ "print(body)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "API_makerest = \"/v1/services\""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "try:\n",
+ " response = requests.post(\"{}{}\".format(Db2RESTful,API_makerest), headers=admin_headers, json=body)\n",
+ "except Exception as e:\n",
+ " print(\"Unable to call RESTful service. Error={}\".format(repr(e)))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Service Created\n"
+ ]
+ }
+ ],
+ "source": [
+ "# A response of 400 frequently means that the service already exists\n",
+ "# and you need to delete it using the delete cells below.\n",
+ "# Certain SQL errors can cause a 400 error too.\n",
+ "print(response)\n",
+ "if (response.status_code == 201):\n",
+ " print(\"Service Created\")\n",
+ "else:\n",
+ " print(response.status_code)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Allow the service user to execute the service\n",
+ "In this case we are giving access to the database role \"SERVICE_USER\" so all users in that role can execute the service. See the definition of the grant_auth in a cell near the beginning of this notebook."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Response of 200 Means it works : \n"
+ ]
+ },
+ {
+ "data": {
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 11,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "service_and_version = \"predict_lr_ext_json/1.0\"\n",
+ "grant_auth(service_and_version)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# 9. Execute Service Accepting Model Features from App as JSON "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Execute the Service Providing Features in JSON to Get Prediction \n",
+ "Now you can call the RESTful service, passing a single JSON document to provide the column values. Note that the key/value pairs can be in any order, but the key names must match the key names that are used when the service is created. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "API_runrest = \"/v1/services/predict_lr_ext_json/1.0\""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Create a dictionary (JSON like type) of the fields to insert into the order header table\n",
+ "# Quote marks (any) and certain other secial characters witin any string fields will cause the insert to fail\n",
+ "features_dict= {'female': 1.0,\n",
+ " 'male': 0.0,\n",
+ " 'married': 0.0,\n",
+ " 'single': 1.0,\n",
+ " 'unspecified': 0.0,\n",
+ " 'executive': 0.0,\n",
+ " 'hospitality': 0.0,\n",
+ " 'other': 1.0,\n",
+ " 'professional': 0.0,\n",
+ " 'retail': 0.0,\n",
+ " 'retured': 0.0,\n",
+ " 'sales': 0.0,\n",
+ " 'student': 0.0,\n",
+ " 'trades': 0.0,\n",
+ " 'camping_equipment': 0.0,\n",
+ " 'golf_equipment': 0.0,\n",
+ " 'mountaineering_equipment': 0.0,\n",
+ " 'outdoor_protection': 0.0,\n",
+ " 'personal_accessories': 1.0,\n",
+ " 'age': 0.0961538461538462,\n",
+ " 'is_tent': 0.0}\n",
+ "\n",
+ "print(type(features_dict))\n",
+ "#print(features_dict)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Convert the dictionary to a string because we are passing the input parm as a Db2 CLOB\n",
+ "# Then replace the single quotes in the JSON document with doubles because Db2 likes \n",
+ "# doube quotes around keys and values\n",
+ "features_str=str(features_dict)\n",
+ "features_db2= features_str.replace(\"'\", '\"') \n",
+ "#print(features_db2)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "{'parameters': {'@FEATURESJSON': '{\"female\": 1.0, \"male\": 0.0, \"married\": 0.0, \"single\": 1.0, \"unspecified\": 0.0, \"executive\": 0.0, \"hospitality\": 0.0, \"other\": 1.0, \"professional\": 0.0, \"retail\": 0.0, \"retured\": 0.0, \"sales\": 0.0, \"student\": 0.0, \"trades\": 0.0, \"camping_equipment\": 0.0, \"golf_equipment\": 0.0, \"mountaineering_equipment\": 0.0, \"outdoor_protection\": 0.0, \"personal_accessories\": 1.0, \"age\": 0.0961538461538462, \"is_tent\": 0.0}'}, 'sync': True}\n"
+ ]
+ }
+ ],
+ "source": [
+ "body = {\n",
+ " \"parameters\": {\n",
+ " \"@FEATURESJSON\": features_db2\n",
+ " },\n",
+ " \"sync\": True\n",
+ "}\n",
+ "print(body)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "try:\n",
+ " response = requests.post(\"{}{}\".format(Db2RESTful,API_runrest), headers=service_headers, json=body)\n",
+ "except Exception as e:\n",
+ " print(\"Unable to call RESTful service. Error={}\".format(repr(e)))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "A response of 200 indicates a successful service call."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "{'jobStatus': 4, 'jobStatusDescription': 'Job is complete', 'resultSet': [{'PREDICTED_VALUE': 105.0625}], 'rowCount': 1}\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(response)\n",
+ "# Display the predicted value in a JSON document\n",
+ "print(response.json())"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# * * * * Application Calls Service Getting Features from Table * * * *"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# 10. Create Service Accepting Model Features from Db2 Table "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Create service to get features from a table\n",
+ "One reason you may be using Db2 to host your ML Models is that the data you use in the is already there. This example shows how to create a REST service to call the predict_price stored procedure that executes the model using data already in the database. This service expects the predict price procedure to have been created in the EXTPY schema and that the table described in Step 4 of the Deploying External Models with Python UDF Repo referenced in the prerequisites section of this notebook is created with the name extpy.person_features"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Create query using the Service \"@FEATURESJSON\" input variable\n",
+ "query = \"SELECT EXTPY.predict_price(FEMALE, MALE, MARRIED, SINGLE, UNSPECIFIED, \\\n",
+ " EXECUTIVE, HOSPITALITY, OTHER, PROFESSIONAL, \\\n",
+ " RETAIL, RETIRED, SALES, STUDENT, TRADES, \\\n",
+ " CAMPING_EQUIPMENT, GOLF_EQUIPMENT, \\\n",
+ " MOUNTAINEERING_EQUIPMENT, OUTDOOR_PROTECTION, \\\n",
+ " PERSONAL_ACCESSORIES, AGE, IS_TENT) \\\n",
+ " from extpy.person_features \\\n",
+ " where id = @PERSONID\"\n",
+ "\n",
+ "#print(query)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "##### Create the \"predict_lr_ext_tab\" service"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "{'isQuery': True, 'parameters': [{'datatype': 'INT', 'name': '@PERSONID'}], 'schema': 'REST_SERVICES', 'serviceDescription': 'Call Python UDF for Linear Prediction with Table Data', 'serviceName': 'predict_lr_ext_tab', 'sqlStatement': 'SELECT EXTPY.predict_price(FEMALE, MALE, MARRIED, SINGLE, UNSPECIFIED, EXECUTIVE, HOSPITALITY, OTHER, PROFESSIONAL, RETAIL, RETIRED, SALES, STUDENT, TRADES, CAMPING_EQUIPMENT, GOLF_EQUIPMENT, MOUNTAINEERING_EQUIPMENT, OUTDOOR_PROTECTION, PERSONAL_ACCESSORIES, AGE, IS_TENT) from extpy.person_features where id = @PERSONID', 'version': '1.0'}\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Notice that isQuery is set to true because a prediction will be returned from the service. \n",
+ "body = {\"isQuery\": True,\n",
+ " \"parameters\": [\n",
+ " {\n",
+ " \"datatype\": \"INT\",\n",
+ " \"name\": \"@PERSONID\"\n",
+ " }\n",
+ " ],\n",
+ " \"schema\": \"REST_SERVICES\",\n",
+ " \"serviceDescription\": \"Call Python UDF for Linear Prediction with Table Data\",\n",
+ " \"serviceName\": \"predict_lr_ext_tab\",\n",
+ " \"sqlStatement\": query,\n",
+ " \"version\": \"1.0\"\n",
+ "}\n",
+ "print(body)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "API_makerest = \"/v1/services\""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 21,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "try:\n",
+ " response = requests.post(\"{}{}\".format(Db2RESTful,API_makerest), headers=admin_headers, json=body)\n",
+ "except Exception as e:\n",
+ " print(\"Unable to call RESTful service. Error={}\".format(repr(e)))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 22,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Service Created\n"
+ ]
+ }
+ ],
+ "source": [
+ "# A response of 400 frequently means that the service already exists\n",
+ "# and you need to call a service to delete it using the delete cells below.\n",
+ "# Certain SQL errors can cause a 400 error too.\n",
+ "print(response)\n",
+ "if (response.status_code == 201):\n",
+ " print(\"Service Created\")\n",
+ "else:\n",
+ " print(response.json())"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### Allow the service user to execute the service\n",
+ "In this case we are giving access to the database role \"SERVICE_USER\" so all users in that role can execute the service. See the definition of the grant_auth in a cell near the beginning of this notebook."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 23,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Response of 200 Means it works : \n"
+ ]
+ },
+ {
+ "data": {
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 23,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "service_and_version = \"predict_lr_ext_tab/1.0\"\n",
+ "grant_auth(service_and_version)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# 11. Execute Service Accepting Model Features from Db2 Table "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Execute the Service with the person_id identifying the input row. \n",
+ "Now you can call the RESTful service, passing the id value that identifies the row you want to use in your prediction. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 24,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "API_runrest = \"/v1/services/predict_lr_ext_tab/1.0\""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 25,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# In a real application there would be logic to determine which person\n",
+ "# you need the prediction, but here we just pick the row where the ID column \n",
+ "# is 1\n",
+ "\n",
+ "person = 1\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 26,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "{'parameters': {'@PERSONID': 1}, 'sync': True}\n"
+ ]
+ }
+ ],
+ "source": [
+ "body = {\n",
+ " \"parameters\": {\n",
+ " \"@PERSONID\": person\n",
+ " },\n",
+ " \"sync\": True\n",
+ "}\n",
+ "print(body)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 27,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "try:\n",
+ " response = requests.post(\"{}{}\".format(Db2RESTful,API_runrest), headers=service_headers, json=body)\n",
+ "except Exception as e:\n",
+ " print(\"Unable to call RESTful service. Error={}\".format(repr(e)))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "A response of 200 indicates a successful service call."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 28,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "{'jobStatus': 4, 'jobStatusDescription': 'Job is complete', 'resultSet': [{'1': 105.0625}], 'rowCount': 1}\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(response)\n",
+ "# Display the predicted value in a JSON document\n",
+ "print(response.json())"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# 12. Service Utility Examples "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# * * * * SERVICE UTILITY EXAMPLES * * * *\n",
+ "The following are examples of endpoint utilities for administering services including delete, retrieve service details and listing services."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## List Available Services\n",
+ "You can also list all the user defined services you have access to"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "API_listrest = \"/v1/services\""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "try:\n",
+ " response = requests.get(\"{}{}\".format(Db2RESTful,API_listrest), headers=service_headers)\n",
+ "except Exception as e:\n",
+ " print(\"Unable to call RESTful service. Error={}\".format(repr(e)))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [],
+ "source": [
+ "display(pd.DataFrame(response.json()['Db2Services']))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Retreive Service Details\n",
+ "You can query each service to see its details, including authoritization, input parameters and output results. It is probably worthwile to give this informaiton to the developers who will call the service. Unless you give them direct access to select from the tables in the query, they will not be able to describe he service wiht this command."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "API_listrest = \"/v1/services/predict_lr_ext_json/1.0\"\n",
+ "#API_listrest = \"/v1/services/predict_lr_ext_tab/1.0\""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "try:\n",
+ " response = requests.get(\"{}{}\".format(Db2RESTful,API_listrest), headers=admin_headers)\n",
+ "except Exception as e:\n",
+ " print(\"Unable to call RESTful service. Error={}\".format(repr(e)))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "print(response.json())"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [],
+ "source": [
+ "print(\"Service Details:\")\n",
+ "print(\"Service Name: \" + response.json()['serviceName'])\n",
+ "print(\"Service Version: \" + response.json()['version'])\n",
+ "print(\"Service Description: \" + response.json()['serviceDescription'])\n",
+ "print(\"Service Creator: \" + response.json()['serviceCreator'])\n",
+ "print(\"Service Updater: \" + response.json()['serviceUpdater'])\n",
+ "\n",
+ "\n",
+ "print('Users:')\n",
+ "display(pd.DataFrame(response.json()['grantees']['users']))\n",
+ "print('Groups:')\n",
+ "display(pd.DataFrame(response.json()['grantees']['groups']))\n",
+ "print('Roles:')\n",
+ "display(pd.DataFrame(response.json()['grantees']['roles']))\n",
+ "\n",
+ "print('')\n",
+ "print('Input Parameters:')\n",
+ "display(pd.DataFrame(response.json()['inputParameters']))\n",
+ "\n",
+ "print('Result Set Fields:')\n",
+ "display(pd.DataFrame(response.json()['resultSetFields']))\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Delete a Service\n",
+ "A single call is also available to delete a service. Only delete the service when you are about to create a new one or no longer want the service."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "API_deleteService = \"/v1/services\"\n",
+ "#Service = \"/predict_lr_ext_json\"\n",
+ "Service = \"/predict_lr_ext_tab\"\n",
+ "Version = \"/1.0\""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "try:\n",
+ " response = requests.delete(\"{}{}{}{}\".format(Db2RESTful,API_deleteService,Service,Version), headers=admin_headers)\n",
+ "except Exception as e:\n",
+ " print(\"Unable to call RESTful service. Error={}\".format(repr(e)))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# A response of 204 indicates success.\n",
+ "if (response.status_code == 204):\n",
+ " print (response)\n",
+ " print(\"Service Deleted: \" + Service)\n",
+ "else:\n",
+ " print(response.json())"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Get Service Logs\n",
+ "You can easily download service logs. However you must be authorized as the principal administration user to do so."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "API_listrest = \"/v1/logs\""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "try:\n",
+ " response = requests.get(\"{}{}\".format(Db2RESTful,API_listrest), headers=admin_headers)\n",
+ "except Exception as e:\n",
+ " print(\"Unable to call RESTful service. Error={}\".format(repr(e)))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [],
+ "source": [
+ "if (response.status_code == 200):\n",
+ " myFile = response.content\n",
+ " open('/tmp/logs.zip', 'wb').write(myFile)\n",
+ " print(\"Downloaded\",len(myFile),\"bytes.\")\n",
+ "else:\n",
+ " print(response.json())"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.3"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/In_Db2_Machine_Learning/Deploying External Models with Python UDF/Db2 RESTful Get Token for External Py UDF.ipynb b/In_Db2_Machine_Learning/Deploying External Models with Python UDF/Db2 RESTful Get Token for External Py UDF.ipynb
new file mode 100644
index 0000000..d841563
--- /dev/null
+++ b/In_Db2_Machine_Learning/Deploying External Models with Python UDF/Db2 RESTful Get Token for External Py UDF.ipynb
@@ -0,0 +1,349 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Get the Token for Service Administrator or User"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "This notebook is called by the Db2 RESTful External Py UDF.ipynb notebook that needs a token. It creates a token for for use by a service administraor who creates and manages services or for use by users who call the services. In a real environment this applicaiton would do some real authentication of the user before providing a token.\n",
+ "\n",
+ "##### If you are looking at the REST example, make sure to start by examining the Db2 RESTful External Py UDF.ipynb notebook.\n",
+ "\n",
+ "### Expected Inputs, Outputs and Modifications\n",
+ "\n",
+ "##### INPUTS\n",
+ "usertype - Notebook expects this variable to be populated with a string depending on the level of authentication expected. It should have a value of \"admin\" or \"user\".\n",
+ "\n",
+ "##### OUTPUTS\n",
+ "token - If the token is successfully generated, then a string token will be placed in this variable and available to the calling notebook. If an error occurs then a value of \"TokenNotCreated\" is returned.\n",
+ "\n",
+ "##### MODIFICATIONS\n",
+ "In the modificaitons section below you must change the values for your database server's host, port and database name, as well as the credentials for the database administrator's and service user's userid and password"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### General\n",
+ "Each service is associated with a single SQL statement. Authenticated users of web, mobile, or cloud applications can use these REST endpoints from any REST HTTP client without having to install any Db2 drivers.\n",
+ "\n",
+ "The Db2 REST server accepts an HTTP request, processes the request body, and returns results in JavaScript Object Notation (JSON). \n",
+ "\n",
+ "This notebook is used as example for the db2Dean article for http://www.db2dean.com/Previous/Db2RESTdbUpdates.html\n",
+ "\n",
+ "You can find more information about this service at: https://www.ibm.com/support/producthub/db2/docs/content/SSEPGG_11.5.0/com.ibm.db2.luw.admin.rest.doc/doc/c_rest.html."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Finding the Db2 RESTful Endpoint Service API Documentation\n",
+ "The APIs used in this notebook are documented in the container for the endpoint. If you are running a browser on the host containing the container, you can view the documentation using \"localhost\" host name. If that is your case then you can view the documentaiton by pasting this URL into your browser: https://localhost:50050/docs Otherwise, you would substitute the remote IP or host name if the container is on another host. You would also change https to http if you are running the service in http mode."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Modifications\n",
+ "!!!! YOU MUST CHANGE THESE VALUES !!!\n",
+ "\n",
+ "Change the values of the variables below to reflect the correct connection informatin for your database and the correct credentials for the admin and service users. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "dbHost=\"192.168.0.72\"\n",
+ "dbName=\"SAMPLE\"\n",
+ "dbPort=50000\n",
+ "\n",
+ "ADMINusername=\"service_admin1\"\n",
+ "ADMINpassword=\"password\"\n",
+ "\n",
+ "USERusername=\"service_user1\"\n",
+ "USERpassword=\"password\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Import the required programming libraries\n",
+ "The requests library is the minimum required by Python to construct RESTful service calls. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import requests"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Create the Header File required for getting an authetication token \n",
+ "The RESTful call to the Db2 RESTful Endpoint service is contructed and transmitted as JSON. The first part of the JSON structure is the headers that define the content type of the request."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "headers = {\n",
+ " \"content-type\": \"application/json\"\n",
+ "}"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## RESTful Host\n",
+ "The next part defines where the request is sent to. It provides the location of the RESTful service for our calls. In my case I was running this notebook on the same machine as the REST Endpoint container was running. If you are on a different host you would need to replace \"localhost\" with the actual host name or IP. If you created your Endpoint container using https, then you need to previx the URL with that instead of http."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Use https instead of https if you installed the rest endpoint as https\n",
+ "Db2RESTful = \"http://localhost:50050\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## API Authentication Service\n",
+ "Each service has its own path in the RESTful call. For authentication we need to point to the `v1/auth` service."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "API_Auth = \"/v1/auth\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Authentication\n",
+ "To authenticate to the RESTful service you must provide the connection information for the database you want to query along with the database userid and password. You can also provide an expiry time so that the access token that gets returned will be invalidated after that time period. Note that the dbHost you specify here must be known inside of the Endpoint container. \n",
+ "\n",
+ "\n",
+ "### Determine type of user for token creation and generate the body with the needed information"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Use this for stand alone testing, otherwise keep commented out\n",
+ "#usertype=\"admin\""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Creating token for the service administrator.\n"
+ ]
+ }
+ ],
+ "source": [
+ "token=\"TokenNotCreated\"\n",
+ "try: usertype\n",
+ "except NameError: print(\"You must provide a user type\")\n",
+ "\n",
+ "# Simuates a significant block\n",
+ "# of code to auhenticate user\n",
+ "# before providing a token \n",
+ "\n",
+ "if usertype == \"admin\":\n",
+ " dbusername=ADMINusername\n",
+ " dbpassword=ADMINpassword\n",
+ " print(\"Creating token for the service administrator.\")\n",
+ "else:\n",
+ " dbusername=USERusername\n",
+ " dbpassword=USERpassword\n",
+ " print(\"Creating token for the service user.\") "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Run this cell to create the body of the service call to careate a token that expires in 1 hour\n",
+ "\n",
+ "body = {\n",
+ " \"dbParms\": {\n",
+ " \"dbHost\": dbHost,\n",
+ " \"dbName\": dbName,\n",
+ " \"dbPort\": dbPort,\n",
+ " \"isSSLConnection\": False,\n",
+ " \"username\": dbusername,\n",
+ " \"password\": dbpassword\n",
+ " },\n",
+ " \"expiryTime\": \"1h\"\n",
+ "}"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{'dbParms': {'dbHost': '192.168.0.72',\n",
+ " 'dbName': 'SAMPLE',\n",
+ " 'dbPort': 50000,\n",
+ " 'isSSLConnection': False,\n",
+ " 'username': 'service_admin1',\n",
+ " 'password': 'password'},\n",
+ " 'expiryTime': '1h'}"
+ ]
+ },
+ "execution_count": 9,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "#body"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## API Service\n",
+ "When communicating with the RESTful service, you must provide the name of the service that you want to interact with. In this case the authentication service is */v1/auth*. When the cell below is run, the server will establish a connection to the database server."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "try:\n",
+ " response = requests.post(\"{}{}\".format(Db2RESTful,API_Auth), headers=headers, json=body)\n",
+ "except Exception as e:\n",
+ " print(\"Unable to call RESTful service. Error={}\".format(repr(e)))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "A response code of 200 means that the authentication worked properly, otherwise the error that was generated is printed."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The response includes a connection token that is reused throughout the rest of this lab. It ensures secure a connection without requiring that you reenter a userid and password with each request. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Status of token request. 200=Success\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(\"Status of token request. 200=Success\")\n",
+ "try:\n",
+ " if (response.status_code == 200):\n",
+ " token = response.json()[\"token\"]\n",
+ "# print(\"Token: {}\".format(token))\n",
+ " else: \n",
+ " print(response.json())\n",
+ " print(response)\n",
+ " print(response.status_code)\n",
+ "except Exception as e:\n",
+ " print(\"Call to get Token Failed\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "print(token)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "At this point you can copy the above token to the clip board and paste it into the next notebook."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.3"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/In_Db2_Machine_Learning/Deploying External Models with Python UDF/Db2 RESTful Prep External Py UDF.ipynb b/In_Db2_Machine_Learning/Deploying External Models with Python UDF/Db2 RESTful Prep External Py UDF.ipynb
new file mode 100644
index 0000000..70d0486
--- /dev/null
+++ b/In_Db2_Machine_Learning/Deploying External Models with Python UDF/Db2 RESTful Prep External Py UDF.ipynb
@@ -0,0 +1,355 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Prepare the database for the REST Service Examples"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "This notebook prepares the database for creating and running the rest services shown in the Db2 RESTful External Py UDF.ipynb notebook. It does things like creating roles and granting permissions. It uses the Db2 Magic commands to interact with the database. You can see how to install and use Db2 Magic at this repo: https://github.com/IBM/db2-jupyter\n",
+ "\n",
+ "##### If you are looking at the REST example, make sure to start by examining the Db2 RESTful External Py UDF.ipynb notebook."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Conversly, you can execute the SQL/DDL shown here in your favorite Db2 client if this all seems like too much trouble. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Run the notebook to set up Db2 Magic"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!wget https://raw.githubusercontent.com/IBM/db2-jupyter/master/db2.ipynb"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Warning: PANDAS level does not support Db2 typing which will can increase memory usage.\n",
+ " Install PANDAS version 1.3+ for more efficient dataframe creation.\n",
+ "Warning: On OSX you need to pip install the multiprocess package if you want\n",
+ " parallelism to work.\n",
+ "Warning: Parallelism is unavailable and THREADS option will be ignored.\n",
+ " Install MULTIPROCESSING/MULTIPROCESS(OSX) if you want allow\n",
+ " multiple SQL threads to run in parallel.\n",
+ "Db2 Extensions Loaded.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# This is the newest notebook and has graphing capabilities\n",
+ "%run \"db2 (4).ipynb\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Connect to the Database"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Connect with a user with DBADM and SECADM authorities. Change these values to connect to your database."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "DB=\"SAMPLE\"\n",
+ "USER=\"db2inst1\"\n",
+ "PW=\"ibmdb2aa\"\n",
+ "HOST=\"localhost\"\n",
+ "PORT=50000"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Connection successful.\n"
+ ]
+ }
+ ],
+ "source": [
+ "%sql CONNECT TO {DB} USER {USER} USING {PW} HOST {HOST} PORT {PORT}"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "##### Create the needed schemas"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "EXTPY schema holds objects for the external Python UDF"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "REST_SERVICES schema holds objects used by the Db2 REST Endpoint"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "%%sql\n",
+ "CREATE SCHEMA EXTPY;\n",
+ "CREATE SCHEMA REST_SERVICES;"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "##### Create the needed roles"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Command completed.\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%sql\n",
+ "drop role service_admin;\n",
+ "drop role service_user;\n",
+ "create role service_admin;\n",
+ "create role service_user;"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "##### Grant the roles to users"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Command completed.\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%sql\n",
+ "GRANT ROLE service_admin TO USER service_admin1;\n",
+ "GRANT ROLE service_user TO USER service_user1;"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### The SERVICE_ADMIN role is intended to users who create services\n",
+ "\n",
+ "- Min privileges to create, execute and describe services\n",
+ "- Also allows GRANT and REVOKE permissions to service users\n",
+ "- Allows token creation for this user\n",
+ "- Allows all tables in the GOSALES schema to be selected.\n",
+ "- Assumes REST_SERVICES schema already created\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Command completed.\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%sql \n",
+ "GRANT CONNECT ON DATABASE TO ROLE SERVICE_ADMIN;\n",
+ "GRANT SELECTIN ON SCHEMA SYSCAT TO ROLE SERVICE_ADMIN;\n",
+ "GRANT SELECT, INSERT, UPDATE, DELETE ON DB2REST.RESTSERVICE TO ROLE SERVICE_ADMIN;\n",
+ "GRANT SELECT ON SYSIBM.SYSDUMMY1 TO ROLE SERVICE_ADMIN;\n",
+ "GRANT ALL ON SCHEMA REST_SERVICES TO ROLE SERVICE_ADMIN WITH GRANT OPTION;\n",
+ "GRANT EXECUTE ON PROCEDURE SYSPROC.ADMIN_CMD TO ROLE SERVICE_ADMIN;\n",
+ "GRANT EXECUTE ON FUNCTION SYSPROC.AUTH_LIST_AUTHORITIES_FOR_AUTHID TO ROLE SERVICE_ADMIN;\n",
+ "\n",
+ "-- Grant authorities on tables used in any service created by this user id\n",
+ "GRANT SELECTIN, INSERTIN, UPDATEIN, DELETEIN, EXECUTEIN ON SCHEMA EXTPY TO ROLE SERVICE_ADMIN;"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### The SERVICE_USER role is intended for users who execute the services\n",
+ "\n",
+ "- Mostly just needs authority to execue services created by others\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Command completed.\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%sql\n",
+ "GRANT CONNECT ON DATABASE TO ROLE SERVICE_USER;\n",
+ "GRANT EXECUTE ON PROCEDURE SYSPROC.ADMIN_CMD TO ROLE SERVICE_USER;\n",
+ "GRANT EXECUTEIN ON SCHEMA REST_SERVICES TO ROLE SERVICE_USER;\n",
+ "GRANT EXECUTE ON FUNCTION SYSPROC.AUTH_LIST_AUTHORITIES_FOR_AUTHID TO ROLE SERVICE_USER;"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "##### Create the table used by the UDF when executed with table data. This is described in section 4 of the readme"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "%%sql\n",
+ "drop table if exists extpy.person_features;\n",
+ "CREATE TABLE extpy.person_features (\n",
+ "ID INT NOT NULL GENERATED ALWAYS AS IDENTITY (START WITH 1, INCREMENT BY 1, NO CACHE),\n",
+ "FEMALE FLOAT,\n",
+ "MALE FLOAT,\n",
+ "MARRIED FLOAT,\n",
+ "SINGLE FLOAT,\n",
+ "UNSPECIFIED FLOAT,\n",
+ "EXECUTIVE FLOAT,\n",
+ "HOSPITALITY FLOAT,\n",
+ "OTHER FLOAT,\n",
+ "PROFESSIONAL FLOAT,\n",
+ "RETAIL FLOAT,\n",
+ "RETIRED FLOAT,\n",
+ "SALES FLOAT,\n",
+ "STUDENT FLOAT,\n",
+ "TRADES FLOAT,\n",
+ "CAMPING_EQUIPMENT FLOAT,\n",
+ "GOLF_EQUIPMENT FLOAT,\n",
+ "MOUNTAINEERING_EQUIPMENT FLOAT,\n",
+ "OUTDOOR_PROTECTION FLOAT,\n",
+ "PERSONAL_ACCESSORIES FLOAT,\n",
+ "AGE FLOAT,\n",
+ "IS_TENT FLOAT,\n",
+ "PRIMARY KEY (ID))\n",
+ "ORGANIZE BY ROW;"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Connection closed.\n"
+ ]
+ }
+ ],
+ "source": [
+ "%sql CONNECT RESET"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.3"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/In_Db2_Machine_Learning/Deploying External Models with Python UDF/README.md b/In_Db2_Machine_Learning/Deploying External Models with Python UDF/README.md
index 135c956..6cd396a 100644
--- a/In_Db2_Machine_Learning/Deploying External Models with Python UDF/README.md
+++ b/In_Db2_Machine_Learning/Deploying External Models with Python UDF/README.md
@@ -9,7 +9,9 @@ This repository contains notebooks and datasets that will allow Db2 customers to
3. [Register the UDF](#Register)
4. [Download and Store the Test Data in a Db2 Table](#ImportData)
5. [Call your UDF to Make a Prediction](#Predict)
-6. [Reference and Further Reading](#Reference)
+6. [Create and Call your UDF to Make a Prediction using REST services ](#RESTAPI)
+7. [Reference and Further Reading](#Reference)
+
## 0. Prerequisites
@@ -113,7 +115,16 @@ GOLF_EQUIPMENT, MOUNTAINEERING_EQUIPMENT, OUTDOOR_PROTECTION, PERSONAL_ACCESSORI
AGE, IS_TENT) as model_prediction from TEST_INPUT;"
```
-## 6. Reference and Further Reading
+## 6. Create and Call your UDF to Make a Prediction using a REST API
+
+Use the three "Db2 RESTful..." notebooks to create a REST services that can call the UDF to make a prediction. Two services are created -- one where you give it the features for the model from your applicaiton and the other where the features are already in a table, and you point the service to the correct row in the table. This would be useful in mobile and other applications where it is not convenient to connect to the database directly, but instead call a service that gets the prediction for you.
+
+1. Download the notebooks to a directory where you keep Jupyter notebooks
+2. Edit the Db2 RESTful Prep External Py UDF.ipynb notebook to use your database connect information and credentials and run it to prepare your database with the needed objects and grants.
+3. Edit the Db2 RESTful Get Token for External Py UDF.ipynb notebook to use your database connect information and credentials. It will be called by the notebook that creates and uses the services to get the needed token for the Db2 REST endpoint.
+4. Run the Db2 RESTful External Py UDF.ipynb for examples of creating and running services that use the UDF.
+
+## 7. Reference and Further Reading
- [Creating Python UDX in Db2](https://www.ibm.com/support/knowledgecenter/SSHRBY/com.ibm.swg.im.dashdb.udx.doc/doc/udx_t_create_udx_python.html)
- [Deploying Python UDX in Db2](https://www.ibm.com/support/knowledgecenter/SSHRBY/com.ibm.swg.im.dashdb.udx.doc/doc/udx_t_deploying_python.html)
\ No newline at end of file