examples/providers/kdbai/ChatGPT_QA_Demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# KDB.AI for Q&A with ChatGPT Retrieval Plugin"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This example will demonstrate how to use the ChatGPT Retrieval Plugin to embed and store data in KDB.AI, as well as to query the embedded data. The plugin enables ChatGPT large language models, such as GPT-3.5, to be used for querying data that it wasn't trained on. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How does the ChatGPT Retrieval Plugin work?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![kdbai-chatgpt-overview](kdbai-chatgpt-overview.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The diagram above provides a high-level overview of the system, and explains how ChatGPT, the Retrieval Plugin, and KDB.AI all interact with one another. The system can be broken down into two main sub-systems: upserting (shown in red) and querying (shown in blue).\n",
    "\n",
    "\n",
    "### Upserting\n",
    "\n",
    "Upserting is the process of taking our own dataset and uploading it to our vector database. In order to upsert data, we must pre-process it to remove any part we do not want to store in the database, change the format to a dictionary, and divide it into batches. When we call  `/upsert`, each batch of data is embedded using an OpenAI LLM (Large Language Model), to generate a vector embedding for similarity search. The data and its embedding are then inserted into a KDB.AI table.\n",
    "\n",
    "\n",
    "### Querying\n",
    "\n",
    "Querying involves taking an input query, or question, transforming it into an embedded vector, and running similarity search in the vector database to find its nearest neighbours - data with closest relevancy to the query. When we call `/query`,  the input query is embedded with the same OpenAI LLM used for upserting, to generate a query vector. A similarity search algorithm running within KDB.AI will return the N closest matches with highest relevancy to the query. This \"context\" is then sent back to ChatGPT via the Retrieval Plugin. Finally, using this contextual information, ChatGPT outputs a human-like response to the query.\n",
    "\n",
    "Having an external source of data greatly increases the applicability of ChatGPT to different tasks. For example: GPT-3.5 can only answer questions with information it has seen during training, which is capped at 2021.. Therefore, if you asked it who the 2023 UK Prime Minister is, the model does not have the necessary information. However, if you were to download a dataset of political data up to the present day, ChatGPT would be able to use this data to answer the question. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Getting Started\n",
    "\n",
    "### Prerequisites\n",
    "\n",
    "- Python 3\n",
    "- Pip\n",
    "- Git\n",
    "\n",
    "### Install the KDB.AI ChatGPT Retrieval Plugin server app\n",
    "\n",
    "```\n",
    "git clone https://github.com/KxSystems/chatgpt-retrieval-plugin -b KDB.AI\n",
    "cd chatgpt-retrieval-plugin\n",
    "pip install poetry\n",
    "poetry install\n",
    "```\n",
    "\n",
    "### Run the KDB.AI ChatGPT Retrieval Plugin server app\n",
    "\n",
    "```\n",
    "export BEARER_TOKEN='<BEARER TOKEN>'  # you can create your own bearer token on auth0.com\n",
    "export DATASTORE=kdbai\n",
    "export KDBAI_ENDPOINT='<KDB.AI ENDPOINT>'\n",
    "export KDBAI_API_KEY='<KDB.AI API KEY>'\n",
    "export OPENAI_API_KEY='<OPENAI API KEY>'  # You can get a free API key on https://platform.openai.com\n",
    "\n",
    "poetry run start\n",
    "```\n",
    "\n",
    "### Install a separate Jupyter environment to run this notebook\n",
    "\n",
    "```\n",
    "pip install datasets jupyter openai tqdm\n",
    "```\n",
    "\n",
    "### Run Jupyter\n",
    "\n",
    "```\n",
    "export BEARER_TOKEN='<BEARER TOKEN>'  # Same bearer token as above\n",
    "export OPENAI_API_KEY='<OPENAI API KEY>'\n",
    "\n",
    "jupyter notebook\n",
    "```\n",
    "\n",
    "Then open this notebook in Jupyter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from pprint import pprint\n",
    "import random\n",
    "\n",
    "from datasets import load_dataset\n",
    "import openai\n",
    "import requests\n",
    "from tqdm.auto import tqdm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "BEARER_TOKEN = os.environ.get(\"BEARER_TOKEN\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load Dataset from Hugging Face\n",
    "\n",
    "The Adversarial_QA dataset is chosen for this demonstration. It consists of questions that current state-of-the-art models find challenging, paired with contextual data that can be used to formulate an answer. Some of these questions have very poor grammar (\"What sare the benifts of the blood brain barrir?\"), and others are purposefully vague (\"What is at the highest level?\"), as shown in the example below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of unique contexts: 2648\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>context</th>\n",
       "      <th>question</th>\n",
       "      <th>answers</th>\n",
       "      <th>metadata</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7ba1e8f4261d3170fcf42e84a81dd749116fae95</td>\n",
       "      <td>Brain</td>\n",
       "      <td>Another approach to brain function is to exami...</td>\n",
       "      <td>What sare the benifts of the blood brain barrir?</td>\n",
       "      <td>{'text': ['isolated from the bloodstream'], 'a...</td>\n",
       "      <td>{'split': 'train', 'model_in_the_loop': 'Combi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>936a8460bfffe437b54cf3ec1e825a3b7b5627a1</td>\n",
       "      <td>Brain</td>\n",
       "      <td>Motor systems are areas of the brain that are ...</td>\n",
       "      <td>What do you think with?</td>\n",
       "      <td>{'text': ['brain'], 'answer_start': [467]}</td>\n",
       "      <td>{'split': 'train', 'model_in_the_loop': 'Combi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>e40737d487964dbcd26a223f2799cf56390a98a8</td>\n",
       "      <td>Brain</td>\n",
       "      <td>The brain is an organ that serves as the cente...</td>\n",
       "      <td>How are neurons connected?</td>\n",
       "      <td>{'text': ['synapses'], 'answer_start': [602]}</td>\n",
       "      <td>{'split': 'train', 'model_in_the_loop': 'Combi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>a0f8e785a10f6e21e24207d24ba2823162383062</td>\n",
       "      <td>Brain</td>\n",
       "      <td>The SCN projects to a set of areas in the hypo...</td>\n",
       "      <td>The body's central biological clock is contain...</td>\n",
       "      <td>{'text': ['SCN'], 'answer_start': [4]}</td>\n",
       "      <td>{'split': 'train', 'model_in_the_loop': 'Combi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53</th>\n",
       "      <td>6d753d4a8878b5f5bc496d9a369b8c0b212079a0</td>\n",
       "      <td>Brain</td>\n",
       "      <td>The brain contains several motor areas that pr...</td>\n",
       "      <td>What is at the highest level?</td>\n",
       "      <td>{'text': ['the primary motor cortex'], 'answer...</td>\n",
       "      <td>{'split': 'train', 'model_in_the_loop': 'Combi...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                          id  title  \\\n",
       "0   7ba1e8f4261d3170fcf42e84a81dd749116fae95  Brain   \n",
       "12  936a8460bfffe437b54cf3ec1e825a3b7b5627a1  Brain   \n",
       "24  e40737d487964dbcd26a223f2799cf56390a98a8  Brain   \n",
       "37  a0f8e785a10f6e21e24207d24ba2823162383062  Brain   \n",
       "53  6d753d4a8878b5f5bc496d9a369b8c0b212079a0  Brain   \n",
       "\n",
       "                                              context  \\\n",
       "0   Another approach to brain function is to exami...   \n",
       "12  Motor systems are areas of the brain that are ...   \n",
       "24  The brain is an organ that serves as the cente...   \n",
       "37  The SCN projects to a set of areas in the hypo...   \n",
       "53  The brain contains several motor areas that pr...   \n",
       "\n",
       "                                             question  \\\n",
       "0    What sare the benifts of the blood brain barrir?   \n",
       "12                            What do you think with?   \n",
       "24                         How are neurons connected?   \n",
       "37  The body's central biological clock is contain...   \n",
       "53                      What is at the highest level?   \n",
       "\n",
       "                                              answers  \\\n",
       "0   {'text': ['isolated from the bloodstream'], 'a...   \n",
       "12         {'text': ['brain'], 'answer_start': [467]}   \n",
       "24      {'text': ['synapses'], 'answer_start': [602]}   \n",
       "37             {'text': ['SCN'], 'answer_start': [4]}   \n",
       "53  {'text': ['the primary motor cortex'], 'answer...   \n",
       "\n",
       "                                             metadata  \n",
       "0   {'split': 'train', 'model_in_the_loop': 'Combi...  \n",
       "12  {'split': 'train', 'model_in_the_loop': 'Combi...  \n",
       "24  {'split': 'train', 'model_in_the_loop': 'Combi...  \n",
       "37  {'split': 'train', 'model_in_the_loop': 'Combi...  \n",
       "53  {'split': 'train', 'model_in_the_loop': 'Combi...  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = load_dataset(\"adversarial_qa\", 'adversarialQA', ignore_verifications=True)['train'].to_pandas()\n",
    "data = data.drop_duplicates(subset=[\"context\"])\n",
    "print(f\"Number of unique contexts: {len(data)}\")\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To use this dataset for Q&A with ChatGPT, we need to insert the relevant data into our vector datastore - KDB.AI. The only column of data we will insert is the \"context\" column, which we reformat into a dictionary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'text': 'Another approach to brain function is to examine the consequences of '\n",
      "         'damage to specific brain areas. Even though it is protected by the '\n",
      "         'skull and meninges, surrounded by cerebrospinal fluid, and isolated '\n",
      "         'from the bloodstream by the blood–brain barrier, the delicate nature '\n",
      "         'of the brain makes it vulnerable to numerous diseases and several '\n",
      "         'types of damage. In humans, the effects of strokes and other types '\n",
      "         'of brain damage have been a key source of information about brain '\n",
      "         'function. Because there is no ability to experimentally control the '\n",
      "         'nature of the damage, however, this information is often difficult '\n",
      "         'to interpret. In animal studies, most commonly involving rats, it is '\n",
      "         'possible to use electrodes or locally injected chemicals to produce '\n",
      "         'precise patterns of damage and then examine the consequences for '\n",
      "         'behavior.'}\n"
     ]
    }
   ],
   "source": [
    "# extract text data from the dataset\n",
    "documents = [\n",
    "    {\n",
    "        'id': r['id'],\n",
    "        'text': r['context'],\n",
    "        'metadata': {\n",
    "            'title': r['title']\n",
    "        }\n",
    "    } for r in data.to_dict(orient='records')\n",
    "]\n",
    "pprint(documents[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Insert data to the KDB.AI table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# initialise an HTTP session with the KDB.AI ChatGPT Retrieval Plugin app\n",
    "s = requests.Session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `/upsert` instruction is used to insert data to the KDB.AI datastore in batches, with each batch being embedded with OpenAI Embedding Model `text-embedding-ada-002` before it is added to the table. We take our contextual data, which has been reformatted into a dictionary, and call `/upsert` on batches of 100 documents at a time. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "bbcc7b88e39a49039d3983aae95a2b3d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/27 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "batchSize = 100\n",
    "\n",
    "# upsert documents from the dataset in batches\n",
    "for i in tqdm(range(0, len(documents), batchSize)):\n",
    "    i_end = min(len(documents), i+batchSize)\n",
    "    \n",
    "    res = s.post(\n",
    "        \"http://localhost:8000/upsert\",\n",
    "        \n",
    "        headers = {\n",
    "            \"Authorization\": f\"Bearer {BEARER_TOKEN}\"\n",
    "        },\n",
    "        \n",
    "        json = {\n",
    "            \"documents\": documents[i:i_end]\n",
    "        }\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Query the KDB.AI table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can take the list of questions from our dataset and format them into a dictionary. We will randomly choose 5 of these questions to be our queries in this example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[{'query': 'What roads are part of the Schnellweg?'}, {'query': 'Little Britain is actually?'}, {'query': 'What is the second smallest large urban area listed?'}, {'query': 'Which country is governed by a national assembly?'}, {'query': 'What was Ireland an exception to?'}]\n"
     ]
    }
   ],
   "source": [
    "# extract questions and reformat into queries\n",
    "queries = data['question'].tolist()\n",
    "queries = [{'query': queries[i]} for i in range(len(queries))]\n",
    "\n",
    "# choose 5 queries at random \n",
    "i = random.randint(0, len(queries)-5)\n",
    "searchQueries = queries[i:i+5]\n",
    "\n",
    "print(searchQueries)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `/query` instruction is used to extract relevant information from the KDB.AI datastore. The queries are embedded into vectors, and a similarity search algorithm is used to calculate its nearest neighbours, representing the most relevant entries in the table. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<Response [200]>\n"
     ]
    }
   ],
   "source": [
    "# query the vector database\n",
    "results = requests.post(\n",
    "    \"http://localhost:8000/query\",\n",
    "    \n",
    "    headers = {\n",
    "        \"Authorization\": f\"Bearer {BEARER_TOKEN}\"\n",
    "    },\n",
    "    \n",
    "    json = {\n",
    "        'queries': searchQueries\n",
    "    }\n",
    ")\n",
    "\n",
    "print(results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the cell below, we iterate through each query and its results, and pass this data to ChatGPT, which uses it to respond to the query with natural language. Here, you can see the three \"nearest neighbour\" pieces of context that ChatGPT uses to form its answer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "QUERY:\n",
      "What roads are part of the Schnellweg?\n",
      "\n",
      "CONTEXT:\n",
      "0.21: The Schnellweg (en: expressway) system, a number of Bundesstraße roads, forms a structure loosely resembling a large ring road together with A2 and A7. The roads are B 3, B 6 and B 65, called Westschnellweg (B6 on the northern part, B3 on the southern part), Messeschnellweg (B3, becomes A37 near Burgdorf, crosses A2, becomes B3 again, changes to B6 at Seelhorster Kreuz, then passes the Hanover fairground as B6 and becomes A37 again before merging into A7) and Südschnellweg (starts out as B65, becomes B3/B6/B65 upon crossing Westschnellweg, then becomes B65 again at Seelhorster Kreuz).\n",
      "0.21: The Schnellweg (en: expressway) system, a number of Bundesstraße roads, forms a structure loosely resembling a large ring road together with A2 and A7. The roads are B 3, B 6 and B 65, called Westschnellweg (B6 on the northern part, B3 on the southern part), Messeschnellweg (B3, becomes A37 near Burgdorf, crosses A2, becomes B3 again, changes to B6 at Seelhorster Kreuz, then passes the Hanover fairground as B6 and becomes A37 again before merging into A7) and Südschnellweg (starts out as B65, becomes B3/B6/B65 upon crossing Westschnellweg, then becomes B65 again at Seelhorster Kreuz).\n",
      "0.21: The Schnellweg (en: expressway) system, a number of Bundesstraße roads, forms a structure loosely resembling a large ring road together with A2 and A7. The roads are B 3, B 6 and B 65, called Westschnellweg (B6 on the northern part, B3 on the southern part), Messeschnellweg (B3, becomes A37 near Burgdorf, crosses A2, becomes B3 again, changes to B6 at Seelhorster Kreuz, then passes the Hanover fairground as B6 and becomes A37 again before merging into A7) and Südschnellweg (starts out as B65, becomes B3/B6/B65 upon crossing Westschnellweg, then becomes B65 again at Seelhorster Kreuz).\n",
      "\n",
      "RESPONSE: \n",
      "The roads that are part of the Schnellweg system are B3, B6, and B65. Specifically, the Westschnellweg consists of B6 on the northern part and B3 on the southern part, the Messeschnellweg includes B3/A37, and the Südschnellweg includes B65/B3/B6/B65 sections.\n",
      "\n",
      "----------------------------------------------------------------------\n",
      "\n",
      "QUERY:\n",
      "Little Britain is actually?\n",
      "\n",
      "CONTEXT:\n",
      "0.39: The Greco-Egyptian scientist Claudius Ptolemy referred to the larger island as great Britain (μεγάλης Βρεττανίας - megális Brettanias) and to Ireland as little Britain (μικρής Βρεττανίας - mikris Brettanias) in his work Almagest (147–148 AD). In his later work, Geography (c. 150 AD), he gave these islands the names Alwion, Iwernia, and Mona (the Isle of Man), suggesting these may have been names of the individual islands not known to him at the time of writing Almagest. The name Albion appears to have fallen out of use sometime after the Roman conquest of Great Britain, after which Britain became the more commonplace name for the island called Great Britain.\n",
      "0.39: The Greco-Egyptian scientist Claudius Ptolemy referred to the larger island as great Britain (μεγάλης Βρεττανίας - megális Brettanias) and to Ireland as little Britain (μικρής Βρεττανίας - mikris Brettanias) in his work Almagest (147–148 AD). In his later work, Geography (c. 150 AD), he gave these islands the names Alwion, Iwernia, and Mona (the Isle of Man), suggesting these may have been names of the individual islands not known to him at the time of writing Almagest. The name Albion appears to have fallen out of use sometime after the Roman conquest of Great Britain, after which Britain became the more commonplace name for the island called Great Britain.\n",
      "0.39: The Greco-Egyptian scientist Claudius Ptolemy referred to the larger island as great Britain (μεγάλης Βρεττανίας - megális Brettanias) and to Ireland as little Britain (μικρής Βρεττανίας - mikris Brettanias) in his work Almagest (147–148 AD). In his later work, Geography (c. 150 AD), he gave these islands the names Alwion, Iwernia, and Mona (the Isle of Man), suggesting these may have been names of the individual islands not known to him at the time of writing Almagest. The name Albion appears to have fallen out of use sometime after the Roman conquest of Great Britain, after which Britain became the more commonplace name for the island called Great Britain.\n",
      "\n",
      "RESPONSE: \n",
      "Little Britain, according to the Greco-Egyptian scientist Claudius Ptolemy, refers to Ireland. In his work Almagest, Ptolemy referred to the larger island as Great Britain and to Ireland as Little Britain.\n",
      "\n",
      "----------------------------------------------------------------------\n",
      "\n",
      "QUERY:\n",
      "What is the second smallest large urban area listed?\n",
      "\n",
      "CONTEXT:\n",
      "0.4: In its 2013 ParkScore ranking, The Trust for Public Land reported that the park system in New York City was the second best park system among the 50 most populous U.S. cities, behind the park system of Minneapolis. ParkScore ranks urban park systems by a formula that analyzes median park size, park acres as percent of city area, the percent of city residents within a half-mile of a park, spending of park services per resident, and the number of playgrounds per 10,000 residents.\n",
      "0.4: In its 2013 ParkScore ranking, The Trust for Public Land reported that the park system in New York City was the second best park system among the 50 most populous U.S. cities, behind the park system of Minneapolis. ParkScore ranks urban park systems by a formula that analyzes median park size, park acres as percent of city area, the percent of city residents within a half-mile of a park, spending of park services per resident, and the number of playgrounds per 10,000 residents.\n",
      "0.4: In its 2013 ParkScore ranking, The Trust for Public Land reported that the park system in New York City was the second best park system among the 50 most populous U.S. cities, behind the park system of Minneapolis. ParkScore ranks urban park systems by a formula that analyzes median park size, park acres as percent of city area, the percent of city residents within a half-mile of a park, spending of park services per resident, and the number of playgrounds per 10,000 residents.\n",
      "\n",
      "RESPONSE: \n",
      "Based on the information provided, I don't have the specific details about the sizes of the urban areas listed. Without more specific information or a list of the urban areas, I cannot determine the second smallest large urban area.\n",
      "\n",
      "----------------------------------------------------------------------\n",
      "\n",
      "QUERY:\n",
      "Which country is governed by a national assembly?\n",
      "\n",
      "CONTEXT:\n",
      "0.37: In fact, there was an eighth province, the County of Drenthe, but this area was so poor it was exempt from paying federal taxes and as a consequence was denied representation in the States General. Each province was governed by the Provincial States, the main executive official (though not the official head of state) was a raadspensionaris. In times of war, the stadtholder, who commanded the army, would have more power than the raadspensionaris.\n",
      "0.37: In fact, there was an eighth province, the County of Drenthe, but this area was so poor it was exempt from paying federal taxes and as a consequence was denied representation in the States General. Each province was governed by the Provincial States, the main executive official (though not the official head of state) was a raadspensionaris. In times of war, the stadtholder, who commanded the army, would have more power than the raadspensionaris.\n",
      "0.37: In fact, there was an eighth province, the County of Drenthe, but this area was so poor it was exempt from paying federal taxes and as a consequence was denied representation in the States General. Each province was governed by the Provincial States, the main executive official (though not the official head of state) was a raadspensionaris. In times of war, the stadtholder, who commanded the army, would have more power than the raadspensionaris.\n",
      "\n",
      "RESPONSE: \n",
      "The Netherlands is governed by a national assembly called the States General.\n",
      "\n",
      "----------------------------------------------------------------------\n",
      "\n",
      "QUERY:\n",
      "What was Ireland an exception to?\n",
      "\n",
      "CONTEXT:\n",
      "0.33: Newfoundland reverted to colonial status in 1933, suffering from financial difficulties during the Great Depression. Ireland distanced itself further from Britain with the introduction of a new constitution in 1937, making it a republic in all but name.\n",
      "0.33: Newfoundland reverted to colonial status in 1933, suffering from financial difficulties during the Great Depression. Ireland distanced itself further from Britain with the introduction of a new constitution in 1937, making it a republic in all but name.\n",
      "0.33: Newfoundland reverted to colonial status in 1933, suffering from financial difficulties during the Great Depression. Ireland distanced itself further from Britain with the introduction of a new constitution in 1937, making it a republic in all but name.\n",
      "\n",
      "RESPONSE: \n",
      "Ireland was an exception to the British rule, as it distanced itself further from Britain with the introduction of a new constitution in 1937, making it a republic in all but name.\n",
      "\n",
      "----------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "# iterate through each set of queries/results\n",
    "for query_response in results.json()['results']:\n",
    "    query = query_response['query']\n",
    "    answers = []\n",
    "    scores = []\n",
    "    \n",
    "    # extract answers and scores from each result\n",
    "    for result in query_response['results']:\n",
    "        \n",
    "        # answer = textual information related to the query\n",
    "        answers.append(result['text'])\n",
    "        \n",
    "        # score = distance between the query vector and the answer vector (smaller=better!)\n",
    "        scores.append(round(result['score'], 2))\n",
    "    \n",
    "    # print the query\n",
    "    print(\"\\nQUERY:\\n\"+query)\n",
    "    \n",
    "    # print the query responses, and their scores\n",
    "    print(\"\\nCONTEXT:\\n\"+\"\\n\".join([f\"{s}: {a}\" for a, s in zip(answers, scores)])+\"\\n\")\n",
    "    \n",
    "    # format the query and its answers into GPT messages\n",
    "    messages = [\n",
    "        {\"role\": \"system\", \"content\": f\"You are a helpful assistant with the following knowledge: {answers}\"},\n",
    "        {\"role\": \"user\", \"content\": f\"Using your knowledge, answer this query: {query}\"}\n",
    "    ]\n",
    "    \n",
    "    # send the messages to a GPT model\n",
    "    response = openai.chat.completions.create(\n",
    "      model=\"gpt-3.5-turbo\",\n",
    "      messages=messages,\n",
    "      max_tokens=100,\n",
    "      n=1,\n",
    "      stop=None,            \n",
    "    )\n",
    "\n",
    "    # extract the generated response from the API response\n",
    "    generated_response = response.choices[0].message.content\n",
    "    \n",
    "    # Print the generated response\n",
    "    print(f\"RESPONSE: \\n{generated_response}\\n\")\n",
    "    print(\"-\"*70)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Delete the KDB.AI table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To delete the KDB.AI table, run the code cell below. Once this is done, the ChatGPT Retrieval Plugin app will need to be restarted to create another table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "## One would need to restart the KDB.AI ChatGPT Retrieval Plugin server app\n",
    "## after this, to recreate the table\n",
    "res = requests.delete(\n",
    "    \"http://localhost:8000/delete\",\n",
    "    headers = {\n",
    "        \"Authorization\": f\"Bearer {BEARER_TOKEN}\"\n",
    "    }, \n",
    "    json = {\n",
    "        \"delete_all\": True\n",
    "    }\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}