Module 4 - Fullstack Semantic Search.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e569883b",
   "metadata": {},
   "source": [
    "# Full Stack Semantic Search Web Application"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58b0a950",
   "metadata": {},
   "source": [
    "In the previous modules, we've demonstrated both keyword and semantic search with Amazon OpenSearch Service. In this module, we will now create a search enabled application using a sage maker endpoint and a serverless web application.\n",
    "\n",
    "By the end of this module, the architecture will look as follows:\n",
    "\n",
    "![full stack semantic search](semantic_search_fullstack.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "537e400b",
   "metadata": {},
   "source": [
    "### 1.Import PyTorch and check version.\n",
    "\n",
    "As in the previous modules, let's import PyTorch and confirm that have have the latest version of PyTorch. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d7fb1c18",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "print(torch.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1a2d361",
   "metadata": {},
   "source": [
    "### 2. Retrieve notebook variables\n",
    "\n",
    "The line below will retrieve your shared variables from the previous notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f19b70e0",
   "metadata": {},
   "outputs": [],
   "source": [
    "%store -r"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f25eff2b",
   "metadata": {},
   "source": [
    "### 3. Initialize boto3\n",
    "\n",
    "We will use boto3 to interact with other AWS services.\n",
    "\n",
    "Note: You can ignore any PythonDeprecationWarning warnings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ae008253",
   "metadata": {},
   "outputs": [],
   "source": [
    "import boto3\n",
    "import re\n",
    "import time\n",
    "import sagemaker\n",
    "from sagemaker import get_execution_role\n",
    "\n",
    "s3_resource = boto3.resource(\"s3\")\n",
    "s3 = boto3.client('s3')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cfb23f5a",
   "metadata": {},
   "source": [
    "### 4. Save pre-trained BERT model to S3\n",
    "\n",
    "First off, we will host a pretrained BERT model in a SageMaker Pytorch model server to generate 768x1 dimension fixed length sentence embedding from [sentence-transformers](https://github.com/UKPLab/sentence-transformers) using [HuggingFace Transformers](https://huggingface.co/sentence-transformers/distilbert-base-nli-stsb-mean-tokens). \n",
    "\n",
    "This SageMaker endpoint will be called by the application to generate vector for the search query. First we'll get a pre-trained model and upload to S3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "45fa8f63",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from transformers import AutoTokenizer, AutoModel\n",
    "from transformers import DistilBertTokenizer, DistilBertModel\n",
    "import os\n",
    "from transformers import AutoTokenizer, AutoModel\n",
    "\n",
    "model_name = \"sentence-transformers/distilbert-base-nli-stsb-mean-tokens\"\n",
    "saved_model_dir = 'transformer'\n",
    "os.makedirs(saved_model_dir, exist_ok=True)\n",
    "\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_name)\n",
    "model = AutoModel.from_pretrained(model_name) \n",
    "\n",
    "tokenizer.save_pretrained(saved_model_dir)\n",
    "model.save_pretrained(saved_model_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f40f5aa1",
   "metadata": {},
   "source": [
    "Create a SageMaker session and get the execution role to be used later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "11557444",
   "metadata": {},
   "outputs": [],
   "source": [
    "sagemaker_session = sagemaker.Session()\n",
    "role = sagemaker.get_execution_role()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17e4ce65",
   "metadata": {},
   "source": [
    "Unpack the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "87917e39",
   "metadata": {},
   "outputs": [],
   "source": [
    "!cd transformer && tar czvf ../model.tar.gz *"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f8d464c",
   "metadata": {},
   "source": [
    "And finally upload the model to S3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "959f87c1",
   "metadata": {},
   "outputs": [],
   "source": [
    "inputs = sagemaker_session.upload_data(path='model.tar.gz', key_prefix='sentence-transformers-model')\n",
    "inputs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34e83f9e",
   "metadata": {},
   "source": [
    "### 5. Create PyTorch Model Object\n",
    "\n",
    "Next we need to create a PyTorchModel object. The deploy() method on the model object creates an endpoint which serves prediction requests in real-time. If the instance_type is set to a SageMaker instance type (e.g. ml.m5.large) then the model will be deployed on SageMaker. If the instance_type parameter is set to local then it will be deployed locally as a Docker container and ready for testing locally.\n",
    "\n",
    "We need to create a Predictor class to accept TEXT as input and output JSON. The default behaviour is to accept a numpy array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "789314b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.pytorch import PyTorch, PyTorchModel\n",
    "from sagemaker.predictor import Predictor\n",
    "from sagemaker import get_execution_role\n",
    "\n",
    "class StringPredictor(Predictor):\n",
    "    def __init__(self, endpoint_name, sagemaker_session):\n",
    "        super(StringPredictor, self).__init__(endpoint_name, sagemaker_session, content_type='text/plain')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ada22880",
   "metadata": {},
   "source": [
    "### 6. Deploy the BERT model to SageMaker Endpoint\n",
    "Now that we have the predictor class, let's deploy a SageMaker endpoint for our application to invoke.\n",
    "\n",
    "#### Note: This process will take about 5 minutes to complete.\n",
    "\n",
    "You can ignore the \"content_type is a no-op in sagemaker>=2\" warning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e023ed23",
   "metadata": {},
   "outputs": [],
   "source": [
    "pytorch_model = PyTorchModel(model_data = inputs, \n",
    "                             role=role, \n",
    "                             entry_point ='inference.py',\n",
    "                             source_dir = './code',\n",
    "                             py_version = 'py39', \n",
    "                             framework_version = '1.13.1',\n",
    "                             predictor_cls=StringPredictor)\n",
    "\n",
    "predictor = pytorch_model.deploy(instance_type='ml.m5d.large', \n",
    "                                 initial_instance_count=1, \n",
    "                                 endpoint_name = f'semantic-search-model-{int(time.time())}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28ad6cc8",
   "metadata": {},
   "source": [
    "### 7. Test the SageMaker Endpoint.\n",
    "\n",
    "Now that the endpoint is created, let's quickly test it out."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "af7bd141",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "original_payload = 'Does this work with xbox?'\n",
    "features = predictor.predict(original_payload)\n",
    "vector_data = json.loads(features)\n",
    "\n",
    "vector_data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e045511",
   "metadata": {},
   "source": [
    "----\n",
    "\n",
    "# Deploying a full-stack semantic search application\n",
    "\n",
    "We are now ready to build a real-world full-stack ML-powered web app. The Serverless Application Model (SAM) template we create below will deploy an Amazon API Gateway and AWS Lambda function. The Lambda function runs your code in response to HTTP requests that are sent to the API Gateway."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "958d0c6b",
   "metadata": {},
   "source": [
    "### 8. Build lambda zip file\n",
    "\n",
    "First, we need to package our lambda function for deployment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3fc9b433",
   "metadata": {},
   "outputs": [],
   "source": [
    "%cd backend/lambda\n",
    "!sh build-lambda.sh\n",
    "!unzip -l lambda.zip\n",
    "%cd /home/ec2-user/SageMaker/semantic-search-with-amazon-opensearch"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8949266d",
   "metadata": {},
   "source": [
    "### Get  account information"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "54296046",
   "metadata": {},
   "outputs": [],
   "source": [
    "account_id = boto3.client('sts').get_caller_identity().get('Account')\n",
    "my_account = f'{account_id}'\n",
    "print(\"account id: \" + my_account)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f92d301",
   "metadata": {},
   "source": [
    "### Disable S3 account level public access block\n",
    "\n",
    "Please refer S3 documenation for more information. https://docs.aws.amazon.com/AmazonS3/latest/userguide/configuring-block-public-access-account.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "68b62b0f",
   "metadata": {},
   "outputs": [],
   "source": [
    "client = boto3.client('s3control')\n",
    "response = client.put_public_access_block(\n",
    "    PublicAccessBlockConfiguration={\n",
    "        'BlockPublicAcls': False,\n",
    "        'IgnorePublicAcls': False,\n",
    "        'BlockPublicPolicy': False,\n",
    "        'RestrictPublicBuckets': False\n",
    "    },\n",
    "    AccountId=my_account\n",
    ")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40f2e1b0",
   "metadata": {},
   "source": [
    "### Disable S3 bucket level public access block\n",
    "\n",
    "Please refer S3 documenation for more informaiton. https://docs.aws.amazon.com/AmazonS3/latest/userguide/configuring-block-public-access-bucket.html\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cd78dcfc",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(bucket)\n",
    "s3 = boto3.client('s3')\n",
    "response = s3.put_public_access_block(\n",
    "    Bucket=f'{bucket}',\n",
    "    PublicAccessBlockConfiguration={\n",
    "        'BlockPublicAcls': False,\n",
    "        'IgnorePublicAcls': False,\n",
    "        'BlockPublicPolicy': False,\n",
    "        'RestrictPublicBuckets': False\n",
    "    }\n",
    ")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04b965fa",
   "metadata": {},
   "source": [
    "### Change S3 bucket ownership settings\n",
    "\n",
    "Please refer S3 documenation for more information. https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-ownership-existing-bucket.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3b93b667",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = s3.put_bucket_ownership_controls(\n",
    "    Bucket=f'{bucket}',\n",
    "    OwnershipControls={\n",
    "        'Rules': [\n",
    "            {\n",
    "                'ObjectOwnership': 'BucketOwnerPreferred'\n",
    "            },\n",
    "        ]\n",
    "    }\n",
    ")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82198d58",
   "metadata": {},
   "source": [
    "Upload the packaged Lambda zip file to S3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d106baa8",
   "metadata": {},
   "outputs": [],
   "source": [
    "s3_resource.Object(bucket, 'lambda/lambda.zip').upload_file('./backend/lambda/lambda.zip',ExtraArgs={'ACL':'public-read'})\n",
    "lambda_zip_url = f'{bucket}'\n",
    "print(\"lambada zip file url: \" + lambda_zip_url)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0dcb572",
   "metadata": {},
   "source": [
    "### 9. Deploy a CloudFormation stack to create API Gateway and Lambda function\n",
    "\n",
    "Next, we'll create a link to deploy a CloudFormation stack for our SAM application. Execute the following code block to generate a web link.\n",
    "\n",
    "### Note: Click the generated link to deploy new CloudFormation template for Lambda and API Gateway. Mark all the checkboxes at the end of the form and click \"Create Stack\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0b9f7105",
   "metadata": {},
   "outputs": [],
   "source": [
    "s3_resource.Object(bucket, 'backend/template.yaml').upload_file('./backend/template.yaml', ExtraArgs={'ACL':'public-read'})\n",
    "\n",
    "\n",
    "sam_template_url = f'https://{bucket}.s3.amazonaws.com/backend/template.yaml'\n",
    "print(\"cloudformation template url:\" + sam_template_url)\n",
    "\n",
    "\n",
    "# Generate the CloudFormation Quick Create Link\n",
    "\n",
    "print(\"Click the URL below to create the backend API for semantic search:\\n\")\n",
    "print((\n",
    "    'https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/review'\n",
    "    f'?templateURL={sam_template_url}'\n",
    "    '&stackName=semantic-search-api'\n",
    "    f'&param_BucketName={outputs[\"s3BucketTraining\"]}'\n",
    "    f'&param_DomainName={outputs[\"OpenSearchDomainName\"]}'\n",
    "    f'&param_ElasticSearchURL={outputs[\"OpenSearchDomainEndpoint\"]}'\n",
    "    f'&param_SagemakerEndpoint={predictor.endpoint}'\n",
    "    f'&param_LambdaZipFile={lambda_zip_url}'\n",
    "))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8582509",
   "metadata": {},
   "source": [
    "### 10. Wait for the CloudFormation stack to complete.\n",
    "Before proceeding further, wait for the CloudFormation stack to become complete. The status should change to \"CREATE_COMPLETE\".\n",
    "\n",
    "### 11. Update the front end config\n",
    "\n",
    "Next, we need to update the config of the front end with the API values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "47c593c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "cfn = boto3.client('cloudformation')\n",
    "\n",
    "def get_cfn_outputs(stackname):\n",
    "    outputs = {}\n",
    "    for output in cfn.describe_stacks(StackName=stackname)['Stacks'][0]['Outputs']:\n",
    "        outputs[output['OutputKey']] = output['OutputValue']\n",
    "    return outputs\n",
    "\n",
    "api_endpoint = get_cfn_outputs('semantic-search-api')['TextSimilarityApi']\n",
    "\n",
    "with open('./frontend/src/config/config.json', 'w') as outfile:\n",
    "    json.dump({'apiEndpoint': api_endpoint}, outfile)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9050f8d9",
   "metadata": {},
   "source": [
    "### 12. Deploy frontend services\n",
    "\n",
    "Now that we've updated the configuration, we need to build and deploy our front end."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "65d7b3c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# add NPM to the path so we can assemble the web frontend from our notebook code\n",
    "\n",
    "from os import environ\n",
    "\n",
    "npm_path = ':/home/ec2-user/anaconda3/envs/JupyterSystemEnv/bin'\n",
    "\n",
    "if npm_path not in environ['PATH']:\n",
    "    ADD_NPM_PATH = environ['PATH']\n",
    "    ADD_NPM_PATH = ADD_NPM_PATH + npm_path\n",
    "else:\n",
    "    ADD_NPM_PATH = environ['PATH']\n",
    "    \n",
    "%set_env PATH=$ADD_NPM_PATH"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c0ff07e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "%cd ./frontend/\n",
    "\n",
    "!npm install"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "57e85ffd",
   "metadata": {},
   "outputs": [],
   "source": [
    "!npm run-script build"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6faef05",
   "metadata": {},
   "source": [
    "### Disable web page S3  bucket level public access block"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eea3e2cc",
   "metadata": {},
   "outputs": [],
   "source": [
    "host_bucket = f\"{outputs['s3BucketHostingBucketName']}\"\n",
    "print(host_bucket)\n",
    "response = s3.put_public_access_block(\n",
    "    Bucket=host_bucket,\n",
    "    PublicAccessBlockConfiguration={\n",
    "        'BlockPublicAcls': False,\n",
    "        'IgnorePublicAcls': False,\n",
    "        'BlockPublicPolicy': False,\n",
    "        'RestrictPublicBuckets': False\n",
    "    }\n",
    ")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51adb039",
   "metadata": {},
   "source": [
    "### Change web page  S3 bucket ownership settings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4905e757",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = s3.put_bucket_ownership_controls(\n",
    "    Bucket=host_bucket,\n",
    "    OwnershipControls={\n",
    "        'Rules': [\n",
    "            {\n",
    "                'ObjectOwnership': 'BucketOwnerPreferred'\n",
    "            },\n",
    "        ]\n",
    "    }\n",
    ")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "95cd925a",
   "metadata": {},
   "outputs": [],
   "source": [
    "hosting_bucket = f\"s3://{outputs['s3BucketHostingBucketName']}\"\n",
    "\n",
    "!aws s3 sync ./build/ $hosting_bucket --acl public-read"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46635b94",
   "metadata": {},
   "source": [
    "### 13. Browse to the application\n",
    "\n",
    "Now that the application is deployed, let's browse to the front end and test it out. \n",
    "\n",
    "### Note: Execute the following and click on the link generated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b5f7a617",
   "metadata": {},
   "outputs": [],
   "source": [
    "print('Click the URL below:\\n')\n",
    "print(outputs['S3BucketSecureURL'] + '/index.html')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3e98b16",
   "metadata": {},
   "source": [
    "You can search the question, for example \"does this work with xbox?\", compare the search result. you will see the difference between keyword search and semantic search.\n",
    "\n",
    "![full stack semantic search](full-stack-semantic-search-ui.jpg)\n",
    "\n",
    "In keyword search, some questions like \"Does this work for a switch?\", \"does this work with pc\" which include \"does this work\" are searched however the meaning is totally different with query.\n",
    "\n",
    "In semantic search, some questions like \"Do I need to buy anything extra to used in xbox one s controller?\", \"How do these headphones connect to the Xbox360 controller?\" are searched. The meaning is very close to the query.\n",
    "![full stack semantic search](full-stack-semantic-search-ui-2.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3cb7c404",
   "metadata": {},
   "source": [
    "## Cleanup"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08cad511",
   "metadata": {},
   "source": [
    "### Restore S3 bucket ownership"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3d135998",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = s3.put_bucket_ownership_controls(\n",
    "    Bucket=f'{bucket}',\n",
    "    OwnershipControls={\n",
    "        'Rules': [\n",
    "            {\n",
    "                'ObjectOwnership': 'BucketOwnerEnforced'\n",
    "            },\n",
    "        ]\n",
    "    }\n",
    ")\n",
    "print(response)\n",
    "\n",
    "response = s3.put_bucket_ownership_controls(\n",
    "    Bucket=f'{host_bucket}',\n",
    "    OwnershipControls={\n",
    "        'Rules': [\n",
    "            {\n",
    "                'ObjectOwnership': 'BucketOwnerEnforced'\n",
    "            },\n",
    "        ]\n",
    "    }\n",
    ")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62003c1a",
   "metadata": {},
   "source": [
    "### Block S3 public access"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "90099bf9",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = s3.put_public_access_block(\n",
    "    Bucket=f'{bucket}',\n",
    "    PublicAccessBlockConfiguration={\n",
    "        'BlockPublicAcls': True,\n",
    "        'IgnorePublicAcls': True,\n",
    "        'BlockPublicPolicy': True,\n",
    "        'RestrictPublicBuckets': True\n",
    "    }\n",
    ")\n",
    "response = s3.put_public_access_block(\n",
    "    Bucket=f'{host_bucket}',\n",
    "    PublicAccessBlockConfiguration={\n",
    "        'BlockPublicAcls': True,\n",
    "        'IgnorePublicAcls': True,\n",
    "        'BlockPublicPolicy': True,\n",
    "        'RestrictPublicBuckets': True\n",
    "    }\n",
    ")\n",
    "\n",
    "response = client.put_public_access_block(\n",
    "    PublicAccessBlockConfiguration={\n",
    "        'BlockPublicAcls': True,\n",
    "        'IgnorePublicAcls': True,\n",
    "        'BlockPublicPolicy': True,\n",
    "        'RestrictPublicBuckets': True\n",
    "    },\n",
    "    AccountId=my_account\n",
    ")\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ed9e1920",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3cf35368",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_pytorch_p310",
   "language": "python",
   "name": "conda_pytorch_p310"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}