labs/GPT-4o-inferencing/GPT-4o-inferencing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# APIM ❤️ OpenAI\n",
    "\n",
    "## GPT-4o inferencing lab\n",
    "![flow](../../images/GPT-4o-inferencing.gif)\n",
    "\n",
    "Playground to try the new GPT-4o model. GPT-4o (\"o\" for \"omni\") is designed to handle a combination of text, audio, and video inputs, and can generate outputs in text, audio, and image formats.\n",
    "\n",
    "### Prerequisites\n",
    "- [Python 3.12 or later version](https://www.python.org/) installed\n",
    "- [Pandas Library](https://pandas.pydata.org/) installed\n",
    "- [VS Code](https://code.visualstudio.com/) installed with the [Jupyter notebook extension](https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter) enabled\n",
    "- [Azure CLI](https://learn.microsoft.com/cli/azure/install-azure-cli) installed\n",
    "- [An Azure Subscription](https://azure.microsoft.com/free/) with Contributor permissions\n",
    "- [Access granted to Azure OpenAI](https://aka.ms/oai/access) or just enable the mock service\n",
    "- [Sign in to Azure with Azure CLI](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='0'></a>\n",
    "### 0️⃣ Initialize notebook variables\n",
    "\n",
    "- Resources will be suffixed by a unique string based on your subscription id.\n",
    "- Adjust the location parameters according your preferences and on the [product availability by Azure region.](https://azure.microsoft.com/explore/global-infrastructure/products-by-region/?cdn=disable&products=cognitive-services,api-management) \n",
    "- Adjust the OpenAI model and version according the [availability by region.](https://learn.microsoft.com/azure/ai-services/openai/concepts/models) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "metadata": {}
   },
   "outputs": [],
   "source": [
    "import os, sys, json\n",
    "sys.path.insert(1, '../../shared')  # add the shared directory to the Python path\n",
    "import utils\n",
    "\n",
    "deployment_name = os.path.basename(os.path.dirname(globals()['__vsc_ipynb_file__']))\n",
    "resource_group_name = f\"lab-{deployment_name}\" # change the name to match your naming style\n",
    "resource_group_location = \"uksouth\"\n",
    "\n",
    "apim_sku = 'Basicv2'\n",
    "\n",
    "openai_resources = [ {\"name\": \"openai1\", \"location\": \"uksouth\"}]\n",
    "\n",
    "openai_model_name = \"gpt-4o\"\n",
    "openai_model_version = \"2024-08-06\"\n",
    "openai_model_sku = \"GlobalStandard\"\n",
    "openai_deployment_name = \"gpt-4o\"\n",
    "openai_api_version = \"2024-10-21\"\n",
    "\n",
    "utils.print_ok('Notebook initialized')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='1'></a>\n",
    "### 1️⃣ Verify the Azure CLI and the connected Azure subscription\n",
    "\n",
    "The following commands ensure that you have the latest version of the Azure CLI and that the Azure CLI is connected to your Azure subscription."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output = utils.run(\"az account show\", \"Retrieved az account\", \"Failed to get the current az account\")\n",
    "\n",
    "if output.success and output.json_data:\n",
    "    current_user = output.json_data['user']['name']\n",
    "    tenant_id = output.json_data['tenantId']\n",
    "    subscription_id = output.json_data['id']\n",
    "\n",
    "    utils.print_info(f\"Current user: {current_user}\")\n",
    "    utils.print_info(f\"Tenant ID: {tenant_id}\")\n",
    "    utils.print_info(f\"Subscription ID: {subscription_id}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='2'></a>\n",
    "### 2️⃣ Create deployment using 🦾 Bicep\n",
    "\n",
    "This lab uses [Bicep](https://learn.microsoft.com/azure/azure-resource-manager/bicep/overview?tabs=bicep) to declarative define all the resources that will be deployed in the specified resource group. Change the parameters or the [main.bicep](main.bicep) directly to try different configurations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the resource group if doesn't exist\n",
    "utils.create_resource_group(resource_group_name, resource_group_location)\n",
    "\n",
    "# Define the Bicep parameters\n",
    "bicep_parameters = {\n",
    "    \"$schema\": \"https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#\",\n",
    "    \"contentVersion\": \"1.0.0.0\",\n",
    "    \"parameters\": {\n",
    "        \"apimSku\": { \"value\": apim_sku },\n",
    "        \"openAIConfig\": { \"value\": openai_resources },\n",
    "        \"openAIDeploymentName\": { \"value\": openai_deployment_name },\n",
    "        \"openAIModelName\": { \"value\": openai_model_name },\n",
    "        \"openAIModelSKU\": { \"value\": openai_model_sku },\n",
    "        \"openAIModelVersion\": { \"value\": openai_model_version },\n",
    "        \"openAIAPIVersion\": { \"value\": openai_api_version }\n",
    "    }\n",
    "}\n",
    "\n",
    "# Write the parameters to the params.json file\n",
    "with open('params.json', 'w') as bicep_parameters_file:\n",
    "    bicep_parameters_file.write(json.dumps(bicep_parameters))\n",
    "\n",
    "# Run the deployment\n",
    "output = utils.run(f\"az deployment group create --name {deployment_name} --resource-group {resource_group_name} --template-file main.bicep --parameters params.json\",\n",
    "    f\"Deployment '{deployment_name}' succeeded\", f\"Deployment '{deployment_name}' failed\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='3'></a>\n",
    "### 3️⃣ Get the deployment outputs\n",
    "\n",
    "Retrieve the required outputs from the Bicep deployment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Obtain all of the outputs from the deployment\n",
    "output = utils.run(f\"az deployment group show --name {deployment_name} -g {resource_group_name}\", f\"Retrieved deployment: {deployment_name}\", f\"Failed to retrieve deployment: {deployment_name}\")\n",
    "\n",
    "if output.success and output.json_data:\n",
    "    apim_service_id = utils.get_deployment_output(output, 'apimServiceId', 'APIM Service Id')\n",
    "    apim_resource_gateway_url = utils.get_deployment_output(output, 'apimResourceGatewayURL', 'APIM API Gateway URL')\n",
    "    apim_subscription_key = utils.get_deployment_output(output, 'apimSubscription1Key', 'APIM Subscription 1 Key (masked)', True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='requsts'></a>\n",
    "### 🧪 Test the API using a direct HTTP call\n",
    "Requests is an elegant and simple HTTP library for Python that will be used here to make raw API requests and inspect the responses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time, requests\n",
    "runs = 1\n",
    "sleep_time_ms = 1000\n",
    "\n",
    "url = apim_resource_gateway_url + \"/openai/deployments/\" + openai_deployment_name + \"/chat/completions?api-version=\" + openai_api_version\n",
    "\n",
    "for i in range(runs):\n",
    "    print(\"▶️ Run: \", i+1)\n",
    "    messages={\"messages\":[\n",
    "        {\"role\": \"system\", \"content\": \"You are a sarcastic, unhelpful assistant.\"},\n",
    "        {\"role\": \"user\", \"content\": \"Can you tell me the time, please?\"}\n",
    "    ]}\n",
    "    response = requests.post(url, headers = {'api-key':apim_subscription_key}, json = messages)\n",
    "    print(\"status code: \", response.status_code)\n",
    "    print(\"headers \", response.headers)\n",
    "    print(\"x-ms-region: \", response.headers.get(\"x-ms-region\")) # this header is useful to determine the region of the backend that served the request\n",
    "    if (response.status_code == 200):\n",
    "        data = json.loads(response.text)\n",
    "        print(\"💬 \", data.get(\"choices\")[0].get(\"message\").get(\"content\"))\n",
    "    else:\n",
    "        print(response.text)\n",
    "    time.sleep(sleep_time_ms/1000)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='sdk1'></a>\n",
    "### 🧪 Test the API using the Azure OpenAI Python SDK\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "from openai import AzureOpenAI\n",
    "runs = 1\n",
    "sleep_time_ms = 1000\n",
    "\n",
    "for i in range(runs):\n",
    "    print(\"▶️ Run: \", i+1)\n",
    "    messages=[\n",
    "        {\"role\": \"system\", \"content\": \"You are a sarcastic, unhelpful assistant.\"},\n",
    "        {\"role\": \"user\", \"content\": \"Can you tell me the time, please?\"}\n",
    "    ]\n",
    "    client = AzureOpenAI(\n",
    "        azure_endpoint=apim_resource_gateway_url,\n",
    "        api_key=apim_subscription_key,\n",
    "        api_version=openai_api_version\n",
    "    )\n",
    "    response = client.chat.completions.create(model=openai_model_name, messages=messages)\n",
    "    print(\"💬 \", response.choices[0].message.content)\n",
    "    time.sleep(sleep_time_ms/1000)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='sdk2'></a>\n",
    "### 🧪 Image Processing\n",
    "\n",
    "GPT-4o can directly process images and take intelligent actions based on the image. \n",
    "\n",
    "The following example is adapted from the [OpenAI Cookbook](https://cookbook.openai.com/examples/gpt4o/introduction_to_gpt4o)\n",
    "\n",
    "#### Input image:\n",
    "![image](https://upload.wikimedia.org/wikipedia/commons/e/e2/The_Algebra_of_Mohammed_Ben_Musa_-_page_82b.png)\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import display, Markdown\n",
    "response = client.chat.completions.create(\n",
    "    model=openai_model_name,\n",
    "    messages=[\n",
    "        {\"role\": \"system\", \"content\": \"You are a helpful assistant that responds in Markdown. Help me with my math homework!\"},\n",
    "        {\"role\": \"user\", \"content\": [\n",
    "            {\"type\": \"text\", \"text\": \"What's the area of the triangle?\"},\n",
    "            {\"type\": \"image_url\", \"image_url\": {\n",
    "                \"url\": \"https://upload.wikimedia.org/wikipedia/commons/e/e2/The_Algebra_of_Mohammed_Ben_Musa_-_page_82b.png\"}\n",
    "            }\n",
    "        ]}\n",
    "    ],\n",
    "    temperature=0.0,\n",
    ")\n",
    "\n",
    "display(Markdown(response.choices[0].message.content))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='sdk3'></a>\n",
    "### 🧪 Base64 Image Processing\n",
    "\n",
    "#### Input image:\n",
    "<img src=\"meme.png\" width=\"600px\">\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import display, Markdown, HTML\n",
    "import base64\n",
    "def encode_image(image_path):\n",
    "    with open(image_path, \"rb\") as image_file:\n",
    "        return base64.b64encode(image_file.read()).decode(\"utf-8\")\n",
    "base64_image = encode_image(\"meme.png\")\n",
    "response = client.chat.completions.create(\n",
    "    model=openai_model_name,\n",
    "    messages=[\n",
    "        {\"role\": \"system\", \"content\": \"You are a helpful assistant that helps to interprete memes!\"},\n",
    "        {\"role\": \"user\", \"content\": [\n",
    "            {\"type\": \"text\", \"text\": \"What's funny about this picture?\"},\n",
    "            {\"type\": \"image_url\", \"image_url\": {\n",
    "                \"url\": f\"data:image/png;base64,{base64_image}\"}\n",
    "            }\n",
    "        ]}\n",
    "    ],\n",
    "    temperature=0.0,\n",
    ")\n",
    "HTML(f'<font size=\"5\">{response.choices[0].message.content}</font>')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='clean'></a>\n",
    "### 🗑️ Clean up resources\n",
    "\n",
    "When you're finished with the lab, you should remove all your deployed resources from Azure to avoid extra charges and keep your Azure subscription uncluttered.\n",
    "Use the [clean-up-resources notebook](clean-up-resources.ipynb) for that."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}