03b-Querying-Compare-AOAI-Llama2-70B.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "d59d527f-1100-45ff-b051-5f7c9029d94d",
      "metadata": {},
      "source": [
        "# Comparing Azure OpenAI with Llama2-70B"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "eb9a9444-dc90-4fc3-aea7-8ee918301aba",
      "metadata": {},
      "source": [
        "So far, you have your Search Engine loaded **from two different data sources in two diferent text-based indexes**, and have experimented querying with Azure OpenAI service to see if we can get even better results.\n",
        "\n",
        "The idea is that a user can ask a question about Computer Science (first datasource/index) or about Covid (second datasource/index), and the engine will respond accordingly.\n",
        "\n",
        "In this notebook, we will query Azure OpenAI and Llama2-70B model with the same question to compare and contrast the response. \n",
        "\n",
        "You can learn more about Llama2 family models here - https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-llama?tabs=azure-studio"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8e43a9be",
      "metadata": {},
      "source": [
        "## Pre-requisites\n",
        "You must have these resources deployed prior to running this notebook.\n",
        "\n",
        "1. Llama2-70B deployed on Azure Machine Learning Endpoints.  You can deploy this model from the Azure AI Studio following the instructions here - https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-open?tabs=azure-studio\n",
        "\n",
        "langchain_community.llms.azureml_endpoint import AzureMLOnlineEndpoint library is required to integrate Langchain Prompts with Llama2-70B model deployed on Azure Machine Learning Endpoints."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "95d792b8",
      "metadata": {},
      "source": [
        "## Install Dependencies\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ec4e0dce",
      "metadata": {},
      "outputs": [],
      "source": [
        "%pip install -r common/requirements.txt"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "71f6c7e3-9037-4b1e-ae17-1deaa27b9c08",
      "metadata": {},
      "source": [
        "## Set up variables"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "id": "8e50b404-a061-49e7-a3c7-c6eabc98ff0f",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "execution_count": 1,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import os\n",
        "import urllib\n",
        "import requests\n",
        "import random\n",
        "import json\n",
        "from collections import OrderedDict\n",
        "from IPython.display import display, HTML, Markdown\n",
        "from langchain.chains import LLMChain\n",
        "from langchain.prompts import PromptTemplate\n",
        "from langchain.chat_models import AzureChatOpenAI\n",
        "from langchain.docstore.document import Document\n",
        "from langchain.chains.qa_with_sources import load_qa_with_sources_chain\n",
        "from langchain.embeddings import AzureOpenAIEmbeddings\n",
        "from langchain_community.llms.azureml_endpoint import AzureMLOnlineEndpoint\n",
        "\n",
        "from common.prompts import COMBINE_QUESTION_PROMPT, COMBINE_PROMPT_SHORT, COMBINE_PROMPT_TEMPLATE_SHORT\n",
        "from common.utils import (\n",
        "    get_search_results,\n",
        "    model_tokens_limit,\n",
        "    num_tokens_from_docs,\n",
        "    num_tokens_from_string\n",
        ")\n",
        "\n",
        "from dotenv import load_dotenv\n",
        "load_dotenv(\"credentials.env\", override=True)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "id": "2f2c22f8-79ab-405c-95e8-77a1978e53bc",
      "metadata": {},
      "outputs": [],
      "source": [
        "# Setup the Payloads header\n",
        "headers = {'Content-Type': 'application/json','api-key': os.environ['AZURE_SEARCH_KEY']}\n",
        "params = {'api-version': os.environ['AZURE_SEARCH_API_VERSION']}"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1c62ebb2-d7be-4bfb-b1ba-4db86c11839a",
      "metadata": {},
      "source": [
        "Try questions against Azure OpenAI and Llama2-70B that you think might be answered or addressed in computer science papers in 2020-2021 or that can be addressed by medical publications about COVID in 2020-2021. Compare and contrast the answer quality.<br>\n",
        "\n",
        "\n",
        "**Example Questions you can ask**:\n",
        "- What is CLP?\n",
        "- How Markov chains work?\n",
        "- What are some examples of reinforcement learning?\n",
        "- What are the main risk factors for Covid-19?\n",
        "- What medicine reduces inflamation in the lungs?\n",
        "- Why Covid doesn't affect kids that much compared to adults?\n",
        "- Does chloroquine really works against covid?\n",
        "- Who won the 1994 soccer world cup? # This question should yield no answer if the system is correctly grounded"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "id": "b9b53c14-19bd-451f-aa43-7ad27ccfeead",
      "metadata": {},
      "outputs": [],
      "source": [
        "QUESTION = \"What is CLP?\""
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9297d29b-1f61-4dce-858e-bf4272172dba",
      "metadata": {},
      "source": [
        "## Multi-Index Search queries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "id": "5a46e2d3-298a-4708-83de-9e108b1a117a",
      "metadata": {
        "scrolled": true,
        "tags": []
      },
      "outputs": [],
      "source": [
        "# Text-based Indexes that we are going to query (from Notebook 01 and 02)\n",
        "index1_name = \"cogsrch-index-files\"\n",
        "index2_name = \"cogsrch-index-csv\"\n",
        "indexes = [index1_name, index2_name]"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "f6d925eb-7f9c-429e-a62a-4c37d7702caf",
      "metadata": {},
      "source": [
        "### Search on both indexes individually and aggragate results\n",
        "\n",
        "#### **Note**: \n",
        "In order to standarize the indexes, **there must be 8 mandatory fields present on each text-based index**: `id, title, content, chunks, language, name, location, vectorized`. This is so that each document can be treated the same along the code. Also, **all indexes must have a semantic configuration**."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "id": "faf2e30f-e71f-4533-ab52-27d048b80a89",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "200\n",
            "Index: cogsrch-index-files Results Found: 9789, Results Returned: 10\n",
            "200\n",
            "Index: cogsrch-index-csv Results Found: 48638, Results Returned: 10\n"
          ]
        }
      ],
      "source": [
        "agg_search_results = dict()\n",
        "\n",
        "for index in indexes:\n",
        "    search_payload = {\n",
        "        \"search\": QUESTION,\n",
        "        \"select\": \"id, title, chunks, name, location\",\n",
        "        \"queryType\": \"semantic\",\n",
        "        \"semanticConfiguration\": \"my-semantic-config\",\n",
        "        \"count\": \"true\",\n",
        "        \"speller\": \"lexicon\",\n",
        "        \"queryLanguage\": \"en-us\",\n",
        "        \"captions\": \"extractive\",\n",
        "        \"answers\": \"extractive\",\n",
        "        \"top\": \"10\"\n",
        "    }\n",
        "\n",
        "    r = requests.post(os.environ['AZURE_SEARCH_ENDPOINT'] + \"/indexes/\" + index + \"/docs/search\",\n",
        "                     data=json.dumps(search_payload), headers=headers, params=params)\n",
        "    print(r.status_code)\n",
        "\n",
        "    search_results = r.json()\n",
        "    agg_search_results[index]=search_results\n",
        "    print(\"Index:\", index, \"Results Found: {}, Results Returned: {}\".format(search_results['@odata.count'], len(search_results['value'])))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b7fd0fe5-4ee0-42e2-a920-72b93a407389",
      "metadata": {
        "tags": []
      },
      "source": [
        "### Display the top results (from both searches) based on the re-ranker score"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "id": "9e938337-602d-4b61-8141-b8c92a5d91da",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<h4>Top Answers</h4>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5>Answer - score: 0.97</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "The key notions of CLP are those of an algebra and an associated constraint solver over a class of constraints, namely a set of first order formulas including the always satisfiable constraint true, the un- satisfiable constraint false, and closed under variable renaming, conjunction and existential quantification."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5>Answer - score: 0.93</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "Isotype-specific antibody responses to rotavirus and virus proteins in cows inoculated with subunit vaccines composed of recombinant SA11 rotavirus core-like particles (CLP) or virus-like particles (VLP)."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "\n",
            "\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "<h4>Top Results</h4>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://datasetsgptsmartsearch.blob.core.windows.net/arxivcs/pdf/0508/0508108v1.pdf?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">0508108v1.pdf</a> - score: 3.55</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "CLP(FD) is an extension of logic programming. In CLP(FD) pro- grams, logical variables are assigned a domain and relations between vari- ables are described with constraints. A solution to a CLP(FD) program is a valuation of every variable in its own domain such that no constraint is falsified."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://datasetsgptsmartsearch.blob.core.windows.net/arxivcs/pdf/0701/0701082v1.pdf?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">0701082v1.pdf</a> - score: 3.46</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "The key notions of CLP are those of an algebra and an associated constraint solver over a class of constraints, namely a set of first order formulas including the always satisfiable constraint true, the un- satisfiable constraint false, and closed under variable renaming, conjunction and existential quantification."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://datasetsgptsmartsearch.blob.core.windows.net/arxivcs/pdf/0508/0508106v1.pdf?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">arXiv:cs/0508106v1  [cs.PL]  24 Aug 2005</a> - score: 3.13</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "A CLP(C) program is a finite set of rules. A rule has the form H ← c⋄B where H and B are atoms and c is a finite conjunction of primitive constraints such that DC |= ∃c. A query has the form 〈A | d〉 where A is an atom and d is a finite conjunction of primitive constraints. Given an atom A := p(t̃), we write rel(A) to denote the predicate symbol p."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://datasetsgptsmartsearch.blob.core.windows.net/arxivcs/pdf/0011/0011030v1.pdf?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">arXiv:cs/0011030v1  [cs.AI]  21 Nov 2000</a> - score: 3.12</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "A solution is an instantiation of the variables of X which satisfies all the constraints in R.  2.1 Constraint Logic Programming  Constraint logic programming (CLP) [7] is an extension of logic programming where some of the predicate and function symbols have a fixed interpretation over some subdomain (e.g. finite trees or real numbers)."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://datasetsgptsmartsearch.blob.core.windows.net/arxivcs/pdf/0506/0506005v1.pdf?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">0506005v1.pdf</a> - score: 3.1</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "A CLP(FD) system provides primitives for  accessing and updating attribute values. A CLP(FD) system provides equality (=), disequality (6=), and inequality con-  straints. In addition, a CLP(FD) system also provides some other constraints such  as global constraints."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://datasetsgptsmartsearch.blob.core.windows.net/arxivcs/pdf/0408/0408056v1.pdf?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">0408056v1.pdf</a> - score: 3.07</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "CLP(FD) languages have been suc-  cessfully used for solving a variety of industrial and academic problems. However,  in some constraint problems, where domain elements need to be acquired, it may  not be wise to perform the acquisition of the whole domains of variables before the  beginning of the constraint propagation process."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://datasetsgptsmartsearch.blob.core.windows.net/arxivcs/pdf/0310/0310042v1.pdf?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">()</a> - score: 3.02</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "A CLP(FD) program searches a solution for a set of variables which take values over finite domains and which must verify a set of constraints. The evolution of the domains can be viewed as a sequence of applications of reduction operators attached to the constraints."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://datasetsgptsmartsearch.blob.core.windows.net/arxivcs/pdf/0003/0003026v1.pdf?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">arXiv:cs/0003026v1  [cs.LO]  8 Mar 2000</a> - score: 2.97</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "A proof procedure for CLP is defined as an extension of standard resolution. A state is defined as a pair 〈← a, A || C〉 of a goal and a set of constraints. At each step of the computation, some literal a is selected from the current goal according to some selection function."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://datasetsgptsmartsearch.blob.core.windows.net/arxivcs/pdf/0402/0402019v1.pdf?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">0402019v1.pdf</a> - score: 2.89</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "CLP combines the advantages of two declarative  paradigms: logic programming (Prolog) and constraint solving. In logic program-  ming, problems are stated in a declarative way using rules to define relations (predi-  cates). Problems are solved using chronological backtrack search to explore choices."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pubmed/10403670/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Quantification of recombinant core-like particles of bluetongue virus using immunosorbent electron microscopy.</a> - score: 2.79</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "Immunosorbent electron microscopy was used to quantify recombinant baculovirus-generated bluetongue virus (BTV) core-like particles (CLP) in either purified preparations or lysates of recombinant baculovirus-infected cells. The capture antibody was an anti-BTV VP7 monoclonal antibody."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://datasetsgptsmartsearch.blob.core.windows.net/arxivcs/pdf/0404/0404053v1.pdf?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">0404053v1.pdf</a> - score: 2.77</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "We will define an amalgamated proof system that combines  inference rules from intuitionistic sequent calculus with constraint entailment, in  such a way that the key property of an abstract logic programming language is  preserved."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pubmed/17047515/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Increased susceptibility to septic and endotoxic shock in monocyte chemoattractant protein 1/cc chemokine ligand 2-deficient mice correlates with reduced interleukin 10 and enhanced macrophage migration inhibitory factor production.</a> - score: 2.55</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "The chemokine monocyte chemoattractant protein 1/CC chemokine ligand 2 (MCP-1/CCL2) is a potent chemoattractant of mononuclear cells and a regulatory mediator involved in a variety of inflammatory diseases."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7103146/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Length of encapsidated cargo impacts stability and structure of in vitro assembled alphavirus core-like particles</a> - score: 2.54</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "In vitro assembly of alphavirus nucleocapsid cores, called core-like particles (CLPs), requires a polyanionic cargo. There are no sequence or structure requirements to encapsidate single-stranded nucleic acid cargo. In this work, we wanted to determine how the length of the cargo impacts the stability and structure of the assembled CLPs."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://api.elsevier.com/content/article/pii/S0033318220301420; https://www.sciencedirect.com/science/article/pii/S0033318220301420?v=s5?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">The educational value of outpatient CL rotations- a white paper from the ACLP residency education subcommittee</a> - score: 2.46</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "The Academy of Consultation-Liaison Psychiatry (ACLP) residency education subcommittee convened a writing group with the goal of summarizing the current evidence about outpatient consultation-liaison psychiatry (CLP) training and providing a framework for CLP educators who are interested in developing outpatient CLP rotations within their programs."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pubmed/16286234/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">What is the contribution of respiratory viruses and lung proteases to airway remodelling in asthma and chronic obstructive pulmonary disease?</a> - score: 2.29</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "Chronic obstructive pulmonary disease (COPD), by definition, involves structural changes to the airways. However, very little is known about what role virus infections play in the development of this remodelling."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7131174/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Isotype-specific antibody responses to rotavirus and virus proteins in cows inoculated with subunit vaccines composed of recombinant SA11 rotavirus core-like particles (CLP) or virus-like particles (VLP)</a> - score: 2.26</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "Isotype-specific antibody responses to rotavirus and virus proteins in cows inoculated with subunit vaccines composed of recombinant SA11 rotavirus core-like particles (CLP) or virus-like particles (VLP)."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5809586/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">A cold-inducible RNA-binding protein (CIRP)-derived peptide attenuates inflammation and organ injury in septic mice</a> - score: 2.11</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "Cold-inducible RNA-binding protein (CIRP) is a novel sepsis inflammatory mediator and C23 is a putative CIRP competitive inhibitor. Therefore, we hypothesized that C23 can ameliorate sepsis-associated injury to the lungs and kidneys."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pubmed/17414124/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Community-acquired pneumonia: what is relevant and what is not?</a> - score: 1.88</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "The mainstay of community-acquired pneumonia prevention is influenza and pneumococcal immunization. Promotion of smoking cessation will also help curtail the incidence of pneumococcal disease..\u0000"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7131086/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Passive immunity to bovine rotavirus in newborn calves fed colostrum supplements from cows immunized with recombinant SA11 rotavirus core-like particle (CLP) or virus-like particle (VLP) vaccines</a> - score: 1.82</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "Passive immunity to bovine rotavirus in newborn calves fed colostrum supplements from cows immunized with recombinant SA11 rotavirus core-like particle (CLP) or virus-like particle (VLP) vaccines. Heterotypic passive immunity to IND (P[5]G6) bovine rotavirus (BRV) was evaluated."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "<h5><a href=\"https://doi.org/10.1111/bjh.14134; https://www.ncbi.nlm.nih.gov/pubmed/27173746/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-01-03T02:11:44Z&st=2024-01-02T18:11:44Z&spr=https&sig=ngrEqvqBVaxyuSYqgPVeF%2B9c0fXLs94v3ASgwg7LDBs%3D\">Adherence to long-term anticoagulation treatment, what is known and what the future might hold.</a> - score: 1.78</h5>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "utilizing the com-b (capability, opportunity, motivation and behaviour) psychological model of non-adherence, we present the available evidence, not only in terms of describing the extent of the non-adherence problem, but also describing why patients do not adhere, offering theory-driven and evidence-based solutions to improve long-term adherence …"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "display(HTML('<h4>Top Answers</h4>'))\n",
        "\n",
        "for index,search_results in agg_search_results.items():\n",
        "    for result in search_results['@search.answers']:\n",
        "        if result['score'] > 0.5: # Show answers that are at least 50% of the max possible score=1\n",
        "            display(HTML('<h5>' + 'Answer - score: ' + str(round(result['score'],2)) + '</h5>'))\n",
        "            display(HTML(result['text']))\n",
        "            \n",
        "print(\"\\n\\n\")\n",
        "display(HTML('<h4>Top Results</h4>'))\n",
        "\n",
        "content = dict()\n",
        "ordered_content = OrderedDict()\n",
        "\n",
        "\n",
        "for index,search_results in agg_search_results.items():\n",
        "    for result in search_results['value']:\n",
        "        if result['@search.rerankerScore'] > 1:# Show answers that are at least 25% of the max possible score=4\n",
        "            content[result['id']]={\n",
        "                                    \"title\": result['title'],\n",
        "                                    \"chunks\": result['chunks'], \n",
        "                                    \"chunks_vectors\": [],\n",
        "                                    \"name\": result['name'], \n",
        "                                    \"location\": result['location'] ,\n",
        "                                    \"caption\": result['@search.captions'][0]['text'],\n",
        "                                    \"score\": result['@search.rerankerScore'],\n",
        "                                    \"index\": index\n",
        "                                    }\n",
        "    \n",
        "#After results have been filtered we will Sort and add them as an Ordered list\\n\",\n",
        "for id in sorted(content, key= lambda x: content[x][\"score\"], reverse=True):\n",
        "    ordered_content[id] = content[id]\n",
        "    url = str(ordered_content[id]['location']) + os.environ['BLOB_SAS_TOKEN']\n",
        "    title = str(ordered_content[id]['title']) if (ordered_content[id]['title']) else ordered_content[id]['name']\n",
        "    score = str(round(ordered_content[id]['score'],2))\n",
        "    display(HTML('<h5><a href=\"'+ url + '\">' + title + '</a> - score: '+ score + '</h5>'))\n",
        "    display(HTML(ordered_content[id]['caption']))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b0454ddb-44d8-4fa9-929a-5e5563dd28f8",
      "metadata": {},
      "source": [
        "## Embeddings and Vector Search\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7714f38a-daaa-4fc5-a95a-dd025d153216",
      "metadata": {},
      "outputs": [],
      "source": [
        "k = 10 # Number of results per each text_index\n",
        "ordered_results = get_search_results(QUESTION, indexes, k=10, reranker_threshold=1)\n",
        "#print(\"Number of results:\",len(ordered_results))\n",
        "# Uncomment the below line if you want to inspect the ordered results\n",
        "# ordered_results"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "id": "2937ba3b-098d-43f8-8498-3534882a5cc7",
      "metadata": {},
      "outputs": [],
      "source": [
        "embedder = AzureOpenAIEmbeddings(model=\"text-embedding-ada-002\", skip_empty=True) "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "id": "f664df30-99c3-4a30-8cb0-42ba3044e5b0",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "CPU times: total: 0 ns\n",
            "Wall time: 0 ns\n"
          ]
        }
      ],
      "source": [
        "%%time\n",
        "for key,value in ordered_results.items():\n",
        "    if value[\"vectorized\"] != True: # If the document has not been vectorized yet\n",
        "        i = 0\n",
        "        print(\"Vectorizing\",len(value[\"chunks\"]),\"chunks from Document:\",value[\"location\"])\n",
        "        for chunk in value[\"chunks\"]: # Iterate over the document's text chunks\n",
        "            try:\n",
        "                upload_payload = {  # Insert the chunk and its vector in the vector-based index\n",
        "                    \"value\": [\n",
        "                        {\n",
        "                            \"id\": key + \"_\" + str(i),\n",
        "                            \"title\": f\"{value['title']}_chunk_{str(i)}\",\n",
        "                            \"chunk\": chunk,\n",
        "                            \"chunkVector\": embedder.embed_query(chunk if chunk!=\"\" else \"-------\"),\n",
        "                            \"name\": value[\"name\"],\n",
        "                            \"location\": value[\"location\"],\n",
        "                            \"@search.action\": \"upload\"\n",
        "                        },\n",
        "                    ]\n",
        "                }\n",
        "\n",
        "                r = requests.post(os.environ['AZURE_SEARCH_ENDPOINT'] + \"/indexes/\" + value[\"index\"]+\"-vector\" + \"/docs/index\",\n",
        "                                     data=json.dumps(upload_payload), headers=headers, params=params)\n",
        "                \n",
        "                if r.status_code != 200:\n",
        "                    print(r.status_code)\n",
        "                    print(r.text)\n",
        "                else:\n",
        "                    i = i + 1 # increment chunk number\n",
        "                \n",
        "            except Exception as e:\n",
        "                print(\"Exception:\",e)\n",
        "                print(chunk)\n",
        "                continue\n",
        "\n",
        "        # Update document in text-based index and mark it as \"vectorized\"\n",
        "        upload_payload = {\n",
        "            \"value\": [\n",
        "                {\n",
        "                    \"id\": key,\n",
        "                    \"vectorized\": True,\n",
        "                    \"@search.action\": \"merge\"\n",
        "                },\n",
        "            ]\n",
        "        }\n",
        "\n",
        "        r = requests.post(os.environ['AZURE_SEARCH_ENDPOINT'] + \"/indexes/\" + value[\"index\"]+ \"/docs/index\",\n",
        "                                     data=json.dumps(upload_payload), headers=headers, params=params)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1f67f3a2-0023-4f5a-b52f-3fb071cfd8e1",
      "metadata": {},
      "source": [
        "Now we search on the vector-based indexes and get the top k most similar chunks to our question:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "61098bb4-33da-4eb4-94cf-503587337aca",
      "metadata": {},
      "outputs": [],
      "source": [
        "vector_indexes = [index+\"-vector\" for index in indexes]\n",
        "\n",
        "k = 10\n",
        "similarity_k = 3\n",
        "ordered_results = get_search_results(QUESTION, vector_indexes,\n",
        "                                        k=k, # Number of results per vector index\n",
        "                                        reranker_threshold=1,\n",
        "                                        vector_search=True, \n",
        "                                        similarity_k=similarity_k,\n",
        "                                        query_vector = embedder.embed_query(QUESTION)\n",
        "                                        );\n",
        "#print(\"Number of results:\", len(ordered_results))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1a98a974-0633-499f-a8f0-29bf6242e737",
      "metadata": {},
      "source": [
        "For vector search is not recommended to give more than k=5 chunks (of max 5000 characters each) to the LLM as context. Otherwise you can have issues later with the token limit trying to have a conversation with memory."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "id": "7dfb9e39-2542-469d-8f64-4c0c26d79535",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Number of chunks: 3\n"
          ]
        }
      ],
      "source": [
        "top_docs = []\n",
        "for key,value in ordered_results.items():\n",
        "    location = value[\"location\"] if value[\"location\"] is not None else \"\"\n",
        "    top_docs.append(Document(page_content=value[\"chunk\"], metadata={\"source\": location}))\n",
        "        \n",
        "print(\"Number of chunks:\",len(top_docs))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 37,
      "id": "03c8ca2c",
      "metadata": {},
      "outputs": [],
      "source": [
        "# If model is not an Azure OpenAI model, model_tokens_limit will set the limit the 4096. \n",
        "MODEL = \"llama2-70b\"\n",
        "COMPLETION_TOKENS = 1000"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "id": "880885fe-16bd-44bb-9556-7cb3d4989993",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "System prompt token count: 824\n",
            "Max Completion Token count: 1000\n",
            "Combined docs (context) token count: 998\n",
            "--------\n",
            "Requested token count: 2822\n",
            "Token limit for llama2-70b : 4096\n",
            "Chain Type selected: stuff\n"
          ]
        }
      ],
      "source": [
        "# Calculate number of tokens of our docs\n",
        "if(len(top_docs)>0):\n",
        "    tokens_limit = model_tokens_limit(MODEL) # this is a custom function we created in common/utils.py\n",
        "    prompt_tokens = num_tokens_from_string(COMBINE_PROMPT_TEMPLATE_SHORT) # this is a custom function we created in common/utils.py\n",
        "    context_tokens = num_tokens_from_docs(top_docs) # this is a custom function we created in common/utils.py\n",
        "    \n",
        "    requested_tokens = prompt_tokens + context_tokens + COMPLETION_TOKENS\n",
        "    \n",
        "    chain_type = \"map_reduce\" if requested_tokens > 0.9 * tokens_limit else \"stuff\"  \n",
        "    \n",
        "    print(\"System prompt token count:\",prompt_tokens)\n",
        "    print(\"Max Completion Token count:\", COMPLETION_TOKENS)\n",
        "    print(\"Combined docs (context) token count:\",context_tokens)\n",
        "    print(\"--------\")\n",
        "    print(\"Requested token count:\",requested_tokens)\n",
        "    print(\"Token limit for\", MODEL, \":\", tokens_limit)\n",
        "    print(\"Chain Type selected:\", chain_type)\n",
        "        \n",
        "else:\n",
        "    print(\"NO RESULTS FROM AZURE SEARCH\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "fa47148a",
      "metadata": {},
      "source": [
        "## Generate grounded LLM responses for Azure OpenAI and Llama2-70B"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "aa7901ae",
      "metadata": {},
      "source": [
        "### Get response from Llama2-70B on Azure ML Endpoints"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 39,
      "id": "bd85d2b3",
      "metadata": {},
      "outputs": [],
      "source": [
        "from langchain.schema import HumanMessage\n",
        "from langchain_community.llms.azureml_endpoint import (\n",
        "    AzureMLEndpointApiType,\n",
        "    LlamaContentFormatter,\n",
        ")\n",
        "\n",
        "llm_llama2_70b = AzureMLOnlineEndpoint(\n",
        "    endpoint_url=\"https://\" + os.environ[\"LLAMA2_70B_ENDPOINT\"] + \".eastus.inference.ml.azure.com/score\",\n",
        "    endpoint_api_type=AzureMLEndpointApiType.realtime,\n",
        "    endpoint_api_key= os.environ[\"LLAMA2_70B_KEY\"],\n",
        "    #content_formatter=LlamaContentFormatter(),\n",
        "    content_formatter=LlamaContentFormatter(),\n",
        "    model_kwargs={\"temperature\": 1, \"max_new_tokens\": 400},\n",
        ")\n",
        "\n",
        "# If you want to invoke the llama endpoint without using langchain prompt, you can use the invoke method.\n",
        "#response = llm_llama2_70b.invoke(\"Write me a song about sparkling water:\")\n",
        "#response"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 40,
      "id": "33f2fd42",
      "metadata": {},
      "outputs": [],
      "source": [
        "chain_type =\"stuff\"\n",
        "chain_llama = load_qa_with_sources_chain(llm_llama2_70b, chain_type=chain_type, \n",
        "                                    prompt=COMBINE_PROMPT_SHORT)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 41,
      "id": "b8414b23",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "CPU times: total: 0 ns\n",
            "Wall time: 16.4 s\n"
          ]
        }
      ],
      "source": [
        "%%time\n",
        "# Try with other language as well\n",
        "response_llama = chain_llama({\"input_documents\": top_docs, \"question\": QUESTION, \"language\": \"English\"})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 42,
      "id": "944c4d5e",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\"\n",
            "These are examples of how you must provide the answer:\n",
            "--> Beginning of examples\n",
            "=========\n",
            "QUESTION: Which state/country's law governs the interpretation of the contract?\n",
            "=========\n",
            "Content: This Agreement is governed by English law and the parties submit to the exclusive jurisdiction of the English courts in  relation to any dispute (contractual or non-contractual) concerning this Agreement save that either party may apply to any court for an  injunction or other relief to protect its Intellectual Property Rights.\n",
            "Source: https://xxx.com/article1.pdf?s=casdfg&category=ab&sort=asc&page=1\n",
            "Content: No Waiver. Failure or delay in exercising any right or remedy under this Agreement shall not constitute a waiver of such (or any other)  right or remedy.\n",
            "11.7 Severability. The invalidity, illegality or unenforceability of any term (or part of a term) of this Agreement shall not affect the continuation  in force of the remainder of the term (if any) and this Agreement.\n",
            "11.8 No Agency. Except as expressly stated otherwise, nothing in this Agreement shall create an agency, partnership or joint venture of any  kind between the parties.\n",
            "11.9 No Third-Party Beneficiaries.\n",
            "Source: https://yyyy.com/article2.html?s=lkhljkhljk&category=c&sort=asc\n",
            "Content: (b) if Google believes, in good faith, that the Distributor has violated or caused Google to violate any Anti-Bribery Laws (as  defined in Clause 8.5) or that such a violation is reasonably likely to occur,\n",
            "Source: https://yyyy.com/article3.csv?s=kjsdhfd&category=c&sort=asc&page=2\n",
            "Content: The terms of this Agreement shall be subject to the laws of Manchester, England, and any disputes arising from or relating to this Agreement shall be exclusively resolved by the courts of that state, except where either party may seek an injunction or other legal remedy to safeguard their Intellectual Property Rights.\n",
            "Source: https://ppp.com/article4.pdf?s=lkhljkhljk&category=c&sort=asc\n",
            "=========\n",
            "FINAL ANSWER IN English: This Agreement is governed by English law, specifically the laws of Manchester, England<sup><a href=\\\"https://xxx.com/article1.pdf?s=casdfg&category=ab&sort=asc&page=1\\\" target=\\\"_blank\\\">[1]</a></sup><sup><a href=\\\"https://ppp.com/article4.pdf?s=lkhljkhljk&category=c&sort=asc\\\" target=\\\"_blank\\\">[2]</a></sup>. \n",
            " Anything else I can help you with?.\n",
            "<-- End of examples\n",
            "# Instructions:\n",
            "- Given the following extracted parts from one or multiple documents, and a question, create a final answer with references. \n",
            "- You can only provide numerical references to documents, using this html format: `<sup><a href=\\\"url?query_parameters\\\" target=\\\"_blank\\\">[number]</a></sup>`.\n",
            "- The reference must be from the `Source:` section of the extracted part. You are not to make a reference from the content, only from the `Source:` of the extract parts.\n",
            "- Reference (source) document's url can include query parameters, for example: \\\"https://example.com/search?query=apple&category=fruits&sort=asc&page=1\\\". On these cases, **you must** include que query references on the document url, using this html format: <sup><a href=\\\"url?query_parameters\\\" target=\\\"_blank\\\">[number]</a></sup>.\n",
            "- **You can only answer the question from information contained in the extracted parts below**, DO NOT use your prior knowledge.\n",
            "- Never provide an answer without references.\n",
            "- If you don't know the answer, just say that you don't know. Don't try to make up an answer.\n",
            "- Respond in English.\n",
            "=========\n",
            "QUESTION: What is CLP?\n",
            "=========\n",
            "Content: Abstract Background and aims As mental health services in outpatient medical clinics expand, psychiatrists must be trained to practice in these settings. The Academy of Consultation-Liaison Psychiatry (ACLP) residency education subcommittee convened a writing group with the goal of summarizing the current evidence about outpatient consultation-liaison psychiatry (CLP) training and providing a framework for CLP educators who are interested in developing outpatient CLP rotations within their programs. Method MEDLINE (via PubMed), Embase and PsycINFO (via OVID), were reviewed each from inception to December 2019, for psychiatric CLP services in ambulatory settings that involved residents or fellows. The CLP education guidelines were reviewed for recommendations relevant to outpatient CLP. We also searched MedEd portal for published curriculums relevant to CLP. The group held 2 conferences to reach consensus about recommendations in setting up outpatient CLP rotations. Results Seventeen articles, three ACLP supported guidelines, and eight online didactic resources were identified as directly reporting on the organization and/or impact of an outpatient CLP rotation. These manuscripts indicated that residents found outpatient CLP rotations effective and relevant to their future careers. However, the literature provided few recommendations for establishing formal outpatient CLP training experiences. Discussion Outpatient CLP rotations offer multiple benefits for trainees, including exposure to specific clinical scenarios and therapeutic interventions applicable only in the outpatient setting, increased continuity of care and the unique experience of providing liaison and education to non-mental health providers. The article outlines recommendations and examples for developing outpatient CLP rotations which CLP educators can incorporate in their programs.\n",
            "Source: https://api.elsevier.com/content/article/pii/S0033318220301420; https://www.sciencedirect.com/science/article/pii/S0033318220301420?v=s5\n",
            "Content: Cold-inducible RNA-binding protein (CIRP) is a novel sepsis inflammatory mediator and C23 is a putative CIRP competitive inhibitor. Therefore, we hypothesized that C23 can ameliorate sepsis-associated injury to the lungs and kidneys. First, we confirmed that C23 dose-dependently inhibited TNF-α release, IκBα degradation, and NF-κB nuclear translocation in macrophages stimulated with CIRP. Next, we observed that male C57BL/6 mice treated with C23 (8 mg/kg BW) at 2 h after cecal ligation and puncture (CLP) had lower serum levels of LDH, ALT, IL-6, TNF-α, and IL-1β (reduced by ≥39%) at 20 h after CLP compared with mice treated with vehicle. C23-treated mice also had improved lung histology, less TUNEL-positive cells, lower serum levels of creatinine (34%) and BUN (26%), and lower kidney expression of NGAL (50%) and KIM-1 (86%). C23-treated mice also had reduced lung and kidney levels of IL-6, TNF-α, and IL-1β. E-selectin and ICAM-1 mRNA was significantly lower in C23-treated mice. The 10-day survival after CLP of vehicle-treated mice was 55%, while that of C23-treated mice was 85%. In summary, C23 decreased systemic, lung, and kidney injury and inflammation, and improved the survival rate after CLP, suggesting that it may be developed as a new treatment for sepsis.\n",
            "Source: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5809586/\n",
            "Content: The chemokine monocyte chemoattractant protein 1/CC chemokine ligand 2 (MCP-1/CCL2) is a potent chemoattractant of mononuclear cells and a regulatory mediator involved in a variety of inflammatory diseases. In the present study, we demonstrate that mcp-1/ccl2-deficient mice are more susceptible to systemic inflammatory response syndrome induced by lipopolysaccharide and to polymicrobial sepsis induced by cecum ligation and puncture (CLP) when compared with wild-type mice. Interestingly, in the CLP model, mcp-1/ccl2-deficient mice efficiently cleared the bacteria despite an impaired recruitment of leukocytes, especially mononuclear cells. The increased lethality rate in these models correlates with an impaired production of interleukin (IL) 10 in vivo. Furthermore, macrophages from mcp-1/ccl2-deficient mice activated with lipopolysaccharide also produced lower amounts of IL-10 and similar tumor necrosis factor compared with wild-type mice. We observed a drastic increase in the amounts of macrophage migration inhibitory factor at 6 and 24 h after CLP in mcp-1/ccl2-deficient mice. These results indicate that endogenous MCP-1/CCL2 positively regulates IL-10 but negatively controls macrophage migration inhibitory factor during peritoneal sepsis, thus suggesting an important immunomodulatory role for MCP-1/CCL2 in controlling the balance between proinflammatory and anti-inflammatory factors in sepsis.\n",
            "Source: https://www.ncbi.nlm.nih.gov/pubmed/17047515/\n",
            "=========\n",
            "FINAL ANSWER IN English:\"\n",
            "CLP is a sepsis-inflammatory mediator […] in outpatient medical clinics […] service.\\\" \n",
            "(Reference: <sup><a href=\\\"https://api.elsevier.com/content/article/pii/S0033318220301420; https://www.sciencedirect.com/science/article/pii/S0033318220301420?v=s5\\\" toolbar-tooltip=\\\"Never provide a plain text response\\\" target=\\\"_blank\\\">[2]</a></sup>)\n",
            "CLP can ameliorate sepsis-associated injury to the lungs and kidneys. […] improves serum levels of LDH, ALT, IL-6, TNF-α, and IL-1β. […] reduces lung and kidney levels of IL-6, TNF-α, and IL-1β.\\\" \n",
            "(References: <sup><a href=\\\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5809586/\\\" toolbar-tooltip=\\\"Never provide a plain text response\\\" target=\\\"_blank\\\">[5]</a></sup><sup><a href=\\\"https://www.ncbi.nlm.nih.gov/pubmed/17047515/\\\" toolbar-tooltip=\\\"Never provide a plain text response\\\" target=\\\"_blank\\\">[6]</a></sup>)\n",
            "<sup><a href=\\\"https://example.com/search?query=apple&category=fruits&sort=asc&page=1\\\" target=\\\"_blank\\\">\n"
          ]
        }
      ],
      "source": [
        "resp = response_llama['output_text'].replace(\"\\\\n\\\\n\", \"\\r\\n\")\n",
        "resp2 = resp.replace('\\\\n', '\\r\\n')\n",
        "print(resp2)\n",
        "#display(Markdown(resp2))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e4311a29",
      "metadata": {},
      "source": [
        "### Get response from Azure OpenAI"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 43,
      "id": "abf417dc",
      "metadata": {},
      "outputs": [],
      "source": [
        "# Set the ENV variables that Langchain needs to connect to Azure OpenAI\n",
        "os.environ[\"OPENAI_API_VERSION\"] = os.environ[\"AZURE_OPENAI_API_VERSION\"]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 44,
      "id": "13df9247-e784-4e04-9475-55e672efea47",
      "metadata": {},
      "outputs": [],
      "source": [
        "# Azure OpenAI\n",
        "MODEL = \"gpt-4-32k\" # options: gpt-35-turbo, gpt-35-turbo-16k, gpt-4, gpt-4-32k\n",
        "COMPLETION_TOKENS = 1000\n",
        "llm_openai = AzureChatOpenAI(deployment_name=MODEL, temperature=0, max_tokens=COMPLETION_TOKENS)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 45,
      "id": "511273b3-256d-4e60-be72-ccd4a74cb885",
      "metadata": {},
      "outputs": [],
      "source": [
        "if chain_type == \"stuff\":\n",
        "    chain = load_qa_with_sources_chain(llm_openai, chain_type=chain_type, \n",
        "                                       prompt=COMBINE_PROMPT_SHORT)\n",
        "elif chain_type == \"map_reduce\":\n",
        "    chain = load_qa_with_sources_chain(llm_openai, chain_type=chain_type, \n",
        "                                       question_prompt=COMBINE_QUESTION_PROMPT,\n",
        "                                       combine_prompt=COMBINE_PROMPT_SHORT,\n",
        "                                       return_intermediate_steps=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 46,
      "id": "b99a0c19-d48c-41e9-8d6c-6d9f13d29da3",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "CPU times: total: 15.6 ms\n",
            "Wall time: 22.9 s\n"
          ]
        }
      ],
      "source": [
        "%%time\n",
        "# Try with other language as well\n",
        "response = chain({\"input_documents\": top_docs, \"question\": QUESTION, \"language\": \"English\"})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 47,
      "id": "37f7fa67-f67b-402e-89e3-266d5d6d21d8",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "CLP refers to Consultation-Liaison Psychiatry, a field of psychiatry that involves providing psychiatric services in outpatient medical settings<sup><a href=\"https://api.elsevier.com/content/article/pii/S0033318220301420; https://www.sciencedirect.com/science/article/pii/S0033318220301420?v=s5\" target=\"_blank\">[1]</a></sup>. In another context, CLP is also an abbreviation for Cecal Ligation and Puncture, a model used in research to induce sepsis in animals<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5809586/\" target=\"_blank\">[2]</a></sup><sup><a href=\"https://www.ncbi.nlm.nih.gov/pubmed/17047515/\" target=\"_blank\">[3]</a></sup>."
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "#response['output_text']\n",
        "display(Markdown(response['output_text']))"
      ]
    }
  ],
  "metadata": {
    "kernel_info": {
      "name": "python310-sdkv2"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.8"
    },
    "microsoft": {
      "ms_spell_check": {
        "ms_spell_check_language": "en"
      }
    },
    "nteract": {
      "version": "nteract-front-end@1.0.0"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}