04_Prompt_Engineering_with_ChatGPT/Prompt_Engg_Meetup_NoteBook.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prompt Engineering with OpenAI and LangChain"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is prompt engineering"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prompt engineering refers to the process of designing and refining prompts to achieve better results when using language models like GPT-3.5. It involves carefully crafting the input instructions or questions provided to the model to elicit the desired output. By understanding how to structure and frame prompts effectively, users can optimize the model's performance and obtain more accurate and useful responses.\n",
    "\n",
    "Prompt engineering techniques typically involve several strategies:\n",
    "\n",
    "- **Clear instructions**: Providing explicit and specific instructions in the prompt helps guide the model towards the desired output. For example, specifying the format of the answer or asking the model to think step by step before providing a final response.\n",
    "\n",
    "- **Context framing**: Setting the context for the prompt can help the model understand the desired domain or topic. Including relevant information or mentioning a specific scenario can improve the accuracy of the generated response.\n",
    "\n",
    "- **System messages**: Incorporating system-level instructions within the prompt can guide the model's behavior. For instance, instructing the model to consider the pros and cons before providing an opinion or asking it to generate a longer and more detailed response.\n",
    "\n",
    "- **Iterative refinement**: Adjusting and fine-tuning the prompts based on the model's initial output is an iterative process. By analyzing the generated responses, users can identify shortcomings or biases and refine the prompts to address those issues.\n",
    "\n",
    "Prompt engineering is an important aspect of using language models effectively. Through thoughtful design and refinement of prompts, users can enhance the quality, relevance, and usefulness of the generated responses."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Importing OpenAI and reading the OpenAI API key from the envirnoment variables\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "import openai\n",
    "import os\n",
    "\n",
    "from dotenv import load_dotenv, find_dotenv\n",
    "_ = load_dotenv(find_dotenv()) # read local .env file\n",
    "\n",
    "openai.api_key  = os.getenv('OPENAI_API_KEY')"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We create a helper function `get_completion()` which uses ChatGPT (gpt-3.5-turbo) model as default. The function takes user prompt as input and returns the response from the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "MODEL = \"gpt-3.5-turbo\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "def get_completion(prompt, model=MODEL):\n",
    "    \"\"\"Function to call the OpenAI API to get a completion of the prompt using ChatGPT model\n",
    "\n",
    "    Args:\n",
    "        prompt (str): user prompt to be completed\n",
    "        model (str, optional): _description_. Defaults to \"gpt-3.5-turbo\".\n",
    "\n",
    "    Returns:\n",
    "        _type_: response from the ChatGPT model\n",
    "    \"\"\"\n",
    "    messages = [{\"role\": \"user\", \"content\": prompt}]\n",
    "    response = openai.ChatCompletion.create(\n",
    "        model=model,\n",
    "        messages=messages,\n",
    "        temperature=0, # this is the degree of randomness of the model's output\n",
    "    )\n",
    "    return response.choices[0].message[\"content\"]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We also create another helper function `create_notebook()`, which will be used later to create a new notebook with the response from the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "def create_notebook(result, filename):\n",
    "    \n",
    "    allowed_chars = [' ', '{', '\\n']\n",
    "    \n",
    "    directory = \"generated_notebooks\"\n",
    "    if not filename.endswith('.ipynb'):\n",
    "        filename = filename + '.ipynb'\n",
    "    filepath = os.path.join(os.getcwd(), directory, filename)\n",
    "    \n",
    "    if len(result.split('```')) > 1:\n",
    "        data = result.split('```')[1]\n",
    "        if data[0] not in allowed_chars:\n",
    "            data = data[4:]\n",
    "    elif len(result.split('```')) == 1:\n",
    "        data = result.split('```')[0]\n",
    "        \n",
    "    print('Creating notebook: '+ os.path.join(directory, filename))\n",
    "    # data = result.split('```')[1]\n",
    "    with open(filepath, 'w') as f:\n",
    "         f.write(data)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We will now read a jupyter notebook as a json. The Notebook lists some tasks to be completed.\n",
    "We will ask the ChatGPT to read the jupyter notebook and complete the tasks"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The Input Notebook"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/task_nb.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'cells': [{'cell_type': 'markdown',\n",
      "            'metadata': {},\n",
      "            'source': ['## Introduction']},\n",
      "           {'cell_type': 'markdown',\n",
      "            'metadata': {},\n",
      "            'source': ['### Your task is to\\n',\n",
      "                       ' - Read the csv file into a pandas dataframe.\\n',\n",
      "                       ' - Find the list of unique cities where matches were '\n",
      "                       'played\\n',\n",
      "                       ' - Find the columns which contains null values if any '\n",
      "                       '?\\n',\n",
      "                       ' - List down top 5 most played venues\\n',\n",
      "                       ' - Get top 5 goal scorers of the tournament\\n']}],\n",
      " 'metadata': {'kernelspec': {'display_name': 'Python 3 (ipykernel)',\n",
      "                             'language': 'python',\n",
      "                             'name': 'python3'},\n",
      "              'language_info': {'codemirror_mode': {'name': 'ipython',\n",
      "                                                    'version': 3},\n",
      "                                'file_extension': '.py',\n",
      "                                'mimetype': 'text/x-python',\n",
      "                                'name': 'python',\n",
      "                                'nbconvert_exporter': 'python',\n",
      "                                'pygments_lexer': 'ipython3',\n",
      "                                'version': '3.11.3'}},\n",
      " 'nbformat': 4,\n",
      " 'nbformat_minor': 2}\n"
     ]
    }
   ],
   "source": [
    "from pprint import pprint\n",
    "import json\n",
    "\n",
    "with open(\"test_file.ipynb\", \"r\") as f:\n",
    "    data = json.load(f)\n",
    "\n",
    "pprint(data)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Prompting with OpenAI"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Now we create a prompt which has a task to be followed. It also tells the model that what the input questions it has to work on"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Prompt #1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "## Solution\n",
      "\n",
      "First, we need to import the necessary libraries.\n",
      "\n",
      "```python\n",
      "import pandas as pd\n",
      "```\n",
      "\n",
      "### Reading the CSV file into a pandas dataframe\n",
      "\n",
      "We can use the `read_csv()` function from pandas to read the CSV file into a dataframe.\n",
      "\n",
      "```python\n",
      "df = pd.read_csv('matches.csv')\n",
      "```\n",
      "\n",
      "### Finding the list of unique cities where matches were played\n",
      "\n",
      "We can use the `unique()` function to get the unique values in a column. In this case, we want to get the unique cities where matches were played. The city column in the dataframe is 'city'.\n",
      "\n",
      "```python\n",
      "unique_cities = df['city'].unique()\n",
      "print(unique_cities)\n",
      "```\n",
      "\n",
      "### Finding the columns which contains null values if any\n",
      "\n",
      "We can use the `isnull()` function to check for null values in the dataframe. We can then use the `any()` function to check if any of the columns contain null values.\n",
      "\n",
      "```python\n",
      "null_columns = df.columns[df.isnull().any()]\n",
      "print(null_columns)\n",
      "```\n",
      "\n",
      "### Listing down top 5 most played venues\n",
      "\n",
      "We can use the `value_counts()` function to get the count of each unique value in a column. In this case, we want to get the count of each venue where matches were played. The venue column in the dataframe is 'venue'. We can then use the `head()` function to get the top 5 most played venues.\n",
      "\n",
      "```python\n",
      "top_venues = df['venue'].value_counts().head(5)\n",
      "print(top_venues)\n",
      "```\n",
      "\n",
      "### Getting top 5 goal scorers of the tournament\n",
      "\n",
      "We need to first create a new dataframe with the player names and their corresponding goals. We can use the `groupby()` function to group the dataframe by player name and then use the `sum()` function to get the total goals scored by each player.\n",
      "\n",
      "```python\n",
      "player_goals = df.groupby('player_of_match')['player_of_match'].count().sort_values(ascending=False).head(5)\n",
      "print(player_goals)\n",
      "```\n",
      "\n",
      "### Final code\n",
      "\n",
      "```python\n",
      "import pandas as pd\n",
      "\n",
      "# Reading the CSV file into a pandas dataframe\n",
      "df = pd.read_csv('matches.csv')\n",
      "\n",
      "# Finding the list of unique cities where matches were played\n",
      "unique_cities = df['city'].unique()\n",
      "print(unique_cities)\n",
      "\n",
      "# Finding the columns which contains null values if any\n",
      "null_columns = df.columns[df.isnull().any()]\n",
      "print(null_columns)\n",
      "\n",
      "# Listing down top 5 most played venues\n",
      "top_venues = df['venue'].value_counts().head(5)\n",
      "print(top_venues)\n",
      "\n",
      "# Getting top 5 goal scorers of the tournament\n",
      "player_goals = df.groupby('player_of_match')['player_of_match'].count().sort_values(ascending=False).head(5)\n",
      "print(player_goals)\n",
      "```\n"
     ]
    }
   ],
   "source": [
    "prompt = f\"\"\"\n",
    "You are a Data Scientist. Your task is listed in the\n",
    "json input which will be followed by ```. The questions \n",
    "which are to be answered are under \"cell_type\":\"markdown\". \n",
    "Your output should be a step by step jupyter notebook with \n",
    "comments for the code in python.\n",
    "\n",
    "The input: ```{data}```\n",
    "\"\"\"\n",
    "\n",
    "response = get_completion(prompt)\n",
    "print(response)\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We see that the output is just a python code which is not in the correct format in which we asked for, i.e. the jupyter notebook format"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Prompt #2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      " \"cells\": [\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"## Introduction\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"### Your task is to\\n\",\n",
      "    \" - Read the csv file into a pandas dataframe.\\n\",\n",
      "    \" - Find the list of unique cities where matches were played\\n\",\n",
      "    \" - Find the columns which contains null values if any ?\\n\",\n",
      "    \" - List down top 5 most played venues\\n\",\n",
      "    \" - Get top 5 goal scorers of the tournament\\n\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"code\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"#importing necessary libraries\\n\",\n",
      "    \"import pandas as pd\\n\",\n",
      "    \"\\n\",\n",
      "    \"#reading the csv file into a pandas dataframe\\n\",\n",
      "    \"df = pd.read_csv('matches.csv')\\n\",\n",
      "    \"\\n\",\n",
      "    \"#finding the list of unique cities where matches were played\\n\",\n",
      "    \"unique_cities = df['city'].unique()\\n\",\n",
      "    \"print('List of unique cities where matches were played:')\\n\",\n",
      "    \"print(unique_cities)\\n\",\n",
      "    \"\\n\",\n",
      "    \"#finding the columns which contains null values if any\\n\",\n",
      "    \"null_columns = df.columns[df.isnull().any()]\\n\",\n",
      "    \"print('Columns which contains null values:')\\n\",\n",
      "    \"print(null_columns)\\n\",\n",
      "    \"\\n\",\n",
      "    \"#listing down top 5 most played venues\\n\",\n",
      "    \"top_venues = df['venue'].value_counts().head()\\n\",\n",
      "    \"print('Top 5 most played venues:')\\n\",\n",
      "    \"print(top_venues)\\n\",\n",
      "    \"\\n\",\n",
      "    \"#getting top 5 goal scorers of the tournament\\n\",\n",
      "    \"top_scorers = df.groupby('player_of_match')['player_of_match'].count().nlargest(5)\\n\",\n",
      "    \"print('Top 5 goal scorers of the tournament:')\\n\",\n",
      "    \"print(top_scorers)\"\n",
      "   ]\n",
      "  }\n",
      " ],\n",
      " \"metadata\": {\n",
      "  \"kernelspec\": {\n",
      "   \"display_name\": \"Python 3 (ipykernel)\",\n",
      "   \"language\": \"python\",\n",
      "   \"name\": \"python3\"\n",
      "  },\n",
      "  \"language_info\": {\n",
      "   \"codemirror_mode\": {\n",
      "    \"name\": \"ipython\",\n",
      "    \"version\": 3\n",
      "   },\n",
      "   \"file_extension\": \".py\",\n",
      "   \"mimetype\": \"text/x-python\",\n",
      "   \"name\": \"python\",\n",
      "   \"nbconvert_exporter\": \"python\",\n",
      "   \"pygments_lexer\": \"ipython3\",\n",
      "   \"version\": \"3.11.3\"\n",
      "  }\n",
      " },\n",
      " \"nbformat\": 4,\n",
      " \"nbformat_minor\": 2\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "prompt = f\"\"\"\n",
    "You are a Data Scientist. Your task is listed in the\n",
    "json input which will be followed by ```. The questions \n",
    "which are to be answered are under \"cell_type\":\"markdown\". \n",
    "\n",
    "Your output should be a step by step jupyter notebook in \"json\" format.\n",
    "Add comments for the python output in \"cell_type\":\"markdown\" of the jupyter notebook.\n",
    "    \n",
    "Only and only output python code in json format of jupyter notebook.\n",
    "\n",
    "The input: ```{data}```\n",
    "\"\"\"\n",
    "response = get_completion(prompt)\n",
    "print(response)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    " Prompt #1         |  Prompt #2\n",
    ":-------------------------:|:-------------------------:\n",
    "![](images/prompt1.png)    |  ![](images/prompt2.png)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Nicely done this time. We get the output in the way we wanted. \n",
    "Now we can save this response using our helper function `create_notebook()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating notebook: generated_notebooks/prompt_with_openai.ipynb\n"
     ]
    }
   ],
   "source": [
    "create_notebook(result=response, filename=\"prompt_with_openai\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Implementing with LangChain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "from langchain.chat_models import ChatOpenAI\n",
    "\n",
    "OPENAI_API_KEY  = os.getenv('OPENAI_API_KEY')\n",
    "\n",
    "chat = ChatOpenAI(\n",
    "    openai_api_key=OPENAI_API_KEY,\n",
    "    temperature=0,\n",
    "    model_name='gpt-3.5-turbo'\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "from langchain.prompts.chat import (\n",
    "    ChatPromptTemplate,\n",
    "    SystemMessagePromptTemplate,\n",
    "    AIMessagePromptTemplate,\n",
    "    HumanMessagePromptTemplate,\n",
    ")\n",
    "\n",
    "from langchain.schema import (\n",
    "    HumanMessage,\n",
    "    AIMessage,   ## not used in this notebook\n",
    "    SystemMessage ## not used in this notebook\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "LangChain gives us flexibility to use different prompt templates. \n",
    "These templates can be used for Few-Shot learning as well. That means we can provide some examples to the model before it actually answers user questions. In this notebook we are going to use One-Shot Learning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "## This is a json format of a jupyter notebook\n",
    "## It is used as an example for One-Shot learning to the Model\n",
    "## This helps us to guide the model to the path we actually want it to follow\n",
    "\n",
    "response_example = \"\"\"{\n",
    " \"cells\": [\n",
    "  {\n",
    "   \"attachments\": {},\n",
    "   \"cell_type\": \"markdown\",\n",
    "   \"metadata\": {},\n",
    "   \"source\": [\n",
    "    \"## Reading csv with Pandas\"\n",
    "   ]\n",
    "  },\n",
    "  {\n",
    "   \"cell_type\": \"code\",\n",
    "   \"execution_count\": null,\n",
    "   \"metadata\": {},\n",
    "   \"outputs\": [],\n",
    "   \"source\": [\n",
    "    \"import pandas as pd\\n\",\n",
    "    \"\\n\",\n",
    "    \"# Reading the csv file into a pandas dataframe\\n\",\n",
    "    \"matches_df = pd.read_csv('matches.csv')\"\n",
    "   ]\n",
    "  }\n",
    " ]\n",
    "}\n",
    "\"\"\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "- **SystemMessagePromptTemplate** - It provides a default message to the model, that what it's actual task is\n",
    "- **HumanMessagePromptTemplate** - It gives the model a prompt template from the users it can expect.\n",
    "- **AIMessagePromptTemplate** - It tells the model what kind of response it should give to the user's question.\n",
    "- **ChatPromptTemplate** - It combines all the Prompt Templates together. This returns a PromptValue, which we can convert to a string or Message object, depending on whether we want to use the formatted value as input to an llm or chat model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[SystemMessage(lc_kwargs={'content': 'You are a helpful assistant. You solve python problems and provide solutions in a jupyter notebook ipynb format when asked for.', 'additional_kwargs': {}}, content='You are a helpful assistant. You solve python problems and provide solutions in a jupyter notebook ipynb format when asked for.', additional_kwargs={}),\n",
       " HumanMessage(lc_kwargs={'content': 'Hi AI, How to read a csv file with pandas using jupyter notebook json format with comments', 'additional_kwargs': {}}, content='Hi AI, How to read a csv file with pandas using jupyter notebook json format with comments', additional_kwargs={}, example=False),\n",
       " AIMessage(lc_kwargs={'content': '{\\n \"cells\": [\\n  {\\n   \"attachments\": {},\\n   \"cell_type\": \"markdown\",\\n   \"metadata\": {},\\n   \"source\": [\\n    \"## Reading csv with Pandas\"\\n   ]\\n  },\\n  {\\n   \"cell_type\": \"code\",\\n   \"execution_count\": null,\\n   \"metadata\": {},\\n   \"outputs\": [],\\n   \"source\": [\\n    \"import pandas as pd\\n\",\\n    \"\\n\",\\n    \"# Reading the csv file into a pandas dataframe\\n\",\\n    \"matches_df = pd.read_csv(\\'matches.csv\\')\"\\n   ]\\n  }\\n ]\\n}\\n', 'additional_kwargs': {}}, content='{\\n \"cells\": [\\n  {\\n   \"attachments\": {},\\n   \"cell_type\": \"markdown\",\\n   \"metadata\": {},\\n   \"source\": [\\n    \"## Reading csv with Pandas\"\\n   ]\\n  },\\n  {\\n   \"cell_type\": \"code\",\\n   \"execution_count\": null,\\n   \"metadata\": {},\\n   \"outputs\": [],\\n   \"source\": [\\n    \"import pandas as pd\\n\",\\n    \"\\n\",\\n    \"# Reading the csv file into a pandas dataframe\\n\",\\n    \"matches_df = pd.read_csv(\\'matches.csv\\')\"\\n   ]\\n  }\\n ]\\n}\\n', additional_kwargs={}, example=False)]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## Defining a system message for the model. \n",
    "## The model should be able to understand that what it has to do is to solve a problem\n",
    "## and provide a solution.\n",
    "system_template = SystemMessagePromptTemplate.from_template(\n",
    "    'You are a helpful assistant. You solve python problems and provide solutions '\n",
    "    'in a jupyter notebook ipynb format when asked for.'\n",
    ")\n",
    "\n",
    "## Defining a human message and AI message for the model. \n",
    "## HumanMessage is the example user message for one-shot learning\n",
    "## AIMessage is the example AI's response for one-shot learning\n",
    "human_template = HumanMessagePromptTemplate.from_template(\"{input}\")\n",
    "ai_template = AIMessagePromptTemplate.from_template(\"{response}\")\n",
    "\n",
    "# create the list of messages\n",
    "chat_prompt = ChatPromptTemplate.from_messages([\n",
    "    system_template,\n",
    "    human_template,\n",
    "    ai_template\n",
    "])\n",
    "\n",
    "# format with required inputs\n",
    "chat_prompt_value = chat_prompt.format_prompt(\n",
    "    input=\"Hi AI, How to read a csv file with pandas using jupyter notebook json format with comments\",\n",
    "    response=response_example\n",
    ")\n",
    "\n",
    "## we can use the promptValue object and use it as a string or a message, depending on the use case.\n",
    "chat_prompt_value.to_messages()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We create a list of messages to keep track of the conversation between the user and the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "messages = chat_prompt_value.to_messages()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We will again use the same example of the Jupyter Notebook (`test_file.ipynb`) with the listed task"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Prompt #1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "## We create an input user prompt for the model.\n",
    "\n",
    "input_prompt = f\"\"\"\n",
    "You are a Data Scientist. Your tasks are listed in the\n",
    "json input which will be followed by ```. The questions \n",
    "which are to be answered are under \"cell_type\":\"markdown\". \n",
    "\n",
    "The output should be under \"cell_type\":\"code\" and the comments \n",
    "of the code should be under \"cell_type\":\"markdown\" of the jupyter notebook.\n",
    "\n",
    "```{data}```\n",
    "\"\"\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We first need to append the input to the list of messages for the Model. We use `HumanMessage` because it is the input prompt given by the user"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "messages.append(\n",
    "    HumanMessage(content=input_prompt)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      " \"cells\": [\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"## Reading csv with Pandas\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"code\",\n",
      "   \"execution_count\": null,\n",
      "   \"metadata\": {},\n",
      "   \"outputs\": [],\n",
      "   \"source\": [\n",
      "    \"# Importing necessary libraries\\n\",\n",
      "    \"import pandas as pd\\n\",\n",
      "    \"\\n\",\n",
      "    \"# Reading the csv file into a pandas dataframe\\n\",\n",
      "    \"matches_df = pd.read_csv('matches.csv')\\n\",\n",
      "    \"\\n\",\n",
      "    \"# Finding the list of unique cities where matches were played\\n\",\n",
      "    \"unique_cities = matches_df['city'].unique()\\n\",\n",
      "    \"print('List of unique cities where matches were played:')\\n\",\n",
      "    \"print(unique_cities)\\n\",\n",
      "    \"\\n\",\n",
      "    \"# Finding the columns which contains null values if any\\n\",\n",
      "    \"null_columns = matches_df.columns[matches_df.isnull().any()]\\n\",\n",
      "    \"print('Columns containing null values:')\\n\",\n",
      "    \"print(null_columns)\\n\",\n",
      "    \"\\n\",\n",
      "    \"# Listing down top 5 most played venues\\n\",\n",
      "    \"top_venues = matches_df['venue'].value_counts().head()\\n\",\n",
      "    \"print('Top 5 most played venues:')\\n\",\n",
      "    \"print(top_venues)\\n\",\n",
      "    \"\\n\",\n",
      "    \"# Getting top 5 goal scorers of the tournament\\n\",\n",
      "    \"player_scores = pd.DataFrame(matches_df, columns=['player_of_match', 'win_by_runs', 'win_by_wickets'])\\n\",\n",
      "    \"player_scores = player_scores.groupby('player_of_match').sum()\\n\",\n",
      "    \"player_scores = player_scores.sort_values(by=['win_by_runs', 'win_by_wickets'], ascending=False).head()\\n\",\n",
      "    \"print('Top 5 goal scorers of the tournament:')\\n\",\n",
      "    \"print(player_scores)\"\n",
      "   ]\n",
      "  }\n",
      " ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "res = chat(messages)\n",
    "\n",
    "print(res.content)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We do not get the desired output format from the model, however the models response in solving the tasks are still correct.\n",
    "\n",
    "Now we will continue to do the prompts iteratively, but this time in a different way in LangChain"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Prompt #2"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We do not have to write the whole input prompt again as we are tracking the conversation between the user and the model. So we can only write new message to the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "input_prompt=\"\"\"No the output should be in jupyter notebook ipynb json format \\\n",
    "        and also add comments in the markdown cells of the jupyter notebook\"\"\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We append the output response from the model to the messages and then again append the user input as HumanMessage"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Here is the solution in Jupyter Notebook ipynb format with comments:\n",
      "\n",
      "```\n",
      "{\n",
      " \"cells\": [\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"## Reading csv with Pandas\\n\",\n",
      "    \"\\n\",\n",
      "    \"In this notebook, we will read a csv file using Pandas and perform some basic data analysis on it.\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"code\",\n",
      "   \"execution_count\": null,\n",
      "   \"metadata\": {},\n",
      "   \"outputs\": [],\n",
      "   \"source\": [\n",
      "    \"# Importing necessary libraries\\n\",\n",
      "    \"import pandas as pd\\n\",\n",
      "    \"\\n\",\n",
      "    \"# Reading the csv file into a pandas dataframe\\n\",\n",
      "    \"matches_df = pd.read_csv('matches.csv')\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"### Finding the list of unique cities where matches were played\\n\",\n",
      "    \"\\n\",\n",
      "    \"Let's find the list of unique cities where matches were played in the dataset.\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"code\",\n",
      "   \"execution_count\": null,\n",
      "   \"metadata\": {},\n",
      "   \"outputs\": [\n",
      "    {\n",
      "     \"name\": \"stdout\",\n",
      "     \"output_type\": \"stream\",\n",
      "     \"text\": [\n",
      "      \"List of unique cities where matches were played:\\n\",\n",
      "      \"['Hyderabad' 'Pune' 'Rajkot' 'Indore' 'Bangalore' 'Mumbai' 'Kolkata'\\n\",\n",
      "      \" 'Delhi' 'Chandigarh' 'Kanpur' 'Jaipur' 'Chennai' 'Cape Town'\\n\",\n",
      "      \" 'Port Elizabeth' 'Durban' 'Centurion' 'East London' 'Johannesburg'\\n\",\n",
      "      \" 'Kimberley' 'Bloemfontein' 'Ahmedabad' 'Cuttack' 'Nagpur' 'Dharamsala'\\n\",\n",
      "      \" 'Kochi' 'Visakhapatnam' 'Raipur' 'Ranchi' 'Abu Dhabi' 'Sharjah'\\n\",\n",
      "      \" 'Dubai' 'Rising Pune Supergiants' 'Kanpur Nagar']\\n\"\n",
      "     ]\n",
      "    }\n",
      "   ],\n",
      "   \"source\": [\n",
      "    \"unique_cities = matches_df['city'].unique()\\n\",\n",
      "    \"print('List of unique cities where matches were played:')\\n\",\n",
      "    \"print(unique_cities)\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"### Finding the columns which contains null values if any\\n\",\n",
      "    \"\\n\",\n",
      "    \"Let's find the columns which contains null values if any in the dataset.\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"code\",\n",
      "   \"execution_count\": null,\n",
      "   \"metadata\": {},\n",
      "   \"outputs\": [\n",
      "    {\n",
      "     \"name\": \"stdout\",\n",
      "     \"output_type\": \"stream\",\n",
      "     \"text\": [\n",
      "      \"Columns containing null values:\\n\",\n",
      "      \"Index(['winner', 'player_of_match', 'umpire1', 'umpire2', 'umpire3'], dtype='object')\\n\"\n",
      "     ]\n",
      "    }\n",
      "   ],\n",
      "   \"source\": [\n",
      "    \"null_columns = matches_df.columns[matches_df.isnull().any()]\\n\",\n",
      "    \"print('Columns containing null values:')\\n\",\n",
      "    \"print(null_columns)\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"### Listing down top 5 most played venues\\n\",\n",
      "    \"\\n\",\n",
      "    \"Let's list down the top 5 most played venues in the dataset.\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"code\",\n",
      "   \"execution_count\": null,\n",
      "   \"metadata\": {},\n",
      "   \"outputs\": [\n",
      "    {\n",
      "     \"name\": \"stdout\",\n",
      "     \"output_type\": \"stream\",\n",
      "     \"text\": [\n",
      "      \"Top 5 most played venues:\\n\",\n",
      "      \"Eden Gardens                                  77\\n\",\n",
      "      \"Wankhede Stadium                              73\\n\",\n",
      "      \"M Chinnaswamy Stadium                         73\\n\",\n",
      "      \"Feroz Shah Kotla                              67\\n\",\n",
      "      \"Rajiv Gandhi International Stadium, Uppal     56\\n\",\n",
      "      \"Name: venue, dtype: int64\\n\"\n",
      "     ]\n",
      "    }\n",
      "   ],\n",
      "   \"source\": [\n",
      "    \"top_venues = matches_df['venue'].value_counts().head()\\n\",\n",
      "    \"print('Top 5 most played venues:')\\n\",\n",
      "    \"print(top_venues)\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"### Getting top 5 goal scorers of the tournament\\n\",\n",
      "    \"\\n\",\n",
      "    \"Let's get the top 5 goal scorers of the tournament from the dataset.\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"code\",\n",
      "   \"execution_count\": null,\n",
      "   \"metadata\": {},\n",
      "   \"outputs\": [\n",
      "    {\n",
      "     \"name\": \"stdout\",\n",
      "     \"output_type\": \"stream\",\n",
      "     \"text\": [\n",
      "      \"Top 5 goal scorers of the tournament:\\n\",\n",
      "      \"                 win_by_runs  win_by_wickets\\n\",\n",
      "      \"player_of_match                              \\n\",\n",
      "      \"CH Gayle                  0               1\\n\",\n",
      "      \"MEK Hussey                0               1\\n\",\n",
      "      \"SR Watson                 0               1\\n\",\n",
      "      \"SR Tendulkar              0               1\\n\",\n",
      "      \"YK Pathan                 0               1\\n\"\n",
      "     ]\n",
      "    }\n",
      "   ],\n",
      "   \"source\": [\n",
      "    \"player_scores = pd.DataFrame(matches_df, columns=['player_of_match', 'win_by_runs', 'win_by_wickets'])\\n\",\n",
      "    \"player_scores = player_scores.groupby('player_of_match').sum()\\n\",\n",
      "    \"player_scores = player_scores.sort_values(by=['win_by_runs', 'win_by_wickets'], ascending=False).head()\\n\",\n",
      "    \"print('Top 5 goal scorers of the tournament:')\\n\",\n",
      "    \"print(player_scores)\"\n",
      "   ]\n",
      "  }\n",
      " ],\n",
      " \"metadata\": {\n",
      "  \"kernelspec\": {\n",
      "   \"display_name\": \"Python 3 (ipykernel)\",\n",
      "   \"language\": \"python\",\n",
      "   \"name\": \"python3\"\n",
      "  },\n",
      "  \"language_info\": {\n",
      "   \"codemirror_mode\": {\n",
      "    \"name\": \"ipython\",\n",
      "    \"version\": 3\n",
      "   },\n",
      "   \"file_extension\": \".py\",\n",
      "   \"mimetype\": \"text/x-python\",\n",
      "   \"name\": \"python\",\n",
      "   \"nbconvert_exporter\": \"python\",\n",
      "   \"pygments_lexer\": \"ipython3\",\n",
      "   \"version\": \"3.11.3\"\n",
      "  },\n",
      "  \"colab\": {\n",
      "   \"name\": \"Python 3\",\n",
      "   \"provenance\": [],\n",
      "   \"collapsed_sections\": [],\n",
      "   \"toc_visible\": true,\n",
      "   \"include_colab_link\": true\n",
      "  }\n",
      " },\n",
      " \"nbformat\": 4,\n",
      " \"nbformat_minor\": 0\n",
      "}\n",
      "```\n"
     ]
    }
   ],
   "source": [
    "\n",
    "messages.append(res)\n",
    "\n",
    "# make new query\n",
    "messages.append(\n",
    "    HumanMessage(content=input_prompt)\n",
    ")\n",
    "\n",
    "res = chat(messages)\n",
    "\n",
    "print(res.content)\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Great, that was much better in getting the desired output. Let's save the output so we can easily visualize the notebook"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Saving the output to a New Notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating notebook: generated_notebooks/prompt_with_langchain.ipynb\n"
     ]
    }
   ],
   "source": [
    "create_notebook(result=res.content, filename=\"prompt_with_langchain\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Generating Task Notebooks with LangChain"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Now lets use LangChain to generate tasks instead of solutions.\n",
    "\n",
    "In this case, we will give the model a topic on which it should create a excersie notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ChatPromptValue(lc_kwargs={'messages': [SystemMessage(lc_kwargs={'content': 'You are a helpful assistant. You create jupyter notebooks on specific tasks     asked by the user. The tasks you create should be solvable in python     You will always create a jupyter notebook in json format', 'additional_kwargs': {}}, content='You are a helpful assistant. You create jupyter notebooks on specific tasks     asked by the user. The tasks you create should be solvable in python     You will always create a jupyter notebook in json format', additional_kwargs={}), HumanMessage(lc_kwargs={'content': 'Hi AI, How to read a csv file with pandas. Use jupyter notebook json format with comments', 'additional_kwargs': {}}, content='Hi AI, How to read a csv file with pandas. Use jupyter notebook json format with comments', additional_kwargs={}, example=False), AIMessage(lc_kwargs={'content': '{\\n \"cells\": [\\n  {\\n   \"attachments\": {},\\n   \"cell_type\": \"markdown\",\\n   \"metadata\": {},\\n   \"source\": [\\n    \"## Reading csv with Pandas\"\\n   ]\\n  },\\n  {\\n   \"cell_type\": \"code\",\\n   \"execution_count\": null,\\n   \"metadata\": {},\\n   \"outputs\": [],\\n   \"source\": [\\n    \"import pandas as pd\\n\",\\n    \"\\n\",\\n    \"# Reading the csv file into a pandas dataframe\\n\",\\n    \"matches_df = pd.read_csv(\\'matches.csv\\')\"\\n   ]\\n  }\\n ]\\n}\\n', 'additional_kwargs': {}}, content='{\\n \"cells\": [\\n  {\\n   \"attachments\": {},\\n   \"cell_type\": \"markdown\",\\n   \"metadata\": {},\\n   \"source\": [\\n    \"## Reading csv with Pandas\"\\n   ]\\n  },\\n  {\\n   \"cell_type\": \"code\",\\n   \"execution_count\": null,\\n   \"metadata\": {},\\n   \"outputs\": [],\\n   \"source\": [\\n    \"import pandas as pd\\n\",\\n    \"\\n\",\\n    \"# Reading the csv file into a pandas dataframe\\n\",\\n    \"matches_df = pd.read_csv(\\'matches.csv\\')\"\\n   ]\\n  }\\n ]\\n}\\n', additional_kwargs={}, example=False)]}, messages=[SystemMessage(lc_kwargs={'content': 'You are a helpful assistant. You create jupyter notebooks on specific tasks     asked by the user. The tasks you create should be solvable in python     You will always create a jupyter notebook in json format', 'additional_kwargs': {}}, content='You are a helpful assistant. You create jupyter notebooks on specific tasks     asked by the user. The tasks you create should be solvable in python     You will always create a jupyter notebook in json format', additional_kwargs={}), HumanMessage(lc_kwargs={'content': 'Hi AI, How to read a csv file with pandas. Use jupyter notebook json format with comments', 'additional_kwargs': {}}, content='Hi AI, How to read a csv file with pandas. Use jupyter notebook json format with comments', additional_kwargs={}, example=False), AIMessage(lc_kwargs={'content': '{\\n \"cells\": [\\n  {\\n   \"attachments\": {},\\n   \"cell_type\": \"markdown\",\\n   \"metadata\": {},\\n   \"source\": [\\n    \"## Reading csv with Pandas\"\\n   ]\\n  },\\n  {\\n   \"cell_type\": \"code\",\\n   \"execution_count\": null,\\n   \"metadata\": {},\\n   \"outputs\": [],\\n   \"source\": [\\n    \"import pandas as pd\\n\",\\n    \"\\n\",\\n    \"# Reading the csv file into a pandas dataframe\\n\",\\n    \"matches_df = pd.read_csv(\\'matches.csv\\')\"\\n   ]\\n  }\\n ]\\n}\\n', 'additional_kwargs': {}}, content='{\\n \"cells\": [\\n  {\\n   \"attachments\": {},\\n   \"cell_type\": \"markdown\",\\n   \"metadata\": {},\\n   \"source\": [\\n    \"## Reading csv with Pandas\"\\n   ]\\n  },\\n  {\\n   \"cell_type\": \"code\",\\n   \"execution_count\": null,\\n   \"metadata\": {},\\n   \"outputs\": [],\\n   \"source\": [\\n    \"import pandas as pd\\n\",\\n    \"\\n\",\\n    \"# Reading the csv file into a pandas dataframe\\n\",\\n    \"matches_df = pd.read_csv(\\'matches.csv\\')\"\\n   ]\\n  }\\n ]\\n}\\n', additional_kwargs={}, example=False)])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "system_template = SystemMessagePromptTemplate.from_template(\n",
    "    'You are a helpful assistant. You create jupyter notebooks on specific tasks \\\n",
    "    asked by the user. The tasks you create should be solvable in python \\\n",
    "    You will always create a jupyter notebook in json format'\n",
    ")\n",
    "\n",
    "human_template = HumanMessagePromptTemplate.from_template(\"{input}\")\n",
    "ai_template = AIMessagePromptTemplate.from_template(\"{response}\")\n",
    "\n",
    "# create the list of messages\n",
    "chat_prompt = ChatPromptTemplate.from_messages([\n",
    "    system_template,\n",
    "    human_template,\n",
    "    ai_template\n",
    "])\n",
    "\n",
    "# format with required inputs\n",
    "chat_prompt_value = chat_prompt.format_prompt(\n",
    "    input=\"Hi AI, How to read a csv file with pandas. Use jupyter notebook json format with comments\",\n",
    "    response=response_example\n",
    ")\n",
    "chat_prompt_value"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Creating Notebook for Linear Regression task"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### 1st prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      " \"cells\": [\n",
      "  {\n",
      "   \"attachments\": {},\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"## Applying Linear Regression on a Dataset\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"### Step 1: Importing Required Libraries\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"code\",\n",
      "   \"execution_count\": null,\n",
      "   \"metadata\": {},\n",
      "   \"outputs\": [],\n",
      "   \"source\": [\n",
      "    \"# Importing Required Libraries\n",
      "\",\n",
      "    \"import pandas as pd\n",
      "\",\n",
      "    \"import numpy as np\n",
      "\",\n",
      "    \"import matplotlib.pyplot as plt\n",
      "\",\n",
      "    \"from sklearn.linear_model import LinearRegression\n",
      "\",\n",
      "    \"from sklearn.model_selection import train_test_split\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"### Step 2: Loading the Dataset\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"code\",\n",
      "   \"execution_count\": null,\n",
      "   \"metadata\": {},\n",
      "   \"outputs\": [],\n",
      "   \"source\": [\n",
      "    \"# Loading the Dataset\n",
      "\",\n",
      "    \"dataset = pd.read_csv('dataset.csv')\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"### Step 3: Exploring the Dataset\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"code\",\n",
      "   \"execution_count\": null,\n",
      "   \"metadata\": {},\n",
      "   \"outputs\": [],\n",
      "   \"source\": [\n",
      "    \"# Exploring the Dataset\n",
      "\",\n",
      "    \"dataset.head()\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"### Step 4: Preprocessing the Dataset\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"#### 4.1: Handling Missing Values\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"code\",\n",
      "   \"execution_count\": null,\n",
      "   \"metadata\": {},\n",
      "   \"outputs\": [],\n",
      "   \"source\": [\n",
      "    \"# Handling Missing Values\n",
      "\",\n",
      "    \"dataset.isnull().sum()\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"#### 4.2: Encoding Categorical Variables\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"#### 4.3: Splitting the Dataset into Dependent and Independent Variables\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"### Step 5: Splitting the Dataset into Training and Testing Sets\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"### Step 6: Training the Linear Regression Model\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"### Step 7: Making Predictions on the Test Set\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"### Step 8: Evaluating the Model Performance\"\n",
      "   ]\n",
      "  }\n",
      " ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "## keeping track of the conversation by appending the Human Message and AI response\n",
    "messages = chat_prompt_value.to_messages()\n",
    "\n",
    "## prompt 1\n",
    "prompt = \"Create a detailed step by step tasks in a jupyter notebook for applying linear regression on a dataset.\" \n",
    "\n",
    "messages.append(\n",
    "    HumanMessage(content=prompt)\n",
    ")\n",
    "\n",
    "res = chat(messages)\n",
    "\n",
    "print(res.content)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "On the first prompt we see that we got the response in the desired format but the response also has solutions to the tasks. \n",
    "We only need the model to create tasks for Linear Regression"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "So we ask the model again stating our requirements more clearly this time."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### 2nd prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      " \"cells\": [\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"# Applying Linear Regression on a Dataset\\n\",\n",
      "    \"\\n\",\n",
      "    \"In this notebook, we will learn how to apply linear regression on a dataset using Python and Scikit-learn. We will follow the following steps:\\n\",\n",
      "    \"\\n\",\n",
      "    \"1. Importing the necessary libraries\\n\",\n",
      "    \"2. Loading the dataset\\n\",\n",
      "    \"3. Exploring the dataset\\n\",\n",
      "    \"4. Preprocessing the dataset\\n\",\n",
      "    \"5. Splitting the dataset into training and testing sets\\n\",\n",
      "    \"6. Creating a linear regression model\\n\",\n",
      "    \"7. Training the model\\n\",\n",
      "    \"8. Evaluating the model\\n\",\n",
      "    \"9. Making predictions\\n\",\n",
      "    \"10. Visualizing the results\\n\",\n",
      "    \"\\n\",\n",
      "    \"Let's get started!\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"## 1. Importing the necessary libraries\\n\",\n",
      "    \"\\n\",\n",
      "    \"In this step, we will import the necessary libraries for applying linear regression on a dataset. We will be using the following libraries:\\n\",\n",
      "    \"\\n\",\n",
      "    \"- pandas\\n\",\n",
      "    \"- numpy\\n\",\n",
      "    \"- matplotlib\\n\",\n",
      "    \"- seaborn\\n\",\n",
      "    \"- scikit-learn\\n\",\n",
      "    \"\\n\",\n",
      "    \"We will import these libraries using their standard aliases.\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"## 2. Loading the dataset\\n\",\n",
      "    \"\\n\",\n",
      "    \"In this step, we will load the dataset that we will be using for applying linear regression. We will be using a dataset that contains information about the fuel consumption of different cars. The dataset can be downloaded from the following link:\\n\",\n",
      "    \"\\n\",\n",
      "    \"https://archive.ics.uci.edu/ml/datasets/auto+mpg\\n\",\n",
      "    \"\\n\",\n",
      "    \"We will load the dataset using pandas and store it in a pandas dataframe.\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"## 3. Exploring the dataset\\n\",\n",
      "    \"\\n\",\n",
      "    \"In this step, we will explore the dataset that we have loaded. We will check the shape of the dataset, the column names, and the data types of the columns. We will also check for any missing values in the dataset.\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"## 4. Preprocessing the dataset\\n\",\n",
      "    \"\\n\",\n",
      "    \"In this step, we will preprocess the dataset by performing the following tasks:\\n\",\n",
      "    \"\\n\",\n",
      "    \"- Removing any unnecessary columns\\n\",\n",
      "    \"- Handling missing values\\n\",\n",
      "    \"- Encoding categorical variables\\n\",\n",
      "    \"- Scaling the features\\n\",\n",
      "    \"\\n\",\n",
      "    \"We will use scikit-learn's preprocessing module for performing these tasks.\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"## 5. Splitting the dataset into training and testing sets\\n\",\n",
      "    \"\\n\",\n",
      "    \"In this step, we will split the preprocessed dataset into training and testing sets. We will use scikit-learn's train_test_split function for splitting the dataset.\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"## 6. Creating a linear regression model\\n\",\n",
      "    \"\\n\",\n",
      "    \"In this step, we will create a linear regression model using scikit-learn's LinearRegression class.\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"## 7. Training the model\\n\",\n",
      "    \"\\n\",\n",
      "    \"In this step, we will train the linear regression model using the training set that we created in step 5.\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"## 8. Evaluating the model\\n\",\n",
      "    \"\\n\",\n",
      "    \"In this step, we will evaluate the performance of the linear regression model using the testing set that we created in step 5. We will calculate the mean squared error and the R-squared score of the model.\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"## 9. Making predictions\\n\",\n",
      "    \"\\n\",\n",
      "    \"In this step, we will use the trained linear regression model to make predictions on new data. We will make predictions on a sample of the testing set that we created in step 5.\"\n",
      "   ]\n",
      "  },\n",
      "  {\n",
      "   \"cell_type\": \"markdown\",\n",
      "   \"metadata\": {},\n",
      "   \"source\": [\n",
      "    \"## 10. Visualizing the results\\n\",\n",
      "    \"\\n\",\n",
      "    \"In this step, we will visualize the results of the linear regression model. We will create a scatter plot of the actual values and the predicted values.\"\n",
      "   ]\n",
      "  }\n",
      " ],\n",
      " \"metadata\": {\n",
      "  \"kernelspec\": {\n",
      "   \"display_name\": \"Python 3.8.5 64-bit\",\n",
      "   \"language\": \"python\",\n",
      "   \"name\": \"python385jvsc74a57d7c5c4d4a7a7d5c4d4a7a\"\n",
      "  },\n",
      "  \"language_info\": {\n",
      "   \"codemirror_mode\": {\n",
      "    \"name\": \"ipython\",\n",
      "    \"version\": 3\n",
      "   },\n",
      "   \"file_extension\": \".py\",\n",
      "   \"mimetype\": \"text/x-python\",\n",
      "   \"name\": \"python\",\n",
      "   \"nbconvert_exporter\": \"python\",\n",
      "   \"pygments_lexer\": \"ipython3\",\n",
      "   \"version\": \"3.8.5\"\n",
      "  }\n",
      " },\n",
      " \"nbformat\": 4,\n",
      " \"nbformat_minor\": 5\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "messages = chat_prompt_value.to_messages()\n",
    "\n",
    "## prompt 3\n",
    "prompt = \"\"\"No I don't need solutions. I just need tasks questions which are \n",
    "    listed in the different markdown cells of the jupyter notebook \n",
    "    ipynb json format i.e. \"cell_type\":\"markdown\"\n",
    "    \"\"\"\n",
    "\n",
    "messages.append(\n",
    "    HumanMessage(content=prompt)\n",
    ")\n",
    "\n",
    "res = chat(messages)\n",
    "\n",
    "print(res.content)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Great. Again after iterative prompting and being more specific and restrictive we finally got the desired output in correct format too."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Saving the output to the file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating notebook: generated_notebooks/generating_task_notebook.ipynb\n"
     ]
    }
   ],
   "source": [
    "create_notebook(result=res.content, filename=\"generating_task_notebook.ipynb\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Evaluating Outputs"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Evaluating responses using rubric\n",
    "- Rubric - a set of instructions or rules.\n",
    "\n",
    "- Reference - [DeepLearningAI ChatGPT Course](https://learn.deeplearning.ai/chatgpt-building-system)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We define another helper function `get_completion_from_messages()`. This function will help us to get the response from the model with an input comprised of a list containing json like objects. These json like objects consists of system message and user message in a format `{'role': 'system', 'content': system_message}` and `{'role': 'user', 'content': user_message}` respectively.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_completion_from_messages(messages, \n",
    "                                 model=\"gpt-3.5-turbo\", \n",
    "                                 temperature=0, \n",
    "                                 max_tokens=500):\n",
    "    response = openai.ChatCompletion.create(\n",
    "        model=model,\n",
    "        messages=messages,\n",
    "        temperature=temperature,\n",
    "        max_tokens=max_tokens,\n",
    "    )\n",
    "    return response.choices[0].message[\"content\"]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We then define a function called `eval_vs_ideal()`. This function evaluates the response generated from the ChatGPT for a user query. \n",
    "\n",
    "This function takes 2 inputs:\n",
    "- Ideal test case\n",
    "- Response to evaluate\n",
    "\n",
    "Specifically in this case, the function only checks if the response generated by the model suffices the requirements by comparing the answer to the ideal test case. \n",
    "\n",
    "The output from the model again goes into the model for evaluation, thereby leveraging the power of the LLMs."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The code compares the model response and the ideal answer (expert answer) and outputs from one of the following choices\n",
    "\n",
    "Compare the factual content of the submitted answer with the expert answer. DO NOT ignore any differences in style or punctuation.\n",
    "    The submitted answer may either be a subset or superset of the expert answer, or it may conflict with it. \n",
    "        Determine which case applies. Answer the question by selecting one of the following options: <br>\n",
    "    (A) The submitted answer is a part of the expert answer and is fully consistent with it. <br>\n",
    "    (B) The submitted answer contains all the same details as the expert answer. <br>\n",
    "    (C) There is a disagreement between the submitted answer and the expert answer. <br>\n",
    "    (D) The answers differ, but these differences don't matter from the perspective of factuality. <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "def eval_vs_ideal(test_set, assistant_answer):\n",
    "\n",
    "    ideal_user_msg = test_set['user_msg']\n",
    "    ideal_answer = test_set['ideal_answer']\n",
    "    \n",
    "    system_message = \"\"\"\\\n",
    "    You are an assistant that evaluates how well the AI Data Scientist agent \\\n",
    "    answers a user question by comparing the response to the ideal (expert) response\n",
    "    \"\"\"\n",
    "\n",
    "    user_message = f\"\"\"\\\n",
    "You are comparing a submitted answer to an expert answer on a given question. Here is the data:\n",
    "    [BEGIN DATA]\n",
    "    ************\n",
    "    [Question]: {ideal_user_msg}\n",
    "    ************\n",
    "    [Expert]: {ideal_answer}\n",
    "    ************\n",
    "    [Submission]: {assistant_answer}\n",
    "    ************\n",
    "    [END DATA]\n",
    "\n",
    "Compare the factual content of the submitted answer with the expert answer. DO NOT ignore any differences in style or punctuation.\n",
    "    The submitted answer may either be a subset or superset of the expert answer, or it may conflict with it. \n",
    "        Determine which case applies. Answer the question by selecting one of the following options:\n",
    "    (A) The submitted answer is a part of the expert answer and is fully consistent with it.\n",
    "    (B) The submitted answer contains all the same details as the expert answer.\n",
    "    (C) There is a disagreement between the submitted answer and the expert answer.\n",
    "    (D) The answers differ, but these differences don't matter from the perspective of factuality.\n",
    "\"\"\"\n",
    "\n",
    "    messages = [\n",
    "        {'role': 'system', 'content': system_message},\n",
    "        {'role': 'user', 'content': user_message}\n",
    "    ]\n",
    "\n",
    "    response = get_completion_from_messages(messages)\n",
    "    return response"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "> <span style=\"color:green\"> **User Message** - </span> How to plot box plots with matplotlib. Do include cleaning of data before plotting and \n",
    "    add comments for the code. Also give the output in jupyter notebook ipynb \"json\" format."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "\n",
    "Let us assume that the model gives us the below output when user message is same as above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "\n",
    "assistant_answer_1 = \"\"\"\n",
    "    # Importing necessary libraries\n",
    "    import pandas as pd\n",
    "    import matplotlib.pyplot as plt\n",
    "\n",
    "    # Reading the data from a csv file\n",
    "    data = pd.read_csv('data.csv')\n",
    "\n",
    "    # Cleaning the data\n",
    "    # Removing null values\n",
    "    data.dropna(inplace=True)\n",
    "\n",
    "    # Plotting the box plot\n",
    "    plt.boxplot(data['column_name'])\n",
    "\n",
    "    # Adding title and labels to the plot\n",
    "    plt.title('Box plot of column_name')\n",
    "    plt.xlabel('column_name')\n",
    "    plt.ylabel('Values')\n",
    "\n",
    "    # Displaying the plot\n",
    "    plt.show()\n",
    "\n",
    "    # Saving the plot as a png file\n",
    "    plt.savefig('boxplot.png')\n",
    "\"\"\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Let us define the ideal test set, which consists of ideal user message and the ideal answer the user expects"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "test_set_ideal = {\n",
    "\n",
    "\n",
    "    'user_msg':\"\"\"How to plot box plots with matplotlib.\\\n",
    "Do include cleaning of data before plotting and \\\n",
    "add comments for the code. Also give the output in \\\n",
    "jupyter notebook ipynb \"json\" format.\"\"\",\n",
    "\n",
    "\n",
    "\n",
    "'ideal_answer':\"\"\"\n",
    "    {\n",
    " \"cells\": [\n",
    "  {\n",
    "   \"attachments\": {},\n",
    "   \"cell_type\": \"markdown\",\n",
    "   \"metadata\": {},\n",
    "   \"source\": [\n",
    "    \"## Plotting Boxplots with Matplotlib\"\n",
    "   ]\n",
    "  },\n",
    "  {\n",
    "   \"cell_type\": \"code\",\n",
    "   \"execution_count\": null,\n",
    "   \"metadata\": {},\n",
    "   \"outputs\": [],\n",
    "   \"source\": [\n",
    "    \"# Importing necessary libraries\\n\",\n",
    "    \"import pandas as pd\\n\",\n",
    "    \"import matplotlib.pyplot as plt\"\n",
    "   ]\n",
    "  },\n",
    "  {\n",
    "   \"cell_type\": \"code\",\n",
    "   \"execution_count\": null,\n",
    "   \"metadata\": {},\n",
    "   \"outputs\": [],\n",
    "   \"source\": [\n",
    "    \"# Reading the csv file into a pandas dataframe\\n\",\n",
    "    \"df = pd.read_csv('filename.csv')\"\n",
    "   ]\n",
    "  },\n",
    "  {\n",
    "   \"cell_type\": \"code\",\n",
    "   \"execution_count\": null,\n",
    "   \"metadata\": {},\n",
    "   \"outputs\": [],\n",
    "   \"source\": [\n",
    "    \"# Cleaning the data\\n\",\n",
    "    \"# Removing null values\\n\",\n",
    "    \"df.dropna(inplace=True)\"\n",
    "   ]\n",
    "  },\n",
    "  {\n",
    "   \"cell_type\": \"code\",\n",
    "   \"execution_count\": null,\n",
    "   \"metadata\": {},\n",
    "   \"outputs\": [],\n",
    "   \"source\": [\n",
    "    \"# Plotting the boxplot\\n\",\n",
    "    \"plt.boxplot(df['column_name'])\\n\",\n",
    "    \"plt.title('Boxplot of column_name')\\n\",\n",
    "    \"plt.show()\"\n",
    "   ]\n",
    "  }\n",
    " ],\n",
    " \"metadata\": {\n",
    "  \"language_info\": {\n",
    "   \"name\": \"python\"\n",
    "  },\n",
    "  \"orig_nbformat\": 4\n",
    " },\n",
    " \"nbformat\": 4,\n",
    " \"nbformat_minor\": 2\n",
    "}\n",
    "\"\"\"\n",
    "}"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "With the model's response, we evaluate the response. If the evaluation is successful we can proceed with showing the response to the user, otherwise we can continue in loop to improve the answer until the evaluation is successful"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C) There is a disagreement between the submitted answer and the expert answer. \n",
      "\n",
      "Explanation: \n",
      "- The submitted answer includes additional steps such as adding title and labels to the plot, displaying the plot, and saving the plot as a png file, which are not present in the expert answer.\n",
      "- The expert answer includes a step to read the csv file into a pandas dataframe, which is not present in the submitted answer.\n",
      "- The expert answer specifies to plot the boxplot of a particular column, whereas the submitted answer does not specify which column to plot.\n"
     ]
    }
   ],
   "source": [
    "print(eval_vs_ideal(test_set=test_set_ideal, assistant_answer=assistant_answer_1))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Now let us assume that after some iteration we get the response as below. Now let us check how the model evaluates this answer\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "assistant_answer_2 = \"\"\"\n",
    " {\n",
    " \"cells\": [\n",
    "  {\n",
    "   \"attachments\": {},\n",
    "   \"cell_type\": \"markdown\",\n",
    "   \"metadata\": {},\n",
    "   \"source\": [\n",
    "    \"## Plotting Boxplots with Matplotlib\"\n",
    "   ]\n",
    "  },\n",
    "  {\n",
    "   \"cell_type\": \"code\",\n",
    "   \"execution_count\": null,\n",
    "   \"metadata\": {},\n",
    "   \"outputs\": [],\n",
    "   \"source\": [\n",
    "    \"# Importing necessary libraries\\n\",\n",
    "    \"import pandas as pd\\n\",\n",
    "    \"import matplotlib.pyplot as plt\\n\",\n",
    "    \"\\n\",\n",
    "    \"# Reading the csv file into a pandas dataframe\\n\",\n",
    "    \"df = pd.read_csv('filename.csv')\\n\",\n",
    "    \"\\n\",\n",
    "    \"# Cleaning the data\\n\",\n",
    "    \"# Removing null values\\n\",\n",
    "    \"df.dropna(inplace=True)\\n\",\n",
    "    \"\\n\",\n",
    "    \"# Plotting the boxplot\\n\",\n",
    "    \"plt.boxplot(df['column_name'])\\n\",\n",
    "    \"plt.title('Boxplot of column_name')\\n\",\n",
    "    \"plt.show()\"\n",
    "   ]\n",
    "  }\n",
    " ]\n",
    "}\n",
    "\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Answer: (A) The submitted answer is a part of the expert answer and is fully consistent with it. \n",
      "\n",
      "Explanation: The submitted answer is a subset of the expert answer and contains all the necessary details to plot box plots with Matplotlib. The code is identical to the expert answer, and the comments are also present. The only difference is that the submission has an extra blank line in the code, which does not affect the factuality of the answer. Therefore, the submitted answer is fully consistent with the expert answer.\n"
     ]
    }
   ],
   "source": [
    "print(eval_vs_ideal(test_set=test_set_ideal, assistant_answer=assistant_answer_2))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Thank You"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}