(participant)_Using_RNN_for_Sentiment_Analysis.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "(participant) Using RNN for Sentiment Analysis.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/kumar4372/sentiment_analysis_hands_on/blob/master/(participant)_Using_RNN_for_Sentiment_Analysis.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2DYHYfgAauWJ"
      },
      "source": [
        "# **Sentiment Analysis Using Recurrent Neural Network**\n",
        "\n",
        "---\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5PpfRxiBX7HT"
      },
      "source": [
        "## choose Hardware accelarator to \"GPU\" for faster computation. Go to \"Runtime\" -> \"Change runtime type\" to change it.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OXpXUF4cbI0b"
      },
      "source": [
        "In this tutorial, we will use RNN/LSTM for sentiment analysis on movie review dataset.\n",
        "\n",
        "**What is sentiment analysis?**\n",
        "\n",
        "Sentiment Analysis is nothing but finding the sentiments of reviews whether it is positive or negative review.\n",
        "\n",
        "**Example Code to refer**: https://keras.io/examples/nlp/bidirectional_lstm_imdb/\n",
        "\n",
        "**Notes**\n",
        "- RNNs are tricky. Choice of batch size is important,\n",
        "choice of loss and optimizer is critical, etc.\n",
        "Some configurations won't converge.\n",
        "- LSTM loss decrease patterns during training can be quite different\n",
        "from what you see with CNNs/MLPs/etc."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YAZNYKDpbL_4"
      },
      "source": [
        "**Importing Libraries**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jzhfuZIPuajX"
      },
      "source": [
        "We start by importing the required dependencies to preprocess our data and build our model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8jZZT3W6j60O",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e584dd9a-33c1-4c29-a783-350358a57586"
      },
      "source": [
        "# Import the dependencies\n",
        "from tensorflow.keras.datasets import imdb\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Embedding, Dense, SimpleRNN,LSTM, GRU\n",
        "from tensorflow.keras.preprocessing import sequence\n",
        "print(\"Imported dependencies.\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Imported dependencies.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LWcnwxoJjnxd"
      },
      "source": [
        "**Loading Data**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WRzugSCquNN5"
      },
      "source": [
        "We will use IMDB sentiment classification dataset which consists of 50,000 movie reviews from IMDB users that are labeled as either positive (1) or negative (0). \n",
        "\n",
        "Continue downloading the IMDB dataset, which is, fortunately, already built into keras."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jlgiU4PSkLUn"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OKwL1Y-aslN7"
      },
      "source": [
        "**Exploring the data**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CmS3ls6YyVLN"
      },
      "source": [
        "You can see in the output above that the dataset is labeled into two categories, — 0 or 1, which represents the sentiment of the review. The whole dataset contains 9,998 unique words and the average review length is 234 words, with a standard deviation of 173 words."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RODKEpHUlaH8"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r1mgv2N_07F8"
      },
      "source": [
        "We should always check how balanced our training and test data is. This helps in deciding evaluation metrics and observing training progress as well. You will observe that both training and test data is perfectly balanced."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VWIgu1y01I09"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G3Ya_GxcuETI"
      },
      "source": [
        "You can see the first review of the dataset, which is labeled as positive (1). "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TdOq8BHEmEXg"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JLWk5R9huA5l"
      },
      "source": [
        "Now we try to map from word index to word so that we can read the reviews.\n",
        "We replace every unknown word with a “#”. It does this by using the get_word_index() function."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-TW5kQUWmQDP",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5b39c9d6-ce46-4dd6-aa1e-e2ee53638d2b"
      },
      "source": [
        "index = imdb.get_word_index() # from word to index mapping\n",
        "reverse_index = dict([(value, key) for (key, value) in index.items()]) # from index to word mapping"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb_word_index.json\n",
            "1646592/1641221 [==============================] - 0s 0us/step\n",
            "1654784/1641221 [==============================] - 0s 0us/step\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-ogflKtFBbwf"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kJHHWlzKBcCF"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "adNJ_dvIV566"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AtEQG_0nqzz3"
      },
      "source": [
        "**Data Preparation**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1B-OZoHct2qi"
      },
      "source": [
        "Now it's time to prepare our data. \n",
        "\n",
        "As we know, each review consists of different number of words. Some reviews could even be of length 1. e.g. \"nice\"\n",
        "\n",
        "We need to fix maximum length of our input sequence. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "842z0XWIuDOo"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8XPeGSTsuGuR"
      },
      "source": [
        "Check how many sentences are of length less than x where x is any integer"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cr4lzF42uERT"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1QvGkeDKuyPq"
      },
      "source": [
        "Here we consider maximum length of our input sequence to be 200.\n",
        "\n",
        "**Please feel free to choose your own maximum length**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DNS1Jr-omVC3"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mDLJtIfowuMZ"
      },
      "source": [
        "To visualize how padding is happening, let's print a few examples of seqeunces for before and after padding\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L_eUysG_fucz"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7jDIdSwmwFkX"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Hfupbfuaqg0f"
      },
      "source": [
        "**BUILDING AND TRAINING THE MODEL**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oPqDpti_tjHd"
      },
      "source": [
        "Now our data is ready for some modelling!\n",
        "\n",
        "Deep learning models have layers.\n",
        "\n",
        "The top layer takes in the data we've just prepared, the middle layers do some math on this data and the final layer produces an output we can hopefully make use of.\n",
        "\n",
        "In our case, our model has three layers, \n",
        "\n",
        "1. Embedding layer\n",
        "2. LSTM layer\n",
        "3. Dense layer.\n",
        "\n",
        "Our model begins with the line model = Sequential(). Think of this as simply stating \"our model will flow from input to output layer in a sequential manner\" or \"our model goes one step at a time\".\n",
        "\n",
        "**Embedding layer**\n",
        "\n",
        "The Embedding layer creates a database of the relationships between words.\n",
        "\n",
        "model.add(Embedding(max_words, embedding_vector_length, input_length=max_review_length)) is saying: add an Embedding layer to our model and use it to turn each of our words into embedding_vector_length dimensional vector which have some mathematical relationship to each other.\n",
        "\n",
        "So each of our words will become vectors of dimension embedding_vector_length.\n",
        "\n",
        "For example, vector of \"the\" = [0.556433, 0.223122, 0.789654....].\n",
        "\n",
        "Don't worry for now how this is computed, Keras does it for us.\n",
        "\n",
        "**LSTM layer**\n",
        "\n",
        "model.add(LSTM(128)) is saying: add a LSTM layer after our embedding layer in our model and give it 128 units.\n",
        "\n",
        "**Dense layer**\n",
        "\n",
        "model.add(Dense(1, activation='sigmoid')) is saying: add a Dense layer to the end of our model and use a sigmoid activation function to produce a meaningful output.\n",
        "\n",
        "A dense layer is also known as a fully-connected layer. This layer connects the 128 LSTM units in the previous layer to 1 unit. This last unit them takes all this information and runs it through a sigmoid function.\n",
        "\n",
        "**Please feel free to change the model architecture.**\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Rwu39HI97hcm"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n6807KVQhXRz"
      },
      "source": [
        "**Compiling the model**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W-FzmE3mqXYc"
      },
      "source": [
        "Now we compile our model, which is nothing but configuring the model for training. We use the “adam” optimizer, an algorithm that changes the weights and biases during training. We also choose binary-crossentropy as loss (because we deal with binary classification) and accuracy as our evaluation metric."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ghfPqcms-S2V"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8Hzb0SrkhyBM"
      },
      "source": [
        "**Summarize the model**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CYvRQw0uh4yr"
      },
      "source": [
        "Making a summary of the model will give us an idea of what's happening at each layer.\n",
        "\n",
        "In the embedding layer, each of our words is being turned into a vector of dimension 128. Because there are 10000 words (max_words), there are 1,280,000 parameters (128 x 10000).\n",
        "\n",
        "Parameters are individual pieces of information. The goal of the model is to take a large number of parameters and reduce them down to something we can understand and make use of (less parameters).\n",
        "\n",
        "The LSTM layer reduces the number of parameters to 131584 = 4 × [128(128+128) + 128].\n",
        "\n",
        "The final dense layer connects each of the outputs of the LSTM units into one cell (128 + 1)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5C36PeZgiKjI"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8JKKMT6ZiR9Y"
      },
      "source": [
        "**Fitting the model to the training data**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nz3QpxeuiW2o"
      },
      "source": [
        "Now our model is compiled, it's ready to be set loose on our training data.\n",
        "\n",
        "We'll be training for 3 epochs with a batch_size of 64.\n",
        "\n",
        "Because of our loss and optimzation functions, the model accuracy should improve after each cycle.\n",
        "\n",
        "model.fit(X_train, y_train, epochs=3, batch_size=64) is saying: fit the model we've built on the training dataset for 3 cycles and go over 64 reviews at a time.\n",
        "\n",
        "I use test data as validation data. Use validation_split parameter in model.fit if you want to split training data into train and val.\n",
        "\n",
        "**Please Feel free to change the number of epochs or batch_size**\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "STmd3VIPpAHk"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Cn2g3ZRoqGK_"
      },
      "source": [
        "It is time to evaluate our model:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2bs_I_gWbIIh"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sXZaFmY5dauk"
      },
      "source": [
        "Let's analyze the results by looking at some examples."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aETOH47QY_tE"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JrvqtcqVZXsp"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8r5GkH_umPrX"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "awHoVtqOZ5gj"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fBY-Yv-KmXVD"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BeiAaANaZjUX"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iC0uqVH7ZmLx"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Se8vLfYQaBdS"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DQsFZZDCad2T"
      },
      "source": [
        "## some samples of wrong predictions"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "G1tBg9mkaFp7"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gGoFsTbhcdTI"
      },
      "source": [
        "## some samples of correct predictions"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ihLVvshlaFs2"
      },
      "source": [
        "# enter your code"
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}