1_Perceptron.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Perceptron Tutorial\n",
    "\n",
    "The perceptron algorithm is a supervised algorithm for binary classification. This linear classifier was introduced by Frank Rosenblatt in 1957 in his paper, 'The Perceptron--a perceiving and recognizing automaton.' The perceptron makes predictions based on a linear combination of it's weights with the feature vector. It also allows for online learning, which means that it can process each element in a training set one at a time. \n",
    "\n",
    "The perceptron is trained with the 'perceptron learning rule' which is defined as follows:\n",
    "\n",
    "1. Initialize the weight vector (Wo), either to 0 or small random numbers. \n",
    "2. Calculate the initial 'guess' as a linear combination of that weight vector and the first input.\n",
    "3. Compare the perceptron output to the target output (error). Since this is a binary classification algorithm the outputs and targets will either be values of 0 or 1.\n",
    "4. Update the new weight vector (Wn) depending on how the target compares to the perceptron output. If the target is less than the output than the weights will be decreased and if the target is greater than the output the weights will be increased. \n",
    "5. Steps 1-4 are repeated until every output matches the target value for each training example.\n",
    "\n",
    "Here we are going to code up a perceptron and train it with the perceptron learning rule using a hard limit as the activation function. I am using numpy here because it is readible and optimized for linear algebra operations. I am defining the perceptron rule as:\n",
    "\n",
    "```\n",
    "Wn = Wo + alpha * error * input.T\n",
    "error = true - predicted\n",
    "```\n",
    "\n",
    "The complete code for the perceptron is below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 316,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import random\n",
    "import numpy as np\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "plt.style.use('ggplot')\n",
    "\n",
    "def hardlim(a):\n",
    "    if a > 0.0:\n",
    "        return 1.\n",
    "    else:\n",
    "        return 0.\n",
    "\n",
    "class Perceptron(object):\n",
    "    '''\n",
    "    simple feed forward perceptron with a hard limit activation\n",
    "    trained with the perceptron learning rule\n",
    "    '''\n",
    "    def __init__(self):\n",
    "        self.alpha = None\n",
    "        self.w = None\n",
    "\n",
    "    def response(self, X):\n",
    "        '''\n",
    "        perceptron response\n",
    "        :param X: input vector\n",
    "        :return: perceptron out\n",
    "        '''\n",
    "        a = hardlim(np.dot(self.w.T, X))\n",
    "        return a\n",
    "\n",
    "    def updateWeight(self, X, error):\n",
    "        '''\n",
    "        update the vector of input weights\n",
    "        :param X: input data\n",
    "        :param error: prediction != true\n",
    "        :return: updated weight vector\n",
    "        '''\n",
    "        self.w += self.alpha * error * X\n",
    "\n",
    "    def train(self, X, y, alpha, iterations):\n",
    "        '''\n",
    "        trains perceptron on vector data by looping each row and updating the weight vector\n",
    "        :param X: input data\n",
    "        :param y: correct y value\n",
    "        :return: updated parameters\n",
    "        '''\n",
    "        # initialize the learning rate and count data size\n",
    "        self.alpha = alpha\n",
    "        num_examples, num_features = np.shape(X)\n",
    "\n",
    "        # set up bias\n",
    "        bias = np.ones(shape=(num_examples,1))\n",
    "        X = np.hstack((X, bias))\n",
    "\n",
    "        # initialize weight vector\n",
    "        self.w = np.random.rand(num_features + 1)\n",
    "        \n",
    "        error_count = []\n",
    "        for i in range(iterations):\n",
    "            for j in range(num_examples):\n",
    "                prediction = self.response(X[j])\n",
    "                error = int(y[j]) - prediction \n",
    "                self.updateWeight(X[j], error)\n",
    "                error_count.append(error)\n",
    "                \n",
    "        error_count = np.array(error_count)\n",
    "        plt.plot(error_count)\n",
    "        plt.ylim([-1,1])\n",
    "        plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The perceptron is contained in a python class. Within the class there are functions to create the output, update the weights based on the error and a function to contain the training loop. \n",
    "\n",
    "* **Response** - This simply calcualtes the linear combination of the weights and input vectors. Then pushes that output through the hard limit function which will generate either a 0 or 1 for the class prediction. \n",
    "* **UpdateWeight** - This function will take the error term and update the perceptron weights as defined by the perceptron learning rule.\n",
    "* **Train** - The train function first initializes some of the parameters of the perceptron. It will take an 'alpha' which should be any number between 0 and 1. Then it will determine the proper size of the weight vector by setting it to the corresponding size of the input vectors. Finally before training it will initialize that weight vector to small random numbers and tack on a bias. During training the loop will run for a predetermined number of iterations. On each iteration the perceptron will; generate it's prediciton, compare that prediction to the target (error), then update the weights as necessary based on the error term.\n",
    "\n",
    "At the end of training the perceptron will generate a plot which shows the error generated by each training example. When the plot flatlines at 0 it may be safe to assume that the perceptron converged to a good solution.\n",
    "\n",
    "Now we will come up with some data for the perceptron to solve. Let's take the data from [Neural Network Design](http://hagan.okstate.edu/NNDesign.pdf) page 4-7 and plot the points below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 309,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAD/CAYAAAAQaHZxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFRtJREFUeJzt3V9MW+cdxvGHPwkQcCAHJUUERWLIqKnHUCpDpFGJLm2u\nehGmqp6iSbtgSiRIaKBrugUJqRJLQLRr0mZNVSVhkbYrOmncVOpEqjWVmm6JV0iZ15ZAdzGKSFq7\nDhCaCrB30QVBwcA5Njjwfj9XHPs9fn/8dPLEfnltp0Sj0agAABtaarILAACsPsIeAAywIcM+EAgk\nu4R1hX7ZQ7/soV/2rFa/CHvQL5volz30yx7CHgDgGGEPAAZIYeslAGx86ckuYCkjIyOOznO5XBof\nH09wNRsX/bKHftlDv+yJt1+FhYWL3s4yDgAY4IF+Zr9SoVBIb3d26sv+fqVPTCg9LU3TMzOazsnR\n9rIyPVVbK8uykl0mACwQCoXU2fm2+vu/1MREutLS0jUzM62cnGmVlW1Xbe1TCcmvB3rNfiXLOJ/e\nuKHuhga9MDSkokXuH5bUUVKimrNn9XB5ecJr3Ah4mW0P/bKHfsV248anamjo1tDQC1KMBCsp6dDZ\nszUqL394RY+5ass4586d06FDh/SrX/0q5pjOzk49++yzOn78uP7zn//EO+WsT2/c0FtNTToTI+il\n79p3ZmhIf25sVP/16wmbGwDicePGp2pqektDQ2e0eNBLUpGGhs6osfHPun69P6754g77n/zkJ2pu\nbo55/0cffaRbt27ptdde0+HDh3XhwoV4p5T03Uuf7oYGvf7ZZ8v+EqmSfj8woGv19frHu+8mZH4A\ncCoUCqmhoVufffa6lo/hVA0M/F719df07rv/cDxn3GG/e/duZWdnx7zf7/erurpakuR2u3X37l2F\nw+F4p9XbnZ16YWhoxb9AqqSWkRFdaW3V5ORk3PMDgFOdnW//f+lm5Qk2MtKi1tYrjvNr1XfjhEIh\n5efnzx7n5+crFArF/bhf9vfHfOGzlCM3b6r7/Pm45wcAp/r7v1TspZvYbt48ovPnux3NuSZbL1fj\nb8DpExOOznNLGvX7E1sMANgwMeF0I6Rbfv+oozNXfeulZVkKBoOzx8FgcNFtRIFAYN4HAPl8Prlc\nrpiPm56W5rimzKmpJR/bNJs3b6YfNtAve+jXQmlpzqN3aipz2X52dXXN/uzxeOTxeFY/7L1er/76\n17+qqqpKAwMDys7OVl5e3oJx9wuaa6ntWtMzM45rurdpE1vB5mBrnD30yx76tdDMzLTjczdturdk\nP10ul3w+34Lb4w77M2fO6JNPPtHY2Jjq6ur0zDPPaOb/Qbx//349+uij6u3tVUNDgzIzM1VXVxfv\nlJKk6ZwcR+fdlFTg9SakBgBwIifHadjflNdb4OjMuMO+sbFx2TG//OUv451mge1lZRq+fNnWnzii\nkl53u9Vw+HDC6wGAlSor267Ll4dl74+0Ubndr+vw4QZHc67bz8Z5qrZWHSUliqxwfETSbwsLVd3S\noqysrNUsDQCWVFv7lEpKOiQbCVZY+Fu1tFQ7zq91G/aWZanm7FkdLS1dtl0RSUdLS1V57pz2PvHE\nWpQHADFZlqWzZ2tUWnpUywd+RKWlR3XuXKWeeGKv4znXbdhL0sPl5TrQ0aFTMT4LQvpu6eZkYaEO\ndHSorKJi7YoDgCWUlz+sjo4DKiw8tcSoqAoLT6qj44AqKsrimm9dh70klVVUyNPermNut25+776b\nkhrdbv2wvZ2gB/DAqagoU3u7R273MWmRBHO7G9Xe/sO4g17aAJ96ed/k5KS6z5/XqN+vzKkp3du0\nSQVer2oOHdKWLVtWscr1j61x9tAve+jX8iYnJ3X+fLf8/lFNTWVq06Z78noLdOhQje38ivWplxsm\n7Ofi4rKHftlDv+yhX/bwTVUAAMcIewAwAGEPAAYg7AHAAIQ9ABiAsAcAAxD2AGAAwh4ADEDYA4AB\nCHsAMABhDwAGIOwBwACEPQAYgLAHAAMQ9gBgAMIeAAxA2AOAAQh7ADAAYQ8ABiDsAcAAhD0AGICw\nBwADEPYAYADCHgAMQNgDgAEIewAwAGEPAAYg7AHAAIQ9ABiAsAcAAxD2AGAAwh4ADEDYA4ABCHsA\nMABhDwAGIOwBwADp8T5AX1+fLl26pEgkon379qmmpmbe/YFAQB0dHXrooYckSXv37tXTTz8d77QA\nABviCvtIJKKLFy+qpaVFlmXpxIkT8nq9KioqmjfukUce0a9//eu4CgUAOBfXMs7g4KAKCgq0Y8cO\npaenq6qqSn6/f8G4aDQazzQAgDjFFfahUEj5+fmzx5ZlKRQKzRuTkpKigYEBHT9+XG1tbRoeHo5n\nSgCAA3Gv2S+nuLhYb7zxhjIyMtTb26uXXnpJr7766oJxgUBAgUBg9tjn88nlcjmac/PmzY7PNRH9\nsod+2UO/7ElEv7q6umZ/9ng88ng88YW9ZVkKBoOzx8FgUJZlzRuTlZU1+/OePXt04cIFTUxMKCcn\nZ964+wXNNT4+7qgul8vl+FwT0S976Jc99MueePvlcrnk8/kW3B7XMk5JSYlGR0d1+/ZtTU9P6+rV\nq/J6vfPGhMPh2TX7wcFBSVoQ9ACA1RXXM/u0tDTV1tbq5MmTs1svi4qK1NPTI0nav3+//v73v6un\np0epqanKyMjQsWPHElI4AGDlUqIP8FaZkZERR+fxstEe+mUP/bKHftkTb78KCwsXvZ130AKAAQh7\nADAAYQ8ABiDsAcAAhD0AGICwBwADEPYAYADCHgAMQNgDgAEIewAwAGEPAAYg7AHAAIQ9ABiAsAcA\nAxD2AGAAwh4ADEDYA4ABCHsAMABhDwAGIOwBwACEPQAYgLAHAAMQ9gBgAMIeAAxA2AOAAQh7ADAA\nYQ8ABiDsAcAAhD0AGICwBwADEPYAYADCHgAMQNgDgAEIewAwAGEPAAYg7AHAAIQ9ABiAsAcAAxD2\nAGAAwh4ADEDYA4ABCHsAMEB6vA/Q19enS5cuKRKJaN++faqpqVkwprOzU319fcrIyFB9fb2Ki4vj\nnRYAYENcz+wjkYguXryo5uZmvfLKK/rggw80PDw8b8xHH32kW7du6bXXXtPhw4d14cKFuAoGANgX\nV9gPDg6qoKBAO3bsUHp6uqqqquT3++eN8fv9qq6uliS53W7dvXtX4XA4nmkBADbFFfahUEj5+fmz\nx5ZlKRQKLTkmPz9/wRgAwOqKe81+JaLR6LJjAoGAAoHA7LHP55PL5XI03+bNmx2fayL6ZQ/9sod+\n2ZOIfnV1dc3+7PF45PF44gt7y7IUDAZnj4PBoCzLsj1mbkFzjY+PO6rL5XI5PtdE9Mse+mUP/bIn\n3n65XC75fL4Ft8e1jFNSUqLR0VHdvn1b09PTunr1qrxe77wxXq9X77//viRpYGBA2dnZysvLi2da\nAIBNcT2zT0tLU21trU6ePDm79bKoqEg9PT2SpP379+vRRx9Vb2+vGhoalJmZqbq6uoQUDgBYuZTo\nShbUk2RkZMTRebxstId+2UO/7KFf9sTbr8LCwkVv5x20AGAAwh4ADEDYA4ABCHsAMABhDwAGIOwB\nwACEPQAYgLAHAAMQ9gBgAMIeAAxA2AOAAQh7ADAAYQ8ABiDsAcAAhD0AGICwBwADEPYAYADCHgAM\nQNgDgAEIewAwAGEPAAYg7AHAAIQ9ABiAsAcAAxD2AGAAwh4ADEDYA4ABCHsAMABhDwAGIOwBwACE\nPQAYgLAHAAMQ9gBgAMIeAAxA2AOAAQh7ADAAYQ8ABiDsAcAAhD0AGICwBwADEPYAYADCHgAMkO70\nxImJCZ0+fVpfffWVtm/frqamJmVnZy8Yd+TIEWVlZSk1NVVpaWlqa2uLq2AAgH2Ow767u1s/+tGP\ndODAAXV3d6u7u1s///nPFx374osvKicnx3GRAID4OF7G8fv9qq6uliQ9/vjjun79esyx0WjU6TQA\ngARw/Mz+zp07ysvLkyTl5ubqzp07i45LSUlRa2urUlNT9eSTT+rJJ590OiUAwKElw761tVXhcHjB\n7QcPHpx3nJKSsuRjbNu2TWNjY2ptbdXOnTu1e/fuBeMCgYACgcDssc/nk8vlWvYXWMzmzZsdn2si\n+mUP/bKHftmTiH51dXXN/uzxeOTxeJYO+5aWlpj35ebmKhwOKy8vT19//bVyc3MXHbdt2zZJ0tat\nW1VZWanBwcFFw/5+QXONj48vVV5MLpfL8bkmol/20C976Jc98fbL5XLJ5/MtuN3xmr3X69V7770n\nSbpy5YoqKioWjPn222/1zTffSJLu3bunjz/+WLt27XI6JQDAIcdr9jU1NTp9+rT+9re/zW69lKRQ\nKKQ333xTJ06cUDgc1ssvvyxJikQieuyxx1ReXp6YygEAK5YSfYC3yoyMjDg6j5eN9tAve+iXPfTL\nnnj7VVhYuOjtvIMWAAxA2AOAAQh7ADAAYQ8ABiDsAcAAhD0AGICwBwADEPYAYADCHgAMQNgDgAEI\newAwAGEPAAYg7AHAAIQ9ABiAsAcAAxD2AGAAwh4ADEDYA4ABCHsAMABhDwAGIOwBwACEPQAYgLAH\nAAMQ9gBgAMIeAAxA2AOAAQh7ADAAYQ8ABiDsAcAAhD0AGICwBwADEPYAYADCHgAMQNgDgAEIewAw\nAGEPAAYg7AHAAIQ9ABiAsAcAA6Qnu4BECIVCeruzU1/29yt9YkLpaWmanpnRdE6OtpeV6anaWlmW\nlewyAWCBtcqvlGg0Gk1AvatiZGRk2TGf3rih7oYGvTA0pKJF7h+W1FFSopqzZ/VweXnCa9wIXC6X\nxsfHk13GukG/7KFfsa1GfhUWFi56u+NlnA8//FDPPfecfvazn+nzzz+POa6vr0+NjY169tln1d3d\n7XS6RX1644beamrSmRiNkqQiSWeGhvTnxkb1X7+e0PkBwKm1zi/HYb9r1y49//zzeuSRR2KOiUQi\nunjxopqbm/XKK6/ogw8+0PDwsNMp5wmFQupuaNDrn3227C+RKun3AwO6Vl+vf7z7bkLmBwCnkpFf\njsN+586dMV8u3Dc4OKiCggLt2LFD6enpqqqqkt/vdzrlPG93duqFoaEV/wKpklpGRnSltVWTk5MJ\nqQEAnEhGfq3qbpxQKKT8/PzZY8uyFAqFEvLYX/b3x3zps5QjN2+q+/z5hNQAAE4kI7+W3I3T2tqq\ncDi84PaDBw/K6/U6mjBR0icmHJ3nljSaoFcXAOBEMvJrybBvaWlx9KD3WZalYDA4exwMBmNuIQoE\nAgoEArPHPp9PLpcr5mOnp6U5ritzamrJxzbN5s2b6YcN9Mse+rXQaudXV1fX7M8ej0cej2d199mX\nlJRodHRUt2/flmVZunr1qo4dO7bo2PsFzbXUdq3pmRnHdd3btImtYHOwNc4e+mUP/VpoNfPL5XLJ\n5/MtuN3xmv21a9dUV1engYEBtbW16dSpU5K+W6dva2uTJKWlpam2tlYnT55UU1OTfvzjH6uoyMlK\n1ULTOTmOzrspqSDJS1AAzJaM/HL8zL6yslKVlZULbrcsSydOnJg93rNnj/bs2eN0mpi2l5Vp+PJl\nW3/kiEp63e1Ww+HDCa8HAFYqGfm1bj8b56naWnWUlCiywvERSb8tLFR1S4uysrJWszQAWFIy8mvd\nhr1lWao5e1ZHS0uXbVhE0tHSUlWeO6e9TzyxFuUBQEzJyK91G/aS9HB5uQ50dOjUEm/uiko6WVio\nAx0dKquoWLviAGAJa51f6zrsJamsokKe9nYdc7t183v33ZTU6Hbrh+3tBD2AB85a5te6/9TL+yYn\nJ9V9/rxG/X5lTk3p3qZNKvB6VXPokLZs2bKKVa5/bI2zh37ZQ7+Wl8j8ivUxNhsm7Ofi4rKHftlD\nv+yhX/bE2691GfYAgMRY92v2i5n7VmEsj37ZQ7/soV/2rFa/NmTYAwDmI+wBwAAbMuy//4FqWBr9\nsod+2UO/7FmtfvEHWgAwwIZ8Zg8AmI+wBwADrOqXl6yVDz/8UG+99Za++OILtbW16Qc/+MGi4/r6\n+nTp0iVFIhHt27dPNTU1a1zpg2FiYkKnT5/WV199pe3bt6upqUnZ2dkLxh05ckRZWVlKTU1VWlra\n7PcUmGIl10tnZ6f6+vqUkZGh+vp6FRcXJ6HSB8Ny/QoEAuro6NBDDz0kSdq7d6+efvrpZJSadOfO\nnVNvb6+2bt2q3/3ud4uOSfi1Fd0AhoeHo1988UX0xRdfjA4NDS06ZmZmJnr06NHorVu3olNTU9Hn\nn38++t///neNK30w/PGPf4x2d3dHo9Fo9C9/+Uv0T3/606Lj6uvro+Pj42tZ2gNjJdfLP//5z+ip\nU6ei0Wg0OjAwEG1ubk5GqQ+ElfTrX//6V7S9vT1JFT5Y/v3vf0c///zz6HPPPbfo/atxbW2IZZyd\nO3fGfIvwfYODgyooKNCOHTuUnp6uqqoq+Q394nG/36/q6mpJ0uOPP67r16/HHBs19O/3K7le5vbR\n7Xbr7t27CofDySg36Vb678vU6+n7du/eveir6ftW49raEGG/EqFQSPn5+bPHlmUpFAolsaLkuXPn\njvLy8iRJubm5unPnzqLjUlJS1Nraqt/85je6fPnyWpaYdCu5Xr4/Jj8/39hraiX9SklJ0cDAgI4f\nP662tjYNDw+vdZnrxmpcW+tmzb61tXXR/9kOHjwoL98pu8BS/ZorJSVlycfYtm2bxsbG1Nraqp07\nd2r37t0Jr3U945nqyhUXF+uNN95QRkaGent79dJLL+nVV19NdlkPrERfW+sm7FtaWuI637IsBYPB\n2eNgMCjLsuIt64G1VL9yc3MVDoeVl5enr7/+Wrm5uYuO27ZtmyRp69atqqys1ODgoDFhv5LrxbRr\naikr6cXcr9Pbs2ePLly4oImJCeU4/PLtjWw1ri1jlnFKSko0Ojqq27dva3p6WlevXjX2FYHX69V7\n770nSbpy5YoqFvlihG+//VbffPONJOnevXv6+OOPtWvXrrUsM6lWcr14vV69//77kqSBgQFlZ2fP\nLo+ZZiX9CofDs89WBwcHJYmgj2E1rq0N8Q7aa9eu6Q9/+IPGxsa0ZcsWFRcXq7m5WaFQSG+++aZO\nnDghSert7Z23NeynP/1pkitPjlhbL+f269atW3r55ZclSZFIRI899phx/Vrseunp6ZEk7d+/X5J0\n8eJF9fX1KTMzU3V1dTG3/ZpguX6988476unpUWpqqjIyMvSLX/xCpaWlSa46Oc6cOaNPPvlEY2Nj\nysvL0zPPPKOZmRlJq3dtbYiwBwAszZhlHAAwGWEPAAYg7AHAAIQ9ABiAsAcAAxD2AGAAwh4ADEDY\nA4AB/gcNkeLUhiCmDAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x112c53610>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# generate training data\n",
    "X = np.array([[-1,-1],\n",
    "              [-1,1],\n",
    "              [1,-1],\n",
    "              [1,1]])\n",
    "\n",
    "y = np.array([[0],\n",
    "              [0],\n",
    "              [0],\n",
    "              [1]])\n",
    "\n",
    "# plot the points\n",
    "plt.plot(X[0][0],X[0][1], 'ro', markersize=15)\n",
    "plt.plot(X[1][0],X[1][1], 'ro', markersize=15)\n",
    "plt.plot(X[2][0],X[2][1], 'ro', markersize=15)\n",
    "plt.plot(X[3][0],X[3][1], 'bo', markersize=15)\n",
    "plt.axis([-1.1,1.1,-1.1,1.1])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we are trying to draw a decision boundary to seperate the blue from the red circles. \n",
    "\n",
    "Lets train the network and see how it does. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 321,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEECAYAAADOJIhPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnWtsXNX57p89Y3s8nhnbGZckJGn+DbdzgqvSICeVGkpo\nmvCXWqlN1WKJ9CIO6CAFSgsVPZD0oKbHStM2Tbi0XFoBh150jmo+EClfWiG1gARFEHBE61ByUoog\nhNjEju2Z7evMrPNhewZf5rL3XmvN3vZ6flKk2N7e+/U7M+t513rf9S5LCCFACCHESCJBG0AIISQ4\nKAKEEGIwFAFCCDEYigAhhBgMRYAQQgyGIkAIIQbTIHuDRx55BH19fWhtbcXhw4fLXvPkk0/ixIkT\niMViuO2227BhwwbZxxJCCFGA9Ezg85//PPbt21fx56+//joGBgbw0EMP4dZbb8Xjjz/u6r79/f2y\nptWFpWDnUrARoJ2qoZ1qWa52SovAxo0bkUgkKv78+PHj2LZtGwDg8ssvh23bGBkZqXnf5erwIFgK\nNgK0UzW0Uy3L1U7tOYHh4WF0dHSUvu7o6MDw8LDuxxJCCHFBXRLD7ExBCCHhxFLRO2hwcBA/+9nP\nyiaGf/Ob36CzsxNbt24FANx5553Yv38/2tvb513X398/bxrT3d0taxYhhBhJb29v6f+dnZ3o7Oys\neK10dVAturq68Oc//xlbt27FqVOnkEgkFgkAUN7Qs2fP6jZPmlQqhUwmE7QZVVkKNgK0UzW0Uy1L\nxc41a9Z4CqKlReCBBx7Am2++ibGxMezZswc33HAD8vk8AGDnzp24+uqr0dfXhzvuuAPNzc3Ys2eP\n7CMJIYQoQloE7rzzzprX3HLLLbKPIYQQogHuGCaEEIOhCBBCiMFQBAghxGAoAoQQYjAUAUIIMRiK\nACGEGAxFgBBCDIYiQAghBkMRIIQQg6EIEEKIwVAECCHEYCgChBBiMBQBQggxGIoAIYQYDEWAEEIM\nhiJACCEGQxEghBCDoQgQQojBUAQIIcRgKAKEEGIwFAFCCDEYigAhhBgMRYAQQgyGIkAIIQZDESCE\nEIOhCBBCiMFQBAghxGAoAoQQYjAUAUIIMRiKACGEGAxFgBBCDIYiQAghBkMRIIQQg6EIEEKIwVAE\nCCHEYCgChBBiMBQBQggxGIoAIYQYDEWAEEIMhiJACCEGQxEghBCDoQgQQojBUAQIIcRgKAKEEGIw\nFAFCCDEYigAhhBgMRYAQQgymQfYGJ06cwFNPPYVCoYDt27dj165d837e39+Pn//851i1ahUA4DOf\n+Qy+9rWvyT6WEEKIAqREoFAo4IknnsB9992HdDqNvXv3oqurC+vWrZt33ZVXXol77rlHylBCCCHq\nkVoOOn36NFavXo2VK1eioaEBW7duxfHjxxddJ4SQeQwhhBBNSInA8PAwOjo6Sl+n02kMDw/Pu8ay\nLJw6dQo/+MEPcPDgQZw5c0bmkYQQQhQinROoxYYNG/Doo48iFouhr68Phw4dwoMPPrjouv7+fvT3\n95e+7u7uRiqV0m2eNE1NTaG3cynYCNBO1dBOtSwVOwGgt7e39P/Ozk50dnZWvFZKBNLpNIaGhkpf\nDw0NIZ1Oz7smHo+X/r9p0yY8/vjjyGazSCaT864rZ2gmk5Exry6kUqnQ27kUbARop2pop1qWkp3d\n3d2ur5daDrr00ktx7tw5DA4OIpfL4aWXXkJXV9e8a0ZGRko5gdOnTwPAIgEghBASDFIzgWg0iptv\nvhkHDhwolYiuW7cOzz77LABg586dePnll/Hss88iEokgFovhe9/7nhLDCSGEyGOJEJfunD17NmgT\narIUpohLwUaAdqqGdqplqdi5Zs0aT9dzxzAhhBgMRYAQQgyGIkAIIQZDESCEEIOhCBBCiMFQBAgh\nxGAoAoQQYjAUAUIIMRiKACGEGAxFgBBCDIYiQAghBkMRIIQQg6EIEEKIwVAECCHEYCgChBBiMBQB\nQggxGIoAIYQYDEWAEEIMhiJACCEGQxEghBCDoQgQQojBUAQIIcRgKAKEEGIwFAFCCDEYigAhhBgM\nRYAQQgyGIkAIIQZDESCEEIOhCBBCiMFQBAghxGAoAoQQYjAUAUIIMRiKACGEGAxFgBBCDIYiQAgh\nBkMRIIQQg6EIEEKIwVAECCHEYCgChBBiMBQBQggxGIoAIYQYDEWAEEIMhiJACCEGQxEghBCDoQgQ\nQojBUAQIIcRgKAKEEGIwDbI3OHHiBJ566ikUCgVs374du3btWnTNk08+iRMnTiAWi+G2227Dhg0b\nZB9LCCFEAVIzgUKhgCeeeAL79u3DkSNH8OKLL+LMmTPzrnn99dcxMDCAhx56CLfeeisef/xxKYMJ\nIYSoQ0oETp8+jdWrV2PlypVoaGjA1q1bcfz48XnXHD9+HNu2bQMAXH755bBtGyMjIzKPJYQQogip\n5aDh4WF0dHSUvk6n0zh9+nTVazo6OjA8PIz29naZR0sjBs8CH5ypftGqtbBWr9VnQz4PnDwBFPKV\nL4pGgSs/DSsS9fcMITDz99cgRiWF9+KPw1p5sdw9qiByM5jpexnCtitfFG0AOjfBsix/zxDC8Xdu\nxqeVDvkrOoF4wvfviw/PAWfflbIBjU3Axqt8+8IN4sw7wNCgtvsXmYnHISYmtD9HllDY2d4B6z8u\nVXpL6ZyAG4QQNa/p7+9Hf39/6evu7m6kUiltNmUf+x3EyDCsthVlfy7GRmAlUkju/VnV+zQ1Nfm2\nM/f/TsJ+4gii/+WTla/55xtI3ncEDRuu8PWMwvCHGDv0QzR88mpfvw8AYmQYkY+tROL7/8v3PWox\n84/XMf7ozxC9bGPFa3InTyB14DFE16739Yz8B2eQeeynaLjy037NhBg+j+n/uASpPff6vof9+GEU\nPjwHqz3t+x65f7yO1iO/ReRjqypeI/PeBICx//0ArEQKVrzF9z3cMBOJIFIoaH2GCsJgZ8PGq9D8\nydrv397e3tL/Ozs70dnZWfmeMgal02kMDQ2Vvh4aGkI6nfZ8TSVDM5mMjHlVyV8YQuSG/wZcdmXZ\nn4u330L+//y6pg2pVMq3nWLgHMSGKyD27K180ZH7MH7uA1gf8xeFiw/OIrJ6bfVn1LrHP9/AzLH/\nq/X1KAyeQ8N//RQK//3uyhf99H/AHvgAVmt54a6FOHcWuPjjcr5441XkX/izlC/yF4YQ2fVNYONV\nvu+BH38P2XMfwIpVHqBl3psAUBgdQeT2/wmkP+b7Hm5okbSzXoTBzhkAMy7GpO7ubtf3lMoJXHrp\npTh37hwGBweRy+Xw0ksvoaura941XV1deOGFFwAAp06dQiKRCHwpCACQzQDJ1so/T7YC2TGtJgh7\nDFY1GwBYyVYIW+KNZ2cQSbX5/31g1hea3/zZDKzWGnYmWwFb4jWxa7zmbki2Qsi+L1TYkZL0RQ2E\nEM79k/pm4yQcSM0EotEobr75Zhw4cKBUIrpu3To8++yzAICdO3fi6quvRl9fH+644w40Nzdjz549\nSgyXxs4AiWoikHKu0Uk2U/tDlkhJDcAim3GExPcdUB9fuBHERMr5e3w+wvGF5KCWTEFkxnzbAMB5\nPRNydsj6oiZTk0AkCqsppusJJCRI5wQ2bdqETZs2zfvezp075319yy23yD5GKaKQByZsIFEluRdP\nADPTELkZWA2NegyxXQwGyZRk9DsGKyUZdSacmYAQQl8iMpuBtWZd9Wtkxcgeqy78bki0opAdhb80\nvcIIW7cw2y4CFLIsMHPH8LgNNLdUrbixLAtoSQJ2Vp8ddZgJYHYmIIPV2Ag0NABTGisj7AwitexU\n4AvpgS3eAkxNQuRy/n5/ahKIROQjbFlf1ELBbIUsDcwUgazLSCyR0poXEFkXkWlS0oasgpkA4Pgi\no9cXVrJWTkDeF9LLMJEIrJYkMO5zAHbzmrtB1he1yI7J5y3IksBQEXCZmNOdELVrr1GHIjEMzCZl\n9UaetRLDVrIVQiY/YmeUCKKVavP/vlC1zKL5vemIMkXABMwUATdr8YD8enwt3ERbCbkPu1CwHARg\nNvLULYgufCHzeiha4rBSEq+J2wCklg2JVgid7023nxGy5DFSBNxWiRQrMLThOjEsNxOwFMwErERK\nbkZSi6yLKF1WiBRF4ZbErEhkx2CpGFx1i7KK/AlZEhgpAq6rRDRWYAgh6pQYVjSt1zjoiHwemJpw\n1tqrkZB8PbI1yoJdEklJ7BVQtRwk64tacCZgDGaKgOvEsMYNY1OTQMSqXSUSbwFy0xAzPvvduImw\n3aDTF3YGaEnAitR4O86Ksps2JAtxRFfN5icrKZETUJCcBlCnxDBFwAQMFQG3iWGNU26Xu0Yty5qN\n+rx/4F1H2G6Q3a1bDZeVKFZDo9M4bWLc+zOmJoCGBliNTT4MXGBHqs2/LxTlBObuY9GBslwSCT1G\nioBwUZUDKKjMqYaXJKXfJSE7A7Qka0fYbtApiF594ec1UVj3LpUYVpWX0L2PpdaOerJsMFIEXA8I\nOvcJ2B7qsP3mJhTu+tSaGPbSS8dvaaRKX0gEByKbgaVqcNVZJsrlIGMwUwTcDgg6E8PZjPsqEb9l\noip3fepMDHupmPG5NKbSF5EQzAQA+PeFG5gYNgYzRcDtbkidnUQ97Mi0/FajqNz1qdMXHmYCls8u\nnio3PzmbxfzmBML/mojcDDAz7RQlkGWPcSJQKs10MyVPpIBxG0LHQRJe6rB9Lks5A5/CmYC2/IiH\npQe/MxKFde8y+wSU5iaSmvaxzNqo89QyEh6ME4GPqkRqdwa1olEgFnc6jqrGy3RbJiegav05Fgdy\nOf+lqtWoR2JYRQfRWaxkKzCe9VyqKnIzQE5hhK1rrwCXgozCPBHwGonpWgv3PBMIOPq1LG1tNNxW\nawEIx0ygoQFoinkPDrKz1VqqIuwwvDfJksc8EfCamNNUISTsMddVIlbSZ2WO6ohOV/tiLzt5fZeI\nKtqkNdcOr75QcaLYQht0JIYVzppI+DFPBDIeE3O6yvC8bBrymQBU3glSV3LYS2LYZ5JcuGlQ5wU/\nvlBcdun4Qv17U2kuiYQe40RA2B5KM1GMwjUMfF4GBJkSUZUfZl3JYc++8FkiGrQvFPUuKqGrlYeq\nXc1kSWCcCHgeDHQtgSy1xDD0dFUVQgDjWSDhsrWFlC8URuE+fCFsxRG2LlHm0ZJGYZ4IeF3v1PBB\n81yH3ZIEJmznbGQvqN71qaNp2YQNNDa5P8dZKkmueDnI6wxR9ZGNWnM0FAFTME8EPM8ENOQE7Kyn\nKhErGgWa487ZyC7xHGG7IaHhdDGvA06sGSjkIaanXP+KmJkBcjOOD1XhOzGsWATGs8r3sXiq1iJL\nHgNFwHti2HfveFU2zNrhKQqfsIGmmPsIW4cNbvBYMeOUqnoUZtuZESnd/OTHF16LEmpQCg5U72NR\ndQ4yWRIYJwJ+EsNaol+vkZbXyFPDlF7LDlU/S1Zey0R1LG/4qNHXEmHrWBLiPgGjME4E/CWGVUe/\nPiItr60KdOz61LBD1VdXTa9RuIZEp6+uqjraM2uZnamdsZBwY54IeB0cdSSGXZ5xPBfP1Sg6ojkd\nO1Rtc2YCWl4TxcIsCgUn96TiICKyJDBPBLxWicwmhv0caVgRP1G6x5YNXnYkuyYMiWF4X5ZSvmkO\n8OcLW/GuZWhYopuwgea4k28gRmCUCPipErFiMSBiOWcCqyI7Bng999frtN/PM2qRSPgrVa2GnamP\nL5TPirzZIAp5J8JWPiNRvByUUS9UJNwYJQK+q0RUR8B+lifCkBiORJ2zbW2F1Sh+Bh2vSyAa1uKt\nWAwAIKZclqqOa4qwVedpVPc3IqHHLBHwOzAqrsDwUyXiuYmcrl2fipuW+aqY8boer6vaxYsvdG3A\nUp2nYcsI4zBLBPwOjKpbKPupww7BTACAnkHHoy+shLczfr2WBbvGiy90RdiJVqW9rZxcEpeDTMIs\nEfA5MCrvmeNHjDyWiGrb9all+cHPTCDgnADgzReaRNnSMhOgCJiEUSLgu0okpTj5Vo8dw4p3pxbx\ne8ZvRfzUpHtODOuJwr34QkuFEqA+Maz63AUSeowSAd8RocKcgO867NmIz3Wpqo6NSXPsUIGYmgIK\nwjmly6sNASeGS3a4Xg4KwWzEDewgahxmiYDfwUDlhjGfddhWYxMQjTpnJLtB66CjKPL0W60VTwCT\nExD52qWqopB3fJ5I+DSyCgkPnUQ152hU7WNxNjIyMWwSZomA3/VOlZ1EZQYDl5GnmPYZYbuyIXhf\nWJEI0JJwuqTWwraB5hanvFU1nhPDGnICTbP7WDx0Va0KD5k3DqNEwG+ViNLTxfzkA4q4nfrPip3S\nrpmz+D7vuBwyFTNuhVnXjAjwtBQjVJ8qNs8OlcLMvkGmYZQI+H6Dq45+/X7Ikq1OwrfmMzR+kBUm\nIoVMJYrbCiGNde+ezjvWeW6vysN+eKCMcRgmAn6Xg9R9yGTqsK2ky/p4nVN6lRvnZPobuT3ZKyy+\n0LkT188pZ2UQQuidOZFQYpYIhCExLBP9uhx0pCLsWij1hf8Bx+3eDT8dW13jxRcaI2xl+1imJoFI\n1MkzEGMwRgSkqkTiCWBm2jkbWBaZyNTtzmUdHUSLqOyqKpskd5Uf0XhKlsu1eO0RtiphZnmokRgj\nAk4DL39VIpZlOXX9totqlFrUYSagc9en1dgINDS4L1Wthsyg49YXOgffeAswPQmRy1W/bmoSiET0\nRdiqluiYDzASc0RANlmqKCEqtXPUrQ26KzzcJqhrIFWT7toXGhPDkYgTHIzXGIDr8XqoyFmxMshI\nDBIByehYVbQlsRzkujxTd623ql2qsr5wsxSjq4dSETfvC92vh6Jd3CLL5nEmYo4IyH4QVXUSldon\n4HINWmcyFFDXOkLWF25eD91LHG58obk9s6WqkyjPEjASY0RAdmBUVoEhnRgOfibg65D1ckjvE3CZ\nGNY9E6jhC+0RtjJRZmLYRIwRAed8V5mcgPwSiBCiTolh3WvQ8oOOyOed5HLcZ08ft0tSuprHzeIs\nS9WIwnVX3YRgeY4sXcwRAdmIUEXyTbZKJN4C5Kads5Kroft0KBW+sDNAS9JJrvqywRn4qpWqSouu\nKztcLNEtqcQwRcA0Gvz+Yjabxf3334/z58/joosuwl133YVEmRr822+/HfF4HJFIBNFoFAcPHpQy\n2DfZDNCxyv/vJ1LAB2fkbJCMCC3L+qiLZ3tH2WtEPg9MTzqCoYtEKzAQsC8aGoHGJmBi3GkmV46p\nCaChwenAqgs3uYlsBlh5sT4b4i2lfSxWQ6Pv24hsBhGNsyYSTnyLwNGjR/GpT30KX/nKV3D06FEc\nPXoU3/jGN8peu3//fiSTHvvnK0bYGURkBp1kKwqyU24VScriklAFEYCdAeIJ/xG2G5Ip4F+SvvBz\nwPxCissglUSgHnXvyRQw8H71a+wMkLxCmwnz9rG0rfB/I24WMxLfI8Xx48exbds2AMB1112HV199\nteK1qnqdSyE7IKjoH+TnFK2F1MpN1OGDrCQxrKISpdZSTB2WN9z4QmQz+nZwF1HR5JD7BIzE90xg\ndHQU7e3tAIC2tjaMjo6Wvc6yLPT09CASiWDHjh3YsWOH30fKITs4qkgMZxUceF6rTLRe0a9sYlhF\nxUytA27C4ot6RNgqDvthYthIqopAT08PRkZGFn3/xhtvnPd1tb71PT09WLFiBcbGxtDT04O1a9di\n48aNi67r7+9Hf39/6evu7m6kUurekKN2BslVaxDxec/C6jXI2JlFNjU1Nbm2c2pmGvl0B1ok/q7x\ndAeiuSnEKtxjOj+D6fY0knN+7sVGN+RXrUF2PCt1z8ncNET6Y4hL2GmvSKMxN4OmKr6YWdGBhMK/\nHZhvZ37VxbBr+GLUziCx6mJEFdsxF3tFGo35+b7w4k+Rm8HozDRSK1dpOYeiGqrfn7pYKnYCQG9v\nb+n/nZ2d6OzsrHhtVRG47777Kv6sra0NIyMjaG9vx4ULF9DW1lb2uhUrnDXK1tZWbNmyBadPny4r\nAuUMzWTUdKsUQkBkxpBFBJbPewphQdhZjI2OzltvT6VSru0sDH0INDVL/V2FpmbMnP8Q0xXuUfhw\nAGiOz3uGFxvdICwLIjMq93cMnQcSSeQk7CzE4sidH8RUJV+c/xCIyfm7HHPtFFYEhRq+EJkx2BLv\nPTcUYnHkPpzvCy/+FCPDQEsS2ayC/lgeUf3+1MVSsrO7u9v19b5zAl1dXXjuuecAAM8//zw2b968\n6JqpqSlMTDiNxiYnJ/HGG29g/fr1fh/pn1KViP/KCSsaBWJxpxOpX1RMt93kBHSvP8fiQC5Xu1S1\nGlmFieFKyO4NcUNLChjPVsx7idwMkJvWW60FyO8V4G5hY/GdE9i1axfuv/9+/PWvfy2ViALA8PAw\nfv3rX2Pv3r0YGRnBL37xCwBAoVDANddcg6uuukqN5V5QtTZcXP/1e69sBthwuZwNiRTw/rvVn6E7\nGWpZH7XRqFSlVAPZai0Ajg3VynazY8CqtXLPqIHV0OCc5TxhOxU6i2yY3Q+he4klmZJr6sfdwsbi\nWwSSyWTZ5aJ0Oo29e/cCAFatWoVDhw75t04VqhJzxQqhVWt8/bqwx6TrsK1kqnqpqq25Jr1IrVLV\nWqg4c7dW9JvNAJfUYWAr+qKcCNQrwk6kgHMSezfqMWsiocSMHcMZRaVvsmV4Knby1tgdKtWqWqEd\nNVEwONY649fpIBqwL+q0C9fxhf/3ptB5BjIJNUaIgLAVlGai2MpZZsqtYEBwUyJajw+zbMmsMl/U\nKBEN2hcqZjxuqOWLWvBAGWMxQgSUDQayZwosl8Qw5LqqCiGA8SyQkNxFXssXKpLPLqjmC2HXKcKW\nFWXuFjYWM0RA1XqnxAdN5GaAGQVVIi1JYMJ2zkwuR72agCUldlBP2EBjk1SfGwC1Rble6/HJKv2D\n6hVhywYoupsOktBihggomwlI5ATsrJIqESsaBZrjzpnJC1AWYbsh0eo/8lQ1MMaagUIeYnpq0Y/E\nzAyQyzm+0k21AbheEXZitlS1UPD169pPYCOhxRARUJcYrtk7XrcNs3aUjcInbKApJh9hy9jgBkUR\nulOqWkGYZw+Yr8vu12q+UFWUUINScOB3H0uW1UGmYoQIqEwMS0W/qiKtSpFnHZN7bs/4LYtqX5R7\nTeqZ6KzSP6iuEbbMkhD3CRiLESKgNjHsN/pVGGklKyzF1LMBmMQOVWGPqeuqWSkKr+MBKVU7idYp\nUQ9AcnZWnyQ6CR9miICqwVEmMazw8PeK1Sj1jOZkOomqHKAriVE9BbGaL+r5mvgUZlEoODkmioCR\nmCECqiofZhPDvs5HUDkoFVs2LEBphF2LMCSGUXlZyhHdEPiijhG27yW6CRtojjt5BWIcy14EnCqR\nGSVVIlYsBkQs56xgr2THgFQdlkBUPaMWiUT1UtVq2Jn6+KJus6LyNohCvr4Rtt/lIBWnvJEly7IX\nAeVVIn4jYJWJyjAkhiNRIJ4AbB/VKKp9UXE5qD6CaMViAAAxtaBUdbzOEbbfPA07iBrN8hcB1QOj\nzwoMlVUiTvuKCgNfPSs8fJ5mpbRiptJ6fD1nAkB5X9S7FYPfPA1bRhjN8hcB1QNjhfX4mqisww7B\nTACA/0Eno84XVqK1rCAqOcrTC+V8Ue8IO9Hqq7eV09qCMwFTWf4ioHhg9N0zR6UYVSgRrfuuT6nl\nB5UzgTIDXyCzogW+qLMoWzIzAe4RMJZlLwLKWyunfCbf6rFjuE67U4tYfndQ23XwRZ174ZTzRd3a\nehfxmxiuU6M9Ek6WvQgoXxv2kRMo1WGXO3TED7MR36JS1XpuTJpjhxfE9BRQEM5pXKpsCDgxXLJj\n0XJQEHmJgGdmZMmx/EVA9WDgZ8OY4jpsq7EJiEads5PnEsig4zHynF16UFatFU8AkxMQ+Y9KVUUh\n7/g8kVDzDDckynQSDShH43Ufi6jXmQcklCx/EVC93umnk6iOwWBB5Kk8wnZlgx9fqF16sCIRoCXh\ndE8tYttAc4tTxlovKiaG65gTaJrdx1Kmq2pV2EHUaJa9CKhqHlfE1+liKvMBRRZO/VVH2C6oWKpa\nDR0VMwuFud4zIqDsUkwgEbavIKW+uSQSLpa9CCh/g/uKfjUMfMlWJxFcekYAH2QfiUihoxJlYYVQ\nAAeklD3vOIhze/0c9sN9AkZjgAioXg7y/iFzevqo/ZBZyQX18fVsmFbEz8Y5Hf2NFp7sFUS1Szlf\nBLETt9opZ2UQQgQzcyKhYfmLQBgSwzqi3wWDjpYIuxYh8cXCvRuBnJJVzhcBRNie97FMTQKRiJNP\nIEayrEVAS5VIPAHMTDtnBrtFR5S+cOdyPTuIFvHTVVVHlL5wAA7BWnxgEbZXYa53KS0JHctaBJwG\nXmqrRCzLcur97Wzti4vUYSYQxK5Pq7ERaGhYXKpaDR0VMwt9EcTgG28Bpichcjnn66AibK9LdNwt\nbDzLWwR0JUs9JkS17BxdaENQFR4LE9Q10NLnf5EvAkgMRyJOcDA+OwAH+Xp4yVmxMsh4lrkIaIpy\nvEZbGpaDFpVnBpEYBrzvUtXli3n5kQCqcoD574ugXg+Pu7hFVn3RAllaLG8R0PVB9NpJVMs+gdZF\nieFABj6vrSN0+WJefiQEA3AAsxGg2FXVw3uTLSOMZ1mLgK6B0XMFhrbEcPAzgaqHrJdDx6CzUIiC\nWueeMysKLML2LMrcI2A6y1oEnPNddeQE3C+BCCHqlBgOag3a/aAj8nlgcsKpsFLJwiWpgCpenGWp\nsY9sCFiIXMFTxYxneYuArtOlvCTfdFWJxFuA3LRzhjIQ2PKDJ1/YGaAl6SRRldrgDHxCCH2i68qO\nOUt0SyoxzJmAySxzEdA0MHpJDGuKCC3LKnXxFPk8MD3pCEO9Kdc9sxK6fNHQCDQ2ARPjzkyjocHp\ntFpv5voiqGWWeIunfSzOCWycCZjMshYBXTtHF7VsqIbOwaAoRnYGiCfUR9hu8LIGrdsXdibYA1Lm\n+iKg5SDP+1iYGDaehqAN0IquQcdL/yCVp2gtpJibiEYD+yBbiRQKbgVRqy9ml2IshMIXIptBJKgI\n20uTQ56TQJjQAAAKZklEQVQqZjzLWwR0RTleEsM6DzwvlolGouGIfmsgMhorZkoH3Fjh8EWQEbaX\nw36YGDae5S0CYdgxrDFBWGxfbEUiwX2QvSaGdfmieMavFanvub5zmeuLIHfiunxNRG4GmJkOJpdE\nQsOyFYFSlYiOKXkiBYzbztnBtdBZqTK7LCUikeBOhvLSsEynL4pRuGUFF4HP9UWA9fcLd1BXJDtb\nrVXHg4hI+Fi+ieGpYpVIo/JbW9EoEIs7HUproXMTV3HQCbITZCwO5HIflapWQ6cvSonhAH3RkgLG\ns44vcgFG2G73CnApiGA5i4DuSMztWrj2mUAm0E6QVjHydrEGrbW1RfH1CPCAFKuhwTnj+fxAsBG2\nl/cmk8LGs4xFQPNg4LJCyGkfoGsdfLZlQ1C9coq4rZbKatrBXbShNBMI2BcDZ4KNsL28HpwJGM+y\nzQlo30HrtgxP55R7NgEoIhFEgvwwh8AXVqoVhewYYFmB+0Kcez/Q2nsr1eqqbFfYAXVbJaFi2YqA\nsDWWZqIYhbuNtupQIhrkh9ltcli7L8aCTQwDzrMHzgZ7WlfCZcVW0LMmEgqWrQhoXyd32zqiHonh\naDTQQafYVbXaCrgQAhjPAomkHiNKQhTgPgHM+mLgfVir1gZmg2tRtjNAqk2/PSTULN+cgK4OokVc\nfNC012G3JJ0Kpcxo8NFvrchzYhxobHL6/OhgbguNoJfGBs4Gn5dgYpi4ZPmKgPbEsIspt+Y6bCsa\nBZrjswlXTRG2GxIucgK62xPEmoFCHsjlHJ8ERSIFjI0EK8qJ2VLVGvtYtBx7SpYcy1gE9CeGa27I\nqUdUmmwF4i36Imy3NtTKj2j2hVOq2gokU8Fufir+jQEOrsXgQIzXaCIX9KyJhALfOYG//e1vePrp\np/H+++/j4MGDuOSSS8ped+LECTz11FMoFArYvn07du3a5dtYLwg7g4j2xHCt6LcO9fuJFOBm57JG\nrGQKhZozgTr5Qgi9z6jF7N8YeNVNIgWRGQOSVdb8A9xfQsKD75nA+vXrcffdd+PKK6+seE2hUMAT\nTzyBffv24ciRI3jxxRdx5swZv4/0Rl0Sw7WiX815CcCJ5IJe13WxQ1XY+vZLlJidCQRJqSIt6B79\nxV5K1bDZQZRIiMDatWuxZs2aqtecPn0aq1evxsqVK9HQ0ICtW7fi+PHjfh/pDd0bqNwkhutw+LuV\nSAU+8LnaoVqvmUDQg1rxbwz6NUmkIDKjFX8sCnlg3A7eXyRwtOYEhoeH0dHRUfo6nU5jeHhY5yM/\nQvduyNnEsKi2/FCPXu3JVPAnQ7lKkuv3hZVMBZ/oLL4WAQ+uVrK6CGDcBmJxJ39AjKZqTqCnpwcj\nIyOLvn/jjTeiq6tLm1FF8r/s8feLQgB5vVUiViwGRKKwf74X+XyFNflzZ2Bd+5/abADg1HkHvQ6e\nSABTE9Vfr7Pvwrpecz4o1RZ4fgTJ1tl9G0HPSNoweeyPKLz4l/I/n54KfrZCQoElqoaytfnxj3+M\nb33rW2UTw6dOncLTTz+NH/7whwCAZ555BpZllU0O9/f3o7+/v/R1d3e3jFmEEGIsvb29pf93dnai\ns7Oz4rVal4MuvfRSnDt3DoODg8jlcnjppZcqziA6OzvR3d1d+jf3jwgzS8HOpWAjQDtVQzvVspTs\nnDuWVhMAQEIEXnnlFezZswenTp3CwYMH8ZOf/ASAkwc4ePAgACAajeLmm2/GgQMHcNddd+Gzn/0s\n1q1b5/eRhBBCFON7n8CWLVuwZcuWRd9Pp9PYu3dv6etNmzZh06ZNfh9DCCFEI9H9+/fvD9qISqxc\nuTJoE1yxFOxcCjYCtFM1tFMty9FO6cQwIYSQpcvy7R1ECCGkJhQBQggxmNAdKhNUwzmv3H777YjH\n44hEIohGo6WKqKB55JFH0NfXh9bWVhw+fBgAkM1mcf/99+P8+fO46KKLcNdddyGRSITOzt7eXvzl\nL39Ba6uz63b37t349Kc/HZiN58+fx8MPP4zR0VFYloUvfOEL+OIXvxg6f1ayM2z+nJ6exv79+zEz\nM4NcLofNmzdj9+7dofNnJTvD5s8ihUIB9957L9LpNO69917v/hQhIp/Pi+985ztiYGBAzMzMiLvv\nvlu89957QZtVlttuu01kMpmgzVjEyZMnxdtvvy2+//3vl773+9//Xhw9elQIIcQzzzwj/vCHPwRl\nXolydvb29opjx44FaNV8Lly4IP79738LIYSYmJgQ3/3ud8V7770XOn9WsjNs/hRCiMnJSSGEELlc\nTuzbt0+8+eabofOnEOXtDKM/hRDi2LFj4sEHHxQ//elPhRDeP++hWg4KtOGcD0QIc+obN25cpPrH\njx/Htm3bAADXXXcdXn311SBMm0c5O4Fw+bS9vR2f+MQnAADNzc1Yu3YthoeHQ+fPSnYC4fInAMRi\nMQBALpdDoVBAIpEInT+B8nYC4fPn0NAQ+vr6sH379pJtXv0ZquWgcg3nTp8+HaBFlbEsCz09PYhE\nItixYwd27NgRtEkVGR0dRXt7OwCgra0No6NVGosFzJ/+9Ce88MILuOSSS/Dtb3878GWrIoODg3jn\nnXdw+eWXh9qfRTuvuOIKvPXWW6HzZ6FQwD333IOBgQFcf/31+PjHPx5Kf5az8+WXXw6dP3/729/i\nm9/8JiYmJkrf8+rPUInAUqKnpwcrVqzA2NgYenp6sHbtWmzcuDFos2oS6KlbNbj++uvx9a9/HQDw\nxz/+Eb/73e+wZ8+egK0CJicncfjwYdx0002Ix+c3JQyTPycnJ3HkyBHcdNNNaG5uDqU/I5EIDh06\nhPHxcRw4cAD/+Mc/5v08LP5caGd/f3/o/Pnaa6+htbUVGzZsmNd3bS5u/Bmq5aB0Oo2hoaHS10ND\nQ0in0wFaVJkVK1YAAFpbW7Fly5bQzlgAJxoodoO9cOEC2tqqnDYVIG1tbbAsC5ZlYfv27aHwaS6X\nw+HDh3HttdeWdsiH0Z9FOz/3uc/NszNs/izS0tKCTZs24e233w6lP4sU7fzXv/4VOn++9dZbeO21\n13D77bfjwQcfRH9/P375y1969meoRMBLw7kgmZqaKk2/Jicn8cYbb2D9+vUBW1WZrq4uPPfccwCA\n559/Hps3bw7WoApcuHCh9P9XXnklcJ8KIfDYY49h7dq1+NKXvlT6ftj8WcnOsPlzbGwMtm0DcCpw\n/v73v2PDhg2h82clO+e21Q+DP3fv3o1HH30UDz/8MO688050dnbijjvu8OzP0O0Y7uvrm1ci+tWv\nfjVokxYxODiIQ4cOAXDWDq+55prQ2PnAAw/gzTffxNjYGNrb29Hd3Y3NmzeHqgSvnJ033HADTp48\niXfeeQeWZeGiiy7CrbfeWlrbDIJ//vOf+NGPfoT169eXptW7d+/GZZddFip/lrPzxhtvxIsvvhgq\nf7777rt4+OGHUSgUIITAtddeiy9/+cuhKxGtZOevfvWrUPlzLidPnsSxY8dwzz33ePZn6ESAEEJI\n/QjVchAhhJD6QhEghBCDoQgQQojBUAQIIcRgKAKEEGIwFAFCCDEYigAhhBgMRYAQQgzm/wMUO03H\niwW/1gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11304b4d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model = Perceptron()\n",
    "model.train(X, y, alpha=0.1, iterations=10)    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This plot allows us to see how many examples the perceptron needed to see in order to converge to a solution. Anytime the error shows -1 the true prediction was 0 and our perceptron predicted 1. Anytime the error shows 1 the true was 1 and the perceptron predicted 0. When the error is 0 that's when the perceptron got that training case correct.\n",
    "\n",
    "\n",
    "### Decision Boundaries\n",
    "\n",
    "Now lets plot the final decision boundary from the perceptron."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 320,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAD/CAYAAAAQaHZxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt4lOWZP/Dv/eR8mBAmgjEgEkLCISBFA/QHKgr1tNoC\nUoOIiERBOaQQBIRIWmoIhAQSlIobTsu17rXbxe4229a2LrgVt6JiKiExEkKwVgE5mCEcRFjI8/z+\niKZgDmROeWcy389fmZl35v1yX+PtM/e877xijDEgIqJOTVkdgIiIvI/NnogoAHTKZl9VVWV1BL/C\nejmH9XIO6+Ucb9WLzZ5YLyexXs5hvZzDZk9ERC5jsyciCgDCQy+JiDq/YKsDtOXo0aMuPc9ms+Hs\n2bMeTtMx9J63YX65CfLDRyB3/gNExOv79Od6WYH1cg7r5Rx365WQkNDi/T7d7AORGn4HTO++0CWF\nMNUVUNMyIZHRVsciIj/n02Oc9q7sHQ4HXt+6FScrKxF87hyCg4JwuaEBl6Oj0W3wYDyQkQG73e7l\ntJ5lLl2C+Y9tMOXvQ81cBOnTz2v74srLOayXc1ivtjkcDmzd+joqK0/i3LlgBAUFo6HhMqKjL2Pw\n4G7IyHjAqf7V2sre75t99b59KM3MxOJDh9CzhccPAyhISsL49evRf8gQj2f0NrP3PehXX4bc9xDk\nB+MgyvPfqfM/RuewXs5hvVq3b181MjNLcejQYqCVDpaUVID168djyJD+7XpNrzX7DRs2YO/evYiJ\nicHatWtb3Gbr1q0oLy9HWFgYZs+ejcTExHa99rWaffW+fXgtKwsvHzjQ5mFFGsDclBSMKyjA4GHD\n2rVvX2K+PA69sRCIjoGaPh9ii/Ho6/M/RuewXs5hvVq2b181srJew4EDL6PtAyM1UlLmoqBgHIYN\nG3zN122t2bu9TLzrrruQnZ3d6uMffvghjh8/jpdeegkzZ87E5s2b3d0lgMaPPqWZmdds9EDjP/IX\nNTXYM3s23n/zTY/svyPJdddDLc6HJNwIvWI+TA1PUiHyZw6HA5mZpe1o9ACgUFPzC8yevQdvvvm+\ny/t0u9kPGDAAUVFRrT5eVlaG0aNHAwCSk5Px1Vdfob6+3t3d4vWtW7H40KF2/wMUgJyjR7ErNxfn\nz593e/8dTYKDoX48HeqxOdAlq6Ff3w6jG6yORUQu2Lr19W9GN+3vYEeP5iA3d5fL/cvrJ1U5HA7E\nxcU13Y6Li4PD4XD7dU9WVrY44bqWOQcPonTTJrf3bxUZfCvUsmKYqg+h1y2HOXPK6khE5KTKypNo\neUbftoMH52DTplKX9tkhZ9B64zvg4HPnXHpeMoBjZWWeDdPBpGsc1LN5kKT+0LlZMPv3WR2JiJxw\n7pyrR70no6zsmEvP9Ppx9na7HXV1dU236+rqWjyMqKqq6qofAEpPT4fNZmv1dYODglzOFH7pUpuv\n7TceewaXhgzD+Q35CLnrfoRPnAZxoS6hoaGdox4dhPVyDuvVXFCQ66330qXwa9Zz+/btTX+npqYi\nNTXV+80+LS0Nb7zxBkaNGoWamhpERUUhNja22XbfBrpSW9/gX25wfV59ISSk8xwd0DsFsmwtLm4u\nwsWfz4N6aiGka9y1n3cFHi3hHNbLOaxXcw0Nl11+bkjIhTbrabPZkJ6e3ux+t5v9unXrsH//fpw5\ncwazZs3Cww8/jIZvGvHdd9+NW265BXv37kVmZibCw8Mxa9Ysd3cJALgc7dpZpQcBxKeleSSDr5CY\nrlDzfw7zh19Br8iCeuInkMGd699I1JlER7va7A8iLS3epWe63eznz59/zW2efPJJd3fTTLfBg3F4\n506nvuIwAF5OTkbmzJkez2M1UQryQDpMcir0lrWQYbdDxk+FBPMXMYh8zeDB3bBz52E49yWtQXLy\ny5g5M9OlffrtTxw/kJGBgqQk6HZurwGsSEjA6JwcREREeDOapSQlFWrZOpijn0MXLoX58rjVkYjo\nOzIyHkBSUgHgRAdLSFiBnJzRLvcvv232drsd49evx9yUlGuW69szaIdv2IARY8d2RDxLiS0Gau4y\nyK0joVcuhPnwXasjEdEV7HY71q8fj5SUubh2w288g3bDhuEYO3aEy/v022YPAP2HDMG4ggKsbOX0\nYKBxdJOXkOC3P5XgKlEK6p4JUHOXQW/fAv1vG2EuXbI6FhF9Y8iQ/igoGIeEhJVtbGWQkJDX7p9K\naItfN3sAGDxsGFLz8zEvORkHv/PYQQDzk5MxKD8/oBr9laRPP6ifroOpr4POXwxzwrVrBBCR5w0b\nNhj5+alITp4HtNDBkpPnIz9/kNuNHugEv3r5rfPnz6N00yYcKytD+KVLuBASgvi0NIyfMQORkZFe\nTOkfjDEwb/0e5re/hDwyA2r4HU2P8dA457BezmG9ru38+fPYtKkUZWXHcOlSOEJCLiAtLR4zZox3\nun912p84bgnfXK0znx2CLimA9L8ZMukpSGgY6+Uk1ss5rJdzvHWlKr8f45BzpFcSVE4xcOHrxi9v\nv/jc6khE1AHY7AOQhEdCnnoWMvaH0AVL8X+7/mh1JCLyMp5xE6BEBHL7PTB9+uHCpjUw5Xsgjz4D\nCe+85yAQBTKu7AOc9LgJtrxXAKWg8xbAHP6r1ZGIyAvY7AkSHgH1xDzIA+nQa3Ogd/3RKz9LTUTW\n4RiHmqjv3wXTOxm6pBCorgCmzoFEtn4VMiLyH1zZ01UkvidUdiEQbYNekQXz6XdP9CAif8RmT81I\nSCjUlFlQDz0O/dIL0G/+lmMdIj/HMQ61StJug7qpL3RJAUx1RePv5EfxikNE/ogre2qTdIuHWrIa\ncl184/VuD1VbHYmIXMBmT9ckwSFQk56EemQG9Mt50H/4Dxjd3t/hJiJfwGZP7SbfGwH1fBHMvveh\n178Ac/a01ZGIqJ3Y7MkpEtcNauFKyI2J0C/MhznwkdWRiKgd2OzJaRIcDPXQNKhpmdCbCqF/+0sY\n3WB1LCJqA5s9uUwG3QK1rAjmQCV00U9h6h1WRyKiVrDZk1skNg5qwQuQlEGNJ2FV7bU6EhG1gMfZ\nk9tEBUF+NBkmJRV6SzFk5BjIjx6FBAVZHY2IvsGVPXmM9L8ZKqcY5m+10GuyYRwnrY5ERN9gsyeP\nkphYqJ/8DHLzcOi8Z2H27bE6EhGBYxzyAlEKcv9EmOSB0JvWQKorIRMfhwSHWB2NKGBxZU9eI30H\nNI51Tn4BvXoJzMljVkciClhs9uRVEh0DNed5yIg7oFctgvnLO1ZHIgpIHOOQ14kI5AfjYPoOhN5Y\n2DjWSc+AhIRaHY0oYHBlTx1GeidDLSsGzp6GXrkI5tgRqyMRBQw2e+pQEhkFeXoxZPR90Kufg37v\nLasjEQUEjnGow4kI5M77YZL6Q5cUQB+ohDwyExIWZnU0ok6LK3uyjNyYCLWsCLh8CTpvAcyRz6yO\nRNRpsdmTpSQ8ApKRBbn3Ieg12dB/3sHr3RJ5Acc4ZDkRgYwaC5OYDF1SAOyvAKbOgoRHWh2NqNPg\nyp58hiT0gspeC4SFQecugPnsE6sjEXUabPbkUyQsDOrxuZAfTYYu/in0n37PsQ6RB3CMQz5JjRgN\n0zsZemMBTHUF1LS5kMhoq2MR+S2u7MlnyfUJUEsKIF26Qudmwfy1xupIRH6LzZ58moSEQj36NNTD\n06HX50L/dynHOkQu4BiH/ILcMhLqxj7Qm9bAHKiEmj4PEh1jdSwiv8GVPfkN6RYPtXgVJL4ndO58\nmIMfWx2JyG+4vbIvLy/Htm3boLXGmDFjMH78+Kser6qqQkFBAa6//noAwIgRIzBx4kR3d0sBSoJD\nIA9Ph+k3CPof8yFjfwi5byJEcd1C1Ba3mr3WGlu2bEFOTg7sdjuWLl2KtLQ09OzZ86rtBg4ciOee\ne86toERXkpuHQT1f9PexzpNZkJiuVsci8lluLYdqa2sRHx+P7t27Izg4GKNGjUJZWVmz7fiFGnmD\n2K+DWpgH6Z3SeLTO/n1WRyLyWW41e4fDgbi4uKbbdrsdDofjqm1EBDU1NVi0aBFWrVqFw4cPu7NL\noqtIUBDUhMegps+D3lIM/V//CqMbrI5F5HO8fjROYmIiXnnlFYSFhWHv3r0oLCzEiy++2Gy7qqoq\nVFVVNd1OT0+HzWZzaZ+hoaEuPzcQdYp6jbgDut8gnP9FHsy65YjKfB7K3s0ru+oU9epArJdzPFGv\n7du3N/2dmpqK1NRU95q93W5HXV1d0+26ujrY7fartomIiGj6e+jQodi8eTPOnTuH6Oirz4b8NtCV\nzp4961Ium83m8nMDUaepV1AITGYOzO9/hTNLZkI9MQ8y+FaP76bT1KuDsF7OcbdeNpsN6enpze53\na4yTlJSEY8eO4cSJE7h8+TJ2796NtLS0q7apr69vmtnX1tYCQLNGT+QpooKgHpwE9fRi6Fdfhv7V\nNpjLl62ORWQ5t1b2QUFByMjIQF5eXtOhlz179sSOHTsAAHfffTfee+897NixA0ophIWFYd68eR4J\nTtQWSRkElVMMvXUdzJpsqBkLIXHdrY5FZBkxPnyozNGjR116Hj82Oqcz18toDbPjv2De+E+ox+dA\nvvd9t1+zM9fLG1gv57hbr4SEhBbv588lUKcmSkHunQDTdwD0pjWQ6krIxCcgISFWRyPqUDztkAKC\nJPWHylkHU3cSOn8xzAnXPjUS+Ss2ewoYEhUNNXspZORY6FWLoT/4s9WRiDoMxzgUUEQEMvZBmL79\noUsKoKsrIJOehISGWR2NyKu4sqeAJDf1hcpZB3z9FfTKhTBf8Mxu6tzY7ClgSUQkZMZCyJgHoQuW\nQO/+H6sjEXkNxzgU0EQEcse9MH36/X2sM+UZSFi41dGIPIoreyIA0rM31LIiQAR6xQKYw59aHYnI\no9jsib4hYeGNlzu8/8fQa5dBv/0Gf56bOg2OcYi+Q40cA5OYAr2xAKiuAKbOgUREWh2LyC1c2RO1\nQG7oCbW0EIiIarze7d9qrY5E5BY2e6JWSGgY1NTZkAlTodcth37zdxzrkN/iGIfoGtSw22Fu6gu9\nsRCmugJ6brbVkYicxpU9UTtI9xugnlsNua47zi6ZAXOo2upIRE5hsydqJwkJgZr0FCKfmAv9ch70\nG/8Jo7XVsYjahWMcIieFpN0Gdd0N0JvWwFRXQmXMh9i6WB2LqE1c2RO5QOK6Qy1cCenZGzo3C6bm\nI6sjEbWJzZ7IRRIcDDVxGtTjc6A3FkL/7t9hdIPVsYhaxGZP5CYZdCvUsiKY/fug1y2HOX3K6khE\nzbDZE3mAxMZBLciFJA1oHOt8vNfqSERX4Re0RB4iQUGQcY/CpKRCby2GjPwB5EeTIUFBVkcj4sqe\nyNNkwBConGKYT2ug1zwP4/jS6khEbPZE3iAxXaHmLYfcnAadtwCm4gOrI1GA4xiHyEtEKcj9P4bp\nOxB68xrIgUrIhKmQ4BCro1EA4sqeyMskeSBUzjqYY0egC5bCnDxmdSQKQGz2RB1AomOg5i6DDLsd\netUimA93Wx2JAgzHOEQdREQgd4+D6TsAemMhpLoS8vB0SEio1dEoAHBlT9TBJDGl8Wid06eg8xfD\nHD9qdSQKAGz2RBaQyGioZ56D3HYPdP5i6Pd3WR2JOjmOcYgsIiKQu/4BJqk/dEkB9IFKyKQZkLAw\nq6NRJ8SVPZHFpFcfqJwi4P8uQq98FuboZ1ZHok6IzZ7IB0h4JOTJBZB7xkMXZkO/s5PXuyWP4hiH\nyEeICGTUD2B6p0CXrAaqK4Apz0DCI62ORp0AV/ZEPkZ69IJ6vggIDoFe8SzM53+1OhJ1Amz2RD5I\nwsKgpmVCHpwEXZQD/dYfONYht3CMQ+TD1PfvhOmdDL2xoHGs8/hcSGSU1bHID3FlT+TjJL4H1NJC\nwNYFekUWzF8PWh2J/BCbPZEfkJBQqCnPQE2cBr3+Beid/8WxDjmFYxwiPyK3joLqlQS9sRCmuhJq\n+jxIlM3qWOQHuLIn8jPSLR7quXxI9xugX5gPU/ux1ZHID7DZE/khCQ6BSn8S6tGnoTesgv7Dr2C0\ntjoW+TA2eyI/JkOGQy0rgqn4APrFn8Ocqbc6Evkot2f25eXl2LZtG7TWGDNmDMaPH99sm61bt6K8\nvBxhYWGYPXs2EhMT3d0tEX1D7N2gFq6E+c2/QufOh3pyAaT/zVbHIh/j1spea40tW7YgOzsbRUVF\neOedd3D48OGrtvnwww9x/PhxvPTSS5g5cyY2b97sVmAiak6CgqAmTIV6Yh705iLo3/wbjG6wOhb5\nELeafW1tLeLj49G9e3cEBwdj1KhRKCsru2qbsrIyjB49GgCQnJyMr776CvX1/KhJ5A2SOrRxrHOw\nCrropzD1dVZHIh/hVrN3OByIi4trum232+FwONrcJi4urtk2ROQ5EmuHyvo5pN9g6BULYD760OpI\n5AM65Dj79pz8UVVVhaqqqqbb6enpsNlcO344NDTU5ecGItbLOX5Tr0dn4NKQYTj/ch5Cbr8H4Q9P\nhwR3/Kk1flMvH+GJem3fvr3p79TUVKSmprrX7O12O+rq/v4xsa6uDna73eltrgx0pbNnz7qUy2az\nufzcQMR6Ocev6tUrCbKsGBe3FuPizzKhZiyCxHXr0Ah+VS8f4G69bDYb0tPTm93v1hgnKSkJx44d\nw4kTJ3D58mXs3r0baWlpV22TlpaGt99+GwBQU1ODqKgoxMbGurNbInKC2LpAZf4U8r0R0HkLYMrf\ntzoSWcCtlX1QUBAyMjKQl5fXdOhlz549sWPHDgDA3XffjVtuuQV79+5FZmYmwsPDMWvWLI8EJ6L2\nE6Ug902ESU6F3lgIOVAJmTgNEhxidTTqIGJ8+NeUjh496tLz+LHROayXc/y9Xuars9DbXgJO1UHN\nXATpfoNX9+fv9epo7tYrISGhxft5Bi1RgJEoG9TsbMj/uws6fzFM2Z+tjkQdgL96SRSARAQy9ocw\nSf3/PtZJfxISEmp1NPISruyJApj0ToZaVgycOwu9ciHMscPXfA75JzZ7ogAnkVGQmYsgd/0D9Ool\n0O/+yepI5AUc4xBR41jnjvtg+vSDLimArq6APPo0JCzc6mjkIVzZE1ET6ZkI9XwRYDR03rMwR/5m\ndSTyEDZ7IrqKhEdAZWRB7nsIes3z0P/737zebSfAMQ4RtUiNHAuTmAJdUgBUVwCPzYZERFodi1zE\nlT0RtUpuuBEqew0QHgG9Igvmb4esjkQuYrMnojZJaBjU1DmQcVOg1/0M+n9+x7GOH+IYh4jaRQ2/\nA6Z3X+iSQpgDlVDTMiGR0VbHonbiyp6I2k26J0AtKYB0vQ76hfkwnxywOhK1E5s9ETlFQkKgHpkB\nNekp6F+sgH7j1zBaWx2LroFjHCJyiQz9PtSNidAbvxnrTJ8PscVYHYtawZU9EblMrrseanE+JOFG\n6BXzYWqqrv0ksgSbPRG5RYKDoX48HeqxOdAlq6Ff3w6jG6yORd/BZk9EHiGDb4VaVgxT9SH0uuUw\np09ZHYmuwGZPRB4jXeOgns2DJPWHzs2C+bjc6kj0DX5BS0QeJUFBkHFTYFIGQW8txtefHoS59yFI\nUJDV0QIaV/ZE5BUyYAhUTjEaaj+GLloGc6rO6kgBjc2eiLxGYroiamkBZODQxt/WqSyzOlLA4hiH\niLxKlIJ6IB0mORV681rIsNshE6ZCgtl+OhJX9kTUISQlFSpnHcwXn0MXLIH58rjVkQIKmz0RdRix\nxUDNXQZJG9V4gfMP37U6UsDg5ygi6lCiFOSeCTB9B0JvLIQcqIT8eDokJMTqaJ0aV/ZEZAnp0w/q\np+tg6uug8xfDnDhqdaROjc2eiCwjkdFQzyyB3PYD6FWLofe8bXWkTotjHCKylIhA7noAJqk/dEkB\ndHUF5JEZkNAwq6N1KlzZE5FPkF5JUDnFwMULjV/efvG51ZE6FTZ7IvIZEh4JeepZyNgfQhcshd79\nptWROg2OcYjIp4gI5PZ7YPr0axzr7K+ATHkGEh5hdTS/xpU9Efkk6XET1PNrgSAFnbcA5vBfrY7k\n19jsichnSVg41BPzIA+kQ6/Ngd71RxhjrI7llzjGISKfp75/F0zvZOiSQqC6Apg6BxIZZXUsv8KV\nPRH5BYnvCZVdCETbGn9B89ODVkfyK2z2ROQ3JCQUasosqIceh37pBeidv+FYp504xiEivyNpt0Hd\n1Be6pACmugJq+jxIlM3qWD6NK3si8kvSLR5qyWpItxsar3d7qNrqSD6NzZ6I/JYEh0BNehLqkRnQ\nL+dB/+E/YLS2OpZPYrMnIr8n3xsB9XwRzL73oV/6OcyZeqsj+Rw2eyLqFCSuG9TClZBefRrHOgc+\nsjqST2GzJ6JOQ4KDoR6aBjUtE3pTIfRvfwmjG6yO5RNcPhrn3LlzKC4uxpdffolu3bohKysLUVHN\nT3KYM2cOIiIioJRCUFAQVq1a5VZgIqJrkUG3QC0rgt5cBHOgEuqpZyGxdqtjWcrlZl9aWoqbb74Z\n48aNQ2lpKUpLSzFlypQWt12+fDmio6NdDklE5CyJjYNa8ALM77ZDr8iCmj4fkjrU6liWcXmMU1ZW\nhtGjRwMA7rzzTnzwwQetbsuTHojICqKCoH40GeqpZ6G3vQT9n/8M0xCYYx2Xm/3p06cRGxsLAOjS\npQtOnz7d4nYigtzcXCxZsgQ7d+50dXdERC6T/jdD5RTDfHYIek02jOOk1ZE6XJtjnNzcXNTXNz+E\nafLkyVfdFpE2X6Nr1644c+YMcnNz0aNHDwwYMKDZdlVVVaiqqmq6nZ6eDpvNtTPiQkNDXX5uIGK9\nnMN6Ocdn6mWzwTy/Bhd/+++4uHIhImYuRMitI61O1Ywn6rV9+/amv1NTU5Gamtp2s8/JyWn1sS5d\nuqC+vh6xsbE4deoUunTp0uJ2Xbt2BQDExMRg+PDhqK2tbbHZfxvoSmfPnm0rXqtsNpvLzw1ErJdz\nWC/n+Fy9xjwI6ZWErzatgezdA5n4OCQ4xOpUTdytl81mQ3p6erP7XR7jpKWl4a233gIA7Nq1C8OG\nDWu2zcWLF/H1118DAC5cuICKigr06tXL1V0SEXmE9B3QONY5+QX06iUwJ49ZHcnrXD4aZ/z48Sgu\nLsaf/vSnpkMvAcDhcKCkpARLly5FfX091qxZAwDQWuO2227DkCFDPJOciMgNEh0DNed5mDd/A71q\nEdSUZyC3jrI6lteI8eFDZY4ePerS83zuY6OPY72cw3o5xx/qZT49CL2xEJJ6CyQ9AxISalkWd+uV\nkJDQ4v08g5aIAp70ToZaVgxzth565SKYY0esjuRxbPZERAAkMgrq6ecgo++DXv0c9HtvWR3Jo3jx\nEiKib4gI5M77YZL6Q5cUQFdXQCbPhISFWx3NbVzZExF9h9yYCLWsCGi4DJ33LMyRz6yO5DY2eyKi\nFkh4BCQjC3LvQ9BrsqH/vMOvf/qFYxwiolaICGTUWJjEZOiSAmB/BTB1FiQ80upoTuPKnojoGiSh\nF1T2WiAsDDp3Acxnh6yO5DQ2eyKidpCwMKjH50J+NBm6+GfQf/q9X411OMYhInKCGjEapncy9MYC\nmOoKqGlzIZG+f70OruyJiJwk1ydALSmAdOnaeL3bv9ZYHema2OyJiFwgIaFQjz4N9fB06PW50P9d\n6tNjHY5xiIjcILeMhLqxD/SmNY3Xu50+DxIdY3WsZriyJyJyk3SLh1q8ChLfEzp3PszBj62O1Ayb\nPRGRB0hwCNTD06GmzIL+x3zo378Go7XVsZqw2RMReZDcPAzq+SKYyr9Av7gc5swpqyMBYLMnIvI4\nsV8HtTAP0jul8Wid/fusjsQvaImIvEGCgiATHoPplwq9pRhy+z2QH06CqCBL8nBlT0TkRTJwaOP1\nbg/th16bA3OqzpIcbPZERF4mXbpCzV8OGTAEekUWTOVfOjwDxzhERB1AVBDkwUkwKanQm4sgw++A\njH8MEtwxbZgreyKiDiQpgxrHOkf+Bl24FKbuRIfsl82eiKiDia0LVGYO5JaRjVfCKn/P6/vkGIeI\nyAKiFOTeCTB9B0BvWgOproRMfMJr++PKnojIQpLUHypnHUzdSej8xWg4dsQr+2GzJyKymERFQ81e\nChk5Fudy5kB/8L8e3webPRGRDxARqLEPImrpaphfvwr96ssw/3fRY6/PZk9E5EOC+/SDylkHfH0e\neuVCmC8Oe+R12eyJiHyMRERCZiyEjHkQumAJ9O7/cfs1eTQOEZEPEhHIHffC9OkHXVIAXV0BefRp\nSHiES6/HlT0RkQ+Tnr2hlhUBIo3H5B/+1KXXYbMnIvJxEhbeeLnD+38MvXYZ9NtvOH29W45xiIj8\nhBo5BiYxBbpkNVBdAUydA4mIbN9zvZyNiIg8SG7oCZW9BoiIarze7d9q2/U8NnsiIj8joWFQU2dD\nJkyFXrcc+s3fXnOswzEOEZGfUsNuh7mpL/TGQpjqSqgnftL6th2Yi4iIPEy63wD13GrIdd2hc+e3\nuh1X9kREfk5CQiCTnoLpN6jVbbiyJyLqJOR732/1MTZ7IqIA0CnGOA6HA69v3YqTlZUIPncOwUFB\nuNzQgMvR0eg2eDAeyMiA3W63OiYRUTMd1b/EOHsaVgc6evToNbep3rcPpZmZWHzoEHq28PhhAAVJ\nSRi/fj36Dxni8Yydgc1mw9mzZ62O4TdYL+ewXq3zRv9KSEho8X6XxzjvvvsuFixYgEmTJuGTTz5p\ndbvy8nLMnz8fP/nJT1BaWurq7lpUvW8fXsvKwrpWCgUAPQGsO3QIv5o/H5UffODR/RMRuaqj+5fL\nzb5Xr15YuHAhBg4c2Oo2Wmts2bIF2dnZKCoqwjvvvIPDhz3z28wOhwOlmZl4+cCBa/4jFIBf1NRg\nz+zZeP/NNz2yfyIiV1nRv1xu9j169Gj148K3amtrER8fj+7duyM4OBijRo1CWVmZq7u8yutbt2Lx\noUPt/gcoADlHj2JXbi7Onz/vkQxERK6won959Wgch8OBuLi4ptt2ux0Oh8Mjr32ysrLVjz5tmXPw\nIEo3bfJIBiIiV1jRv9o8Gic3Nxf19fXN7p88eTLS0tJc2qGnBJ8759LzkgEc89CnCyIiV1jRv9ps\n9jk5OS6lGcxeAAAGNElEQVS96Lfsdjvq6uqabtfV1bV6CFFVVRWqqqqabqenp8Nms7X62sFBQS7n\nCr90qc3XDjShoaGshxNYL+ewXs15u39t37696e/U1FSkpqZ69zj7pKQkHDt2DCdOnIDdbsfu3bsx\nb968Frf9NtCV2jpc63JDg8u5LoSE8FCwK/DQOOewXs5hvZrzZv+y2WxIT09vdr/LM/s9e/Zg1qxZ\nqKmpwapVq7By5UoAjXP6VatWAQCCgoKQkZGBvLw8ZGVlYeTIkejZ05VJVXOXo6Ndet5BAPEWj6CI\nKLBZ0b9cXtkPHz4cw4cPb3a/3W7H0qVLm24PHToUQ4cOdXU3reo2eDAO79zp1JccBsDLycnInDnT\n43mIiNrLiv7lt7+N80BGBgqSkqDbub0GsCIhAaNzchAR4drV2YmIPMGK/uW3zd5ut2P8+vWYm5Jy\nzYJpAHNTUjB8wwaMGDu2I+IREbXKiv7lt80eAPoPGYJxBQVY2cbJXQZAXkICxhUUYPCwYR0Xjoio\nDR3dv/y62QPA4GHDkJqfj3nJyTj4nccOApifnIxB+fls9ETkczqyf/n9r15+6/z58yjdtAnHysoQ\nfukSLoSEID4tDeNnzEBkZKQXU/o/HhrnHNbLOazXtXmyf7X2MzadptlfiW8u57BezmG9nMN6Ocfd\nevllsyciIs/w+5l9S648VZiujfVyDuvlHNbLOd6qV6ds9kREdDU2eyKiANApm/13f1CN2sZ6OYf1\ncg7r5Rxv1Ytf0BIRBYBOubInIqKrsdkTEQUAr168pKO8++67eO2113DkyBGsWrUKffr0aXG78vJy\nbNu2DVprjBkzBuPHj+/gpL7h3LlzKC4uxpdffolu3bohKysLUVFRzbabM2cOIiIioJRCUFBQ03UK\nAkV73i9bt25FeXk5wsLCMHv2bCQmJlqQ1Ddcq15VVVUoKCjA9ddfDwAYMWIEJk6caEVUy23YsAF7\n9+5FTEwM1q5d2+I2Hn9vmU7g8OHD5siRI2b58uXm0KFDLW7T0NBg5s6da44fP24uXbpkFi5caD7/\n/PMOTuobXn31VVNaWmqMMebXv/61+Zd/+ZcWt5s9e7Y5e/ZsR0bzGe15v/zlL38xK1euNMYYU1NT\nY7Kzs62I6hPaU6+PPvrI5OfnW5TQt3z88cfmk08+MQsWLGjxcW+8tzrFGKdHjx6tniL8rdraWsTH\nx6N79+4IDg7GqFGjUBagFx4vKyvD6NGjAQB33nknPvjgg1a3NQH6/X173i9X1jE5ORlfffUV6uvr\nrYhrufb+9xWo76fvGjBgQIufpr/ljfdWp2j27eFwOBAXF9d02263w+FwWJjIOqdPn0ZsbCwAoEuX\nLjh9+nSL24kIcnNzsWTJEuzcubMjI1quPe+X724TFxcXsO+p9tRLRFBTU4NFixZh1apVOHz4cEfH\n9BveeG/5zcw+Nze3xf+zTZ48GWm8pmwzbdXrSiLS5mt07doVZ86cQW5uLnr06IEBAwZ4PKs/40q1\n/RITE/HKK68gLCwMe/fuRWFhIV588UWrY/ksT7+3/KbZ5+TkuPV8u92Ourq6ptt1dXWw2+3uxvJZ\nbdWrS5cuqK+vR2xsLE6dOoUuXbq0uF3Xrl0BADExMRg+fDhqa2sDptm35/0SaO+ptrSnFldeTm/o\n0KHYvHkzzp07h2gXL77dmXnjvRUwY5ykpCQcO3YMJ06cwOXLl7F79+6A/USQlpaGt956CwCwa9cu\nDGvhwggXL17E119/DQC4cOECKioq0KtXr46Maan2vF/S0tLw9ttvAwBqamoQFRXVNB4LNO2pV319\nfdNqtba2FgDY6FvhjfdWpziDds+ePfinf/onnDlzBpGRkUhMTER2djYcDgdKSkqwdOlSAMDevXuv\nOjRswoQJFie3RmuHXl5Zr+PHj2PNmjUAAK01brvttoCrV0vvlx07dgAA7r77bgDAli1bUF5ejvDw\ncMyaNavVw34DwbXq9cc//hE7duyAUgphYWF4/PHHkZKSYnFqa6xbtw779+/HmTNnEBsbi4cffhgN\nDQ0AvPfe6hTNnoiI2hYwYxwiokDGZk9EFADY7ImIAgCbPRFRAGCzJyIKAGz2REQBgM2eiCgAsNkT\nEQWA/w9XO744BVYhkQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x110907e50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "n = np.linalg.norm(model.w[0:2])\n",
    "ww = (model.w[0:2]) / n\n",
    "ww1 = [float(ww[1])* 2., -1.0*float(ww[0])* 2.] \n",
    "ww2 = [-1.0*float(ww[1]) * 2., float(ww[0]) * 2.]\n",
    "plt.plot([ww1[0]-model.w[2],ww2[0]-model.w[2]], [ww1[1]-model.w[2],ww2[1]-model.w[2]])\n",
    "plt.plot(X[0][0],X[0][1], 'ro', markersize=15)\n",
    "plt.plot(X[1][0],X[1][1], 'ro', markersize=15)\n",
    "plt.plot(X[2][0],X[2][1], 'ro', markersize=15)\n",
    "plt.plot(X[3][0],X[3][1], 'bo', markersize=15)\n",
    "plt.axis([-1.1,1.1,-1.1,1.1])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "\n",
    "\n",
    "\n",
    "\n",
    "Here is a cool animation of the how the decision boundary updates after every full iteration through the training data. This animation was created on a different run of the algorithm so the final boundary is not the same as the one above. It is worth noting that the decision boundary may not always be the same.\n",
    "\n",
    "![](http://i.imgur.com/aEufKbg.gif)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}