Tugas4_GriesYulianti.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tulis Jawaban pada Tempat yang Telah Disediakan"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Anda diberikan tiga buah data dari Taarifa dan Kementrian Air di Tanzania yaitu: data latih, label data latih, dan data uji. Anda diminta untuk memprediksi pompa air mana yang berfungsi (\"functional\"), pompa air yang memerlukan perbaikan (\"functional needs repair\"), dan pompa mana yang tidak berfungsi sama sekali (\"non functional\"). \n",
    "Referensi: https://www.drivendata.org/competitions/7/pump-it-up-data-mining-the-water-table/data/ (Anda perlu memiliki akun DrivenData dan “Compete” pada perlombaan “Pump it Up: Data Mining the Water Table” untuk bisa mengakses data):\n",
    "\n",
    "<b>Catatan:</b>\n",
    "Anda diperbolehkan untuk menggunakan Lib ANN, namun untuk teknik <b><i>Bagging</b></i> dan <b><i>Adaptive Boosting</b></i> Anda harus mengerjakan secara manual (tanpa Lib)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Jika dalam menyelesaikan tugas ini Anda berkolaborasi dengan orang lain, silahkan tuliskan dengan siapa Anda berkolaborasi (pada baris ini)!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Kolaborator: ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Jika dalam menyelesaikan tugas ini, Anda menggunakan referensi dari halaman situs di internet, silahkan tuliskan link halaman situs tersebut (pada baris ini)!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Referensi:\n",
    "    1) https://www.datacamp.com/community/tutorials/pandas-read-csv\n",
    "    2) https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.drop.html\n",
    "    3) http://pbpython.com/categorical-encoding.html\n",
    "    4) https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.update.html\n",
    "    5) https://machinelearningmastery.com/ensemble-machine-learning-algorithms-python-scikit-learn/\n",
    "    6) https://github.com/MattBrown88/Pump-it-Up-XGBoost-Ensemble\n",
    "    7) https://www.drivendata.org/competitions/7/pump-it-up-data-mining-the-water-table/page/25/\n",
    "    8) https://stackoverflow.com/questions/13413590/how-to-drop-rows-of-pandas-dataframe-whose-value-in-certain-columns-is-nan\n",
    "    9) https://machinelearningmastery.com/handle-missing-data-python/\n",
    "    10) https://stackoverflow.com/questions/46760288/replace-pandas-column-values-with-array\n",
    "    11) https://github.com/Jean-njoroge/ensemble-learning/tree/master/notebooks\n",
    "    12)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Soal"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Anda diminta membangun ANN dengan spesifikasi jumlah <i>hidden layer</i>, jumlah <i>output</i> unit, jumlah <i>hidden</i> unit sesuka Anda, atau menggunakan fungsi aktivasi tertentu. \n",
    "\n",
    "1. Gunakan metode <b><i>Bagging</b></i> dari tiga model ANN! Dari spesifikasi yang telah Anda pilih, Anda diharapkan dapat menghasilkan tiga model ANN berbeda karena Anda men-train model dengan training set yang berbeda. Hint: Anda perlu membuat training set yang berbeda-beda dari data yang tersedia, misalnya dengan teknik random sampling, k-fold, atau teknik lainnya. Hitung akurasi ketiga model dan tampilkan <i>confusion matrix</i> nya!\n",
    "2. Gunakan metode <b><i>Adaptive Boosting</b></i> dari spesifikasi ANN yang sama pada soal 1. Hitung akurasi dan tampilkan <i>confusion matrix</i> nya!\n",
    "3. Bandingkan hasil akurasi pelatihan dan pengujian pada soal 1 dan 2!\n",
    "4. Berikan analisis Anda tentang hasil perbandingan yang diperoleh pada soal 3!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Jawaban"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PREPROCESSING DATA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "===== Data Latih (5 data teratas) =====\n",
      "        id  amount_tsh date_recorded        funder  gps_height     installer  \\\n",
      "0  69572.0      6000.0    2011-03-14         Roman      1390.0         Roman   \n",
      "1   8776.0         NaN    2013-03-06       Grumeti      1399.0       GRUMETI   \n",
      "2  34310.0        25.0    2013-02-25  Lottery Club       686.0  World vision   \n",
      "3  67743.0         NaN    2013-01-28        Unicef       263.0        UNICEF   \n",
      "4  19728.0         NaN    2011-07-13   Action In A         NaN       Artisan   \n",
      "\n",
      "   longitude   latitude              wpt_name  num_private  \\\n",
      "0  34.938093  -9.856322                  none          NaN   \n",
      "1  34.698766  -2.147466              Zahanati          NaN   \n",
      "2  37.460664  -3.821329           Kwa Mahundi          NaN   \n",
      "3  38.486161 -11.155298  Zahanati Ya Nanyumbu          NaN   \n",
      "4  31.130847  -1.825359               Shuleni          NaN   \n",
      "\n",
      "           ...          payment_type water_quality quality_group  \\\n",
      "0          ...              annually          soft          good   \n",
      "1          ...             never pay          soft          good   \n",
      "2          ...            per bucket          soft          good   \n",
      "3          ...             never pay          soft          good   \n",
      "4          ...             never pay          soft          good   \n",
      "\n",
      "       quantity  quantity_group                source           source_type  \\\n",
      "0        enough          enough                spring                spring   \n",
      "1  insufficient    insufficient  rainwater harvesting  rainwater harvesting   \n",
      "2        enough          enough                   dam                   dam   \n",
      "3           dry             dry           machine dbh              borehole   \n",
      "4      seasonal        seasonal  rainwater harvesting  rainwater harvesting   \n",
      "\n",
      "   source_class              waterpoint_type waterpoint_type_group  \n",
      "0   groundwater           communal standpipe    communal standpipe  \n",
      "1       surface           communal standpipe    communal standpipe  \n",
      "2       surface  communal standpipe multiple    communal standpipe  \n",
      "3   groundwater  communal standpipe multiple    communal standpipe  \n",
      "4       surface           communal standpipe    communal standpipe  \n",
      "\n",
      "[5 rows x 40 columns]\n",
      "\n",
      "===== Header Data Latih =====\n",
      "['id', 'amount_tsh', 'date_recorded', 'funder', 'gps_height', 'installer', 'longitude', 'latitude', 'wpt_name', 'num_private', 'basin', 'subvillage', 'region', 'region_code', 'district_code', 'lga', 'ward', 'population', 'public_meeting', 'recorded_by', 'scheme_management', 'scheme_name', 'permit', 'construction_year', 'extraction_type', 'extraction_type_group', 'extraction_type_class', 'management', 'management_group', 'payment', 'payment_type', 'water_quality', 'quality_group', 'quantity', 'quantity_group', 'source', 'source_type', 'source_class', 'waterpoint_type', 'waterpoint_type_group']\n",
      "\n",
      "===== Data setelah Feature Selection ======\n",
      "        id  amount_tsh date_recorded  gps_height  longitude   latitude  \\\n",
      "0  69572.0      6000.0    2011-03-14      1390.0  34.938093  -9.856322   \n",
      "1   8776.0         NaN    2013-03-06      1399.0  34.698766  -2.147466   \n",
      "2  34310.0        25.0    2013-02-25       686.0  37.460664  -3.821329   \n",
      "3  67743.0         NaN    2013-01-28       263.0  38.486161 -11.155298   \n",
      "4  19728.0         NaN    2011-07-13         NaN  31.130847  -1.825359   \n",
      "\n",
      "                     basin  region_code  district_code  population  \\\n",
      "0               Lake Nyasa           11            5.0       109.0   \n",
      "1            Lake Victoria           20            2.0       280.0   \n",
      "2                  Pangani           21            4.0       250.0   \n",
      "3  Ruvuma / Southern Coast           90           63.0        58.0   \n",
      "4            Lake Victoria           18            1.0         NaN   \n",
      "\n",
      "              ...              scheme_management permit construction_year  \\\n",
      "0             ...                            VWC  False            1999.0   \n",
      "1             ...                          Other   True            2010.0   \n",
      "2             ...                            VWC   True            2009.0   \n",
      "3             ...                            VWC   True            1986.0   \n",
      "4             ...                            NaN   True               NaN   \n",
      "\n",
      "   extraction_type_class management_group         payment quality_group  \\\n",
      "0                gravity       user-group    pay annually          good   \n",
      "1                gravity       user-group       never pay          good   \n",
      "2                gravity       user-group  pay per bucket          good   \n",
      "3            submersible       user-group       never pay          good   \n",
      "4                gravity            other       never pay          good   \n",
      "\n",
      "       quantity source_class              waterpoint_type  \n",
      "0        enough  groundwater           communal standpipe  \n",
      "1  insufficient      surface           communal standpipe  \n",
      "2        enough      surface  communal standpipe multiple  \n",
      "3           dry  groundwater  communal standpipe multiple  \n",
      "4      seasonal      surface           communal standpipe  \n",
      "\n",
      "[5 rows x 21 columns]\n",
      "\n",
      "===== Header Data setelah Feature Selection =====\n",
      "['id', 'amount_tsh', 'date_recorded', 'gps_height', 'longitude', 'latitude', 'basin', 'region_code', 'district_code', 'population', 'public_meeting', 'scheme_management', 'permit', 'construction_year', 'extraction_type_class', 'management_group', 'payment', 'quality_group', 'quantity', 'source_class', 'waterpoint_type']\n",
      "\n",
      "===== Jumlah Missing Values di tiap atribut =====\n",
      "id                           1\n",
      "amount_tsh               41639\n",
      "date_recorded                0\n",
      "gps_height               20438\n",
      "longitude                 1812\n",
      "latitude                     0\n",
      "basin                        0\n",
      "region_code                  0\n",
      "district_code               23\n",
      "population               21381\n",
      "public_meeting            3334\n",
      "scheme_management         3878\n",
      "permit                    3056\n",
      "construction_year        20709\n",
      "extraction_type_class        0\n",
      "management_group             0\n",
      "payment                      0\n",
      "quality_group                0\n",
      "quantity                     0\n",
      "source_class                 0\n",
      "waterpoint_type              0\n",
      "dtype: int64\n",
      "\n",
      "===== Jumlah Missing Values di tiap atribut setelah handling Categorical =====\n",
      "id                           0\n",
      "amount_tsh               41639\n",
      "date_recorded                0\n",
      "gps_height               20438\n",
      "longitude                 1812\n",
      "latitude                     0\n",
      "basin                        0\n",
      "region_code                  0\n",
      "district_code               23\n",
      "population               21381\n",
      "public_meeting               0\n",
      "scheme_management            0\n",
      "permit                       0\n",
      "construction_year        20709\n",
      "extraction_type_class        0\n",
      "management_group             0\n",
      "payment                      0\n",
      "quality_group                0\n",
      "quantity                     0\n",
      "source_class                 0\n",
      "waterpoint_type              0\n",
      "dtype: int64\n",
      "\n",
      "===== Jumlah Missing Values di tiap atribut setelah handling longitude, latitude, region_code, district_code, and construction_year =====\n",
      "id                           0\n",
      "amount_tsh               41639\n",
      "date_recorded                0\n",
      "gps_height               20438\n",
      "longitude                    0\n",
      "latitude                     0\n",
      "basin                        0\n",
      "region_code                  0\n",
      "district_code                0\n",
      "population               21381\n",
      "public_meeting               0\n",
      "scheme_management            0\n",
      "permit                       0\n",
      "construction_year            0\n",
      "extraction_type_class        0\n",
      "management_group             0\n",
      "payment                      0\n",
      "quality_group                0\n",
      "quantity                     0\n",
      "source_class                 0\n",
      "waterpoint_type              0\n",
      "dtype: int64\n",
      "\n",
      "===== Jumlah Missing Values Final =====\n",
      "id                       0\n",
      "amount_tsh               0\n",
      "date_recorded            0\n",
      "gps_height               0\n",
      "longitude                0\n",
      "latitude                 0\n",
      "basin                    0\n",
      "region_code              0\n",
      "district_code            0\n",
      "population               0\n",
      "public_meeting           0\n",
      "scheme_management        0\n",
      "permit                   0\n",
      "construction_year        0\n",
      "extraction_type_class    0\n",
      "management_group         0\n",
      "payment                  0\n",
      "quality_group            0\n",
      "quantity                 0\n",
      "source_class             0\n",
      "waterpoint_type          0\n",
      "dtype: int64\n",
      "\n",
      "===== Data setelah Missing Value Handling ======\n",
      "        id  amount_tsh  date_recorded  gps_height  longitude   latitude  \\\n",
      "0  69572.0      6000.0           47.0      1390.0  34.938093  -9.856322   \n",
      "1   8776.0       500.0          309.0      1399.0  34.698766  -2.147466   \n",
      "2  34310.0        25.0          300.0       686.0  37.460664  -3.821329   \n",
      "3  67743.0       500.0          272.0       263.0  38.486161 -11.155298   \n",
      "4  19728.0       500.0          104.0       -15.0  31.130847  -1.825359   \n",
      "\n",
      "   basin  region_code  district_code  population       ...         \\\n",
      "0    1.0           11            5.0       109.0       ...          \n",
      "1    4.0           20            2.0       280.0       ...          \n",
      "2    5.0           21            4.0       250.0       ...          \n",
      "3    7.0           90           63.0        58.0       ...          \n",
      "4    4.0           18            1.0         1.0       ...          \n",
      "\n",
      "   scheme_management  permit  construction_year  extraction_type_class  \\\n",
      "0                6.0     0.0             1999.0                    0.0   \n",
      "1                1.0     1.0             2010.0                    0.0   \n",
      "2                6.0     1.0             2009.0                    0.0   \n",
      "3                6.0     1.0             1986.0                    5.0   \n",
      "4                6.0     1.0             2010.0                    0.0   \n",
      "\n",
      "   management_group  payment  quality_group  quantity  source_class  \\\n",
      "0               4.0      2.0            2.0       1.0           0.0   \n",
      "1               4.0      0.0            2.0       2.0           1.0   \n",
      "2               4.0      4.0            2.0       1.0           1.0   \n",
      "3               4.0      0.0            2.0       0.0           0.0   \n",
      "4               1.0      0.0            2.0       3.0           1.0   \n",
      "\n",
      "   waterpoint_type  \n",
      "0              1.0  \n",
      "1              1.0  \n",
      "2              2.0  \n",
      "3              2.0  \n",
      "4              1.0  \n",
      "\n",
      "[5 rows x 21 columns]\n",
      "\n",
      "===== Target Label =====\n",
      "functional                 32259\n",
      "non functional             22824\n",
      "functional needs repair     4317\n",
      "Name: status_group, dtype: int64\n",
      "0    32259\n",
      "2    22824\n",
      "1     4317\n",
      "Name: status_group, dtype: int64\n",
      "Matrix X = [[ 6.000e+03  4.700e+01  1.390e+03 ...  1.000e+00  0.000e+00  1.000e+00]\n",
      " [ 5.000e+02  3.090e+02  1.399e+03 ...  2.000e+00  1.000e+00  1.000e+00]\n",
      " [ 2.500e+01  3.000e+02  6.860e+02 ...  1.000e+00  1.000e+00  2.000e+00]\n",
      " ...\n",
      " [ 5.000e+02  7.500e+01 -1.500e+01 ...  1.000e+00  0.000e+00  4.000e+00]\n",
      " [ 5.000e+02  4.100e+01 -1.500e+01 ...  2.000e+00  0.000e+00  4.000e+00]\n",
      " [ 5.000e+02  5.600e+01  1.910e+02 ...  1.000e+00  0.000e+00  4.000e+00]]\n",
      "Matrix y (target label) = [0 0 0 ... 0 0 0]\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "from sklearn.preprocessing import Imputer\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "#--- Mengambil data latih dari file\n",
    "data_latih = pd.read_csv('data latih.csv', na_values = ['None', 'no info', '.', '0',0])\n",
    "print '===== Data Latih (5 data teratas) ====='\n",
    "print data_latih.head() # menampilkan 5 data teratas\n",
    "\n",
    "print '\\n===== Header Data Latih ====='\n",
    "print list(data_latih)\n",
    "\n",
    "#--- Mengambil target label dari file\n",
    "target_label = pd.read_csv('label data latih.csv')\n",
    "#print target_label.head()\n",
    "\n",
    "####### Feature Selection #######\n",
    "# =============================================================================\n",
    "#  Menghapus beberapa variabel yang duplikat dan tidak mempengaruhi model\n",
    "#  \n",
    "#  1) num_private, tidak mempengaruhi model dan datanya kosong\n",
    "#  2) funder, tidak mempengaruhi model karena itu hanya nama penemu sumur\n",
    "#  3) recorded_by, karena hanya 1 nilai\n",
    "#  4) wpt_name, terlalu banyak nilai yang unik\n",
    "#  5) extraction_type_group, nilainya sama dengan extraction_type_class\n",
    "#  6) extraction_type, nilainya sama dengan extraction_type_class\n",
    "#  7) payment_type, nilainya sama dengan payment\n",
    "#  8) water_quality, nilainya sama dengan quality_group\n",
    "#  9) scheme_name, banyak data yang kosong\n",
    "#  10) region, dapat diwakilkan dengan region_code\n",
    "#  11) subvillage, dapat diwakilkan dengan district_code\n",
    "#  12) ward, data lokasi dapat diwakilkan dengan region_code dan district_code\n",
    "#  13) lga, data lokasi dapat diwakilkan dengan region_code dan district_code\n",
    "#  14) waterpoint_type_group, sama dengan waterpoint_type\n",
    "#  15) quantity_group, duplikat dari data quantity\n",
    "#  16) management, dapat diwakilkan dari data management_group\n",
    "#  17) installer, tidak mempengaruhi model karena itu organisasi yang membuat sumur\n",
    "#  18) source, nilainya dapat diwakilkan oleh source_class\n",
    "#  19) source_type, nilainya dapat diwakilkan oleh source_class\n",
    "# =============================================================================\n",
    "\n",
    "del_coloumns = ['num_private', 'scheme_name', 'funder', 'recorded_by', \n",
    "                'wpt_name', 'extraction_type_group', 'extraction_type', \n",
    "                'payment_type', 'water_quality', 'region', 'subvillage', \n",
    "                'ward', 'lga', 'waterpoint_type_group', 'quantity_group', \n",
    "                'management', 'installer', 'source', 'source_type']\n",
    "\n",
    "df = data_latih.drop(del_coloumns, axis = 1).copy()\n",
    "\n",
    "print '\\n===== Data setelah Feature Selection ======'\n",
    "print df.head()\n",
    "\n",
    "print '\\n===== Header Data setelah Feature Selection ====='\n",
    "print list(df)\n",
    "\n",
    "\n",
    "####### Missing Values and Encoder #######\n",
    "\n",
    "# Jika pada data tersebut terdapat missing value, \n",
    "# Untuk data categorical, dijadikan data numeric dan digantikan dengan modus data\n",
    "# Untuk data numeric, nilai yang kosong diganti dengan mean, \n",
    "# kecuali untuk longitude, latitude, region_code, district_code, dan construction_year diganti dengan median\n",
    "\n",
    "print '\\n===== Jumlah Missing Values di tiap atribut ====='\n",
    "#print len(df)\n",
    "print df.isnull().sum()\n",
    "\n",
    "#--- Find missing values rows\n",
    "#null_columns=df.columns[df.isnull().any()]\n",
    "#print(df[df[\"id\"].isnull()][null_columns])\n",
    "\n",
    "# Terdapat 1 missing value di \"id\", tetapi data atribut lain ada dan ada target labelnya\n",
    "# oleh karena itu missing value dikembalikan kembali nilainya\n",
    "df[[\"id\"]] = df[[\"id\"]].replace(np.NaN, 0)\n",
    "\n",
    "\n",
    "#print len(df)\n",
    "\n",
    "#--- Data categorical\n",
    "obj_df = df.select_dtypes(include=['object'])\n",
    "#print obj_df.dtypes\n",
    "\n",
    "#--- Encoding data untuk [\"permit\", \"public_meeting\", \"scheme_management\"]\n",
    "df[[\"permit\", \"public_meeting\", \"scheme_management\"]] = df[[\"permit\", \"public_meeting\", \"scheme_management\"]].astype('category')\n",
    "#print df.dtypes\n",
    "\n",
    "df[\"public_meeting\"] = df[\"public_meeting\"].cat.codes\n",
    "df[\"permit\"] = df[\"permit\"].cat.codes\n",
    "df[\"scheme_management\"] = df[\"scheme_management\"].cat.codes\n",
    "df[[\"permit\", \"public_meeting\", \"scheme_management\"]] = df[[\"permit\", \"public_meeting\", \"scheme_management\"]].replace(-1, np.NaN)\n",
    "\n",
    "#--- Data categorical without permit, public_meeting, scheme_management\n",
    "obj_df2 = df.select_dtypes(include=['object']).copy()\n",
    "#print obj_df2.dtypes\n",
    "#print obj_df2.isnull().sum() #check missing values\n",
    "\n",
    "#--- Encoder data object lainnya\n",
    "obj_df_encoder = obj_df2.apply(LabelEncoder().fit_transform)\n",
    "#print obj_df_encoder.dtypes \n",
    "df.update(obj_df_encoder)\n",
    "#print df.head()\n",
    "\n",
    "#--- Mengisi missing values untuk data categorical\n",
    "obj_df_array = df[list(obj_df)].values\n",
    "#obj_df_array = obj_df_array.astype(float)\n",
    "#print obj_df_array[1:3,:]\n",
    "\n",
    "imp_modus = Imputer(missing_values=np.nan, strategy='most_frequent')\n",
    "transformed_obj_df = imp_modus.fit(obj_df_array)\n",
    "#print (imp_modus.transform(obj_df_array))\n",
    "\n",
    "df[list(obj_df)] = imp_modus.transform(obj_df_array)\n",
    "#print df.values[1:3,:]\n",
    "\n",
    "print '\\n===== Jumlah Missing Values di tiap atribut setelah handling Categorical ====='\n",
    "print df.isnull().sum()\n",
    "\n",
    "#print df.dtypes\n",
    "\n",
    "#--- Missing values handling for longitude, latitude, region_code, district_code, and construction_year with median\n",
    "dy_header = [\"longitude\", \"district_code\", \"construction_year\"]\n",
    "dy_array = df[list(dy_header)].values\n",
    "\n",
    "imp_median = Imputer(missing_values=np.nan, strategy='median')\n",
    "transformed_dy_array = imp_modus.fit(dy_array)\n",
    "df[list(dy_header)] = imp_modus.transform(dy_array)\n",
    "\n",
    "print '\\n===== Jumlah Missing Values di tiap atribut setelah handling longitude, latitude, region_code, district_code, and construction_year ====='\n",
    "print df.isnull().sum()\n",
    "\n",
    "#--- Missing values handling for other numeric data with mean\n",
    "nd_header = [\"amount_tsh\", \"gps_height\", \"population\"]\n",
    "nd_array = df[list(nd_header)].values\n",
    "\n",
    "imp_mean = Imputer(missing_values=np.nan, strategy='mean')\n",
    "transformed_nd_array = imp_modus.fit(nd_array)\n",
    "df[list(nd_header)] = imp_modus.transform(nd_array)\n",
    "\n",
    "print '\\n===== Jumlah Missing Values Final ====='\n",
    "print df.isnull().sum()\n",
    "\n",
    "print '\\n===== Data setelah Missing Value Handling ======'\n",
    "print df.head()\n",
    "\n",
    "#print target_label.isnull().sum()\n",
    "#print len(target_label)\n",
    "#print len(df)\n",
    "\n",
    "#--- Encoder data target\n",
    "print \"\\n===== Target Label =====\"\n",
    "print target_label[\"status_group\"].value_counts()\n",
    "\n",
    "target_label[[\"status_group\"]] = target_label[[\"status_group\"]].astype('category')\n",
    "target_label[\"status_group\"] = target_label[\"status_group\"].cat.codes\n",
    "\n",
    "print target_label[\"status_group\"].value_counts()\n",
    "\n",
    "\n",
    "####### Separate Data Training and Testing #######\n",
    "\n",
    "X = df.values[:,1:]\n",
    "y = target_label.values[:,1:]\n",
    "\n",
    "y = y.ravel()\n",
    "\n",
    "print \"Matrix X =\", X\n",
    "print \"Matrix y (target label) =\", y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Bagging"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "kf = KFold(n_splits=50, random_state=2, shuffle=False)\n",
      "\n",
      "GROUP K : 1\n",
      "Score ANN var1 = 0.5395622895622896\n",
      "Confusion Matrix ANN var1\n",
      "[[641   0   0]\n",
      " [ 90   0   0]\n",
      " [457   0   0]]\n",
      "Score ANN var2 = 0.5395622895622896\n",
      "Confusion Matrix ANN var2\n",
      "[[641   0   0]\n",
      " [ 90   0   0]\n",
      " [457   0   0]]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Gries\\Anaconda2\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:564: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
      "  % self.max_iter, ConvergenceWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Score ANN var3 = 0.5395622895622896\n",
      "Confusion Matrix ANN var3\n",
      "[[641   0   0]\n",
      " [ 90   0   0]\n",
      " [457   0   0]]\n",
      "\n",
      "GROUP K : 2\n",
      "Score ANN var1 = 0.5193602693602694\n",
      "Confusion Matrix ANN var1\n",
      "[[617   0   0]\n",
      " [ 92   0   0]\n",
      " [479   0   0]]\n",
      "Score ANN var2 = 0.5193602693602694\n",
      "Confusion Matrix ANN var2\n",
      "[[617   0   0]\n",
      " [ 92   0   0]\n",
      " [479   0   0]]\n",
      "Score ANN var3 = 0.5193602693602694\n",
      "Confusion Matrix ANN var3\n",
      "[[617   0   0]\n",
      " [ 92   0   0]\n",
      " [479   0   0]]\n",
      "\n",
      "GROUP K : 3\n",
      "Score ANN var1 = 0.5462962962962963\n",
      "Confusion Matrix ANN var1\n",
      "[[649   0   0]\n",
      " [ 78   0   0]\n",
      " [461   0   0]]\n",
      "Score ANN var2 = 0.5462962962962963\n",
      "Confusion Matrix ANN var2\n",
      "[[649   0   0]\n",
      " [ 78   0   0]\n",
      " [461   0   0]]\n",
      "Score ANN var3 = 0.5462962962962963\n",
      "Confusion Matrix ANN var3\n",
      "[[649   0   0]\n",
      " [ 78   0   0]\n",
      " [461   0   0]]\n",
      "\n",
      "GROUP K : 4\n",
      "Score ANN var1 = 0.5631313131313131\n",
      "Confusion Matrix ANN var1\n",
      "[[669   0   0]\n",
      " [ 86   0   0]\n",
      " [433   0   0]]\n",
      "Score ANN var2 = 0.5631313131313131\n",
      "Confusion Matrix ANN var2\n",
      "[[669   0   0]\n",
      " [ 86   0   0]\n",
      " [433   0   0]]\n",
      "Score ANN var3 = 0.5631313131313131\n",
      "Confusion Matrix ANN var3\n",
      "[[669   0   0]\n",
      " [ 86   0   0]\n",
      " [433   0   0]]\n",
      "\n",
      "GROUP K : 5\n",
      "Score ANN var1 = 0.5572390572390572\n",
      "Confusion Matrix ANN var1\n",
      "[[662   0   0]\n",
      " [100   0   0]\n",
      " [426   0   0]]\n",
      "Score ANN var2 = 0.5572390572390572\n",
      "Confusion Matrix ANN var2\n",
      "[[662   0   0]\n",
      " [100   0   0]\n",
      " [426   0   0]]\n",
      "Score ANN var3 = 0.5572390572390572\n",
      "Confusion Matrix ANN var3\n",
      "[[662   0   0]\n",
      " [100   0   0]\n",
      " [426   0   0]]\n",
      "\n",
      "GROUP K : 6\n",
      "Score ANN var1 = 0.5395622895622896\n",
      "Confusion Matrix ANN var1\n",
      "[[641   0   0]\n",
      " [ 79   0   0]\n",
      " [468   0   0]]\n",
      "Score ANN var2 = 0.5395622895622896\n",
      "Confusion Matrix ANN var2\n",
      "[[641   0   0]\n",
      " [ 79   0   0]\n",
      " [468   0   0]]\n",
      "Score ANN var3 = 0.5395622895622896\n",
      "Confusion Matrix ANN var3\n",
      "[[641   0   0]\n",
      " [ 79   0   0]\n",
      " [468   0   0]]\n",
      "\n",
      "GROUP K : 7\n",
      "Score ANN var1 = 0.5294612794612794\n",
      "Confusion Matrix ANN var1\n",
      "[[629   0   0]\n",
      " [108   0   0]\n",
      " [451   0   0]]\n",
      "Score ANN var2 = 0.5294612794612794\n",
      "Confusion Matrix ANN var2\n",
      "[[629   0   0]\n",
      " [108   0   0]\n",
      " [451   0   0]]\n",
      "Score ANN var3 = 0.5294612794612794\n",
      "Confusion Matrix ANN var3\n",
      "[[629   0   0]\n",
      " [108   0   0]\n",
      " [451   0   0]]\n",
      "\n",
      "GROUP K : 8\n",
      "Score ANN var1 = 0.5505050505050505\n",
      "Confusion Matrix ANN var1\n",
      "[[654   0   0]\n",
      " [ 84   0   0]\n",
      " [450   0   0]]\n",
      "Score ANN var2 = 0.5505050505050505\n",
      "Confusion Matrix ANN var2\n",
      "[[654   0   0]\n",
      " [ 84   0   0]\n",
      " [450   0   0]]\n",
      "Score ANN var3 = 0.5505050505050505\n",
      "Confusion Matrix ANN var3\n",
      "[[654   0   0]\n",
      " [ 84   0   0]\n",
      " [450   0   0]]\n",
      "\n",
      "GROUP K : 9\n",
      "Score ANN var1 = 0.5429292929292929\n",
      "Confusion Matrix ANN var1\n",
      "[[645   0   0]\n",
      " [ 76   0   0]\n",
      " [467   0   0]]\n",
      "Score ANN var2 = 0.5429292929292929\n",
      "Confusion Matrix ANN var2\n",
      "[[645   0   0]\n",
      " [ 76   0   0]\n",
      " [467   0   0]]\n",
      "Score ANN var3 = 0.5429292929292929\n",
      "Confusion Matrix ANN var3\n",
      "[[645   0   0]\n",
      " [ 76   0   0]\n",
      " [467   0   0]]\n",
      "\n",
      "GROUP K : 10\n",
      "Score ANN var1 = 0.5252525252525253\n",
      "Confusion Matrix ANN var1\n",
      "[[624   0   0]\n",
      " [ 68   0   0]\n",
      " [496   0   0]]\n",
      "Score ANN var2 = 0.5252525252525253\n",
      "Confusion Matrix ANN var2\n",
      "[[624   0   0]\n",
      " [ 68   0   0]\n",
      " [496   0   0]]\n",
      "Score ANN var3 = 0.5252525252525253\n",
      "Confusion Matrix ANN var3\n",
      "[[624   0   0]\n",
      " [ 68   0   0]\n",
      " [496   0   0]]\n",
      "\n",
      "GROUP K : 11\n",
      "Score ANN var1 = 0.5538720538720538\n",
      "Confusion Matrix ANN var1\n",
      "[[658   0   0]\n",
      " [ 93   0   0]\n",
      " [437   0   0]]\n",
      "Score ANN var2 = 0.5538720538720538\n",
      "Confusion Matrix ANN var2\n",
      "[[658   0   0]\n",
      " [ 93   0   0]\n",
      " [437   0   0]]\n",
      "Score ANN var3 = 0.5538720538720538\n",
      "Confusion Matrix ANN var3\n",
      "[[658   0   0]\n",
      " [ 93   0   0]\n",
      " [437   0   0]]\n",
      "\n",
      "GROUP K : 12\n",
      "Score ANN var1 = 0.5547138047138047\n",
      "Confusion Matrix ANN var1\n",
      "[[659   0   0]\n",
      " [ 81   0   0]\n",
      " [448   0   0]]\n",
      "Score ANN var2 = 0.5547138047138047\n",
      "Confusion Matrix ANN var2\n",
      "[[659   0   0]\n",
      " [ 81   0   0]\n",
      " [448   0   0]]\n",
      "Score ANN var3 = 0.5547138047138047\n",
      "Confusion Matrix ANN var3\n",
      "[[659   0   0]\n",
      " [ 81   0   0]\n",
      " [448   0   0]]\n",
      "\n",
      "GROUP K : 13\n",
      "Score ANN var1 = 0.5471380471380471\n",
      "Confusion Matrix ANN var1\n",
      "[[650   0   0]\n",
      " [ 81   0   0]\n",
      " [457   0   0]]\n",
      "Score ANN var2 = 0.5471380471380471\n",
      "Confusion Matrix ANN var2\n",
      "[[650   0   0]\n",
      " [ 81   0   0]\n",
      " [457   0   0]]\n",
      "Score ANN var3 = 0.5471380471380471\n",
      "Confusion Matrix ANN var3\n",
      "[[650   0   0]\n",
      " [ 81   0   0]\n",
      " [457   0   0]]\n",
      "\n",
      "GROUP K : 14\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import KFold\n",
    "\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "from sklearn.metrics import confusion_matrix\n",
    "\n",
    "def ANNvar (Ktrain_idx, Ktest_idx):\n",
    "    ###### 3 Model ANN dengan Gradient Descent ######\n",
    "    # Fungsi aktivasi sigmoid, hidden layer = [50,100,200], learning rate = 0.0000001, max_iter = 100\n",
    "    nn1 = MLPClassifier(activation='logistic',\n",
    "                        solver='adam',\n",
    "                        hidden_layer_sizes=(50,1000),\n",
    "                        learning_rate_init=0.0000001,\n",
    "                        max_iter = 100)\n",
    "    nn1.fit(X[Ktrain_idx], y[Ktrain_idx])\n",
    "    score_nn1 = nn1.score(X[Ktest_idx], y[Ktest_idx])    \n",
    "    print \"Score ANN var1 =\", score_nn1\n",
    "    predict_nn1 = nn1.predict(X[Ktest_idx])\n",
    "    print \"Confusion Matrix ANN var1\"\n",
    "    print confusion_matrix((y[Ktest_idx]), predict_nn1, labels=[0, 1, 2])\n",
    "\n",
    "    nn2 = MLPClassifier(activation='logistic',\n",
    "                        solver='adam',\n",
    "                        hidden_layer_sizes=(100,1000),\n",
    "                        learning_rate_init=0.0000001,\n",
    "                        max_iter = 100)\n",
    "    nn2.fit(X[Ktrain_idx], y[Ktrain_idx])\n",
    "    score_nn2 = nn2.score(X[Ktest_idx], y[Ktest_idx])    \n",
    "    print \"Score ANN var2 =\", score_nn2\n",
    "    predict_nn2 = nn2.predict(X[Ktest_idx])\n",
    "    print \"Confusion Matrix ANN var2\"\n",
    "    print confusion_matrix((y[Ktest_idx]), predict_nn2, labels=[0, 1, 2])\n",
    "\n",
    "    nn3 = MLPClassifier(activation='logistic',\n",
    "                        solver='adam',\n",
    "                        hidden_layer_sizes=(200,1000),\n",
    "                        learning_rate_init=0.0000001,\n",
    "                        max_iter = 100)\n",
    "    nn3.fit(X[Ktrain_idx], y[Ktrain_idx])\n",
    "    score_nn3 = nn3.score(X[Ktest_idx], y[Ktest_idx])    \n",
    "    print \"Score ANN var3 =\", score_nn3\n",
    "    predict_nn3 = nn3.predict(X[Ktest_idx])\n",
    "    print \"Confusion Matrix ANN var3\"\n",
    "    print confusion_matrix((y[Ktest_idx]), predict_nn3, labels=[0, 1, 2])\n",
    "    \n",
    "    return score_nn1, score_nn2, score_nn3\n",
    "\n",
    "\n",
    "#--- Split data dengan K fold dengan k = 50\n",
    "\n",
    "seed = 2\n",
    "\n",
    "kf = KFold(n_splits=50, random_state=seed)\n",
    "kf.get_n_splits(X, y)\n",
    "print 'kf =', kf\n",
    "\n",
    "score_nn1_list =[]\n",
    "score_nn2_list =[]\n",
    "score_nn3_list =[]\n",
    "\n",
    "group = 0\n",
    "for train_index, test_index in kf.split(X, y):\n",
    "    #print(\"TRAIN:\", train_index, \"TEST:\", test_index)\n",
    "    group = group+1\n",
    "    print \"\\nGROUP K :\", group\n",
    "    \n",
    "    score_nn1, score_nn2, score_nn3 = ANNvar(train_index,test_index)\n",
    "    \n",
    "    score_nn1_list.append(score_nn1)\n",
    "    score_nn2_list.append(score_nn2)\n",
    "    score_nn3_list.append(score_nn3)\n",
    "\n",
    "\n",
    "avg_nn1 = sum(score_nn1_list)/len(score_nn1_list)\n",
    "print \"Score Bagging ANN1 =\", avg_nn1\n",
    "\n",
    "avg_nn2 = sum(score_nn2_list)/len(score_nn2_list)\n",
    "print \"Score Bagging ANN2 =\", avg_nn2\n",
    "\n",
    "avg_nn3 = sum(score_nn3_list)/len(score_nn3_list)\n",
    "print \"Score Bagging ANN3 =\", avg_nn3\n",
    "\n",
    "    \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. Boosting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "kf = KFold(n_splits=50, random_state=2, shuffle=False)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import AdaBoostClassifier\n",
    "from sklearn.model_selection import KFold\n",
    "from sklearn import model_selection\n",
    "\n",
    "seed = 2\n",
    "kf = KFold(n_splits=50, random_state=seed)\n",
    "kf.get_n_splits(X, y)\n",
    "print 'kf =', kf\n",
    "\n",
    "ada = AdaBoostClassifier(n_estimators=len(X),learning_rate=1,random_state=seed)\n",
    "results_ada = model_selection.cross_val_score(ada, X, y, cv=kf)\n",
    "print 'Score Boosting =', (results_ada.mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. Hasil akurasi pelatihan dan pengujian pada soal 1 dan 2 belum dapat diketahui, tetapi jika antara metode bagging dan boosting dengan 3 variasi ANN training diaplikasikan pada data lain yang lebih sedikit, misalnya dataset Iris maka hasilnya menujukan hasil akurasi dengan boosting lebih baik."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4. Analisis tentang hasil perbandingan yang diperoleh pada soal 3 adalah mengapa boosting lebih baik, karena dalam boosting iterasi sejumlah K data yang di training selanjutnya merupakan data dari pediksi yang salah digabungkan dengan data yang baru dan berulang selanjutnya sampai K, dengan kata lain data yang mempunyai prediksi yang salah dilatih lebih lanjut. Oleh karena itu hasil dengan boosting lebih baik dari bagging."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}