udacity/python/Supervised Learning/Training and Tuning/Grid_Search_Lab.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Improving a model with Grid Search\n",
    "\n",
    "In this mini-lab, we'll fit a decision tree model to some sample data. This initial model will overfit heavily. Then we'll use Grid Search to find better parameters for this model, to reduce the overfitting.\n",
    "\n",
    "First, some imports."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Reading and plotting the data\n",
    "Now, a function that will help us read the csv file, and plot the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJztnX2QHGd54H/P2lp9tOZibC+xWXtjVtok2NT5sFUGzJIzMSRmF3C0MVknSjA5pVQkUJfN3pW0WtcRgu2yo6tQ2UhJiLXxxcopZC8gwAkDWOBQ3rmLnUgqGWMMSN4jZd26YmMS7+z6A4ye+2N6pNlRz0zPTH/386vq2pnp3n6feWfmfbqfT1FVDMMwjPzRE7cAhmEYRjyYAjAMw8gppgAMwzByiikAwzCMnGIKwDAMI6eYAjAMw8gppgAMwzByiikAwzCMnNK1AhCRy0Xk70XkSRF5QkR+2+MYEZE/EpGTIvJ1Ebmm23ENwzCM7jg/gHO8CvwXVT0mIgXgqIgcVtVv1hzzbmDI3d4M/Kn7tykXX3yxXnHFFQGIaBiGkQ+OHj36PVXt83Ns1wpAVZ8BnnEfl0XkSaAfqFUANwMHtFJ34hERuUBELnX/tyFXXHEFR44c6VZEwzCM3CAi/+z32EB9ACJyBfAm4NG6Xf3A0zXPT7mvGYZhGDERmAIQkY3AZ4AJVV2q3+3xL55V6ERkh4gcEZEjzz33XFDiGYZhGHUEogBEZA2Vxf+gqh7yOOQUcHnN88uARa9zqeq9qrpFVbf09fkyYxmGYRgdEEQUkAB/Djypqp9ocNgDwAfcaKC3AC+0sv8bhmEY4RJEFNDbgF8DHheR4+5r08AAgKp+EigCI8BJ4EXg1wMY1zAMw+iCIKKASnjb+GuPUeDD3Y5lGIZhBIdlAhuGYeQUUwCGYRg5xRSAYRhGTjEFYBiGkVNMARiGYeSUIMJADSMSyuUyc3NznDixwNDQIOPj4xQKhbjFMozUYncARioolUr0929iYqLInj0OExNF+vs3USqV4hbNMFKL3QEYiadcLjMyMka5fBB4FwArKwCHGRkZY3FxgY0bN8YpomGkErsDMBLP3Nwcp08PU138z/IuTp8eZm5uLg6xDCP1mAIwEs+JEwusrFzruW9l5RpOnlyIWCLDyAZmAsoYWXSUDg0N4jhF1+yzGsc5xubNo9ELZRgZQCplepLJli1b1DqC+adUKjEyMsbp08OsrFyL4xylp6dEsXiI4eHhuMXrmHK5TH//plU+gAqHKRS2mQ/AMGoQkaOqusXPsXYHkBGy7CgtFAoUi4dqlNs1OM6xM8otre/LMOLGfAAZIeuO0uHhYRYXF5iZGWVq6iVmZkZZXFxI9Z2NYcSN3QFkhDw4Sjdu3Mj27dvjFsMwMoPdAWSEiqP0qOe+iqN0MGKJDMNIOqYAMsL4+Dg9PSXgcN2ew/T0lBgfH49DLMMwEoyZgDKCOUoNw2iXQBSAiNwHvAd4VlXf6LH/BuDzwP91Xzqkqh8PYmzjLFVH6dzcHCdPLrB58yjj4wds8TcMw5Og7gD+AtgHHGhyzLyqvieg8YwGmKPUMAy/BOIDUNWHge8HcS7DMAwjGqJ0Ar9VRB4TkS+KyFURjmsYhmF4EJUT+BjwE6q6LCIjwOeAIa8DRWQHsANgYGAgIvEMwzDyRyR3AKq6pKrL7uMisEZELm5w7L2qukVVt/T19UUhnmEYRi6JRAGIyCUiIu7j69xxn49ibMMwDMOboMJAPwXcAFwsIqeA3wXWAKjqJ4FbgN8UkVeBl4BbNcllSA3DMHJAIApAVX+5xf59VMJEDSOVZLHPgmFYKQjDaIE1pDeyipWCMIwmZLnPgmHYHYBhNCHrfRaMfGMKwDCakIc+C0Z+MROQYTSh24b05jw2kow1hTd8kdeFrJuG9KVSqaY897U4ztEz5bmtlaURFu00hTcFYLQk7wvZ6ve/us9Co/ffjeIwjG5oRwGYCchoikXBdNZnwY/zOKtlu/N6t5hGTAEYTcnzQlZLu30W8uo8Pvduscjk5HRu7hbThikAoyl5Xci6pVvncRqxu8X0YWGgRlMqC9lRz32VhWwwYonSwfj4OD09JeBw3Z7D9PSUGB8fj0OsULGcifRhdwBGU8bHx5mcnKaykK12ZlYWsmZdQPNLoVCgWDzU0Hnc7ZVw1c6+cOIEg0NDibCzJ/1u0XwT52IKwGhK2AtZlunEeeyHUqnE2MgIw6dPc+3KCkXHYXpykkPFYqx29iSbvcw34Y2FgRq+WF5erlnIKldPtvhHT7lcZlN/PwfL5br7MdhWKLCwuBjb55LU0NekyhUWFgZqBE67UTBGOMzNzTF8+rSHlR2GT5+ONSorqXeLFsnWGFMAhpEiFk6c4FovGwtwzcoKCydPRizRasIye3VD0n0TcWIKwDC6JErn4uDQEEXHwcvQfsxxGN28OZRx2yFpd4tJ9k3ETSA+ABG5D3gP8KyqvtFjvwAzwAjwIvBBVT3W6rzmAzCSTtRlMpLsA0gq5gNoTFB3AH9BpeVjo5jAdwND7vZm4E/dv4aRWuJIfCoUChwqFs9EAV2zssIxx6HU08OhYjFTC1lQJNU3kQSC6gn8sIhc0eSQm4EDbiP4R0TkAhG5VFWfCWJ8w4iDuJyLw8PDLCwuVvIATp5kdPNmDrSIysp7DHzVN3H//ffzd39XBGB09KNcffXVMUsWL1H5APqBp2uen3JfMwVgpJY4nYvt2NktBr7C8ePH2b37987Mw/z8Q0xPfzx381BLVApAPF7zdD6IyA5gB8DAwECYMhlGV6TBuWj1eSrYPHgTVS2gU8DlNc8vAxa9DlTVe1V1i6pu6evri0Q4w+iENNT7sfo8FWwevIlKATwAfEAqvAV4wez/RtqpOhcLhW04zhhwJ44zRqGwLTHORYuBr2Dz4E0gJiAR+RRwA3CxiJwCfhdYA6CqnwSKVEJAT1IJA/31IMY1jLhJYuJTLWkwU0WBzYM3VgvIMDJM3mLgG5GnebBaQIZhABYDX8XmwRu7AzCMHGDVXCvkYR7auQMwBWCklriaoiSxGYthVDEFYGSe+qYoR2vKIYSZ1BPXuHFjSi89mAJIIXlP1W+HuAqi5bUQW16VXlppRwFYU/gEUCqV6O/fxMREkT17HCYmivT3b6JUKsUtWiLx0xQlS+PGSblcZmxkhIPlModWVrgdOLSywkH39eXl5bhFNLrAFEDM1Kaor6wcAm5nZeUQ5fJBRkbG7AfmQVxNUZLejCUM8qj08oQpgJjpJkW9XC4zOzvLrl3TzM7OUi6XQ5U1KQwODXHUcTz3HXMcBkNqihLXuHGSR6WXJ0wBxEynKep5NhuNj49T6unxqMADpZ6e0GrwxDVunORR6eUJUwAxU0lRP+q5r5KiPnjO63k3G1WbomwrFBhzHO4ExhyHbe7rYTli4xo3TvKo9PKERQHFTCcp6rOzs0xMFN3FfzWOM8bMzGiierKGRTWpZ+HkSQY3b44sqSeuceOiNgqovgOZRQElDysFkSI6SVG3yoYV4mo+nrSm52HjpwOZhTGnE1MACaDdipJW2TDfxJGU1UzpWcex9JIZE1Car0DalT2tlQ3T/BklhaQlZaX1u5hl2jEBoaqJ3a699lr1w/z8vBYKfeo4WxXuVMfZqoVCn87Pz/v6/zjpVPbV/3dH4t9zmj+jpLC0tKR9hYI+CKo124OgfYWClsvlyGXav3+/+5nqOZvjbNXZ2dlI5VlaWtL9+/frzp27df/+/bq0tBTp+EkAOKI+19jYF/lmmx8FsLS0pIVCn8KDdV/AB7VQ6IvlR+GXbmUvl8s6OzurU1PTOjs7m9j3msTPqLpQ7N65MzULxf79+3Wr45y70oJudZzIF1tV1Z07dyvc6akA4A6dmpqOTBa7yKjQjgJIvQ/ATyJVUh123cqeFmdkN+8zDHt3vRml6DhMT04mPqoliUlZUfqjmpkQrel7Z6Q+DyDNETFplr0dukl229TfT3FiAmfPHooTE2zq7+8q2S3NtW2SmJQ1Pj5OT08JPDIFenpKgeUJtEp8tKbvnRGIAhCRm0Tk2yJyUkSmPPZ/UESeE5Hj7vYbQYwLnSVSJYU0y94OnSa7hbFQp7m2TRKTsqphzIXCNhxnDLgTxxmjUNgWWKctP4mPebmYCpquFYCInAf8MfBu4Ergl0XkSo9D51T1P7jbbLfjVonqCiQM0ix7O3TyPsNaqJNoRvFLUjORq2HMMzOjTE29xMzMKIuLC4GZ0/xc3eflYipogvABXAecVNUFABH5a+Bm4JsBnLslae71mWbZ26GT9xnWQj04NETRcfAyWh9zHEYTVNvGy//hJykrDsL0R/m5up+enmJycprKRcbqcNTKRcaBUGRLO0EogH7g6Zrnp4A3exz3iyLyM8B3gN9R1ac9jumIdhOpkkSaZW+Hdt9nWAv1+Pg405OTHstExYxyoMFdV9TJV60c1Wlw/geFH0dzXi6mAsdvuFCjDXg/MFvz/NeAvXXHXASsdR9/CHioyfl2AEeAIwMDAyEFShlJJ8yY9/n5ee0rFHSr4+gdbghlX6HQMFyw9vg7fRzfLUmM94+TdsKI0xIaHSZEmQcAvBX4cs3z3cDuJsefB7zg59x+E8GMbNLuQt0O1YViemqq6UIRx2KcxHj/uElb4mOctKMAgjAB/RMwJCKvB/4fcCvwK7UHiMilqvqM+/R9wJMBjGsknG7NJmHau/3arP04o4M2x6TZUR0WeTGVRk3XCkBVXxWRjwBfpnJ1f5+qPiEiH6eiiR4A/rOIvA94Ffg+8MFuxzWSTVDJVnEnu8WxGKfJUR0lcX8XskggmcCqWgSKda99tObxbiqmISMH1Mbwn7lyXlnhMDA2MsLC4mJqrtziWIw7dVQbRrukPhPYSB5pSLby2085juSrpMb7G9kj9bWAjOSRdBt2O/Xrq4txo45YYS3GSY33N7KFKQCjKZ04cpNsw+6kaFhci7HZvI2wyUxDGCN4Om0+Ui6X2dTfv9oHQMVssq1QiNUH0Kyf8oYN7+WWWy6k/5JLIuu0ZWSDOLq0NcJ6Ahtd040jNy6ziR8alxUocfrFB3n+U8pP/vCHsZaITtJiYrQmreXFgfQ3hDHCIYhkJL/JVlHi3cFqSdeRjMzbqLOOje5IYtY2bSSCWRSQ4UkQjtyqDfuuu+9m+/btiXBgelcmnePt/DD2qKU09yrIK2mIeGuGKQDDkyQ2HwkCr/r1a8/fw3/kZc/jo4xaSupi4jdkNo8kPeKtFaYADE+S2HwkKOrr14//yls5kgBll8TFpFUnrryT+gslv7aiOLZ2fADVJt87d+5OTZPvpBNmMbYksbS0pBdu2OBpx71wwwbfdtxuG83v3btX39Tbq7tB94MuxVwErp0qnHkl7T6A2Bf5ZptfBbC6UuCdVikwQJLoyA2apaUlXb/+Al3Hen0XFWX3Lhxdx3pdv/4CX++5W+ft/Py89m3cqD8Hlf8H7QOdj3Ex8XaYVzbH2ZrLqqReJO1CqR0FkPow0E4Sewz/tJOMlNbwxbm5OXp63sEKBzjMHF/hJMpmYByn5wMtK352W/vozP8vL5+TN/FuYN3GjXw2hvBZ67PrjzRnbadeAfjpF2rZlOGT5ljoswvdRmA7tamRfha6bktGN/v/G9au5aZ77ollDv104soz1QueEycWGBoaTM0FTy2pdwLbVUr8pD18sduG4t06b5v9/5tfeYXFU6ea/n9YeIfMwtk+u+kNBOiWrDjHU68Auv3xGt2T1PBFv3S70HUbCZLUSBKvkFnHGaNQ2JbrPru1ZudKSZHbWVk5RLl8kJGRscRf8NSSegVgVynxE2b4YjUGfXrXrlBi0Ku38b/w3nexdu0YGza8F6+Frpkc3YbMJjnktj5kdmZmlMXFhcSb9cLEj9k5LaTeB1C9Sjlb3vcaHOcYPT2lXF+lRElY1T/D9ivUn//djsPXfvQVtm57De94x9mWg63k8Kp99OjatTysyod+8zcr4XZN8Fs7KS4nu1UlXU2mzM5+w4WabcBNwLeBk8CUx/61wJy7/1HgCj/nbScPoBquODU1HXm4Yt5zELqJhW40d2HGVy8tLenevXvVWbNGf6cu3r7+/O3IUS6XddeuXbp2zXrt7X2Twn9rKyS5Wcit1QhKDkkPjyXKPAAqfYCfAgaBXuAx4Mq6Y34L+KT7+FZgzs+501AMznIQKnQSC91s7oIoRtdMzvesXXtOvL3X+duRI6zEqSQmG+WZpCfItaMAgjABXQecVNUFABH5a+Bm4Js1x9wMfMx9/Glgn4iIK2xqsRyEs7QbC91q7nb8xq8G7lfwjNenYmsfAxaoBILWnr8d/0ZYIcndhpkawZIls3MQCqAfeLrm+SngzY2OUdVXReQF4CLgewGMHxuWg7CadmzFrebuueefZyFgv0LThZSKjXJ73fkHh4b4woYN8OKLLeUIyzacxBpBeafqHJ+bm+PkyQU2bz7rM0oTQSgA8Xit/srezzGVA0V2ADsABgYGupMsZDLlDIqYVnN30UX/xhfdyJj6q/WHRfjZl15ieteutpyhTRdSKncA1cibA27kzcDAAF9+8UVPOWqPg/ASp5LcYjPPZME5HkQY6Cng8prnlwGLjY4RkfOBHwO+73UyVb1XVbeo6pa+vr4AxAsPy0HonFZzd9VVb+BQsci2QoExx+FOYMxx+KX169HTp3loagpnzx6KExNs6u/3lYDTLN7+EeCLvb1scyNyqqGfv3rLLdwFbKNiJrqTij3zZuB/fvrTq674wgpJTnKYaJKwstUd4NdZ0GijchexALyes07gq+qO+TCrncD/y8+5k+4ETrozKGnUVsvcu3evbtx4Ucu5q42M2bdvn/Zt3NixM7SZM/Xf9fbqvn37Vp2j1gFcBp0FnXb/3uQeX89qx/YdgQUFJK3gWNKwYIyzEHU1UGAE+A6VaKDb3dc+DrzPfbwO+BsqYaD/CAz6OW/SFYBqeD/4rOEVxnjhhg26fv0FvucuiMigdhbS3Tt36p1esX6gd4CuXbPe8//CCknOQ2XWTrALsdW0owACSQRT1SJQrHvtozWPXwbeH8RYSSMrzqAwaVYtc9vGHn73nhs5dWqx5dwF4QxtJ1qpme39YRxe+eEOz2ivVrbhThO6smBzDgMLxuic1GcCJwH7YTanafSNKuvWrePuu+8683qjBTIoZ6jfz2t8fJzpyUlPB/A8PcDHOX36u20tMGmumppULBijc0wBGKHTzpV7swWy2YJcH5ETBNUSDaM33sibfwA/ww94GId5eniZIrCxrQWm274Bjc6Zxh4MQWJlq7vAr60oji0NPgCjNX5t934yXuNwhu7du1d7e9+kwpTCrEK5o9T/oLObrTxEBfMBrIa8tYQ0ko3fUgZ+F8ionaFBLTCtnMrTU1PnjNuox7CVh1iNBWOcpR0FYCYg4xyCNiv4rXbp11QUtc8lqNT/dnwYrXwFQZeHSLspyYIxOsSvpohjszuA6AnTrNDqyj2sAnBB0W14p9+r9upxnwPdD7rb/fu5muPavZtohpmSsgVmAjI6IW6zQtPxN27UvXv3pr7kth8fxv79+/Xt69ZpH5VqpbVVS9++bp3Ozs4Gpizj/syN4GlHAaS+I5gRHHG3dqyaiurLP9y6YQPLPzqfqamHUt1/Fc7mIYzOzPDS1BSjMzMsLC6uCgF98hvf4Osvv8xBoNJwsPL3IPD1l1/mW9/8ZmDlIeL+zI14MR+AcYYkVJ2sT9S68bLLODz1MV5a/iuyUnK7lQ/je//6r1yPV1oTXA889/zzvv0qrUjCZ27Eh90BGGdISnPy6gJ51913s3btWlTfThb6r/ql78ILeVuDfdcD337iCWZnZ7n66qtb3k20IimfuREPpgCMMySx6mQeszx/+qqr+Md16zz3/W/gkiNHzlRBPX78+BlluX379rbvhpL4mRvRYQogZpJUwraRDb62RHLU5LHk9vj4OP+wZo3nonwU+Evg0MoKB93M4uXl5Y7HSuJnbkSHVJzGyWTLli165MiRuMUIjVKpVBNbfi2Oc/RMbHmcdWGWl5fP2OAHN29mvElrx7Apl8v0929a1TqywmEKhW2p9AH4oTYP4JqVFf4PcISKM/hqKt3LFoAv9fay/ROf4MMf/nBX4yXpMze6Q0SOquoWX8eaAoiHZgtbb+8t/MEf3MVtt92WqmScdmgn8Wi1olydhJXlAmrVRfm+/ft57aOP8pfAcSqNaYaBa4F/AOZ7e/nCV7+a6bkw/GMKIAXMzs4yMVFkZeWQx9730Nu7yNq1pzK5yNVnuR6tiV5p9F6ri2Ely3MwV1eos7OzFCcmuH9lhU1UwkHri+FtKxQ6KiZnZA9TAClg165p9uxxqER513Mn8BJwQ+bMHOVymU39/asrYmKLWDOqc/ar5TLfpWIGqmfMcRidmbGy5EZbCsCcwDHRzLkJx4BBshjqaIlH7VN11N67Zg3e8VAWs290himAmGjWQBxKQCX8LmuhjpZ41BnDw8Pc84lP8MjatZ77LWbf6ISuFICIXCgih0XkhPv3NQ2O+5GIHHe3B7oZMytUK0wWCttYu/a9VMw+Y8A2Kjf5FTNI1kIdu0k8qobMTu/aFXvIbBzcdtttPNrbazH7RnD4LRrktQF7gCn38RTw+w2OW+7k/HkoBlcul3Xfvn26Zo2j8Durmo1ksaFFp8XHrGJlhTga4gRBtbdB2ov5pQHaKAbXlRNYRL4N3KCqz4jIpcDXVPWnPI5bVtW2PXtZdgLXk6dQx/oY92MtooA6cRynvb59M9IWs5/UfJesElkUkIj8m6peUPP8X1X1HDOQiLxKJYT5VeAeVf2cn/PnSQFAvkId21nEqmGQhzx8B17RL52EmRrhkNdEvjhpRwG0rAYqIl8BLvHY5RW/2IgBVV0UkUHgIRF5XFWfajDeDmAHwMDAQBtDpJ+oO13FSTvvtR3HcRiN12vJ8p1FGMzNzXH69DDNivnl5TufRFo6gVX1nar6Ro/t88C/uKYf3L/PNjjHovt3Afga8KYm492rqltUdUtfX18Hb8nIGu04jsMMMy2VSmzq76c4MYGzZ8+Zgmxp7EsQFXks5pcmuu0H8ABwG3CP+/fz9Qe4kUEvquorInIx8DYqzmMjBqpXsCdOLDA0NNj1FWwUV8Tj4+NMT05ymHMzYEs9PRyoiX4JK8w0jDuLMOcuKXcqlXyXolcbZDfCbTRymYwa/HqLvTbgIuCrwAn374Xu61uAWffx9cDjwGPu3+1+z5+HKKAomZ+f10KhTx1nq8Kd6jhbtVDo6ziCJMrIHL/RL2H1FQ76vGHOXZIippaWlrRQ6FN4sG7ashfhlhSwnsBGPUH/EOPoJduqqXyYcgXZhD3MuUtij9/VFx53dH3hYTSnHQVgmcA5wY8zrt3zRV3SobZTWKPmJ2HVtw+yc1aYc5fEUhvDw8MsLi4wMzPK1NRLzMyMsri4YBFZCcB6AueEoJ1xSS7pUN9XeHTzZg50GVLbjh+iFWHOXVI/lzxFuKUJUwA5IWhn3ODQEEXHweuExxyH0ZDr0rRycga94ATVhB3Cnbu4P5dmBB2AYASAX1tRHJv5AIIjCz6AKnE6Of34IVrRbO4u2rhR9+7dq7t37uyoZEISfQCqwQcgGI3BnMCGF0E74+KoS5PUBa5dvObugvXr9cING7pWbEmrF2SRQNHSjgKwhjA5I+hyE1HXpWm3LESSqZ271112Gb83NcXB5eVAGuUkqV5Qs+53jjPGzMxoaj6zWpJq0gq0FISRLYK2jUft3Euqk7MTaududnaWYdWm0TvtzHOSnK7NAxCu4jOfOcR3vvNUohZRaL7An1vgrsjk5HTqCtxZGKiRKoIMx0wSWVJs9TTuflcC/pCHHuphzx6HiYki/f2bElFao1Qq0d+/iYmJ4jmylctlRkbGKJcPunc1t7Oycohy+SAjI2MsLy/HLb5vTAEYiae2EczLL7/MwyKZa4qSVcUGjbrflYGbgUO88srfkqRFtNUCf//99weaUxMnZgIyEk19aeeHHAdV5ZfWr+cdPT1dhWMmCT95Bkmp79Mu1e53tf0uensP8YMfbCGJVUJbJU1+4QtfZGXles//TVuBO1MARmJpVoBt28aN3HjPPSyeOhVIolfctMozOH78+CpFWHQcpicnU9PjoJoNXA1AOH78Ur70pWQuoq2SJkX+Acc5mokCd6YAjMTStKyBKuvWreOuu++OQ7RQaJTBrKrndkQLsMdBVNQ7vefnk1kltFXS5OjoCKXS74HH/VpPT4nx8QMRSRoAfuNF49gsDyDfBFmALc2EVeE0TpKcG+BHtiQXuKONPAC7AzASS5LLGkRJFiOEvPwCtX2w47yj8SNbvUlr8+ZRxscPnJE7qTkC9VgimJFYOmkGn0WylPxWT5L7YHcq27k5AkfPKI8o/DWRNYUPG1MARm0UUL1jNGnOz7CidEwRpodyuUx//ybK5YPU+wcKhW0sLi6E/lmZAjAyRZLKGjSiPlz1aACKqtaMAD/if/zJnzCsmnhFmGeSUPYislIQIvJ+4GPAG4DrVNVztRaRm4AZ4DwqrSLv6WbcrBC3nTDu8f2SpLIGXoTRL9jLjCCyjp/8yHZeEslE6GsWCbrvRuj49RZ7bVQW/p8CvgZsaXDMecBTwCDQS6U38JV+zp/lKKC4y+PGPX6WCDpKJ4gImaWlJd2/f7/u3Lm7o7LSRmfs37/f/U2d+3VwnK2RRGwRdTnoFgrgrcCXa57vBnb7OW9WFUDcIXBxj581gg5X7XYRyZNyT5qiS8Jvqx0FEEUtoH7g6Zrnp9zXckvQ/XnTNn4t1To/u3ZNMzs7S7lcjmzsoAi6jk83ZoQsFSprRbOCbXFRDSEtFLbhOGPAnTjOGIXCttjDW71oqQBE5Csi8g2P7WafY4jHaw09zyKyQ0SOiMiR5557zucQ6SJuO2Hc41dJ4g+4E8bHxyn19ARWoK5x9cxqluxgw/9NknIPkyQrumqOwMzMKFNTLzEzM8ri4kIinfUtncCq+s4uxzgFXF7z/DJgscl49wL3QiUKqMuxE0nQ/XnTNj6s/gFXF6uKPIcZGRmLJFwuKILsFwwVhTI5OU0npQaSotzDxo9EZcJ3AAAKPUlEQVSiizNwIOmBC1WiMAH9EzAkIq8XkV7gVuCBCMZNLN7lceHsDzzcksZxjw/Zu1Kt1vEZnZnhpakpRmdmWFhc7OiqrxszQjd3D2kiL4oubLoNA90K7AX6gC+IyHFV/XkReR2VcM8RVX1VRD4CfJlKRNB9qvpE15KnmLjT4OMeH7L5Aw7yqq9VqYFGdHP3kCaScBebBSwRLEbiToOPc/woE2bSku8QFKtzCFYr9yTaoTshCRm3ScUygY3EE9UPOO66LHER98VFFCRB0SWxSY8pACMVhP0DtqvE7BOnoguj/EcQmAIwUkOYP+Ak1GUxskmSC/RFVgvIMLolzHC5LDqajWTQtFvd6dOxh6H6xRSAkSiCdNhapIgRFllp0hNFHoBh+CLozOAk5DsY2STo8h9xYT4AwxdhRzuE5bBNQqRImshbyGynZMUHYAogBJIYGtYNUUQ7hOmwzUNIZBDkNWS2U5Larc6cwDFSv1gWHYfpycnYvxSdEkazEy/CdNimpS5LnGSpNlNUVMt/VLvVpbFJjymAAIlqsYySqKIdzGEbL0kvrpZU0n5xYU7gAPGzWKaNqKIdzGEbL1GFzFb7P0zv2pXa/g9ZwhRAgGQlNKyWqKId0tZII2tEUUW0VCqxqb+f4sQEzp49FCcm2NTfn7r+D5nCb+uwOLa0tYQMujdsElhaWtK+QkEfrHs/D4L2FQqBt7grl8s6OzurU1PTOjs7a+0pIyLsVoZRf4/yDFH3BA5rS5sCCOtLHnff0/n5ee0rFHSr4+gdrjLrKxQy2WM2z6zuJXxHoL2Es3hxlFTaUQDmBA6QoDtDgVdoXpHJyelIQ/OyEO1gtKbTHgR+yKJ5NAuYAgiYIBfLJIXmpT3awfBHWJ/z4NAQRcfBK8zrmOMwmpLM2axhiWAJxqpZGlkhyZmzWcMSwTKCVbM0skIY5lGje7rtCfx+4GPAG4DrVNXzcl1EvguUgR8Br/rVTnnHkqOMLGG+pOTRlQlIRN4AnAb+DPivLRTAFlX9Xjvnz7sJyDpaGYbRLpGZgFT1SXfAbk5jNKCaHNWomqUt/oZhdENUPgAFHhQRBf5MVe+NaNzUE2ZonmEY+aalAhCRrwCXeOy6XVU/73Oct6nqooi8FjgsIt9S1YcbjLcD2AEwMDDg8/TZxkIwDcMIg5YKQFXf2e0gqrro/n1WRD4LXAd4KgD37uBeqPgAuh3bMOLCmqsYSSf0YnAi4ohIofoY+DngG2GPaxhxEnR7S8MIg27DQLcCe4E+4AsiclxVf15EXgfMquoI8OPAZ11H8fnAX6nql7qU2zASS5IyuA2jGV3dAajqZ1X1MlVdq6o/rqo/776+6C7+qOqCql7tblep6l1BCG4YScVPcxXDSALWD8AwAsYyuI20YArAMAImiuYqhhEEpgAMI2CsvaWRFqwYnGEEjGVwG2nBFIBhhECYGdyWX2AEhfUDMIwUcW6HuKNn7iyi6hBnJBvrB2AYGcTyC4ygMSewYaQEyy8wgsYUgGGkBMsvMILGFIBhpATLLzCCxhSAYaQEyy8wgsacwIaREiy/wAgaUwCGkSKsQ5wRJKYADCNlWIc4IyjMB2AYhpFTTAEYhmHkFFMAhmEYOaUrBSAi/11EviUiXxeRz4rIBQ2Ou0lEvi0iJ0VkqpsxDcMwjGDo9g7gMPBGVf33wHeA3fUHiMh5wB8D7wauBH5ZRK7sclzDMAyjS7rtCfygqr7qPn0EuMzjsOuAk25v4B8Afw3c3M24hmEYRvcE6QP4T8AXPV7vB56ueX7Kfc0wDMOIkZZ5ACLyFeASj123q+rn3WNuB14FDnqdwuO1hk0IRGQHsANgYGCglXiGYRhGh7RUAKr6zmb7ReQ24D3AjerdXeYUcHnN88uAxSbj3QvcC5WGMK3kMwzDMDqj2yigm4BdwPtU9cUGh/0TMCQirxeRXuBW4IFuxjUMwzC6p6uWkCJyElgLPO++9IiqfkhEXgfMquqIe9wI8IfAecB9qnqXz/M/B/xzxwIahmHkj59Q1T4/Bya6J7BhGIYRHpYJbBiGkVNMARiGYeQUUwCGYRg5xRSAYRhGTjEFYBiGkVMSrQCSXm1URN4vIk+IyGkR2dLkuO+KyOMiclxEjkQlX5syxlaxVUQuFJHDInLC/fuaBsf9yJ3D4yISSS5Jq3kRkbUiMufuf1RErohCrjbk+6CIPFczb78RpXyuDPeJyLMi8o0G+0VE/sh9D18XkWsSKOMNIvJCzTx+NGL5LheRvxeRJ93f8297HNP+PKpqYjfg54Dz3ce/D/y+xzHnAU8Bg0Av8BhwZUTyvQH4KeBrwJYmx30XuDimOWwpY5xz6I6/B5hyH095fc7uvuWI567lvAC/BXzSfXwrMJcw+T4I7Ivju1cjw88A1wDfaLB/hEodMQHeAjyaQBlvAP4uxjm8FLjGfVygUn25/rNuex4TfQegCa82qqpPquq3oxirU3zKGHfF1puB+93H9wO/EOHYzfAzL7Wyfxq4UUS86l/FJV/sqOrDwPebHHIzcEArPAJcICKXRiNdBR8yxoqqPqOqx9zHZeBJzi2q2fY8JloB1JHmaqMKPCgiR91id0kj7jn8cVV9BipfdOC1DY5bJyJHROQREYlCSfiZlzPHuBcrLwAXRSDbqrFdGn1uv+iaBD4tIpd77I+buL9/fnmriDwmIl8UkaviEsI1M74JeLRuV9vz2LIYXNhEXW00DPl88DZVXRSR1wKHReRb7hVHUmQMdQ6huYxtnGbAncdB4CEReVxVnwpGQk/8zEvoc9cEP2P/LfApVX1FRD5E5W7lZ0OXrD3inEO/HKNSYmHZLW3zOWAoaiFEZCPwGWBCVZfqd3v8S9N5jF0BaMTVRoOWz+c5Ft2/z4rIZ6ncugemAAKQMdQ5hOYyisi/iMilqvqMe8v6bINzVOdxQUS+RuUqKEwF4GdeqsecEpHzgR8jOlNCS/lU9fmap/up+NKSRujfv26pXWxVtSgifyIiF6vq96KSQUTWUFn8D6rqIY9D2p7HRJuAJAPVRkXEEZFC9TEVx7ZnpEGMxD2HDwC3uY9vA865axGR14jIWvfxxcDbgG+GLJefeamV/RbgoQYXKrHIV2cDfh8V23HSeAD4gBvF8hbghapJMCmIyCVV346IXEdl7Xy++X8FOr4Afw48qaqfaHBY+/MYl1fbp+f7JBWb1nF3q0ZbvA4o1nm/v0PlavD2COXbSkXrvgL8C/DlevmoRGg85m5PRCmfXxnjnEN37IuArwIn3L8Xuq9voVJVFuB64HF3Hh8Htkck2znzAnycykUJwDrgb9zv6j8CgxHPXSv57na/d48Bfw/8dJTyuTJ8CngG+KH7XdwOfAj4kLtfqPQNf8r9bBtG1MUo40dq5vER4PqI5RumYs75es16ONLtPFo1UMMwjJySaBOQYRiGER6mAAzDMHKKKQDDMIycYgrAMAwjp5gCMAzDyCmmAAzDMHKKKQDDMIycYgrAMAwjp/x/xbT2gfYGCtQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f087fcc5cc0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def load_pts(csv_name):\n",
    "    data = np.asarray(pd.read_csv(csv_name, header=None))\n",
    "    X = data[:,0:2]\n",
    "    y = data[:,2]\n",
    "\n",
    "    plt.scatter(X[np.argwhere(y==0).flatten(),0], X[np.argwhere(y==0).flatten(),1],s = 50, color = 'blue', edgecolor = 'k')\n",
    "    plt.scatter(X[np.argwhere(y==1).flatten(),0], X[np.argwhere(y==1).flatten(),1],s = 50, color = 'red', edgecolor = 'k')\n",
    "    \n",
    "    plt.xlim(-2.05,2.05)\n",
    "    plt.ylim(-2.05,2.05)\n",
    "    plt.grid(False)\n",
    "    plt.tick_params(\n",
    "        axis='x',\n",
    "        which='both',\n",
    "        bottom='off',\n",
    "        top='off')\n",
    "\n",
    "    return X,y\n",
    "\n",
    "X, y = load_pts('data.csv')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Splitting our data into training and testing sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import f1_score, make_scorer\n",
    "\n",
    "#Fixing a random seed\n",
    "import random\n",
    "random.seed(42)\n",
    "\n",
    "# Split the data into training and testing sets\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Fitting a Decision Tree model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "# Define the model (with default hyperparameters)\n",
    "clf = DecisionTreeClassifier(random_state=42)\n",
    "\n",
    "# Fit the model\n",
    "clf.fit(X_train, y_train)\n",
    "\n",
    "# Make predictions\n",
    "train_predictions = clf.predict(X_train)\n",
    "test_predictions = clf.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's plot the model, and find the testing f1_score, to see how we did."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following function will help us plot the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_model(X, y, clf):\n",
    "    plt.scatter(X[np.argwhere(y==0).flatten(),0],X[np.argwhere(y==0).flatten(),1],s = 50, color = 'blue', edgecolor = 'k')\n",
    "    plt.scatter(X[np.argwhere(y==1).flatten(),0],X[np.argwhere(y==1).flatten(),1],s = 50, color = 'red', edgecolor = 'k')\n",
    "\n",
    "    plt.xlim(-2.05,2.05)\n",
    "    plt.ylim(-2.05,2.05)\n",
    "    plt.grid(False)\n",
    "    plt.tick_params(\n",
    "        axis='x',\n",
    "        which='both',\n",
    "        bottom='off',\n",
    "        top='off')\n",
    "\n",
    "    r = np.linspace(-2.1,2.1,300)\n",
    "    s,t = np.meshgrid(r,r)\n",
    "    s = np.reshape(s,(np.size(s),1))\n",
    "    t = np.reshape(t,(np.size(t),1))\n",
    "    h = np.concatenate((s,t),1)\n",
    "\n",
    "    z = clf.predict(h)\n",
    "\n",
    "    s = s.reshape((np.size(r),np.size(r)))\n",
    "    t = t.reshape((np.size(r),np.size(r)))\n",
    "    z = z.reshape((np.size(r),np.size(r)))\n",
    "\n",
    "    plt.contourf(s,t,z,colors = ['blue','red'],alpha = 0.2,levels = range(-1,2))\n",
    "    if len(np.unique(z)) > 1:\n",
    "        plt.contour(s,t,z,colors = 'k', linewidths = 2)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJztnXt8VOWZ+L9PEsgk0QoWCkJItS0KtNqgUamyW7e7SY1SQ7fa2osVV0tti2GtsdqVXj6WtbWksURQq6xV/Fnb1UpMrbLQ6rYFxSJmVBRU7CoJWMELqJNMru/vj5kJk+TM/cycc2ae7+eTT2bOOXPeJyfJ+7zvcxVjDIqiKErhUeS0AIqiKIozqAJQFEUpUFQBKIqiFCiqABRFUQoUVQCKoigFiioARVGUAkUVgKIoSoGiCkBRFKVAyVgBiMgMEXlURHaIyHMistTiGhGRVhHZJSLPiMiJmY6rKIqiZEaJDfcYAK4wxjwlIocD20RkozHm+ahr6oGZ4a9TgZvD3+MyYcIkM23a0TaIqCiKUhjs2LHtDWPM5GSuzVgBGGNeA14Lv35XRHYA04FoBdAArDWhuhNbRGSCiBwV/mxMpk07mrvuejJTERVFUQqGmhp5NdlrbfUBiMjRwFzgiVGnpgOdUe+7wscURVEUh7BNAYjIYcBvgX83xrwz+rTFRyyr0InIYhF5UkSefPvt/XaJpyiKoozCFgUgIuMITf53G2Put7ikC5gR9b4S2Gt1L2PMrcaYGmNMzcSJSZmxFEVRlDSwIwpIgP8CdhhjWmJc1g58NRwNNA84mMj+ryiKomQXO6KATgcuAJ4VEX/42H8AVQDGmFuAh4CzgF1AN3CRDeMqiqIoGWBHFNAmrG380dcY4FuZjqUoiqLYh2YCK4qiFCiqABRFUQoUVQCKoigFiioARVGUAkUVgKIoSoFiRxioouSEQCDAxo0b2L17D1VV06mtraOiosJpsRTFs6gCUDyB399BY+OVDA1VEwzOxufbTEvLalpbV1BdPddp8RTFk6gCUFxPIBCgsfFKuruXA/MACAYBttDYeCXr17dTXl7upIiK4knUB6C4no0bNzA0VE1k8j/EPIaGqtmwYYMTYimK51EFoLie3bv3EAzOtjwXDM6iq2tPjiVSlPxATUB5Rj46SquqpuPzbQ6bfUbi8+2ksnJ+7oVSlDxAdwB5hN/fQX19A83Nm1m7tozm5s3U1zfg93c4LVpG1NbWUVTkB7aMOrOFoiI/dXV1ToilKJ5HdwB5Qj47SisqKmhtXREVBTQLn28nRUV+WltXePbnUhSnUQWQJyTjKF24cKETotlCdfVc1q9vZ8OGDXR17aGycj51ddfq5K8oGaAKIE8oBEdpeXm5p5WYorgN9QHkCSFH6Q7LcyFH6fQcS6QoittRBZAnqKNUUZRUURNQnqCOUkVRUsUWBSAitwMLgH3GmI9ZnD8DeAD4v/Ch+40x19oxtnIIdZQqipIKdu0A7gBWAWvjXPMXY8wCm8ZTYqCOUkVRksUWH4Ax5s/AW3bcS1EURckNuXQCf0JEnhaRh0XkozkcV1EURbEgV07gp4APGmPeE5GzgDZgptWFIrIYWAwwdWpVjsRTFEUpPHKyAzDGvGOMeS/8+iFgnIhMinHtrcaYGmNMzcSJk3MhnqIoSkGSEwUgIlNFRMKvTwmP+2YuxlYURVGssSsM9B7gDGCSiHQBPwDGARhjbgHOBb4hIgNAD3C+McbYMbaiKIqSHrYoAGPMFxOcX0UoTFRRPEk+9llQFM0EVpQEaEN6JV9RBaAoccjnPguKosXgFCUO2pBeyWdUAShKHAqhz4JSuKgJSFHikGlDenUeK25GdwBKUgQCAdra1tHauoq2tnUEAgGnRcoJmfRZ8Ps7qK9voLl5M2vXltHcvJn6+gb8/o6syqwoySJuDsefM6fG3HXXk06LUfCMjYLZMdxnoBCiYEb+/CP7LMT6+QOBAPX1DSOcxyG2UF6+TJ3HStaoqZFtxpiaZK5VE5ASF42CSa/PQjLO43wt261mL++gCkCJSyFPZNGk2mehUJ3HmjPhLVQBKHEp1IksUzJ1HnsR3S16D3UCK3EJTWQ7LM+FJrLpOZbIG2TiPPYqmjPhPXQHoMSltraOlpbVhCaykc7M0ESmrZ2tqKiooLV1RUzncaYr4YidvWt3J5VVM1xhZ3f7blF9E2NRBaDEJdsTWT6TjvM4Gfz+Dpoal1I9NMTsYA+bfGWsarmB5taVjtrZ3Wz2Ut+ENRoGqiRFd3d31EQ2nbq6Op38HSAQCNBQX8/y7sCo/RgsK6+gff16x34vbg19datc2ULDQBXbSTUKRskOGzduoHpoyMLKDtVDQ45GZbl1t6iRbLFRBaAoHqJrdyezgz2W52YFe9jT1ZljiUaSLbNXJrjdN+EkqgAUJUNy6VysrJrBJl8ZWCiBnb4y5lfOyMq4qeC23aKbfRNOY0sYqIjcLiL7RGR7jPMiIq0isktEnhGRE+0YV1GcJtf1fmpr6/AXFVkEl4K/qCgvw0szpRBDcpPFrh3AHYRaPq6Ncb4emBn+OhW4OfxdUTyLE4lPFRUVNLeuHI4CmhXsYaevDH9REc2tK/PKmWkXbvVNuAG7egL/WUSOjnNJA7A23Ah+i4hMEJGjjDGv2TG+ojiBU87F6uq5tK9fz4YNG9jT1cn8yhlcmyAqq9Bj4CO+iQcffJC//GUzAPPnf42ZM491WDJnyZUPYDoQ7Z3qCh9TBaB4Fiedi6nY2TUGPsSLL77AqlW3Dj+Hjo6trF59W8E9h2hypQDE4phlAoKILAYWA0ydWpVNmRQlI7zgXNT6PCH0OViTq1pAXUB0eEIlsNfqQmPMrcaYGmNMzcSJk3MinKKkgxeci1qfJ4Q+B2tytQNoB5aIyK8JOX8Pqv1f8TpecC5qDHwIfQ7W2KIAROQe4Axgkoh0AT8AxgEYY24BHgLOAnYB3cBFdoyrKE7jxsSnaLxgpsoF+hys0VpAipLHFFodnFgU0nPQWkBKWjz66Dr+/OffZXWMk0/+FGed9ZWY519++XnuuWclg4P9WZUDoLTUx0UXfZcpUw65p15/vYtf/vIn9PZ2p33fiROn8LWvfY+yskMTyvbtT/Db3/4iI3nTZe7cw3j88X/FmCMx5ghEDiLyFnPnnsKKFUsckckJ9DmMRXcACgCf+9wcXn3VuvGL3UyePI2HHx5rc73ppmu4/fbrciJDNN///u2cc85FPPjgWn74wwttu+99973A0Ucfy/e+91Uefvgu2+6rKAlIegegHcEUgJxN/gD79++lu/u9Mcd/85vVOZMhmjvu+PGI73Zx3303AbBhw69tva+i2IWagJQx/PHmm7Ny33/+xjeGXw8NDY05H70bXfO973HMtGkx7/XSrl38YtUqPjw0xAf7+3l13DheLiri60uWMPMjH0koy4YtW7j+zjsBGBwcCo9/SKYrL7iAM087LeVxv7NyJdt27gzfd2DM5x9oaeGwsrKE8rmNnmCQbdu28cbrrzNpyhROOukkynw+p8VSLIj+P0uEKgCX4JZU/SIRPnXyyVm597iSEvoHxk6KVnzyxBP5yAzrypaBQIDrrriCm3p7D7nz+vtDTVFuvjmppijvBgJxz5/6sY+NeQ7JjDuzqmpYAVjxTyedxOEeK8HQ4fdz3RVXUD00xMeDQXb4fNxw//2saG1lbnW10+IpGaAmIBeQ64qSXmfDxo0Jm6Lk07hOEggEuLKxkeXd3TQHg1wMNAeDLO/u5srGRrq703eWK86jCsBholPUg8Fm4GKCwWa6u5eHj+s/2Gj27N7NbKuAbmBWMMierq68GtdJClHpFRKqABwmkxT1QCBAW9s6WltX0da2jkACs0a+ML2qih0x7M87fT6mV1bm1bhOUohKr5BQBeAw6aaoF7LZqK621pGmKE6N6ySFqPQKCVUADhNKUbcOwQylqE8fc7zQzUYVFRWsaG1lWXk5TT4fa4Amn49l5eWsaG3NWkanU+M6SSEqvUJCo4Acpra2jpaW1YT+pUamqIcqSl475jNONSJxE3Orq6OaonQxv7IyYVMUL4/rFBGld2VjY7gDWZCdPh/+oqK8VXqFhCoAh0mnoqRWNgzhVPNxtzU9T5bBwUFO+spXePqll9L6/B8jL8I+gRMvucQewVzEtMmTeen++yn3YK5GOqgCcAGpVpTUyoaFTSAQYMPGjezZvZvpVVXU1dYmlTPy3dWr0578C4W9+/fTcMUVbLzpJqdFyQl5owDckkiVDmNlr427tU7HbOQ2fv/737FgwULP/I7cQoffP2yOmR0MstnnY3VLS1JJWZ1///uI93Y0XBoaGuLgwYPAYYQrwIfpB97jiCOOoKjI/a7Gd955ezhz+/W33nJYmtyRFwrAyz1P05HdC41IRtPh9zMQlQXc2rqF1at/6YnfkVuITsoaVvvBIFuAKxsbk8qAjnDeed/iqqtWZSxTW9s6mps3h4MRRuLzNXHZZfNzai5LdyH4+9/fxQ9+8NUcSOguPK8AvNzrMxPZ3d6IJJpAIMDSxiYMxcAgAL29/wk879jvqCfKfhYIvOeJHIpkkrJy7Ztwkz/KywtBp/C8AvByREymsnvFGblh48bwz7lp1JnEP2e69u54dPj9XHftITNZ8MABGurrGX9Yb0b3zTZuTMrKpT8q3ureywtBJ3G/cS4BblqBpIqXZU+Fzt1d9KTxc3b4/TTU17O5uZmytWvZ3NxMQ309HX5/2rJEzChf6esbPvZ+DMu7A+zf93ra980FbkzKqq2to6jIDxaZAiF/lD15AokSH7Xpe3rYogBE5EwReUFEdonI1RbnF4nIfhHxh79six9LJ5HKLXhZ9lSYUVVJWYo/Z3e42JjdRcgiZpTjRh2fB7i9uLEbk7Ii/qjy8mX4fE3AGny+JsrLl9nmj0om8bFQFlN2k7ECEJFiYDVQD8wBvigicywu/Y0xpjr8tSbTcSPkagWSDbwseyrU1daGf87R3edi/5ybNm/OShGyeGaU8YztUeAm3JqJHPFHNTXNZ9GiXpqa5rN+fbttdvdkVveFspiyGzt8AKcAu4wxfwMQkV8DDcDzNtw7IV6MiIngZdlToaKigpWtzZx0ySXDKqC09BqKi3fE/Dlf37s3K/bu6VVVbPb5+AeLe/e5zCJq5f9wayZyNv1RyazuL7pokedDo53ADgUwHeiMet8FnGpx3edE5B+BF4HLjTGdFtekhZciYkbjZdlTYW51NSVRDWEaGz/BZz5zfcyfc8q0aSF7t8VEvdPnY36a9u662lpWt7TwgVHHtwDW6iZENpzR8UgU7+8F579dJONoLpTFlN3YoQDE4tjovf7vgHuMMb0icilwJ/Apy5uJLAYWA0ydWpW0EF6JiLHCy7Kny9lnL4j7Tzn/9NO5/447LNZzIXv3tWmaxyJmlAu/+U0IO4LfRFhWXsHkw6awb9+rYz6TSfJVOtgZ758PJJv4WCiLKTuxQwF0AdG9+yqBvdEXGGPejHp7G3B9rJsZY24FbgWYM6dmtCJRCoTysF07G0XI5lZXc80PfsD511wDgG/CBNrb13PBBWNt1k5Mxm6M93eSVFb3hbiYygQ7FMBWYKaIHAPsAc4HvhR9gYgcZYx5Lfz2HMDaW6PkFaPNJqmSTXu3r7R0+HVFxWEx7+nEZOzGeH+n0dV9dshYARhjBkRkCfA/QDFwuzHmORG5FnjSGNMONIrIOcAA8BawKNNxFXdjZTYZSLIhfDROr+icmIwjjmq7/R9ex+m/hXzElkxgY8xDwEOjjn0/6vV3ge/aMZbifmKZTVYSWgF4CScm44ij2m7/h6KMxl1xb0peEMtsYhUt4BTJ1gJyIvnKrfH+Sv7h+VpAivuIZzZxAx1+P9dee93w+wMHgtTXN3CYRS0gpzpiuTXeX8kvVAEocUkn/j2e2cRpIpVJe/u+TCggDWAS3d3L6e62jEx2bDJWm7eSbVQBKDFJN/49lg3bDTG9hyqTWlUDKgV6AHjir3/lzHnzhpWdTsZKPHKdKGgXqgAUSzKJf49lNhl0wY4gdmXSDiKTP8DO9nYaHn44a8le8fDqZFKo5DpR0E7UCaxYkkz8ezwiZpP5TU30LlrE/KYmSkqcX29YVyYdYjxLmRy1R7mwvz/jyqPpkI0S2Er2iF4o2Vm1NleoAlAssSP+PWI2+daSJa4xnxyqTPpC1NFu5jLA6P1MppVHU8Xrk0khkulCyWlUASiWuLH5iB1EKpOWjr876ug7nIh1N7BcZt66dTIJBAK0ta2jtXUVbW3rPNE+M1d4PWtbFYBiiRubj9jF3OpqfvCDa4bfl5WVssNXZnltLpWdGyeTRJ24Ch2vL5ScN8raRLx+oUrqOBX/niuiawFNnDiRp94ylI66ZgvwFCSdeRsIBOiKmqRTLX0xecoU2ktK6BkYYDpQB0T+gp0oAaF9dhPj9aztvFAAfn9HVKXA2fh8m2lpWU1r6wrbuhIVIoWUjNRrxvFO1Ib4NkLvxST3LxKJBNkftYJ/8IF2zj7760l//rZVqzhuYIAyYDOhNnsrgF6cmUyS6cTlFt+OU3h9oeR5BaCrlOySSvy7V8MXu7u7EakhVMx2FwAvcg4TWUqZfD/hRBftvF3JIffyvKF+mhqXkigDYvjzPT1jVpGNwPiyMpodmEy0z25yeHmh5HkFoKsUd+DlWOiBgYHwRPe7qKOnAOX0BGfRmWCii3berow6PhUoHhriT0Pxew3Hc/6eNG4cp112mSPPMJlOXIVMZMHTubuLGVWVnlnwRON5J7CuUpzH6+GLJSUlMRuKl/l2MiNBQ/H4ztseEu0A4n3+Y/397N+3L+7ns0VtbV04ZHZsKECoE5e77dvZpMPvp76+gZ81b+LOtWX8rHkT9fUNnsvX8LwCCK1SrP95Q6uU+P+8Sua4NXwxWcrLy8MTXc+oM8lNdPEjQcpIVAfVrZEkkU5c5eXL8PmagDX4fE2Uly8r6D67kXpSge7l9ASbgYvpCTYT6F7O0sYm1y94ovG8AtBVivNkM3wxEAiwrq2NVa2trGtry0oMend3N2f846nA/uFj48etpaJ8GStbmykvL48rR6yQ2b8Tct4WFcX/N3NzyG2kE1dT03wWLeqlqWk+69e3F3RwxaF6UrHNzl7B8z6AVPqFKtkhW01TcuVXCB44wAfXr6eMQ3uAc86ZxQ+X3kx5eXlCOaIjQbqCQQjb/B+jiC+fey533/143PGTjSRxysmuhfBGErueFEn5jNyELQpARM4k5P8qBtYYY34y6nwpsBY4CXgT+IIx5hU7xgbn+4UWeg5CJrHQ0c9uYKB/+Hh32H+QjWbsgUCAv2zaNPz+SAwXAw8xxKvhY9XV1cMr/2TkiESC/MNXvwqvvALAILP47//ezeBg4nyARJEkXnay5xuhelKb6LHY9IZ8Rt5xjmesAESkmFDIci3QBWwVkXZjzPNRl10MvG2M+YiInA9cD3wh07GjcWqVojkI6cdCj3520Q0jN23enJVm7JGJtCxqt/IaMNp15/f7+VJ9fUpN4Y0x7N4dvfo7nmCwGfg5MJhQtlh/w5lUZlXsp662lhtaVoHFkidkdr7WIclSx44dwCnALmPM3wBE5NdAAxCtABqAH4Zf3wesEhExxrihRHzaaA7CIVKNhe7uHvvs4NtAaBfQ1dlpu18heiJ9HXgwfPxIoAk4LOrat998E0jNv7Fh40ZgAtG+hBBFJKMAYpGKElKyT6Se1NLGJoaGqukJzqIsbHaO+Iy8gh0KYDrQGfW+Czg11jXGmAEROQi8H3jDhvEdQ3MQRpLKLuzRRx+J8exCvPvuu6HIGBv9CtET6QNRx33ATOCZqGMT3/9+IOTf2JSkHJ27uxgYOoKxCiAz3FgjqNCZW109bHbu7NrDjBybne3CDgVgFeM2emWfzDWhC0UWA4sBpk6tykyyLKM5COnT2bk35rMDOPzwI/hrODJmtF+hQ4STe3tZ1dqakjM07kQKbIt6Xx22q0+dOpUtYXNLIv/GjKpKSooOQvy8r5TJlpNdyYx8cI7bEQbaBcyIel8J7I11jYiUAEcAb1ndzBhzqzGmxhhTM3HiZBvEyx6ag5A+M2ZMi/nsQuens6K1lWXl5TT5fKwBmnw+ri4txRjD1htvTLlhSrx4++1AT9Q6pXT8eAKBAN/7znf4JrCMkJloDSFD1RXAj3760xErvrraWuCAxd0z0whuDhN1E1q2OnXsUABbgZkicoyIjAfOB9pHXdMOXBh+fS7wiNft/6A5CJnQ29uHSAdjn12I+fPnj+kqdkpjIyVFRVwXDKaVcRxrIg0CHSXjOHzikSOOR0xGXyb0BzyfUGG2fwROLCmhc/fuEddXVFRwUk204/+5cAJV+vb/yH2tlOGy8nJPFBzLBVq2Oj0yNgGFbfpLgP8hFAZ6uzHmORG5FnjSGNMO/Bdwl4jsIrTyPz/Tcd2A5iAkT4ffP6I88lO/+AXjpJih0qsRqSEYnEV0Jm5ZWag+f/Q2e11bG9XGpO0MjY5Wqujvh/6Qw/kthNtuuZkf/eh83n77NQDOveoqRASM4ecW9xoaGGDVT3/KF1esCF0XZjCq7s9xxw1y3nnz+fGPVyYVChoPLxccyzYajJE+tuQBGGMeAh4adez7Ua+DwHl2jOU2nM5B8AKR6JtiDgV6Lu8Nsp1ilpUVsfiyU9i3bz+/+lUJfX39Me9jhzM0MpH+6MYbab/3XgCOmjaN6uq5VFZ+hFdffXH42sgmNd76fcgYiLGZPffcC1i4cCE//vFIF1i6CV35YHPOBhqMkT6eLwXhBiL/mEuWfIuFCxfq5D+KiClldCTAPKDaGMaPH8+SJd+ipGTciM+MLrtgV82c8vJy5p16KFBNJPRv0NLSzrx5n+aww45I/oezQKSIc8/9Bp/97NfGnHv62We16bvNaDBG+ni+FITifhJVy9zT1Tnm+LY1azitr29Exmu2uy8VFxezatX6Ecf8/g6WXvoNPjYAc+mngzI6KKKPlcBcYA2LFvWyZMm3khpj2ZVX8pPouv8ZJnR5tQeDnWjZ6vTRHYCSdRJVy5xeGQkiO2RK+W5f3xgnr4jk3BlaXT2Xb15+OdtKPswtLOIJmuhjPaHJP/VorxNsrJra4ffrbgINxsgE3QEoWSeych9tKR+9co/VQ3e0kzfXztAFCxawevVtDAycTKap/8f19loet/JhxFvda3mIQ2gwRvqoAlDGYLdZIRJ9c9IllwwfW1bqY3vxeJpbVw7/gw4Oxo6Xj54gc+0MtXOCeaG0FCyUwOiErkTF3+wuDxEpyte1u5PKqhmeK2iowRjpoQpAGYEBGurrba86Obe6mpKSEvrDq/xPNDZy3WdGOsyLi4sYjBFy43TGq10TzDMxspujd0KR1f2y7m7eJtSp+PRgkAUcWt3bWR7C7++gqXFp+HfewyZfGatabqC5daWnChpqlFTqqAJQRmCMyYlZ4eyzF4y5V0lJCX19Y6/dAvjD5R9aW1c51n/Vjglm+YoVLPvOd+JWTd2wcSNHDwywHKgGZgObCVUsPXpggA0bNthWHiIQCNDUuJTl3YGo33kPW4CmxqUFZUoqRFQBKGOYxciCBrOAWYODrGtv5+yzz077vkMJk78PBYr+cPx4Tunr48XSUp4G+gaLuKH1MXp6j6Os9FFW/KyVn1y/nBOOPz4tWQ46VCbg48cfn9CH8cquXbzU18f1jN0pXNXXxyv/9398bfFiWyKiNm7coJVGCxhVAMoYJlod7O1ldXMzNDfnRIa7+/q4OzxuWABC6+DNoZfAJxsbcyKL3STaSRx4910+jlVaE3wcePvgwbR7MIyma3cns4OjeyGHiBWiq+QPGgaqADBhwqScjeXzVVBRcfiY45/85Dk5kyGaz3zmwsQX5ZAJ73sfH49x7gTg1ZdfZl1bG8fOnDmiVtL8piba169PyVdTWTWDHb4yy3MjQ3SVfER3AAoAGzfu46mnNvH1r3+aqQwRvX7sBv5OEcccc8xw1my6nHDCJ/iP/7hlRP2cCD/60V185Svf5qabvsdrr4WaM7755hscPFhEqG3LaN5iwoQhjjwyPeU1fnwpl112Paee+s9pfT5bHP3hD/OX8eOxcog8Dbx/xw42NzcPO+czMdHU1taxquWGrCXXKe5GFYDDuKWfsIhQVuZj4cJFPPy73+MDJg/2sb94PG8DCz9zNkcdNc2WsX75y+vinj/hhHmccEJoOnr++efZtOlVBgfrx1xXXPwwH//40cyZE7uvQCK2b3+c7dvjN21PlYkTp/CZzyxi3LhxiS+2YDjjua9vzKS8g1Bl0nKbnPMVFRU0t64cjgKaFexhp68Mf1HRiBBdJT9RBeAgbuonfOGF83juuSeG378S/opUQmtruzmn8ozlsTFHBgfhT396jD/9yQFxEvDjH3+d9vZXOOqo1JsaWdn3nyHUY3UFoVDddYTCQ6f09fHggw/y+c9/Pm1Zq6vnRjmmO5lfOUMrjRYIqgAcIl4J20sv/Xcuv/ybLFiwIGe7gejJX8kcYwz33HMD3/72DWl9Prr8c3tbGxO3b6cdeIFQg+1IeOikgQFWt7Qw89hjM8rT0Bj6wkQVgEPEK2E7MFDNDTe0s3r1bY7sBm675pqs3n93Zyf33nMPM4aGOGpwkNeKi+ksKuK8L36RqhnWTse+vj6ee+453nr7AEdOnMBHP/pRxo8fn1U50+H6O+9kVzgJq78/dmnrZIhMygbYvGsXJhjkSuDQkiHEloGBgiv/oNiDKgCHiFfCFo5nYKCXgYGTct7QokiESz772azdPxAI0FBfzy/7+w9NYoODbBkcZNl993G1xyexP27dOqwA7CLiE/gFoZW/xuwrdqFhoA4Rr58w7ASmE93QIl9IpoaNMpKIT+D+khJiLRlSLf+gKKAKwDHilbANJf2Hwu/yraGFnTVsCom51dVcdvnlPBsjsiiVhjiKEiEjBSAiR4rIRhF5KfzdMolURAZFxB/+Gt0wviCJVJgsL1/GuHH/DqHq9sAyQrEeITNIqvXm3U4mXb0CgQDr2trGdAorFBYsWMD2ceOslwxFRVr3XkmZTHcAVwN/NMbMBP4Yfm9FjzGmOvzlTLqnC4lUmLz88tPP5rqqAAAVP0lEQVQoKbkDmEYoyjvi9M2/hhZ1tbX4wxUxo0k0iWnzk0OmoFw2xLGLQCBAW9s6WltX0da2ruCUt1vJ1AncAJwRfn0n8L/AVRnes6AoLy/n85//PMceOzOcE7A3rxtapFPDJp3mJ/naKjE6PDRXDXEyxU35LspIMlUAU4wxrwEYY14TkQ/EuM4nIk8CA8BPjDFtGY6bdxRSQ4tUJ7FUm58kaqbidbwUsx8v3yXXEW7KWBIqABH5AzDV4lQqweJVxpi9IvIh4BERedYY83KM8RYDiwGmTk09i9LLeOkfO1NS+VlTcRxnu1Vivu4sskW8fJdIhFuh/M27kYQ+AGPMvxhjPmbx9QDwuogcBRD+vi/GPfaGv/+NkJko5r7PGHOrMabGGFMzceLkNH4kJd9IxXGczTBT9UOkTrx8l3yLcPMimZqA2oELgZ+Evz8w+oJwZFC3MaZXRCYBpwM/zXBcJU3sLj6XixXxcHE0EleszFaYaTZ2Ftl8dlb3doJQvstmq8Zl4Qi3+bkXShkm0yignwC1IvISUBt+j4jUiMia8DWzgSdF5GngUUI+gOczHFdJA7+/g/r6BpqbN7N2bRnNzZupr2/A7+9I6365WhGnEv2SSZhpPOzeWWTz2cW699sHDiT+sM3Ey3fJtwg3L5LRDsAY8yYwppi6MeZJ4JLw68eA9Pr2KbaRyBmXzv2yaWsfTbKO41R2C6lg584im88u3r0bnnwyrXtmQiTf5VAUUP5GuHkRrQVUICRyxoXSOJIn1cgcO0jGcWxXq8TR2NWEHbL77OLdeyIxnHRZppAi3LyGKoACIZEzLlXcXNIhG7Hy6e0shsYc2bBlC398/HFKg0HWW3xiIBhk45Yt+D4QK6I6Phvj3DswNFaeXOGlCLfX3niD9Y+N7T+Rj6gCKBASOeNizOUxsXNFnA6JHKh2TzjJ7iyKi4uHP3PvvTfzhS8sHXGfc686lCf581iD/eEP3PSHP2Qkb8x7hykuzv2/vlu631kR/TzeOHCA+sZGB6XJHWKMcVqGmMyZU2Puuiv3dst8JBAIUF/fMMIHEGIL5eXL6O4OmYCKRBjcujWp+zXU14+0NRNaES8rL89qbfrRiV47oibibCd6dXd3D+8spldWUjdqZ/H0iy9S/aUvDb+vrv4HKis/xIMP3plVuVJl3bqXmDHjIzkbb2w28I5hP4AbsoHfe+8dPvWp9zM0NOC0KHawzRhTk8yFqgAKiJH/hCOdcZdcciKQvAKAkRPx6BVxtiZiJxVPstzwq1/x7ZYWAGbPPpm77vorL7zQQVvbGowJmWH273+Dx/6yiUnGMMEMcEBK2AcIMBmYaAZ4W0p4Q4TT/mE+kycn3/je6t6R+8ycOYeLLvoupaXWkVLZINHiwy3ZwENDQ9x9dwt79ljmqHqG++67JWkFoCagAsJuZ5wTdWmccD6nSllp6Zhjxx03l6uuWj3i2KHdRCeTPzCFW2+8keU9UYrNwBYDy7a+Tvv6O1N6rtH3nl45Y8xOJZd4JRu4qKiICy5oSvp6t5q07rvvlqSvVQVQYNhtG8+1c8/NzudUiX52bW3rqDbGNsXmJqdr/ACED/PII4+ye3eXqyZRiD/B50uBO20Io3iKbCV6OU3X7k5mB3ssz80K9rCnqzPHEtlH7O53HcCv2LpVbElMtJN4SZPROTXBYDNwMcFgM93dy8PHu50WP2lUASiuJ7oRTF9vLx0iedcUpbJqBjt8ZZbndvrKmF45I8cS2Yd1NnAAuAJYQX//z3HTJJpogn/wwQcTmrS8gpqAFFczOuJnq8+HAa4uLaVGxLZEL6epra1jVcsNcfMMIiaJrt2dVFbNcJW5JB5W2cAlJY8yMDAbN/oFEvksNm16jGDwBMvPeq3AnSoAxbXEK2uwrKyMUy67jP379nmiKUoiKioqaG5dSVPj0nBUVQ87fWX4i4pobl3Jiy++MHxudrCHTb4yVrXcQHPrSk/YnEcHILzwwiQef9ydk2iipEmRZ/H5duRFgTtVAIpriRvxYwzjx4/nW0uWOCFaVqiunhsVVdXJ/MoZXFtXhzEmHPoaiFKEPWwBmhqXuiL0NRlGO707OtxZJTRR0uT8+afj998KFvu1UIG7a3MkaeaoD0BxLfkU8ZMskUnyW0suY+HChZSXl7Nx44as9ThwCjdXCU0k24IFC8KF7Jbh8zUBa/D5migvX+a5Ane6A1Bci9PlJtxCPkYIublKaDKyJcqpcWuOwGhUASiuJVulnb1GZdUMNvnKwEIJ7PSVMd+jEUJurhKajGyxci28lCOgCkBxLdkq7ZwtIgXq/veRR2y9bzIRQl7FTQlro0lHtkR9N9xS9iKCKgDF1ThRbiIdosNVx0WZrLq7A2nfM9qMcM6557Ls3nupNmZMhJDbnkUh45WyFxEyUgAich7wQ0JtH08JdwKzuu5MYCVQDKwxxvwkk3HzBafthNHjuxk3rxJhbLjqLcD/C5/rfPVVuru7U56krapniozng19YSK8wHCGkk7+7SBRC6rYcgUx3ANuBfwV+EesCESkGVhPqGdwFbBWR9kLvC+y0nXD0+BHcWxvWvcQKVwUoh5RXffHMCPfem1z1TKcXF4VKohBSt+UIZNoTeAeAiMS77BRglzHmb+Frfw00AAWrAJy2E1qNH27hjDEmrRXraLp7elj1m9/w9nvvZSasB9i2dSslwSDfDb+P9gCUMZRylE6mZgSnFxe5xG2Krra2jpaW1XglRyAXPoDpQPR/QBdwag7GdS1O2wljjx8i0/E3+/3Mv+SStD/vVR62OBakOOU6PpmYEZxeXOQSNyo6N4e3WpFQAYjIH4CpFqeuMcY8kMQYVtuDmJYGEVkMLAaYOrUqidt7D6fthPHGB+jMcPyrV63K6PP5xND4D6ec1JSJGcHpxUWucLOic3N462gSKgBjzL9kOEYXEL0EqgT2xhnvVuBWCHUEy3BsV+K0nTDe+AAzKqdndP+BwcHh12ec8Vk+/enzx1wTDAa57rqf0t9/PnBs1JkXGTfu11xzzVWUWjRWcSuvvPI31t5+B0cbw/SBPv5e4qNz3CR+dmPq4aqZmBGcXlzkCrcrOrcHLkTIhQloKzBTRI4B9gDnA1+K/5H8xmk7YezxQ9iZhn/yyZ+itvbzY463ta2juPhM+vuXjTlXXHyAgYHDWbDA/f9A0Xz5y43DdXzmZdCFKxMzgtOLi1xRKIou22QaBvpZ4EZCbUx/LyJ+Y8ynRWQaoXDPs4wxAyKyBPgfQmGgtxtjnstYcg/jtJ3QavwIIpKTrWo+/gPbuepL14zgxOLCGMM777xNLvuLT5r0PkpLn6C3d+zzLi3t4MgjT+XAgTdzJo9XyTQKaB2wzuL4XuCsqPcPAQ9lMla+4bSdcPT4d9wROh43nstGcrlSdVukSLKko1ByvbjYtu1PXHrpP+V08h/JTWOO9PZCS8tNtLQ4II7HEOd+cYmZM6fG3HWXZW6ZYjM1NaGpv0iEwa1bM7rXJy66iC3PPgvAlVfeyBe+MLZkcyAQoL6+YVQoKsAWysuTi3VPBquEqshkmG8hkdFEmsKHFhfTs9YU/pOfnEAgcND2+yoZsc0YU5PMha4uBTEw0M8bb/zdaTGULJCLlaqbI0WyTa6ckIOD/cOvj62qorzMuq1lthgaGuKdd96hv7+fcePG8b73vY+iotxUue/p7mZPVxdlxuADgkCPCNMrKylz8O/K/8ILSV/ragXw0kvPcOaZRzkthpIlsm0Gc3ukSL7xx5tvpnLKFKfFyAmBQICG+nruMGbk/tUYlr35Ju133+3Y4kJqklr8Ay5XAErumXf88TkdL5sr1Xx0NCvuIG63unCTHi8sLlytACp8Po7/yEecFqMwEOGb557LBWef7agYdjpsCyUkUsk9+dKtztUKYNYxx/B4JDxFyXvsTu13Ot9CyV/ypVudqxWA4h4izU727N7N9Koq6mprbQ2lzIbD1ul8Cy/i1ZDZXJMv3epUAWSBbE+WuSa62cnsYJDNPh+rW1pY0drK3OpqW8bIlsPW6XwLL+HG4mpuxWvd6mKhCsBmcjFZ5pLRzU4ACAbZAlzZ2Ej7+vW2/LFn02HrlbosTlLIIbPp4pVudfFQBWAjuZosc0muoh3UYessGjKbHl5fXOQmY6JASGay9Bq5inaora2jqMhPyIoaTcRh6w2bqlfJVchsIBBgXVsbq1pbWdfWRiCQfs9kJXN0B2Aj+RIaFk2uoh3UYessudiB5Zt5NB/QHYCNTK+qYofPZ3lup8/HdI+EhkVTV1uLv6jIYl0einawc2Uecdg2Nc1n0aJemprms359uzogc0C2d2DR5tHmYJCLgeZgkOXd3VzZ2Eh3d3dG91fSw9U7gG07dqSU1uwWfmZ1MBjkZ8uXw/LluRYnacaPG8djt9/OSbMPmQJyHe3gdZuqV8n2DixfMmfzDVcrACW39PX3c9q//Ru9jz8+4ng+RDsoiclmyGw+mkfzAVcrgKKiYg477H1Oi5EyxkB/fx+Dg0MUFxcxbtx4JI1C+0ND8N577wDljPxVDQDdHH74+9K6rxXvvPN26M4DA5bndWVeGGTr95wvmbP5hqsVwHHHVVPI/QDa2tbR3LyZYLB5zDmfr4mlS+fb9s8a6QegKNkgXzJn8w1XK4BCR6tZKvlCvmTO5huZ9gQ+D/ghMBs4xRhjuVwXkVeAd4FBYCDZbjWFjiZHKfmE+pLcR6Y7gO3AvwK/SOLafzLGvJHheAWFVrNU8g31JbmLTJvC7wAQuzyRygg0OUpRlGySKx+AATaIiAF+YYy5NUfjeh6tZqkoSrZIqABE5A/AVItT1xhjHkhynNONMXtF5APARhHZaYz5c4zxFgOLAaZOrUry9vmNbpsVRckGCRWAMeZfMh3EGLM3/H2fiKwDTgEsFUB4d3ArwJw5NSbTsRXFKbS5iuJ2sl4LSEQqROTwyGugjpDzWFHyFr+/g/r6BpqbN7N2bRnNzZupr2/A7+9wWjRFGSbTMNDPAjcCk4Hfi4jfGPNpEZkGrDHGnAVMAdaFHcUlwK+MMeszlFtRXIs2V1G8QqZRQOuAdRbH9wJnhV//Dfh4JuMoipfQ5iqKV9By0IpiM5rBrXgFVQCKYjOhDO4dludCGdzTcyyRolijCkBRbEbbWypeQYvBKYrNaAa34hVUAShKFshmBrfmFyh2oQpAUbJENjK4/f6OqJ3FbHy+zbS0rKa1dYX2TlZSRhWAongEzS9Q7EadwIriEZLJL1CUVFAFoCgeQfMLFLtRBaAoHkHzCxS7UQWgKB5B8wsUu1EnsKJ4BM0vUOxGFYCieAjtEKfYiSoARfEY2iFOsQv1ASiKohQoqgAURVEKFFUAiqIoBUpGCkBEVojIThF5RkTWiciEGNedKSIviMguEbk6kzEVRVEUe8h0B7AR+Jgx5gTgReC7oy8QkWJgNVAPzAG+KCJzMhxXURRFyZCMFIAxZoMxZiD8dgtQaXHZKcAuY8zfjDF9wK+BhkzGVRRFUTLHTh/AvwEPWxyfDnRGve8KH1MURVEcJGEegIj8AZhqceoaY8wD4WuuAQaAu61uYXHMxBlvMbAYYOrUqkTiKYqiKGmSUAEYY/4l3nkRuRBYAPyzMcZqYu8CZkS9rwT2xhnvVuBWgDlzamIqCkVRFCUzMo0COhO4CjjHGNMd47KtwEwROUZExgPnA+2ZjKsoiqJkjlgv2pP8sMguoBR4M3xoizHmUhGZBqwxxpwVvu4s4OdAMXC7MeY/k7z/fuDVtAVUFEUpPD5ojJmczIUZKQBFURTFu2gmsKIoSoGiCkBRFKVAUQWgKIpSoKgCUBRFKVBUASiKohQorlYAbq82KiLnichzIjIkIjVxrntFRJ4VEb+IPJkr+VKU0bGKrSJypIhsFJGXwt8nxrhuMPwM/SKSk1ySRM9FREpF5Dfh80+IyNG5kCsF+RaJyP6o53ZJLuULy3C7iOwTke0xzouItIZ/hmdE5EQXyniGiByMeo7fz7F8M0TkURHZEf5/XmpxTerP0Rjj2i+gDigJv74euN7immLgZeBDwHjgaWBOjuSbDRwH/C9QE+e6V4BJDj3DhDI6+QzD4/8UuDr8+mqr33P43Hs5fnYJnwvwTeCW8Ovzgd+4TL5FwCon/vaiZPhH4ERge4zzZxGqIybAPOAJF8p4BvCgg8/wKODE8OvDCVVfHv27Tvk5unoHYFxebdQYs8MY80IuxkqXJGV0umJrA3Bn+PWdgFsa3ibzXKJlvw/4ZxGxqn/llHyOY4z5M/BWnEsagLUmxBZggogclRvpQiQho6MYY14zxjwVfv0usIOxRTVTfo6uVgCj8HK1UQNsEJFt4WJ3bsPpZzjFGPMahP7QgQ/EuM4nIk+KyBYRyYWSSOa5DF8TXqwcBN6fA9lGjB0m1u/tc2GTwH0iMsPivNM4/feXLJ8QkadF5GER+ahTQoTNjHOBJ0adSvk5JiwGl21yXW00G/IlwenGmL0i8gFgo4jsDK843CJjVp8hxJcxhdtUhZ/jh4BHRORZY8zL9khoSTLPJevPLg7JjP074B5jTK+IXEpot/KprEuWGk4+w2R5ilCJhffCpW3agJm5FkJEDgN+C/y7Mead0actPhL3OTquAEyOq43aLV+S99gb/r5PRNYR2rrbpgBskDGrzxDiyygir4vIUcaY18Jb1n0x7hF5jn8Tkf8ltArKpgJI5rlErukSkRLgCHJnSkgonzHmzai3txHypbmNrP/9ZUr0ZGuMeUhEbhKRScaYN3Ilg4iMIzT5322Mud/ikpSfo6tNQJIH1UZFpEJEDo+8JuTYtow0cBCnn2E7cGH49YXAmF2LiEwUkdLw60nA6cDzWZYrmecSLfu5wCMxFiqOyDfKBnwOIdux22gHvhqOYpkHHIyYBN2CiEyN+HZE5BRCc+eb8T9l6/gC/BewwxjTEuOy1J+jU17tJD3fuwjZtPzhr0i0xTTgoVHe7xcJrQavyaF8nyWkdXuB14H/GS0foQiNp8Nfz+VSvmRldPIZhsd+P/BH4KXw9yPDx2sIVZUFOA14NvwcnwUuzpFsY54LcC2hRQmAD7g3/Lf6V+BDOX52ieT7cfjv7mngUWBWLuULy3AP8BrQH/5bvBi4FLg0fF4I9Q1/Ofy7jRlR56CMS6Ke4xbgtBzLN5+QOeeZqPnwrEyfo1YDVRRFKVBcbQJSFEVRsocqAEVRlAJFFYCiKEqBogpAURSlQFEFoCiKUqCoAlAURSlQVAEoiqIUKKoAFEVRCpT/D+9njH6uAi5kAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f08751c5ba8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The Training F1 Score is 1.0\n",
      "The Testing F1 Score is 0.7\n"
     ]
    }
   ],
   "source": [
    "plot_model(X, y, clf)\n",
    "print('The Training F1 Score is', f1_score(train_predictions, y_train))\n",
    "print('The Testing F1 Score is', f1_score(test_predictions, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Woah! Some heavy overfitting there. Not just from looking at the graph, but also from looking at the difference between the high training score (1.0) and the low testing score (0.7).Let's see if we can find better hyperparameters for this model to do better. We'll use grid search for this.\n",
    "\n",
    "### 4. (TODO) Use grid search to improve this model.\n",
    "\n",
    "In here, we'll do the following steps:\n",
    "1. First define some parameters to perform grid search on. We suggest to play with `max_depth`, `min_samples_leaf`, and `min_samples_split`.\n",
    "2. Make a scorer for the model using `f1_score`.\n",
    "3. Perform grid search on the classifier, using the parameters and the scorer.\n",
    "4. Fit the data to the new classifier.\n",
    "5. Plot the model and find the f1_score.\n",
    "6. If the model is not much better, try changing the ranges for the parameters and fit it again.\n",
    "\n",
    "**_Hint:_ If you're stuck and would like to see a working solution, check the solutions notebook in this same folder.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import make_scorer\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "clf = DecisionTreeClassifier(random_state=42)\n",
    "\n",
    "# TODO: Create the parameters list you wish to tune.\n",
    "parameters = None\n",
    "\n",
    "# TODO: Make an fbeta_score scoring object.\n",
    "scorer = None\n",
    "\n",
    "# TODO: Perform grid search on the classifier using 'scorer' as the scoring method.\n",
    "grid_obj = None\n",
    "\n",
    "# TODO: Fit the grid search object to the training data and find the optimal parameters.\n",
    "grid_fit = None\n",
    "\n",
    "# TODO: Get the estimator.\n",
    "best_clf = None\n",
    "\n",
    "# Fit the new model.\n",
    "best_clf.fit(X_train, y_train)\n",
    "\n",
    "# Make predictions using the new model.\n",
    "best_train_predictions = best_clf.predict(X_train)\n",
    "best_test_predictions = best_clf.predict(X_test)\n",
    "\n",
    "# Calculate the f1_score of the new model.\n",
    "print('The training F1 Score is', f1_score(best_train_predictions, y_train))\n",
    "print('The testing F1 Score is', f1_score(best_test_predictions, y_test))\n",
    "\n",
    "# Plot the new model.\n",
    "plot_model(X, y, best_clf)\n",
    "\n",
    "# Let's also explore what parameters ended up being used in the new model.\n",
    "best_clf"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}