{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature Scaling - Solution\n",
    "\n",
    "With any distance based machine learning model (regularized regression methods, neural networks, and now kmeans), you will want to scale your data.  \n",
    "\n",
    "If you have some features that are on completely different scales, this can greatly impact the clusters you get when using K-Means. \n",
    "\n",
    "In this notebook, you will get to see this first hand.  To begin, let's read in the necessary libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.cluster import KMeans\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn import preprocessing as p\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "plt.rcParams['figure.figsize'] = (16, 9)\n",
    "import helpers2 as h\n",
    "import tests as t\n",
    "\n",
    "\n",
    "# Create the dataset for the notebook\n",
    "data = h.simulate_data(200, 2, 4)\n",
    "df = pd.DataFrame(data)\n",
    "df.columns = ['height', 'weight']\n",
    "df['height'] = np.abs(df['height']*100)\n",
    "df['weight'] = df['weight'] + np.random.normal(50, 10, 200)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`1.` Next, take a look at the data to get familiar with it.  The dataset has two columns, and it is stored in the **df** variable.  It might be useful to get an idea of the spread in the current data, as well as a visual of the points.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>height</th>\n",
       "      <th>weight</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>200.000000</td>\n",
       "      <td>200.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>569.726207</td>\n",
       "      <td>52.141503</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>246.966215</td>\n",
       "      <td>11.342247</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>92.998481</td>\n",
       "      <td>16.263732</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>357.542793</td>\n",
       "      <td>45.228737</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>545.766752</td>\n",
       "      <td>52.065198</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>773.310607</td>\n",
       "      <td>61.207241</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1096.222348</td>\n",
       "      <td>77.908867</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            height      weight\n",
       "count   200.000000  200.000000\n",
       "mean    569.726207   52.141503\n",
       "std     246.966215   11.342247\n",
       "min      92.998481   16.263732\n",
       "25%     357.542793   45.228737\n",
       "50%     545.766752   52.065198\n",
       "75%     773.310607   61.207241\n",
       "max    1096.222348   77.908867"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 19184 characters]\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff0789bb1d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(df['height'], df['weight']);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we've got a dataset, let's look at some options for scaling the data.  As well as how the data might be scaled.  There are two very common types of feature scaling that we should discuss:\n",
    "\n",
    "\n",
    "**I.  MinMaxScaler**\n",
    "\n",
    "In some cases it is useful to think of your data in terms of the percent they are as compared to the maximum value.  In these cases, you will want to use **MinMaxScaler**.\n",
    "\n",
    "**II. StandardScaler**\n",
    "\n",
    "Another very popular type of scaling is to scale data so that it has mean 0 and variance 1.  In these cases, you will want to use **StandardScaler**.  \n",
    "\n",
    "It is probably more appropriate with this data to use **StandardScaler**.  However, to get practice with feature scaling methods in python, we will perform both.\n",
    "\n",
    "`2.` First let's fit the **StandardScaler** transformation to this dataset.  I will do this one so you can see how to apply preprocessing in sklearn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_ss = p.StandardScaler().fit_transform(df) # Fit and transform the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 18372 characters]\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff071a04c18>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_ss = pd.DataFrame(df_ss) #create a dataframe\n",
    "df_ss.columns = ['height', 'weight'] #add column names again\n",
    "\n",
    "plt.scatter(df_ss['height'], df_ss['weight']); # create a plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`3.` Now it's your turn.  Try fitting the **MinMaxScaler** transformation to this dataset. You should be able to use the previous example to assist."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_mm = p.MinMaxScaler().fit_transform(df) # fit and transform"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 18744 characters]\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff06f9adfd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_mm = pd.DataFrame(df_mm) #create a dataframe\n",
    "df_mm.columns = ['height', 'weight'] #change the column names\n",
    "\n",
    "plt.scatter(df_mm['height'], df_mm['weight']); #plot the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`4.`  Now let's take a look at how kmeans divides the dataset into different groups for each of the different scalings of the data.  Did you end up with different clusters when the data was scaled differently?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 55480 characters]\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff06f673f98>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def fit_kmeans(data, centers):\n",
    "    '''\n",
    "    INPUT:\n",
    "        data = the dataset you would like to fit kmeans to (dataframe)\n",
    "        centers = the number of centroids (int)\n",
    "    OUTPUT:\n",
    "        labels - the labels for each datapoint to which group it belongs (nparray)\n",
    "    \n",
    "    '''\n",
    "    kmeans = KMeans(centers)\n",
    "    labels = kmeans.fit_predict(data)\n",
    "    return labels\n",
    "\n",
    "labels = fit_kmeans(df, 10) #fit kmeans to get the labels\n",
    "    \n",
    "# Plot the original data with clusters\n",
    "plt.scatter(df['height'], df['weight'], c=labels, cmap='Set1');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 53160 characters]\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff06f8070b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "labels = fit_kmeans(df_mm, 10) #fit kmeans to get the labels\n",
    "    \n",
    "#plot each of the scaled datasets\n",
    "plt.scatter(df_mm['height'], df_mm['weight'], c=labels, cmap='Set1');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 51828 characters]\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff06f768128>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "labels = fit_kmeans(df_ss, 10)\n",
    "\n",
    "plt.scatter(df_ss['height'], df_ss['weight'], c=labels, cmap='Set1');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Different from what was stated in the video - In this case, the scaling did end up changing the results.  In the video, the kmeans algorithm was not refit to each differently scaled dataset.  It was only using the one clustering fit on every dataset.  In this notebook, you see that clustering was recomputed with each scaling, which changes the results!**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
