project 1 updated

python/Supervised Learning/Project/README.md

@@ -0,0 +1,56 @@
+# Data Scientist Nanodegree
+# Supervised Learning
+## Project: Finding Donors for CharityML
+
+### Install
+
+This project requires **Python 3.x** and the following Python libraries installed:
+
+- [NumPy](http://www.numpy.org/)
+- [Pandas](http://pandas.pydata.org)
+- [matplotlib](http://matplotlib.org/)
+- [scikit-learn](http://scikit-learn.org/stable/)
+
+You will also need to have software installed to run and execute an [iPython Notebook](http://ipython.org/notebook.html)
+
+We recommend students install [Anaconda](https://www.continuum.io/downloads), a pre-packaged Python distribution that contains all of the necessary libraries and software for this project.
+
+### Code
+
+Template code is provided in the `finding_donors.ipynb` notebook file. You will also be required to use the included `visuals.py` Python file and the `census.csv` dataset file to complete your work. While some code has already been implemented to get you started, you will need to implement additional functionality when requested to successfully complete the project. Note that the code included in `visuals.py` is meant to be used out-of-the-box and not intended for students to manipulate. If you are interested in how the visualizations are created in the notebook, please feel free to explore this Python file.
+
+### Run
+
+In a terminal or command window, navigate to the top-level project directory `finding_donors/` (that contains this README) and run one of the following commands:
+
+```bash
+ipython notebook finding_donors.ipynb
+```  
+or
+```bash
+jupyter notebook finding_donors.ipynb
+```
+
+This will open the iPython Notebook software and project file in your browser.
+
+### Data
+
+The modified census dataset consists of approximately 32,000 data points, with each datapoint having 13 features. This dataset is a modified version of the dataset published in the paper *"Scaling Up the Accuracy of Naive-Bayes Classifiers: a Decision-Tree Hybrid",* by Ron Kohavi. You may find this paper [online](https://www.aaai.org/Papers/KDD/1996/KDD96-033.pdf), with the original dataset hosted on [UCI](https://archive.ics.uci.edu/ml/datasets/Census+Income).
+
+**Features**
+- `age`: Age
+- `workclass`: Working Class (Private, Self-emp-not-inc, Self-emp-inc, Federal-gov, Local-gov, State-gov, Without-pay, Never-worked)
+- `education_level`: Level of Education (Bachelors, Some-college, 11th, HS-grad, Prof-school, Assoc-acdm, Assoc-voc, 9th, 7th-8th, 12th, Masters, 1st-4th, 10th, Doctorate, 5th-6th, Preschool)
+- `education-num`: Number of educational years completed
+- `marital-status`: Marital status (Married-civ-spouse, Divorced, Never-married, Separated, Widowed, Married-spouse-absent, Married-AF-spouse)
+- `occupation`: Work Occupation (Tech-support, Craft-repair, Other-service, Sales, Exec-managerial, Prof-specialty, Handlers-cleaners, Machine-op-inspct, Adm-clerical, Farming-fishing, Transport-moving, Priv-house-serv, Protective-serv, Armed-Forces)
+- `relationship`: Relationship Status (Wife, Own-child, Husband, Not-in-family, Other-relative, Unmarried)
+- `race`: Race (White, Asian-Pac-Islander, Amer-Indian-Eskimo, Other, Black)
+- `sex`: Sex (Female, Male)
+- `capital-gain`: Monetary Capital Gains
+- `capital-loss`: Monetary Capital Losses
+- `hours-per-week`: Average Hours Per Week Worked
+- `native-country`: Native Country (United-States, Cambodia, England, Puerto-Rico, Canada, Germany, Outlying-US(Guam-USVI-etc), India, Japan, Greece, South, China, Cuba, Iran, Honduras, Philippines, Italy, Poland, Jamaica, Vietnam, Mexico, Portugal, Ireland, France, Dominican-Republic, Laos, Ecuador, Taiwan, Haiti, Columbia, Hungary, Guatemala, Nicaragua, Scotland, Thailand, Yugoslavia, El-Salvador, Trinadad&Tobago, Peru, Hong, Holand-Netherlands)
+
+**Target Variable**
+- `income`: Income Class (<=50K, >50K)

python/Supervised Learning/Project/visuals.py

@@ -0,0 +1,143 @@
+###########################################
+# Suppress matplotlib user warnings
+# Necessary for newer version of matplotlib
+import warnings
+warnings.filterwarnings("ignore", category = UserWarning, module = "matplotlib")
+#
+# Display inline matplotlib plots with IPython
+from IPython import get_ipython
+get_ipython().run_line_magic('matplotlib', 'inline')
+###########################################
+
+import matplotlib.pyplot as pl
+import matplotlib.patches as mpatches
+import numpy as np
+import pandas as pd
+from time import time
+from sklearn.metrics import f1_score, accuracy_score
+
+
+def distribution(data, transformed = False):
+    """
+    Visualization code for displaying skewed distributions of features
+    """
+    
+    # Create figure
+    fig = pl.figure(figsize = (11,5));
+
+    # Skewed feature plotting
+    for i, feature in enumerate(['capital-gain','capital-loss']):
+        ax = fig.add_subplot(1, 2, i+1)
+        ax.hist(data[feature], bins = 25, color = '#00A0A0')
+        ax.set_title("'%s' Feature Distribution"%(feature), fontsize = 14)
+        ax.set_xlabel("Value")
+        ax.set_ylabel("Number of Records")
+        ax.set_ylim((0, 2000))
+        ax.set_yticks([0, 500, 1000, 1500, 2000])
+        ax.set_yticklabels([0, 500, 1000, 1500, ">2000"])
+
+    # Plot aesthetics
+    if transformed:
+        fig.suptitle("Log-transformed Distributions of Continuous Census Data Features", \
+            fontsize = 16, y = 1.03)
+    else:
+        fig.suptitle("Skewed Distributions of Continuous Census Data Features", \
+            fontsize = 16, y = 1.03)
+
+    fig.tight_layout()
+    fig.show()
+
+
+def evaluate(results, accuracy, f1):
+    """
+    Visualization code to display results of various learners.
+    
+    inputs:
+      - learners: a list of supervised learners
+      - stats: a list of dictionaries of the statistic results from 'train_predict()'
+      - accuracy: The score for the naive predictor
+      - f1: The score for the naive predictor
+    """
+  
+    # Create figure
+    fig, ax = pl.subplots(2, 3, figsize = (11,7))
+
+    # Constants
+    bar_width = 0.3
+    colors = ['#A00000','#00A0A0','#00A000']
+    
+    # Super loop to plot four panels of data
+    for k, learner in enumerate(results.keys()):
+        for j, metric in enumerate(['train_time', 'acc_train', 'f_train', 'pred_time', 'acc_test', 'f_test']):
+            for i in np.arange(3):
+                
+                # Creative plot code
+                ax[j//3, j%3].bar(i+k*bar_width, results[learner][i][metric], width = bar_width, color = colors[k])
+                ax[j//3, j%3].set_xticks([0.45, 1.45, 2.45])
+                ax[j//3, j%3].set_xticklabels(["1%", "10%", "100%"])
+                ax[j//3, j%3].set_xlabel("Training Set Size")
+                ax[j//3, j%3].set_xlim((-0.1, 3.0))
+    
+    # Add unique y-labels
+    ax[0, 0].set_ylabel("Time (in seconds)")
+    ax[0, 1].set_ylabel("Accuracy Score")
+    ax[0, 2].set_ylabel("F-score")
+    ax[1, 0].set_ylabel("Time (in seconds)")
+    ax[1, 1].set_ylabel("Accuracy Score")
+    ax[1, 2].set_ylabel("F-score")
+    
+    # Add titles
+    ax[0, 0].set_title("Model Training")
+    ax[0, 1].set_title("Accuracy Score on Training Subset")
+    ax[0, 2].set_title("F-score on Training Subset")
+    ax[1, 0].set_title("Model Predicting")
+    ax[1, 1].set_title("Accuracy Score on Testing Set")
+    ax[1, 2].set_title("F-score on Testing Set")
+    
+    # Add horizontal lines for naive predictors
+    ax[0, 1].axhline(y = accuracy, xmin = -0.1, xmax = 3.0, linewidth = 1, color = 'k', linestyle = 'dashed')
+    ax[1, 1].axhline(y = accuracy, xmin = -0.1, xmax = 3.0, linewidth = 1, color = 'k', linestyle = 'dashed')
+    ax[0, 2].axhline(y = f1, xmin = -0.1, xmax = 3.0, linewidth = 1, color = 'k', linestyle = 'dashed')
+    ax[1, 2].axhline(y = f1, xmin = -0.1, xmax = 3.0, linewidth = 1, color = 'k', linestyle = 'dashed')
+    
+    # Set y-limits for score panels
+    ax[0, 1].set_ylim((0, 1))
+    ax[0, 2].set_ylim((0, 1))
+    ax[1, 1].set_ylim((0, 1))
+    ax[1, 2].set_ylim((0, 1))
+
+    # Create patches for the legend
+    patches = []
+    for i, learner in enumerate(results.keys()):
+        patches.append(mpatches.Patch(color = colors[i], label = learner))
+    pl.legend(handles = patches, bbox_to_anchor = (-.80, 2.53), \
+               loc = 'upper center', borderaxespad = 0., ncol = 3, fontsize = 'x-large')
+    
+    # Aesthetics
+    pl.suptitle("Performance Metrics for Three Supervised Learning Models", fontsize = 16, y = 1.10)
+    pl.tight_layout()
+    pl.show()
+    
+
+def feature_plot(importances, X_train, y_train):
+    
+    # Display the five most important features
+    indices = np.argsort(importances)[::-1]
+    columns = X_train.columns.values[indices[:5]]
+    values = importances[indices][:5]
+
+    # Creat the plot
+    fig = pl.figure(figsize = (9,5))
+    pl.title("Normalized Weights for First Five Most Predictive Features", fontsize = 16)
+    pl.bar(np.arange(5), values, width = 0.6, align="center", color = '#00A000', \
+          label = "Feature Weight")
+    pl.bar(np.arange(5) - 0.3, np.cumsum(values), width = 0.2, align = "center", color = '#00A0A0', \
+          label = "Cumulative Feature Weight")
+    pl.xticks(np.arange(5), columns)
+    pl.xlim((-0.5, 4.5))
+    pl.ylabel("Weight", fontsize = 12)
+    pl.xlabel("Feature", fontsize = 12)
+    
+    pl.legend(loc = 'upper center')
+    pl.tight_layout()
+    pl.show()