dtomlinson-cv/README.md

dtomlinson-cv

Repo showcasing example work

Image Classifier using a Nerual network

This is an example of the Udacity project Machine Learning - Introduction Nanodegree Program I completed showing the steps taken to train an image classifier neural network on images of flowers.

The data set is too large for git but examples of the images are available in the jupter workbook. We create a neural network to identify 102 different categories of flowers.

We then create utility functions train.py and predict.py to update the model and predict a flower from the command line, without having to retrain the whole model.

train.py

You can select either Alexnet or VGG13 as a base model

Arguments:

1. data_dir: str - Specify path to data directory
2. save_dir: str - Specify path to output directory
3. arch: str - Choose the base model
4. lrn: float - Specify the learning rate
5. hidden_units: int - Specify the hidden layers
6. epochs: int - Specify the epochs
7. GPU: str - If a GPU is available set this to true

predict.py

Arguments:

1. image_dir: str - Specify path to image
2. load_dir: str - Specify path to load in
3. top_k: str - Specify top k classes that are most likely
4. category_names: str -- Specify JSON of mapping of categories to real names
5. GPU: str - If a GPU is available set this to true

Anomaly Detection using a Neural network

Included is a jupyter notebook outlining the general steps during the development of a Neural Network to identify anomalies in key business metrics.

The general idea is

1. Fit a distribution to the data, once we have this we can find the probability of a value being an anomaly. This turns our problem form unsupervised to supervised as we can say (with a certain probability) whether or not we think a value is an anomaly. This supervised set is passed on.

2. Create a baseline Neural Network, experiment with the hyperparameters and Stochastic Gradient Descent and a Gradient Boosting classifier.

Vrops API

A python wrapper for the VROPS api. No such wrapper exists publicly to automate pulling statistics from VMs using VROPS. See the README.md inside the project for detailed documentation.

This will be released publically as a module available in pypi.org (pip)

Emoji finder

A simple script (for fun) that can lookup an emoji on the command line by name, tag or description.

(Udacity) Supervised Learning Project

Supervised learning project from the Udacity Machine Learning course. This showcases an example of fitting several supervised learning models to a given dataset where the aim is to classify people on how likely they are to donate to a charity, based on their income, job, marital status etc. See the finding_donors.ipynd jupter workbook for a walktrhough through the project.

Splunk dashboard screenshotter

For Splunk there is no easy way to automate screenshots of dashboards. You can set a report to send a single graph as a PDF, or you can use the API to pull your data and use matplotlib (for example) to graph and send your data via email.

This script uses Selenium to log in, hide certain elements, and screenshot the body. It uses postfix to send an email directly from the server.

The script is complete - I will write this as a custom splunk command (| dashboardscreenshot $NAME $EMAIL ) and bundle it into a Splunk app to release publicly on SplunkBase.

Included is an example PNG of a random website to showcase the functionality.