adding projects to repo

This commit is contained in:
dtomlinson
2019-11-04 14:38:31 +00:00
parent 6728b1ecf3
commit a285fad1ee
65 changed files with 171244 additions and 1 deletions

View File

@@ -1,3 +1,69 @@
# dtomlinson-cv
Repo showcasing example work
Repo showcasing example work
## Image Classifier using a Nerual network
This is an example of the Udacity project `Machine Learning - Introduction Nanodegree Program` I completed showing the steps taken to train an image classifier neural network on images of flowers.
The data set is too large for git but examples of the images are available in the jupter workbook. We create a neural network to identify 102 different categories of flowers.
We then create utility functions `train.py` and `predict.py` to update the model and predict a flower from the command line, without having to retrain the whole model.
### `train.py`
You can select either Alexnet or VGG13 as a base model
Arguments:
1. `data_dir: str` - Specify path to data directory
2. `save_dir: str` - Specify path to output directory
3. `arch: str` - Choose the base model
4. `lrn: float` - Specify the learning rate
5. `hidden_units: int` - Specify the hidden layers
6. `epochs: int` - Specify the epochs
7. `GPU: str` - If a GPU is available set this to true
### `predict.py`
Arguments:
1. `image_dir: str` - Specify path to image
2. `load_dir: str` - Specify path to load in
3. `top_k: str` - Specify top k classes that are most likely
4. `category_names: str` -- Specify JSON of mapping of categories to real names
5. `GPU: str` - If a GPU is available set this to true
## Anomaly Detection using a Neural network
Included is a jupyter notebook outlining the general steps during the development of a Neural Network to identify anomalies in key business metrics.
The general idea is
1. Fit a distribution to the data, once we have this we can find the probability of a value being an anomaly. This turns our problem form unsupervised to supervised as we can say (with a certain probability) whether or not we think a value is an anomaly. This supervised set is passed on.
2. Create a baseline Neural Network, experiment with the hyperparameters and Stochastic Gradient Descent and a Gradient Boosting classifier.
## Vrops API
A python wrapper for the VROPS api. No such wrapper exists publicly to automate pulling statistics from VMs using VROPS. See the `README.md` inside the project for detailed documentation.
This will be released publically as a module available in pypi.org (`pip`)
## Emoji finder
A simple script (for fun) that can lookup an emoji on the command line by name, tag or description.
## (Udacity) Supervised Learning Project
Supervised learning project from the Udacity Machine Learning course. This showcases an example of fitting several supervised learning models to a given dataset where the aim is to classify people on how likely they are to donate to a charity, based on their income, job, marital status etc. See the `finding_donors.ipynd` jupter workbook for a walktrhough through the project.
## Splunk dashboard screenshotter
For Splunk there is no easy way to automate screenshots of dashboards. You can set a report to send a single graph as a PDF, or you can use the API to pull your data and use matplotlib (for example) to graph and send your data via email.
This script uses Selenium to log in, hide certain elements, and screenshot the body. It uses postfix to send an email directly from the server.
The script is complete - I will write this as a custom splunk command `(| dashboardscreenshot $NAME $EMAIL )` and bundle it into a Splunk app to release publicly on SplunkBase.
Included is an example PNG of a random website to showcase the functionality.