# Udacity Deep Learning Nanodegree

** Published:**

I completed Udacity’s Deep Learning Foundation Nanodegree Program. In this post I will share and discuss my projects that I completed. I will also share resources I found useful for certain topics within deep learning (specifically topics covered in the nanodgree). This course was broken up into 4 sections, Neural Networks, Convolutional Neural Networks, Recurrent Neural Networks, and Generative Adversarial Networks.

All code will be uploaded to github soon!

## Preliminary Information

## Neural Networks

**Overview:**In this section we had an intro to the course and got introduced to platforms such as Ancaonda, tools like Jupyter Notebooks, and libraries like Pandas, Scikit-learn, and Matplotlib. We also looked at real applications of deep learning by exploring some already working code. We looked at Style Transfer, DeepTraffic, and Flappy Bird. In this section we were also recommended books (Grokking Deep Learning, Neural Networks and Deep Learning, Deep Learning - links below). We also looked into basic concepts like regression, matrix math, and numpy. After all of the intro we finally explored neural network basics which included studying the perceptron, gradient descent, multilayer perceptrons (fully connected network), and backpropagation.**Resources:**- Conda Documentation
- Jupyter
- 10 Minutes to pandas
- Scikit-Learn's offical tutorial
- Matplotlib's offical tutorial
- Style Transfer
- DeepTraffict
- Flappy Bird
- Grokking Deep Learning by Andrew Task
- Neural Networks and Deep Learning by Michael Nielsen
- Deep Learning by Ian Goodfello, Yoshua Bengia, and Aaron Courville
- Yes you should understand backprop
- CS231n Winter 2016 Lec 4 Backprop

**Project 1 'Your First Neural Network'**- This description is copied directly from Udacity: 'In this project, you'll get to build a neural network from scratch to carry out a prediction problem on a real dataset! By building a neural network from the ground up, you'll have a much better understanding of gradient descent, backpropagation, and other concepts that are important to know before we move to higher level tools such as Tensorflow. You'll also get to see how to apply these networks to solve real prediction problems!'
- link to code coming soon

## Convolutional Neural Networks

**Overview:**In this section we got introduced into concepts like model evaluation and validation. Then we had a slight tangent and discussed Sentiment Analysis with Andrew Trask. We were then introduced int TFLearn, Cloud Computing (using AWS), Keras, and had an introduction to TensorFlow. After covering the basics we dove into Deep Neural Networks and learned the basics of CNNs and their components. We looked into parameter sharing, filters (padding, stride, width, height depth) and basics of convolution. We also explore concepts like max pooling and RELU activations. Finally, we explored image classification.**Resources:****Project 2 'Image Classification'**- This description is copied directly from Udacity: In this project, you'll classify images from the CIFAR-10 dataset. The dataset consists of airplanes, dogs, cats, and other objects. The dataset will need to be preprocessed, then train a convolutional neural network on all the samples. You'll normalize the images, one-hot encode the labels, build a convolutional layer, max pool layer, and fully connected layer. At then end, you'll see their predictions on the sample images.
- link to code coming soon

## Recurrent Neural Networks

**Overview:**This section started with an introduction to RNNs. Siraj, walked us through the first sequential dataset example we examined which was stock prediction.Then we looked at hyperparameters, word embeddings, and text summarization. We got introduced to TensorBoard. Other examples of using RNNs was with music generation and sentiment prediction. Then came the 3rd project. After the third project we looked at transfer learning in TensorFlow, a sequence to sequence architecture, and an intro to reinforcement learning. This section ended with the 4th project. Reference list is long because this section contained two projects!**Resources:**- cs231n Lec 10 RNN
- Understanding LSTM Networks colah's blog
- RNN in Tensorflow
- LSTM Networks for Sentiment Analysis
- A Beginner's Guide to RNNs and LSTMs
- Time Series Prediction with LSTM in Keras
- Word2Vec Tutorial-The Skip-Gram Model
- Efficient Estimation of Word Representations in Vector Space
- Vector Representations of Words
- Hands-on TensorBoard (TensorFlow Dev Summit 2017)
- TensorBoard: Visualizing Learning
- Siraj's Music Generation in TensorFlow Note: He has additional references in video description.
- How to Make a Text Summarizer - Siraj
- cs231n Transfer Learning
- Pre Trained VGGnet
- How to retrain inception's final layer
- Siraj's Language Translation
- Tensorflow-seq2seq tutorial
- tf-stanford-tutorials chatbots
- Seq2seq Deep Learning (Quoc Le, Google)
- Siraj's Chatbot
- Simple Reinforcement Learning with Tensorflow Part 0 This is an entire series and it's awesome. This link goes to the beginning
- DQN-Tensorflow
- Cart-Pole Balancing with Q-Learning
- Siraj's Reinforcement Learning
- Siraj's Image Generation
- Deconvolution and Checkerboard Artifacts

**Project 3 'Generate TV Scripts**- This description is copied directly from Udacity: In this project, you'll generate your own Simpsons TV scripts using RNNs. You'll be using part of the Simpsons dataset of scripts from 27 seasons. The Neural Network you'll build will generate a new TV script for a scene at Moe's Tavern.
- link to code coming soon

**Project 4 'Translation Project**- This description is copied directly from Udacity: In this project, you’re going to take a peek into the realm of neural network machine translation. You’ll be training a sequence to sequence model on a dataset of English and French sentences that can translate new sentences from English to French.
- link to code coming soon

## Generative Adversarial Networks

**Overview:**This was the final section in our studies. The lessons included a section on GANs, video generation, one-shot learning, hyperparameters, deep convolutional GANs.**Resources:****Project 5 'Generate Faces'**- This description is copied directly from Udacity: In this project, you'll use generative adversarial networks to generate new images of faces.
- link to code coming soon