Neural Network Classifier

Table of Contents

1. About The Project
2. Usage
   • Requirements
   • Run
3. Phases
   • Preprocess Data
   • Forward Propagation
   • Non-Vectorized Backward Propagetion
   • Vectorized Backward Propagetion
   • Final Test
   • Hyperparameters Analyzing
   • Learning Rate Decay
   • Deeper Network !!!
4. License
5. Contact

About The Project

This project implements a feed-forward neural network from scratch with numpy and uses the 360 Fruits Dataset for training and testing model. there are seven phases in this project which complete neural network implementation step by step, which includes the following steps

• Preprocess Data
• Forward Propagation
• Non-Vectorized Backward Propagetion
• Vectorized Backward Propagetion
• Final Test
• Hyperparameters Analyzing
• Learning Rate Decay
• Deeper Network !!!

As input data are images, so there are some feature extraction codes that can transform images into fixed dimensional vectors, so the model uses .pkl extension files as input (not direct images). In the beginning, non-vectorized backward propagation will be implemented, and after that implementation will be changed to vectorized backward propagation.

(back to top)

Usage

Requirements

Project uses numpy array for increasing speed and Linear algebra, then you should install numpy library

$ pip install numpy

also, project needs matplotlib for visualizing plots

$ pip install matplotlib

and you should install tqdm for better experince (you can delete it from code if you want)

$ pip install tqdm

(optional)
if you want to run feature extraction codes yourself, please install scikit-image too

$ pip install scikit-image

Run

Run all cells in order, but you should run phase 7 before running the train function's cell at phase 3.

(back to top)

Phases

Preprocess Data

In this phase data is transformed from images into vectors, at beginning feature extraction reads all images and with the use of histogram function and number of 360 bins, images are transformed into vectors, then features with specific standard deviation are deleted, and finally, data is ready!

Forward Propagation

In this phase forward function is implemented and weights and biases are initialized, also the test function is implemented for testing the accuracy of the model. As activation function, this model uses sigmoid for all layers.

Non-Vectorized Backward Propagetion

In this phase nonvectorized backward function is implemented, but it is so slow.

Vectorized Backward Propagetion

In this phase vectorized backward function is implemented, as you can see it is super fast.

Final Test

Hyperparameters Analyzing

Run cells to see effect of different hyperparameters on accuracy.

Learning Rate Decay

I used a custom learning decay for improving convergence.

class LR_Decay:
    def __init__(self, initial_lr, k):
        self.initial_lr = initial_lr
        self.t = 0
        self.k = k
        
    def step(self):
        value = self.initial_lr * np.exp(-self.k * self.t)
        self.t += 1
        return max(value, 1)

Deeper Network !!!

In the last phase, 3 fruits were added to the dataset, so there are seven fruits and also we need Deeper Model !!!

⚠️ warning: feel free to add more fruits to the dataset, but you should modify feature extraction a little bit and make your model more complex.

(back to top)

License

Distributed under the MIT license. See LICENSE.md for more information.

(back to top)

Contact

Amirreza Naziri
Email: naziriamirreza@gmail.com

(back to top)