Table of Contents
- Introduction
- Prerequisites
- Setting Up
- Overview of Neural Networks
- Building a Neural Network
- Training the Neural Network
- Making Predictions
- Conclusion
Introduction
In this tutorial, we will learn how to build a Neural Network from scratch in Python. Neural Networks are powerful machine learning models inspired by the human brain. By the end of this tutorial, you will have a clear understanding of the inner workings of Neural Networks and be able to build one yourself. We will cover the basic concepts of Neural Networks, implement a simple Neural Network model, train it on a dataset, and use it to make predictions.
Prerequisites
Before starting this tutorial, you should have a basic understanding of Python programming language, as well as the numpy library for numerical computations. Familiarity with machine learning concepts and linear algebra will be helpful, but not mandatory.
Setting Up
To follow along with the examples in this tutorial, we need to set up our Python environment. Here are the steps to get started:
-
Install Python: If you don’t have Python installed, visit the official Python website and download the latest version for your operating system. Follow the installation instructions to complete the setup.
-
Install numpy: Open a terminal or command prompt and run the following command to install the numpy library:
pip install numpyThis will install the necessary dependencies for numerical computations.
-
Create a new Python file: Open a text editor or an integrated development environment (IDE) of your choice and create a new Python file. Save it with a .py extension.
Now we are ready to dive into the world of Neural Networks!
Overview of Neural Networks
Neural Networks are a type of artificial intelligence models that are capable of learning from data. They consist of interconnected nodes called neurons, which mimic the neurons in the human brain. Each neuron takes inputs, applies a mathematical transformation to them, and produces an output.
The basic building block of a Neural Network is called a perceptron. A perceptron takes input values, multiplies them by corresponding weights, applies an activation function, and produces an output. By combining multiple perceptrons, we can create more complex Neural Networks.
Neural Networks excel at solving complex problems such as image classification, natural language processing, and time series analysis. They have become an essential tool in the field of machine learning.
Building a Neural Network
To build a Neural Network, we will start by defining the structure of the network. This includes the number of input nodes, output nodes, hidden layers, and the number of neurons in each layer. For this tutorial, let’s create a simple Neural Network with one input layer, one hidden layer, and one output layer.
Here is the structure of our simple Neural Network:
- Input layer: 2 nodes
- Hidden layer: 3 nodes
- Output layer: 1 node
Now, let’s create the Neural Network class in Python:
python
class NeuralNetwork:
def __init__(self):
self.weights = {
'hidden': np.random.rand(2, 3),
'output': np.random.rand(3, 1)
}
self.biases = {
'hidden': np.random.rand(1, 3),
'output': np.random.rand(1, 1)
}
In the __init__ method, we initialize the weights and biases of the Neural Network randomly using the np.random.rand function from the numpy library. The weights are stored in a dictionary, where the keys represent the layers (‘hidden’ and ‘output’) and the values are the weight matrices. Similarly, we store the biases in another dictionary.
Next, let’s add the feedforward method to our Neural Network class:
python
def feedforward(self, inputs):
hidden_layer_output = self.sigmoid(np.dot(inputs, self.weights['hidden']) + self.biases['hidden'])
output_layer_output = self.sigmoid(np.dot(hidden_layer_output, self.weights['output']) + self.biases['output'])
return output_layer_output
The feedforward method takes the inputs, computes the outputs of the hidden layer and the output layer using matrix multiplication and the sigmoid activation function, and returns the final output of the Neural Network.
We also need to define the sigmoid activation function:
python
def sigmoid(self, x):
return 1 / (1 + np.exp(-x))
The sigmoid function maps any real number to a value between 0 and 1. It is commonly used in Neural Networks to introduce non-linearity.
Now we have a basic Neural Network structure in place!
Training the Neural Network
To train the Neural Network, we need a labeled dataset. The dataset should contain input features and corresponding target values. For simplicity, let’s create a small dataset manually:
python
inputs = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
targets = np.array([[0], [1], [1], [0]])
Next, let’s add the backpropagation method to our Neural Network class:
```python
def backpropagation(self, inputs, targets, learning_rate):
hidden_layer_output = self.sigmoid(np.dot(inputs, self.weights[‘hidden’]) + self.biases[‘hidden’])
output_layer_output = self.sigmoid(np.dot(hidden_layer_output, self.weights[‘output’]) + self.biases[‘output’])
output_error = targets - output_layer_output
output_delta = output_error * self.sigmoid_derivative(output_layer_output)
hidden_error = output_delta.dot(self.weights[‘output’].T)
hidden_delta = hidden_error * self.sigmoid_derivative(hidden_layer_output)
self.weights['output'] += learning_rate * hidden_layer_output.T.dot(output_delta)
self.biases['output'] += learning_rate * np.sum(output_delta, axis=0, keepdims=True)
self.weights['hidden'] += learning_rate * inputs.T.dot(hidden_delta)
self.biases['hidden'] += learning_rate * np.sum(hidden_delta, axis=0, keepdims=True)
``` In the `backpropagation` method, we first compute the outputs of the hidden layer and the output layer using the feedforward process. Then, we calculate the errors and deltas for both layers. The deltas represent the amount by which the weights and biases should be adjusted to minimize the error.
Finally, we update the weights and biases using the learning rate and the deltas. This process of propagating the errors backward through the network and adjusting the weights and biases is called backpropagation.
We also need to define the sigmoid derivative function:
python
def sigmoid_derivative(self, x):
return x * (1 - x)
The sigmoid derivative represents the rate of change of the sigmoid function at a given point. It is used in backpropagation to adjust the weights and biases.
Now we can train our Neural Network!
python
neural_network = NeuralNetwork()
for epoch in range(10000):
neural_network.backpropagation(inputs, targets, learning_rate=0.1)
In this example, we train the Neural Network for 10000 epochs with a learning rate of 0.1. Adjust the number of epochs and the learning rate based on your specific problem and dataset.
Making Predictions
Once the Neural Network is trained, we can use it to make predictions on new inputs. Here is an example:
python
test_input = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
predictions = neural_network.feedforward(test_input)
print(predictions)
The feedforward method takes the test inputs, computes the corresponding outputs of the Neural Network, and returns the predictions. We can then print the predictions to see the results.
Conclusion
In this tutorial, we have learned how to build a Neural Network from scratch in Python. We started by understanding the basic concepts of Neural Networks and their structure. We then implemented a simple Neural Network class, trained it on a dataset using backpropagation, and made predictions. Neural Networks are versatile models that can be applied to various machine learning tasks. With this knowledge, you can further explore the field of deep learning and build more sophisticated Neural Networks. Keep experimenting and have fun exploring the exciting world of Neural Networks!
This tutorial belongs to the categories: Python Libraries and Modules, Python for Data Science