The Project | About Us | Contribute | Donations | License
HOME My courses Completed
◄ Previous
◄

Creating a Simple Neural Network

Next ►
►

In this section, we will learn how to create a simple neural network using PyTorch. We will cover the following steps:

  1. Defining the Network Architecture
  2. Initializing the Network
  3. Forward Pass
  4. Backward Pass and Optimization

  1. Defining the Network Architecture

A neural network in PyTorch is defined by creating a class that inherits from torch.nn.Module. This class will contain the layers of the network and the forward pass logic.

Example:

import torch
import torch.nn as nn
import torch.nn.functional as F

class SimpleNeuralNetwork(nn.Module):
    def __init__(self):
        super(SimpleNeuralNetwork, self).__init__()
        # Define layers
        self.fc1 = nn.Linear(784, 128)  # Input layer to hidden layer
        self.fc2 = nn.Linear(128, 64)   # Hidden layer to another hidden layer
        self.fc3 = nn.Linear(64, 10)    # Hidden layer to output layer

    def forward(self, x):
        # Define forward pass
        x = F.relu(self.fc1(x))  # Apply ReLU activation function
        x = F.relu(self.fc2(x))  # Apply ReLU activation function
        x = self.fc3(x)          # Output layer
        return x

Explanation:

  • __init__ Method: This method initializes the layers of the network. We use nn.Linear to create fully connected layers.
  • forward Method: This method defines the forward pass of the network. We use F.relu to apply the ReLU activation function to the outputs of the layers.

  1. Initializing the Network

Once the network architecture is defined, we need to create an instance of the network.

Example:

# Create an instance of the network
model = SimpleNeuralNetwork()
print(model)

Explanation:

  • model: This variable holds the instance of the SimpleNeuralNetwork class. Printing the model will display the architecture of the network.

  1. Forward Pass

The forward pass involves passing input data through the network to get the output.

Example:

# Create a random tensor with shape (1, 784) to simulate a single input
input_data = torch.randn(1, 784)

# Perform a forward pass
output = model(input_data)
print(output)

Explanation:

  • input_data: This tensor simulates a single input to the network. The shape (1, 784) represents a batch size of 1 and 784 input features.
  • output: This tensor contains the output of the network after the forward pass.

  1. Backward Pass and Optimization

To train the network, we need to define a loss function and an optimizer. The backward pass involves computing the gradients and updating the weights.

Example:

import torch.optim as optim

# Define a loss function
criterion = nn.CrossEntropyLoss()

# Define an optimizer
optimizer = optim.SGD(model.parameters(), lr=0.01)

# Simulate a target output
target = torch.tensor([3])  # Assume the correct class is 3

# Zero the gradients
optimizer.zero_grad()

# Perform a forward pass
output = model(input_data)

# Compute the loss
loss = criterion(output, target)

# Perform a backward pass
loss.backward()

# Update the weights
optimizer.step()

print(f'Loss: {loss.item()}')

Explanation:

  • criterion: This variable holds the loss function. We use nn.CrossEntropyLoss for classification tasks.
  • optimizer: This variable holds the optimizer. We use stochastic gradient descent (SGD) with a learning rate of 0.01.
  • target: This tensor contains the correct class for the input data.
  • optimizer.zero_grad(): This method clears the gradients of all optimized tensors.
  • loss.backward(): This method computes the gradient of the loss with respect to the network parameters.
  • optimizer.step(): This method updates the network parameters based on the computed gradients.

Practical Exercise

Task:

  1. Define a neural network with the following architecture:

    • Input layer: 784 units
    • Hidden layer 1: 256 units, ReLU activation
    • Hidden layer 2: 128 units, ReLU activation
    • Output layer: 10 units

  2. Initialize the network and print its architecture.

  3. Create a random input tensor with shape (1, 784) and perform a forward pass.

  4. Define a loss function and an optimizer.

  5. Simulate a target output and perform a backward pass and optimization step.

Solution:

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

class CustomNeuralNetwork(nn.Module):
    def __init__(self):
        super(CustomNeuralNetwork, self).__init__()
        self.fc1 = nn.Linear(784, 256)
        self.fc2 = nn.Linear(256, 128)
        self.fc3 = nn.Linear(128, 10)

    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

# Initialize the network
model = CustomNeuralNetwork()
print(model)

# Create a random input tensor
input_data = torch.randn(1, 784)

# Perform a forward pass
output = model(input_data)
print(output)

# Define a loss function and an optimizer
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.01)

# Simulate a target output
target = torch.tensor([3])

# Zero the gradients
optimizer.zero_grad()

# Compute the loss
loss = criterion(output, target)

# Perform a backward pass
loss.backward()

# Update the weights
optimizer.step()

print(f'Loss: {loss.item()}')

Conclusion

In this section, we learned how to create a simple neural network in PyTorch. We covered defining the network architecture, initializing the network, performing a forward pass, and executing the backward pass and optimization. These steps form the foundation for building and training more complex neural networks. In the next section, we will delve deeper into activation functions and their role in neural networks.

◄ Previous
◄
Next ►
►
The Project | About Us | Contribute | Donations | License
© Copyright 2024. All rights reserved