The Project | About Us | Contribute | Donations | License
HOME My courses Completed
◄ Previous
◄

Building an RNN from Scratch

Next ►
►

In this section, we will build a Recurrent Neural Network (RNN) from scratch using PyTorch. RNNs are powerful for sequence data, such as time series, text, and speech. We will cover the following steps:

  1. Understanding RNNs
  2. Setting Up the Environment
  3. Creating the RNN Model
  4. Training the RNN
  5. Evaluating the RNN

  1. Understanding RNNs

Recurrent Neural Networks (RNNs) are a class of neural networks designed to handle sequential data. Unlike traditional feedforward neural networks, RNNs have connections that form directed cycles, allowing them to maintain a memory of previous inputs.

Key Concepts:

  • Hidden State: The hidden state is a vector that captures information from previous time steps.
  • Recurrent Connections: These connections allow the network to pass information from one time step to the next.

  1. Setting Up the Environment

Before we start coding, ensure you have PyTorch installed. You can install it using pip:

pip install torch

We will also use some additional libraries for data handling and visualization:

pip install numpy matplotlib

  1. Creating the RNN Model

Let's start by defining our RNN model. We will use PyTorch's nn.Module to create our custom RNN.

Code Example:

import torch
import torch.nn as nn

class SimpleRNN(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(SimpleRNN, self).__init__()
        self.hidden_size = hidden_size
        self.rnn = nn.RNN(input_size, hidden_size, batch_first=True)
        self.fc = nn.Linear(hidden_size, output_size)
    
    def forward(self, x):
        h0 = torch.zeros(1, x.size(0), self.hidden_size).to(x.device)
        out, _ = self.rnn(x, h0)
        out = self.fc(out[:, -1, :])
        return out

Explanation:

  • __init__ Method: Initializes the RNN with an input size, hidden size, and output size. We define an RNN layer and a fully connected layer.
  • forward Method: Defines the forward pass. We initialize the hidden state h0, pass the input through the RNN, and then through the fully connected layer.

  1. Training the RNN

Next, we will train our RNN on a simple dataset. For this example, let's use a sine wave as our dataset.

Code Example:

import numpy as np
import matplotlib.pyplot as plt

# Generate sine wave data
def generate_sine_wave(seq_length, num_samples):
    X = np.linspace(0, 100, num_samples)
    y = np.sin(X)
    data = []
    for i in range(len(y) - seq_length):
        data.append(y[i:i+seq_length+1])
    data = np.array(data)
    return data[:, :-1], data[:, -1]

seq_length = 10
num_samples = 1000
X, y = generate_sine_wave(seq_length, num_samples)

# Convert to PyTorch tensors
X = torch.tensor(X, dtype=torch.float32).unsqueeze(-1)
y = torch.tensor(y, dtype=torch.float32)

# Split into training and test sets
train_size = int(0.8 * len(X))
X_train, X_test = X[:train_size], X[train_size:]
y_train, y_test = y[:train_size], y[train_size:]

# Initialize the model, loss function, and optimizer
input_size = 1
hidden_size = 50
output_size = 1
model = SimpleRNN(input_size, hidden_size, output_size)
criterion = nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.01)

# Training loop
num_epochs = 100
for epoch in range(num_epochs):
    model.train()
    outputs = model(X_train)
    loss = criterion(outputs, y_train)
    
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    
    if (epoch+1) % 10 == 0:
        print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {loss.item():.4f}')

Explanation:

  • Data Generation: We generate a sine wave and create sequences of length seq_length.
  • Data Preparation: Convert the data to PyTorch tensors and split it into training and test sets.
  • Model Initialization: Initialize the RNN model, loss function (Mean Squared Error), and optimizer (Adam).
  • Training Loop: Train the model for a specified number of epochs, printing the loss every 10 epochs.

  1. Evaluating the RNN

Finally, we will evaluate our trained RNN on the test set and visualize the results.

Code Example:

# Evaluation
model.eval()
with torch.no_grad():
    predictions = model(X_test)
    loss = criterion(predictions, y_test)
    print(f'Test Loss: {loss.item():.4f}')

# Plot the results
plt.plot(y_test.numpy(), label='True')
plt.plot(predictions.numpy(), label='Predicted')
plt.legend()
plt.show()

Explanation:

  • Evaluation: Set the model to evaluation mode and compute the loss on the test set.
  • Visualization: Plot the true and predicted values to visualize the model's performance.

Conclusion

In this section, we built a simple RNN from scratch using PyTorch. We covered the following steps:

  • Understanding the basic concepts of RNNs.
  • Setting up the environment.
  • Creating the RNN model.
  • Training the RNN on a sine wave dataset.
  • Evaluating and visualizing the RNN's performance.

In the next section, we will dive deeper into more advanced RNN architectures, such as Long Short-Term Memory (LSTM) networks.

◄ Previous
◄
Next ►
►
The Project | About Us | Contribute | Donations | License
© Copyright 2024. All rights reserved