The Project | About Us | Contribute | Donations | License
HOME My courses Completed
◄ Previous
◄

Generative Adversarial Networks (GANs)

Next ►
►

Generative Adversarial Networks (GANs) are a class of machine learning frameworks designed by Ian Goodfellow and his colleagues in 2014. GANs consist of two neural networks, the generator and the discriminator, which compete against each other in a zero-sum game. The generator creates data that mimics real data, while the discriminator evaluates the authenticity of the data. This competition drives both networks to improve over time.

Key Concepts

  1. Generator

  • Purpose: To generate data that is indistinguishable from real data.
  • Input: Random noise (usually from a normal distribution).
  • Output: Synthetic data (e.g., images, text).

  1. Discriminator

  • Purpose: To distinguish between real data and data generated by the generator.
  • Input: Both real data and synthetic data.
  • Output: Probability that the input data is real.

  1. Adversarial Training

  • Objective: The generator tries to fool the discriminator, while the discriminator tries to correctly identify real vs. fake data.
  • Loss Functions:
    • Generator Loss: Measures how well the generator fools the discriminator.
    • Discriminator Loss: Measures how well the discriminator distinguishes real from fake data.

Practical Example: Building a Simple GAN

Step 1: Import Libraries

import torch
import torch.nn as nn
import torch.optim as optim
import torchvision
import torchvision.transforms as transforms
import matplotlib.pyplot as plt
import numpy as np

Step 2: Define the Generator and Discriminator

class Generator(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(Generator, self).__init__()
        self.main = nn.Sequential(
            nn.Linear(input_size, hidden_size),
            nn.ReLU(True),
            nn.Linear(hidden_size, hidden_size),
            nn.ReLU(True),
            nn.Linear(hidden_size, output_size),
            nn.Tanh()
        )

    def forward(self, x):
        return self.main(x)

class Discriminator(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(Discriminator, self).__init__()
        self.main = nn.Sequential(
            nn.Linear(input_size, hidden_size),
            nn.LeakyReLU(0.2, inplace=True),
            nn.Linear(hidden_size, hidden_size),
            nn.LeakyReLU(0.2, inplace=True),
            nn.Linear(hidden_size, output_size),
            nn.Sigmoid()
        )

    def forward(self, x):
        return self.main(x)

Step 3: Initialize the Networks

input_size = 100
hidden_size = 256
output_size = 784  # For MNIST dataset (28x28 images)

G = Generator(input_size, hidden_size, output_size)
D = Discriminator(output_size, hidden_size, 1)

Step 4: Define Loss Function and Optimizers

criterion = nn.BCELoss()
lr = 0.0002

optimizerD = optim.Adam(D.parameters(), lr=lr)
optimizerG = optim.Adam(G.parameters(), lr=lr)

Step 5: Training Loop

num_epochs = 100
batch_size = 100
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.5,), (0.5,))
])

trainset = torchvision.datasets.MNIST(root='./data', train=True, download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=batch_size, shuffle=True)

for epoch in range(num_epochs):
    for i, (data, _) in enumerate(trainloader):
        # Train Discriminator
        D.zero_grad()
        real_data = data.view(-1, 28*28)
        batch_size = real_data.size(0)
        labels = torch.ones(batch_size, 1)
        output = D(real_data)
        lossD_real = criterion(output, labels)
        lossD_real.backward()

        noise = torch.randn(batch_size, input_size)
        fake_data = G(noise)
        labels = torch.zeros(batch_size, 1)
        output = D(fake_data.detach())
        lossD_fake = criterion(output, labels)
        lossD_fake.backward()
        optimizerD.step()

        # Train Generator
        G.zero_grad()
        labels = torch.ones(batch_size, 1)
        output = D(fake_data)
        lossG = criterion(output, labels)
        lossG.backward()
        optimizerG.step()

    print(f'Epoch [{epoch+1}/{num_epochs}], Loss D: {lossD_real+lossD_fake:.4f}, Loss G: {lossG:.4f}')

Step 6: Visualize Generated Data

def show_generated_images(epoch):
    noise = torch.randn(64, input_size)
    fake_images = G(noise).view(-1, 1, 28, 28)
    grid = torchvision.utils.make_grid(fake_images, nrow=8, normalize=True)
    plt.imshow(np.transpose(grid.detach().numpy(), (1, 2, 0)))
    plt.title(f'Epoch {epoch}')
    plt.show()

show_generated_images(num_epochs)

Practical Exercises

Exercise 1: Modify the Generator and Discriminator

  • Task: Add more layers to the generator and discriminator. Observe how the performance changes.
  • Solution: Add additional nn.Linear and activation layers in the Generator and Discriminator classes.

Exercise 2: Experiment with Different Loss Functions

  • Task: Try using different loss functions such as Mean Squared Error (MSE) instead of Binary Cross-Entropy (BCE).
  • Solution: Replace nn.BCELoss() with nn.MSELoss() and adjust the labels accordingly.

Exercise 3: Implement Conditional GANs (cGANs)

  • Task: Implement a conditional GAN where the generator and discriminator are conditioned on class labels.
  • Solution: Concatenate the class labels with the input noise for the generator and with the real/fake data for the discriminator.

Common Mistakes and Tips

  • Vanishing Gradients: Ensure that the learning rates are not too high, which can cause the gradients to vanish.
  • Mode Collapse: The generator may produce limited varieties of outputs. Use techniques like mini-batch discrimination to mitigate this.
  • Training Stability: GANs can be unstable to train. Regularly monitor the loss values and generated outputs to ensure the training is progressing as expected.

Conclusion

In this section, we introduced GANs, a powerful framework for generating synthetic data. We covered the basic concepts, implemented a simple GAN, and provided exercises to deepen your understanding. In the next module, we will explore reinforcement learning with PyTorch, building on the knowledge gained here.

◄ Previous
◄
Next ►
►
The Project | About Us | Contribute | Donations | License
© Copyright 2024. All rights reserved