3 Generative Adversarial Networks (GANs): Shape and Number Generation
This chapter introduces Generative Adversarial Networks (GANs) and shows how to build and understand them end to end with PyTorch. It explains the adversarial setup—where a generator learns to produce realistic samples while a discriminator learns to distinguish real from fake—and highlights the role of the latent space for diversity and control. Emphasis is placed on practical skills: constructing generator and discriminator networks from scratch, choosing and optimizing loss functions, organizing training loops and batches, applying early stopping, and evaluating model quality. The material is framed as a foundation for later applications to richer modalities, while also noting how synthetic data and injected noise can be useful for downstream modeling by mitigating overfitting.
The first project uses GANs to generate two-dimensional points that conform to an exponential growth curve, y = 1.08^x. The workflow covers preparing a dataset of (x, y) pairs over a chosen x-range, batching with a data loader, and designing networks: an MLP discriminator that outputs a real/fake probability via a sigmoid, and an MLP generator that maps a latent vector to (x, y). Both models are trained with binary cross-entropy in an alternating fashion over real and generated batches. Performance is assessed visually and quantitatively via mean squared error against the known function; training is stopped when improvements plateau. After training, the discriminator is discarded, the generator is saved and reloaded, and new curve-conforming samples are produced directly from latent noise.
The second project extends the same GAN recipe to discrete pattern generation: producing sequences of ten integers in the range 0–99 where every number is a multiple of five. Integers are translated to and from machine-ready one-hot vectors, a simple discriminator classifies one-hot inputs as real or fake, and a generator maps latent noise to one-hot-like outputs. A task-specific metric measures how close generated integers are to the target pattern (e.g., the MSE of remainders modulo five against zero), enabling early stopping. As before, the trained generator is saved and reused to sample new sequences. The approach is easily adapted to other numerical patterns (such as multiples of different bases or odd/even constraints), reinforcing core GAN skills that transfer to more advanced content generation tasks later in the book.
A diagram of the steps involved in training GANs to generate an exponential growth curve and the dual-network architecture in GANs. The generator obtains a random noise vector Z from the latent space (top left) to create a fake sample and presents it to the discriminator (middle). The discriminator classifies a sample as real or fake (created by the generator). The predictions are compared to the ground truth and both the discriminator and the generator learn from the predictions. After many iterations of training, the generator learns to create shapes that are indistinguishable from real samples.
Subplots of the comparison of the generated shape with the true exponential growth curve shape at different stages of the training process. At epoch 25, the generated samples don’t form any meaningful shape. At epoch 200, the samples start to look like an exponential growth curve shape. At epoch 1025, the generated samples align closely with the exponential growth curve.
Summary
- GANs consist of two networks: a discriminator to distinguish fake samples from real samples, and a generator to create samples that are indistinguishable from those in the training set.
- The steps involved in GANs are preparing training data, creating a discriminator and a generator, training the model and deciding when to stop training, and finally, discarding the discriminator and using the trained generator to create new samples.
- The content generated by GANs depends on the training data. When the training dataset contains data pairs (x, y) that form an exponential growth curve, the generated samples are also data pairs that mimic such a shape. When the training dataset has sequences of numbers that are all multiples of five, the generated samples are also sequences of numbers, with multiples of five in them.
- GANs are versatile and capable of generating many different formats of content.
FAQ
What is a Generative Adversarial Network (GAN) and how do its two networks interact?
GANs consist of two neural networks trained in a zero-sum game: the generator (G) creates samples intended to look real, and the discriminator (D) predicts whether a sample is real (from the training set) or fake (from G). During training, D learns to better distinguish real from fake, while G learns to fool D by producing increasingly realistic samples.How do I prepare training data to learn the exponential relation y = 1.08^x?
- Sample x uniformly, e.g., 2048 values in [0, 50].- Compute y = 1.08^x for each x.
- Store pairs (x, y) in a tensor with shape (N, 2).
- Wrap the dataset with a DataLoader using a batch size (e.g., 128) and shuffle=True to stabilize training.
- Visualize x vs. y to verify the exponential shape.
What is the latent space Z and why do we use random noise vectors?
The latent space is a conceptual space from which random vectors (noise) are sampled as inputs to the generator. Each point in Z can be transformed by G into a data instance. Sampling noise allows G to learn a mapping that covers the variety of possible outputs; interpolations in Z can smoothly vary attributes of generated samples.What model architectures are used for the generator and discriminator in the shape project?
- Discriminator (D): a dense network taking 2 inputs (x, y) and outputting a single probability via a Sigmoid. Hidden layers use ReLU and Dropout to reduce overfitting (e.g., widths 256 → 128 → 64).- Generator (G): a dense network mapping a low-dimensional noise vector (e.g., 2D) to a 2D output (x, y), with ReLU activations in hidden layers. The latent dimension can be changed (e.g., 2, 5, 10) without altering the overall approach.
Which loss functions and optimizers are used, and why?
- Loss: Binary cross-entropy (BCE) for both networks because D performs binary classification (real vs. fake), and G is trained to make D classify fake as real.- Optimizer: Adam with a small learning rate (e.g., 0.0005) for stable updates of both D and G.
How does the training loop work and what labels are used?
- For each batch: (1) Train D on real samples with label 1s. (2) Generate fake samples and train D on them with label 0s. (3) Train G by generating fake samples, sending them through D, and using label 1s so G learns to fool D.- Gradients are zeroed each step; weights are updated by backpropagation in their respective optimizers.
How do I evaluate GAN performance and decide when to stop training?
- Visual inspection: plot generated samples against real samples to see if the shape matches (e.g., the exponential curve).- Quantitative metric (for known relations): compute MSE between generated y values and the true function value at generated x (e.g., y_true = 1.08^x).
- Early stopping: stop when performance (e.g., MSE) fails to improve for a set patience (e.g., 1000 epochs).
How do I save, load, and use a trained generator?
- Save: script the trained generator with torch.jit.script and save to a .pt file.- Load: torch.jit.load with map_location to CPU or CUDA.
- Use: sample noise from the latent space, pass it through the loaded generator, and post-process or plot the outputs.
How does the numbers-with-patterns project generate multiples of five?
- Data representation: convert integers 0–99 to 100-dimensional one-hot vectors for model compatibility; convert back with argmax for readability.- Task: G maps 100-D noise to a 100-D output per integer; D classifies one-hot vectors as real or fake.
- Training signal: use BCE for adversarial training; measure progress by the MSE of remainders (generated_number % 5) against zeros; when near zero, outputs are multiples of five.
How can I adapt these projects to other shapes or patterns?
- Shapes: change the training relation (e.g., y = sin(x), y = x^2), adjust x ranges and plotting limits, and retrain.- Patterns: alter the constraint (e.g., multiples of 2 or 3, odds/evens) by modifying the distance/constraint function used for monitoring progress.
- Tunables: batch size, latent dimension, hidden layer sizes, dropout, learning rate, and patience. Use visualization or task-specific metrics to guide choices.
Learn Generative AI with PyTorch ebook for free
