Every Model in Machine Learning (Supervised, Unsupervised, Regression) explained

Deep learning is a subset of machine learning that uses artificial neural networks to process and analyze information. Neural networks are composed of computational nodes that are layered within deep learning algorithms. Each layer contains an input layer, an output layer, and a hidden layer. The neural network is fed training data which helps the algorithm learn and improve accuracy. When a neural network is composed of three or more layers it is said to be “deep,” thus deep learning.

Deep learning algorithms are inspired by the workings of the human brain and are used for analysis of data with a logical structure. Deep learning is used in many of the tasks we think of as AI today, including image and speech recognition, object detection, and natural language processing. Deep learning can make non-linear, complex correlations within datasets though requires more training data and computational resources than machine learning.

Mathematics behind each neuron is represented as below:

Here’s a clean, step-by-step breakdown of the mathematical operations inside a single neuron, stripped of all non-mathematical content:

1. Weighted Sum (Net Input)
$$
z = \sum_{i=1}^n w_i x_i + b
$$
– $x_i$: Input features
– $w_i$: Weights
– $b$: Bias term

2. Activation Function
$$
a = \phi(z)
$$
Common activations:
– Sigmoid:
$$\phi(z) = \frac{1}{1 + e^{-z}}$$
– ReLU:
$$\phi(z) = \max(0, z)$$
– Tanh:
$$\phi(z) = \tanh(z)$$

3. Error Calculation (Output Layer)
For a target $t$ and predicted output $y = a^{(L)}$ (where $L$ is the output layer):
$$
\delta^{(L)} = (y – t) \odot \phi'(z^{(L)})
$$
– $\odot$: Element-wise multiplication
– $\phi’$: Derivative of the activation function

4. Backpropagation (Hidden Layers)
For layer $l$:
$$
\delta^{(l)} = \left( (W^{(l+1)})^T \delta^{(l+1)} \right) \odot \phi'(z^{(l)})
$$
– $W^{(l+1)}$: Weight matrix of the next layer

5. Weight Update (Gradient Descent)
$$
w_{ij} \leftarrow w_{ij} – \eta \, \delta_j^{(l)} \, a_i^{(l-1)}
$$
$$
b_j \leftarrow b_j – \eta \, \delta_j^{(l)}
$$
– $\eta$: Learning rate

6. Summary of Steps
1. Forward Pass:
$$z = Wx + b \quad \rightarrow \quad a = \phi(z)$$
2. Backward Pass:
$$\delta = \text{Error} \odot \phi'(z)$$
3. Update:
$$W \leftarrow W – \eta \, \delta \, a^\top$$
$$b \leftarrow b – \eta \, \delta$$

This is the core mathematical flow of a neuron during training.

Here’s a step-by-step derivation of all key equations in a neural network neuron, from forward propagation to backpropagation, with mathematical rigor:

1. Forward Propagation (Expanded)
Weighted Sum:
$$
z_j = \sum_{i=1}^n w_{ji} x_i + b_j
$$
– Derivation:
Each input \(x_i\) is multiplied by its corresponding weight \(w_{ji}\), and the bias \(b_j\) is added. This is a linear transformation.

Activation Function:
$$
a_j = \phi(z_j)
$$
– Derivative Examples:
– Sigmoid:
$$\phi'(z_j) = \phi(z_j)(1 – \phi(z_j))$$
– ReLU:
$$\phi'(z_j) = \begin{cases}
1 & \text{if } z_j > 0 \\
0 & \text{otherwise}
\end{cases}$$
– Tanh:
$$\phi'(z_j) = 1 – \tanh^2(z_j)$$

2. Backpropagation (Derivation)
Output Layer Error:
For loss function \( \mathcal{L} \) (e.g., Mean Squared Error):
$$
\delta_j^{(L)} = \frac{\partial \mathcal{L}}{\partial z_j^{(L)}}
$$
– Derivation for MSE (\(\mathcal{L} = \frac{1}{2}(y_j – t_j)^2\)):
$$
\delta_j^{(L)} = \underbrace{(y_j – t_j)}_{\text{Loss gradient}} \cdot \underbrace{\phi'(z_j^{(L)})}_{\text{Activation gradient}}
$$

Hidden Layer Error:
$$
\delta_j^{(l)} = \sum_k \left( \delta_k^{(l+1)} w_{kj}^{(l+1)} \right) \phi'(z_j^{(l)})
$$
– Derivation:
Uses the chain rule to propagate error backward:
$$
\frac{\partial \mathcal{L}}{\partial z_j^{(l)}} = \sum_k \frac{\partial \mathcal{L}}{\partial z_k^{(l+1)}} \cdot \frac{\partial z_k^{(l+1)}}{\partial a_j^{(l)}} \cdot \frac{\partial a_j^{(l)}}{\partial z_j^{(l)}}
$$
– \(\frac{\partial z_k^{(l+1)}}{\partial a_j^{(l)}} = w_{kj}^{(l+1)}\) (weight linking neuron \(j\) to \(k\))
– \(\frac{\partial a_j^{(l)}}{\partial z_j^{(l)}} = \phi'(z_j^{(l)})\)

3. Weight/Bias Update (Derivation)
Gradient Descent Rule:
$$
\frac{\partial \mathcal{L}}{\partial w_{ji}^{(l)}} = \delta_j^{(l)} a_i^{(l-1)}
$$
$$
\frac{\partial \mathcal{L}}{\partial b_j^{(l)}} = \delta_j^{(l)}
$$
– Derivation:
– From \(z_j^{(l)} = \sum_i w_{ji}^{(l)} a_i^{(l-1)} + b_j^{(l)}\):
– \(\frac{\partial z_j^{(l)}}{\partial w_{ji}^{(l)}} = a_i^{(l-1)}\)
– \(\frac{\partial z_j^{(l)}}{\partial b_j^{(l)}} = 1\)
– Apply chain rule:
$$
\frac{\partial \mathcal{L}}{\partial w_{ji}^{(l)}} = \underbrace{\frac{\partial \mathcal{L}}{\partial z_j^{(l)}}}_{\delta_j^{(l)}} \cdot \frac{\partial z_j^{(l)}}{\partial w_{ji}^{(l)}}
$$

Update Equations:
$$
w_{ji}^{(l)} \leftarrow w_{ji}^{(l)} – \eta \frac{\partial \mathcal{L}}{\partial w_{ji}^{(l)}}
$$
$$
b_j^{(l)} \leftarrow b_j^{(l)} – \eta \frac{\partial \mathcal{L}}{\partial b_j^{(l)}}
$$

4. Full Algorithm Summary
1. Forward Pass:
– Compute \(z_j^{(l)} = W^{(l)} a^{(l-1)} + b^{(l)}\)
– Apply activation: \(a_j^{(l)} = \phi(z_j^{(l)})\)

2. Backward Pass:
– Output layer:
$$\delta^{(L)} = \nabla_a \mathcal{L} \odot \phi'(z^{(L)})$$
– Hidden layers (\(l = L-1, \dots, 1\)):
$$\delta^{(l)} = (W^{(l+1)})^\top \delta^{(l+1)} \odot \phi'(z^{(l)})$$

3. Update Parameters:
– \(\Delta W^{(l)} = -\eta \, \delta^{(l)} (a^{(l-1)})^\top\)
– \(\Delta b^{(l)} = -\eta \, \delta^{(l)}\)

Key Insights:
– Chain Rule: Backpropagation is just repeated application of the chain rule.
– Locality: Each neuron only needs its own \(z_j\), \(a_j\), and \(\delta_j\) to compute updates.
– Modularity: Change \(\phi\) or \(\mathcal{L}\) by plugging in different derivatives.

Here’s a complete numerical example of a neural network forward/backward pass with concrete numbers, using a single data point:

Network Architecture
– Inputs: \( x = [2.0, 3.0] \)
– Weights: \( W = \begin{bmatrix} 0.5 & -1.2 \\ 0.8 & 0.7 \end{bmatrix} \) (Layer 1)
– Bias: \( b = [0.1, -0.3] \)
– Activation: Sigmoid (\( \phi(z) = \frac{1}{1+e^{-z}} \))
– Target: \( t = 1.0 \) (binary classification)

Step 1: Forward Pass
1. Weighted Sum (\( z = Wx + b \)):
\[
z_1 = (0.5 \times 2.0) + (-1.2 \times 3.0) + 0.1 = -2.5
\]
\[
z_2 = (0.8 \times 2.0) + (0.7 \times 3.0) + (-0.3) = 3.0
\]
– Output: \( z = [-2.5, 3.0] \)

2. Activation (\( a = \phi(z) \)):
\[
a_1 = \frac{1}{1+e^{2.5}} \approx 0.076
\]
\[
a_2 = \frac{1}{1+e^{-3.0}} \approx 0.953
\]
– Activated Output: \( a = [0.076, 0.953] \)

Step 2: Output Layer Error (Binary Cross-Entropy Loss)
– Predicted Probability: \( y = a_2 = 0.953 \)
– Error Gradient (\( \delta^{(L)} = y – t \)):
\[
\delta^{(L)} = 0.953 – 1.0 = -0.047
\]

Step 3: Backpropagation (Hidden Layer)
1. Weight Gradient (\( \nabla W = \delta \cdot a^\top \)):
\[
\nabla W_{21} = \delta^{(L)} \times a_1 = -0.047 \times 0.076 \approx -0.0036
\]
\[
\nabla W_{22} = \delta^{(L)} \times a_2 = -0.047 \times 0.953 \approx -0.0448
\]

2. Bias Gradient (\( \nabla b = \delta \)):
\[
\nabla b_2 = -0.047
\]

Step 4: Update Weights (Learning Rate \( \eta = 0.1 \))
– New Weights:
\[
W_{21} \leftarrow 0.8 – (0.1 \times -0.0036) \approx 0.8004
\]
\[
W_{22} \leftarrow 0.7 – (0.1 \times -0.0448) \approx 0.7045
\]
– New Bias:
\[
b_2 \leftarrow -0.3 – (0.1 \times -0.047) \approx -0.2953
\]

Key Observations
1. Forward Pass:
– Input \( [2.0, 3.0] \) → Activated output \( \approx 0.953 \).
2. Backward Pass:
– Small error (\( -0.047 \)) slightly adjusts weights toward the target \( 1.0 \).
3. Sigmoid Behavior:
– \( z_1 = -2.5 \) → Near \( 0 \) (inactive neuron).
– \( z_2 = 3.0 \) → Near \( 1 \) (active neuron).

Why This Matters
– Transparency: Every number is traceable to show how gradients flow.
– Debugging: Verify your implementation matches these exact values.

Some common types of neural networks used for deep learning include:

Feedforward neural networks (FF) are one of the oldest forms of neural networks, with data flowing one way through layers of artificial neurons until the output is achieved.

Training a Feedforward Neural Network

Training a Feedforward Neural Network involves adjusting the weights of the neurons to minimize the error between the predicted output and the actual output. This process is typically performed using backpropagation and gradient descent.

1. 1. Forward Propagation: During forward propagation the input data passes through the network and the output is calculated.
   2. Loss Calculation: The loss (or error) is calculated using a loss function such as Mean Squared Error (MSE) for regression tasks or Cross-Entropy Loss for classification tasks.
   3. Backpropagation: In backpropagation the error is propagated back through the network to update the weights. The gradient of the loss function with respect to each weight is calculated and the weights are adjusted using gradient descent.

Evaluation of Feedforward neural network

Evaluating the performance of the trained model involves several metrics:

• Accuracy: The proportion of correctly classified instances out of the total instances.
• Precision: The ratio of true positive predictions to the total predicted positives.
• Recall: The ratio of true positive predictions to the actual positives.
• F1 Score: The harmonic mean of precision and recall, providing a balance between the two.
• Confusion Matrix: A table used to describe the performance of a classification model, showing the true positives, true negatives, false positives, and false negatives.

Code Implementation of Feedforward neural network

Model is compiled with the Adam optimizer, SparseCategoricalCrossentropy loss function and SparseCategoricalAccuracy metric and then trained for 5 epochs on the training data.

This code demonstrates the process of building, training and evaluating a neural network model using TensorFlow and Keras to classify handwritten digits from the MNIST dataset.

The model architecture is defined using the Sequential API consisting of:

• a Flatten layer to convert the 2D image input into a 1D array
• a Dense layer with 128 neurons and ReLU activation
• a final Dense layer with 10 neurons and softmax activation to output probabilities for each digit class.

An epoch is when all the training data is used at once and is defined as the total number of iterations of all the training data in one cycle for training the machine learning model.

Recurrent neural networks (RNN) differ from feedforward neural networks in that they typically use time series data or data that involves sequences. Recurrent neural networks have “memory” of what happened in the previous layer as contingent to the output of the current layer.

Imagine reading a sentence and you try to predict the next word, you don’t rely only on the current word but also remember the words that came before. RNNs work similarly by “remembering” past information and passing the output from one step as input to the next i.e it considers all the earlier words to choose the most likely next word. This memory of previous steps helps the network understand context and make better predictions.

Key Components of RNNs

There are mainly two components of RNNs that we will discuss.

1. Recurrent Neurons

The fundamental processing unit in RNN is a Recurrent Unit. They hold a hidden state that maintains information about previous inputs in a sequence. Recurrent units can “remember” information from prior steps by feeding back their hidden state, allowing them to capture dependencies across time.

2. RNN Unfolding

RNN unfolding or unrolling is the process of expanding the recurrent structure over time steps. During unfolding each step of the sequence is represented as a separate layer in a series illustrating how information flows across each time step.

This unrolling enables backpropagation through time (BPTT) a learning process where errors are propagated across time steps to adjust the network’s weights enhancing the RNN’s ability to learn dependencies within sequential data.

Types Of Recurrent Neural Networks

There are four types of RNNs based on the number of inputs and outputs in the network:

1. One-to-One RNN

This is the simplest type of neural network architecture where there is a single input and a single output. It is used for straightforward classification tasks such as binary classification where no sequential data is involved.

2. One-to-Many RNN

In a One-to-Many RNN the network processes a single input to produce multiple outputs over time. This is useful in tasks where one input triggers a sequence of predictions (outputs). For example in image captioning a single image can be used as input to generate a sequence of words as a caption.

3. Many-to-One RNN

The Many-to-One RNN receives a sequence of inputs and generates a single output. This type is useful when the overall context of the input sequence is needed to make one prediction. In sentiment analysis the model receives a sequence of words (like a sentence) and produces a single output like positive, negative or neutral.

4. Many-to-Many RNN

The Many-to-Many RNN type processes a sequence of inputs and generates a sequence of outputs. In language translation task a sequence of words in one language is given as input and a corresponding sequence in another language is generated as output.

Variants of Recurrent Neural Networks (RNNs)

There are several variations of RNNs, each designed to address specific challenges or optimize for certain tasks:

1. Vanilla RNN

This simplest form of RNN consists of a single hidden layer where weights are shared across time steps. Vanilla RNNs are suitable for learning short-term dependencies but are limited by the vanishing gradient problem, which hampers long-sequence learning.

2. Bidirectional RNNs

Bidirectional RNNs process inputs in both forward and backward directions, capturing both past and future context for each time step. This architecture is ideal for tasks where the entire sequence is available, such as named entity recognition and question answering.

3. Long Short-Term Memory Networks (LSTMs)

Long Short-Term Memory Networks (LSTMs) introduce a memory mechanism to overcome the vanishing gradient problem. Each LSTM cell has three gates:

• Input Gate: Controls how much new information should be added to the cell state.
• Forget Gate: Decides what past information should be discarded.
• Output Gate: Regulates what information should be output at the current step. This selective memory enables LSTMs to handle long-term dependencies, making them ideal for tasks where earlier context is critical.

4. Gated Recurrent Units (GRUs)

Gated Recurrent Units (GRUs) simplify LSTMs by combining the input and forget gates into a single update gate and streamlining the output mechanism. This design is computationally efficient, often performing similarly to LSTMs and is useful in tasks where simplicity and faster training are beneficial.

Long/short term memory (LSTM) is an advanced form of RNN that can use memory to “remember” what happened in previous layers.

LSTMs can capture long-term dependencies in sequential data making them ideal for tasks like language translation, speech recognition and time series forecasting.

Unlike traditional RNNs which use a single hidden state passed through time LSTMs introduce a memory cell that holds information over extended periods addressing the challenge of learning long-term dependencies.

Applications of LSTM

Some of the famous applications of LSTM includes:

• Language Modeling: Used in tasks like language modeling, machine translation and text summarization. These networks learn the dependencies between words in a sentence to generate coherent and grammatically correct sentences.
• Speech Recognition: Used in transcribing speech to text and recognizing spoken commands. By learning speech patterns they can match spoken words to corresponding text.
• Time Series Forecasting: Used for predicting stock prices, weather and energy consumption. They learn patterns in time series data to predict future events.
• Anomaly Detection: Used for detecting fraud or network intrusions. These networks can identify patterns in data that deviate drastically and flag them as potential anomalies.
• Recommender Systems: In recommendation tasks like suggesting movies, music and books. They learn user behavior patterns to provide personalized suggestions.
• Video Analysis: Applied in tasks such as object detection, activity recognition and action classification. When combined with Convolutional Neural Networks (CNNs) they help analyze video data and extract useful information.

Convolutional neural networks (CNN) include some of the most common neural networks in modern artificial intelligence and use several distinct layers (a convolutional layer, then a pooling layer) that filter different parts of an image before putting it back together (in the fully connected layer).

 This is particularly useful for visual datasets such as images or videos, where data patterns play a crucial role. CNNs are widely used in computer vision applications due to their effectiveness in processing visual data.

CNNs consist of multiple layers like the input layer, Convolutional layer, pooling layer, and fully connected layers.

How Convolutional Layers Works?

Convolution Neural Networks are neural networks that share their parameters.

Imagine you have an image. It can be represented as a cuboid having its length, width (dimension of the image), and height (i.e the channel as images generally have red, green, and blue channels).

Now imagine taking a small patch of this image and running a small neural network, called a filter or kernel on it, with say, K outputs and representing them vertically.

Now slide that neural network across the whole image, as a result, we will get another image with different widths, heights, and depths. Instead of just R, G, and B channels now we have more channels but lesser width and height. This operation is called Convolution. If the patch size is the same as that of the image it will be a regular neural network. Because of this small patch, we have fewer weights.

Mathematical Overview of Convolution

Now let’s talk about a bit of mathematics that is involved in the whole convolution process.

• Convolution layers consist of a set of learnable filters (or kernels) having small widths and heights and the same depth as that of input volume (3 if the input layer is image input).
• For example, if we have to run convolution on an image with dimensions 34x34x3. The possible size of filters can be axax3, where ‘a’ can be anything like 3, 5, or 7 but smaller as compared to the image dimension.
• During the forward pass, we slide each filter across the whole input volume step by step where each step is called stride (which can have a value of 2, 3, or even 4 for high-dimensional images) and compute the dot product between the kernel weights and patch from input volume.
• As we slide our filters we’ll get a 2-D output for each filter and we’ll stack them together as a result, we’ll get output volume having a depth equal to the number of filters. The network will learn all the filters.

Layers Used to Build ConvNets

A complete Convolution Neural Networks architecture is also known as covnets. A covnets is a sequence of layers, and every layer transforms one volume to another through a differentiable function.

Let’s take an example by running a covnets on of image of dimension 32 x 32 x 3.

• Input Layers: It’s the layer in which we give input to our model. In CNN, Generally, the input will be an image or a sequence of images. This layer holds the raw input of the image with width 32, height 32, and depth 3.
• Convolutional Layers: This is the layer, which is used to extract the feature from the input dataset. It applies a set of learnable filters known as the kernels to the input images. The filters/kernels are smaller matrices usually 2×2, 3×3, or 5×5 shape. it slides over the input image data and computes the dot product between kernel weight and the corresponding input image patch. The output of this layer is referred as feature maps. Suppose we use a total of 12 filters for this layer we’ll get an output volume of dimension 32 x 32 x 12.
• Activation Layer: By adding an activation function to the output of the preceding layer, activation layers add nonlinearity to the network. it will apply an element-wise activation function to the output of the convolution layer. Some common activation functions are RELU: max(0, x),  Tanh, Leaky RELU, etc. The volume remains unchanged hence output volume will have dimensions 32 x 32 x 12.
• Pooling layer: This layer is periodically inserted in the covnets and its main function is to reduce the size of volume which makes the computation fast reduces memory and also prevents overfitting. Two common types of pooling layers are max pooling and average pooling. If we use a max pool with 2 x 2 filters and stride 2, the resultant volume will be of dimension 16x16x12.
• Flattening: The resulting feature maps are flattened into a one-dimensional vector after the convolution and pooling layers so they can be passed into a completely linked layer for categorization or regression.
• Fully Connected Layers: It takes the input from the previous layer and computes the final classification or regression task.
• Output Layer: The output from the fully connected layers is then fed into a logistic function for classification tasks like sigmoid or softmax which converts the output of each class into the probability score of each class.

Example: Applying CNN to an Image

Let’s consider an image and apply the convolution layer, activation layer, and pooling layer operation to extract the inside feature.

Input image:

Step:

• import the necessary libraries
• set the parameter
• define the kernel
• Load the image and plot it.
• Reformat the image
• Apply convolution layer operation and plot the output image.
• Apply activation layer operation and plot the output image.
• Apply pooling layer operation and plot the output image.

NOTE: AM USING COLAB

OUTPUT:

Advantages of CNNs

1. Good at detecting patterns and features in images, videos, and audio signals.
2. Robust to translation, rotation, and scaling invariance.
3. End-to-end training, no need for manual feature extraction.
4. Can handle large amounts of data and achieve high accuracy.

Disadvantages of CNNs

1. Computationally expensive to train and require a lot of memory.
2. Can be prone to overfitting if not enough data or proper regularization is used.
3. Requires large amounts of labeled data.
4. Interpretability is limited, it’s hard to understand what the network has learned.

Generative adversarial networks (GAN) involve two neural networks (a “generator” and a “discriminator”) that compete against each other in a game that ultimately improves the accuracy of the output. It is introduced by Ian Goodfellow and his team in 2014 and they have transformed how computers generate images, videos, music and more. Unlike traditional models that only recognize or classify data, they take a creative way by generating entirely new content that closely resembles real-world data. This ability helped various fields such as art, gaming, healthcare and data science. In this article, we will see more about GANs and its core concepts.

Architecture of GANs

GANs consist of two main models that work together to create realistic synthetic data which are as follows:

1. Generator Model

The generator is a deep neural network that takes random noise as input to generate realistic data samples like images or text. It learns the underlying data patterns by adjusting its internal parameters during training through backpropagation. Its objective is to produce samples that the discriminator classifies as real.

How does a GAN work?

GANs train by having two networks the Generator (G) and the Discriminator (D) compete and improve together. Here’s the step-by-step process

1. Generator’s First Move

The generator starts with a random noise vector like random numbers. It uses this noise as a starting point to create a fake data sample such as a generated image. The generator’s internal layers transform this noise into something that looks like real data.

2. Discriminator’s Turn

The discriminator receives two types of data:

• Real samples from the actual training dataset.
• Fake samples created by the generator.

D’s job is to analyze each input and find whether it’s real data or something G cooked up. It outputs a probability score between 0 and 1. A score of 1 shows the data is likely real and 0 suggests it’s fake.

3. Adversarial Learning

• If the discriminator correctly classifies real and fake data it gets better at its job.
• If the generator fools the discriminator by creating realistic fake data, it receives a positive update and the discriminator is penalized for making a wrong decision.

4. Generator’s Improvement

• Each time the discriminator mistakes fake data for real, the generator learns from this success.
• Through many iterations, the generator improves and creates more convincing fake samples.

5. Discriminator’s Adaptation

• The discriminator also learns continuously by updating itself to better spot fake data.
• This constant back-and-forth makes both networks stronger over time.

6. Training Progression

• As training continues, the generator becomes highly proficient at producing realistic data.
• Eventually the discriminator struggles to distinguish real from fake shows that the GAN has reached a well-trained state.
• At this point, the generator can produce high-quality synthetic data that can be used for different applications.

Types of GANs

There are several types of GANs each designed for different purposes. Here are some important types:

1. Vanilla GAN

Vanilla GAN is the simplest type of GAN. It consists of:

• A generator and a discriminator both are built using multi-layer perceptrons (MLPs).
• The model optimizes its mathematical formulation using stochastic gradient descent (SGD).

While foundational, Vanilla GANs can face problems like:

• Mode collapse: The generator produces limited types of outputs repeatedly.
• Unstable training: The generator and discriminator may not improve smoothly.

2. Conditional GAN (CGAN)

Conditional GANs (CGANs) adds an additional conditional parameter to guide the generation process. Instead of generating data randomly they allow the model to produce specific types of outputs.

Working of CGANs:

• A conditional variable (y) is fed into both the generator and the discriminator.
• This ensures that the generator creates data corresponding to the given condition (e.g generating images of specific objects).
• The discriminator also receives the labels to help distinguish between real and fake data.

Example: Instead of generating any random image, CGAN can generate a specific object like a dog or a cat based on the label.

3. Deep Convolutional GAN (DCGAN)

Deep Convolutional GANs (DCGANs) are among the most popular types of GANs used for image generation.

They are important because they:

• Uses Convolutional Neural Networks (CNNs) instead of simple multi-layer perceptrons (MLPs).
• Max pooling layers are replaced with convolutional stride helps in making the model more efficient.
• Fully connected layers are removed, which allows for better spatial understanding of images.

DCGANs are successful because they generate high-quality, realistic images.

4. Laplacian Pyramid GAN (LAPGAN)

Laplacian Pyramid GAN (LAPGAN) is designed to generate ultra-high-quality images by using a multi-resolution approach.

Working of LAPGAN:

• Uses multiple generator-discriminator pairs at different levels of the Laplacian pyramid.
• Images are first down sampled at each layer of the pyramid and upscaled again using Conditional GANs (CGANs).
• This process allows the image to gradually refine details and helps in reducing noise and improving clarity.

Due to its ability to generate highly detailed images, LAPGAN is considered a superior approach for photorealistic image generation.

5. Super Resolution GAN (SRGAN)

Super-Resolution GAN (SRGAN) is designed to increase the resolution of low-quality images while preserving details.

Working of SRGAN:

• Uses a deep neural network combined with an adversarial loss function.
• Enhances low-resolution images by adding finer details helps in making them appear sharper and more realistic.
• Helps to reduce common image upscaling errors such as blurriness and pixelation.

Implementation of Generative Adversarial Network (GAN) using PyTorch

Generative Adversarial Networks (GANs) can generate realistic images by learning from existing image datasets. Here we will be implementing a GAN trained on the CIFAR-10 dataset using PyTorch.

Step 1: Importing Required Libraries

We will be using Pytorch, Torchvision, Matplotlib and Numpy libraries for this. Set the device to GPU if available otherwise use CPU.

Dataset: https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz   or https://www.cs.toronto.edu/~kriz/cifar.html

Application Of Generative Adversarial Networks (GANs)

1. Image Synthesis & Generation: GANs generate realistic images, avatars and high-resolution visuals by learning patterns from training data. They are used in art, gaming and AI-driven design.
2. Image-to-Image Translation: They can transform images between domains while preserving key features. Examples include converting day images to night, sketches to realistic images or changing artistic styles.
3. Text-to-Image Synthesis: They create visuals from textual descriptions helps applications in AI-generated art, automated design and content creation.
4. Data Augmentation: They generate synthetic data to improve machine learning models helps in making them more robust and generalizable in fields with limited labeled data.
5. High-Resolution Image Enhancement: They upscale low-resolution images which helps in improving clarity for applications like medical imaging, satellite imagery and video enhancement.

Advantages of GAN

Lets see various advantages of the GANs:

1. Synthetic Data Generation: GANs produce new, synthetic data resembling real data distributions which is useful for augmentation, anomaly detection and creative tasks.
2. High-Quality Results: They can generate photorealistic images, videos, music and other media with high quality.
3. Unsupervised Learning: They don’t require labeled data helps in making them effective in scenarios where labeling is expensive or difficult.
4. Versatility: They can be applied across many tasks including image synthesis, text-to-image generation, style transfer, anomaly detection and more.