PyTorch -I

content

Introduction — What is Deep Learning & Why PyTorch?
Setting Up the PyTorch Environment
PyTorch Basics — Tensors, Operations, and GPU Acceleration
Autograd and Computational Graphs

🧠 1. Introduction — What is Deep Learning & Why PyTorch?

🌍 The Rise of Intelligent Machines

Over the past decade, the world has witnessed a digital transformation powered by Artificial Intelligence (AI). From voice assistants like Alexa and Google Assistant, to self-driving cars, facial recognition, and even medical diagnosis systems — AI is now embedded in nearly every aspect of modern life.

But behind these intelligent systems lies a powerful technology:
👉 Deep Learning (DL) — a subset of Machine Learning that uses artificial neural networks to mimic the way the human brain learns from data.

🧩 What Exactly is Deep Learning?

At its core, Deep Learning is about teaching machines to recognize patterns and make decisions — without being explicitly programmed to perform those tasks.

Instead of handcrafting rules, we feed a model massive amounts of data, and it learns to extract relevant features automatically.

Let’s simplify this:

Traditional Machine Learning	Deep Learning
Needs manual feature extraction (e.g., color, shape, size).	Learns features automatically from raw data.
Works well for small datasets.	Excels with large datasets and high-dimensional data.
Example: Logistic Regression, Decision Trees.	Example: CNNs, RNNs, Transformers.

Example:
Suppose you’re building a system to identify cats and dogs in images.

A traditional ML approach might require you to define features like “fur length,” “ear shape,” or “eye color.”
A deep learning model (like a CNN) will learn these features on its own directly from pixels.

That’s the magic of deep learning — it automates feature discovery.

🧠 How Deep Learning Works — A Simple Overview

Deep Learning models are composed of layers of neurons — inspired by biological brains. Each neuron receives inputs, applies weights, adds bias, and passes the result through an activation function.

Mathematically:
[
y = f(Wx + b)
]

Where:

( x ): Input vector
( W ): Weights
( b ): Bias
( f ): Activation function (like ReLU or Sigmoid)
( y ): Output

These layers are stacked to form a neural network — and the more layers it has, the “deeper” it is.

During training:

The model makes a prediction.
The prediction is compared to the true label (via a loss function).
The difference (error) is propagated backward.
The model adjusts its weights to minimize the error.

This process — known as backpropagation — enables the model to learn from mistakes over multiple iterations (epochs).

⚙️ Why Deep Learning Has Become So Popular

Deep Learning’s success is due to a combination of:

Big Data availability (e.g., millions of images or documents)
Powerful GPUs that accelerate training
Advanced architectures (like CNNs, LSTMs, and Transformers)
Frameworks like PyTorch and TensorFlow that make model development simpler than ever.

🔥 Enter PyTorch — The Researcher’s Deep Learning Framework

PyTorch, developed by Facebook’s AI Research Lab (FAIR) in 2016, quickly became one of the most loved frameworks in the deep learning community.

It is open-source, easy to learn, and incredibly powerful for building both research prototypes and production-grade models.

🧩 Key Features that Make PyTorch Stand Out

Feature	Description
🧮 Dynamic Computation Graphs	Unlike TensorFlow 1.x, PyTorch builds graphs on the fly, making it intuitive and flexible.
⚡ Pythonic and Easy to Debug	Feels like native Python; easy to print tensors, set breakpoints, and debug.
💾 GPU Acceleration	Seamless CUDA integration for high-speed tensor computations.
🧠 Strong Ecosystem	Includes `torchvision`, `torchtext`, `torchaudio`, and integrates with Hugging Face and PyTorch Lightning.
🚀 Research to Production	TorchScript, ONNX, and TorchServe help deploy models efficiently.

🧰 PyTorch vs. TensorFlow: A Quick Comparison

Feature	PyTorch	TensorFlow
Graph Type	Dynamic	Static (eager mode added later)
Syntax	More Pythonic	More verbose
Debugging	Easier (like native Python)	Requires graph inspection
Popularity	Favored by researchers	Favored by production teams
Ecosystem	TorchVision, TorchText, Lightning	Keras, TFX, TF Lite

👉 In short, PyTorch = Flexibility + Simplicity,
while TensorFlow = Scalability + Deployment.

That’s why most research papers, including OpenAI’s GPT and Meta’s Segment Anything, are written in PyTorch.

🌐 Real-World Use Cases of PyTorch

PyTorch powers some of the world’s most advanced AI systems:

Industry	Application	Example
🧠 Research	Language & Vision models	OpenAI’s GPT, Meta’s Segment Anything
🏥 Healthcare	Medical image segmentation	Detecting tumors from MRI scans
🚗 Automotive	Self-driving systems	Tesla & Waymo’s perception networks
📱 Social Media	Content moderation	Facebook & Instagram’s AI filters
🛒 E-commerce	Recommendation systems	Personalized shopping predictions

🧩 Example:

Imagine you’re building a recommendation system for an e-commerce site.
PyTorch lets you:

Represent user interactions as tensors.
Build a neural network to predict user preferences.
Deploy it as a real-time service.

All with just a few lines of elegant, Pythonic code.

🧱 The PyTorch Ecosystem — More Than Just a Framework

PyTorch isn’t a single library — it’s a complete deep learning ecosystem.
Here’s what it includes:

Library	Purpose
torch	Core tensor operations and autograd
torchvision	Image datasets, transforms, pretrained CNNs
torchtext	NLP datasets and preprocessing tools
torchaudio	Audio and speech processing
PyTorch Lightning	Simplified training workflows
TorchServe	Model deployment on servers
TorchScript & ONNX	Model serialization and cross-platform execution

Together, they make PyTorch ideal for both beginners (easy syntax) and professionals (scalable architecture).

🧩 Simple Example — Your First PyTorch Program

Let’s start with a mini demo — creating a tensor, performing operations, and checking GPU acceleration.

import torch

# Create two tensors
a = torch.tensor([2.0, 3.0])
b = torch.tensor([4.0, 5.0])

# Perform simple operations
print("Addition:", a + b)
print("Multiplication:", a * b)

# Check GPU availability
if torch.cuda.is_available():
    a = a.to('cuda')
    print("Tensor moved to GPU!")

🧠 Output:

Addition: tensor([6., 8.])
Multiplication: tensor([8., 15.])
Tensor moved to GPU!

Just like that, you’ve taken your first step into the PyTorch world — working with tensors and leveraging GPU acceleration.

🚀 Why You Should Learn PyTorch in 2025

As of 2025, PyTorch has become the industry standard for AI development.

Universities teach it as the primary deep learning tool.
Research labs publish models in PyTorch format.
Major cloud platforms (AWS, GCP, Azure) have built-in PyTorch support.

Whether you’re aiming for academic research, AI startups, or production-level model deployment, PyTorch gives you the flexibility and power to build almost anything — from simple neural networks to billion-parameter models.

Sponsor Key-Word

"This Content Sponsored by SBO Digital Marketing.
Mobile-Based Part-Time Job Opportunity by SBO!
Earn money online by doing simple content publishing and sharing tasks. Here's how:
Job Type: Mobile-based part-time work
Work Involves:
Content publishing
Content sharing on social media
Time Required: As little as 1 hour a day
Earnings: ₹300 or more daily
Requirements:
Active Facebook and Instagram account
Basic knowledge of using mobile and social media
For more details:
WhatsApp your Name and Qualification to 9994104160
a.Online Part Time Jobs from Home
b.Work from Home Jobs Without Investment
c.Freelance Jobs Online for Students
d.Mobile Based Online Jobs
e.Daily Payment Online Jobs
Keyword & Tag: #OnlinePartTimeJob #WorkFromHome #EarnMoneyOnline #PartTimeJob #jobs #jobalerts #withoutinvestmentjob"

🧭 Transition to Next Section

Now that you understand what Deep Learning is and why PyTorch is the preferred framework, it’s time to dive deeper.

In the next section, we’ll set up your PyTorch environment, explore how to work with tensors, and learn how PyTorch handles computations under the hood.

🔜 Next: “Setting Up the PyTorch Environment — Your First Step Toward AI Development”

Section 2: Setting Up the PyTorch Environment

Before diving deep into neural networks and model training, it’s essential to properly set up your PyTorch environment. This ensures a smooth workflow, GPU utilization, and compatibility across systems.

This section will walk you through everything — from installation to verifying GPU support — and setting up your first PyTorch notebook.

🧩 2.1 What You Need Before Installing PyTorch

Before installing PyTorch, make sure your system is ready with the following requirements:

✅ Prerequisites

Python 3.8+
(Recommended: Python 3.10 or later)
pip (Python package installer)
Virtual environment tools (optional but recommended)
CUDA Toolkit (only if you have an NVIDIA GPU)
Operating System: Windows, macOS, or Linux

🧠 Why a Virtual Environment?

Virtual environments help isolate your project dependencies so that installing PyTorch (and its packages) doesn’t conflict with other Python projects.

You can create a virtual environment using either:

# Using venv
python -m venv pytorch_env

# Activate environment
# On Windows
pytorch_env\Scripts\activate

# On Mac/Linux
source pytorch_env/bin/activate

Once activated, your terminal will show something like:

(pytorch_env) user@machine:~$

This confirms that your environment is active and ready for PyTorch installation.

⚡ 2.2 Installing PyTorch

PyTorch can be installed in a variety of ways — through pip, conda, or directly from source. The simplest method is using the official PyTorch installation command.

💻 Step 1: Visit the Official Installation Selector

Go to:
👉 https://pytorch.org/get-started/locally/

You’ll see a dropdown menu where you can choose:

Your OS (Windows, Mac, Linux)
Package (pip or conda)
Language (Python)
Compute Platform (CPU or CUDA)

Once selected, it generates a custom install command.

💡 Step 2: Example Installation (CPU Only)

If you don’t have a GPU, install the CPU version:

pip install torch torchvision torchaudio

This installs:

torch: The main PyTorch library
torchvision: Tools for image datasets and transforms
torchaudio: Tools for handling audio datasets

⚙️ Step 3: Example Installation (with CUDA GPU)

If you have an NVIDIA GPU, you can enable CUDA acceleration.

Example for CUDA 12.1:

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

💡 Note: Make sure you have compatible NVIDIA drivers and CUDA toolkit installed.
You can check your driver version using:

nvidia-smi

🧠 2.3 Verifying Your Installation

After installation, verify that PyTorch is properly set up.

import torch

# Check version
print(torch.__version__)

# Check if CUDA is available
print("CUDA Available:", torch.cuda.is_available())

# Check current device
print("Device:", torch.device("cuda" if torch.cuda.is_available() else "cpu"))

Example Output:

2.5.0
CUDA Available: True
Device: cuda

If you see True for CUDA, congratulations 🎉 — your PyTorch can now leverage your GPU for deep learning tasks!

🧠 2.4 Installing Supporting Libraries

Deep learning projects often require additional packages for data processing, visualization, and experiment tracking. Let’s install the most common ones:

pip install numpy matplotlib pandas scikit-learn tqdm seaborn

NumPy – numerical computations
Matplotlib/Seaborn – data visualization
Pandas – data manipulation
Scikit-learn – classic ML utilities (train-test split, metrics, etc.)
tqdm – progress bars for training loops

🧪 2.5 Setting Up a Jupyter Notebook (Optional but Recommended)

Interactive notebooks are perfect for experimenting with PyTorch code, visualizing results, and documenting research.

Install Jupyter using:

pip install notebook

Then launch it with:

jupyter notebook

A new browser tab will open with a file explorer.
Create a new notebook, name it PyTorch_Intro.ipynb, and start experimenting!

⚡ 2.6 Running Your First PyTorch Test

Let’s confirm your setup with a simple test.

import torch

# Create a random tensor
x = torch.rand(3, 3)
print("Tensor:\n", x)

# Perform a matrix multiplication
y = torch.mm(x, torch.ones(3, 3))
print("\nResult:\n", y)

# Move to GPU (if available)
if torch.cuda.is_available():
    x = x.to("cuda")
    print("\nTensor moved to GPU successfully!")

Output Example:

Tensor:
 tensor([[0.45, 0.88, 0.32],
         [0.19, 0.67, 0.48],
         [0.75, 0.11, 0.59]])
Result:
 tensor([[1.65, 1.65, 1.65],
         [1.34, 1.34, 1.34],
         [1.45, 1.45, 1.45]])
Tensor moved to GPU successfully!

🧠 2.7 Common Installation Issues (and Fixes)

❌ 1. Version Conflict (pip vs conda)

If you face version errors, uninstall and reinstall:

pip uninstall torch torchvision torchaudio
pip cache purge
pip install torch torchvision torchaudio

❌ 2. CUDA Not Detected

Update your NVIDIA driver
Reinstall with the correct CUDA version (e.g., cu118, cu121)
Run:
```
python -m torch.utils.collect_env
```
This prints detailed info about your CUDA, Python, and PyTorch versions.

❌ 3. Import Error: DLL Load Failed (Windows)

Run:

pip install --upgrade setuptools wheel

and reinstall PyTorch.

🧩 2.8 Folder Structure for PyTorch Projects

Organizing your files early helps maintain a scalable project. Here’s a recommended structure:

pytorch_project/
│
├── data/                  # Datasets
├── models/                # Model architecture files
├── notebooks/             # Jupyter notebooks
├── outputs/               # Trained weights, logs
├── utils/                 # Helper functions (metrics, preprocessing)
├── requirements.txt       # Package dependencies
└── train.py               # Main training script

This structure works well even for large-scale projects — easy to maintain and extend.

Sponsor Key-Word

"This Content Sponsored by SBO Digital Marketing.
Mobile-Based Part-Time Job Opportunity by SBO!
Earn money online by doing simple content publishing and sharing tasks. Here's how:
Job Type: Mobile-based part-time work
Work Involves:
Content publishing
Content sharing on social media
Time Required: As little as 1 hour a day
Earnings: ₹300 or more daily
Requirements:
Active Facebook and Instagram account
Basic knowledge of using mobile and social media
For more details:
WhatsApp your Name and Qualification to 9994104160
a.Online Part Time Jobs from Home
b.Work from Home Jobs Without Investment
c.Freelance Jobs Online for Students
d.Mobile Based Online Jobs
e.Daily Payment Online Jobs
Keyword & Tag: #OnlinePartTimeJob #WorkFromHome #EarnMoneyOnline #PartTimeJob #jobs #jobalerts #withoutinvestmentjob"

✅ Summary of Section 2

By now, you should have:

Installed PyTorch (CPU or GPU version)
Verified your setup with a small test
Installed supporting libraries
Organized your working directory
Prepared your environment for deep learning experiments

You’re officially ready to start coding deep learning models in PyTorch! 🚀

Section 3: PyTorch Basics — Tensors, Operations, and GPU Acceleration

Everything in PyTorch revolves around tensors.
They are to PyTorch what arrays are to NumPy — but with superpowers like GPU support and automatic differentiation.

In this section, we’ll cover:

What tensors are
How to create and manipulate them
Mathematical operations
GPU acceleration
Real-world use cases

⚙️ 3.1 What Are Tensors?

A tensor is simply a multi-dimensional array that can represent data of any type — scalars, vectors, matrices, or higher dimensions.

You can think of tensors as the data containers for neural networks.

Tensor Type	Dimensions	Example	Description
Scalar	0D	`torch.tensor(5)`	Single number
Vector	1D	`torch.tensor([1, 2, 3])`	Sequence of numbers
Matrix	2D	`torch.tensor([[1, 2], [3, 4]])`	Rows and columns
3D Tensor	3D	Image tensor `[3, 224, 224]`	RGB channels
4D Tensor	4D	`[Batch, Channel, Height, Width]`	Batch of images

🧩 3.2 Creating Tensors in PyTorch

PyTorch provides multiple ways to create tensors.

1. From Python lists

import torch

data = [[1, 2], [3, 4]]
tensor = torch.tensor(data)
print(tensor)

Output:

tensor([[1, 2],
        [3, 4]])

2. Using built-in functions

x_ones = torch.ones((2, 3))
x_zeros = torch.zeros((2, 3))
x_random = torch.rand((2, 3))
x_identity = torch.eye(3)

3. From NumPy arrays

import numpy as np

arr = np.array([[1, 2, 3], [4, 5, 6]])
tensor_from_np = torch.from_numpy(arr)

4. Tensor with specific data type

x = torch.tensor([2.5, 0.5], dtype=torch.float32)

🧮 3.3 Tensor Attributes

Each tensor has important attributes that define its properties:

tensor = torch.rand(3, 4)

print("Shape:", tensor.shape)
print("Datatype:", tensor.dtype)
print("Device:", tensor.device)

Output:

Shape: torch.Size([3, 4])
Datatype: torch.float32
Device: cpu

🧠 3.4 Tensor Operations

Tensors support a wide variety of mathematical and logical operations.

1. Element-wise Operations

x = torch.tensor([1, 2, 3])
y = torch.tensor([3, 2, 1])
print(x + y)      # addition
print(x * y)      # multiplication
print(x ** 2)     # exponentiation

2. Matrix Operations

a = torch.rand(2, 3)
b = torch.rand(3, 2)
print(torch.mm(a, b))  # matrix multiplication

3. Statistical Operations

x = torch.randn(4, 4)
print(x.mean(), x.std(), x.max(), x.min())

4. Reshaping and Slicing

x = torch.arange(1, 10).reshape(3, 3)
print(x)
print(x[0])       # first row
print(x[:, 1])    # second column
print(x[1:, 1:])  # slicing

Output:

tensor([[1, 2, 3],
        [4, 5, 6],
        [7, 8, 9]])

⚡ 3.5 GPU Acceleration in PyTorch

One of the reasons PyTorch is so powerful is its seamless integration with GPUs using CUDA (NVIDIA’s parallel computing platform).

Checking for GPU availability

device = "cuda" if torch.cuda.is_available() else "cpu"
print(device)

Moving tensors to GPU

x = torch.rand(2, 2)
x_gpu = x.to(device)
print(x_gpu)

Performing operations on GPU

y = torch.rand(2, 2).to(device)
z = x_gpu + y
print(z.device)

All calculations performed on the GPU are significantly faster for large tensors and deep learning workloads.

🧩 3.6 Tensor Broadcasting

PyTorch supports broadcasting, a mechanism that automatically expands tensors with smaller dimensions to match larger tensors during operations.

Example:

x = torch.ones(3, 1)
y = torch.ones(1, 3)
z = x + y
print(z)

Output:

tensor([[2., 2., 2.],
        [2., 2., 2.],
        [2., 2., 2.]])

PyTorch automatically “stretches” dimensions to perform element-wise addition — this is broadcasting.

🧮 3.7 Automatic Type Conversion

PyTorch can easily convert data types:

x = torch.tensor([1, 2, 3], dtype=torch.int32)
x = x.to(torch.float32)
print(x)

And you can move between CPU and GPU anytime:

x_cpu = x.to("cpu")
x_gpu = x_cpu.to("cuda")

📈 3.8 Real-World Example: Representing Image Data

Suppose you have a color image with 3 channels (RGB) and dimensions 64×64 pixels.

image = torch.rand(3, 64, 64)
print(image.shape)

Output:

torch.Size([3, 64, 64])

To handle a batch of 10 such images:

batch = torch.rand(10, 3, 64, 64)
print(batch.shape)  # (Batch, Channels, Height, Width)

This structure is exactly what convolutional neural networks (CNNs) expect as input.

Sponsor Key-Word

"This Content Sponsored by SBO Digital Marketing.
Mobile-Based Part-Time Job Opportunity by SBO!
Earn money online by doing simple content publishing and sharing tasks. Here's how:
Job Type: Mobile-based part-time work
Work Involves:
Content publishing
Content sharing on social media
Time Required: As little as 1 hour a day
Earnings: ₹300 or more daily
Requirements:
Active Facebook and Instagram account
Basic knowledge of using mobile and social media
For more details:
WhatsApp your Name and Qualification to 9994104160
a.Online Part Time Jobs from Home
b.Work from Home Jobs Without Investment
c.Freelance Jobs Online for Students
d.Mobile Based Online Jobs
e.Daily Payment Online Jobs
Keyword & Tag: #OnlinePartTimeJob #WorkFromHome #EarnMoneyOnline #PartTimeJob #jobs #jobalerts #withoutinvestmentjob"

🧠 3.9 Combining PyTorch and NumPy

PyTorch integrates seamlessly with NumPy. You can convert between them easily:

Tensor → NumPy

a = torch.ones(5)
b = a.numpy()

NumPy → Tensor

import numpy as np
arr = np.zeros(5)
tensor = torch.from_numpy(arr)

They share memory locations — so changing one changes the other:

tensor[0] = 10
print(arr)  # updated automatically!

🧩 3.10 Tensor Cloning and Detaching

Sometimes you want to copy a tensor without affecting the original or detach it from the computational graph.

x = torch.tensor([2.0], requires_grad=True)
y = x ** 2
z = y.detach()  # detached tensor (no gradient tracking)

Or create a copy safely:

x_copy = x.clone()

⚡ 3.11 Practical Example: Dot Product & Similarity

In machine learning, comparing embeddings or features is common.
You can use tensor operations like dot product and cosine similarity.

a = torch.tensor([1, 2, 3], dtype=torch.float32)
b = torch.tensor([4, 5, 6], dtype=torch.float32)

# Dot product
dot = torch.dot(a, b)
print("Dot Product:", dot.item())

# Cosine similarity
cos_sim = torch.nn.functional.cosine_similarity(a.unsqueeze(0), b.unsqueeze(0))
print("Cosine Similarity:", cos_sim.item())

✅ Summary of Section 3

You’ve now mastered the foundation of PyTorch:

Creating and manipulating tensors
Performing mathematical and matrix operations
Utilizing GPU acceleration
Converting between NumPy and PyTorch
Handling real-world data shapes (like images and batches)

These are the core tools every deep learning engineer uses daily.

Section 4: Autograd and Computational Graphs in PyTorch

Deep learning models learn through optimization — adjusting their internal parameters (weights and biases) to minimize the error between predictions and actual outputs.
At the heart of this learning process lies backpropagation, which requires computing gradients (partial derivatives) for every parameter.

PyTorch automates this entire process using its powerful Autograd engine.
Let’s explore what it is, how it works, and how you can use it effectively in your models.

🧠 4.1 What Is Autograd?

Autograd is PyTorch’s automatic differentiation system.
It records all operations performed on tensors that have requires_grad=True, creating a dynamic computational graph.

When you call .backward(), PyTorch automatically traverses this graph from the output back to the inputs — computing all gradients along the way.

This mechanism makes backpropagation simple and efficient.

🧩 4.2 What Is a Computational Graph?

A computational graph is a structure that represents how data flows through a series of mathematical operations.

Each node in the graph is an operation (like addition or multiplication),
and each edge represents a tensor (data).

Let’s see a simple example:

Example function:

[
y = (x + 2)^2
]

Computational graph:

   x
   |
 (+2)
   |
 (^2)
   |
   y

When .backward() is called, PyTorch uses the chain rule to compute derivatives:
[
\frac{dy}{dx} = 2(x + 2)
]

🧮 4.3 Enabling Autograd in PyTorch

Let’s build a simple example using PyTorch tensors.

import torch

# Enable gradient tracking
x = torch.tensor(3.0, requires_grad=True)

# Define a simple function
y = x**2 + 2*x + 1

# Compute gradients
y.backward()

# Display gradient
print(x.grad)

Output:

tensor(8.)

✅ Because ( y = x^2 + 2x + 1 ),
the derivative is ( dy/dx = 2x + 2 = 8 ) when ( x = 3 ).

Sponsor Key-Word

"This Content Sponsored by SBO Digital Marketing.
Mobile-Based Part-Time Job Opportunity by SBO!
Earn money online by doing simple content publishing and sharing tasks. Here's how:
Job Type: Mobile-based part-time work
Work Involves:
Content publishing
Content sharing on social media
Time Required: As little as 1 hour a day
Earnings: ₹300 or more daily
Requirements:
Active Facebook and Instagram account
Basic knowledge of using mobile and social media
For more details:
WhatsApp your Name and Qualification to 9994104160
a.Online Part Time Jobs from Home
b.Work from Home Jobs Without Investment
c.Freelance Jobs Online for Students
d.Mobile Based Online Jobs
e.Daily Payment Online Jobs
Keyword & Tag: #OnlinePartTimeJob #WorkFromHome #EarnMoneyOnline #PartTimeJob #jobs #jobalerts #withoutinvestmentjob"

⚙️ 4.4 Autograd in Action: Multiple Variables

Autograd works even when your computation involves multiple tensors.

x = torch.tensor(2.0, requires_grad=True)
y = torch.tensor(3.0, requires_grad=True)

z = x**2 + y**3 + 2*x*y
z.backward()

print("dz/dx:", x.grad)
print("dz/dy:", y.grad)

Output:

dz/dx: tensor(7.)
dz/dy: tensor(15.)

🧩 Let’s verify:
[
z = x^2 + y^3 + 2xy
]
[
\frac{dz}{dx} = 2x + 2y = 7
]
[
\frac{dz}{dy} = 3y^2 + 2x = 15
]
✅ Matches perfectly.

🔄 4.5 Dynamic Computational Graphs

Unlike TensorFlow 1.x (which used static graphs),
PyTorch builds its graph dynamically during the forward pass.
That means each iteration can have different structures — perfect for models like RNNs that change over time.

Example:

for i in range(3):
    x = torch.tensor(float(i), requires_grad=True)
    y = x ** 2
    y.backward()
    print(f"x={i}, grad={x.grad}")

Output:

x=0, grad=0.0
x=1, grad=2.0
x=2, grad=4.0

Each loop iteration creates its own computational graph.

🧱 4.6 Stopping Gradient Tracking

Sometimes, you don’t want PyTorch to track operations (e.g., during inference or when using pre-trained models).

You can stop gradient tracking in three ways:

(a) Using `with torch.no_grad()`

x = torch.tensor(2.0, requires_grad=True)
y = x**2

with torch.no_grad():
    z = y * 3
print(z.requires_grad)  # False

(b) Using `.detach()`

x = torch.tensor(3.0, requires_grad=True)
y = x**3
z = y.detach()  # creates a tensor without grad history

(c) Disabling globally

torch.set_grad_enabled(False)

This saves memory and improves inference performance.

⚡ 4.7 Working with Gradients

Gradients accumulate in PyTorch.
That means if you call .backward() multiple times, gradients will be added to the existing ones.

Example:

x = torch.tensor(2.0, requires_grad=True)
y1 = x**2
y2 = 3*x

y1.backward(retain_graph=True)
print(x.grad)  # tensor(4.)

y2.backward()
print(x.grad)  # tensor(7.) -> (4 + 3)

To reset gradients before each backward pass:

x.grad.zero_()

🧮 4.8 Gradient Calculation with Vectors

For vector outputs, you must provide a gradient argument to .backward() to compute the Jacobian product.

x = torch.tensor([1.0, 2.0, 3.0], requires_grad=True)
y = x * 2
z = y.sum()

z.backward()
print(x.grad)

Output:

tensor([2., 2., 2.])

🧠 4.9 Real-World Example: Linear Regression Using Autograd

Let’s build a linear regression model manually — using only PyTorch’s Autograd (no nn.Module yet).

Step 1: Create Dataset

import torch

# y = 2x + 1
X = torch.tensor([[1.0], [2.0], [3.0], [4.0]])
Y = torch.tensor([[3.0], [5.0], [7.0], [9.0]])

# Random initialization
w = torch.randn(1, requires_grad=True)
b = torch.randn(1, requires_grad=True)

Step 2: Training Loop

lr = 0.01

for epoch in range(500):
    # Forward pass
    Y_pred = w * X + b

    # Loss (MSE)
    loss = torch.mean((Y_pred - Y) ** 2)

    # Backward pass
    loss.backward()

    # Update weights manually
    with torch.no_grad():
        w -= lr * w.grad
        b -= lr * b.grad

    # Zero gradients
    w.grad.zero_()
    b.grad.zero_()

print(f"Trained Weight: {w.item():.3f}, Bias: {b.item():.3f}")

Output:

Trained Weight: 2.000, Bias: 1.000

✅ The model correctly learns ( y = 2x + 1 ) using only PyTorch’s Autograd engine!

🔍 4.10 Visualizing the Gradient Flow (Conceptual)

You can imagine the gradient flow like a river that starts at the loss function and flows backward through the network layers.

Input → Linear → Activation → Loss
              ↑          ↑
          Backprop   Gradients

Every operation records how it affects the output.
When .backward() is called, PyTorch computes the gradient for each variable in reverse order using the chain rule.

✅ Summary of Section 4

You’ve now learned:

How PyTorch’s Autograd engine tracks computations dynamically
How to compute and manage gradients
How to stop gradient tracking for inference
How to implement manual backpropagation for simple models

These concepts form the mathematical and computational backbone of deep learning training.

Sponsor Key-Word

"This Content Sponsored by SBO Digital Marketing.
Mobile-Based Part-Time Job Opportunity by SBO!
Earn money online by doing simple content publishing and sharing tasks. Here's how:
Job Type: Mobile-based part-time work
Work Involves:
Content publishing
Content sharing on social media
Time Required: As little as 1 hour a day
Earnings: ₹300 or more daily
Requirements:
Active Facebook and Instagram account
Basic knowledge of using mobile and social media
For more details:
WhatsApp your Name and Qualification to 9994104160
a.Online Part Time Jobs from Home
b.Work from Home Jobs Without Investment
c.Freelance Jobs Online for Students
d.Mobile Based Online Jobs
e.Daily Payment Online Jobs
Keyword & Tag: #OnlinePartTimeJob #WorkFromHome #EarnMoneyOnline #PartTimeJob #jobs #jobalerts #withoutinvestmentjob"

PyTorch - 1 - Introduction, Setting Up the Environment, basics, Autograd and Computational Graphs

PyTorch -I

content

Introduction — What is Deep Learning & Why PyTorch?Setting Up the PyTorch Environment PyTorch Basics — Tensors, Operations, and GPU AccelerationAutograd and Computational Graphs

🧠 1. Introduction — What is Deep Learning & Why PyTorch?

🌍 The Rise of Intelligent Machines

🧩 What Exactly is Deep Learning?

🧠 How Deep Learning Works — A Simple Overview

⚙️ Why Deep Learning Has Become So Popular

🔥 Enter PyTorch — The Researcher’s Deep Learning Framework

🧩 Key Features that Make PyTorch Stand Out

🧰 PyTorch vs. TensorFlow: A Quick Comparison

🌐 Real-World Use Cases of PyTorch

🧩 Example:

🧱 The PyTorch Ecosystem — More Than Just a Framework

🧩 Simple Example — Your First PyTorch Program

🚀 Why You Should Learn PyTorch in 2025

Sponsor Key-Word

🧭 Transition to Next Section

Section 2: Setting Up the PyTorch Environment

🧩 2.1 What You Need Before Installing PyTorch

✅ Prerequisites

🧠 Why a Virtual Environment?

⚡ 2.2 Installing PyTorch

💻 Step 1: Visit the Official Installation Selector

💡 Step 2: Example Installation (CPU Only)

⚙️ Step 3: Example Installation (with CUDA GPU)

🧠 2.3 Verifying Your Installation

🧠 2.4 Installing Supporting Libraries

🧪 2.5 Setting Up a Jupyter Notebook (Optional but Recommended)

⚡ 2.6 Running Your First PyTorch Test

🧠 2.7 Common Installation Issues (and Fixes)

❌ 1. Version Conflict (pip vs conda)

❌ 2. CUDA Not Detected

❌ 3. Import Error: DLL Load Failed (Windows)

🧩 2.8 Folder Structure for PyTorch Projects

Sponsor Key-Word

✅ Summary of Section 2

Section 3: PyTorch Basics — Tensors, Operations, and GPU Acceleration

⚙️ 3.1 What Are Tensors?

🧩 3.2 Creating Tensors in PyTorch

1. From Python lists

2. Using built-in functions

3. From NumPy arrays

4. Tensor with specific data type

🧮 3.3 Tensor Attributes

🧠 3.4 Tensor Operations

1. Element-wise Operations

2. Matrix Operations

3. Statistical Operations

4. Reshaping and Slicing

⚡ 3.5 GPU Acceleration in PyTorch

Checking for GPU availability

Moving tensors to GPU

Performing operations on GPU

🧩 3.6 Tensor Broadcasting

Example:

🧮 3.7 Automatic Type Conversion

📈 3.8 Real-World Example: Representing Image Data

Sponsor Key-Word

🧠 3.9 Combining PyTorch and NumPy

Tensor → NumPy

NumPy → Tensor

🧩 3.10 Tensor Cloning and Detaching

⚡ 3.11 Practical Example: Dot Product & Similarity

✅ Summary of Section 3

Section 4: Autograd and Computational Graphs in PyTorch

🧠 4.1 What Is Autograd?

🧩 4.2 What Is a Computational Graph?

Example function:

🧮 4.3 Enabling Autograd in PyTorch

Sponsor Key-Word

⚙️ 4.4 Autograd in Action: Multiple Variables

🔄 4.5 Dynamic Computational Graphs

🧱 4.6 Stopping Gradient Tracking

(a) Using with torch.no_grad()

(b) Using .detach()

(c) Disabling globally

⚡ 4.7 Working with Gradients

Example:

Introduction — What is Deep Learning & Why PyTorch?
Setting Up the PyTorch Environment
PyTorch Basics — Tensors, Operations, and GPU Acceleration
Autograd and Computational Graphs

(a) Using `with torch.no_grad()`

(b) Using `.detach()`