PyTorch torch.no_grad() is a context manager “used to disable gradient calculation within a specific block of code.” This is helpful for inference or evaluation phases when you are sure you will not call backward(). Disabling gradient calculation can reduce memory consumption and speed up computations.
In this mode, the result of every computation will have requires_grad=False, even when the inputs have requires_grad=True.
with torch.no_grad(): # Your code here
The torch.no_grad() context manager is typically used with a with statement to specify the block of code where gradients should not be computed:
Example 1: How to Use torch.no_grad()
Here’s a simple example showing how the torch.no_grad() works:
import torch x = torch.tensor([1.0, 2.0], requires_grad=True) y = x * 2 with torch.no_grad(): z = y + 2 print("Does z require gradients? : ", z.requires_grad)
Does z require gradients? : False
In this code example, the tensor z will not require gradients, even though it is computed from y, which does require gradients.
When evaluating a model, wrapping the evaluation code in a torch is a good practice.no_grad() block. This can help save memory and make the model run faster.
import torch import torch.nn as nn class SimpleModel(nn.Module): def __init__(self): super(SimpleModel, self).__init__() self.fc = nn.Linear(10, 10) def forward(self, x): return self.fc(x) # Initialize the model and set it to evaluation mode model = SimpleModel() model.eval() # Create a sample input tensor input_tensor = torch.randn(1, 10) # Evaluate the model without gradient computation with torch.no_grad(): output = model(input_tensor) print("Does output require gradients? : ", output.requires_grad)
Does output require gradients? : False
In this example, the output tensor does not require gradients, which is what you would typically want during evaluation.