Updated April 15, 2023
Introduction to PyTorch GPU
As PyTorch helps to create many machine learning frameworks where scientific and tensor calculations can be done easily, it is important to use Graphics Processing Unit or GPU in PyTorch to enable deep learning where the works can be completed efficiently. Moreover, memory in the system can be easily manipulated and modified to store several processing computations, and hence computational graphs can be drawn easily with a rather simple interface.
What is PyTorch GPU?
GPU helps to perform a huge number of computations in a parallel format so that the work is completed faster. Operations are carried out in queuing form so that users can view both synchronous and asynchronous operations where data is copied simultaneously between CPU and GPU or between two GPUs. Queuing ensures that the operations are performed in a synchronous fashion, and parallel operations are carried out. Cross GPU operations cannot be done in PyTorch.
How to use PyTorch GPU?
The initial step is to check whether we have access to GPU.
import torch
torch.cuda.is_available()
The result must be true to work in GPU. So the next step is to ensure whether the operations are tagged to GPU rather than working with CPU.
A_train = torch.FloatTensor([4., 5., 6.])
A_train.is_cuda
We can use an API to transfer tensors from CPU to GPU, and this logic is followed in models as well.
A_train = A_train.to(device)
A_train.is_cuda
Model_trial = MyModel(args)
Model_trial.to(device)
The device is a variable initialized in PyTorch so that it can be used to hold the device where the training is happening either in CPU or GPU.
device = torch.device("cuda:4" if torch.cuda.is_available() else "cpu")
print(device)torch.cuda package supports CUDA tensor types but works with GPU computations. Hence, if GPU is used, it is common to use CUDA.
torch.cuda.current_device()
torch.cuda.device_count()
torch.cuda.get_device_name(0)
PyTorch GPU Example
GPUs are preferred over numpy due to the speed and the computational efficiency where several data can be computed along with graphs within a few minutes. If we have the proper device, it is easy to link GPU and work on the same. Forward and backward passes must be implemented in the network so that the computations are done faster.
import torch
import math
data_type = torch.float
device = torch.device("cpu")
device = torch.device("cuda:0")
x = torch.linspace(-math.pi, math.pi, 1500, device=device, dtype=data_type)
y = torch.sin(x)
a = torch.randn((), device=device, dtype=data_type)
b = torch.randn((), device=device, dtype=data_type)
c = torch.randn((), device=device, dtype=data_type)
d = torch.randn((), device=device, dtype=data_type)
learning_rate = 1e-6
for i in range(1500):
y_pred = a + b * m + c * m ** 2 + d * m ** 3
loss = (y_pred - y).pow(2).sum().item()
if i % 100 == 99:
print(i, loss)
grad_a = grad_y_pred.sum()
grad_b = (grad_y_pred * m).sum()
grad_c = (grad_y_pred * m** 2).sum()
grad_d = (grad_y_pred * m ** 3).sum()
a -= learning_rate * grad_a
b -= learning_rate * grad_b
c -= learning_rate * grad_c
d -= learning_rate * grad_d
print(f'Result: y = {a.item()} + {b.item()} x + {c.item()} x^2 + {d.item()} x^3')
GPU for Deep Learning
GPU helps in training models at a faster rate because all the models are run in parallel, and hence waiting time is not there. First, let us look into the GPUs that support deep learning.
- NVIDIA GeForce RTX 2060: NVIDIA ensures that the operations are running at a faster rate with Turing architecture involved in the system where RTX does the operation with speed faster than 6 times compared to its previous versions. RTX is known for supporting all types of games with its visual effects as well.
- NVIDIA GeForce GTX 1080: GTX 1080 has Pascal architecture, thus helping the system to focus into the power and efficiency of the system. In addition, there is a vapor chamber cooling available, thus reducing the heating issues while gaming or doing deep learning experiments.
- ZOTAC GeForce GTX 1070: This also follows Pascal architecture, where high performance, improved memory, and power efficiency are promised.
- NVIDIA Tesla K80: Dual GPU is offered in the system where performance is increased with improved reliability and aggregate memory bandwidth. In addition, Tesla K80 also manages server optimization.
Other GPUs include NVIDIA GeForce RTX 2080, NVIDIA GeForce RTX 3060, NVIDIA Titan RTX, NVIDIA Tesla v100, NVIDIA A100 and ASUS ROG Strix Radeon RX 570.
PyTorch GPU Idea
It is important that both data and network should co-exist in GPU so that computations can be performed easily. This applies to CPU as well. Both CPU and GPU are computational devices, and hence if any data calculations are to be carried out in the network, they should be inside the device.
The first step is to do the tensor computations, and here we should give the device as CPU or GPU based on our requirement. If we see CPU as the device, we can change it to CUDA, the GPU. We can do the same process in neural networks as well, where GPU is preferred more than CPU.
PyTorch GPU Network
The network’s parameter has to be moved to the device to make it work in GPU. We have weight and bias in convolution and functions parameters where it must be applied, and the system has to be initialized with parameter values. GPU initializes these parameters, and it must be noted that tensors inside networks are important for a device. Consistency to be maintained between network modules and PyTorch sensors.
All the new networks will be CPU by default, and we should move it to GPU to make it work.
try:
pred = network(sample.to('cuda'))
print(pred)
except Exception as e:
print(e)
tensor([[-0.0576, 0.0138, 0.1393, 0.2895, 0.1378, 0.0111, -0.0250, -0.0088, -0.0632, 0.0940]]
, device='cuda:0'
, grad_fn=
)
We can write agnostic code for the device where the code will not depend on any devices and work independently.
Conclusion
With more experience, we can improve the accuracy by trying with different epoch conditions, and we can try with different models where the training and test data can be given in different conditions. It is always unnecessary to train the models to complete to know the results to visualize them easily.
Recommended Articles
We hope that this EDUCBA information on “PyTorch GPU” was beneficial to you. You can view EDUCBA’s recommended articles for more information.
