当前位置:网站首页>To: pytoch model small example
To: pytoch model small example
2022-07-20 10:34:00 【III】
Link to the original text : Written in five minutes Pytorch Model
If you want to write in five minutes Pytorch Model , There are four steps to complete :
Import and preprocessing ( Data sets ) data , And batch it ( Data loader )
Use neural networks to build models .
Write a training cycle and run it .
Validation on validation set .
because MNIST It has been done very thoroughly , We will show you how to import torchvision Data sets , And write some code in five minutes . For this reason , It won't be very beautiful , But it will work .
1、 Download and import data
because MNIST It has been done to death , We will search for standard torchvision Data sets , See if there is anything else we want to try and predict .
Let's see Kuzushiji MNIST, It is from 70000 Composed of images MNIST Hiragana alternatives to datasets .
First , We find the mean and standard deviation for each channel . The reason behind this is , We want to normalize the training data , but Pytorch The transformation needs to give the normalized mean and standard deviation in advance . therefore , We will use the data set to find these values , Then re import it , And pass a normalized transformation with predefined values .
from torchvision import datasets
from torchvision import transforms
# Specify the dataset path as data, If the dataset does not exist, download it
# adopt train=False Get the test set
data_path = "data/"
kmnist = datasets.KMNIST(data_path, train=True, download=True,
transform=transforms.Compose([
transforms.ToTensor()]))
kmnist_val = datasets.KMNIST(data_path, train=False, download=True,
transform=transforms.ToTensor())
Of the above code Kuzushiji MNIST The data will be downloaded automatically , give the result as follows :
Please note that ,kmnist Is a dataset , So loop it will give us every image and every tag . therefore , If we cycle through each image in the dataset , And superimpose them along the extra fourth dimension , We will get the tensor of all the images .
# t.stack: Dimension expansion stack
imgs = torch.stack([img_t for img_t, _ in kmnist], dim=3)
print(f"imgs.shape: {imgs.shape}")
# torch.Size([1, 28, 28, 60000])
Now we calculate the average value for each channel . Please note that , call imgs.view(1,-1) All tensors will be compressed to the second dimension . We average the pixel values in this second dimension ( therefore dim=1) And standard deviation .
# compute mean and std per channel
mean = imgs.view(1, -1).mean(dim=1)
std = imgs.view(1, -1).std(dim=1)
print(f"mean:{mean},std:{std}")
# mean:tensor([0.1918]),std:tensor([0.3483])
We can now re import the data , Use Normalize Transform and transform arrays into tensors . Please note that ,Normalize Take the average value and standard deviation of pixel values as parameters .
# We can now re import the data , Use Normalize Transform and transform arrays into tensors .
# Please note that ,Normalize Take the average value and standard deviation of pixel values as parameters .
# normalised data
t_kmnist = datasets.KMNIST(data_path, train=True, download=False,
transform=transforms.Compose([
transforms.ToTensor(),
# transforms.Normalize(0.1307,0.3081),
transforms.Normalize(mean, std)
]))
t_kmnist_val = datasets.KMNIST(data_path, train=False, download=False,
transform=transforms.Compose([
transforms.ToTensor()]))
Now we have data sets , We need to input this data into the data loader for batch processing . If you're using CPU, Make sure to set a smaller batch size , And will num_workers=1( This is a GPU problem , Don't worry too much about ).
# Data loader
train_loader = torch.utils.data.DataLoader(kmnist, batch_size=128,
shuffle=True,
num_workers=1)
val_loader = torch.utils.data.DataLoader(kmnist_val, batch_size=128,
shuffle=False,
num_workers=1)
We can view some samples from the dataset .
# Traverse
figure = plt.figure(figsize=(10, 8))
cols, rows = 5, 5
for i in range(1, cols * rows + 1):
# randint return tensor,size=(1,) One dimensional column ,item Returns the value
sample_idx = torch.randint(len(t_kmnist), size=(1,)).item()
img, label = t_kmnist[sample_idx]
figure.add_subplot(rows, cols, i)
plt.title(label)
plt.axis("off")
plt.imshow(img.squeeze(), cmap="gray")
plt.show()
Build the model
This is not a tutorial on how to build a deep learning model theoretically . therefore , We will introduce the model implementation here .
First , The model needs to be instantiated as nn.Module Example . secondly , You need to use the usual Python Method initialization class . Last , You need a model initialization , Here we define all the model layers , And then a forward Method , Here we tell the model how to get input and pass it to these layers .
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.conv1 = nn.Conv2d(in_channels=1, out_channels=16, kernel_size=5,
stride=1, padding=2)
self.conv2 = nn.Conv2d(in_channels=16, out_channels=32, kernel_size=5, stride=1,
padding=2)
self.layer1 = nn.Sequential(self.conv1, nn.ReLU(), nn.MaxPool2d(kernel_size=2))
self.layer2 = nn.Sequential(self.conv2, nn.ReLU(), nn.MaxPool2d(kernel_size=2))
self.out = nn.Linear(32 * 7 * 7, 10) # output 10 classes
def forward(self, x):
x = self.layer1(x)
x = self.layer2(x)
# flatten the output of conv2 to (batch_size, 32*7*7)
x = x.view(x.size(0), -1)
output = self.out(x)
return output
At this stage , By getting from dataloader It is always important to give an example in to debug the model . then , We pass this image to the model , And check that it outputs the correct size of content .
img, _ = next(iter(train_loader))
img = img[0]
model = Net()
model(img.unsqueeze(0)).shape
>>> torch.Size([1, 10])
Perfectly we built a model , The model uses K-MNIST Images , And the output 10 Classes , Represents every possible number 0 To 9 Of 10 Different probabilities .
Write and run training cycles
As usual , Our training steps are similar . Forward propagation , Calculate the loss , Reset gradient (Pytorch). Back propagation to calculate the gradient of the loss . Update our weights with these gradients .
def training_loop(n_epochs, model, loss_fn, optimiser, train_loader):
model.train()
total_step = len(train_loader)
for n in range(1, n_epochs+1):
for i, (imgs, labels) in enumerate(train_loader):
output = model(Variable(imgs)) # forward pass
loss = loss_fn(output, Variable(labels)) # compute loss
optimiser.zero_grad() # reset gradients
loss.backward() # backpropagation
optimiser.step() # update the weights
if (i+1) % (n_epochs/10) == 0:
print ('{} Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}'
.format(datetime.datetime.now(), n, n_epochs,
i + 1, total_step, loss.item()))
then , We instantiate our model , Set up Adam Optimizer , And use cross entropy loss ( Because this is a multi class classification problem ).
model = Net()
optimiser = torch.optim.Adam(model.parameters(), lr=1e-2)
loss_fn = nn.CrossEntropyLoss()
Then pass these parameters to the training loop .
training_loop(
n_epochs = 10,
model = model,
loss_fn = loss_fn,
optimiser = optimiser,
train_loader = train_loader
)
Validate the model
Iteratively validate images and labels in the data loader , Forward propagation , By finding the index with the highest value in the output tensor ( remember , We output 10 A vector of probability ) Get predictions .data.squeeze() To get the actual scalar itself . Last , Count the number of samples with the predicted value equal to the label , Divide by the total number of tags .
def validate(model, train_loader, val_loader):
model.eval() # set to eval mode to avoid batchnorm
with torch.no_grad(): # avoid calculating gradients
correct, total = 0, 0
for images, labels in val_loader:
test_output = model(images)
pred_y = torch.max(test_output, 1)[1].data.squeeze()
accuracy = (pred_y == labels).sum().item() / float(labels.size(0))
print('VALIDATION SET ACCURACY: %.2f' % accuracy)
validate(model, train_loader, val_loader)
>>> VALIDATION SET ACCURACY: 0.95
The accuracy of the validation set is 95%.
Conclusion
Make the model better :
Print verification set indicators during model training : obviously , It's nice to see the training losses with each epoch And less . however , Until we validate the model after training , We really understand the performance of the model . If you print to verify accuracy during operation , You will better understand the success of the model .
Stop early : Once the accuracy of the verification is within a certain period ( Called patience ) No improvement , Just go back to the best epoch And use these weights .
Look at other indicators : For example, the area under the curve (AUC).
Implement other network architectures : since CNN Since its introduction , Computer vision has made great progress . You can try other architectures to improve performance .
边栏推荐
- 二叉树
- Regular Expression
- Lombok cooperates with logback to realize the simplest log output
- ftp port
- Statistical experiment data naming, data naming rules like:d8i8 txt
- 【资源记录】如何用Pytorch的DataLoader加载自己的数据集
- Kingbasees database administrator's Guide -- 13 table management
- 【资源记录】Invertible Neural Networks 可逆神经网络是什么,与VAE,GAN的关系;什么是Bits per pixel,Bits per dim
- vs2017 编译遇到COM 组件的调用返回了错误 HRESULT E_FAIL
- Nat overview
猜你喜欢
Lunix开机引导和故障修复
Openstack queens create fixed IP virtual machine
Kubernetes命令行管理工具 — Kubectl
静态路由工作原理与配置
Network packet capturing to understand the establishment process of TCP triple handshake
狂神。SMBMS(超市订单管理系统)
FIO test hard disk performance
[resource record] VAE learning notes
LVM and disk quotas
Disk and file system
随机推荐
分布式唯一ID
Statistical experiment data naming, data naming rules like:d8i8 txt
xpath过滤标签内的其他标签获得全部内容
VLAN概述
Variational Inference 笔记 from UCB CS 285 Sergey Levine
ZABBIX automatically discovers and monitors GPU
Process and planned task management
ZABBIX automatically discovers and monitors CEPH OSD
pytorch梯度的计算过程
[resource record] the relationship between manifold learning and PCA
oracle 查询语句总结
论文笔记:Neural Baby Talk
invalid syntax
CEPH deploy configure CEPH distributed cluster
SUSE 11 enter single user
Server hardware and RAID configuration and Practice
Scala case (companion object)
Mikrotik ROS soft route setting Internet access mode (II)
Openstack configures GPU passthrough (pci_passthrough)
Latex ieee paper首页标题下方插入跨分栏图片,并解决脚注问题