莱芜手机网站建设电话,wordpress怎样设置导航栏,网站板块怎么做,廊坊网站排名优化报价一个使用PyTorch实现图像分类的迁移学习实例 1. 导入模块2. 加载数据3. 模型处理4. 训练及验证模型5. 微调6. 其他代码 在特征提取中#xff0c;可以在预先训练好的网络结构后修改或添加一个简单的分类器#xff0c;然后将源任务上预先训练好的网络作为另一个目标任务的特征提… 一个使用PyTorch实现图像分类的迁移学习实例 1. 导入模块2. 加载数据3. 模型处理4. 训练及验证模型5. 微调6. 其他代码 在特征提取中可以在预先训练好的网络结构后修改或添加一个简单的分类器然后将源任务上预先训练好的网络作为另一个目标任务的特征提取器只对最后增加的分类器参数重新学习而预先训练好的网络参数不被修改或冻结。 在完成新任务的特征提取时使用的是源任务中学习到的参数而不用重新学习所有参数。下面的示例用一个实例具体说明如何通过特征提取的方法进行图像分类。
1. 导入模块
from datetime import datetimeimport matplotlib.pyplot as plt
import numpy as np
import torch
import torchvision
import torchvision.transforms as transforms
from torch import nn
from torchvision import models2. 加载数据
这里需要事先将CIFAR10数据下载到本地因为比较耗时因此将downloadFalse。除此之外还增加了一些预处理功能比如数据标准化、对图片进行裁剪等。
def load_data(data, batch_size64, num_workers2, meanNone, stdNone):if std is None:std [0.229, 0.224, 0.225]if mean is None:mean [0.485, 0.456, 0.406]trans_train transforms.Compose([transforms.RandomResizedCrop(224), transforms.RandomHorizontalFlip(),transforms.ToTensor(), transforms.Normalize(meanmean, stdstd)])trans_valid transforms.Compose([transforms.Resize(256), transforms.CenterCrop(224), transforms.ToTensor(),transforms.Normalize(meanmean, stdstd)])train_set torchvision.datasets.CIFAR10(rootdata, trainTrue, downloadTrue, transformtrans_train)trainloader torch.utils.data.DataLoader(train_set, batch_sizebatch_size, shuffleTrue, num_workersnum_workers)test_set torchvision.datasets.CIFAR10(rootdata, trainFalse, downloadTrue, transformtrans_valid)testloader torch.utils.data.DataLoader(test_set, batch_sizebatch_size, shuffleFalse, num_workersnum_workers)return trainloader, testloader3. 模型处理
这个部分包含三个操作
下载预训练模型使用的预训练模型为resnet18且已经在ImageNet大数据集上训练好了冻结模型参数使其在反向传播时不会更新修改最后一层的输出类别数该数据集中有1000个类别即原始输出为512×1000现将其修改为512×10因为这里使用的新数据集有10个类别
def freeze_net(num_class10):# 下载预训练模型net models.resnet18(pretrainedTrue)# 冻结模型参数for params in net.parameters():params.requires_grad False# 修改最后一层的输出类别数net.fc nn.Linear(512, num_class)# 查看冻结前后的参数情况total_params sum(p.numel() for p in net.parameters())print(f原总参数个数{total_params})total_trainable_params sum(p.numel() for p in net.parameters() if p.requires_grad)print(f需训练参数个数{total_trainable_params})return net原总参数个数11181642 需训练参数个数5130
从输出上可知如果不冻结需要更新的参数太多了冻结之后只需要更新全连接层的参数即可。
4. 训练及验证模型
这里选用交叉熵作为损失函数使用SGD作为优化器学习率为1e-3权重衰减设为1e-3代码如下
# 训练及验证模型
def train(net, train_data, valid_data, num_epochs, optimizer, criterion):prev_time datetime.now()for epoch in range(num_epochs):train_loss 0train_acc 0net net.train()for im, label in train_data:im im.to(device) # (bs, 3, h, w)label label.to(device) # (bs, h, w)# forwardoutput net(im)loss criterion(output, label)# backwardoptimizer.zero_grad()loss.backward()optimizer.step()train_loss loss.item()train_acc get_acc(output, label)cur_time datetime.now()h, remainder divmod((cur_time - prev_time).seconds, 3600)m, s divmod(remainder, 60)time_str Time %02d:%02d:%02d % (h, m, s)if valid_data is not None:valid_loss 0valid_acc 0net net.eval()for im, label in valid_data:im im.to(device) # (bs, 3, h, w)label label.to(device) # (bs, h, w)output net(im)loss criterion(output, label)valid_loss loss.item()valid_acc get_acc(output, label)epoch_str (Epoch %d. Train Loss: %f, Train Acc: %f, Valid Loss: %f, Valid Acc: %f, % (epoch, train_loss / len(train_data),train_acc / len(train_data), valid_loss / len(valid_data),valid_acc / len(valid_data)))else:epoch_str (Epoch %d. Train Loss: %f, Train Acc: %f, %(epoch, train_loss / len(train_data),train_acc / len(train_data)))prev_time cur_timeprint(epoch_str time_str)运行结果 Epoch 0. Train Loss: 1.474121, Train Acc: 0.498322, Valid Loss: 0.901339, Valid Acc: 0.713177, Time 00:03:26 Epoch 1. Train Loss: 1.222752, Train Acc: 0.576946, Valid Loss: 0.818926, Valid Acc: 0.730494, Time 00:04:35 Epoch 2. Train Loss: 1.172832, Train Acc: 0.592651, Valid Loss: 0.777265, Valid Acc: 0.737759, Time 00:04:23 Epoch 3. Train Loss: 1.158157, Train Acc: 0.596228, Valid Loss: 0.761969, Valid Acc: 0.746517, Time 00:04:28 Epoch 4. Train Loss: 1.143113, Train Acc: 0.600643, Valid Loss: 0.757134, Valid Acc: 0.742138, Time 00:04:24 Epoch 5. Train Loss: 1.128991, Train Acc: 0.607797, Valid Loss: 0.745840, Valid Acc: 0.747014, Time 00:04:24 Epoch 6. Train Loss: 1.131602, Train Acc: 0.603561, Valid Loss: 0.740176, Valid Acc: 0.748109, Time 00:04:21 Epoch 7. Train Loss: 1.127840, Train Acc: 0.608336, Valid Loss: 0.738235, Valid Acc: 0.751990, Time 00:04:19 Epoch 8. Train Loss: 1.122831, Train Acc: 0.609275, Valid Loss: 0.730571, Valid Acc: 0.751692, Time 00:04:18 Epoch 9. Train Loss: 1.118955, Train Acc: 0.609715, Valid Loss: 0.731084, Valid Acc: 0.751692, Time 00:04:13 Epoch 10. Train Loss: 1.111291, Train Acc: 0.612052, Valid Loss: 0.728281, Valid Acc: 0.749602, Time 00:04:09 Epoch 11. Train Loss: 1.108454, Train Acc: 0.612712, Valid Loss: 0.719465, Valid Acc: 0.752787, Time 00:04:15 Epoch 12. Train Loss: 1.111189, Train Acc: 0.612012, Valid Loss: 0.726525, Valid Acc: 0.751294, Time 00:04:09 Epoch 13. Train Loss: 1.114475, Train Acc: 0.610594, Valid Loss: 0.717852, Valid Acc: 0.754080, Time 00:04:06 Epoch 14. Train Loss: 1.112658, Train Acc: 0.608596, Valid Loss: 0.723336, Valid Acc: 0.751393, Time 00:04:14 Epoch 15. Train Loss: 1.109367, Train Acc: 0.614950, Valid Loss: 0.721230, Valid Acc: 0.752588, Time 00:04:06 Epoch 16. Train Loss: 1.107644, Train Acc: 0.614230, Valid Loss: 0.711586, Valid Acc: 0.755275, Time 00:04:08 Epoch 17. Train Loss: 1.100239, Train Acc: 0.613411, Valid Loss: 0.722191, Valid Acc: 0.749303, Time 00:04:11 Epoch 18. Train Loss: 1.108576, Train Acc: 0.611013, Valid Loss: 0.721263, Valid Acc: 0.753483, Time 00:04:08 Epoch 19. Train Loss: 1.098069, Train Acc: 0.618027, Valid Loss: 0.705413, Valid Acc: 0.757962, Time 00:04:06
从结果上看验证集的准确率达到75%左右。下面采用微调数据增强的方法继续提升准确率。
5. 微调
微调允许修改预训练好的网络参数来学习目标任务所以训练时间要比特征抽取方法长但精度更高。微调的大致过程是再预训练的网络上添加新的随机初始化层此外预训练的网络参数也会被更新但会使用较小的学习率以防止预训练好的参数发生较大改变。
常用的方法是固定底层的参数调整一些顶层或具体层的参数。这样可以减少训练参数的数量也可以避免过拟合的发生。尤其是针对目标任务的数据量不够大的时候该方法会很有效。
实际上微调优于特征提取因为它能对迁移过来的预训练网络参数进行优化使其更加适合新的任务。 1数据预处理 对训练数据添加了几种数据增强方法比如图片裁剪、旋转、颜色改变等方法。测试数据与特征提取的方法一样。 if fine_tuning is False:trans_train transforms.Compose([transforms.RandomResizedCrop(224),transforms.RandomHorizontalFlip(),transforms.ToTensor(),transforms.Normalize(meanmean, stdstd)])else:trans_train transforms.Compose([transforms.RandomResizedCrop(size256, scale(0.8, 1.0)),transforms.RandomRotation(degrees15),transforms.ColorJitter(),transforms.RandomResizedCrop(224),transforms.RandomHorizontalFlip(),transforms.ToTensor(),transforms.Normalize(meanmean, stdstd)])2修改模型的分类器层 修改最后全连接层把类别数由原来的1000改为10。
def freeze_net(num_class10, fine_tuningFalse):# 下载预训练模型net models.resnet18(pretrainedTrue)print(net)if fine_tuning is False:# 冻结模型参数for params in net.parameters():params.requires_grad False# 修改最后一层的输出类别数net.fc nn.Linear(512, num_class)# 查看冻结前后的参数情况total_params sum(p.numel() for p in net.parameters())print(f原总参数个数{total_params})total_trainable_params sum(p.numel() for p in net.parameters() if p.requires_grad)print(f需训练参数个数{total_trainable_params})# 打印出第一层的权重print(f第一层的权重{net.conv1.weight.type()})return net训练结果 Epoch 0. Train Loss: 1.455535, Train Acc: 0.488460, Valid Loss: 0.832547, Valid Acc: 0.721400, Time 00:14:48 Epoch 1. Train Loss: 1.342625, Train Acc: 0.530280, Valid Loss: 0.815430, Valid Acc: 0.723500, Time 10:31:48 Epoch 2. Train Loss: 1.319122, Train Acc: 0.535680, Valid Loss: 0.866512, Valid Acc: 0.699000, Time 00:12:02 Epoch 3. Train Loss: 1.310949, Train Acc: 0.541700, Valid Loss: 0.789511, Valid Acc: 0.728000, Time 00:12:03 Epoch 4. Train Loss: 1.313486, Train Acc: 0.538500, Valid Loss: 0.762553, Valid Acc: 0.741300, Time 00:12:19 Epoch 5. Train Loss: 1.309776, Train Acc: 0.540680, Valid Loss: 0.777906, Valid Acc: 0.736100, Time 00:11:43 Epoch 6. Train Loss: 1.302117, Train Acc: 0.541780, Valid Loss: 0.779318, Valid Acc: 0.737200, Time 00:12:00 Epoch 7. Train Loss: 1.304539, Train Acc: 0.544320, Valid Loss: 0.795917, Valid Acc: 0.726500, Time 00:13:16 Epoch 8. Train Loss: 1.311748, Train Acc: 0.542400, Valid Loss: 0.785983, Valid Acc: 0.728000, Time 00:14:48 Epoch 9. Train Loss: 1.302069, Train Acc: 0.544820, Valid Loss: 0.781665, Valid Acc: 0.734700, Time 00:14:15 Epoch 10. Train Loss: 1.298019, Train Acc: 0.547040, Valid Loss: 0.771555, Valid Acc: 0.742200, Time 00:16:11 Epoch 11. Train Loss: 1.310127, Train Acc: 0.538700, Valid Loss: 0.764313, Valid Acc: 0.739300, Time 00:17:33 Epoch 12. Train Loss: 1.300172, Train Acc: 0.544720, Valid Loss: 0.765881, Valid Acc: 0.734200, Time 00:12:04 Epoch 13. Train Loss: 1.289607, Train Acc: 0.546980, Valid Loss: 0.753371, Valid Acc: 0.742500, Time 00:11:49 Epoch 14. Train Loss: 1.295938, Train Acc: 0.546280, Valid Loss: 0.821099, Valid Acc: 0.721900, Time 00:11:43 …
使用微调训练方式的时间明显大于使用特征提取方式的时间但是验证集上的准确率并没有提高这是因为由于GPU内存限制这里将batch_size设为了16。
6. 其他代码
if __name__ __main__:data_path ./dataclasses (plane, car, bird, cat, deer, dog, forg, horse, ship, truck)if torch.cuda.is_available():device torch.device(cuda:0)torch.cuda.empty_cache()else:device torch.device(cpu)# 加载数据train_loader, test_loader load_data(datadata_path, fine_tuningTrue)# 随机获取部分训练数据data_iter iter(train_loader)images, labels data_iter.next()# 显示图像imshow(torchvision.utils.make_grid(images[:4]))# 打印标签print( .join(%5s % classes[labels[j]] for j in range(4)))# 加载模型net freeze_net(num_classlen(classes), fine_tuningTrue)net net.to(device)# 定义损失函数及优化器criterion nn.CrossEntropyLoss()# 只需要优化最后一层参数optimizer torch.optim.SGD(net.fc.parameters(), lr1e-3, weight_decay1e-3, momentum0.9)# 训练及验证模型train(net, train_loader, test_loader, 20, optimizer, criterion)
