🍨 本文为🔗365天深度学习训练营 中的学习记录博客
🍖 原作者:K同学啊|接辅导、项目定制
1 理论基础
Inception v3论文
Inception v3由谷歌研究员Christian Szegedy等人在2015年的论文《Rethinking the Inception Architecture for Computer Vision》中提出。Inception v3是Inception网络系列的第三个版本,它在ImageNet图像识别竞赛中取得了优异成绩,尤其是在大规模图像识别任务中表现出色。
Inception v3的主要特点如下:
1、更深的网络结构:Inception v3比之前的Inception网络结构更深,包含了48层卷积层。这使得网络可以提取更多层次的特征,从而在图像识别任务上取得更好的效果。
2、使用Factorized Convolutions:Inception v3采用了Factorized Convolutions(分解卷积),将较大的卷积核分解为多个较小卷积核。这种方法可以降低网络的参数数量,减少计算复杂度,同时保持良好的性能。
3、使用Batch Normalization:Inception v3在每个卷积层之后都添加了Batch Normalization(BN),这有助于网络的收敛和泛化能力。BN可以减少Internal Covariate Shift(内部协变量偏移)现象,加快训练速度,同时提高模型的鲁棒性。
4、辅助分类器:Inception v3引入了辅助分类器,可以在网络训练过程中提供额外的梯度信息,帮助网络更好的学习特征。辅助分类器位于网络的某个中间层,其输出会与主分类器的输出进行加权融合,从而得到最终的预测结果。
5、基于RMSProp的优化器:Inception v3使用了RMSProp优化器进行训练。相比于传统的随机梯度下降(SGD)方法,RMSProp可以自适应地调整学习率,使得训练过程更加稳定,收敛速度更快。
Inception v3在图像分类、物体检测和图像分割等计算机视觉任务中均取得了显著的效果。然而,由于其较大的网络结构和计算复杂度,Inception v3在实际应用中可能需要较高的硬件要求。
相对于Inception v1的Inception Module结构,Inception v3中做出了如下改动:
(1)将5x5的卷积分解为两个3x3的卷积运算已提升计算速度。尽管这有点违反直觉,但一个5x5的卷积在计算成本上是一个3x3卷积的2.78倍。所以叠加两个3x3卷积实际上在性能上会有所提升,如下图所示:
(2)此外,作者将nxn的卷积核尺寸分解为1xn和nx1两个卷积,例如, 一个3x3的卷积等价于首先执行一个1x3的卷积,再执行一个3x1的卷积。他们还发现这种方法在成本上要比单个3x3的卷积降低33%,这一结构如下图所示:
此处如果n=3,则与上一张图像一致。最左侧的5x5卷积可被表示为两个3x3卷积,太慢又可以表示为1x3和3x1卷积。
模块中的滤波器组被扩展(即变得更宽而不是更深),以解决表征性瓶颈。如果该模块没有被拓展宽度,而是变得更深,那么维度会过多减少,造成信息损失。如下图所示:
最后实现的inception v3网络是如下所示:
2 代码实现
2.1 开发环境
电脑系统:ubuntu16.04
编译器:Jupter Lab
语言环境:Python 3.7
深度学习环境:Pytorch
2.2 前期准备
2.2.1 设置GPU
import torch
import torch.nn as nn
import torchvision.transforms as transforms
import torchvision
from torchvision import transforms, datasets
import os, PIL, pathlib, warningswarnings.filterwarnings("ignore")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")print(device)
2.2.2 导入数据
import os,PIL,random,pathlib
data_dir = '../data/4-data/'
data_dir = pathlib.Path(data_dir)
data_dirdata_paths = list(data_dir.glob('*'))
classNames = [str(path).split('\\')[-1] for path in data_paths]
print('classNames:', classNames , '\n')total_dir = '../data/4-data/'
train_transforms = transforms.Compose([transforms.Resize([299, 299]), # resize输入图片transforms.ToTensor(), # 将PIL Image或numpy.ndarray转换成tensortransforms.Normalize(mean=[0.485, 0.456, 0.406],std=[0.229, 0.224, 0.225]) # 从数据集中随机抽样计算得到
])total_data = datasets.ImageFolder(total_dir, transform=train_transforms)
print(total_data, '\n')print(total_data.class_to_idx)
输出结果如下:
2.2.3 划分数据集
train_size = int(0.8 * len(total_data))
test_size = len(total_data) - train_size
train_dataset, test_dataset = torch.utils.data.random_split(total_data, [train_size, test_size])
print(train_dataset, test_dataset)batch_size = 4
train_dl = torch.utils.data.DataLoader(train_dataset, batch_size=batch_size,shuffle=True,num_workers=1,pin_memory=False)
test_dl = torch.utils.data.DataLoader(test_dataset, batch_size=batch_size,shuffle=True,num_workers=1,pin_memory=False)for X, y in test_dl:print("Shape of X [N, C, H, W]:", X.shape)print("Shape of y:", y.shape, y.dtype)break
输出结果如下所示:
2.3 Inception v3的实现
2.3.1 Inception-A
import torch
import torch.nn as nn
import torch.nn.functional as Fclass BasicConv2d(nn.Module):def __init__(self, in_channels, out_channels, **kwargs):super(BasicConv2d, self).__init__()self.conv = nn.Conv2d(in_channels, out_channels, bias=False, **kwargs)self.bn = nn.BatchNorm2d(out_channels, eps=0.001)def forward(self, x):x = self.conv(x)x = self.bn(x)return F.relu(x, inplace=True)class InceptionA(nn.Module):def __init__(self, in_channels, pool_features):super(InceptionA, self).__init__()# 1x1 conv branchself.branch1x1 = BasicConv2d(in_channels, 64, kernel_size=1) #1self.branch5x5_1 = BasicConv2d(in_channels, 48, kernel_size=1)self.branch5x5_2 = BasicConv2d(48, 64, kernel_size=5, padding=2)self.branch3x3dbl_1 = BasicConv2d(in_channels, 64, kernel_size=1)self.branch3x3dbl_2 = BasicConv2d(64, 96, kernel_size=3, padding=1)self.branch3x3dbl_3 = BasicConv2d(96, 96, kernel_size=3, padding=1)self.branch_pool = BasicConv2d(in_channels, pool_features, kernel_size=1)def forward(self, x):branch1x1 = self.branch1x1(x)branch5x5 = self.branch5x5_1(x)branch5x5 = self.branch5x5_2(branch5x5)branch3x3dbl = self.branch3x3dbl_1(x)branch3x3dbl = self.branch3x3dbl_2(branch3x3dbl)branch3x3dbl = self.branch3x3dbl_3(branch3x3dbl)branch_pool = F.avg_pool2d(x, kernel_size=3, stride=1, padding=1)branch_pool = self.branch_pool(branch_pool)outputs = [branch1x1, branch5x5, branch3x3dbl, branch_pool]return torch.cat(outputs, 1)
2.3.2 Inception-B
class InceptionB(nn.Module):def __init__(self, in_channels, channels_7x7):super(InceptionB, self).__init__()# 1x1 conv branchself.branch1x1 = BasicConv2d(in_channels, 192, kernel_size=1)c7 = channels_7x7self.branch7x7_1 = BasicConv2d(in_channels, c7, kernel_size=1)self.branch7x7_2 = BasicConv2d(c7, c7, kernel_size=(1,7), padding=(0,3))self.branch7x7_3 = BasicConv2d(c7, 192, kernel_size=(7,1), padding=(3,0))self.branch7x7dbl_1 = BasicConv2d(in_channels, c7, kernel_size=1)self.branch7x7dbl_2 = BasicConv2d(c7, c7, kernel_size=(7,1), padding=(3,0))self.branch7x7dbl_3 = BasicConv2d(c7, c7, kernel_size=(1,7), padding=(0,3))self.branch7x7dbl_4 = BasicConv2d(c7, c7, kernel_size=(7,1), padding=(3,0))self.branch7x7dbl_5 = BasicConv2d(c7, 192, kernel_size=(1,7), padding=(0,3))self.branch_pool = BasicConv2d(in_channels, 192, kernel_size=1)def forward(self, x):branch1x1 = self.branch1x1(x)branch7x7 = self.branch7x7_1(x)branch7x7 = self.branch7x7_2(branch7x7)branch7x7 = self.branch7x7_3(branch7x7)branch7x7dbl = self.branch7x7dbl_1(x)branch7x7dbl = self.branch7x7dbl_2(branch7x7dbl)branch7x7dbl = self.branch7x7dbl_3(branch7x7dbl)branch7x7dbl = self.branch7x7dbl_4(branch7x7dbl)branch7x7dbl = self.branch7x7dbl_5(branch7x7dbl)branch_pool = F.avg_pool2d(x, kernel_size=3, stride=1, padding=1)branch_pool = self.branch_pool(branch_pool)outputs = [branch1x1, branch7x7, branch7x7dbl, branch_pool]return torch.cat(outputs, 1)
2.3.3 Inception-C
class InceptionC(nn.Module):def __init__(self, in_channels):super(InceptionC, self).__init__()self.branch1x1 = BasicConv2d(in_channels, 320, kernel_size=1) #1self.branch3x3_1 = BasicConv2d(in_channels, 384, kernel_size=1)self.branch3x3_2a = BasicConv2d(384, 384, kernel_size=(1,3), padding=(0,1))self.branch3x3_2b = BasicConv2d(384, 384, kernel_size=(3,1), padding=(1,0))self.branch3x3dbl_1 = BasicConv2d(in_channels, 448, kernel_size=1)self.branch3x3dbl_2 = BasicConv2d(448, 384, kernel_size=3, padding=1)self.branch3x3dbl_3a = BasicConv2d(384, 384, kernel_size=(1,3), padding=(0,1))self.branch3x3dbl_3b = BasicConv2d(384, 384, kernel_size=(3,1), padding=(1,0))self.branch_pool = BasicConv2d(in_channels, 192, kernel_size=1)def forward(self, x):branch1x1 = self.branch1x1(x)branch3x3 = self.branch3x3_1(x)branch3x3 = [self.branch3x3_2a(branch3x3),self.branch3x3_2b(branch3x3),]branch3x3 = torch.cat(branch3x3, 1)branch3x3dbl = self.branch3x3dbl_1(x)branch3x3dbl = self.branch3x3dbl_2(branch3x3dbl)branch3x3dbl = [self.branch3x3dbl_3a(branch3x3dbl),self.branch3x3dbl_3b(branch3x3dbl),]branch3x3dbl = torch.cat(branch3x3dbl, 1)branch_pool = F.avg_pool2d(x, kernel_size=3, stride=1, padding=1)branch_pool = self.branch_pool(branch_pool)outputs = [branch1x1, branch3x3, branch3x3dbl, branch_pool]return torch.cat(outputs, 1)
2.3.4 Reduction-A
class ReductionA(nn.Module):def __init__(self, in_channels):super(ReductionA, self).__init__()self.branch3x3 = BasicConv2d(in_channels, 384, kernel_size=3, stride=2) self.branch3x3dbl_1 = BasicConv2d(in_channels, 64, kernel_size=1)self.branch3x3dbl_2 = BasicConv2d(64, 96, kernel_size=3, padding=1)self.branch3x3dbl_3 = BasicConv2d(96, 96, kernel_size=3, stride=2)def forward(self, x):branch3x3 = self.branch3x3(x)branch3x3dbl = self.branch3x3dbl_1(x)branch3x3dbl = self.branch3x3dbl_2(branch3x3dbl)branch3x3dbl = self.branch3x3dbl_3(branch3x3dbl)branch_pool = F.max_pool2d(x, kernel_size=3, stride=2)outputs = [branch3x3, branch3x3dbl, branch_pool]return torch.cat(outputs, 1)
2.3.5 Reduction-B
class ReductionB(nn.Module):def __init__(self, in_channels):super(ReductionB, self).__init__()self.branch3x3_1 = BasicConv2d(in_channels, 192, kernel_size=1) self.branch3x3_2 = BasicConv2d(192, 320, kernel_size=3, stride=2) self.branch7x7x3_1 = BasicConv2d(in_channels, 192, kernel_size=1)self.branch7x7x3_2 = BasicConv2d(192, 192, kernel_size=(1,7), padding=(0,3))self.branch7x7x3_3 = BasicConv2d(192, 192, kernel_size=(7,1), padding=(3,0))self.branch7x7x3_4 = BasicConv2d(192, 192, kernel_size=3, stride=2)def forward(self, x):branch3x3 = self.branch3x3_1(x)branch3x3 = self.branch3x3_2(branch3x3)branch7x7x3 = self.branch7x7x3_1(x)branch7x7x3 = self.branch7x7x3_2(branch7x7x3)branch7x7x3 = self.branch7x7x3_3(branch7x7x3)branch7x7x3 = self.branch7x7x3_4(branch7x7x3)branch_pool = F.max_pool2d(x, kernel_size=3, stride=2)outputs = [branch3x3, branch7x7x3, branch_pool]return torch.cat(outputs, 1)
2.3.6 辅助分支
class InceptionAux(nn.Module):def __init__(self, in_channels, num_classes):super(InceptionAux, self).__init__()self.conv0 = BasicConv2d(in_channels, 128, kernel_size=1)self.conv1 = BasicConv2d(128, 768, kernel_size=5)self.conv1.stddev = 0.01self.fc = nn.Linear(768, num_classes)self.fc.stddev = 0.001def forward(self, x):# 17x17x768x = F.avg_pool2d(x, kernel_size=5, stride=3)# 5x5x768x = self.conv0(x)# 5x5x128x = self.conv1(x)# 1x1x128x = x.view(x.size(0), -1)# 768x = self.fc(x)# num_classesreturn x
2.3.7 模型搭建
class InceptionV3(nn.Module):def __init__(self, num_classes=1000, aux_logits=False, transform_input=False):super(InceptionV3, self).__init__()self.aux_logits = aux_logitsself.transform_input = transform_inputself.conv2d_1a_3x3 = BasicConv2d(3, 32, kernel_size=3, stride=2)self.conv2d_2a_3x3 = BasicConv2d(32, 32, kernel_size=3)self.conv2d_2b_3x3 = BasicConv2d(32, 64, kernel_size=3, padding=1)self.conv2d_3b_1x1 = BasicConv2d(64, 80, kernel_size=1)self.conv2d_4a_3x3 = BasicConv2d(80, 192, kernel_size=3)self.Mixed_5b = InceptionA(192, pool_features=32)self.Mixed_5c = InceptionA(256, pool_features=64)self.Mixed_5d = InceptionA(288, pool_features=64)self.Mixed_6a = ReductionA(288)self.Mixed_6b = InceptionB(768, channels_7x7=128)self.Mixed_6c = InceptionB(768, channels_7x7=160)self.Mixed_6d = InceptionB(768, channels_7x7=160)self.Mixed_6e = InceptionB(768, channels_7x7=192)if aux_logits:self.AuxLogits = InceptionAux(768, num_classes)self.Mixed_7a = ReductionB(768)self.Mixed_7b = InceptionC(1280)self.Mixed_7c = InceptionC(2048)self.fc = nn.Linear(2048, num_classes)def forward(self, x):if self.transform_input:x = x.clone()x[:, 0] = x[:, 0] * (0.229 / 0.5) + (0.485 - 0.5) / 0.5x[:, 1] = x[:, 1] * (0.224 / 0.5) + (0.456 - 0.5) / 0.5x[:, 2] = x[:, 2] * (0.225 / 0.5) + (0.406 - 0.5) / 0.5# 299*299*3x = self.conv2d_1a_3x3(x)# 149*149*32x = self.conv2d_2a_3x3(x)# 147*147*32x = self.conv2d_2b_3x3(x)# 147*147*64x = F.max_pool2d(x, kernel_size=3, stride=2)# 73*73*64x = self.conv2d_3b_1x1(x)# 73*73*80x = self.conv2d_4a_3x3(x)# 71*71*192x = F.max_pool2d(x, kernel_size=3, stride=2)# 35*35*192x = self.Mixed_5b(x)# 35*35*256x = self.Mixed_5c(x)# 35*35*288x = self.Mixed_5d(x)# 35*35*288x = self.Mixed_6a(x)# 17*17*768x = self.Mixed_6b(x)# 17*17*768x = self.Mixed_6c(x)# 17*17*768x = self.Mixed_6d(x)# 17*17*768x = self.Mixed_6e(x)# 17*17*768if self.training and self.aux_logits:aux = self.AuxLogits(x)# 17*17*768x = self.Mixed_7a(x)# 8*8*1280x = self.Mixed_7b(x)# 8*8*2048x = self.Mixed_7c(x)# 8*8*2048x = F.avg_pool2d(x, kernel_size=8)# 1*1*2048x = F.dropout(x, training=self.training)# 1*1*2048x = x.view(x.size(0), -1)# 2048x = self.fc(x)# num_classesif self.training and self.aux_logits:return x, auxreturn x
2.3.8 查看模型详情
# 统计模型参数量以及其他指标
import torchsummary# 调用并将模型转移到GPU中
model = InceptionV3().to(device)# 显示网络结构
torchsummary.summary(model, (3, 299, 299))
print(model)
输出结果如下(由于内容较多,只展示前后部分内容):
(中间内容省略)
(中间内容省略)
2.4 训练模型
2.4.1 编写训练函数
# 训练循环
def train(dataloader, model, loss_fn, optimizer):size = len(dataloader.dataset) # 训练集的大小num_batches = len(dataloader) # 批次数目, (size/batch_size,向上取整)train_loss, train_acc = 0, 0 # 初始化训练损失和正确率for X, y in dataloader: # 获取图片及其标签X, y = X.to(device), y.to(device)# 计算预测误差pred = model(X) # 网络输出loss = loss_fn(pred, y) # 计算网络输出pred和真实值y之间的差距,y为真实值,计算二者差值即为损失# 反向传播optimizer.zero_grad() # grad属性归零loss.backward() # 反向传播optimizer.step() # 每一步自动更新# 记录acc与losstrain_acc += (pred.argmax(1) == y).type(torch.float).sum().item()train_loss += loss.item()train_acc /= sizetrain_loss /= num_batchesreturn train_acc, train_loss
2.4.2 编写测试函数
def test(dataloader, model, loss_fn):size = len(dataloader.dataset) # 训练集的大小num_batches = len(dataloader) # 批次数目, (size/batch_size,向上取整)test_loss, test_acc = 0, 0 # 初始化测试损失和正确率# 当不进行训练时,停止梯度更新,节省计算内存消耗# with torch.no_grad():for imgs, target in dataloader: # 获取图片及其标签with torch.no_grad():imgs, target = imgs.to(device), target.to(device)# 计算误差tartget_pred = model(imgs) # 网络输出loss = loss_fn(tartget_pred, target) # 计算网络输出和真实值之间的差距,targets为真实值,计算二者差值即为损失# 记录acc与losstest_loss += loss.item()test_acc += (tartget_pred.argmax(1) == target).type(torch.float).sum().item()test_acc /= sizetest_loss /= num_batchesreturn test_acc, test_loss
2.4.3 正式训练
import copyoptimizer = torch.optim.Adam(model.parameters(), lr = 1e-4)
loss_fn = nn.CrossEntropyLoss() #创建损失函数epochs = 40train_loss = []
train_acc = []
test_loss = []
test_acc = []best_acc = 0 #设置一个最佳准确率,作为最佳模型的判别指标if hasattr(torch.cuda, 'empty_cache'):torch.cuda.empty_cache()for epoch in range(epochs):model.train()epoch_train_acc, epoch_train_loss = train(train_dl, model, loss_fn, optimizer)#scheduler.step() #更新学习率(调用官方动态学习率接口时使用)model.eval()epoch_test_acc, epoch_test_loss = test(test_dl, model, loss_fn)#保存最佳模型到best_modelif epoch_test_acc > best_acc:best_acc = epoch_test_accbest_model = copy.deepcopy(model)train_acc.append(epoch_train_acc)train_loss.append(epoch_train_loss)test_acc.append(epoch_test_acc)test_loss.append(epoch_test_loss)#获取当前的学习率lr = optimizer.state_dict()['param_groups'][0]['lr']template = ('Epoch: {:2d}. Train_acc: {:.1f}%, Train_loss: {:.3f}, Test_acc:{:.1f}%, Test_loss:{:.3f}, Lr: {:.2E}')print(template.format(epoch+1, epoch_train_acc*100, epoch_train_loss, epoch_test_acc*100, epoch_test_loss, lr))PATH = './J7_best_model.pth'
torch.save(model.state_dict(), PATH)print('Done')
输出结果如下:
2.5 结果可视化
import matplotlib.pyplot as plt
#隐藏警告
import warnings
warnings.filterwarnings("ignore") #忽略警告信息
plt.rcParams['font.sans-serif'] = ['SimHei'] # 用来正常显示中文标签
plt.rcParams['axes.unicode_minus'] = False # 用来正常显示负号
plt.rcParams['figure.dpi'] = 100 #分辨率epochs_range = range(epochs)plt.figure(figsize=(12, 3))
plt.subplot(1, 2, 1)plt.plot(epochs_range, train_acc, label='Training Accuracy')
plt.plot(epochs_range, test_acc, label='Test Accuracy')
plt.legend(loc='lower right')
plt.title('Training and Validation Accuracy')plt.subplot(1, 2, 2)
plt.plot(epochs_range, train_loss, label='Training Loss')
plt.plot(epochs_range, test_loss, label='Test Loss')
plt.legend(loc='upper right')
plt.title('Training and Validation Loss')
plt.show()
输出结果如下:
3 总结
总体而言,Inception v3主要提出了分解卷积,将大卷积因式分解成小卷积核非对称卷积,体现在数学上就是矩阵的分解,即一个大矩阵可以分解成多个小矩阵相乘。
由于简单的增大Inception网络的规模是不可行的,这样会导致计算效率变低,Inception v3在v2的基础上去除低层辅助分类器,高层辅助分类器加入BN层作为正则化器。将较大的卷积核分解为串联的小卷积核,能够进行维度缩减,同时小卷积核在多次串联后,并不会缩小感受野,进而提取的特征所代表的感受野不受影响。而并联卷积核池化,避免了表征瓶颈。这样同时增加宽度核深度,平衡了网络的宽度和深度,因此提高了网络的质量,优化了网络的特征提取效果。