如何对网站页面进行优化,个人建购物网站怎么备案,佛山主题网站设计多少钱,手机网站是怎么制作的YOLOv11 NCNN安卓部署
前言
yolov11 NCNN安卓部署 目前的帧率可以稳定在20帧左右#xff0c;下面是这个项目的github地址#xff1a;https://github.com/gaoxumustwin/ncnn-android-yolov11
上面的检测精度很低时因为这个模型只训练了5个epoch#xff0c;使用3090训练一个…YOLOv11 NCNN安卓部署
前言
yolov11 NCNN安卓部署 目前的帧率可以稳定在20帧左右下面是这个项目的github地址https://github.com/gaoxumustwin/ncnn-android-yolov11
上面的检测精度很低时因为这个模型只训练了5个epoch使用3090训练一个epoch需要15分钟后续会把训练50个epoch和100个epoch的权重更新到仓库中
在之前复现了一个yolov8pose ncnn安卓部署的项目在逛github的时候发现了一个关于yolov11的ncnn仓库,看了一下代码发现作者是根据三木君大佬的代码进行改写所以跟yolov8pose ncnn的非常的类似所以就趁着刚改写的热乎劲把yolov11 ncnn 安卓部署的代码改写出来
环境配置
写这个blog的时候安装时间为2024年11月29日
pip install ultralytics安装后的ultralytics版本为8.3.39安装后的路径为:/root/miniconda3/lib/python3.8/site-packages/ultralytics
数据配置
yolov11的默认检测模型是使用COCO2017数据集进行训练如果训练COCO数据集建议在autodl上进行训练因为coco2017数据集在autodl上是公开数据集
如何查看autodl的共享数据
rootautodl-container-3686439328-168c7bd7:~# ls /root/autodl-pub/
ADEChallengeData2016 COCO2017 DIV2K ImageNet100 VOCdevkit mvtec_anomaly_detection.tar.xz
Aishell CUB200-2011 DOTA KITTI_Depth_Completion.tar Vimeo-90k nuScenes
BERT-Pretrain-Model CULane GOT10k KITTI_Object cifar-100
CASIAWebFace CelebA ImageNet SemanticKITTI cityscapes数据制作
如果在实例中找到了自己需要的数据集想使用共享数据不能直接解压会出现只读错误需要解压到自己的数据盘中/root/autodl-tmp
按照下面的流程操作即可
cd /root/autodl-tmp/
mkdir images
cd images
unzip /root/autodl-pub/COCO2017/train2017.zip
unzip /root/autodl-pub/COCO2017/val2017.zip此时images下面只有 train2017 val2017
下载COCO2017的标签
cd /root/autodl-tmp
mkdir labels
cd labels
wget https://github.com/ultralytics/assets/releases/download/v0.0.0/coco2017labels.zip
unzip coco2017labels.zip
rm coco2017labels.zip
cd coco
rm -r annotations/
rm -r images/
rm -r LICENSE
rm -r README.txt
rm -r test-dev2017.txt
rm -r train2017.txt
rm -r val2017.txt
mv labels/* ../
rm -r coco/ 此时labels下面只有 train2017 val2017
数据配置文件
复制COCO2017的配置文件到训练目录下
# workspace root
mkdir train
cp /root/miniconda3/lib/python3.8/site-packages/ultralytics/cfg/datasets/coco.yaml ./train修改coco.yaml中的path、train和val
# Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]
path: /root/autodl-tmp # dataset root dir
train: images/train2017 # train images (relative to path) 118287 images
val: images/val2017 # val images (relative to path) 5000 images更换激活函数
YOLOv11默认使用的激活函数是SiLU,换成计算更高效的ReLU
更换激活函数后原有的Pytorch模型需要重新训练再导出ONNX
修改/root/miniconda3/lib/python3.8/site-packages/ultralytics/nn/modules/conv.py中的第39行左右的default_act nn.SiLU() 修改为 default_act nn.ReLU()
训练
下载预训练权重
wget https://github.com/ultralytics/assets/releases/download/v8.3.0/yolo11n.pt训练
训练脚本train.py
from ultralytics import YOLOmodel YOLO(yolo11.yaml).load(yolo11n.pt) # 加载预训练模型 还是有用的 有助于训练results model.train(data./coco.yaml, epochs100, imgsz640, batch64, projectruns)模型导出
模型结构修改
使用下面的方式修改模型结构不影响训练
修改/root/miniconda3/lib/python3.8/site-packages/ultralytics/nn/modules/head.py文件修改Detect类的导出函数在其forward函数中加如下代码
if self.export or torch.onnx.is_in_onnx_export():results self.forward_export(x)return tuple(results)同时在Detect类新加上如下函数
def forward_export(self, x):results []for i in range(self.nl):dfl self.cv2[i](x[i]).permute(0, 2, 3, 1)cls self.cv3[i](x[i]).sigmoid().permute(0, 2, 3, 1)results.append(torch.cat((dfl, cls), -1))return results修改后的整体代码效果如下
class Detect(nn.Module):YOLO Detect head for detection models.dynamic False # force grid reconstructionexport False # export modeformat None # export formatend2end False # end2endmax_det 300 # max_detshape Noneanchors torch.empty(0) # initstrides torch.empty(0) # initlegacy False # backward compatibility for v3/v5/v8/v9 modelsdef __init__(self, nc80, ch()):Initializes the YOLO detection layer with specified number of classes and channels.super().__init__()self.nc nc # number of classesself.nl len(ch) # number of detection layersself.reg_max 16 # DFL channels (ch[0] // 16 to scale 4/8/12/16/20 for n/s/m/l/x)self.no nc self.reg_max * 4 # number of outputs per anchorself.stride torch.zeros(self.nl) # strides computed during buildc2, c3 max((16, ch[0] // 4, self.reg_max * 4)), max(ch[0], min(self.nc, 100)) # channelsself.cv2 nn.ModuleList(nn.Sequential(Conv(x, c2, 3), Conv(c2, c2, 3), nn.Conv2d(c2, 4 * self.reg_max, 1)) for x in ch)self.cv3 (nn.ModuleList(nn.Sequential(Conv(x, c3, 3), Conv(c3, c3, 3), nn.Conv2d(c3, self.nc, 1)) for x in ch)if self.legacyelse nn.ModuleList(nn.Sequential(nn.Sequential(DWConv(x, x, 3), Conv(x, c3, 1)),nn.Sequential(DWConv(c3, c3, 3), Conv(c3, c3, 1)),nn.Conv2d(c3, self.nc, 1),)for x in ch))self.dfl DFL(self.reg_max) if self.reg_max 1 else nn.Identity()if self.end2end:self.one2one_cv2 copy.deepcopy(self.cv2)self.one2one_cv3 copy.deepcopy(self.cv3)def forward(self, x):Concatenates and returns predicted bounding boxes and class probabilities.if self.export or torch.onnx.is_in_onnx_export():results self.forward_export(x)return tuple(results)if self.end2end:return self.forward_end2end(x)for i in range(self.nl):x[i] torch.cat((self.cv2[i](x[i]), self.cv3[i](x[i])), 1)if self.training: # Training pathreturn xy self._inference(x)return y if self.export else (y, x)def forward_export(self, x):results []for i in range(self.nl):dfl self.cv2[i](x[i]).permute(0, 2, 3, 1)cls self.cv3[i](x[i]).sigmoid().permute(0, 2, 3, 1)results.append(torch.cat((dfl, cls), -1))return results导出的名字修改
如果需要修改输出的名称则要去修改/root/miniconda3/lib/python3.8/site-packages/ultralytics/engine/exporter.py 的 export_onnx函数
导出
导出脚本export.py
from ultralytics import YOLO# load model
model YOLO(best.pt)# export onnx
model.export(formatonnx, opset11, simplifyTrue, dynamicFalse, imgsz640)NCNN转化和优化
$ ./onnx2ncnn best.onnx yolov11-relu.param yolov11-relu.bin$ ./ncnnoptimize yolov11-relu.param yolov11-relu.bin yolov11-relu-opt.param yolov11-relu-opt.bin 1
fuse_convolution_activation /model.0/conv/Conv /model.0/act/Relu
fuse_convolution_activation /model.1/conv/Conv /model.1/act/Relu
fuse_convolution_activation /model.2/cv1/conv/Conv /model.2/cv1/act/Relu
fuse_convolution_activation /model.2/m.0/cv1/conv/Conv /model.2/m.0/cv1/act/Relu
fuse_convolution_activation /model.2/m.0/cv2/conv/Conv /model.2/m.0/cv2/act/Relu
fuse_convolution_activation /model.2/cv2/conv/Conv /model.2/cv2/act/Relu
fuse_convolution_activation /model.3/conv/Conv /model.3/act/Relu
fuse_convolution_activation /model.4/cv1/conv/Conv /model.4/cv1/act/Relu
fuse_convolution_activation /model.4/m.0/cv1/conv/Conv /model.4/m.0/cv1/act/Relu
fuse_convolution_activation /model.4/m.0/cv2/conv/Conv /model.4/m.0/cv2/act/Relu
fuse_convolution_activation /model.4/cv2/conv/Conv /model.4/cv2/act/Relu
fuse_convolution_activation /model.5/conv/Conv /model.5/act/Relu
fuse_convolution_activation /model.6/cv1/conv/Conv /model.6/cv1/act/Relu
fuse_convolution_activation /model.6/m.0/cv1/conv/Conv /model.6/m.0/cv1/act/Relu
fuse_convolution_activation /model.6/m.0/cv2/conv/Conv /model.6/m.0/cv2/act/Relu
fuse_convolution_activation /model.6/m.0/m/m.0/cv1/conv/Conv /model.6/m.0/m/m.0/cv1/act/Relu
fuse_convolution_activation /model.6/m.0/m/m.0/cv2/conv/Conv /model.6/m.0/m/m.0/cv2/act/Relu
fuse_convolution_activation /model.6/m.0/m/m.1/cv1/conv/Conv /model.6/m.0/m/m.1/cv1/act/Relu
fuse_convolution_activation /model.6/m.0/m/m.1/cv2/conv/Conv /model.6/m.0/m/m.1/cv2/act/Relu
fuse_convolution_activation /model.6/m.0/cv3/conv/Conv /model.6/m.0/cv3/act/Relu
fuse_convolution_activation /model.6/cv2/conv/Conv /model.6/cv2/act/Relu
fuse_convolution_activation /model.7/conv/Conv /model.7/act/Relu
fuse_convolution_activation /model.8/cv1/conv/Conv /model.8/cv1/act/Relu
fuse_convolution_activation /model.8/m.0/cv1/conv/Conv /model.8/m.0/cv1/act/Relu
fuse_convolution_activation /model.8/m.0/cv2/conv/Conv /model.8/m.0/cv2/act/Relu
fuse_convolution_activation /model.8/m.0/m/m.0/cv1/conv/Conv /model.8/m.0/m/m.0/cv1/act/Relu
fuse_convolution_activation /model.8/m.0/m/m.0/cv2/conv/Conv /model.8/m.0/m/m.0/cv2/act/Relu
fuse_convolution_activation /model.8/m.0/m/m.1/cv1/conv/Conv /model.8/m.0/m/m.1/cv1/act/Relu
fuse_convolution_activation /model.8/m.0/m/m.1/cv2/conv/Conv /model.8/m.0/m/m.1/cv2/act/Relu
fuse_convolution_activation /model.8/m.0/cv3/conv/Conv /model.8/m.0/cv3/act/Relu
fuse_convolution_activation /model.8/cv2/conv/Conv /model.8/cv2/act/Relu
fuse_convolution_activation /model.9/cv1/conv/Conv /model.9/cv1/act/Relu
fuse_convolution_activation /model.9/cv2/conv/Conv /model.9/cv2/act/Relu
fuse_convolution_activation /model.10/cv1/conv/Conv /model.10/cv1/act/Relu
fuse_convolution_activation /model.10/m/m.0/ffn/ffn.0/conv/Conv /model.10/m/m.0/ffn/ffn.0/act/Relu
fuse_convolution_activation /model.10/cv2/conv/Conv /model.10/cv2/act/Relu
fuse_convolution_activation /model.13/cv1/conv/Conv /model.13/cv1/act/Relu
fuse_convolution_activation /model.13/m.0/cv1/conv/Conv /model.13/m.0/cv1/act/Relu
fuse_convolution_activation /model.13/m.0/cv2/conv/Conv /model.13/m.0/cv2/act/Relu
fuse_convolution_activation /model.13/cv2/conv/Conv /model.13/cv2/act/Relu
fuse_convolution_activation /model.16/cv1/conv/Conv /model.16/cv1/act/Relu
fuse_convolution_activation /model.16/m.0/cv1/conv/Conv /model.16/m.0/cv1/act/Relu
fuse_convolution_activation /model.16/m.0/cv2/conv/Conv /model.16/m.0/cv2/act/Relu
fuse_convolution_activation /model.16/cv2/conv/Conv /model.16/cv2/act/Relu
fuse_convolution_activation /model.17/conv/Conv /model.17/act/Relu
fuse_convolution_activation /model.23/cv2.0/cv2.0.0/conv/Conv /model.23/cv2.0/cv2.0.0/act/Relu
fuse_convolution_activation /model.23/cv2.0/cv2.0.1/conv/Conv /model.23/cv2.0/cv2.0.1/act/Relu
fuse_convolution_activation /model.23/cv3.0/cv3.0.0/cv3.0.0.1/conv/Conv /model.23/cv3.0/cv3.0.0/cv3.0.0.1/act/Relu
fuse_convolution_activation /model.19/cv1/conv/Conv /model.19/cv1/act/Relu
fuse_convolution_activation /model.19/m.0/cv1/conv/Conv /model.19/m.0/cv1/act/Relu
fuse_convolution_activation /model.23/cv3.0/cv3.0.1/cv3.0.1.1/conv/Conv /model.23/cv3.0/cv3.0.1/cv3.0.1.1/act/Relu
fuse_convolution_activation /model.19/m.0/cv2/conv/Conv /model.19/m.0/cv2/act/Relu
fuse_convolution_activation /model.23/cv3.0/cv3.0.2/Conv /model.23/Sigmoid
fuse_convolution_activation /model.19/cv2/conv/Conv /model.19/cv2/act/Relu
fuse_convolution_activation /model.20/conv/Conv /model.20/act/Relu
fuse_convolution_activation /model.23/cv2.1/cv2.1.0/conv/Conv /model.23/cv2.1/cv2.1.0/act/Relu
fuse_convolution_activation /model.23/cv2.1/cv2.1.1/conv/Conv /model.23/cv2.1/cv2.1.1/act/Relu
fuse_convolution_activation /model.23/cv3.1/cv3.1.0/cv3.1.0.1/conv/Conv /model.23/cv3.1/cv3.1.0/cv3.1.0.1/act/Relu
fuse_convolution_activation /model.22/cv1/conv/Conv /model.22/cv1/act/Relu
fuse_convolution_activation /model.22/m.0/cv1/conv/Conv /model.22/m.0/cv1/act/Relu
fuse_convolution_activation /model.22/m.0/cv2/conv/Conv /model.22/m.0/cv2/act/Relu
fuse_convolution_activation /model.23/cv3.1/cv3.1.1/cv3.1.1.1/conv/Conv /model.23/cv3.1/cv3.1.1/cv3.1.1.1/act/Relu
fuse_convolution_activation /model.22/m.0/m/m.0/cv1/conv/Conv /model.22/m.0/m/m.0/cv1/act/Relu
fuse_convolution_activation /model.23/cv3.1/cv3.1.2/Conv /model.23/Sigmoid_1
fuse_convolution_activation /model.22/m.0/m/m.0/cv2/conv/Conv /model.22/m.0/m/m.0/cv2/act/Relu
fuse_convolution_activation /model.22/m.0/m/m.1/cv1/conv/Conv /model.22/m.0/m/m.1/cv1/act/Relu
fuse_convolution_activation /model.22/m.0/m/m.1/cv2/conv/Conv /model.22/m.0/m/m.1/cv2/act/Relu
fuse_convolution_activation /model.22/m.0/cv3/conv/Conv /model.22/m.0/cv3/act/Relu
fuse_convolution_activation /model.22/cv2/conv/Conv /model.22/cv2/act/Relu
fuse_convolution_activation /model.23/cv2.2/cv2.2.0/conv/Conv /model.23/cv2.2/cv2.2.0/act/Relu
fuse_convolution_activation /model.23/cv2.2/cv2.2.1/conv/Conv /model.23/cv2.2/cv2.2.1/act/Relu
fuse_convolution_activation /model.23/cv3.2/cv3.2.0/cv3.2.0.1/conv/Conv /model.23/cv3.2/cv3.2.0/cv3.2.0.1/act/Relu
fuse_convolution_activation /model.23/cv3.2/cv3.2.1/cv3.2.1.1/conv/Conv /model.23/cv3.2/cv3.2.1/cv3.2.1.1/act/Relu
fuse_convolution_activation /model.23/cv3.2/cv3.2.2/Conv /model.23/Sigmoid_2
fuse_convolutiondepthwise_activation /model.23/cv3.0/cv3.0.0/cv3.0.0.0/conv/Conv /model.23/cv3.0/cv3.0.0/cv3.0.0.0/act/Relu
fuse_convolutiondepthwise_activation /model.23/cv3.0/cv3.0.1/cv3.0.1.0/conv/Conv /model.23/cv3.0/cv3.0.1/cv3.0.1.0/act/Relu
fuse_convolutiondepthwise_activation /model.23/cv3.1/cv3.1.0/cv3.1.0.0/conv/Conv /model.23/cv3.1/cv3.1.0/cv3.1.0.0/act/Relu
fuse_convolutiondepthwise_activation /model.23/cv3.1/cv3.1.1/cv3.1.1.0/conv/Conv /model.23/cv3.1/cv3.1.1/cv3.1.1.0/act/Relu
fuse_convolutiondepthwise_activation /model.23/cv3.2/cv3.2.0/cv3.2.0.0/conv/Conv /model.23/cv3.2/cv3.2.0/cv3.2.0.0/act/Relu
fuse_convolutiondepthwise_activation /model.23/cv3.2/cv3.2.1/cv3.2.1.0/conv/Conv /model.23/cv3.2/cv3.2.1/cv3.2.1.0/act/Relu
Input layer images without shape info, shape_inference skipped
Input layer images without shape info, estimate_memory_footprint skipped安卓代码的修改
参考这两个代码进行修改
https://github.com/gaoxumustwin/ncnn-android-yolov8-pose
https://github.com/zhouweigogogo/yolo11-ncnn
对于yolo11-ncnn有以下几个修改的地方
将softmax函数修改为了使用快速指数fast_exp的sigmoid将 cv::dnn::NMSBoxes 修改了使用纯C代码的实现
对于ncnn-android-yolov8-pose修改为ncnn-android-yolov11主要为将各种与yolov8pose相关的内容替换为yolov11
具体的代码过程有兴趣的可以去查看
本人技术水平不高代码肯定还有提升优化的地方
参考资料
https://github.com/gaoxumustwin/ncnn-android-yolov8-pose
https://github.com/zhouweigogogo/yolo11-ncnn
https://github.com/triple-Mu/ncnn-examples/blob/main/cpp/yolov8/src/triplemu-yolov8.cpp
https://zhuanlan.zhihu.com/p/769076635
https://blog.csdn.net/u012863603/article/details/142977809?ops_request_miscrequest_idbiz_id102utm_termyolov11%E7%9A%84%E8%BE%93%E5%87%BA%E6%98%AF%E4%BB%80%E4%B9%88utm_mediumdistribute.pc_search_result.none-task-blog-2allsobaiduweb~default-1-142977809.142v100pc_search_result_base2spm1018.2226.3001.4187
