linux下 yolov8 tensorrt模型部署

TensorRT系列之 Windows10下yolov8 tensorrt模型加速部署TensorRT系列之 Linux下 yolov8 tensorrt模型加速部署TensorRT系列之 Linux下 yolov7 tensorrt模型加速部署TensorRT系列之 Linux下 yolov6 tensorrt模型加速部署TensorRT系列之 Linux下 yolov5 tensorrt模型加速部署TensorRT系列之 Linux下 yolox tensorrt模型加速部署TensorRT系列之 Linux下 u2net tensorrt模型加速部署更多点我进去…文章目录ubuntu下yolov8 tensorrt模型加速部署【实战】一、加速结果展示1.1 性能速览1.2精度对齐二、Ubuntu18.04环境配置2.1 安装工具链和opencv2.2 安装Nvidia相关库2.2.1 安装Nvidia显卡驱动2.2.2 安装 cuda11.32.2.3 安装 cudnn8.22.2.4 下载 tensorrt8.4.2.42.2.5 下载仓库TensorRT-Alpha并设置三、YOLOv8模型部署3.1 获取YOLOv8onnx文件3.2 编译 onnx3.3 编译运行四、如何部署自己训练的yolov8模型五、参考ubuntu下yolov8 tensorrt模型加速部署【实战】TensorRT-Alpha基于tensorrtcuda c实现模型end2end的gpu加速支持win10、linux在2023年已经更新模型YOLOv8, YOLOv7, YOLOv6, YOLOv5, YOLOv4, YOLOv3, YOLOX, YOLOR,pphumanseg,u2net,EfficientDet。仓库TensorRT-Alphahttps://github.com/FeiYull/TensorRT-Alpha/tree/legacy一、加速结果展示1.1 性能速览快速看看yolov8n 在移动端RTX2070m(8G)的新能表现modelvideo resolutionmodel input sizeGPU Memory-UsageGPU-Utilyolov8n1920x10808x3x640x6401093MiB/7982MiB14%下图是yolov8n的运行时间开销单位是ms更多TensorRT-Alpha测试录像在B站视频B站YOLOv8nB站YOLOv8s1.2精度对齐下面是左边是python框架推理结果右边是TensorRT-Alpha推理结果。yolov8n : Offical( left ) vs Ours( right )yolov7-tiny : Offical( left ) vs Ours( right )yolov6s : Offical( left ) vs Ours( right )yolov5s : Offical( left ) vs Ours( right )YOLOv4 YOLOv3 YOLOR YOLOX略。二、Ubuntu18.04环境配置如果您对tensorrt不是很熟悉请务必保持下面库版本一致。请注意: Linux系统安装以下库务必去进入系统bios下关闭安全启动(设置 secure boot 为 disable)2.1 安装工具链和opencvsudoapt-getupdatesudoapt-getinstallbuild-essentialsudoapt-getinstallgitsudoapt-getinstallgdbsudoapt-getinstallcmakesudoapt-getinstalllibopencv-dev# pkg-config --modversion opencv2.2 安装Nvidia相关库注Nvidia相关网站需要注册账号。2.2.1 安装Nvidia显卡驱动ubuntu-drivers devicessudoadd-apt-repository ppa:graphics-drivers/ppasudoaptupdatesudoaptinstallnvidia-driver-470-server# for ubuntu18.04nvidia-smi2.2.2 安装 cuda11.3进入链接: https://developer.nvidia.com/cuda-toolkit-archive选择CUDA Toolkit 11.3.0(April 2021)选择[Linux] - [x86_64] - [Ubuntu] - [18.04] - [runfile(local)]在网页你能看到下面安装命令我这里已经拷贝下来wgethttps://developer.download.nvidia.com/compute/cuda/11.3.0/local_installers/cuda_11.3.0_465.19.01_linux.runsudoshcuda_11.3.0_465.19.01_linux.runcuda的安装过程中需要你在bash窗口手动作一些选择这里选择如下select[continue] - [accept] - 接着按下回车键取消Driver和465.19.01这个选项如下图(it is important!) - [Install]bash窗口提示如下表示安装完成## Summary ##Driver: Not Selected#Toolkit: Installed in /usr/local/cuda-11.3/#......把cuda添加到环境变量vim~/.bashrc把下面拷贝到 .bashrc里面# cuda v11.3exportPATH/usr/local/cuda-11.3/bin${PATH::${PATH}}exportLD_LIBRARY_PATH/usr/local/cuda-11.3/lib64${LD_LIBRARY_PATH::${LD_LIBRARY_PATH}}exportCUDA_HOME/usr/local/cuda-11.3刷新环境变量和验证source~/.bashrc nvcc-Vbash窗口打印如下信息表示cuda11.3安装正常nvcc: NVIDIA(R)Cuda compiler driverbrCopyright(c)2005-2021 NVIDIA CorporationbrBuilt on Sun_Mar_21_19:15:46_PDT_2021brCuda compilation tools, release11.3, V11.3.58brBuild cuda_11.3.r11.3/compiler.29745058_0br2.2.3 安装 cudnn8.2进入网站https://developer.nvidia.com/rdp/cudnn-archive选择 Download cuDNN v8.2.0 (April 23rd, 2021), for CUDA 11.x选择 cuDNN Library for Linux (x86_64)你将会下载这个压缩包: “cudnn-11.3-linux-x64-v8.2.0.53.tgz”# 解压tar-zxvfcudnn-11.3-linux-x64-v8.2.0.53.tgz将cudnn的头文件和lib拷贝到cuda11.3的安装目录下sudocpcuda/include/cudnn.h /usr/local/cuda/include/sudocpcuda/lib64/libcudnn* /usr/local/cuda/lib64/sudochmodar /usr/local/cuda/include/cudnn.hsudochmodar /usr/local/cuda/lib64/libcudnn*2.2.4 下载 tensorrt8.4.2.4本教程中tensorrt只需要下载\、解压即可不需要安装。进入网站https://developer.nvidia.cn/nvidia-tensorrt-8x-download网站更新2023.12https://developer.nvidia.com/nvidia-tensorrt-8x-download顺便法克 Nvidia把这个打勾 I Agree To the Terms of the NVIDIA TensorRT License Agreement选择: TensorRT 8.4 GA Update 1选择: TensorRT 8.4 GA Update 1 for Linux x86_64 and CUDA 11.0, 11.1, 11.2, 11.3, 11.4, 11.5, 11.6 and 11.7 TAR Package你将会下载这个压缩包: “TensorRT-8.4.2.4.Linux.x86_64-gnu.cuda-11.6.cudnn8.4.tar.gz”# 解压tar-zxvfTensorRT-8.4.2.4.Linux.x86_64-gnu.cuda-11.6.cudnn8.4.tar.gz# 快速验证一下tensorrtcudacudnn是否安装正常cdTensorRT-8.4.2.4/samples/sampleMNISTmakecd../../bin/导出tensorrt环境变量(it is important!)注将LD_LIBRARY_PATH:后面的路径换成你自己的后续编译onnx模型的时候也需要执行下面第一行命令exportLD_LIBRARY_PATH$LD_LIBRARY_PATH:/home/xxx/temp/TensorRT-8.4.2.4/lib ./sample_mnistbash窗口打印类似如下图的手写数字识别表明cudacudnntensorrt安装正常2.2.5 下载仓库TensorRT-Alpha并设置gitclone https://github.com/FeiYull/tensorrt-alphagitfetch origingitcheckout legacy设置您自己TensorRT根目录:gitclone https://github.com/FeiYull/tensorrt-alphagitfetch origingitcheckout legacycdtensorrt-alpha/cmakevimcommon.cmake# 在文件common.cmake中的第20行中设置成你自己的目录别和我设置一样的路径eg:# set(TensorRT_ROOT /root/TensorRT-8.4.2.4)三、YOLOv8模型部署3.1 获取YOLOv8onnx文件直接在网盘下载 weiyun or google driver 或者使用如下命令导出onnx:# yolov8 官方仓库: https://github.com/ultralytics/ultralytics# yolov8 官方教程: https://docs.ultralytics.com/quickstart/# TensorRT-Alpha will be updated synchronously as soon as possible!# 安装 yolov8conda create-nyolov8python3.8-yconda activate yolov8 pipinstallultralytics8.0.5 pipinstallonnx1.12.0# 下载官方权重(.pt file)https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8n.pt https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8s.pt https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8m.pt https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8l.pt https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8x.pt https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8x6.pt导出 onnx:# 640yolomodeexportmodelyolov8n.ptformatonnxdynamicTrueopset12#simplifyTrueyolomodeexportmodelyolov8s.ptformatonnxdynamicTrueopset12#simplifyTrueyolomodeexportmodelyolov8m.ptformatonnxdynamicTrueopset12#simplifyTrueyolomodeexportmodelyolov8l.ptformatonnxdynamicTrueopset12#simplifyTrueyolomodeexportmodelyolov8x.ptformatonnxdynamicTrueopset12#simplifyTrue# 1280yolomodeexportmodelyolov8x6.ptformatonnxdynamicTrueopset12#simplifyTrue新增更新如何使用代码导出onnx文件。在yolov8官方源码目录下新建my_export.py文件如下代码yaml是官方提供的配置文件如果是自己训练的模型那就设置为自定义yaml文件即可。fromultralyticsimportYOLO# Load a modelmodelYOLO(ultralytics/cfg/models/v8/yolov8.yaml)modelYOLO(yolov8n.pt)# load an official model# Export the modelmodel.export(formatonnx,dynamicTrue,opset12)# 第2、3个参数禁止修改3.2 编译 onnx# 把你的onnx文件放到这个路径:tensorrt-alpha/data/yolov8cdtensorrt-alpha/data/yolov8# 请把LD_LIBRARY_PATH:换成您自己的路径。exportLD_LIBRARY_PATH$LD_LIBRARY_PATH:~/TensorRT-8.4.2.4/lib# 640../../../../TensorRT-8.4.2.4/bin/trtexec--onnxyolov8n.onnx--saveEngineyolov8n.trt--buildOnly--minShapesimages:1x3x640x640--optShapesimages:4x3x640x640--maxShapesimages:8x3x640x640../../../../TensorRT-8.4.2.4/bin/trtexec--onnxyolov8s.onnx--saveEngineyolov8s.trt--buildOnly--minShapesimages:1x3x640x640--optShapesimages:4x3x640x640--maxShapesimages:8x3x640x640../../../../TensorRT-8.4.2.4/bin/trtexec--onnxyolov8m.onnx--saveEngineyolov8m.trt--buildOnly--minShapesimages:1x3x640x640--optShapesimages:4x3x640x640--maxShapesimages:8x3x640x640../../../../TensorRT-8.4.2.4/bin/trtexec--onnxyolov8l.onnx--saveEngineyolov8l.trt--buildOnly--minShapesimages:1x3x640x640--optShapesimages:4x3x640x640--maxShapesimages:8x3x640x640../../../../TensorRT-8.4.2.4/bin/trtexec--onnxyolov8x.onnx--saveEngineyolov8x.trt--buildOnly--minShapesimages:1x3x640x640--optShapesimages:4x3x640x640--maxShapesimages:8x3x640x640# 1280../../../../TensorRT-8.4.2.4/bin/trtexec--onnxyolov8x6.onnx--saveEngineyolov8x6.trt--buildOnly--minShapesimages:1x3x1280x1280--optShapesimages:4x3x1280x1280--maxShapesimages:8x3x1280x1280你将会的到例如yolov8n.trt、yolov8s.trt、yolov8m.trt等文件。3.3 编译运行gitclone https://github.com/FeiYull/tensorrt-alphagitfetch origingitcheckout legacycdtensorrt-alpha/yolov8mkdirbuildcdbuild cmake..make-j10# 注: 效果图默认保存在路径 tensorrt-alpha/yolov8/build# 下面参数解释# --show 表示可视化结果# --savePath 表示保存默认保存在build目录# --savePath../ 保存在上一级目录## 640# 推理图片./app_yolov8--model../../data/yolov8/yolov8n.trt--size640--batch_size1--img../../data/6406407.jpg--show--savePath./app_yolov8--model../../data/yolov8/yolov8n.trt--size640--batch_size4--video../../data/people.mp4--show--savePath# 推理视频./app_yolov8--model../../data/yolov8/yolov8n.trt--size640--batch_size4--video../../data/people.mp4--show--savePath../# 在线推理相机视频./app_yolov8--model../../data/yolov8/yolov8n.trt--size640--batch_size2--cam_id0--show## 1280# infer camera./app_yolov8--model../../data/yolov8/yolov8x6.trt--size1280--batch_size2--cam_id0--showyolov8 tensorrt cuda模型推理加速部署TensorRT-Alpha《ski facility》yolov8 tensorrt cuda模型推理加速部署TensorRT-Alpha《NewYork-Stree》yolov7 tensorrt cuda模型推理加速部署TensorRT-Alpha《Korea-Night》四、如何部署自己训练的yolov8模型未必避免文章太长直接给出演示视频yolov8 tensorrt 部署自己训练的模型五、参考https://github.com/FeiYull/TensorRT-Alpha