目标追踪

注：所有模型转换都是在docker环境中的

先进入docker

这里我们是要在docker环境里编译的，所以先进入docker

:~/tpu-nntc# docker run -v $PWD/:/workspace  -it sophgo/tpuc_dev:latest

初始化环境

root@2bb02a2e27d5:/workspace/tpu-nntc# source ./scripts/envsetup.sh

docker里安装编译器

root@2bb02a2e27d5:/workspace/sophon-demo/sample/YOLOv5/cpp/yolov5_bmcv/build# sudo apt-get install  gcc-aarch64-linux-gnu g++-aarch64-linux-gnu libeigen3-dev

本C++例程依赖Eigen，您需要在编译c++程序的机器上运行如下命令安装：

sudo apt install libeigen3-dev

先下载相关文件，主要是追踪的测试视频，测试图片，目标追踪的权重，目标检测的权重

# 安装unzip，若已安装请跳过
sudo apt install unzip
chmod -R +x scripts/
./scripts/download.sh

然后编译c++代码

/workspace/sophon-demo/sample/DeepSORT/cpp/deepsort_bmcv/build# 
cd cpp/deepsort_bmcv
mkdir build && cd build
# 请根据实际情况修改-DSDK的路径，需使用绝对路径。
cmake -DTARGET_ARCH=soc -DSDK=/workspace/soc-sdk ..  
make

这时会生成deepsort_bmcv.soc文件，复制到盒子里

:/workspace/sophon-demo/sample/DeepSORT/cpp/deepsort_bmcv# scp -r  deepsort_bmcv.soc linaro@192.168.17.125:/data/yolo/sophon-demo/sample/DeepSORT/cpp

测试视频

./deepsort_bmcv.soc --input=rtsp://admin:sangfor@123@192.168.17.253 --bmodel_detector=../../BM1684/yolov5s_v6.1_3output_int8_1b.bmodel --bmodel_extractor=../../BM1684/extractor_fp32_1b.bmodel --dev_id=0

运行相关代码，这个是检测图片的

cd python
python3 deepsort_opencv.py --input ../datasets/mot15_trainset/ADL-Rundle-6/img1 --bmodel_detector ../models/BM1684/yolov5s_v6.1_3output_int8_1b.bmodel --bmodel_extractor ../models/BM1684/extractor_fp32_1b.bmodel --dev_id=0

对视频追踪

python3 deepsort_opencv.py --input ../datasets/test_car_person_1080P.mp4 --bmodel_detector ../models/BM1684/yolov5s_v6.1_3output_int8_1b.bmodel --bmodel_extractor ../models/BM1684/extractor_fp32_1b.bmodel --dev_id=0

对本地摄像头视频追踪

python3 deepsort_opencv.py --input rtsp://admin:sangfor@123@192.168.17.253 --bmodel_detector ../models/BM1684/yolov5s_v6.1_3output_int8_1b.bmodel --bmodel_extractor ../models/BM1684/extractor_fp32_1b.bmodel --dev_id=0

人体姿态估计

python3 python/openpose_opencv.py --input rtsp://admin:sangfor@123@192.168.17.253 --bmodel models/BM1684/pose_coco_fp32_1b.bmodel --dev_id 0

---

生成的文件会放在sample/YOLOv5/data/models/BM1684/int8model/anquanmao_batch1里

:~/fugui/sophon-demo_20221027_181652/sophon-demo_v0.1.0_b909566_20221027/sample/YOLOv5/data/models/BM1684/int8model/anquanmao_batch1# ls
compilation.bmodel  input_ref_data.dat  io_info.dat  output_ref_data.dat

然后将转换好的模型推送到开发板

scp compilation.bmodel linaro@{开发板ip地址}:/data/{你的yolov5存放路径}

开发板环境配置

搭建 libsophon 环境

cd libsophon_<date>_<hash>
# 安装依赖库，只需要执行一次
sudo apt install dkms libncurses5
sudo dpkg -i sophon-*.deb
# 在终端执行如下命令，或者log out再log in当前用户后即可使用bm-smi等命令
source /etc/profile

python3 yolov5_new_1.py --input rtsp://admin:1111111a@192.168.16.223 --bmodel yolov5s_v6.1_3output_fp32_1b.bmodel

c++编译环境

安装libsophon

进入sophon-img_20221027_215835这个路径

解压里面的tar包

:~/fugui/sophon-img_20221027_215835# tar -zxvf libsophon_soc_0.4.2_aarch64.tar.gz

将相关库目录和头文件目录拷贝到soc-sdk文件夹中

:~/fugui/sophon-img_20221027_215835/libsophon_soc_0.4.2_aarch64/opt/sophon/libsophon-0.4.2# sudo cp -rf include lib ~/fugui/soc-sdk

安装sophon-opencv 和sophon-ffmpeg

先进入sophon-mw，解压sophon-mw-soc_0.4.0_aarch64.tar.gz这个tar包

:~/fugui/sophon-mw_20221027_183429# tar -zxvf sophon-mw-soc_0.4.0_aarch64.tar.gz

复制相关文件到soc-sdk

:~/fugui/sophon-mw_20221027_183429/sophon-mw-soc_0.4.0_aarch64/opt/sophon# cp -rf sophon-ffmpeg_0.4.0//lib sophon-ffmpeg_0.4.0/include/ ~/fugui/soc-sdk:~/fugui/sophon-mw_20221027_183429/sophon-mw-soc_0.4.0_aarch64/opt/sophon# cp -rf sophon-opencv_0.4.0//lib sophon-opencv_0.4.0/include/ ~/fugui/soc-sdk

很简单，复制过去，交叉编译的环境就搭建好了

TPU-NNTC环境

这里我们是要在docker环境里编译的，所以先进入docker

:~/fugui# docker run -v $PWD/:/workspace  -it sophgo/tpuc_dev:latest

然后进入tpu-nntc，初始化环境

root@2bb02a2e27d5:/workspace/tpu-nntc# source ./scripts/envsetup.sh

docker里安装编译器

root@2bb02a2e27d5:/workspace/sophon-demo/sample/YOLOv5/cpp/yolov5_bmcv/build# sudo apt-get install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu

进入sophon-demo路径

下载相关文件

:~/fugui/sophon-demo/sample/YOLOv5# chmod -R +x scripts/
:~/fugui/sophon-demo/sample/YOLOv5# ./scripts/download.sh

编译yolov5

我们这里只是交叉编译，不能在x86设备上运行，要复制到我们1684平台

先cmake

root@2bb02a2e27d5:/workspace/sophon-demo/sample/YOLOv5/cpp/yolov5_bmcv/build# cmake -DTARGET_ARCH=soc -DSDK=/workspace/soc-sdk ..

在make

root@2bb02a2e27d5:/workspace/sophon-demo/sample/YOLOv5/cpp/yolov5_bmcv/build# make

此时会出现.soc文件

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-eWPar5Yp-1692844732110)(https://gitee.com/lizheng0219/picgo_img/raw/master/img202325/image-20230421134631891.png)]

把输出的文件传导我们开发板上运行下

scp -r yolov5_bmcv linaro@192.168.17.125:/data/sophon-demo/sample/YOLOv5/cpp/

运行推理图片

./yolov5_bmcv.soc --input=../../coco128 --bmodel=../../python/yolov5s_v6.1_3output_fp32_1b.bmodel --dev_id=0 --conf_thresh=0.5 --nms_thresh=0.5 --classnames=../../coco.names 

推理视频

./yolov5_bmcv.soc --input=../../test.avi --bmodel=../../python/yolov5s_v6.1_3output_fp32_1b.bmodel --dev_id=0 --conf_thresh=0.5 --nms_thresh=0.5 --classnames=../../coco.names 

c++推理网络摄像头

./yolov5_bmcv.soc --input=rtsp://admin:sangfor@123@192.168.17.253 --bmodel=/data/ai_box/yolov5s_640_coco_v6.1_3output_int8_1b_BM1684.bmodel --dev_id=0 --conf_thresh=0.5 --nms_thresh=0.5 --classnames=../../coco.names

./yolov5_bmcv.soc --input=rtsp://admin:sangfor@123@192.168.17.253 --bmodel=/data/models/all16_v6.1_3output_int8_4b.bmodel --dev_id=0 --conf_thresh=0.5 --nms_thresh=0.5 --classnames=/data/models/all16.names

网络摄像头：安全帽

 ./yolov5_bmcv.soc  --bmodel=anquanmao.bmodel --dev_id=0 --conf_thresh=0.5 --nms_thresh=0.5 --classnames=../../coco.names 

Python推理

python3 yolov5_opencv.py --input rtsp://admin:1111111a@192.168.16.222  --bmodel ../yolov5s_v6.1_3output_int8_4b.bmodel

前端只展示一路摄像头，我们只需要做一路摄像头使用多个算法推理。

不展示的摄像头也要实时在后台推理，有出现问题时要及时报警。

这样我们需做出单路摄像头推理多算法（单摄像头单算法也行，把所有检测都放到一个模型里，输出时只输出他选择的那个）

把所有模型统一训练比较简单，后台一块推理

sophon-pipeline

本地编译

docker run -v $PWD/:/workspace -p 8001:8001 -it sophgo/tpuc_dev:latest
source scripts/envsetup.sh
sudo apt-get install -y  gcc-aarch64-linux-gnu g++-aarch64-linux-gnu libeigen3-dev

./tools/compile.sh soc /workspace/soc-sdk

开发板运行

linaro@bm1684:/data/sophon-pipeline/release/video_stitch_demo$ ./soc/video_stitch_demo --config=cameras_video_stitch1.json

英码

export PYTHONPATH=$PYTHONPATH:/system/libexport
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/system/lib/

python

pip3 install sophon_arm-master-py3-none-any.whl --force-reinstall 
pip3 install opencv-python-headless<4.3

---

开发板执行命令

python3 python/yolov5_opencv.py --input ../data/images/coco200/000000009772.jpg  --model  ../compilation.bmodel --dev_id 0 --conf_thresh 0.5 --nms_thresh 0.5 

python3 python/yolov5_opencv.py --input ../data/xiyanimg/000017.jpg  --model  ../compilation.bmodel --dev_id 0 --conf_thresh 0.5 --nms_thresh 0.5

python3 python/yolov5_video.py --input rtsp://admin:sangfor@123@192.168.17.253 --model  ../compilation.bmodel

python3 python/yolov5_video.py --input rtsp://admin:1111111a@192.168.16.222  --model  ../compilation.bmodel --dev_id 0 --conf_thresh 0.5 --nms_thresh 0.5 

tar -zxf ~/Release_221201-public/sophon-mw_20221227_040823/sophon-mw-soc_*_aarch64.tar.gz