opencv 解码记录
解码库使用的时候发现瑞芯微以前做过解码库对ffmpeg和gstreamer的支持
然后最近实在不想再调试Rtsp浪费时间了，就从这中间找了一个比较快的方案
ffmpeg 带硬解码库编译
编译流程参考文献
https://blog.csdn.net/T__zxt/article/details/123424359
编译的ffmpeg是这个版本
https://github.com/jjm2473
安装了下面这些东西：
sudo apt-get -y install \autoconf \automake \build-essential \cmake \git-core \libass-dev \libfreetype6-dev \libsdl2-dev \libtool \libva-dev \libvdpau-dev \libvorbis-dev \libxcb1-dev \libxcb-shm0-dev \libxcb-xfixes0-dev \pkg-config \texinfo \wget \zlib1g-dev
编译选项参考的是这哥们的语句：
https://www.reddit.com/r/OrangePI/comments/14f725v/comment/jpcezy2/?context=3
下面这句话，哪个不能用删哪个，一定要有的是 --enable-rkmpp
./configure --enable-nonfree --enable-gpl --enable-version3 --enable-libx264 --enable-libdrm --enable-rkmpp --enable-librtmp --enable-shared --enable-static --enable-librga --enable-libx265 --enable-libmp3lame --enable-libpulse --enable-openssl --enable-libopus --enable-libvorbis
# 我最后用的 应该是sudo apt-get -y install autoconf automake build-essential cmake git-core libass-dev libfreetype6-dev libsdl2-dev libtool libva-dev libvdpau-dev libvorbis-dev libxcb1-dev libxcb-shm0-dev libxcb-xfixes0-dev pkg-config texinfo wget zlib1g-dev
./configure --enable-nonfree --enable-gpl --enable-version3 --enable-libx264 --enable-libdrm --enable-rkmpp --enable-shared --enable-static --enable-librga --enable-libx265 --enable-libpulse --enable-openssl --enable-libvorbis使用板上编译的时候记得多等一会，有点慢，不要急
编译出来的结果需要使用下面的语句去看ffmpeg是否正常
ffmpeg -decoders |grep h264
如果编译之后，ffmpeg不支持，
修改
sudo vim /etc/ld.so.conf
include /etc/ld.so.conf.d/*.conf
/usr/local/lib
打开之前ldd看一眼是不是所有的链接库都找到地方了
ffmpeg-rk-enc-5.1$ ldd ffmpeglinux-vdso.so.1 (0x0000007fbb328000)libavdevice.so.59 => /usr/local/lib/libavdevice.so.59 (0x0000007fbb230000)libavfilter.so.8 => /usr/local/lib/libavfilter.so.8 (0x0000007fbae60000)libavformat.so.59 => /usr/local/lib/libavformat.so.59 (0x0000007fbabe0000)libavcodec.so.59 => /usr/local/lib/libavcodec.so.59 (0x0000007fb9ae0000)libpostproc.so.56 => /usr/local/lib/libpostproc.so.56 (0x0000007fb9ac0000)libswresample.so.4 => /usr/local/lib/libswresample.so.4 (0x0000007fb9a90000)libswscale.so.6 => /usr/local/lib/libswscale.so.6 (0x0000007fb9a10000)libavutil.so.57 => /usr/local/lib/libavutil.so.57 (0x0000007fb9830000)libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007fb9790000)libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007fb95e0000)/lib/ld-linux-aarch64.so.1 (0x0000007fbb2ef000)libdrm.so.2 => /lib/aarch64-linux-gnu/libdrm.so.2 (0x0000007fb95b0000)libxcb.so.1 => /lib/aarch64-linux-gnu/libxcb.so.1 (0x0000007fb9570000)libxcb-shm.so.0 => /lib/aarch64-linux-gnu/libxcb-shm.so.0 (0x0000007fb9550000)libxcb-shape.so.0 => /lib/aarch64-linux-gnu/libxcb-shape.so.0 (0x0000007fb9530000)libxcb-xfixes.so.0 => /lib/aarch64-linux-gnu/libxcb-xfixes.so.0 (0x0000007fb9510000)libasound.so.2 => /lib/aarch64-linux-gnu/libasound.so.2 (0x0000007fb93f0000)libpulse.so.0 => /lib/aarch64-linux-gnu/libpulse.so.0 (0x0000007fb9390000)libSDL2-2.0.so.0 => /lib/aarch64-linux-gnu/libSDL2-2.0.so.0 (0x0000007fb91f0000)libsndio.so.7 => /lib/aarch64-linux-gnu/libsndio.so.7 (0x0000007fb91c0000)libXv.so.1 => /lib/aarch64-linux-gnu/libXv.so.1 (0x0000007fb91a0000)libX11.so.6 => /lib/aarch64-linux-gnu/libX11.so.6 (0x0000007fb9050000)libXext.so.6 => /lib/aarch64-linux-gnu/libXext.so.6 (0x0000007fb9020000)libavrkmpp.so.1 => /usr/local/lib/libavrkmpp.so.1 (0x0000007fb9000000)libva.so.2 => /lib/aarch64-linux-gnu/libva.so.2 (0x0000007fb8fc0000)libz.so.1 => /lib/aarch64-linux-gnu/libz.so.1 (0x0000007fb8f90000)libssl.so.3 => /lib/aarch64-linux-gnu/libssl.so.3 (0x0000007fb8ee0000)libcrypto.so.3 => /lib/aarch64-linux-gnu/libcrypto.so.3 (0x0000007fb8af0000)liblzma.so.5 => /lib/aarch64-linux-gnu/liblzma.so.5 (0x0000007fb8ab0000)librockchip_mpp.so.1 => /lib/aarch64-linux-gnu/librockchip_mpp.so.1 (0x0000007fb88b0000)libvorbis.so.0 => /lib/aarch64-linux-gnu/libvorbis.so.0 (0x0000007fb8870000)libvorbisenc.so.2 => /lib/aarch64-linux-gnu/libvorbisenc.so.2 (0x0000007fb87b0000)libx264.so.163 => /lib/aarch64-linux-gnu/libx264.so.163 (0x0000007fb8560000)libx265.so.199 => /lib/aarch64-linux-gnu/libx265.so.199 (0x0000007fb82a0000)librga.so.2 => /lib/aarch64-linux-gnu/librga.so.2 (0x0000007fb8270000)libva-drm.so.2 => /lib/aarch64-linux-gnu/libva-drm.so.2 (0x0000007fb8250000)libva-x11.so.2 => /lib/aarch64-linux-gnu/libva-x11.so.2 (0x0000007fb8230000)libvdpau.so.1 => /lib/aarch64-linux-gnu/libvdpau.so.1 (0x0000007fb8210000)libXau.so.6 => /lib/aarch64-linux-gnu/libXau.so.6 (0x0000007fb81f0000)libXdmcp.so.6 => /lib/aarch64-linux-gnu/libXdmcp.so.6 (0x0000007fb81d0000)libpulsecommon-15.99.so => /usr/lib/aarch64-linux-gnu/pulseaudio/libpulsecommon-15.99.so (0x0000007fb8140000)libdbus-1.so.3 => /lib/aarch64-linux-gnu/libdbus-1.so.3 (0x0000007fb80e0000)libXcursor.so.1 => /lib/aarch64-linux-gnu/libXcursor.so.1 (0x0000007fb80c0000)libXinerama.so.1 => /lib/aarch64-linux-gnu/libXinerama.so.1 (0x0000007fb80a0000)libXi.so.6 => /lib/aarch64-linux-gnu/libXi.so.6 (0x0000007fb8070000)libXfixes.so.3 => /lib/aarch64-linux-gnu/libXfixes.so.3 (0x0000007fb8050000)libXrandr.so.2 => /lib/aarch64-linux-gnu/libXrandr.so.2 (0x0000007fb8030000)libXss.so.1 => /lib/aarch64-linux-gnu/libXss.so.1 (0x0000007fb8010000)libXxf86vm.so.1 => /lib/aarch64-linux-gnu/libXxf86vm.so.1 (0x0000007fb7ff0000)libgbm.so.1 => /lib/aarch64-linux-gnu/libgbm.so.1 (0x0000007fb7fd0000)libwayland-egl.so.1 => /lib/aarch64-linux-gnu/libwayland-egl.so.1 (0x0000007fb7fb0000)libwayland-client.so.0 => /lib/aarch64-linux-gnu/libwayland-client.so.0 (0x0000007fb7f90000)libwayland-cursor.so.0 => /lib/aarch64-linux-gnu/libwayland-cursor.so.0 (0x0000007fb7f70000)libxkbcommon.so.0 => /lib/aarch64-linux-gnu/libxkbcommon.so.0 (0x0000007fb7f10000)libdecor-0.so.0 => /lib/aarch64-linux-gnu/libdecor-0.so.0 (0x0000007fb7ef0000)libbsd.so.0 => /lib/aarch64-linux-gnu/libbsd.so.0 (0x0000007fb7ec0000)libstdc++.so.6 => /lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000007fb7c90000)libogg.so.0 => /lib/aarch64-linux-gnu/libogg.so.0 (0x0000007fb7c70000)libnuma.so.1 => /lib/aarch64-linux-gnu/libnuma.so.1 (0x0000007fb7c40000)libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000007fb7c10000)libsndfile.so.1 => /lib/aarch64-linux-gnu/libsndfile.so.1 (0x0000007fb7b80000)libX11-xcb.so.1 => /lib/aarch64-linux-gnu/libX11-xcb.so.1 (0x0000007fb7b60000)libsystemd.so.0 => /lib/aarch64-linux-gnu/libsystemd.so.0 (0x0000007fb7a80000)libasyncns.so.0 => /lib/aarch64-linux-gnu/libasyncns.so.0 (0x0000007fb7a60000)libapparmor.so.1 => /lib/aarch64-linux-gnu/libapparmor.so.1 (0x0000007fb7a30000)libXrender.so.1 => /lib/aarch64-linux-gnu/libXrender.so.1 (0x0000007fb7a10000)libwayland-server.so.0 => /lib/aarch64-linux-gnu/libwayland-server.so.0 (0x0000007fb79e0000)libexpat.so.1 => /lib/aarch64-linux-gnu/libexpat.so.1 (0x0000007fb79a0000)libffi.so.8 => /lib/aarch64-linux-gnu/libffi.so.8 (0x0000007fb7980000)libmd.so.0 => /lib/aarch64-linux-gnu/libmd.so.0 (0x0000007fb7960000)libFLAC.so.8 => /lib/aarch64-linux-gnu/libFLAC.so.8 (0x0000007fb7910000)libopus.so.0 => /lib/aarch64-linux-gnu/libopus.so.0 (0x0000007fb78a0000)libzstd.so.1 => /opt/ros/humble/lib/libzstd.so.1 (0x0000007fb7810000)liblz4.so.1 => /lib/aarch64-linux-gnu/liblz4.so.1 (0x0000007fb77e0000)libcap.so.2 => /lib/aarch64-linux-gnu/libcap.so.2 (0x0000007fb77c0000)libgcrypt.so.20 => /lib/aarch64-linux-gnu/libgcrypt.so.20 (0x0000007fb76d0000)libgpg-error.so.0 => /lib/aarch64-linux-gnu/libgpg-error.so.0 (0x0000007fb7690000)至此，编译好了一个支持硬件解码的ffmpeg，记得make install ，然后去编译opencv,让opencv支持ffmpeg
opencv编译带硬解码ffmpeg
原来的opencv是这样的：
OpenCV FFmpeg version:
General configuration for OpenCV 4.5.5 =====================================Version control: unknownPlatform:Timestamp: 2023-09-25T04:22:20ZHost: Linux 5.10.110-rockchip-rk3588 aarch64CMake: 3.22.1CMake generator: Unix MakefilesCMake build tool: /usr/bin/gmakeConfiguration: ReleaseCPU/HW features:Baseline: NEON FP16C/C++:Built as dynamic libs?: YESC++ standard: 11C++ Compiler: /usr/bin/c++ (ver 11.4.0)C++ flags (Release): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG -DNDEBUGC++ flags (Debug): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -fvisibility-inlines-hidden -g -O0 -DDEBUG -D_DEBUGC Compiler: /usr/bin/ccC flags (Release): -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -O3 -DNDEBUG -DNDEBUGC flags (Debug): -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -g -O0 -DDEBUG -D_DEBUGLinker flags (Release): -Wl,--gc-sections -Wl,--as-needed Linker flags (Debug): -Wl,--gc-sections -Wl,--as-needed ccache: NOPrecompiled headers: NOExtra dependencies: dl m pthread rt3rdparty dependencies:OpenCV modules:To be built: calib3d core dnn features2d flann gapi highgui imgcodecs imgproc ml objdetect photo python3 stitching ts video videoioDisabled: worldDisabled by dependency: -Unavailable: java python2Applications: tests perf_tests appsDocumentation: NONon-free algorithms: NOGUI: QT5QT: YES (ver 5.15.3 )QT OpenGL support: YES (Qt5::OpenGL 5.15.3)GTK+: YES (ver 3.24.33)GThread : YES (ver 2.72.4)GtkGlExt: NOOpenGL support: YES (/usr/lib/aarch64-linux-gnu/libGL.so /usr/lib/aarch64-linux-gnu/libGLU.so)VTK support: YES (ver 9.1.0)Media I/O: ZLib: /usr/lib/aarch64-linux-gnu/libz.so (ver 1.2.11)JPEG: /usr/lib/aarch64-linux-gnu/libjpeg.so (ver 80)WEBP: /usr/lib/aarch64-linux-gnu/libwebp.so (ver encoder: 0x020f)PNG: /lib/libpng.so (ver 1.6.37)TIFF: /usr/lib/aarch64-linux-gnu/libtiff.so (ver 42 / 4.3.0)JPEG 2000: OpenJPEG (ver 2.4.0)OpenEXR: /usr/lib/aarch64-linux-gnu/libImath-2_5.so /usr/lib/aarch64-linux-gnu/libIlmImf-2_5.so /usr/lib/aarch64-linux-gnu/libIex-2_5.so /usr/lib/aarch64-linux-gnu/libHalf-2_5.so /usr/lib/aarch64-linux-gnu/libIlmThread-2_5.so (ver 2_5)HDR: YESSUNRASTER: YESPXM: YESPFM: YESVideo I/O:DC1394: YES (2.2.6)FFMPEG: YESavcodec: YES (58.134.100)avformat: YES (58.76.100)avutil: YES (56.70.100)swscale: YES (5.9.100)avresample: NOGStreamer: YES (1.20.3)v4l/v4l2: YES (linux/videodev2.h)Parallel framework: pthreadsTrace: YES (with Intel ITT)Other third-party libraries:Lapack: NOEigen: YES (ver 3.4.0)Custom HAL: YES (carotene (ver 0.0.1))Protobuf: build (3.19.1)OpenCL: YES (no extra features)Include path: /home/orangepi/opencv-4.5.5/3rdparty/include/opencl/1.2Link libraries: Dynamic loadPython 3:Interpreter: /usr/bin/python3 (ver 3.10.12)Libraries: /usr/lib/aarch64-linux-gnu/libpython3.10.so (ver 3.10.12)numpy: /usr/lib/python3/dist-packages/numpy/core/include (ver 1.21.5)install path: lib/python3.10/dist-packages/cv2/python-3.10Python (for build): /usr/bin/python2.7Java: ant: NOJNI: /usr/lib/jvm/default-java/include /usr/lib/jvm/default-java/include/linux /usr/lib/jvm/default-java/includeJava wrappers: NOJava tests: NOInstall to: /usr/local
opencv 4.5.5来讲，这个ffmpeg太新了，编译不过。只能以毒攻毒，去下了opencv4.8.0
回头把这个打包上传到csdn资源，防止它小更新不能用
4.8.0是编译通过的
一样也是板端编译，cmake-gui配置,然后去build文件夹下面 make make install
参考文献：
https://blog.csdn.net/Flag_ing/article/details/109508374
注意 ，如果使用qt，一定要把qt勾选，否则在qt里运行会报错
还有opencl他也不是自动勾选的
其他的应该随意
然后去创建工程使用、测试opencv是不是能正常用
测试代码
// RTSP Server#include "xop/RtspServer.h"
#include "net/Timer.h"
#include <thread>
#include <memory>
#include <iostream>
#include <string>#include <opencv2/opencv.hpp>
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>// 编码库
#include <encoder.h>
// 解码库
#include <decoder.h>
// 海康拉流库
#include "HCNetSDK.h"
#include <chrono>
#include "yolo.h"
// 查看264 二进制 文件
#include <sstream>
#include <fstream>using namespace std;
using namespace cv;
// 定义一个缓冲池结构体
struct Buffer {Mat buf1;Mat buf2;
};// 定义互斥锁
std::mutex mymutex;
Buffer matBuf;
VideoCapture videocapture;
int captureOpen = 0;
/*** @brief 读图* * @return Mat */
Mat captureThread()
{string rtspIN = "rtsp://admin:s745112822@192.168.2.110:554/h264/ch0/main/av_stream";if (captureOpen == 0 ){ videocapture.open(rtspIN,CAP_FFMPEG);captureOpen = 1;//打开图像// capture.open(url);}while (1){//加锁mymutex.lock();// buf2->buf1if(matBuf.buf1.empty())videocapture.read(matBuf.buf1); else if(matBuf.buf2.empty())videocapture.read(matBuf.buf2);else{matBuf.buf1 = matBuf.buf2;videocapture.read(matBuf.buf2);}// 没有读到数据 网络问题 重启rtspif(matBuf.buf1.empty()){videocapture.release();videocapture.open(rtspIN,CAP_FFMPEG);}//读图成功// imgready = true;//解锁mymutex.unlock();xop::Timer::Sleep(10);}
}void writeCharPointerToFile(const char* data, std::size_t size, const std::string& filename) {// 打开文件std::ofstream file(filename, ios::binary);// 检查文件是否成功打开if (!file.is_open()) {std::cerr << "无法打开文件: " << filename << std::endl;return;}// 写入数据file.write(data, size);// 关闭文件file.close();
}bool IsKeyFrame(const char* data, uint32_t size)
{if (size > 4) {//0x67:sps ,0x65:IDR, 0x6: SEIif (data[4] == 0x67 || data[4] == 0x65 || data[4] == 0x6 || data[4] == 0x27) {return true;}}return false;
}
// encoder e ;void SendFrameThread(xop::RtspServer* rtsp_server, xop::MediaSessionId session_id, int& clients);int main(int argc, char **argv)
{ // 读图线程 thread t1(captureThread);int clients = 1;std::string ip = "0.0.0.0";std::string rtsp_url = "rtsp://127.0.0.1:5514/live";// e.init();std::shared_ptr<xop::EventLoop> event_loop(new xop::EventLoop()); std::shared_ptr<xop::RtspServer> server = xop::RtspServer::Create(event_loop.get());if (!server->Start(ip, 5514)) {return -1;}#ifdef AUTH_CONFIGserver->SetAuthConfig("-_-", "admin", "12345");
#endifxop::MediaSession *session = xop::MediaSession::CreateNew("live"); // url: rtsp://ip/livesession->AddSource(xop::channel_0, xop::H264Source::CreateNew()); session->AddSource(xop::channel_1, xop::AACSource::CreateNew(44100,2));// session->startMulticast(); /* 开启组播(ip,端口随机生成), 默认使用 RTP_OVER_UDP, RTP_OVER_RTSP */session->AddNotifyConnectedCallback([] (xop::MediaSessionId sessionId, std::string peer_ip, uint16_t peer_port){printf("RTSP client connect, ip=%s, port=%hu \n", peer_ip.c_str(), peer_port);});session->AddNotifyDisconnectedCallback([](xop::MediaSessionId sessionId, std::string peer_ip, uint16_t peer_port) {printf("RTSP client disconnect, ip=%s, port=%hu \n", peer_ip.c_str(), peer_port);});std::cout << "URL: " << rtsp_url << std::endl;xop::MediaSessionId session_id = server->AddSession(session); //server->removeMeidaSession(session_id); /* 取消会话, 接口线程安全 */std::thread thread(SendFrameThread, server.get(), session_id, std::ref(clients));thread.detach();while(1) {xop::Timer::Sleep(100);}getchar();return 0;
}void SendFrameThread(xop::RtspServer* rtsp_server, xop::MediaSessionId session_id, int& clients)
{ encoder e;// decoder d;// cv::VideoCapture *videocapture = new cv::VideoCapture(0);// std::string rtspIN = "rtsp://admin:s745112822@192.168.2.110:554/h264/ch1/sub/av_stream";// yolo处理// char* modelname = "/home/orangepi/code/realsense/cap2/yolov5s-640-640.rknn";// yolo yolo5;// yolo5.load_model(modelname);// yolo5.model_init();
// decoder相关 结果回传// d.init();std::vector<cv::Mat> resultArray;int lenofmat = 0;
// encoder相关 内存拷贝int size = 0;char* buffer ;cv::Mat colorBar;int i = 0;while(1){if(clients > 0) /* 会话有客户端在线, 发送音视频数据 */{{ xop::AVFrame videoFrame = {0};// opencv 拿图 后续需要优化auto start_time = std::chrono::high_resolution_clock::now(); // videocapture->read(colorBar); // 拿图 while(matBuf.buf1.empty());mymutex.lock();colorBar = matBuf.buf1;matBuf.buf1 = matBuf.buf2;matBuf.buf2.release();mymutex.unlock();auto end_time1 = std::chrono::high_resolution_clock::now();// d.poststream("264stream pointer",lenofmat,resultArray);// printf("width is %d, height is %d",colorBar.rows,colorBar.cols);// yolo 处理// yolo5.load_Mat(colorBar);// yolo5.run();// colorBar = yolo5.getPicResult();auto end_time2 = std::chrono::high_resolution_clock::now();if(0==i){// 第一帧有sps信息 给他两帧拼一起char *buffer1;int size1;e.init(buffer1,size1);videoFrame.size = size1;e.postAframe(colorBar,buffer,size);videoFrame.size += size; videoFrame.buffer.reset(new uint8_t[videoFrame.size]);memcpy(videoFrame.buffer.get(), buffer1, size1);memcpy(videoFrame.buffer.get()+size1, buffer, size);i++;} else{e.postAframe(colorBar,buffer,size);videoFrame.size = size; // 视频帧大小 videoFrame.buffer.reset(new uint8_t[videoFrame.size]);memcpy(videoFrame.buffer.get(), buffer, videoFrame.size);}auto end_time3 = std::chrono::high_resolution_clock::now();auto duration = std::chrono::duration_cast<std::chrono::milliseconds>(end_time1 - start_time);cout<<"===============================A==frame================================================="<<endl;cout << "decode mat took: " << duration.count() <<"ms\r\n";duration = std::chrono::duration_cast<std::chrono::milliseconds>(end_time2 - end_time1);cout << "rknn took: " << duration.count() <<"ms\r\n";duration = std::chrono::duration_cast<std::chrono::milliseconds>(end_time3 - end_time2);cout << "encode took: " << duration.count() <<"ms\r\n";videoFrame.type = IsKeyFrame(buffer, size) ? xop::VIDEO_FRAME_I : xop::VIDEO_FRAME_P;// videoFrame.type = 0; // 建议确定帧类型。I帧(xop::VIDEO_FRAME_I) P帧(xop::VIDEO_FRAME_P)videoFrame.timestamp = xop::H264Source::GetTimestamp(); // 时间戳, 建议使用编码器提供的时间戳// writeCharPointerToFile((char *)videoFrame.buffer.get(), videoFrame.size, "filename.txt"); rtsp_server->PushFrame(session_id, xop::channel_0, videoFrame); //送到服务器进行转发, 接口线程安全/*//获取一帧 H264, 打包xop::AVFrame videoFrame = {0};videoFrame.type = 0; // 建议确定帧类型。I帧(xop::VIDEO_FRAME_I) P帧(xop::VIDEO_FRAME_P)videoFrame.size = video frame size; // 视频帧大小 videoFrame.timestamp = xop::H264Source::GetTimestamp(); // 时间戳, 建议使用编码器提供的时间戳videoFrame.buffer.reset(new uint8_t[videoFrame.size]); memcpy(videoFrame.buffer.get(), video frame data, videoFrame.size); rtsp_server->PushFrame(session_id, xop::channel_0, videoFrame); //送到服务器进行转发, 接口线程安全*/}{ /*//获取一帧 AAC, 打包xop::AVFrame audioFrame = {0};audioFrame.type = xop::AUDIO_FRAME;audioFrame.size = audio frame size; /* 音频帧大小 audioFrame.timestamp = xop::AACSource::GetTimestamp(44100); // 时间戳audioFrame.buffer.reset(new uint8_t[audioFrame.size]); memcpy(audioFrame.buffer.get(), audio frame data, audioFrame.size);rtsp_server->PushFrame(session_id, xop::channel_1, audioFrame); // 送到服务器进行转发, 接口线程安全*/} }// xop::Timer::Sleep(20); /* 实际使用需要根据帧率计算延时! */}videocapture.release();e.deinit((MPP_RET)0);// d.deinit();// yolo5.deinit();}
