live555关于RTSP协议交互流程
live555的核心数据结构值之闭环双向链表
live555 rtsp服务器实战之createNewStreamSource
live555 rtsp服务器实战之doGetNextFrame
注意:该篇文章可能有些绕,最好跟着文章追踪下源码,不了解源码可能就是天书;
概要
live555用于实际项目开发时,createNewStreamSource和doGetNextFrame是必须要实现的两个虚函数,一般会创建两个类来实现这两个函数:假如这两个类为H264LiveVideoServerMediaSubssion和H264FramedLiveSource;H264LiveVideoServerMediaSubssion为实时视频会话类,用于实现createNewStreamSource虚函数;
H264FramedLiveSource为实时视频帧资源类,用户实现doGetNextFrame函数;
那么这两个函数是什么时候被调用以及他们的作用是什么呢?本节将详细介绍;
声明:该文章基于H264视频源为基础分析,其他源类似;
由于这两个类主要是自定义虚函数,所以存在一定的继承关系,首先需要明确这两个类的继承关系(本章只介绍H264FramedLiveSource):
H264FramedLiveSource:FramedSource:MediaSource:Medium
doGetNextFrame调用流程
doGetNextFrame函数声明于FrameSource类:
virtual void doGetNextFrame() = 0;
该声明为纯虚函数,所以必须有子类对该函数进行实现,H264FramedLiveSource类就是用于实现该函数;doGetNextFrame函数只有一个功能:获取视频帧;
void H264FramedLiveSource::doGetNextFrame()
{uint8_t *frame = new uint8_t[1024*1024];int len = 0;//获取一帧视频帧get_frame(&frame, len);//将帧长度赋值给父类的成员变量fFrameSizefFrameSize = len;//将帧数据赋值给父类的成员变量fTomemcpy(fTo, frame, len);delete [] frame;// nextTask() = envir().taskScheduler().scheduleDelayedTask(0,(TaskFunc*)FramedSource::afterGetting, this);//表示延迟0秒后再执行 afterGetting 函数afterGetting(this);return;
}
fFrameSize和fTo的两个父类变量在流传输时会用到;那么问题来了:doGetNextFrame函数在整个流程中在哪里被调用?什么时候调用?
首先doGetNextFrame函数是在PLAY信令交互的时候被调用;调用流程为:
handleCmd_PLAY->startStream->startPlaying(OnDemandServerMediaSubsession)->startPlaying(MediaSink)->continuePlaying(虚函数,子类H264or5VideoRTPSink实现)->continuePlaying(虚函数,子类MultiFramedRTPSink实现)->buildAndSendPacket->packFrame(MultiFramedRTPSink)->getNextFrame->doGetNextFrame
下面详解介绍下这个流程,看过我的另一篇文章:"live555 rtsp服务器实战之createNewStreamSource" 的都知道handleRequestBytes函数调用了handleCmd_SETUP;同样handleCmd_PLAY函数也是在这里被调用的;handleRequestBytes函数就是用来处理各种客户端信令交互信息的;handleRequestBytes函数如下:
void RTSPServer::RTSPClientConnection::handleRequestBytes(int newBytesRead)
{...if (urlIsRTSPS != fOurRTSPServer.fOurConnectionsUseTLS){
#ifdef DEBUGfprintf(stderr, "Calling handleCmd_redirect()\n");
#endifhandleCmd_redirect(urlSuffix);}else if (strcmp(cmdName, "OPTIONS") == 0){// If the "OPTIONS" command included a "Session:" id for a session that doesn't exist,// then treat this as an error:if (requestIncludedSessionId && clientSession == NULL){
#ifdef DEBUGfprintf(stderr, "Calling handleCmd_sessionNotFound() (case 1)\n");
#endifhandleCmd_sessionNotFound();}else{// Normal case:handleCmd_OPTIONS();}}else if (urlPreSuffix[0] == '\0' && urlSuffix[0] == '*' && urlSuffix[1] == '\0'){// The special "*" URL means: an operation on the entire server. This works only for GET_PARAMETER and SET_PARAMETER:if (strcmp(cmdName, "GET_PARAMETER") == 0){handleCmd_GET_PARAMETER((char const *)fRequestBuffer);}else if (strcmp(cmdName, "SET_PARAMETER") == 0){handleCmd_SET_PARAMETER((char const *)fRequestBuffer);}else{handleCmd_notSupported();}}else if (strcmp(cmdName, "DESCRIBE") == 0){handleCmd_DESCRIBE(urlPreSuffix, urlSuffix, (char const *)fRequestBuffer);}else if (strcmp(cmdName, "SETUP") == 0){Boolean areAuthenticated = True;if (!requestIncludedSessionId){// No session id was present in the request.// So create a new "RTSPClientSession" object for this request.// But first, make sure that we're authenticated to perform this command:char urlTotalSuffix[2 * RTSP_PARAM_STRING_MAX];// enough space for urlPreSuffix/urlSuffix'\0'urlTotalSuffix[0] = '\0';if (urlPreSuffix[0] != '\0'){strcat(urlTotalSuffix, urlPreSuffix);strcat(urlTotalSuffix, "/");}strcat(urlTotalSuffix, urlSuffix);if (authenticationOK("SETUP", urlTotalSuffix, (char const *)fRequestBuffer)){clientSession = (RTSPServer::RTSPClientSession *)fOurRTSPServer.createNewClientSessionWithId();}else{areAuthenticated = False;}}if (clientSession != NULL){clientSession->handleCmd_SETUP(this, urlPreSuffix, urlSuffix, (char const *)fRequestBuffer);playAfterSetup = clientSession->fStreamAfterSETUP;}else if (areAuthenticated){
#ifdef DEBUGfprintf(stderr, "Calling handleCmd_sessionNotFound() (case 2)\n");
#endifhandleCmd_sessionNotFound();}}else if (strcmp(cmdName, "TEARDOWN") == 0 || strcmp(cmdName, "PLAY") == 0 || strcmp(cmdName, "PAUSE") == 0 || strcmp(cmdName, "GET_PARAMETER") == 0 || strcmp(cmdName, "SET_PARAMETER") == 0){if (clientSession != NULL){clientSession->handleCmd_withinSession(this, cmdName, urlPreSuffix, urlSuffix, (char const *)fRequestBuffer);}else{
#ifdef DEBUGfprintf(stderr, "Calling handleCmd_sessionNotFound() (case 3)\n");
#endifhandleCmd_sessionNotFound();}}else if (strcmp(cmdName, "REGISTER") == 0 || strcmp(cmdName, "DEREGISTER") == 0){// Because - unlike other commands - an implementation of this command needs// the entire URL, we re-parse the command to get it:char *url = strDupSize((char *)fRequestBuffer);if (sscanf((char *)fRequestBuffer, "%*s %s", url) == 1){// Check for special command-specific parameters in a "Transport:" header:Boolean reuseConnection, deliverViaTCP;char *proxyURLSuffix;parseTransportHeaderForREGISTER((const char *)fRequestBuffer, reuseConnection, deliverViaTCP, proxyURLSuffix);handleCmd_REGISTER(cmdName, url, urlSuffix, (char const *)fRequestBuffer, reuseConnection, deliverViaTCP, proxyURLSuffix);delete[] proxyURLSuffix;}else{handleCmd_bad();}delete[] url;}else{// The command is one that we don't handle:handleCmd_notSupported();}...
}
handleCmd_withinSession函数内部就调用了handleCmd_PLAY函数;仅整理流程,调用途中的函数这里不做解释;直接跳到packFrame(MultiFramedRTPSink)函数:
void MultiFramedRTPSink::packFrame()
{// Get the next frame.// First, skip over the space we'll use for any frame-specific header:fCurFrameSpecificHeaderPosition = fOutBuf->curPacketSize();fCurFrameSpecificHeaderSize = frameSpecificHeaderSize();fOutBuf->skipBytes(fCurFrameSpecificHeaderSize);fTotalFrameSpecificHeaderSizes += fCurFrameSpecificHeaderSize;// See if we have an overflow frame that was too big for the last pktif (fOutBuf->haveOverflowData()){// Use this frame before reading a new one from the sourceunsigned frameSize = fOutBuf->overflowDataSize();struct timeval presentationTime = fOutBuf->overflowPresentationTime();unsigned durationInMicroseconds = fOutBuf->overflowDurationInMicroseconds();fOutBuf->useOverflowData();afterGettingFrame1(frameSize, 0, presentationTime, durationInMicroseconds);}else{// Normal case: we need to read a new frame from the sourceif (fSource == NULL)return;fSource->getNextFrame(fOutBuf->curPtr(), fOutBuf->totalBytesAvailable(),afterGettingFrame, this, ourHandleClosure, this);}
}
这里调用了getNextFrame函数,来看下getNextFrame函数的实现:
void FramedSource::getNextFrame(unsigned char* to, unsigned maxSize,afterGettingFunc* afterGettingFunc,void* afterGettingClientData,onCloseFunc* onCloseFunc,void* onCloseClientData) {// Make sure we're not already being read:if (fIsCurrentlyAwaitingData) {envir() << "FramedSource[" << this << "]::getNextFrame(): attempting to read more than once at the same time!\n";envir().internalError();}fTo = to;fMaxSize = maxSize;fNumTruncatedBytes = 0; // by default; could be changed by doGetNextFrame()fDurationInMicroseconds = 0; // by default; could be changed by doGetNextFrame()fAfterGettingFunc = afterGettingFunc;fAfterGettingClientData = afterGettingClientData;fOnCloseFunc = onCloseFunc;fOnCloseClientData = onCloseClientData;fIsCurrentlyAwaitingData = True;doGetNextFrame();
}
发现啦!发现doGetNextFrame函数啦!所以doGetNextFrame函数就是在getNextFrame内被调用的;但是问题来了:搜索可以发现在live555中doGetNextFrame函数很多;怎么就确定这里的doGetNextFrame和我们自定义的doGetNextFrame有关呢?
问的好!那这的关键点就在于fSource->getNextFrame的fSource变量到底是什么类的对象;才能确定doGetNextFrame到底调用的是哪个类的函数;fSource属于MediaSink类的成员变量:
FramedSource* fSource;
但是FrameSource子类很多;所以需要确定fSource到底指向的是哪个子类;
既然fSource是MediaSink类的成员,那么fSource肯定是在MediaSink或其子类中被赋值;因此查找下这些类;首先明确下MediaSink的继承关系,从fSource调用的类开始查找:
//父类
MultiFramedRTPSink:RTPSink:MediaSink:Medium
//子类
H264VideoRTPSink:H264or5VideoRTPSink:VideoRTPSink:MultiFramedRTPSink
因此在这些类中查找即可;根据调用流程可知startPlaying最先赋值,值是startPlaying的第一个参数;
Boolean MediaSink::startPlaying(MediaSource& source,afterPlayingFunc* afterFunc,void* afterClientData) {// Make sure we're not already being played:if (fSource != NULL) {envir().setResultMsg("This sink is already being played");return False;}// Make sure our source is compatible:if (!sourceIsCompatibleWithUs(source)) {envir().setResultMsg("MediaSink::startPlaying(): source is not compatible!");return False;}fSource = (FramedSource*)&source;fAfterFunc = afterFunc;fAfterClientData = afterClientData;return continuePlaying();
}
而该函数是在StreamState类中的startPlaying函数中调用;
void StreamState ::startPlaying(Destinations *dests, unsigned clientSessionId,TaskFunc *rtcpRRHandler, void *rtcpRRHandlerClientData,ServerRequestAlternativeByteHandler *serverRequestAlternativeByteHandler,void *serverRequestAlternativeByteHandlerClientData)
{...if (!fAreCurrentlyPlaying && fMediaSource != NULL){if (fRTPSink != NULL){fRTPSink->startPlaying(*fMediaSource, afterPlayingStreamState, this);fAreCurrentlyPlaying = True;}else if (fUDPSink != NULL){fUDPSink->startPlaying(*fMediaSource, afterPlayingStreamState, this);fAreCurrentlyPlaying = True;}}
}
startPlaying的第一个参数是fMediaSource;而fMediaSource是在StreamState类的构造函数中被赋值的:
StreamState::StreamState(OnDemandServerMediaSubsession &master,Port const &serverRTPPort, Port const &serverRTCPPort,RTPSink *rtpSink, BasicUDPSink *udpSink,unsigned totalBW, FramedSource *mediaSource,Groupsock *rtpGS, Groupsock *rtcpGS): fMaster(master), fAreCurrentlyPlaying(False), fReferenceCount(1),fServerRTPPort(serverRTPPort), fServerRTCPPort(serverRTCPPort),fRTPSink(rtpSink), fUDPSink(udpSink), fStreamDuration(master.duration()),fTotalBW(totalBW), fRTCPInstance(NULL) /* created later */,fMediaSource(mediaSource), fStartNPT(0.0), fRTPgs(rtpGS), fRTCPgs(rtcpGS)
{
}
StreamState是在OnDemandServerMediaSubsession类的getStreamParameters函数中被调用:
void OnDemandServerMediaSubsession ::getStreamParameters(unsigned clientSessionId,struct sockaddr_storage const &clientAddress,Port const &clientRTPPort,Port const &clientRTCPPort,int tcpSocketNum,unsigned char rtpChannelId,unsigned char rtcpChannelId,TLSState *tlsState,struct sockaddr_storage &destinationAddress,u_int8_t & /*destinationTTL*/,Boolean &isMulticast,Port &serverRTPPort,Port &serverRTCPPort,void *&streamToken)
{if (addressIsNull(destinationAddress)){// normal case - use the client address as the destination address:destinationAddress = clientAddress;}isMulticast = False;if (fLastStreamToken != NULL && fReuseFirstSource){// Special case: Rather than creating a new 'StreamState',// we reuse the one that we've already created:serverRTPPort = ((StreamState *)fLastStreamToken)->serverRTPPort();serverRTCPPort = ((StreamState *)fLastStreamToken)->serverRTCPPort();++((StreamState *)fLastStreamToken)->referenceCount();streamToken = fLastStreamToken;}else{// Normal case: Create a new media source:unsigned streamBitrate;FramedSource *mediaSource = createNewStreamSource(clientSessionId, streamBitrate);...streamToken = fLastStreamToken = new StreamState(*this, serverRTPPort, serverRTCPPort, rtpSink, udpSink,streamBitrate, mediaSource,rtpGroupsock, rtcpGroupsock);}
}
createNewStreamSource函数加上OnDemandServerMediaSubsession类是不是很熟悉?对的OnDemandServerMediaSubsession就是我们自定义的用于实现createNewStreamSource函数的类H264LiveVideoServerMediaSubssion的父类;因为createNewStreamSource在OnDemandServerMediaSubsession中是纯虚函数,因此该处调用的就是我们自定义的createNewStreamSource函数;
因此在startPlaying函数中将fSource赋值为createNewStreamSource的返回值;我们再看一下createNewStreamSource函数吧:
FramedSource* H264LiveVideoServerMediaSubssion::createNewStreamSource(unsigned clientSessionId, unsigned& estBitrate)
{/* Remain to do : assign estBitrate */estBitrate = 1000; // kbps, estimate//创建视频源H264FramedLiveSource* liveSource = H264FramedLiveSource::createNew(envir(), Server_datasize, Server_databuf, Server_dosent);if (liveSource == NULL){return NULL;}// Create a framer for the Video Elementary Stream:return H264VideoStreamFramer::createNew(envir(), liveSource);
}
则fSource就是H264VideoStreamFramer::createNew(envir(), liveSource); 因此getNextFrame函数中运行的doGetNextFrame就是H264VideoStreamFramer中的doGetNextFrame函数;
等等!发现问题没有:这个并不是我们自定义的H264FramedLiveSource类中获取视频帧的函数doGetNextFrame;这是为什么呢?
别急!返回值中H264VideoStreamFramer::createNew(envir(), liveSource); 将doGetNextFrame的类对象liveSource传递进了H264VideoStreamFramer类中;而liveSource最终赋值给了StreamParser类中成员变量fInputSource和FramedFilter类的成员变量fInputSource;更多细节参考我的另一篇文章;
接着往下说:H264VideoStreamFramer中的doGetNextFrame函数会调用MPEGVideoStreamFramer中的doGetNextFrame函数;在这个函数中有一个ensureValidBytes1第一次调用了自定义的函数doGetNextFrame
void StreamParser::ensureValidBytes1(unsigned numBytesNeeded) {
.
.
.fInputSource->getNextFrame(&curBank()[fTotNumValidBytes],maxNumBytesToRead,afterGettingBytes, this,onInputClosure, this);throw NO_MORE_BUFFERED_INPUT;
}
这里仅仅是对流的解析;下面才是真正获取视频流:
void H264or5Fragmenter::afterGettingFrame(void* clientData, unsigned frameSize,unsigned numTruncatedBytes,struct timeval presentationTime,unsigned durationInMicroseconds) {H264or5Fragmenter* fragmenter = (H264or5Fragmenter*)clientData;fragmenter->afterGettingFrame1(frameSize, numTruncatedBytes, presentationTime,durationInMicroseconds);
}void H264or5Fragmenter::afterGettingFrame1(unsigned frameSize,unsigned numTruncatedBytes,struct timeval presentationTime,unsigned durationInMicroseconds) {fNumValidDataBytes += frameSize;fSaveNumTruncatedBytes = numTruncatedBytes;fPresentationTime = presentationTime;fDurationInMicroseconds = durationInMicroseconds;// Deliver data to the client:doGetNextFrame();
}void H264or5Fragmenter::doGetNextFrame() {if (fNumValidDataBytes == 1) {// We have no NAL unit data currently in the buffer. Read a new one:fInputSource->getNextFrame(&fInputBuffer[1], fInputBufferSize - 1,afterGettingFrame, this,FramedSource::handleClosure, this);} else {...}
}
afterGettingFrame函数才是真正调用doGetNextFrame的函数?那么afterGettingFrame是在哪里被调用的呢?
看一下fSource->getNextFrame函数:
fInputSource->getNextFrame(fOutBuf->curPtr(), fOutBuf->totalBytesAvailable(),afterGettingFrame, this, ourHandleClosure, this);
这里第三个参数就调用了afterGettingFrame函数,再看看getNextFrame函数实现;上面已经写过了 为了方便理解这里再写下:
void FramedSource::getNextFrame(unsigned char* to, unsigned maxSize,afterGettingFunc* afterGettingFunc,void* afterGettingClientData,onCloseFunc* onCloseFunc,void* onCloseClientData) {// Make sure we're not already being read:if (fIsCurrentlyAwaitingData) {envir() << "FramedSource[" << this << "]::getNextFrame(): attempting to read more than once at the same time!\n";envir().internalError();}fTo = to;fMaxSize = maxSize;fNumTruncatedBytes = 0; // by default; could be changed by doGetNextFrame()fDurationInMicroseconds = 0; // by default; could be changed by doGetNextFrame()fAfterGettingFunc = afterGettingFunc;fAfterGettingClientData = afterGettingClientData;fOnCloseFunc = onCloseFunc;fOnCloseClientData = onCloseClientData;fIsCurrentlyAwaitingData = True;doGetNextFrame();
}
afterGettingFrame被赋值给了fAfterGettingClientData指针;那么fAfterGettingClientData指针什么时候被调用的?
回望doGetNextFrame的实现,每次读完流都会调用:
afterGetting(this);
//函数实现
void FramedSource::afterGetting(FramedSource* source) {source->nextTask() = NULL;source->fIsCurrentlyAwaitingData = False;// indicates that we can be read again// Note that this needs to be done here, in case the "fAfterFunc"// called below tries to read another frame (which it usually will)if (source->fAfterGettingFunc != NULL) {(*(source->fAfterGettingFunc))(source->fAfterGettingClientData,source->fFrameSize, source->fNumTruncatedBytes,source->fPresentationTime,source->fDurationInMicroseconds);}
}
afterGetting的函数实现中就调用了fAfterGettingFunc指针;这就使doGetNextFrame形成了闭环:
至此函数doGetNextFrame的执行流程已经全部解析完毕,后续还会继续更新关于doGetNextFrame函数获取的帧数据是怎么处理和发送的,H264VideoStreamFramer中的doGetNextFrame函数和MPEGVideoStreamFramer中的doGetNextFrame函数都是什么作用!期待的话关注我,了解最新动态!
该文章持续更新!如果有错误或者模糊的地方欢迎留言探讨!