SurfaceTexture OnFrameAvailableListener 调用流程分析

背景:

最近项目中遇到一个问题, 需要搞清楚OnFrameAvailableListener 回调流程, 本文借此机会做个记录, 巩固印象, 有相关困惑的同学也可以参考下.

本文基于Android 14 framework 源码进行分析

SurfaceTexture.java

OnFrameAvailableListener 设置过程

 public void setOnFrameAvailableListener(@Nullable final OnFrameAvailableListener listener,@Nullable Handler handler) {if (listener != null) {// Although we claim the thread is arbitrary, earlier implementation would// prefer to send the callback on the creating looper or the main looper// so we preserve this behavior here.Looper looper = handler != null ? handler.getLooper() :mCreatorLooper != null ? mCreatorLooper : Looper.getMainLooper();mOnFrameAvailableHandler = new Handler(looper, null, true /*async*/) {@Overridepublic void handleMessage(Message msg) {listener.onFrameAvailable(SurfaceTexture.this);}};} else {mOnFrameAvailableHandler = null;}}

一直很好奇,硬件解码后回调是怎么调用的, 首先java在设置回调的时候,回看调用方是不是有handler, 如果没有handler 会发送到主线程执行, 如果有handler,就会把回调发送这个handler执行,接下来来看下这个mOnFrameAvailableHandler在什么地方调用

mOnFrameAvailableHandler

private static void postEventFromNative(WeakReference<SurfaceTexture> weakSelf) {SurfaceTexture st = weakSelf.get();if (st != null) {Handler handler = st.mOnFrameAvailableHandler;if (handler != null) {handler.sendEmptyMessage(0);}}
}

native 会调用这个方法, 然后这个方法在handler里边发送消息,通知回调,接下来看下native是怎么调用的

postEventFromNative

static void SurfaceTexture_classInit(JNIEnv* env, jclass clazz)
{fields.postEvent = env->GetStaticMethodID(clazz, "postEventFromNative","(Ljava/lang/ref/WeakReference;)V");if (fields.postEvent == NULL) {ALOGE("can't find android/graphics/SurfaceTexture.postEventFromNative");}
}
class JNISurfaceTextureContextCommon {
public:JNISurfaceTextureContextCommon(JNIEnv* env, jobject weakThiz, jclass clazz): mWeakThiz(env->NewGlobalRef(weakThiz)), mClazz((jclass)env->NewGlobalRef(clazz)) {}...void onFrameAvailable(const BufferItem& item) {JNIEnv* env = getJNIEnv();if (env != NULL) {env->CallStaticVoidMethod(mClazz, fields.postEvent, mWeakThiz);} else {ALOGW("onFrameAvailable event will not posted");}}protected:....jobject mWeakThiz;jclass mClazz;
};
class JNISurfaceTextureContextFrameAvailableListener: public JNISurfaceTextureContextCommon,public SurfaceTexture::FrameAvailableListener {
public:JNISurfaceTextureContextFrameAvailableListener(JNIEnv* env, jobject weakThiz, jclass clazz): JNISurfaceTextureContextCommon(env, weakThiz, clazz) {}void onFrameAvailable(const BufferItem& item) override {JNISurfaceTextureContextCommon::onFrameAvailable(item);}
};

JNISurfaceTextureContextFrameAvailableListener 集成了JNISurfaceTextureContextCommon,SurfaceTexture::FrameAvailableListener, 并实现了方法onFrameAvailable,再通过jni去调用java的postEventFromNative, 接下来看 JNISurfaceTextureContextFrameAvailableListener 设置到那里的

JNISurfaceTextureContextFrameAvailableListener

static void SurfaceTexture_init(JNIEnv* env, jobject thiz, jboolean isDetached,jint texName, jboolean singleBufferMode, jobject weakThiz)
{sp<IGraphicBufferProducer> producer;sp<IGraphicBufferConsumer> consumer;BufferQueue::createBufferQueue(&producer, &consumer);sp<SurfaceTexture> surfaceTexture;surfaceTexture = new SurfaceTexture(consumer, texName,GL_TEXTURE_EXTERNAL_OES, true, !singleBufferMode);surfaceTexture->setName(String8::format("SurfaceTexture-%d-%d-%d",(isDetached ? 0 : texName),getpid(),createProcessUniqueId()));SurfaceTexture_setSurfaceTexture(env, thiz, surfaceTexture);SurfaceTexture_setProducer(env, thiz, producer);jclass clazz = env->GetObjectClass(thiz);sp<JNISurfaceTextureContextFrameAvailableListener> ctx(new JNISurfaceTextureContextFrameAvailableListener(env, weakThiz, clazz));surfaceTexture->setFrameAvailableListener(ctx);SurfaceTexture_setFrameAvailableListener(env, thiz, ctx);}

看的出来SurfaceTexture在初始化时候, 会创建生产者(producer),与消费者(consumer), 接着会把ctx设置到surfaceTexture->setFrameAvailableListener, 接下来看下内部代码

surfaceTexture::setFrameAvailableListener

SurfaceTexture 继承了ConsumerBase, SurfaceTexture并没有实现setFrameAvailableListener, 在其父类实现的,接下来看下父类的代码


class ConsumerListener : public virtual RefBase {
public:ConsumerListener() {}virtual ~ConsumerListener();...virtual void onFrameAvailable(const BufferItem& item) = 0;...}
ConsumerBase::ConsumerBase(const sp<IGraphicBufferConsumer>& bufferQueue, bool controlledByApp) :mAbandoned(false),mConsumer(bufferQueue),mPrevFinalReleaseFence(Fence::NO_FENCE) {...wp<ConsumerListener> listener = static_cast<ConsumerListener*>(this);status_t err = mConsumer->consumerConnect(proxy, controlledByApp);...
}void ConsumerBase::setFrameAvailableListener(const wp<FrameAvailableListener>& listener) {CB_LOGV("setFrameAvailableListener");Mutex::Autolock lock(mFrameAvailableMutex);mFrameAvailableListener = listener;
}void ConsumerBase::onFrameAvailable(const BufferItem& item) {sp<FrameAvailableListener> listener;{ // scope for the lockMutex::Autolock lock(mFrameAvailableMutex);listener = mFrameAvailableListener.promote();}if (listener != nullptr) {listener->onFrameAvailable(item);}
}

ConsumerBase 继承了ConsumerListener, 并重写了onFrameAvailable, 在ConsumerBase 构造调用了consumerConnect, 这个方法很关键,接下来看下这个方法在干什么

BufferQueueConsumer::consumerConnect

class BufferQueueConsumer : public BnGraphicBufferConsumer {virtual status_t consumerConnect(const sp<IConsumerListener>& consumer,bool controlledByApp) {return connect(consumer, controlledByApp);}
}
status_t BufferQueueConsumer::connect(const sp<IConsumerListener>& consumerListener, bool controlledByApp) {std::lock_guard<std::mutex> lock(mCore->mMutex);mCore->mConsumerListener = consumerListener;mCore->mConsumerControlledByApp = controlledByApp;return NO_ERROR;
}

这里就很清晰了, consumerConnect 会把上边的回调ConsumerBase::onFrameAvailable 回调设置到mConsumer(BufferQueueConsumer) 的 mCore->mConsumerListener中去,

我们知道BufferQueueProducer与BufferQueueConsumer共享一个BufferQueueCore 如下图所示:

也就说到这里生成者Producer通过Core就可以获取到Consumer的回调了

接下来看下回调是怎么调用的

回调调用

我们已经知道了SurfaceTexture 创建成功后,还需要创建Surface作为其生产者, 接下来从生产者的角度看下回调调用流程是怎么样的.

Surface作为生产者在dequeuebuffer获取一帧数据后, 在填充后, 会调用queuebuffer,我们就从这个queuebuffer 去跟下源码

int Surface::hook_queueBuffer(ANativeWindow* window,ANativeWindowBuffer* buffer, int fenceFd) {Surface* c = getSelf(window);{std::shared_lock<std::shared_mutex> lock(c->mInterceptorMutex);if (c->mQueueInterceptor != nullptr) {auto interceptor = c->mQueueInterceptor;auto data = c->mQueueInterceptorData;return interceptor(window, Surface::queueBufferInternal, data, buffer, fenceFd);}}return c->queueBuffer(buffer, fenceFd);
}


int Surface::queueBuffer(android_native_buffer_t* buffer, int fenceFd) {ATRACE_CALL();ALOGV("Surface::queueBuffer");Mutex::Autolock lock(mMutex);int i = getSlotFromBufferLocked(buffer);IGraphicBufferProducer::QueueBufferOutput output;IGraphicBufferProducer::QueueBufferInput input;getQueueBufferInputLocked(buffer, fenceFd, mTimestamp, &input);applyGrallocMetadataLocked(buffer, input);sp<Fence> fence = input.fence;nsecs_t now = systemTime();status_t err = mGraphicBufferProducer->queueBuffer(i, input, &output);onBufferQueuedLocked(i, fence, output);return err;
}

status_t BufferQueueProducer::queueBuffer(int slot,const QueueBufferInput &input, QueueBufferOutput *output) {ATRACE_CALL();ATRACE_BUFFER_INDEX(slot);//...frameAvailableListener = mCore->mConsumerListener;//..if (frameAvailableListener != nullptr) {frameAvailableListener->onFrameAvailable(item);} //..} // Autolock scope

这个BufferQueueProducer::queueBuffer 代码很多, 这里删除了无关代码, 看到在queueBuffer 入队的时候会获取mCore->mConsumerListener, 这就是上边ConsumerBase::onFrameAvailable回调, 调用这个最终就会到消费者回调中去,从而一层一层通知,最终到SurfaceTexture.java 往handler发送消息, 最终回调出去