1 介绍
1.1 框架中位置
上图为Android的图形显示系统框架图。
首先上层应用通过ViewRoot的scheduleTraversals函数发起绘制任务,并通过HWUI调用OpenGL接口将绘制数据传递给GPU处理;SF会接收所有应用更新的绘制数据,并根据Z-Order、透明度、大小、位置等参数计算出每个应用图层在最终合成图像中的位置;SF完成图层处理后会把所有的应用图层提供给HWC,由HWC来决定这些图层的合成策略并调用屏显驱动做合成;最后屏显驱动会将合成的最终画面送到硬件屏幕显示。
Android系统通过buffer缓冲区来保存图形信息,这个buffer的内存空间是有SF向图形内存分配器Gralloc申请的,而Gralloc是通过ION Driver从kernel中开辟出一块共享内存。在整个图形显示流程中,buffer缓冲区会在App、SF、HWC之间来回流转。
1.2 职责
- surfaceFlinger是一个服务进程
- 负责管理应用程序窗口的创建、显示、更新和销毁等操作
- 向上和WindowManager交互,向下和HwComposer交互。
SF会接收所有应用更新的绘制数据,并根据Z-Order、透明度、大小、位置等参数计算出每个应用图层在最终合成图像中的位置;
SF完成图层处理后会把所有的应用图层提供给HWC,由HWC来决定这些图层的合成策略并调用屏显驱动做合成
负责显示系统界面和应用程序的图形渲染工作。
- 图像合成:SurfaceFlinger 接收来自多个应用程序和系统服务的图像缓冲区,根据它们的位置、大小、透明度、Z轴顺序等属性,将它们合成到一个最终的缓冲区中,然后发送到显示设备上 。
- 管理 Surface:SurfaceFlinger 管理所有 Surface 对象,包括视频、图片等,并为每个 Surface 对象分配 BufferQueue(缓冲区队列),确保每个 Surface 都能按时完成显示 。
- 三缓冲与 VSYNC 机制:SurfaceFlinger 采用双缓冲区机制,即前台和后台两个缓冲区,以及 VSYNC 机制,以提高渲染效率和质量,减少屏幕撕裂现象 。
- 硬件加速:SurfaceFlinger 提供多种硬件加速技术,如 OpenGL ES、Vulkan 等,使应用程序能够更快地渲染 UI 界面 。
- 窗口叠加与透明度支持:SurfaceFlinger 支持窗口叠加、透明度、混合模式等特性,以支持复杂的多层 UI 界面 。
- 与 Hardware Composer 通信:SurfaceFlinger 与硬件抽象层 Hardware Composer 进行通信,利用硬件加速的方式来合成 Surface,提高性能和节省电量 。
- 处理输入事件:SurfaceFlinger 中的 EventThread 方法负责处理输入事件,将输入事件分发给对应的窗口或 Surface 进行处理
2 源码分析
首先看下系统中SurfaceFlinger的进程
代码中入口:
SF是一个独立的可执行程序,路径是/system/bin/surfaceflinger,开机过程中init进程会解析surfaceflinger.rc,启动surfaceflinger进程。
frameworks/native/services/surfaceflinger/surfaceflinger.rc,
service surfaceflinger /system/bin/surfaceflingerclass core animationuser systemgroup graphics drmrpc readproccapabilities SYS_NICEonrestart restart --only-if-running zygotetask_profiles HighPerformancesocket pdx/system/vr/display/client stream 0666 system graphics u:object_r:pdx_display_client_endpoint_socket:s0socket pdx/system/vr/display/manager stream 0666 system graphics u:object_r:pdx_display_manager_endpoint_socket:s0socket pdx/system/vr/display/vsync stream 0666 system graphics u:object_r:pdx_display_vsync_endpoint_socket:s0on property:vendor.debug.sf.restart=1restart surfaceflingeron property:init.svc.zygote=restarting && property:debug.sf.toomanylayers=1restart surfaceflingersetprop debug.sf.toomanylayers "0"
2.1 SurfaceFlinger的初始化
init.rc中调用到SF的main函数,这里是SF进程的入口。
路径:frameworks/native/services/surfaceflinger/main_surfaceflinger.cpp
int main(int, char**) {startGraphicsAllocatorService(); //启动图形服务...sp<SurfaceFlinger> flinger = surfaceflinger::createSurfaceFlinger(); //1. 创建SF实例flinger->init(); //2. 对SF实例进行初始化// publish surface flinger//3. 将SF添加到serviceManager中。 这样客户端通过getService()+SurfaceFlinger的名称,// 就可以获取SF的远程代理对象,进而和SF服务跨进程通信//static char const* getServiceName() ANDROID_API { return "SurfaceFlinger"; } @SurfaceFinger.hsp<IServiceManager> sm(defaultServiceManager());sm->addService(String16(SurfaceFlinger::getServiceName()), flinger, false,IServiceManager::DUMP_FLAG_PRIORITY_CRITICAL | IServiceManager::DUMP_FLAG_PROTO);// publish gui::ISurfaceComposer, the new AIDL interface//4. 新建SurfaceComposerAIDL对象,注册到serviceManager中sp<SurfaceComposerAIDL> composerAIDL = new SurfaceComposerAIDL(flinger);sm->addService(String16("SurfaceFlingerAIDL"), composerAIDL, false,IServiceManager::DUMP_FLAG_PRIORITY_CRITICAL | IServiceManager::DUMP_FLAG_PROTO);// 启动Display服务startDisplayService(); // dependency on SF getting registered above...// run surface flinger in this threadflinger->run(); //5. 启动SF线程
}
2.2 SurfaceFlinger构造函数详解
先来看下SurfaceFlinger的类关系图
//frameworks/native/services/surfaceflinger/SurfaceFlinger.h
ISurfaceComposer的接口定义:
它的职责是:
class ISurfaceComposer: public IInterface {
public:DECLARE_META_INTERFACE(SurfaceComposer)static constexpr size_t MAX_LAYERS = 4096;...//Create a connection with SurfaceFlinger.virtual sp<ISurfaceComposerClient> createConnection() = 0;...
2.2.1 createSurfaceFlinger详解
接下来看下surfaceflinger::createSurfaceFlinger()具体做了什么?
新建一个SurfaceFlinger实例,将DefaultFactory 做为参数传入,其中DefaultFactory继承了surfaceflinger::Factory类。
主要做了什么?
- 创建CompositionEngine
- 添加了WindowInfosListenerInvoker回调
//frameworks/native/services/surfaceflinger/SurfaceFlingerFactory.cpp
sp<SurfaceFlinger> createSurfaceFlinger() {static DefaultFactory factory;return new SurfaceFlinger(factory);
}SurfaceFlinger::SurfaceFlinger(Factory& factory, SkipInitializationTag): mFactory(factory),mPid(getpid()),mInterceptor(mFactory.createSurfaceInterceptor()),mTimeStats(std::make_shared<impl::TimeStats>()),mFrameTracer(mFactory.createFrameTracer()),mFrameTimeline(mFactory.createFrameTimeline(mTimeStats, mPid)),mCompositionEngine(mFactory.createCompositionEngine()), //创建CompositionEngine...//添加了WindowInfosListenerInvoker回调mWindowInfosListenerInvoker(sp<WindowInfosListenerInvoker>::make(*this)) {ALOGI("Using HWComposer service: %s", mHwcServiceName.c_str());
}
//frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp
//SurfaceFlinger构造函数中对一些成员变量进行赋值初始化。
SurfaceFlinger::SurfaceFlinger(Factory& factory) : SurfaceFlinger(factory, SkipInitialization) {ALOGI("SurfaceFlinger is starting");hasSyncFramework = running_without_sync_framework(true);dispSyncPresentTimeOffset = present_time_offset_from_vsync_ns(0);useHwcForRgbToYuv = force_hwc_copy_for_virtual_displays(false);maxFrameBufferAcquiredBuffers = max_frame_buffer_acquired_buffers(2);...
}
2.2.2 SurfaceFlinger的init详解
flinger->init()对SF实例进行初始化,具体做了什么?
void SurfaceFlinger::init() {ALOGI( "SurfaceFlinger's main thread ready to run. ""Initializing graphics H/W...");Mutex::Autolock _l(mStateLock);//构造renderengine的builderauto builder = renderengine::RenderEngineCreationArgs::Builder().setPixelFormat(static_cast<int32_t>(defaultCompositionPixelFormat)).setImageCacheSize(maxFrameBufferAcquiredBuffers).setUseColorManagerment(useColorManagement).setEnableProtectedContext(enable_protected_contents(false)).setPrecacheToneMapperShaderOnly(false).setSupportsBackgroundBlur(mSupportsBlur).setContextPriority(useContextPriority? renderengine::RenderEngine::ContextPriority::REALTIME: renderengine::RenderEngine::ContextPriority::MEDIUM);...//创建renderEngine,并给mCompositionEngine设置RenderEnginemCompositionEngine->setRenderEngine(renderengine::RenderEngine::create(builder.build()));...//调用mCompositionEngine来createHWComposer,给mCompositionEngine设置回调函数。思考:这里回调是什么接口?作用是?mCompositionEngine->setTimeStats(mTimeStats);mCompositionEngine->setHwComposer(getFactory().createHWComposer(mHwcServiceName));mCompositionEngine->getHwComposer().setCallback(*this);...// Process any initial hotplug and resulting display changes.processDisplayHotplugEventsLocked(); //重要调用,第一次会做一次插拔检测const auto display = getDefaultDisplayDeviceLocked();LOG_ALWAYS_FATAL_IF(!display, "Missing primary display after registering composer callback.");const auto displayId = display->getPhysicalId();LOG_ALWAYS_FATAL_IF(!getHwComposer().isConnected(displayId), "Primary display is disconnected.");// initialize our drawing statemDrawingState = mCurrentState; //更新drawState, 思考:其中共有几种状态?都是什么时候切换的?initializeDisplays();...ALOGV("Done initializing");
}
2.2.3 processDisplayHotplugEventsLocked详解
//frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp
void SurfaceFlinger::processDisplayHotplugEventsLocked() {for (const auto& event : mPendingHotplugEvents) {std::optional<DisplayIdentificationInfo> info =getHwComposer().onHotplug(event.hwcDisplayId, event.connection);if (!info) { continue; }//获取displayId和token const auto displayId = info->id;const auto token = mPhysicalDisplayTokens.get(displayId);if (event.connection == hal::Connection::CONNECTED) { //如果此时有显示屏幕连接... //} else { //如果此时没有显示屏幕连接ALOGV("Removing display %s", to_string(displayId).c_str());if (const ssize_t index = mCurrentState.displays.indexOfKey(token->get()); index >= 0) {const DisplayDeviceState& state = mCurrentState.displays.valueAt(index);mInterceptor->saveDisplayDeletion(state.sequenceId);mCurrentState.displays.removeItemsAt(index);}mPhysicalDisplayTokens.erase(displayId);}processDisplayChangesLocked(); //主要步骤,处理显示改变}mPendingHotplugEvents.clear();
}
//
void SurfaceFlinger::processDisplayChangesLocked() {const KeyedVector<wp<IBinder>, DisplayDeviceState>& curr(mCurrentState.displays);const KeyedVector<wp<IBinder>, DisplayDeviceState>& draw(mDrawingState.displays);if (!curr.isIdenticalTo(draw)) {mVisibleRegionsDirty = true;// find the displays that were removed (ie: in drawing state but not in current state)// also handle displays that changed (ie: displays that are in both lists)for (size_t i = 0; i < draw.size(); i++) {const wp<IBinder>& displayToken = draw.keyAt(i);const ssize_t j = curr.indexOfKey(displayToken);if (j < 0) {// in drawing state but not in current stateprocessDisplayRemoved(displayToken);} else {// this display is in both lists. see if something changed.const DisplayDeviceState& currentState = curr[j];const DisplayDeviceState& drawingState = draw[i];processDisplayChanged(displayToken, currentState, drawingState); //核心步骤,调用processDisplayAdded}}...}mDrawingState.displays = mCurrentState.displays;
}
2.2.4 processDisplayAdded详解
主要做了几件事:
1)通过CompositionEngine创建display
2)创建BufferQueue 和 DisplaySurface对象
3)调用initscheduler来初始化整个Vsync系统
void SurfaceFlinger::processDisplayAdded(const wp<IBinder>& displayToken,const DisplayDeviceState& state) {compositionengine::DisplayCreationArgsBuilder builder; builder.setPixels(resolution);builder.setIsSecure(state.isSecure);builder.setPowerAdvisor(mPowerAdvisor.get());builder.setName(state.displayName);// 1. 通过CompositionEngine创建display,具体display参数通过builder设置auto compositionDisplay = getCompositionEngine().createDisplay(builder.build());compositionDisplay->setLayerCachingEnabled(mLayerCachingEnabled);sp<compositionengine::DisplaySurface> displaySurface;sp<IGraphicBufferProducer> producer; //为什么有producer 和BqProducer,两者有什么区别?sp<IGraphicBufferProducer> bqProducer;sp<IGraphicBufferConsumer> bqConsumer;//2. 创建BufferQueuegetFactory().createBufferQueue(&bqProducer, &bqConsumer, /*consumerIsSurfaceFlinger =*/false);//3. 创建DisplaySurface对象if (state.isVirtual()) {const auto displayId = VirtualDisplayId::tryCast(compositionDisplay->getId());auto surface = sp<VirtualDisplaySurface>::make(getHwComposer(), *displayId, state.surface,bqProducer, bqConsumer, state.displayName);displaySurface = surface;producer = std::move(surface);} else {const auto displayId = PhysicalDisplayId::tryCast(compositionDisplay->getId());displaySurface = sp<FramebufferSurface>::make(getHwComposer(), *displayId, bqConsumer,state.physical->activeMode->getResolution(),ui::Size(maxGraphicsWidth, maxGraphicsHeight));producer = bqProducer;}...auto display = setupNewDisplayDeviceInternal(displayToken, std::move(compositionDisplay), state,displaySurface, producer);if (display->isPrimary()) {initScheduler(display); //主要步骤,调用initscheduler来初始化整个Vsync系统}mDisplays.try_emplace(displayToken, std::move(display));
}
2.2.5 initScheduler详解
initscheduler初始化整个Vsync系统,即对Vsync产生机制初始化,主要做几件事:
1)如果没有mScheduler ,创建mScheduler,有的话直接使用
2)创建APP连接
3)创建appSf连接
4)初始化Vsync
void SurfaceFlinger::initScheduler(const sp<DisplayDevice>& display) {if (mScheduler) {// If the scheduler is already initialized, this means that we received// a hotplug(connected) on the primary display. In that case we should// update the scheduler with the most recent display information.ALOGW("Scheduler already initialized, updating instead");mScheduler->setRefreshRateConfigs(display->holdRefreshRateConfigs());return;}//如果没有mScheduler ,创建mSchedulermScheduler = std::make_unique<scheduler::Scheduler>(static_cast<ICompositor&>(*this),static_cast<ISchedulerCallback&>(*this),features);{auto configs = display->holdRefreshRateConfigs();if (configs->kernelIdleTimerController().has_value()) {features |= Feature::kKernelIdleTimer;}mScheduler->createVsyncSchedule(features);mScheduler->setRefreshRateConfigs(std::move(configs));}setVsyncEnabled(false);mScheduler->startTimers();const auto configs = mVsyncConfiguration->getCurrentConfigs();const nsecs_t vsyncPeriod = currRefreshRate.getPeriodNsecs();//创建APP连接,它用于连接生产者和消费者,相互通信,回调消息。 思考:这里的生产者和消费者分别是?mAppConnectionHandle = mScheduler->createConnection("app", mFrameTimeline->getTokenManager(),/*workDuration=*/configs.late.appWorkDuration,/*readyDuration=*/configs.late.sfWorkDuration,impl::EventThread::InterceptVSyncsCallback());//创建appSf连接,它用于连接生产者和消费者,相互通信,回调消息。mSfConnectionHandle = mScheduler->createConnection("appSf", mFrameTimeline->getTokenManager(),/*workDuration=*/std::chrono::nanoseconds(vsyncPeriod),/*readyDuration=*/configs.late.sfWorkDuration,[this](nsecs_t timestamp) {mInterceptor->saveVSyncEvent(timestamp);});//初始化Vsync,initVsync主要作用是绑定一个回调函数MessageQueue::vsyncCallback 到VSyncDispatch上,回调名字"sf"mScheduler->initVsync(mScheduler->getVsyncDispatch(), *mFrameTimeline->getTokenManager(),configs.late.sfWorkDuration);mFpsReporter = new FpsReporter(*mFrameTimeline, *this); mScheduler->onPrimaryDisplayModeChanged(mAppConnectionHandle, display->getActiveMode());
}
思考几个问题:
1)创建mScheduler的用处?
他是一个调度器,是SF的重要部分之一,负责协调和优化渲染流程,以确保流畅的图形更新和响应。
2)mScheduler->createConnection()具体做了什么?做什么用?
分别创建APP和appSf的连接,它用于连接生产者和消费者,相互通信,回调消息。 思考:这里的生产者和消费者分别是?
3)mScheduler->initVsync()做了什么?
initVsync主要作用是绑定一个回调函数 MessageQueue::vsyncCallback 到VSyncDispatch上,回调名字"sf"
2.2.6 mScheduler详解
职责:Scheduler是SurfaceFlinger的调度器, 管理和调度与屏幕渲染相关的任务。
SurfaceFlinger 是 Android 图形系统的核心组件之一,负责合成显示到屏幕上的各种图形层。Scheduler 作为其重要部分,负责协调和优化渲染流程,以确保流畅的图形更新和响应。
Scheduler 的主要作用包括:
- 任务调度:
Scheduler 负责安排不同的渲染任务的执行顺序。它确定何时执行图形合成、缓冲区交换等操作,以优化性能并减少延迟。 - 帧同步:
确保图形渲染与显示的刷新率同步。Scheduler 调整渲染的时间,以保持帧率稳定,减少撕裂和卡顿现象。它通常会利用硬件的垂直同步(V-Sync)信号来确保每一帧都能准确地显示在屏幕上。 - 时间片管理:
Scheduler 处理与时间相关的任务,如管理和分配每个任务的时间片。它负责在合成图层时进行时间上的协调,确保每一层图形在正确的时间被渲染。
围绕上面三个问题,结合代码看下mScheduler做了什么?
- mScheduler定义
1)类关系图
Scheduler继承了impl::MessageQueue类,类关系图:
2)主要接口
- createConnection()
用于创建一个指定名字的连接,它用于连接生产者和消费者,相互通信,回调消息。
ConnectionHandle Scheduler::createConnection(const char* connectionName, frametimeline::TokenManager* tokenManager,std::chrono::nanoseconds workDuration, std::chrono::nanoseconds readyDuration,impl::EventThread::InterceptVSyncsCallback interceptCallback) {//创建DispSyncSource对象auto vsyncSource = makePrimaryDispSyncSource(connectionName, workDuration, readyDuration);auto throttleVsync = makeThrottleVsyncCallback();auto getVsyncPeriod = makeGetVsyncPeriodFunction();//创建EventThread的对象,将传入的参数传给EventThread构造函数中auto eventThread = std::make_unique<impl::EventThread>(std::move(vsyncSource), tokenManager,std::move(interceptCallback),std::move(throttleVsync),std::move(getVsyncPeriod));return createConnection(std::move(eventThread));
}
//创建EventThreadConnection并加入到mConnections中
ConnectionHandle Scheduler::createConnection(std::unique_ptr<EventThread> eventThread) {const ConnectionHandle handle = ConnectionHandle{mNextConnectionHandleId++}; //获取连接handleauto connection = createConnectionInternal(eventThread.get());std::lock_guard<std::mutex> lock(mConnectionsLock);//将handle和EventThreadConnection对象一起保存到mConnections中mConnections.emplace(handle, Connection{connection, std::move(eventThread)});return handle; // 返回handle
}
//创建并返回EventThreadConnection对象
sp<EventThreadConnection> Scheduler::createConnectionInternal(EventThread* eventThread, ISurfaceComposer::EventRegistrationFlags eventRegistration) {return eventThread->createEventConnection([&] { resync(); }, eventRegistration);
}
EventThread的构造函数做了什么?
1)给mVSyncSource设置回调函数
2)创建主线程
3)提高主线程优先级
EventThread::EventThread(std::unique_ptr<VSyncSource> vsyncSource,android::frametimeline::TokenManager* tokenManager,InterceptVSyncsCallback interceptVSyncsCallback,ThrottleVsyncCallback throttleVsyncCallback,GetVsyncPeriodFunction getVsyncPeriodFunction): mVSyncSource(std::move(vsyncSource)),mTokenManager(tokenManager),mInterceptVSyncsCallback(std::move(interceptVSyncsCallback)),mThrottleVsyncCallback(std::move(throttleVsyncCallback)),mGetVsyncPeriodFunction(std::move(getVsyncPeriodFunction)),mThreadName(mVSyncSource->getName()) {mVSyncSource->setCallback(this); //EventThread继承了VSyncSource::Callback,用EventThread给mVSyncSource设置回调函数//回调函数接口是onVSyncEvent(nsecs_t when, VSyncData vsyncData)//创建主线程,同时阻塞等待mThread = std::thread([this]() NO_THREAD_SAFETY_ANALYSIS {std::unique_lock<std::mutex> lock(mMutex);threadMain(lock);});//设置线程优先级为SP_FOREGROUNDpthread_setname_np(mThread.native_handle(), mThreadName);pid_t tid = pthread_gettid_np(mThread.native_handle());...set_sched_policy(tid, SP_FOREGROUND);
}
EventThread类关系图
createEventConnection()做了什么?新建一个EventThreadConnection对象,设置eventThread、resyncCallback和eventRegistration三个参数。
sp<EventThreadConnection> EventThread::createEventConnection(ResyncCallback resyncCallback,ISurfaceComposer::EventRegistrationFlags eventRegistration) const {return new EventThreadConnection(const_cast<EventThread*>(this),IPCThreadState::self()->getCallingUid(),std::move(resyncCallback), eventRegistration);
}
- initVsync()
绑定一个回调函数 MessageQueue::vsyncCallback到VsyncDispatch上,回调名字是"sf"。
通过adb shell dumpsys SurfaceFlinger > sf.txt命令可以查看VsysncDispatch上绑定了那些callbacks?
VsyncDispatch:Timer:DebugState: WaitingmTimerSlack: 0.50ms mMinVsyncDistance: 3.00msmIntendedWakeupTime: 9223078412288.00ms from nowmLastTimerCallback: 2553.19ms ago mLastTimerSchedule: 2554.64ms agoCallbacks:sf: workDuration: 27.60ms readyDuration: 0.00ms earliestVsync: -2655.90ms relative to nowmLastDispatchTime: 2525.89ms agoappSf: workDuration: 10.00ms readyDuration: 27.60ms earliestVsync: -31929.41ms relative to nowmLastDispatchTime: 31929.41ms agoapp: workDuration: 20.00ms readyDuration: 27.60ms earliestVsync: -2648.97ms relative to nowmLastDispatchTime: 2648.97ms ago
void MessageQueue::initVsync(scheduler::VSyncDispatch& dispatch,frametimeline::TokenManager& tokenManager,std::chrono::nanoseconds workDuration) {setDuration(workDuration);//给mVsync赋值,其中registration 是VSyncCallbackRegistration,MessageQueue::vsyncCallback在MessageQueue中定义的回调函数,// bind()是将MessageQueue的成员函数(vsyncCallback)与参数绑定,生成新的可调用对象mVsync.tokenManager = &tokenManager;mVsync.registration = std::make_unique<scheduler::VSyncCallbackRegistration>(dispatch,std::bind(&MessageQueue::vsyncCallback, this, //this表示MessageQueue对象std::placeholders::_1,std::placeholders::_2,std::placeholders::_3),"sf"); //这里将sf名字的回调函数绑定到VsyncDispatch上
}
3 小结
本文介绍了SF的职责,它是一个服务进程,负责显示系统界面和应用程序的图形渲染工作。
- 负责管理应用程序窗口的创建、显示、更新和销毁等操作
- 向上和WindowManager交互,向下和HwComposer交互。
SF会接收所有应用更新的绘制数据,并根据Z-Order、透明度、大小、位置等参数计算出每个应用图层在最终合成图像中的位置;
SF完成图层处理后会把所有的应用图层提供给HWC,由HWC来决定这些图层的合成策略并调用屏显驱动做合成
同时讲述了SurfaceFlinger的进程启动入口和初始化流程,涉及到Vsync系统的初始化(mScheduler做的事情)。