Android Surface 的渲染机制

# 1.Surface 是什么?

1
2
3
4

Handle onto a raw buffer that is being managed by the screen compositor.
A Surface is generally created by or from a consumer of image buffers (such as a SurfaceTexture, android.media.MediaRecorder, or android.renderscript.Allocation), and is handed to some kind of producer (such as OpenGL, MediaPlayer, or CameraDevice) to draw into.
Note: A Surface acts like a weak reference to the consumer it is associated with. By itself it will not keep its parent consumer from being reclaimed.

上面这段话是 Surface.java 这个类的注释,大致意思是:
Surface 是一个句柄,指向一个 raw buffer 缓冲对象,这个缓冲对象由 screen compositer 管理。Surface 通常由一个图像缓冲的消费者创建,例如 SurfaceTexture,MediaRecorder,Allocation。然后被送给各种消费者来绘制。

# window 创建流程

在 Activity 的启动过程中,
ActivityThread.handleLaunchActivity 中

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Override  
public Activity handleLaunchActivity(ActivityClientRecord r,
PendingTransactionActions pendingActions, Intent customIntent) {
...
WindowManagerGlobal.initialize();
...
final Activity a = performLaunchActivity(r, customIntent);
}


private Activity performLaunchActivity(ActivityClientRecord r, Intent customIntent) {
...
ContextImpl appContext = createBaseContextForActivity(r);
...
Activity activity = null;
activity = mInstrumentation.newActivity(
cl, component.getClassName(), r.intent);

...
Application app = r.packageInfo.makeApplication(false, mInstrumentation);
...
activity.attach(appContext, this, getInstrumentation(), r.token,
r.ident, app, r.intent, r.activityInfo, title, r.parent,
r.embeddedID, r.lastNonConfigurationInstances, config,
r.referrer, r.voiceInteractor, window, r.configCallback,
r.assistToken, r.shareableActivityToken);

...
mInstrumentation.callActivityOnCreate(activity, r.state);
}

在 Activity 的 attch 中,创建创建了 Window 对象。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
final void attach(Context context, ActivityThread aThread,  
Instrumentation instr, IBinder token, int ident,
Application application, Intent intent, ActivityInfo info,
CharSequence title, Activity parent, String id,
NonConfigurationInstances lastNonConfigurationInstances,
Configuration config, String referrer, IVoiceInteractor voiceInteractor,
Window window, ActivityConfigCallback activityConfigCallback, IBinder assistToken,
IBinder shareableActivityToken) {

attachBaseContext(context);

mFragments.attachHost(null /*parent*/);
...
mWindow = new PhoneWindow(this, window, activityConfigCallback);
...
mWindow.setWindowManager(
(WindowManager)context.getSystemService(Context.WINDOW_SERVICE),
mToken, mComponent.flattenToString(),
(info.flags & ActivityInfo.FLAG_HARDWARE_ACCELERATED) != 0);
if (mParent != null) {
mWindow.setContainer(mParent.getWindow());
}
mWindowManager = mWindow.getWindowManager();

}

Activity 中的 WindowManager 的初始化
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

public void setWindowManager(WindowManager wm, IBinder appToken, String appName,
boolean hardwareAccelerated) {
mAppToken = appToken;
mAppName = appName;
mHardwareAccelerated = hardwareAccelerated;
if (wm == null) {
wm = (WindowManager)mContext.getSystemService(Context.WINDOW_SERVICE);
}
mWindowManager = ((WindowManagerImpl)wm).createLocalWindowManager(this);
}


//WindowManagerImpl.java
public WindowManagerImpl createLocalWindowManager(Window parentWindow) {
Bridge.getLog().fidelityWarning(ILayoutLog.TAG_UNSUPPORTED,
"The preview does not support multiple windows.",
null, null, null);
return this;
}


ActivityThread.handleResumeActivity 中
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21


//ActivityThread.java
public void handleResumeActivity(ActivityClientRecord r, boolean finalStateRequest,
boolean isForward, String reason) {
if (r.activity.mVisibleFromClient) {
r.activity.makeVisible();
}
}



//Activity.java
void makeVisible() {
if (!mWindowAdded) {
ViewManager wm = getWindowManager();
wm.addView(mDecor, getWindow().getAttributes());
mWindowAdded = true;
}
mDecor.setVisibility(View.VISIBLE);
}

会通过 WindowManager.addView(View,Attributes) 将 DecorView 添加到 Window 上。

WindowManager 实现类是 WindowManagerImpl,他又将添加 View 的操作委托给了 WindowmanagerGlobal。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
//WindowmanagerGlobal
public void addView(View view, ViewGroup.LayoutParams params,
Display display, Window parentWindow, int userId) {
ViewRootImpl root = new ViewRootImpl(view.getContext(), display);
root.setView(view, wparams, panelParentView, userId);
}



//ViewRootImpl
public void setView(View view, WindowManager.LayoutParams attrs, View panelParentView,
int userId) {
//启动Choreographer的Vsync信号监听,
requestLayout();
//通过WindowSession请求wms,这里的WindowSession是一个binder接口。
es = mWindowSession.addToDisplayAsUser(mWindow, mWindowAttributes,
getHostVisibility(), mDisplay.getDisplayId(), userId,
mInsetsController.getRequestedVisibility(), inputChannel, mTempInsets,
mTempControls);

}


//WindowSession初始化
public static IWindowSession getWindowSession() {
synchronized (WindowManagerGlobal.class) {
if (sWindowSession == null) {
try {
// Emulate the legacy behavior. The global instance of InputMethodManager
// was instantiated here. // TODO(b/116157766): Remove this hack after cleaning up @UnsupportedAppUsage InputMethodManager.ensureDefaultInstanceForDefaultDisplayIfNecessary();
IWindowManager windowManager = getWindowManagerService();
sWindowSession = windowManager.openSession(
new IWindowSessionCallback.Stub() {
@Override
public void onAnimatorScaleChanged(float scale) {
ValueAnimator.setDurationScale(scale);
}
});
} catch (RemoteException e) {
throw e.rethrowFromSystemServer();
}
}
return sWindowSession;
}
}

从这里进入了 system_server 中的 WMS 进行执行
1
2
3
4
5
6
7
8
9
10
//Session.java   Session是IWindowSession这个AIDL接口的实现类,是应用与WMS进行交互的接口
@Override
public int addToDisplay(IWindow window, WindowManager.LayoutParams attrs,
int viewVisibility, int displayId, InsetsState requestedVisibility,
InputChannel outInputChannel, InsetsState outInsetsState,
InsetsSourceControl[] outActiveControls) {
return mService.addWindow(this, window, attrs, viewVisibility, displayId,
UserHandle.getUserId(mUid), requestedVisibility, outInputChannel, outInsetsState,
outActiveControls);
}

最后进入了 WindowManagerService 中执行:

# Surface 的创建

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
//1.WindowManagerService.addWindow()
public int addWindow(Session session, IWindow client, LayoutParams attrs, int viewVisibility,
int displayId, int requestUserId, InsetsState requestedVisibility,
InputChannel outInputChannel, InsetsState outInsetsState,
InsetsSourceControl[] outActiveControls) {
final DisplayContent displayContent = getDisplayContentOrCreate(displayId, attrs.token);
WindowToken token = displayContent.getWindowToken(
hasParent ? parentWindow.mAttrs.token : attrs.token);
if (token == null) {
token = new WindowToken.Builder(this, binder, type)
.setDisplayContent(displayContent)
.setOwnerCanManageAppTokens(session.mCanAddInternalSystemWindow)
.setRoundedCornerOverlay(isRoundedCornerOverlay)
.build();
}
final WindowState win = new WindowState(this, session, client, token, parentWindow,
appOp[0], attrs, viewVisibility, session.mUid, userId,
session.mCanAddInternalSystemWindow);

...
win.attach();

}

//2.WindowSession.attach()
void attach() {
mSession.windowAddedLocked();
}

//3.Session.windowAddedLocked()
void windowAddedLocked(){
...
mSurfaceSession = new SurfaceSession();
...
}

//4.SurfaceSession
public SurfaceSession() {
mNativeClient = nativeCreate();
}


5.android_view_SurfaceSession.nativeCreate()
static jlong nativeCreate(){
SurfaceClientComposer* client = new SurfaceClientComposer();
}

# Surface 上屏流程

1.Android 发送了 Vsync 垂直同步脉冲信号

-> DisplaySync
->Choreographer 接收到 Vsync-app 信号
->Choreographer 回调 doFrame
->ViewRootImpl 执行 scheduleTravesals, 同时发送一个同步屏障
-> 取消同步屏障,执行 performTravesals
->performMeasure(),performLayout(),performDraw()
-> 申请 BufferQueue 中的图像缓冲区对象
->RenderThread 发送 OpenGL 命令
->queueBuffer

->DisplaySync
->SurfaceFlinger 收到 Vsync-sf
-> 释放上一次获取 buffer 缓冲区对象
->onMessageReceive (INVALIDATE) 判断当前是否有 Layer 状态发生变化,BufferQueue 中是否有新数据
->onMessageReceive (REFRESH) 获取了 LAYER 后,将 buffer 发送给 HWC 进行合成图层
-> 图层合成后通过 drm 将帧送显上屏
->HWC 归还 buffer 给 surfaceFlinger,待下一次 Vsync-sf 时 release 这次的 buffer。