matrixfurry.com
The open source OpenXR runtime
doc: Add diagrams for swapchain image allocation control flow.
Part-of: <https://gitlab.freedesktop.org/monado/monado/-/merge_requests/2300>
+1
doc/changes/doc/mr.2300.md
+1
doc/changes/doc/mr.2300.md
···
1
+
Illustrate various control flows for swapchain image allocation, including interaction with IPC.
+2
-1
doc/ipc-design.md
+2
-1
doc/ipc-design.md
···
222
222
a @ref xrt_image_native_allocator (aka XINA) and shares it to the server. When
223
223
using D3D11 or D3D12 on Windows, buffers are allocated by the client compositor
224
224
and imported into the native compositor, because Vulkan can import buffers from
225
-
D3D, but D3D cannot import buffers allocated by Vulkan.
225
+
D3D, but D3D cannot import buffers allocated by Vulkan. See @ref swapchains-ipc
226
+
for details.
226
227
227
228
[SCM_RIGHTS]: https://man7.org/linux/man-pages/man3/cmsg.3.html
228
229
[win32handles]: https://lackingrhoticity.blogspot.com/2015/05/passing-fds-handles-between-processes.html
+16
doc/mermaid/swapchain_allocation_inproc.mmd
+16
doc/mermaid/swapchain_allocation_inproc.mmd
···
1
+
%% Copyright 2024, Collabora, Ltd. and the Monado contributors
2
+
%% SPDX-License-Identifier: BSL-1.0
3
+
4
+
%% Simple in-process case
5
+
sequenceDiagram
6
+
participant app
7
+
participant cc as client compositor
8
+
participant native_comp as xrt_compositor_native
9
+
10
+
11
+
app->>+cc: xrCreateSwapchain
12
+
cc->>+native_comp: xrt_comp_create_swapchain
13
+
native_comp->>-cc: xrt_swapchain impl
14
+
Note over cc: Keep reference to inner xrt_swapchain in<br>the object we create
15
+
Note over cc: Import handles from<br/>inner xrt_swapchain into client API
16
+
cc->>-app: return swapchain
+25
doc/mermaid/swapchain_allocation_ipc.mmd
+25
doc/mermaid/swapchain_allocation_ipc.mmd
···
1
+
%% Copyright 2024, Collabora, Ltd. and the Monado contributors
2
+
%% SPDX-License-Identifier: BSL-1.0
3
+
4
+
%% Out of process, typical behavior (server-side allocation)
5
+
sequenceDiagram
6
+
box Client Process
7
+
participant app
8
+
participant cc as client compositor
9
+
participant client_native as client process<br/>native compositor<br/>(IPC Stub)
10
+
end
11
+
box Server Process
12
+
participant server_ipc_handler as Server IPC handler
13
+
participant server_comp as Server xrt_compositor_native
14
+
end
15
+
16
+
app->>+cc: xrCreateSwapchain
17
+
cc->>+client_native: xrt_comp_create_swapchain
18
+
client_native->>+server_ipc_handler: xrt_comp_create_swapchain<br/>(IPC call)
19
+
server_ipc_handler->>+server_comp: xrt_comp_create_swapchain
20
+
server_comp->>-server_ipc_handler: xrt_swapchain impl
21
+
server_ipc_handler->>-client_native: swapchain ID and<br/>image handles<br/>(over IPC)
22
+
client_native->>-cc: return ipc_swapchain<br/>as inner xrt_swapchain
23
+
Note over cc: Keep reference to inner xrt_swapchain in<br>the object we create
24
+
Note over cc: Import handles from<br/>inner xrt_swapchain into client API
25
+
cc->>-app: return swapchain
+27
doc/mermaid/swapchain_allocation_ipc_d3d.mmd
+27
doc/mermaid/swapchain_allocation_ipc_d3d.mmd
···
1
+
%% Copyright 2024, Collabora, Ltd. and the Monado contributors
2
+
%% SPDX-License-Identifier: BSL-1.0
3
+
4
+
%% Out of process, D3D client API
5
+
sequenceDiagram
6
+
box Client Process
7
+
participant app
8
+
participant cc as D3D client compositor
9
+
participant client_native as client process<br/>native compositor<br/>(IPC Stub)
10
+
end
11
+
box Server Process
12
+
participant server_ipc_handler as Server IPC handler
13
+
participant server_comp as Server xrt_compositor_native
14
+
end
15
+
16
+
app->>+cc: xrCreateSwapchain
17
+
Note over cc: create images<br/>(because d3d cannot reliably import from Vulkan)
18
+
Note over cc: export images to handles
19
+
Note over cc: import handles to images for app
20
+
cc->>+client_native: xrt_comp_import_swapchain(xrt_image_native[])<br/>(import handles into xrt_swapchain_native)
21
+
client_native->>+server_ipc_handler: swapchain_import<br/>(IPC call, copies handles)
22
+
server_ipc_handler->>+server_comp: xrt_comp_import_swapchain(xrt_image_native[])
23
+
server_comp->>-server_ipc_handler: xrt_swapchain impl
24
+
server_ipc_handler->>-client_native: swapchain ID (over IPC)
25
+
client_native->>-cc: return ipc_swapchain<br/>as inner xrt_swapchain
26
+
Note over cc: Keep reference to inner xrt_swapchain in<br>the object we create
27
+
cc->>-app: return swapchain
+28
doc/mermaid/swapchain_allocation_ipc_xina.mmd
+28
doc/mermaid/swapchain_allocation_ipc_xina.mmd
···
1
+
%% Copyright 2024, Collabora, Ltd. and the Monado contributors
2
+
%% SPDX-License-Identifier: BSL-1.0
3
+
4
+
%% Out of process, with a "xina" in the IPC client (currently only android with AHB)
5
+
sequenceDiagram
6
+
box Client Process
7
+
participant app
8
+
participant cc as client compositor
9
+
participant client_native as client process<br/>native compositor<br>(IPC Stub)
10
+
participant xina as xrt_image_native_allocator
11
+
end
12
+
box Server Process
13
+
participant server_ipc_handler as Server IPC handler
14
+
participant server_comp as Server xrt_compositor_native
15
+
end
16
+
17
+
app->>+cc: xrCreateSwapchain
18
+
cc->>+client_native: xrt_comp_create_swapchain
19
+
client_native->>+xina: create native images
20
+
xina->>-client_native: collection of<br/>native image handles
21
+
client_native->>+server_ipc_handler: swapchain_import<br/>(IPC call, passing handles)
22
+
server_ipc_handler->>+server_comp: xrt_comp_import_swapchain
23
+
server_comp->>-server_ipc_handler: xrt_swapchain impl
24
+
server_ipc_handler->>-client_native: swapchain ID<br/>(over IPC)
25
+
client_native->>-cc: return ipc_swapchain<br>(handles allocated in client process)<br>as inner xrt_swapchain
26
+
Note over cc: Keep reference to inner xrt_swapchain in<br>the object we create
27
+
Note over cc: Import handles from<br/>inner xrt_swapchain into client API
28
+
cc->>-app: return swapchain
+61
doc/swapchains-ipc.md
+61
doc/swapchains-ipc.md
···
1
+
# Swapchain Allocation and IPC {#swapchains-ipc}
2
+
3
+
<!--
4
+
Copyright 2024, Collabora, Ltd. and the Monado contributors
5
+
SPDX-License-Identifier: BSL-1.0
6
+
-->
7
+
8
+
The control flow in Monado for allocating the images/buffers used for a given
9
+
@ref XrSwapchain can vary widely based on a number of factors.
10
+
11
+
## Simple in-process
12
+
13
+
In the minimal case, which is not common for widespread deployment but which is
14
+
useful for debugging, there is only a single process, with the entire runtime
15
+
loaded into the "client" application. Despite the absence of IPC in this
16
+
scenario, the same basic flow and interaction applies: images are conveyed using
17
+
system ("native") handles.
18
+
19
+
@mermaid{swapchain_allocation_inproc}
20
+
21
+
The control flow makes its way to the main compositor, which, in default
22
+
implementations, uses Vulkan to allocate exportable images.
23
+
24
+
## Simple out-of-process
25
+
26
+
The usual model on desktop is that the images for a given @ref XrSwapchain are
27
+
allocated in the service, as shown below.
28
+
29
+
@mermaid{swapchain_allocation_ipc}
30
+
31
+
However, there are two additional paths possible.
32
+
33
+
## Client-side Allocation with XINA (Out-of-process)
34
+
35
+
In some builds, a "XINA" (like the warrior princess) is used: @ref
36
+
xrt_image_native_allocator. This allocates the "native" (system handle) images
37
+
within the client process, in a central location. Currently only Android builds
38
+
using AHardwareBuffer use this model, but ideally we would probably move toward
39
+
this model for all systems. The main advantage is that the swapchain images are
40
+
allocated in the application process and thus counted against application
41
+
quotas, rather than the service/runtime quotas, avoiding security, privacy, and
42
+
denial-of-service risks.
43
+
44
+
@mermaid{swapchain_allocation_ipc_xina}
45
+
46
+
Generally a XINA used in this way is intended for client-process allocation.
47
+
However, there is an implementation (disabled by default) of a "loopback XINA"
48
+
in the IPC client, which implements the XINA API using the IPC interface. So
49
+
using a XINA in that case would not result in client-process image allocation.
50
+
51
+
## Allocation for D3D client apps (Out-of-process)
52
+
53
+
Direct3D cannot reliably and portably import images allocated in Vulkan. So, if
54
+
an application uses Direct3D, the D3D client compositor does the allocation, on
55
+
the client side, as shown below. The example shown is out-of-process.
56
+
57
+
@mermaid{swapchain_allocation_ipc_d3d}
58
+
59
+
Note that this pattern is used even when running in-process with D3D, the result
60
+
is a hybrid of this diagram and the first (in-process) diagram, dropping the
61
+
client process native compositor and server IPC handler from the diagram.