Skip to content

Commit fcd266c

Browse files
mykola-mokhnachmykola-mokhnach
authored
docs: Create a guide for testing using tunnels (#2753)
1 parent 953a70e commit fcd266c

2 files changed

Lines changed: 332 additions & 0 deletions

File tree

‎docs/guides/remotexpc-tunnels-real-devices.md‎

Lines changed: 331 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,331 @@
1+
---
2+
title: Remote XPC Tunnels
3+
---
4+
5+
The XCUITest driver can use **Remote XPC** (via `appium-ios-remotexpc`) and an IPv6 tunnel to talk to
6+
real iOS and tvOS devices on **OS version 18 or newer**. This guide explains how to:
7+
8+
- Create and inspect tunnels using the `tunnel-creation` script
9+
- Run tests against real devices using those tunnels
10+
- Plan parallel test runs when using a single Appium server or multiple servers
11+
12+
This guide applies to **real iOS/tvOS devices only**. Simulators do not use this tunnel mechanism.
13+
14+
At the moment, the tunnel workflow described here supports **devices physically connected via USB**
15+
to the host running Appium. Support for **wireless tvOS / Apple TV devices** via Remote XPC tunnels
16+
is planned but not yet available in this driver.
17+
18+
## When you need tunnels and Remote XPC
19+
20+
On iOS/tvOS 18+ Apple routes many system services (including XCTest-related ones) over IPv6-only
21+
interfaces and Remote XPC endpoints. The XCUITest driver uses:
22+
23+
- `appium-ios-remotexpc` for:
24+
- Device lockdown / USBMUX communication
25+
- CoreDeviceProxy and Remote XPC connections
26+
- High‑level services (installation proxy, AFC, crash reports, DVT instruments, etc.)
27+
- `appium-ios-tuntap` (used internally by `appium-ios-remotexpc`) to:
28+
- Create a TUN/TAP virtual network interface
29+
- Establish an IPv6 tunnel between the host and the device
30+
31+
For iOS/tvOS < 18 these tunnels are not required and the driver falls back to legacy transport.
32+
33+
## Prerequisites
34+
35+
- **Host OS**:
36+
- macOS or Linux (tunnels rely on TUN/TAP support)
37+
- **Node.js / Appium**:
38+
- Node.js compatible with this driver version (see `package.json` or README)
39+
- Appium 3.x
40+
- **Device setup**:
41+
- Real iOS or tvOS device on **18.x or newer**
42+
- Device paired and trusted on the host
43+
- Developer tools and Xcode installed (for general iOS development and pairing)
44+
- **Optional dependency (required for tunnels)**:
45+
- The driver declares `appium-ios-remotexpc` as an **optional dependency**, so in a normal
46+
installation npm will install it automatically. You only need to install it manually if
47+
the optional dependency step failed or you are wiring a custom environment.
48+
- For details about Remote XPC and IPv6 tunneling, see the
49+
[`appium-ios-remotexpc` README](https://github.com/appium/appium-ios-remotexpc).
50+
51+
- **Privileges**:
52+
- **sudo/root is required** to create TUN/TAP interfaces for the tunnel. You should generally run
53+
the tunnel script with `sudo` (or an equivalent mechanism, such as a root container).
54+
55+
To verify that the optional dependency and tunnel infrastructure are available you can run:
56+
57+
```bash
58+
appium driver doctor xcuitest
59+
```
60+
61+
Look for the optional checks related to `appium-ios-remotexpc` and “tunnel availability”.
62+
63+
## Creating tunnels with the driver script
64+
65+
The XCUITest driver exposes a high‑level convenience script that wraps the lower‑level
66+
`appium-ios-remotexpc` tunnel workflow:
67+
68+
```bash
69+
sudo appium driver run xcuitest tunnel-creation
70+
```
71+
72+
This script:
73+
74+
- Connects to `usbmuxd` and enumerates all connected, trusted iOS/tvOS devices
75+
- For each device:
76+
- Starts a Lockdown session
77+
- Starts `com.apple.internal.devicecompute.CoreDeviceProxy` via Remote XPC
78+
- Creates an IPv6 tunnel using `TunnelManager.getTunnel(...)`
79+
- Starts a packet stream server on a local TCP port (default base: `50000`)
80+
- Builds an in‑memory **tunnel registry** containing:
81+
- Device UDID and device ID
82+
- Tunnel IPv6 address (`Address`) and `RsdPort`
83+
- Packet stream port and basic metadata
84+
- Starts an HTTP **tunnel registry API server** and prints its address
85+
- Persists the chosen registry port in a per‑driver strongbox entry so that the driver can find it
86+
87+
### Command-line options
88+
89+
The script supports a few options:
90+
91+
- **Target a specific device**:
92+
93+
```bash
94+
sudo appium driver run xcuitest tunnel-creation --udid <device-udid>
95+
```
96+
97+
- **Customize packet stream base port**:
98+
99+
```bash
100+
sudo appium driver run xcuitest tunnel-creation --packet-stream-base-port 52000
101+
```
102+
103+
The script will assign `52000`, `52001`, `52002`, … to packet stream servers for each device.
104+
105+
- **Customize tunnel registry port**:
106+
107+
```bash
108+
sudo appium driver run xcuitest tunnel-creation --tunnel-registry-port 43000
109+
```
110+
111+
The registry API will then be available at:
112+
113+
- `http://localhost:43000/remotexpc/tunnels`
114+
115+
The script also stores the chosen port in a strongbox entry for the `appium-xcuitest-driver`
116+
package so that driver instances can locate the registry automatically.
117+
118+
### Inspecting the tunnel registry
119+
120+
After a successful run you should see log lines similar to:
121+
122+
```text
123+
📁 Tunnel registry API:
124+
The tunnel registry is now available through the API at:
125+
http://localhost:<port>/remotexpc/tunnels
126+
...
127+
curl http://localhost:<port>/remotexpc/tunnels/<udid>
128+
```
129+
130+
Useful endpoints:
131+
132+
- **List all tunnels**:
133+
134+
```bash
135+
curl http://localhost:<port>/remotexpc/tunnels
136+
```
137+
138+
- **Get tunnel for a specific UDID**:
139+
140+
```bash
141+
curl http://localhost:<port>/remotexpc/tunnels/<udid>
142+
```
143+
144+
The response contains the IPv6 `address`, `rsdPort`, and other metadata required to establish
145+
Remote XPC connections.
146+
147+
## Running tests on a single Appium server
148+
149+
Once tunnels are running, you can start a standard Appium server with the XCUITest driver and run
150+
tests against iOS/tvOS 18+ real devices using normal capabilities.
151+
152+
### Recommended workflow
153+
154+
1. **Start the tunnels (once per host)**:
155+
156+
```bash
157+
sudo appium driver run xcuitest tunnel-creation
158+
```
159+
160+
Leave this process running in the background while tests execute.
161+
162+
2. **Start the Appium server** (in a separate terminal):
163+
164+
```bash
165+
appium
166+
```
167+
168+
3. **Run your tests** using standard XCUITest capabilities:
169+
170+
```json
171+
{
172+
"platformName": "iOS",
173+
"appium:automationName": "XCUITest",
174+
"appium:platformVersion": "18.4",
175+
"appium:udid": "<device-udid>",
176+
}
177+
```
178+
179+
For tvOS, set `"platformName": "tvOS"` and use the UDID of your Apple TV device.
180+
181+
4. **How the driver uses tunnels**:
182+
183+
- When `platformVersion` is **18 or higher** on a **real device**, the driver:
184+
- Automatically imports `appium-ios-remotexpc` (via `getRemoteXPCServices`)
185+
- Uses Remote XPC services (installation proxy, AFC, diagnostics, DVT instruments, etc.)
186+
instead of the legacy paths
187+
- Relies on the IPv6 tunnels created by the tunnel registry for connectivity
188+
- If `appium-ios-remotexpc` is missing or tunnels are not available, some advanced real‑device
189+
features for 18+ may be unavailable or will fall back to slower/less reliable code paths.
190+
191+
No extra capabilities are required to “enable” tunnels; they are automatically used when:
192+
193+
- `appium-ios-remotexpc` is installed, **and**
194+
- the tunnel registry server is reachable, **and**
195+
- the platform is iOS/tvOS 18+ on a real device.
196+
197+
## Parallel tests with a single Appium server
198+
199+
The tunnel creation script is **multi‑device aware**: it creates and registers an independent tunnel
200+
for each connected device. XCUITest can then run multiple sessions concurrently on a **single**
201+
Appium server as long as:
202+
203+
- Each session uses a **different real device** (`appium:udid` is unique per session)
204+
- The tunnels for all those devices are present in the tunnel registry
205+
206+
### Example: single server, multiple devices
207+
208+
1. **Create tunnels for all connected devices**:
209+
210+
```bash
211+
sudo appium driver run xcuitest tunnel-creation
212+
```
213+
214+
2. **Start one Appium server**:
215+
216+
```bash
217+
appium --port 4723
218+
```
219+
220+
3. **Run tests in parallel**, for example:
221+
222+
- Session A:
223+
224+
```json
225+
{
226+
"platformName": "iOS",
227+
"appium:automationName": "XCUITest",
228+
"appium:platformVersion": "18.1",
229+
"appium:udid": "<iphone-udid>",
230+
"appium:app": "/path/to/iphone/app.app"
231+
}
232+
```
233+
234+
- Session B:
235+
236+
```json
237+
{
238+
"platformName": "tvOS",
239+
"appium:automationName": "XCUITest",
240+
"appium:platformVersion": "18.0",
241+
"appium:udid": "<appletv-udid>",
242+
"appium:app": "/path/to/tvos/app.app"
243+
}
244+
```
245+
246+
4. **Driver behavior**:
247+
248+
- Each session uses the UDID to pick the appropriate tunnel from the registry.
249+
- Underneath, `TunnelManager` maintains a registry of active tunnels and Remote XPC connections
250+
keyed by tunnel address and reuses them when possible.
251+
- Packet stream servers created by the tunnel script are already bound to distinct TCP ports, so
252+
traffic for different devices is isolated.
253+
254+
### Guidelines for single‑server parallelism
255+
256+
- **Do not share a UDID across concurrent sessions** on the same server; use one session per device.
257+
- Ensure the **tunnel script is running before** starting parallel tests so that the registry is
258+
populated.
259+
- If you frequently add/remove devices, re‑run the tunnel script to refresh the registry.
260+
261+
## Parallel tests with multiple Appium servers
262+
263+
You can also run multiple Appium servers in parallel on the same host while **sharing a single
264+
tunnel registry** and tunnel process.
265+
266+
### Recommended pattern: one tunnel process, many servers
267+
268+
1. **Start a single global tunnel process**:
269+
270+
```bash
271+
sudo appium driver run xcuitest tunnel-creation --tunnel-registry-port 43000
272+
```
273+
274+
- Leave this running in the background.
275+
- It creates tunnels for all currently connected devices and exposes the registry on `43000`.
276+
- The registry port is persisted in a strongbox entry for `appium-xcuitest-driver` so that all
277+
driver instances running in the same environment can discover it.
278+
279+
2. **Start multiple Appium servers**, for example:
280+
281+
```bash
282+
# Server 1
283+
appium --port 4723
284+
285+
# Server 2
286+
appium --port 4725
287+
```
288+
289+
3. **Assign devices to servers** via capabilities:
290+
291+
- Server 1 handles device A (iOS 18.x, UDID `<iphone-udid>`)
292+
- Server 2 handles device B (tvOS 18.x, UDID `<appletv-udid>`)
293+
294+
4. **Run tests in parallel** across servers:
295+
296+
- Each server behaves as described in the single‑server section, using the shared tunnel
297+
registry.
298+
- Tunnels are created only once; both servers reuse the same IPv6 tunnel and Remote XPC
299+
infrastructure for each device.
300+
301+
### Alternative: one tunnel process per isolation boundary
302+
303+
In more advanced setups (e.g., Docker, multiple hosts, CI agents), you might:
304+
305+
- Run **one tunnel‑creation process per container/VM**, started together with that container’s
306+
Appium server(s).
307+
- Use distinct `--tunnel-registry-port` values for each isolation boundary.
308+
309+
This keeps tunnel state scoped to each environment, but within that boundary you should still
310+
avoid running multiple competing tunnel‑creation scripts simultaneously, as they may fight over
311+
TUN/TAP configuration and USBMUX connections.
312+
313+
### Guidelines for multi‑server parallelism
314+
315+
- **Prefer a single global tunnel process per physical host** and share it between servers.
316+
- Ensure all Appium servers that should share tunnels:
317+
- Run under the same user or environment where the strongbox entry is accessible, or
318+
- Are configured to discover the same tunnel registry port (by starting the script with an
319+
explicit `--tunnel-registry-port`).
320+
- As with a single server, never assign the **same UDID to multiple concurrent sessions** unless
321+
your test coordination knows exactly what it is doing.
322+
323+
## tvOS‑specific notes
324+
325+
- The tunnel creation and Remote XPC mechanism works the same way for **tvOS 18+** as for iOS 18+.
326+
- Use:
327+
- `"platformName": "tvOS"`
328+
- A tvOS 18+ `platformVersion`
329+
- The UDID of the Apple TV device
330+
- Only devices connected via USB are currently supported. Support for wirelessly connected TV
331+
devices is coming.

‎mkdocs.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ nav:
6161
- guides/parallel-tests.md
6262
- guides/ci-setup.md
6363
- guides/multiple-xcode-versions.md
64+
- guides/remotexpc-tunnels-real-devices.md
6465
- guides/remotexpc-apple-tv-pairing.md
6566
- Improve Session Startup Performance:
6667
- guides/run-preinstalled-wda.md

0 commit comments

Comments
 (0)