Skip to content

Commit dffadb5

Browse files
peterchavepeterchave
authored
Merge pull request #6 from openmoq/feature/generic-runner
Feature/generic runner
2 parents b3108c0 + 960b311 commit dffadb5

33 files changed

Lines changed: 4465 additions & 1 deletion

‎.github/workflows/moq-test-ui-image.yml‎

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
name: moq-test-ui-image
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
tags:
8+
- 'v*'
9+
paths:
10+
- '.github/workflows/moq-test-ui-image.yml'
11+
- 'moq-test-ui/**'
12+
pull_request:
13+
paths:
14+
- '.github/workflows/moq-test-ui-image.yml'
15+
- 'moq-test-ui/**'
16+
workflow_dispatch:
17+
18+
concurrency:
19+
group: moq-test-ui-image-${{ github.ref }}
20+
cancel-in-progress: true
21+
22+
env:
23+
IMAGE_NAME: ghcr.io/${{ github.repository_owner }}/moq-test-ui
24+
25+
jobs:
26+
test:
27+
runs-on: ubuntu-latest
28+
defaults:
29+
run:
30+
working-directory: moq-test-ui
31+
steps:
32+
- name: Check out repository
33+
uses: actions/checkout@v4
34+
35+
- name: Set up Node.js
36+
uses: actions/setup-node@v4
37+
with:
38+
node-version: '22'
39+
40+
- name: Install dependencies
41+
run: npm install
42+
43+
- name: Start server
44+
run: |
45+
node src/server.js > /tmp/moq-test-ui.log 2>&1 &
46+
echo $! > /tmp/moq-test-ui.pid
47+
48+
- name: Smoke test HTTP endpoint
49+
run: |
50+
for _ in $(seq 1 20); do
51+
if curl -fsS http://127.0.0.1:3000/api/tools >/dev/null; then
52+
exit 0
53+
fi
54+
sleep 1
55+
done
56+
cat /tmp/moq-test-ui.log
57+
exit 1
58+
59+
- name: Stop server
60+
if: always()
61+
run: |
62+
if [ -f /tmp/moq-test-ui.pid ]; then
63+
kill "$(cat /tmp/moq-test-ui.pid)" || true
64+
fi
65+
66+
docker:
67+
runs-on: ubuntu-latest
68+
needs: test
69+
permissions:
70+
contents: read
71+
packages: write
72+
steps:
73+
- name: Check out repository
74+
uses: actions/checkout@v4
75+
76+
- name: Set up QEMU
77+
uses: docker/setup-qemu-action@v3
78+
79+
- name: Set up Docker Buildx
80+
uses: docker/setup-buildx-action@v3
81+
82+
- name: Log in to GHCR
83+
if: github.event_name != 'pull_request'
84+
uses: docker/login-action@v3
85+
with:
86+
registry: ghcr.io
87+
username: ${{ github.actor }}
88+
password: ${{ secrets.GITHUB_TOKEN }}
89+
90+
- name: Extract Docker metadata
91+
id: meta
92+
uses: docker/metadata-action@v5
93+
with:
94+
images: ${{ env.IMAGE_NAME }}
95+
tags: |
96+
type=raw,value=latest,enable={{is_default_branch}}
97+
type=ref,event=branch
98+
type=ref,event=pr
99+
type=semver,pattern={{version}}
100+
type=sha
101+
102+
- name: Build container image
103+
id: build
104+
uses: docker/build-push-action@v6
105+
with:
106+
context: ./moq-test-ui
107+
file: ./moq-test-ui/Dockerfile
108+
platforms: linux/amd64
109+
push: ${{ github.event_name != 'pull_request' }}
110+
load: ${{ github.event_name == 'pull_request' }}
111+
tags: ${{ steps.meta.outputs.tags }}
112+
labels: ${{ steps.meta.outputs.labels }}
113+
114+
- name: Smoke test container image
115+
if: github.event_name == 'pull_request'
116+
run: |
117+
docker run -d --rm --name moq-test-ui-smoke -p 3000:3000 "${{ env.IMAGE_NAME }}:pr-${{ github.event.pull_request.number }}"
118+
for _ in $(seq 1 20); do
119+
if curl -fsS http://127.0.0.1:3000/api/tools >/dev/null; then
120+
docker logs moq-test-ui-smoke
121+
docker stop moq-test-ui-smoke
122+
exit 0
123+
fi
124+
sleep 1
125+
done
126+
docker logs moq-test-ui-smoke
127+
docker stop moq-test-ui-smoke
128+
exit 1

‎moq-test-ui/DESIGN.md‎

Lines changed: 194 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,194 @@
1+
# MoQ Test UI Design
2+
3+
## Overview
4+
5+
`moq-test-ui` is a generic web runner for Docker-backed MoQ test tools. It provides a single framework that can:
6+
7+
- discover tools from manifest files
8+
- render parameter forms dynamically
9+
- run each tool inside a Docker container
10+
- stream output to the browser over WebSocket
11+
- apply client-side regex filters to logs
12+
- load optional per-tool renderers for richer result views
13+
- persist run history as JSON
14+
- run a relay self-test workflow across multiple tools
15+
16+
The implementation is centered on a lightweight Node.js server plus a no-build browser client.
17+
18+
## Current Architecture
19+
20+
```text
21+
Browser
22+
- tools view
23+
- self-test view
24+
- history view
25+
- dynamic parameter form
26+
- raw output panel + optional renderer panel
27+
|
28+
| WebSocket + small REST surface
29+
v
30+
Node.js server
31+
- Express static hosting and read APIs
32+
- ws message handling
33+
- tool registry
34+
- Docker executor
35+
- self-test orchestrator
36+
- results store
37+
|
38+
| Docker socket
39+
v
40+
Docker engine
41+
- tool containers
42+
- optional compose deployment
43+
- nginx TLS proxy for containerized deployment
44+
```
45+
46+
## Backend Design
47+
48+
The backend is a small Node.js service using Express and `ws`.
49+
50+
- `src/server.js` initializes the tool registry, Docker executor, results store, and self-test orchestrator.
51+
- Static assets are served from `public/`.
52+
- Tool renderers are exposed directly from `tools/` so the browser can load them on demand.
53+
- REST endpoints are intentionally narrow: list tools, list results, and fetch a saved result.
54+
- Real-time activity uses a single WebSocket connection per browser session.
55+
56+
### Tool Registry
57+
58+
The registry scans `tools/*/tool.json` at startup, validates the required fields, and exposes normalized tool metadata to the UI. The manifest is the primary extension point for the system.
59+
60+
Each tool can define:
61+
62+
- Docker image, command, entrypoint, network mode, extra hosts, and environment
63+
- parameter schema for form generation
64+
- regex log filters
65+
- optional renderer module
66+
- optional `selfTest` configuration
67+
- optional generated input files via `prepareFiles`
68+
69+
### Docker Execution
70+
71+
The Docker executor is responsible for container lifecycle and output streaming.
72+
73+
- Containers are created through `dockerode` using the local Docker socket.
74+
- Commands are built from manifest templates with user parameters substituted into `buildCommand`.
75+
- Output is demultiplexed, ANSI escape sequences are stripped, and lines are forwarded live to the browser.
76+
- Temporary input files can be generated on the host and bind-mounted into sibling containers.
77+
- A duration-based auto-stop path exists for tools that declare a timeout parameter.
78+
- Runs are stopped on request and also when the owning WebSocket session closes.
79+
80+
Generated input files are written under `/tmp/generic-runner` when available so sibling containers can access them correctly when the UI itself runs inside Docker.
81+
82+
### Results Storage
83+
84+
Each completed run is saved as structured JSON under `results/<tool>/...json`. Self-test runs are saved separately under `results/self-test/`.
85+
86+
Stored result data includes:
87+
88+
- tool name
89+
- input parameters
90+
- timestamps
91+
- exit code
92+
- session identifier
93+
- full captured output
94+
95+
This keeps the history view simple and makes later re-rendering possible without rerunning the tool.
96+
97+
## Frontend Design
98+
99+
The frontend is plain ES modules with no build step.
100+
101+
- `public/js/app.js` is the coordinator for tool selection, runs, history, and self-test.
102+
- `param-form.js` renders forms directly from manifest metadata.
103+
- `output-panel.js` renders the raw log stream and applies client-side filters.
104+
- `renderer-loader.js` dynamically imports `tools/<name>/renderer.js` when a tool has a renderer.
105+
- `self-test-ui.js` reuses the same renderer model during self-test runs.
106+
- `history-ui.js` lists saved results and displays them in a modal overlay.
107+
108+
The UI is split into three views:
109+
110+
- Tools: pick a tool, configure inputs, run it, inspect logs and renderer output
111+
- Automated Tests: run all self-test-enabled tools against one relay target
112+
- History: browse and reopen saved result files
113+
114+
## Tool Model
115+
116+
The current tool set is manifest-driven and includes these implemented tool directories:
117+
118+
- `probe`
119+
- `conformance`
120+
- `interop-tests`
121+
- `adaptive-bench`
122+
- `multi-sub-bench`
123+
124+
## Self-Test Design
125+
126+
Self-test is implemented as a server-side orchestrated sequence.
127+
128+
- The orchestrator collects all tools with enabled `selfTest` entries.
129+
- It expands single or array-based self-test configs.
130+
- Tools run sequentially in manifest-defined order.
131+
- Shared relay configuration is merged with per-tool defaults.
132+
- Live output is streamed to the browser as normal progress events.
133+
- Final aggregated results are saved to disk.
134+
135+
The browser-side self-test UI intentionally reuses each tool renderer instead of introducing a separate parsing layer. That keeps self-test support mostly data-driven: adding `selfTest` to a tool plus a renderer is generally enough.
136+
137+
## WebSocket Protocol
138+
139+
The current protocol covers:
140+
141+
- tool listing
142+
- run start and stop
143+
- self-test start and stop
144+
- live output events
145+
- result listing and result fetch
146+
147+
The protocol is intentionally simple and session-scoped. Session isolation is achieved by tracking runs per socket and stopping those runs when the socket disconnects.
148+
149+
## Current Characteristics And Limitations
150+
151+
The current implementation intentionally keeps several areas simple:
152+
153+
- The terminal output view is currently a custom `pre`-based log panel, not a full `xterm.js` terminal. The page still links xterm CSS, but runtime behavior is the simpler implementation.
154+
- Self-test runs all enabled tools; there is no per-run checklist UI.
155+
- Session isolation exists, but reconnect-based active run recovery is not implemented. A disconnect stops the session's runs.
156+
- History is implemented and backed by JSON files, but the list view is intentionally minimal and opens raw saved output in a modal.
157+
- Local startup has been simplified and documented as `npm install` then `npm start`, with Docker Compose as an optional nginx-backed deployment path.
158+
159+
## Deployment Model
160+
161+
Two operating modes are supported.
162+
163+
### Local development
164+
165+
- install dependencies with `npm install`
166+
- start the server with `npm start`
167+
- open `http://localhost:3000`
168+
169+
### Containerized deployment
170+
171+
- `docker compose up --build`
172+
- nginx terminates TLS and proxies to the Node.js app
173+
- the app container mounts the Docker socket so it can launch sibling tool containers
174+
- the compose setup expects certificates in `./certs`
175+
176+
## Operational Notes
177+
178+
The UI stores runtime artifacts that should stay out of source control.
179+
180+
- `results/` contains saved run output and input parameters
181+
- `certs/` contains deployment TLS material
182+
183+
These are operational assets, not source files.
184+
185+
## Design Intent Going Forward
186+
187+
The design remains intentionally data-driven:
188+
189+
- add tools by dropping in a manifest and optional renderer
190+
- keep the backend generic and unaware of tool-specific parsing
191+
- keep self-test composition driven by tool manifests
192+
- preserve saved JSON output as the system-of-record for history and later analysis
193+
194+
That keeps the framework extensible without reintroducing bespoke UI code for each test tool.

‎moq-test-ui/Dockerfile‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
FROM node:22-alpine
2+
3+
RUN apk add --no-cache bash
4+
5+
WORKDIR /app
6+
7+
COPY package.json package-lock.json* ./
8+
RUN npm install --production
9+
10+
COPY src/ ./src/
11+
COPY public/ ./public/
12+
COPY tools/ ./tools/
13+
14+
RUN mkdir -p /app/results
15+
16+
EXPOSE 3000
17+
18+
CMD ["node", "src/server.js"]

0 commit comments

Comments
 (0)