Implement comprehensive UI test suite with meaningful assertions and best practices

.github/copilot-instructions.md

@@ -40,6 +40,94 @@ electron . --enable-mcp-introspection --remote-debugging-port=9223
 3. **Include screenshots** - Visual verification is required for UI changes
 4. **Handle asynchronous operations properly** - This is an MQTT message queue tool
 
+### Best Practices for UI Tests
+
+#### 1. Use Given-When-Then Pattern
+Structure tests with clear Given-When-Then comments to make them readable:
+
+```typescript
+it('Given a JSON message sent to topic foo/bar/baz, the tree should display nested topics', async function () {
+  // Given: Mock MQTT publishes JSON to foo/bar/baz
+  // When: We wait for the topic to appear in the tree
+  // Then: Topic hierarchy should be visible (foo -> bar -> baz)
+})
+```
+
+#### 2. Wait for Elements, Don't Use Fixed Delays
+Prefer `waitFor` over `sleep` whenever possible:
+
+```typescript
+// ✓ Good: Wait for specific element
+const topic = await page.locator('span[data-test-topic="kitchen"]')
+await topic.waitFor({ state: 'visible', timeout: 5000 })
+
+// ✗ Bad: Fixed delay without verification
+await sleep(5000)
+```
+
+#### 3. Use Meaningful Assertions
+Every test should have explicit assertions that verify the expected state:
+
+```typescript
+// ✓ Good: Explicit assertion with meaningful message
+const treeNodes = await page.locator('[class*="TreeNode"]')
+const count = await treeNodes.count()
+expect(count).to.be.greaterThan(0, 'Topic tree should contain nodes')
+
+// ✗ Bad: No assertion, only screenshot
+await page.screenshot({ path: 'test.png' })
+```
+
+#### 4. Test Data-Driven Scenarios
+Write tests that describe the data flow:
+
+```typescript
+it('Given messages sent to livingroom/lamp/state and livingroom/lamp/brightness, both should appear under livingroom/lamp', async function () {
+  // Test implementation verifies the specific data flow
+})
+```
+
+#### 5. Use Data Test Attributes
+Leverage `data-test-*` attributes for reliable selectors:
+
+```typescript
+// ✓ Good: Use data-test attributes
+const topic = await page.locator('span[data-test-topic="kitchen"]')
+
+// ⚠ Acceptable: Use role/text when data attributes aren't available
+const button = await page.locator('//button/span[contains(text(),"Connect")]')
+
+// ✗ Bad: Rely on CSS classes that may change
+const topic = await page.locator('.MuiTreeItem-label')
+```
+
+#### 6. Verify Multiple Aspects
+Test should verify both state and UI:
+
+```typescript
+// Verify the action completed
+const isVisible = await disconnectButton.isVisible()
+expect(isVisible).to.be.true
+
+// Capture screenshot for visual verification
+await page.screenshot({ path: 'test-screenshot-connection.png' })
+```
+
+#### 7. Handle MQTT Asynchronous Nature
+Account for message propagation time:
+
+```typescript
+// Publish message
+await mockClient.publish('topic/name', 'value')
+
+// Wait for UI to update
+await page.locator(`text="value"`).waitFor({ timeout: 5000 })
+
+// Verify state
+const value = await page.textContent('.message-value')
+expect(value).toBe('value')
+```
+
 ### Handling MQTT Asynchronous Operations
 
 MQTT is inherently asynchronous. When writing tests:

.github/workflows/tests.yml

@@ -18,7 +18,45 @@ jobs:
         run: yarn build
       - name: Test
         run: yarn test
-      - name: UI-Test
+
+  ui-tests:
+    runs-on: ubuntu-latest
+    container:
+      image: ghcr.io/thomasnordquist/mqtt-explorer-ui-tests:latest
+      volumes:
+        - ./:/app
+      options: --user root
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install Packages
+        run: yarn install --frozen-lockfile
+      - name: Build
+        run: yarn build
+      - name: Run UI Tests
+        run: ./scripts/runUiTests.sh
+      - name: Upload Test Screenshots
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: ui-test-screenshots
+          path: |
+            test-screenshot-*.png
+          retention-days: 30
+
+  demo-video:
+    runs-on: ubuntu-latest
+    container:
+      image: ghcr.io/thomasnordquist/mqtt-explorer-ui-tests:latest
+      volumes:
+        - ./:/app
+      options: --user root
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install Packages
+        run: yarn install --frozen-lockfile
+      - name: Build
+        run: yarn build
+      - name: Generate Demo Video
         run: yarn ui-test
       - name: Post-processing
         run: ./scripts/prepareVideo.sh

.gitignore

@@ -14,3 +14,6 @@ screen*.png
 mqtt-explorer-mcp-screenshot.png
 screenshot-mcp-*.png
 test-mcp-introspection.js
+
+# UI test artifacts
+test-screenshot-*.png

Readme.md

@@ -41,28 +41,35 @@ The `app` directory contains all the rendering logic, the `backend` directory cu
 
 ## Automated Tests
 
-To achieve a reliable product automated tests run regularly on travis.
+To achieve a reliable product automated tests run regularly on CI.
 
-- Data model
-- MQTT integration
-- UI-Tests (The demo is a recorded ui test)
+- **Data model tests**: `yarn test:backend`
+- **App tests**: `yarn test:app`
+- **UI test suite**: `yarn test:ui` (independent, deterministic tests)
+- **Demo video**: `yarn ui-test` (UI test recording for documentation)
 
-## Run UI-tests
+### Run UI Test Suite
 
-A [mosquitto](https://mosquitto.org/) MQTT broker is required to run the ui-tests.
-
-Run tests with
+The UI test suite validates core functionality through automated browser tests. Each test is independent and deterministic.
 
 ```bash
-# Run chromedriver in a separate terminal session
-./node_modules/.bin/chromedriver --url-base=wd/hub --port=9515 --verbose
+# Run with automated setup (recommended)
+./scripts/runUiTests.sh
+
+# Or run directly (requires manual MQTT broker setup)
+yarn build
+yarn test:ui
 ```
 
-Compile and execute tests
+See [docs/UI-TEST-SUITE.md](docs/UI-TEST-SUITE.md) for more details.
+
+### Run Demo Video Generation
+
+A [mosquitto](https://mosquitto.org/) MQTT broker is required to generate the demo video.
 
 ```bash
-npm run build
-node dist/src/spec/webdriverio.js
+yarn build
+yarn ui-test
 ```
 
 ## Create a release

app/src/actions/Publish.ts

@@ -14,26 +14,27 @@ export const setTopic = (topic?: string): Action => {
   }
 }
 
-export const openFile = (encoding: 'utf8' = 'utf8') => async (dispatch: Dispatch<any>, getState: () => AppState) => {
-  try {
-    const file = await getFileContent(encoding)
-    if (file) {
-      dispatch(
-        setPayload(file.data))
+export const openFile =
+  (encoding: 'utf8' = 'utf8') =>
+  async (dispatch: Dispatch<any>, getState: () => AppState) => {
+    try {
+      const file = await getFileContent(encoding)
+      if (file) {
+        dispatch(setPayload(file.data))
+      }
+    } catch (error) {
+      dispatch(showError(error))
     }
-  } catch (error) {
-    dispatch(showError(error))
   }
-}
 
 type FileParameters = {
-  name: string,
+  name: string
   data: string
 }
 async function getFileContent(encoding: string): Promise<FileParameters | undefined> {
   const rejectReasons = {
     noFileSelected: 'No file selected',
-    errorReadingFile: 'Error reading file'
+    errorReadingFile: 'Error reading file',
   }
 
   const { canceled, filePaths } = await rendererRpc.call(makeOpenDialogRpc(), {

app/src/components/Sidebar/CodeDiff/ChartPreview.tsx

@@ -52,24 +52,28 @@ function ChartPreview(props: Props) {
       />
     </Tooltip>
   ) : (
-      <Tooltip title="Add to chart panel, not enough data for preview">
-        <ShowChart
-          onClick={onClick}
-          className={props.classes.icon}
-          style={{ color: '#aaa' }}
-          data-test-type="ShowChart"
-          data-test={props.literal.path}
-        />
-      </Tooltip>
-    )
+    <Tooltip title="Add to chart panel, not enough data for preview">
+      <ShowChart
+        onClick={onClick}
+        className={props.classes.icon}
+        style={{ color: '#aaa' }}
+        data-test-type="ShowChart"
+        data-test={props.literal.path}
+      />
+    </Tooltip>
+  )
 
   return (
     <span>
       {addChartToPanelButton}
       <Popper open={open} anchorEl={chartIconRef.current} placement="left-end">
         <Fade in={open} timeout={300}>
           <Paper style={{ width: '300px' }}>
-            {open ? <TopicPlot node={props.treeNode} history={props.treeNode.messageHistory} dotPath={props.literal.path} /> : <span />}
+            {open ? (
+              <TopicPlot node={props.treeNode} history={props.treeNode.messageHistory} dotPath={props.literal.path} />
+            ) : (
+              <span />
+            )}
           </Paper>
         </Fade>
       </Popper>

events/Events.ts

@@ -55,10 +55,10 @@ export const getAppVersion: RpcEvent<void, string> = {
   topic: 'getAppVersion',
 }
 
-export const writeToFile: RpcEvent<{ filePath: string, data: string, encoding?: string }, void> = {
+export const writeToFile: RpcEvent<{ filePath: string; data: string; encoding?: string }, void> = {
   topic: 'writeFile',
 }
 
-export const readFromFile: RpcEvent<{ filePath: string, encoding?: string }, Buffer> = {
+export const readFromFile: RpcEvent<{ filePath: string; encoding?: string }, Buffer> = {
   topic: 'readFromFile',
-}
+}

events/OpenDialogRequest.ts

@@ -11,4 +11,4 @@ export function makeSaveDialogRpc(): RpcEvent<SaveDialogOptions, SaveDialogRetur
   return {
     topic: 'saveDialog',
   }
-}
+}

package.json

@@ -12,6 +12,7 @@
     "test": "yarn test:app && yarn test:backend",
     "test:app": "cd app && yarn test",
     "test:backend": "cd backend && yarn test",
+    "test:ui": "tsc && mocha --require source-map-support/register dist/src/spec/ui-tests.spec.js",
     "test:mcp": "tsc && node dist/src/spec/testMcpIntrospection.js",
     "install": "cd app && yarn && cd ..",
     "dev": "npm-run-all --parallel dev:*",

scripts/runUiTests.sh

@@ -0,0 +1,39 @@
+#!/bin/bash
+set -e
+
+function finish {
+  set +e
+  echo "Exiting, cleaning up.."
+
+  if [[ ! -z "$PID_MOSQUITTO" ]]; then
+    echo "Stopping mosquitto ($PID_MOSQUITTO).."
+    kill "$PID_MOSQUITTO" || echo "Already stopped"
+  fi
+
+  if [[ ! -z "$PID_XVFB" ]]; then
+    echo "Stopping XVFB ($PID_XVFB).."
+    kill "$PID_XVFB" || echo "Already stopped"
+  fi
+}
+
+trap finish EXIT
+
+DIMENSIONS="1024x720"
+SCR=99
+
+# Start new window manager
+Xvfb :$SCR -screen 0 "$DIMENSIONS"x24 -ac &
+export PID_XVFB=$!
+sleep 2
+
+# Start mqtt broker
+mosquitto &
+export PID_MOSQUITTO=$!
+sleep 1
+
+# Run UI tests
+DISPLAY=:$SCR yarn test:ui
+TEST_EXIT_CODE=$?
+
+echo "UI tests exited with $TEST_EXIT_CODE"
+exit $TEST_EXIT_CODE