Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions .github/workflows/demo-report.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
name: Demonstrate Reporting
on:
# still run checks on every PR *before* the merge
pull_request:

# run the workflow once more **after** the PR has been merged
push:
branches:
- main
- development

jobs:
build:
runs-on: macos-latest
timeout-minutes: 120
steps:
- name: Checkout (default)
if: github.event_name != 'pull_request'
uses: actions/checkout@v4
with:
fetch-depth: 0
submodules: recursive

# For pull_request_target, explicitly fetch the PR head
- name: Checkout PR head (for pull_request)
if: github.event_name == 'pull_request'
uses: actions/checkout@v4
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
submodules: recursive
- uses: actions/setup-java@v3
with:
distribution: 'temurin'
java-version: '17'
- uses: maxim-lobanov/setup-xcode@v1
with:
xcode-version: 16.2.0
- name: Run tests
run: ./gradlew allTests || true
- name: Test Report
uses: dorny/test-reporter@v2
env:
NODE_OPTIONS: --max-old-space-size=8192
if: success() || failure()
with:
name: Tests
path: '**/build/test-results/**/TEST*.xml'
reporter: java-junit
list-suites: failed
list-tests: failed
use-actions-summary: true
fail-on-error: false
27 changes: 18 additions & 9 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
name: Publish
on: workflow_dispatch
on:
workflow_dispatch:
inputs:
modules:
description: 'Comma-separated list of module names to publish'
required: false
default: 'internals,commons,matrix,freespec,property,datatest,fixturegen,fixturegen-freespec'
permissions:
contents: read
pages: write
Expand All @@ -23,15 +29,18 @@ jobs:
- name: Publish modules serially to Sonatype
run: |
./gradlew clean
./gradlew :internals:publishToSonatype closeSonatypeStagingRepository
./gradlew :commons:publishToSonatype closeSonatypeStagingRepository
./gradlew :matrix:publishToSonatype closeSonatypeStagingRepository
./gradlew :freespec:publishToSonatype closeSonatypeStagingRepository
./gradlew :property:publishToSonatype closeSonatypeStagingRepository
./gradlew :datatest:publishToSonatype closeSonatypeStagingRepository
./gradlew :fixturegen:publishToSonatype closeSonatypeStagingRepository
./gradlew :fixturegen-freespec:publishToSonatype closeSonatypeStagingRepository
IFS=',' read -ra MODULES <<< "$PUBLISH_MODULES"
for module in "${MODULES[@]}"; do
module="$(echo "$module" | xargs)"
[ -z "$module" ] && continue
case "$module" in
*[!a-zA-Z0-9_-]*)
echo "::error::Invalid module name: '$module'"; exit 1 ;;
esac
./gradlew ":${module}:publishToSonatype" closeSonatypeStagingRepository
done
env:
PUBLISH_MODULES: ${{ github.event.inputs.modules }}
ORG_GRADLE_PROJECT_signingKeyId: ${{ secrets.PUBLISH_SIGNING_KEYID }}
ORG_GRADLE_PROJECT_signingKey: ${{ secrets.PUBLISH_SIGNING_KEY }}
ORG_GRADLE_PROJECT_signingPassword: ${{ secrets.PUBLISH_SIGNING_PASSWORD }}
Expand Down
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,22 @@
# Changelog

## 0.12.0
* **Matrix testing: nameless `data` / `property` layers**
* `data` and `property` now have overloads that omit the leading name (e.g. `data(listOf(...)) test { ... }`,
`property(Arb.int()) - { ... }`). A nameless layer skips the intermediate grouping node and registers its rows
directly in the surrounding scope. A named layer remains shorthand for wrapping a nameless one in a labeled
suite. Available in both regular and `compact` scopes, across all overloads (Iterable/Sequence,
`replayIndex(es)` / `replay(s)`, `- { }` and `test { }`).
* **Matrix testing: replay info shows layer kind**
* `Error replay info` frames are now tagged with their layer kind — `(property)` or `(data)` — for named and
nameless layers alike (e.g. `(property) seed: ...`). Nameless layers print the marker without a name; the
general failure/report path still shows no synthesized segment.
* **Matrix testing: replace layer separator slash with TestBalloon separator arrow**
* **Matrix testing: replay info shows the full path**
* The `Error replay info` path now mirrors the compact report path in full — enclosing grouping suites
(`"name" - { ... }`) and plain `test` leaves appear as path segments, not just the replayable `data`/`property`
layers. Only replayable layers still emit a `- ...` argument line; group-only chains produce no replay block.

## 0.11.0
* **Matrix testing: bounded compact concurrency**
* `CompactConcurrency.Shared(n)` runs a compact block through one compact-wide worker budget, so nested virtual
Expand Down
89 changes: 73 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,23 @@ val quickstart by matrixSuite(execution = ExecutionMode.Sequential) {
}
```

A layer's leading name is just a label for a grouping node that wraps the rows it generates. Naming a layer is
shorthand for wrapping a nameless one in a labeled suite by hand — these two produce the same test-tree shape:

```kotlin
data("numbers", listOf(1, 2, 3)) - { /* … */ } // named layer

"numbers" - { // equivalent structure
data(listOf(1, 2, 3)) - { /* … */ }
}
```

Both nest the three rows under a `numbers` node. Drop the name entirely (`data(listOf(1, 2, 3))`) and the rows attach
directly to the surrounding scope with no grouping node at all. The name, when present, additionally becomes the
layer's label in failure/replay output (`(data) numbers: …`); a nameless layer just shows its kind (`(data) …`). The
same applies to `property`. The sections below use named layers throughout, but every `data`/`property` form has a
nameless overload.

### Data-Driven Matrix Layers

`data` layers accept `Iterable` values and lazy `Sequence` values. Use `test { ... }` when each row is the test, and
Expand All @@ -114,7 +131,7 @@ val dataDrivenMatrix by matrixSuite(execution = ExecutionMode.Sequential) {
number shouldBeGreaterThan 0
}

data("as a dimension", listOf("foo", "bar")) - { word ->
data("string data", listOf("foo", "bar")) - { word ->
"has a length" {
word.length shouldBeGreaterThan 0
}
Expand Down Expand Up @@ -189,15 +206,16 @@ all generated checks: compact progress: 512 of 900 queued completed (1200 source

Every matrix failure — a real test-tree leaf or a compacted virtual row — carries an **`Error replay info`**
block whose detail lines are valid `property` / `data` arguments. Copy them back onto the failing layers and the
matrix re-runs exactly those cases.
matrix re-runs exactly those cases. Each frame is tagged with its layer kind — `(property)` or `(data)` — so the
marker still pinpoints the right layer even for nameless layers (see below), where no layer name is printed.

```
at.asitplus.AssertionError: 360888 should be < 256000
Error replay info: first: 4: 2018089192 / second: 2: 2796 / third: 1: 2 / fourth: 3: 1
- first: replay = ReplayInput(seed=4779463605442148766L, iteration=4L)
- second: replay = ReplayInput(seed=-1353176301820643450L, iteration=2L)
- third: replayIndex = 1L
- fourth: replay = ReplayInput(seed=5014696554795393980L, iteration=3L)
Error replay info: (property) first: 4: 2018089192 ↘ (property) second: 2: 2796 ↘ (data) third: 1: 2 ↘ (property) fourth: 3: 1
- (property) first: replay = ReplayInput(seed=4779463605442148766L, iteration=4L)
- (property) second: replay = ReplayInput(seed=-1353176301820643450L, iteration=2L)
- (data) third: replayIndex = 1L
- (property) fourth: replay = ReplayInput(seed=5014696554795393980L, iteration=3L)
```

Paste the text after each layer name into that layer's call:
Expand Down Expand Up @@ -225,30 +243,69 @@ Each pinned layer collapses to just the recorded case, so the whole matrix narro
* A layer's `replay` / `replayIndex` is independent of its `seed`: `seed` pins a deterministic *full* run, while
`replay` selects specific recorded cases (which carry their own seeds).

Replay info inherits the enclosing layers' frames, so a compacted leaf records the full chain (outer real layers
included), and the data index is recorded even when a custom `nameFn` omits it from the displayed name.
The path on the first line mirrors the compact report path in full: it includes the enclosing grouping suites
(`"name" - { ... }`) as plain segments, with the `(property)` / `(data)` layers carrying the markers. Only the
replayable layers get a `- ...` argument line below — grouping suites appear in the path but have nothing to paste.
A compacted leaf therefore records the full chain (outer real layers included), and the data index is recorded even
when a custom `nameFn` omits it from the displayed name.

### Fixtures

`fixture` creates fresh values for each directly nested test or suite. It also works inside compact blocks.
`fixture { ... }` produces a fresh value, and **every test or suite you declare inside its `- { ... }` block gains an
extra lambda parameter that hands you that value**. You name that parameter yourself — it is the freshly generated
fixture, scoped to the element it is attached to:

```kotlin
fixture { Random.nextBytes(16) } - {
// ^ open the fixture scope with `- { }`

"a single test" { bytes -> // <-- `bytes` is the fixture, freshly generated for THIS test
bytes.size shouldBe 16
}

"a nested suite" - { bytes -> // <-- `bytes` is one fixture, shared by everything in this suite
data("word", listOf("foo", "bar")) test { word ->
bytes.isNotEmpty() shouldBe true
}
}
}
```

The parameter appears on **every** nested form — `"name" { value -> }`, `"name" - { value -> }`,
`test(...) { value -> }`, and `testSuite(...) { value -> }` — so the fixture threads in wherever you declare a child.

Freshness follows the element the lambda is attached to:

* a **terminal test** (`"name" { value -> }` / `test { value -> }`) gets its **own** fresh value;
* a **suite** (`"name" - { value -> }`) gets **one** fresh value that all of its children share.

So fixtures isolate per test by default, but give you a single shared instance when you group children under a suite.

Directly inside the fixture block you can only declare tests and suites — not `data`/`property`/`compact` layers. To add
those, open a suite first: a `"name" - { value -> ... }` body is a normal matrix scope, so every layer you declare in it
closes over that suite's `value`. Nesting composes the other direction too: a `fixture` inside a `data` row regenerates
once per row, and a `fixture` inside another fixture's suite threads both values down to the leaves.

```kotlin
val fixtureMatrix by matrixSuite(execution = ExecutionMode.Concurrent()) {
fixture { Random.nextBytes(16) } - {
"regular test with fresh bytes" { bytes ->
"regular test with fresh bytes" { bytes -> // fresh value, this test only
bytes.size shouldBe 16
}

"suite with a fresh fixture" - { bytes ->
data("word", listOf("foo", "bar")) test { word ->
"suite with a shared fixture" - { bytes -> // one value for the whole suite; `- { }` opens a matrix scope,
data("word", listOf("foo", "bar")) test { word -> // so data/property/compact layers are available here
bytes.isNotEmpty() shouldBe true
word.length shouldBeGreaterThan 0
}
}
}

compact("compact checks with fresh fixture") { report = CompactReport.AllCases } - {
data("byte", bytes.toList()) test { byte ->
byte shouldBe byte
// Fixtures also work inside compact blocks — same parameter, same freshness rules:
compact("compact checks with fresh fixture") { report = CompactReport.AllCases } - {
fixture { Random.nextBytes(16) } - {
"fresh bytes per row" { bytes ->
bytes.size shouldBe 16
}
}
}
Expand Down
2 changes: 1 addition & 1 deletion gradle.properties
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ kotlin.code.style=official
org.gradle.jvmargs=-Xmx10g -Dfile.encoding=UTF-8 -Xms200m
kotlin.daemon.jvm.options=-Xmx10G -Xms200m
kotlin.daemon.jvmargs=-Xmx10g -Xms200m
artifactVersion = 0.11.0
artifactVersion = 0.12.0
jdk.version=17
android.minSdk=21
android.compileSdk=34
Expand Down
Loading
Loading