Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .cspell/project-words.txt
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,8 @@ streetsidesoftware
prerelease
workflow_dispatch
checkout
startgroup
endgroup
Comment thread
goanpeca marked this conversation as resolved.
Outdated
ncconfig
nvmrc

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ This folder contains:
- **`codeql.yml`**: CodeQL (SAST) static analysis of the TypeScript source. Runs on PRs to `main`, on push to `main`, and weekly; findings surface in the repo Security tab.
- **`full-lockfile-audit.yml`**: PR/push + weekly + manual `pnpm audit --audit-level high` across the full lockfile (dev/build tooling included). PR failures are informational; default-branch failures manage labeled tracking issues.
- **`full-lockfile-audit-heartbeat.yml`**: daily + manual check that a scheduled, manual, or main-push full-lockfile audit has fired recently; opens, updates, or closes a tracking issue for transient cron lapses after an audit has been observed.
- **`mutation-testing.yml`**: weekly + manual Stryker mutation testing against high-value action-owned parsing, dispatcher, filesystem-boundary, and error-aggregation targets. Default-branch failures manage a labeled tracking issue and upload the mutation report artifact when available.
- **`mutation-testing.yml`**: weekly + manual batched per-file Stryker mutation testing for the files listed in `stryker.conf.json`. Default-branch failures manage a labeled tracking issue and upload the `reports/mutation` JSON artifact when available.
- **`release.yml`**: fires on three-component `vX.Y.Z` tags (a bare `v1` does **not** trigger it): full gate + GitHub Release + floats the major-version tag (`v1`, `v2`, …).
- **`daily-smoke.yml`**: 03:13 UTC cron: real-B2 end-to-end smoke against the test bucket.
- **`large-multipart-smoke.yml`**: weekly real-B2 multipart upload + download SHA-1 integrity check for a payload above B2's recommended part size.
Expand Down
70 changes: 47 additions & 23 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,8 @@ pnpm lint:fix
pnpm typecheck # tsc --noEmit (strict + exactOptionalPropertyTypes)
pnpm test # vitest run: drives against the SDK's in-memory B2Simulator
pnpm test:coverage # same + the 95/85/100/95 coverage gate
pnpm test:mutation # Stryker mutation run against the Vitest suite
pnpm test:mutation # batched per-file Stryker mutation run + aggregate gate
pnpm test:mutation:single # raw Stryker run for focused local investigation
pnpm build # ncc build src/main.ts -o dist
pnpm run audit # pnpm audit --prod --audit-level high (CI gate; needs network)
pnpm spellcheck # cspell across src/, __tests__/, *.md, *.yml, action.yml
Expand Down Expand Up @@ -117,26 +118,46 @@ Interrupted local installs can leave a cache lock directory behind. The next run

## Mutation testing

`pnpm test:mutation` runs Stryker with the Vitest runner. It mutates the
action-owned parsing, dispatcher, filesystem-boundary, and error-aggregation
surfaces listed in the `mutate` array in
[`stryker.conf.json`](./stryker.conf.json).
`pnpm test:mutation` runs
[`scripts/run-batched-mutation.mjs`](./scripts/run-batched-mutation.mjs). The
wrapper runs Stryker once per file listed in the `mutate` array in
[`stryker.conf.json`](./stryker.conf.json), then aggregates the JSON reports and
applies the configured break threshold to the combined score. The per-file
invocation disables Stryker's own break exit so a below-threshold file does not
hide runner, config, or environment failures.

The current mutation scope is the explicit `mutate` list in
`stryker.conf.json`: action-owned `src/*.ts` support modules plus the command
implementations under `src/commands/*.ts` that are named there. Update that
single list when adding or removing mutation targets.

The mutation workflow is scheduled and manually dispatchable only; it is not a
per-PR gate while survivor triage is still being paid down. Reports are written
under `reports/mutation/` locally and uploaded as the `mutation-report`
artifact in CI. The workflow audits the full lockfile and rejects blocked
lookalike dependency names before installing the Stryker toolchain.
artifact in CI. The useful files are:

- `reports/mutation/by-file/*.json`: one Stryker JSON report per mutated file.
- `reports/mutation/aggregate.json`: the wrapper's combined score, threshold,
per-file rows, and status totals.
- `reports/mutation/mutation.json`: Stryker's shared JSON output path from the
last per-file run.

The workflow audits the full lockfile and rejects blocked lookalike dependency
names before installing the Stryker toolchain.

Stryker core and `@stryker-mutator/vitest-runner` are exact-pinned to the same
version because the runner plugin must stay in lockstep with core; the
Dependabot test-runner group updates them together. `stryker.conf.json` also
sets `vitest.related` to `false` so every mutant runs the full Vitest suite.
That is slower, but it avoids missing cross-file assertions in the shared
command fixtures and dispatcher tests while the mutation baseline is still
being triaged.

Initial baseline for this Stryker configuration:
Dependabot test-runner group updates them together. `pnpm test:mutation:single`
runs raw `stryker run` using the config reporters, which is useful for focused
local investigation. Do not use a full-scope raw run as the scheduled gate:
with this suite's `vi.resetModules()` + `vi.doMock()` + dynamic import pattern,
the Vitest runner can mis-attribute mutants when all files are instrumented in
one Stryker process. `stryker.conf.json` also sets `vitest.related` to `false`
so every mutant runs the full Vitest suite. That is slower, but it avoids
missing cross-file assertions in the shared command fixtures and dispatcher
tests while the mutation baseline is still being triaged.

Batched-runner baseline for the current mutation scope:

| Scope | Mutation score | Killed | Timed out | Survived | No coverage |
| --- | ---: | ---: | ---: | ---: | ---: |
Expand All @@ -157,14 +178,17 @@ Survivor triage from the baseline:
report should still be checked before adding disables because some survivors
can be equivalent mutants.

The configured mutation threshold is `break: 65`, with `low: 65` and
`high: 75`. That keeps the scheduled workflow passing the 72.83% baseline with
7.83 points of headroom while still failing on a material regression. Raise the
threshold only after survivors have been triaged and the baseline is re-run.
The headroom and scheduled-only cadence are an intentional bootstrap posture.
Once alerting has proven reliable, either tighten the break threshold toward
the baseline, or add a non-blocking PR information run so sub-threshold drift is
visible before the weekly cron.
The configured aggregate mutation threshold is `break: 65`, with `low: 65` and
`high: 75`. The batched wrapper enforces that threshold only when
`thresholds.break` is a number; setting it to `null` disables the aggregate
failure, matching Stryker's disabled-break semantics. The current 65% break gate
keeps the scheduled workflow passing the 72.83% baseline with 7.83 points of
headroom while still failing on a material regression. Raise the threshold only
after survivors have been triaged and the baseline is re-run. The headroom and
scheduled-only cadence are an intentional bootstrap posture. Once alerting has
proven reliable, either tighten the break threshold toward the baseline, or add
a non-blocking PR information run so sub-threshold drift is visible before the
weekly cron.

Default-branch scheduled and manual failures open or update one
`mutation-testing-failure` tracking issue through
Expand Down Expand Up @@ -212,7 +236,7 @@ listed in the same table and called out explicitly.
| `test` (matrix: ubuntu/macos/windows) | typecheck + vitest unit suite |
| `lint` | biome `--error-on-warnings` |
| `coverage` | vitest with v8 coverage, threshold 95 % statements / 85 % branches / 100 % functions / 95 % lines |
| `mutation-testing` ([mutation-testing.yml](./.github/workflows/mutation-testing.yml)) | Stryker mutation testing against the Vitest suite for high-value parsing, dispatcher, filesystem-boundary, and error-aggregation targets. Runs weekly and manually; it uploads the HTML/JSON report artifact, opens or updates a tracking issue on default-branch failure, and fails if the mutation score drops below the configured break threshold (65%). |
| `mutation-testing` ([mutation-testing.yml](./.github/workflows/mutation-testing.yml)) | Batched per-file Stryker mutation testing against the configured `stryker.conf.json` mutation scope. Runs weekly and manually; it uploads the JSON report artifact, opens or updates a tracking issue on default-branch failure, and fails if the aggregate mutation score drops below the configured break threshold (65%). |
| `build-and-check-dist` | ncc build, then `git diff --exit-code dist/`. **Drift fails CI**: rebuild with `pnpm build` and commit `dist/`. Bundle size is gated hard at 4 MiB. |
| `release-provenance-policy` ([security.yml](./.github/workflows/security.yml)) | parses release workflow YAML and enforces OIDC/attestation isolation, validated-SHA checkouts, tag re-verification, staged release asset upload, and post-upload verification. |
| `github-actions` ([security.yml](./.github/workflows/security.yml)) | runs the shared GitHub Actions security composite action against every workflow, including actionlint, third-party action pin checks, and zizmor audits (see [Pinning third-party actions](#pinning-third-party-actions)) |
Expand Down
115 changes: 115 additions & 0 deletions __tests__/commands/command-logs.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
import { rm, writeFile } from 'node:fs/promises'
import { join } from 'node:path'
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
import { headCommand } from '../../src/commands/head.ts'
import { hideCommand } from '../../src/commands/hide.ts'
import { unhideCommand } from '../../src/commands/unhide.ts'
import { tryStat } from '../../src/fs.ts'
import { captureStdout, makeFixture, makeInputs, seedFile, type TestFixture } from '../_helpers.ts'

// These tests pin the user-visible log surface (group labels + info lines) and
// the missing-source error wording for the small command wrappers, plus the
// tryStat error-swallowing contract. They exist to keep mutation coverage on
// code that is otherwise only exercised for its return value.
describe('head/hide/unhide log + error surface', () => {
let fx: TestFixture

beforeEach(async () => {
fx = await makeFixture('gh-action-logs')
})

afterEach(async () => {
await rm(fx.workDir, { recursive: true, force: true })
})

it('head: groups the probe and logs size/type/sha1', async () => {
await seedFile(fx, 'h.txt', 'head-me')
let result: Awaited<ReturnType<typeof headCommand>> | undefined
const out = await captureStdout(async () => {
result = await headCommand(fx.bucket, makeInputs('head', fx, { source: 'h.txt' }))
})
expect(result?.contentSha1).toBeTruthy()
expect(out).toContain('::group::head b2://gh-action-logs/h.txt')
expect(out).toContain(
`size=${result?.size} type=${result?.contentType} sha1=${result?.contentSha1}`,
)
expect(out).toContain('::endgroup::')
})

it('head: requires a source and names the action in the error', async () => {
await expect(headCommand(fx.bucket, makeInputs('head', fx))).rejects.toThrow(
"'source' input is required for 'head' action (the B2 file name)",
)
})

it('hide: groups the call and logs the created marker', async () => {
await seedFile(fx, 'g.txt', 'hide-me')
let result: Awaited<ReturnType<typeof hideCommand>> | undefined
const out = await captureStdout(async () => {
result = await hideCommand(fx.bucket, makeInputs('hide', fx, { source: 'g.txt' }))
})
expect(out).toContain('::group::hide b2://gh-action-logs/g.txt')
expect(out).toContain(`hidden: g.txt (marker fileId=${result?.fileId})`)
expect(out).toContain('::endgroup::')
})

it('hide: requires a source and names the action in the error', async () => {
await expect(hideCommand(fx.bucket, makeInputs('hide', fx))).rejects.toThrow(
"'source' input is required for 'hide' action (the B2 file name)",
)
})

it('unhide: logs the removed marker when one exists', async () => {
await seedFile(fx, 'u.txt', 'unhide-me')
await fx.bucket.hideFile('u.txt')
let result: Awaited<ReturnType<typeof unhideCommand>> | undefined
const out = await captureStdout(async () => {
result = await unhideCommand(fx.bucket, makeInputs('unhide', fx, { source: 'u.txt' }))
})
expect(result?.removedMarkerFileId).not.toBeNull()
Comment thread
goanpeca marked this conversation as resolved.
Outdated
expect(out).toContain('::group::unhide b2://gh-action-logs/u.txt')
expect(out).toContain(
`removed hide marker fileId=${result?.removedMarkerFileId}, u.txt is now visible`,
)
})

it('unhide: reports a no-op when there is no hide marker', async () => {
await seedFile(fx, 'v.txt', 'visible')
let result: Awaited<ReturnType<typeof unhideCommand>> | undefined
const out = await captureStdout(async () => {
result = await unhideCommand(fx.bucket, makeInputs('unhide', fx, { source: 'v.txt' }))
})
expect(result?.removedMarkerFileId).toBeNull()
expect(out).toContain('no hide marker found for v.txt (already visible or non-existent)')
})

it('unhide: requires a source and names the action in the error', async () => {
await expect(unhideCommand(fx.bucket, makeInputs('unhide', fx))).rejects.toThrow(
"'source' input is required for 'unhide' action (the B2 file name)",
)
})
})

describe('tryStat', () => {
let fx: TestFixture

beforeEach(async () => {
fx = await makeFixture('gh-action-try-stat')
})

afterEach(async () => {
await rm(fx.workDir, { recursive: true, force: true })
})

it('returns Stats for an existing path', async () => {
const p = join(fx.workDir, 'present.txt')
await writeFile(p, 'x')
const s = await tryStat(p)
expect(s?.isFile()).toBe(true)
})

it('returns undefined instead of throwing for a missing path', async () => {
const s = await tryStat(join(fx.workDir, 'does', 'not', 'exist.txt'))
expect(s).toBeUndefined()
})
})
63 changes: 62 additions & 1 deletion __tests__/commands/head-purge-multipresign.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import { type PresignedFile, presignCommand } from '../../src/commands/presign.t
import { purgeCommand } from '../../src/commands/purge.ts'
import { uploadCommand } from '../../src/commands/upload.ts'
import { setSummaryJsonOutput } from '../../src/outputs.ts'
import { captureStdout, makeFixture, makeInputs, type TestFixture } from '../_helpers.ts'
import { captureStdout, makeFixture, makeInputs, seedFile, type TestFixture } from '../_helpers.ts'

function inputs(action: Parameters<typeof makeInputs>[0], over: Record<string, unknown> = {}) {
return makeInputs(action, { bucket: 'gh-action-hpx', ...over })
Expand Down Expand Up @@ -83,6 +83,67 @@ describe('purge command', () => {
/'allow-bucket-purge' must be true/,
)
})

it('purges the whole bucket when allow-bucket-purge is set, warning loudly', async () => {
await seedFile(fx, 'a/x.txt', 'x')
await seedFile(fx, 'top.txt', 'y')
let result: Awaited<ReturnType<typeof purgeCommand>> | undefined
const out = await captureStdout(async () => {
result = await purgeCommand(fx.bucket, inputs('purge', { allowBucketPurge: true }))
})
expect(result?.errors).toBe(0)
expect(result?.files.length).toBeGreaterThanOrEqual(2)
expect(result?.files.every((f) => f.action === 'upload' && !f.skipped)).toBe(true)
expect(out).toContain('::warning::')
expect(out).toContain('permanently delete EVERY version in bucket "gh-action-hpx"')
expect(out).toContain('::group::purge b2://gh-action-hpx/ (all versions)')
expect(out).toMatch(/ {2}purged top\.txt \(/)
const after = await fx.bucket.listFileVersions({})
expect(after.files).toHaveLength(0)
})

it('previews a whole-bucket purge with no warning under dry-run', async () => {
await seedFile(fx, 'a/x.txt', 'x')
let result: Awaited<ReturnType<typeof purgeCommand>> | undefined
const out = await captureStdout(async () => {
result = await purgeCommand(
fx.bucket,
inputs('purge', { allowBucketPurge: true, dryRun: true }),
)
})
expect(result?.files.length).toBeGreaterThan(0)
expect(result?.files.every((f) => f.skipped && f.action === 'skip')).toBe(true)
expect(out).not.toContain('::warning::')
expect(out).toContain('::group::dry-run b2://gh-action-hpx/ (all versions)')
expect(out).toMatch(/ {2}would purge a\/x\.txt \(/)
const after = await fx.bucket.listFileVersions({})
expect(after.files.length).toBeGreaterThan(0)
})

it('appends a trailing slash to a prefix source that lacks one', async () => {
// 'log' must become prefix 'log/', so 'log/inside.txt' is purged but the
// sibling 'logs.txt' (which a bare 'log' prefix would also match) survives.
await seedFile(fx, 'log/inside.txt', 'in')
await seedFile(fx, 'logs.txt', 'out')
const result = await purgeCommand(fx.bucket, inputs('purge', { source: 'log' }))
expect(result.errors).toBe(0)
const inside = await fx.bucket.listFileVersions({ prefix: 'log/' })
expect(inside.files).toHaveLength(0)
const sibling = await fx.bucket.listFileVersions({ prefix: 'logs.txt' })
expect(sibling.files.length).toBeGreaterThan(0)
})

it('warns with the failed-purge message when a version cannot be deleted', async () => {
await seedFile(fx, 'pf/a.txt', 'a')
fx.sim.injectFailure({ on: 'b2_delete_file_version', status: 500, code: 'internal_error' })
let result: Awaited<ReturnType<typeof purgeCommand>> | undefined
const out = await captureStdout(async () => {
result = await purgeCommand(fx.bucket, inputs('purge', { source: 'pf/' }))
})
expect(result?.errors).toBeGreaterThanOrEqual(1)
expect(out).toContain('::warning::')
expect(out).toContain('failed to purge')
})
})

describe('presign command (prefix mode)', () => {
Expand Down
Loading
Loading