Skip to content

[#36] test(e2e): flag transposed orientation as a known flake and bail on first failure#37

Open
yuji-hatakeyama wants to merge 1 commit into
mainfrom
36-e2e_flag_transposed_orientation_flake
Open

[#36] test(e2e): flag transposed orientation as a known flake and bail on first failure#37
yuji-hatakeyama wants to merge 1 commit into
mainfrom
36-e2e_flag_transposed_orientation_flake

Conversation

@yuji-hatakeyama

@yuji-hatakeyama yuji-hatakeyama commented Jul 6, 2026

Copy link
Copy Markdown
Owner

assertDimensions now detects the transposed case and fails with an explicit "known flake — re-run" message instead of a bare expect(1536).toBe(1024). --bail stops the run on the first failure, so the multi-minute Test C (which depends on A/B output) doesn't run needlessly.

Closes #36

@yuji-hatakeyama
yuji-hatakeyama force-pushed the 36-e2e_flag_transposed_orientation_flake branch from e78dbf7 to 8aad9ed Compare July 6, 2026 15:25
Repository owner deleted a comment from chatgpt-codex-connector Bot Jul 6, 2026
@yuji-hatakeyama
yuji-hatakeyama force-pushed the 36-e2e_flag_transposed_orientation_flake branch from ac72e52 to 8aad9ed Compare July 6, 2026 15:54
@yuji-hatakeyama

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: 8aad9eda3e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@yuji-hatakeyama

Copy link
Copy Markdown
Owner Author

E2E test result

$ bun run test:e2e_subscription
$ OPENCODE_MODEL=openai/gpt-5.5 bun test --bail tests/e2e/subscription.test.ts
bun test v1.3.13 (bf2e2cec)

tests/e2e/subscription.test.ts:
WORKDIR: /var/folders/b4/_bwt79d95656w4zf8rlzjy400000gp/T/qa-imagegen-work-mbHXzZ
XDG_CONFIG_HOME: /var/folders/b4/_bwt79d95656w4zf8rlzjy400000gp/T/qa-imagegen-cfg-Ovuyxi
$ rm -rf dist
$ bun build ./src/index.ts --outdir ./dist --target node --format esm --packages external
Bundled 5 modules in 2ms

  index.js  7.94 KB  (entry point)


> build · gpt-5.5

Generating the requested portrait image to `character.png` with the specified subject, style, size, and quality.

⚙ gpt_imagegen {"prompt":"A man wearing a navy-blue samue and a red hachimaki headband, standing in a garden of cherry blossoms in full bloom. Hand-drawn 90s Japanese animation style, soft cel shading, expressive linework, nostalgic color palette, portrait composition.","out":"character.png","quality":"medium","size":"1024x1536","images":[]}

Generated `character.png` at the requested size and quality.

A: /var/folders/b4/_bwt79d95656w4zf8rlzjy400000gp/T/qa-imagegen-work-mbHXzZ/character.png
✓ gpt_imagegen e2e (subscription) > A. man portrait 1024x1536 [67125.55ms]

> build · gpt-5.5

Generating `character.png` with the requested scene, landscape size, and medium quality.

⚙ gpt_imagegen {"prompt":"A woman wearing a yellow yukata and holding a red wagasa parasol, standing in a garden at night with fireflies dancing around her. Hand-drawn 90s Japanese animation style, soft painted background, warm lantern-like firefly glow, atmospheric moonlit garden, elegant character pose, landscape composition.","out":"character.png","quality":"medium","size":"1536x1024","images":[]}

Generated the image, but `character.png` already existed, so the new file was saved as:

`character-v2.png`

B: /var/folders/b4/_bwt79d95656w4zf8rlzjy400000gp/T/qa-imagegen-work-mbHXzZ/character-v2.png
✓ gpt_imagegen e2e (subscription) > B. woman landscape 1536x1024 (auto-versioned) [166552.11ms]

> build · gpt-5.5

Generating the requested combined character image with the two provided references.

⚙ gpt_imagegen {"prompt":"Image 1: reference image for the man wearing a navy samue and red hachimaki. Image 2: reference image for the woman wearing a yellow yukata and holding a red wagasa. Create an image of the man from Image 1 and the woman from Image 2 standing side by side on the engawa veranda of an old Japanese house, smiling at the viewer. Preserve each character's outfit, hairstyle, and props exactly, including the man's navy samue and red hachimaki, and the woman's yellow yukata and red wagasa. Style: hand-drawn 90s Japanese animation style.","out":"together.png","quality":"medium","size":"2048x1152","images":["./character.png","./character-v2.png"]}

Generated `together.png`.

C: /var/folders/b4/_bwt79d95656w4zf8rlzjy400000gp/T/qa-imagegen-work-mbHXzZ/together.png
✓ gpt_imagegen e2e (subscription) > C. compose two characters from A and B references [171414.85ms]

 3 pass
 0 fail
 10 expect() calls
Ran 3 tests across 1 file. [405.17s]
character character-v2 together

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

e2e: surface transposed image orientation as a known flake and bail on first failure

1 participant