Skip to content

Stop promising a review box that nothing unchecks - #214

Merged
MarkusPaulsen merged 2 commits into
mainfrom
docs/pr-template-language-and-review-claim
Aug 26, 2026
Merged

Stop promising a review box that nothing unchecks#214
MarkusPaulsen merged 2 commits into
mainfrom
docs/pr-template-language-and-review-claim

Conversation

@MarkusPaulsen

@MarkusPaulsen MarkusPaulsen commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

The Review progress comment promised that pushing a commit unchecks the affected review
box. Nothing in this repository does that, so a ticked box said less than a reviewer
believed. It no longer claims it. The opening comment now asks for British English, and
the checklist asks for the Javadoc rule AGENTS.md already requires.

Linked issues

No linked issues.

1. Problem

Three things in the pull request template, all in the instructions it gives rather than in
code.

The Review progress comment stated that pushing a commit unchecks the affected box. No
workflow here touches a description: the seven are actionlint, codeql, examples,
issue-labeler, maven, pullrequest-labeler and pullrequest-template. The sentence was
inherited from Artemis, where a bot does it. A reviewer who believed it would read a
ticked box as evidence that the tick had survived the commits under it.

The template never said which variety of English to write in, so it was left to chance.

AGENTS.md requires Javadoc on every method and every field, each under 500 characters and
in simple words. The checklist never mentioned it, so the one convention a reviewer most
often has to raise by hand was the one the template stayed silent about.

2. Improvement from the user's perspective

No Improvement from the user's perspective.

3. Improvement from the maintainer's perspective

A tick in Review progress now means only what somebody actually ticked, rather than
carrying a promise about later commits that nothing keeps.

Contributors are told which variety of English to write in before a reviewer asks, and are
reminded of the Javadoc rule where the rest of the checklist is read rather than in a
review comment afterwards.

4. Testing manual

Prerequisites

  1. A checkout of this branch and a JDK 21 or newer, run from the repository root. Steps 2
    and 3 need nothing but a browser.

Steps

  1. Not reproducible from an exercise. Run
    PR_BODY="$(cat .github/PULL_REQUEST_TEMPLATE.md)" java .github/scripts/CheckPullRequestTemplate.java
    and read every line it prints.
  2. Run grep -ril uncheck .github/.
  3. Open .github/PULL_REQUEST_TEMPLATE.md on this branch in the browser and read the
    checklist and the opening comment.

Expected result

  1. Seven errors, each naming a section of the blank template that is still blank, and none
    naming a missing, a duplicated or an undefined heading. This pull request does not
    touch .github/scripts/CheckPullRequestTemplate.java, and that output is what says the
    template still matches it.
  2. No output. Nothing under .github unchecks a review box, which is why the sentence
    promising it is gone. This is evidence about this repository only: an organisation-level
    GitHub App cannot be ruled out from a checkout, and that is exactly why the template
    should not have stated it as a fact.
  3. The checklist carries a Javadoc line between the tests line and the documentation line,
    linking to the "Documenting Java" section of AGENTS.md. The opening comment asks for
    British English on the line after the one about tick boxes. The Review progress comment
    is two lines shorter and says only that both boxes should be ticked before merge.

Negative case (what must still be rejected)

The check must not have become more permissive. Confirm that PR_BODY="" java .github/scripts/CheckPullRequestTemplate.java still reports that the body is empty and
exits 1, and that deleting any one heading from a copy of the filled template still fails
with a missing section. This description was written against the changed template, so the
pr-template check on this pull request is the change exercising itself.

Modes exercised

No mode-specific behaviour changed.

5. Test case coverage regarding this PR

No production Java code changed.

Breaking changes and migration

No breaking changes or migration.

Checklist

  • Documentation (docs/, README.adoc, Javadoc) was updated where the change is user-facing.
  • CI is green, or every remaining failure is explained above.
  • No secrets, tokens or absolute local paths are contained in the diff.

Review progress

  • Code review
  • Manual test

Markus Paulsen added 2 commits August 26, 2026 11:11
The Review progress comment said that pushing a commit unchecks the affected
box. Nothing here does that: the workflows are actionlint, codeql, examples,
issue-labeler, maven, pullrequest-labeler and pullrequest-template, and none of
them touches the description. The sentence came from Artemis, where a bot does
it, and a reviewer who believed it would take a ticked box as evidence that the
tick survived the commits under it.

The opening comment now also asks for British English, which the template
otherwise left to chance.
AGENTS.md requires Javadoc on every method and field, under 500 characters, in
simple words. The checklist never mentioned it, so the one convention a reviewer
most often has to raise by hand was the one the template stayed silent about.
@MarkusPaulsen
MarkusPaulsen requested a review from a team August 26, 2026 09:14
@MarkusPaulsen
MarkusPaulsen requested review from a team and krusche as code owners August 26, 2026 09:14
@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 206e98fe-ae25-4e59-8454-788fbbd5cf71


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the other Automated area label: other label Aug 26, 2026
@MarkusPaulsen
MarkusPaulsen merged commit e70a128 into main Aug 26, 2026
15 of 17 checks passed
@MarkusPaulsen
MarkusPaulsen deleted the docs/pr-template-language-and-review-claim branch August 26, 2026 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

other Automated area label: other

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant