Skip to content

Say in the security policy what counts as a vulnerability - #213

Open
MarkusPaulsen wants to merge 1 commit into
mainfrom
docs/security-policy-scope-and-dangerous-code
Open

Say in the security policy what counts as a vulnerability#213
MarkusPaulsen wants to merge 1 commit into
mainfrom
docs/security-policy-scope-and-dangerous-code

Conversation

@MarkusPaulsen

Copy link
Copy Markdown
Collaborator

Summary

Adds two sections to the security policy: what in this repository is dangerous on purpose, and what counts as a vulnerability. Both come from the sibling SCORE repositories, which wrote them first. Two existing sentences are corrected in passing.

Linked issues

No linked issues

1. Problem

SECURITY.md said how to report a vulnerability but never said what one is.

Ares is a security tool whose own test tree attacks it on purpose. The classes under src/test/java/de/tum/cit/ase/ares/integration/testuser/subject/ open sockets, execute commands, spawn threads and reach for reserved packages, and the shipped agent and AspectJ instrumentation rewrite bytecode inside the supervised JVM. Anyone reading the repository can find code that looks like a finding, and neither they nor a maintainer had a written criterion for telling a defect apart from the apparatus behaving as designed.

The policy also promised more than GitHub documents, claiming a private report is encrypted and stays private only "until a fix is published".

Nothing is broken in the enforcement itself. This is a documentation gap.

2. Improvement from the user's perspective

Anyone who finds something alarming in Ares now has a written test for it. The policy names the operations the test fixtures attempt on purpose, and separates what is in scope, above all a containment bypass or a wrong enforcement verdict, from what is not, such as a gap on the far side of the documented enforcement boundary.

The most useful of those is the last out-of-scope entry: a finding that presumes an adversary who controls the build. That was already written down in the Ares 1 conversion guide, where it says the build descriptor and the command that invokes it are trusted instructor configuration. The policy now points at it rather than leaving it buried in a migration document nobody reads for this purpose.

3. Improvement from the maintainer's perspective

An incoming report can be triaged against a written criterion instead of a judgement call made afresh each time, and the three repositories of this family now answer the same questions under the same headings, so a reader moving between them does not have to relearn the document.

Two inaccurate claims are gone. The private reporting channel no longer promises encryption, and no longer says privacy ends when a fix is published. GitHub documents neither.

4. Testing manual

Prerequisites

  1. A browser, signed in to GitHub with read access to this repository. No checkout, no build and no local tooling are needed.

Steps

Not reproducible from an exercise. This changes one Markdown file and no code, so there is nothing an exercise can exercise. A reviewer verifies it by reading.

  1. Open the "Files changed" tab of this pull request and confirm the diff touches SECURITY.md and nothing else.
  2. In that diff, confirm the two added sections are "Deliberately dangerous code" and "Scope", and that exactly two existing sentences change: "behavior" becomes "behaviour", and the private reporting bullet loses its encryption and until-a-fix claims.
  3. Open docs/HowToConvertAnAres1ProjectIntoAnAres2Project.md and find the paragraph beginning "What this boundary does not defend against". Compare it with the third out-of-scope entry in the new "Scope" section.
  4. Open SECURITY.md on this branch as GitHub renders it and read the section headings in order.
  5. Look at the checks on this pull request once they have finished.

Expected result

  1. One file changed, roughly 29 lines added and 2 changed. If any other file appears, the branch is wrong.
  2. All three changes present, and no fourth change to existing prose.
  3. The conversion guide limits its statement to the reserved-package boundary. The policy entry must restate that without widening it: it has to name that boundary, not checks in general. If the policy reads as though any Ares check may be removed by whoever controls the build, it overclaims and should be sent back.
  4. Five headings, in this order: Supported Versions, Deliberately dangerous code, Scope, Reporting a bug, Reporting a Vulnerability.
  5. No new failure. No workflow, build file or Java source is touched, so a failure here would be pre-existing or unrelated. The checklist box for green CI is deliberately left unticked, because the checks had not run when this was opened.

Negative case (what must still be rejected)

The policy must not become an excuse. Confirm that the out-of-scope list covers only four things: the fixtures behaving as documented, an operation outside the documented enforcement boundary, an adversary who controls the build, and defects belonging upstream in the JVM, Maven, Gradle or a library.

Anything that would make Ares itself wrong must remain in scope, and the diff must still list all of these: supervised code performing an operation the active policy blocks, supervised code disabling or detaching the enforcement, an enforcement result that does not match what actually happened, a leaked credential, and a build-time download without checksum verification. If any of those has moved to the out-of-scope list, reject the change.

Modes exercised

No mode-specific behaviour changed.

  • ArchUnit + AspectJ
  • ArchUnit + instrumentation
  • WALA + AspectJ
  • WALA + instrumentation

5. Test case coverage regarding this PR

No production Java code changed.

Breaking changes and migration

No breaking changes or migration.

Checklist

  • Documentation (docs/, README.adoc, Javadoc) was updated where the change is user-facing. This pull request is that documentation change.
  • CI is green, or every remaining failure is explained above.
  • No secrets, tokens or absolute local paths are contained in the diff.

Review progress

  • Code review
  • Manual test

The policy said how to report a vulnerability but never what one is. For a
framework whose test tree exists to attack itself, that gap does real work:
neither a reporter nor a maintainer had a written criterion to decide whether
a finding was a defect or the apparatus behaving as designed.

Adds two sections taken from the sibling SCORE repositories, which had them
first. 'Deliberately dangerous code' names what is hostile on purpose, and
'Scope' lists what is in and out. The out-of-scope list defers to the threat
model already written down in the Ares 1 conversion guide: the build descriptor
is trusted instructor configuration, so an adversary who controls the build is
not the threat addressed.

Two existing sentences are corrected in passing. 'behavior' becomes 'behaviour',
and the private reporting channel no longer claims the report is encrypted or
that its privacy ends when a fix is published, neither of which GitHub
documents.
@MarkusPaulsen
MarkusPaulsen requested a review from a team August 25, 2026 16:45
@MarkusPaulsen
MarkusPaulsen requested review from a team and krusche as code owners August 25, 2026 16:45
@coderabbitai

coderabbitai Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: ccfe4850-86cf-4fb9-a4d4-ddf0be1c2b2e


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the docs Automated area label: docs label Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Automated area label: docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant