Detect the supervised package instead of defaulting to it - #174
Conversation
Gradle source-root discovery dropped every srcDirs declaration, because the SOURCE_DIRECTORY alternation tried the singular srcDir first and srcDir is a prefix of srcDirs. The capture then began at the leftover "s = [", which resolveGradlePath cannot resolve, so the plural branch was unreachable and a project declaring its main sources that way looked like a project with no production sources at all. Without a policy the supervised package is scanned rather than pinned, so it then fell through in one step to the configured default, "de.tum.cit.ase" for programming exercises. For an exercise in de.tum.cit.aet that resolved the analysis path to a directory which does not exist; ClassFileImporter answers an empty set for a missing directory, so ArchUnit analysed nothing and the aspects were armed over a package holding no class. Supervised code could then read files, execute commands and start threads unhindered while the run stayed green, under a configuration the setup manual documents as denying all three. Order the alternation longest-first, guard the singular branch with a negative lookahead, and accept += as well as =. Then read the compiled production output as a second detection step, so an unparsed descriptor no longer costs the supervised scope: the build tool writes its output where it says it does, whatever the descriptor looks like. Count only top-level classes, and skip blank and reserved packages as the source scan already does. The configured default remains as the last resort, and now says so in a warning naming the roots that were searched. It stays because Ares's own policy-free integration tests depend on that coarse prefix supervising their test-user classes; rescoping those fixtures changes what they assert and belongs in its own change.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📝 WalkthroughSummary by CodeRabbit
WalkthroughThe project scanner now derives supervised packages from source or compiled production classes, validates derived scope coverage, and uses configured defaults only as a final fallback. Gradle source discovery handles nested descriptors and assignment forms. Timeout advice identifies only ChangesProject scanning and derived scope enforcement
Timeout-thread attribution
Estimated code review effort: 4 (Complex) | ~60 minutes Mergeability Score: 🟡 Moderate · up to The PR improves supervised-package detection and enforcement, but valid Gradle source declarations can still fail during discovery, and the manual currently overstates the guarantees of derived-scope validation. These bounded issues can cause affected builds to fail or lead users to rely on inaccurate security documentation, so they should be addressed before merging. Possibly related issues
Possibly related PRs
Suggested labels: Suggested reviewers: Caution Pre-merge checks failedPlease resolve all errors before merging. Addressing warnings is optional.
❌ Failed checks (1 error, 1 warning)
✅ Passed checks (6 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Discovery and the package scan decide which code Ares supervises, so a path that no test takes is a way for the scope to be wrong with nothing to notice it. The gaps were concentrated exactly there: rejections, fallbacks and the legacy descriptor accessors. Adds ProjectSourcesFinderEdgeCaseTest for the paths that do not resolve: a project root that is not a directory, a selected build tool whose descriptor is absent, a project with no descriptor at all, a source root that is a file, a malformed Maven descriptor, an unreadable Gradle descriptor and unreadable Gradle properties, the conventional-root fallbacks, the Kotlin getByName source set, line-comment stripping, files(...) unwrapping, both quotation styles, a mismatched pair of quotation marks, relative and absolute Maven source directories, and the legacy accessors including both base-directory prefixes and the Kotlin descriptor. Extends the scanner tests with the compiled-output cases that the new fallback introduces: classes in the default package and in a reserved package are skipped, an absent or unreadable output root yields nothing to detect, and the configured build mode is reported. Adds two JUnit 3 resolution fixtures: a bare TestCase reached through a wildcard import, and a self-declared TestCase in a second file of the same package, which resolves through the package rather than through the compilation unit. Also drops a null check on productionOutputRoot(), which is @nonnull, so the condition could never hold.
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@src/test/java/de/tum/cit/ase/ares/api/util/ProjectSourcesFinderEdgeCaseTest.java`:
- Around line 423-426: Require permission-based tests to distinguish fixture
failures from sandbox limitations. In
src/test/java/de/tum/cit/ase/ares/api/util/ProjectSourcesFinderEdgeCaseTest.java:423-426,
add an assumeUnreadable(Path) helper beside assumePosix() and invoke it after
each permission removal in rejectsUnreadableGradleDescriptor,
rejectsUnreadableGradleProperties, and reportsNoLegacyGradleSourceDirectory. In
src/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScannerPackageFallbackTest.java:161-182
and :193-209, after changing permissions, abort when
Files.isReadable(outputRoot) or Files.isReadable(sourceRoot) remains true,
respectively.
- Around line 44-54: Protect mutations of the static descriptor paths in
ProjectSourcesFinderEdgeCaseTest by adding a shared `@ResourceLock` key to the
test class; apply the same lock to ProjectSourcesFinderTest and integration
tests that configure these paths through AresConfiguration or ClassNameScanner,
covering both rememberConfiguredPaths and restoreConfiguredPaths setup/teardown.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 70a933f0-7c26-4b94-a573-3ee1960abdbc
📒 Files selected for processing (4)
src/main/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScanner.javasrc/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScannerAstTest.javasrc/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScannerPackageFallbackTest.javasrc/test/java/de/tum/cit/ase/ares/api/util/ProjectSourcesFinderEdgeCaseTest.java
📜 Review details
⏰ Context from checks skipped due to timeout. (4)
- GitHub Check: Build
- GitHub Check: Run the gradle exercise
- GitHub Check: Run the maven exercise
- GitHub Check: Analyse Java
🧰 Additional context used
📓 Path-based instructions (5)
**/*Test.java
📄 CodeRabbit inference engine (AGENTS.md)
**/*Test.java: A sandboxed test JVM must never spin up its own server (echo server, socket listener, etc.) to test incoming or outgoing connections
Outgoing-connection tests must connect to an external echo server at a configurable endpoint running as a separate process or CI service on the loopback at port 25565, exercising only the student's client behaviour
If the external echo server is not reachable, the test must skip (using JUnit Assumptions.abort) rather than fail
An Ares SecurityException on an explicitly allowed connection is always a real failure and must propagate (never skipped)
Do not hard-code a self-hosted listener as the connection counterpart; use an external echo service to avoid in-JVM BindException/thread/lifecycle flakiness
Files:
src/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScannerAstTest.javasrc/test/java/de/tum/cit/ase/ares/api/util/ProjectSourcesFinderEdgeCaseTest.javasrc/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScannerPackageFallbackTest.java
**/*
⚙️ CodeRabbit configuration file
Dogmatically check all reviewed files for current British English in prose, comments, JavaDoc, documentation, workflow names, step names, issue/PR text, labels, user-facing messages, and review suggestions. Flag American spellings and grammar such as behavior, color, initialize, authorization, canceled, and program when they are natural-language text. Do not flag programming-language syntax, dependency coordinates, API names, class names, method names, package names, paths, URLs, quoted external identifiers, or other literals where American English is required by the technology.
Files:
src/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScannerAstTest.javasrc/test/java/de/tum/cit/ase/ares/api/util/ProjectSourcesFinderEdgeCaseTest.javasrc/main/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScanner.javasrc/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScannerPackageFallbackTest.java
src/test/java/**/*.java
⚙️ CodeRabbit configuration file
Require tests to distinguish fixture failures from sandbox failures. Network tests must not start in-process listeners inside the sandbox; external fixtures may be skipped when absent, but explicit Ares SecurityException failures must propagate.
Files:
src/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScannerAstTest.javasrc/test/java/de/tum/cit/ase/ares/api/util/ProjectSourcesFinderEdgeCaseTest.javasrc/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScannerPackageFallbackTest.java
src/main/java/**/*.java
⚙️ CodeRabbit configuration file
Review as a Java 17 Maven security framework used to test untrusted student code in Artemis programming exercises. Prioritise sandbox escapes, fail-open behaviour, unsafe reflection, classloader/bootstrap boundary mistakes, global mutable state, concurrency races, insufficient canonicalisation, and changes that weaken file, command, thread, network, package, or class access restrictions. Treat unrecognised security-sensitive inputs as a potential fail-closed requirement. Prefer simple Java code and one field or method declaration per line.
Files:
src/main/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScanner.java
src/main/java/de/tum/cit/ase/ares/api/securitytest/**/*.java
⚙️ CodeRabbit configuration file
Check that security-test factory, scanner, creator, and execution code cannot be influenced by student-controlled classes to bypass trusted-package, supervised-code, or test-class boundaries.
Files:
src/main/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScanner.java
🔇 Additional comments (5)
src/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScannerAstTest.java (1)
446-466: LGTM!src/test/java/de/tum/cit/ase/ares/api/util/ProjectSourcesFinderEdgeCaseTest.java (1)
58-66: LGTM!Also applies to: 145-320
src/main/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScanner.java (2)
432-481: LGTM!Also applies to: 515-530
493-498: 🩺 Stability & AvailabilityNo change needed.
BuildToolConfigurationrejectsnullforproductionOutputRootbefore storing it, soFiles.isDirectory(outputRoot)does not dereference a null path.src/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScannerPackageFallbackTest.java (1)
136-159: LGTM!Also applies to: 184-191
The permission-based tests assumed POSIX support but never that the bits are honoured. A superuser ignores them, so the code under test would keep reading the file and the assertion would report a defect where the fixture is what could not be established. Each site now checks that the target really is unreadable after the bits are cleared, and aborts otherwise, so such a run skips instead of failing.
getDefaultPackage() guesses the package of the supervised project when the project declares none. That guess is now de.tum.cit.aet. Only this one literal moves. Every other de.tum.cit.ase in src/main is Ares' own namespace, not a guess about someone else's code: FileTools resolves its own source directory through it, AOPMode and ArchitectureMode use it to identify literals inside copied Ares sources, and the essential-data YAMLs list Ares' own trusted packages. Renaming those would make Ares fail to find itself. Keep the fallback test meaningful. It used de.tum.cit.aet as the detected package precisely to contrast it with the default, so with both sides equal the assertion would have held just as well had the fallback fired, and the test would have proved nothing. Its fixture now sits in de.tum.cit.detected and it asserts against the new default explicitly. The neighbouring cases drive the base scanner, whose default is the empty string, so they stay as they are. Reword the two thread-advice comments that called de.tum.cit.ase the self-test fallback. It never was one; it is the broad scope Ares' own self-tests configure, and after this change the old wording would send a reader looking for a fallback that no longer has that value.
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScannerPackageFallbackTest.java (1)
247-248: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winMake the unavailable Java compiler a skipped fixture failure.
ToolProvider.getSystemJavaCompiler()can returnnullwhen the test JVM lacksjdk.compiler; line 248 then throwsNullPointerExceptioninstead of a test-scope fixturing failure. Confirm Maven surefire always uses a full JDK compiler, otherwise abort this test class withassumeTrue(compiler != null, "Java compiler required for fixture").🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScannerPackageFallbackTest.java` around lines 247 - 248, Update the fixture setup in JavaProjectScannerPackageFallbackTest around ToolProvider.getSystemJavaCompiler() to handle a null compiler before invoking compiler.run. Abort the test class with an assumption such as assumeTrue when the compiler is unavailable, preserving normal compilation behavior when it exists.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/securitytest/TestCaseFactoryAndBuilderManual.md`:
- Line 535: Update the documentation for scanForPackageName() to describe its
fallback order: production source roots, compiled production output, then the
default package. Explicitly document that the final default package may enforce
no security checks when the project does not contain that package, using British
English and avoiding overstated guarantees.
---
Outside diff comments:
In
`@src/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScannerPackageFallbackTest.java`:
- Around line 247-248: Update the fixture setup in
JavaProjectScannerPackageFallbackTest around
ToolProvider.getSystemJavaCompiler() to handle a null compiler before invoking
compiler.run. Abort the test class with an assumption such as assumeTrue when
the compiler is unavailable, preserving normal compilation behavior when it
exists.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 9b128e3d-7be2-411e-8d67-840d443d26eb
📒 Files selected for processing (6)
docs/securitytest/TestCaseFactoryAndBuilderManual.mdsrc/main/java/de/tum/cit/ase/ares/api/aop/java/aspectj/adviceandpointcut/JavaAspectJThreadSystemAdviceDefinitions.ajsrc/main/java/de/tum/cit/ase/ares/api/aop/java/instrumentation/advice/JavaInstrumentationAdviceThreadSystemToolbox.javasrc/main/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProgrammingExerciseProjectScanner.javasrc/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProgrammingExerciseProjectScannerTest.javasrc/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScannerPackageFallbackTest.java
📜 Review details
⏰ Context from checks skipped due to timeout. (4)
- GitHub Check: Analyse Java
- GitHub Check: Run the maven exercise
- GitHub Check: Run the gradle exercise
- GitHub Check: Build
🧰 Additional context used
📓 Path-based instructions (7)
**/*
⚙️ CodeRabbit configuration file
Dogmatically check all reviewed files for current British English in prose, comments, JavaDoc, documentation, workflow names, step names, issue/PR text, labels, user-facing messages, and review suggestions. Flag American spellings and grammar such as behavior, color, initialize, authorization, canceled, and program when they are natural-language text. Do not flag programming-language syntax, dependency coordinates, API names, class names, method names, package names, paths, URLs, quoted external identifiers, or other literals where American English is required by the technology.
Files:
src/main/java/de/tum/cit/ase/ares/api/aop/java/instrumentation/advice/JavaInstrumentationAdviceThreadSystemToolbox.javadocs/securitytest/TestCaseFactoryAndBuilderManual.mdsrc/main/java/de/tum/cit/ase/ares/api/aop/java/aspectj/adviceandpointcut/JavaAspectJThreadSystemAdviceDefinitions.ajsrc/main/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProgrammingExerciseProjectScanner.javasrc/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProgrammingExerciseProjectScannerTest.javasrc/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScannerPackageFallbackTest.java
src/main/java/**/*.java
⚙️ CodeRabbit configuration file
Review as a Java 17 Maven security framework used to test untrusted student code in Artemis programming exercises. Prioritise sandbox escapes, fail-open behaviour, unsafe reflection, classloader/bootstrap boundary mistakes, global mutable state, concurrency races, insufficient canonicalisation, and changes that weaken file, command, thread, network, package, or class access restrictions. Treat unrecognised security-sensitive inputs as a potential fail-closed requirement. Prefer simple Java code and one field or method declaration per line.
Files:
src/main/java/de/tum/cit/ase/ares/api/aop/java/instrumentation/advice/JavaInstrumentationAdviceThreadSystemToolbox.javasrc/main/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProgrammingExerciseProjectScanner.java
src/main/java/de/tum/cit/ase/ares/api/aop/**/*.java
⚙️ CodeRabbit configuration file
Focus on runtime enforcement integrity: intercepted methods, argument extraction, null handling, recursive advice guards, bootstrap classloader interaction, and whether denied operations can reach the JVM or operating system before checks run.
Files:
src/main/java/de/tum/cit/ase/ares/api/aop/java/instrumentation/advice/JavaInstrumentationAdviceThreadSystemToolbox.java
docs/**/*.md
⚙️ CodeRabbit configuration file
Use current British English. Check that security guarantees, known limitations, and required external fixtures are explicit and do not overstate enforcement.
Files:
docs/securitytest/TestCaseFactoryAndBuilderManual.md
src/main/java/de/tum/cit/ase/ares/api/securitytest/**/*.java
⚙️ CodeRabbit configuration file
Check that security-test factory, scanner, creator, and execution code cannot be influenced by student-controlled classes to bypass trusted-package, supervised-code, or test-class boundaries.
Files:
src/main/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProgrammingExerciseProjectScanner.java
**/*Test.java
📄 CodeRabbit inference engine (AGENTS.md)
**/*Test.java: A sandboxed test JVM must never spin up its own server (echo server, socket listener, etc.) to test incoming or outgoing connections
Outgoing-connection tests must connect to an external echo server at a configurable endpoint running as a separate process or CI service on the loopback at port 25565, exercising only the student's client behaviour
If the external echo server is not reachable, the test must skip (using JUnit Assumptions.abort) rather than fail
An Ares SecurityException on an explicitly allowed connection is always a real failure and must propagate (never skipped)
Do not hard-code a self-hosted listener as the connection counterpart; use an external echo service to avoid in-JVM BindException/thread/lifecycle flakiness
Files:
src/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProgrammingExerciseProjectScannerTest.javasrc/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScannerPackageFallbackTest.java
src/test/java/**/*.java
⚙️ CodeRabbit configuration file
Require tests to distinguish fixture failures from sandbox failures. Network tests must not start in-process listeners inside the sandbox; external fixtures may be skipped when absent, but explicit Ares SecurityException failures must propagate.
Files:
src/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProgrammingExerciseProjectScannerTest.javasrc/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScannerPackageFallbackTest.java
🔇 Additional comments (7)
src/main/java/de/tum/cit/ase/ares/api/aop/java/aspectj/adviceandpointcut/JavaAspectJThreadSystemAdviceDefinitions.aj (1)
151-153: LGTM!src/main/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProgrammingExerciseProjectScanner.java (1)
24-29: LGTM!Also applies to: 38-38
src/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProgrammingExerciseProjectScannerTest.java (1)
77-77: LGTM!Also applies to: 269-269
src/main/java/de/tum/cit/ase/ares/api/aop/java/instrumentation/advice/JavaInstrumentationAdviceThreadSystemToolbox.java (1)
147-149: LGTM!src/test/java/de/tum/cit/ase/ares/api/securitytest/java/projectScanner/JavaProjectScannerPackageFallbackTest.java (3)
33-98: LGTM!
100-235: LGTM!
237-246: LGTM!Also applies to: 249-252
Renaming the TUM default package broke SecurityTest.test_useCommonPoolGood, and the test was right. The self-tests declared @Policy without a policy file, so their supervised package came from the scanner's TUM default. That default happened to be a textual prefix of Ares' own packages, so the test subjects counted as student code by coincidence rather than by declaration. Once the default no longer matched, the classifier found no student frame, walked past the subject and reached TimeoutUtils, concluded the operation belonged to Ares' own timeout machinery and exempted it. Enforcement switched itself off and nothing failed but that one assertion. Pin the supervised package for the two self-test users whose assertions depend on their subject being supervised. The policy keeps the architecture and enforcement modes the no-policy branch selects, ArchUnit and AspectJ, so the backend under test does not change; it deliberately narrows the exempt set to the declared test classes and states where it departs from what the no-policy path derived. Key the timeout exemption to the frame that creates the worker. The student's body runs inside TimeoutUtils.rethrowThrowableSafe on that same worker, so TimeoutUtils is on the student's stack too, below their own frames. Accepting any frame of that class made the exemption depend on the restricted package being scoped correctly, which is precisely the assumption that failed. Only executeWithTimeout, which actually submits the invocation, now grants it. Verified against dumped StackWalker frames from both situations rather than inferred; the Javadoc claiming the student's stack contains no TimeoutUtils was wrong and is corrected. Cover both backends. The two advices carry this logic separately and are kept in step by hand, and the self-tests exercise only the restricted-package branch, so each backend gets a regression test that fails when the exemption is keyed to the class again.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@src/test/resources/de/tum/cit/ase/ares/integration/testuser/securitypolicies/java/maven/archunit/aspectj/PolicySelfTestDefaultRestrictive.yaml`:
- Around line 31-35: Update the empty YAML sequences for
regardingFileSystemInteractions, regardingNetworkConnections,
regardingCommandExecutions, regardingThreadCreations, and
regardingPackageImports to use [] without an internal space.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 2ee82a11-a50c-4412-a5c9-5a8fb2d1417e
📒 Files selected for processing (7)
src/main/java/de/tum/cit/ase/ares/api/aop/java/aspectj/adviceandpointcut/JavaAspectJThreadSystemAdviceDefinitions.ajsrc/main/java/de/tum/cit/ase/ares/api/aop/java/instrumentation/advice/JavaInstrumentationAdviceThreadSystemToolbox.javasrc/test/java/de/tum/cit/ase/ares/api/aop/java/aspectj/adviceandpointcut/JavaAspectJThreadSystemAdviceDefinitionsTest.javasrc/test/java/de/tum/cit/ase/ares/api/aop/java/instrumentation/advice/JavaInstrumentationAdviceThreadSystemToolboxTest.javasrc/test/java/de/tum/cit/ase/ares/integration/testuser/PackageAccessUser.javasrc/test/java/de/tum/cit/ase/ares/integration/testuser/SecurityUser.javasrc/test/resources/de/tum/cit/ase/ares/integration/testuser/securitypolicies/java/maven/archunit/aspectj/PolicySelfTestDefaultRestrictive.yaml
📜 Review details
⏰ Context from checks skipped due to timeout. (7)
- GitHub Check: Unit Tests
- GitHub Check: Core Integration Tests
- GitHub Check: Integration Tests (WALA + AspectJ)
- GitHub Check: Integration Tests (ArchUnit + instrumentation)
- GitHub Check: Integration Tests (WALA + instrumentation)
- GitHub Check: Run the gradle exercise
- GitHub Check: Analyse Java
🧰 Additional context used
📓 Path-based instructions (5)
**/*Test.java
📄 CodeRabbit inference engine (AGENTS.md)
**/*Test.java: A sandboxed test JVM must never spin up its own server (echo server, socket listener, etc.) to test incoming or outgoing connections
Outgoing-connection tests must connect to an external echo server at a configurable endpoint running as a separate process or CI service on the loopback at port 25565, exercising only the student's client behaviour
If the external echo server is not reachable, the test must skip (using JUnit Assumptions.abort) rather than fail
An Ares SecurityException on an explicitly allowed connection is always a real failure and must propagate (never skipped)
Do not hard-code a self-hosted listener as the connection counterpart; use an external echo service to avoid in-JVM BindException/thread/lifecycle flakiness
Files:
src/test/java/de/tum/cit/ase/ares/api/aop/java/aspectj/adviceandpointcut/JavaAspectJThreadSystemAdviceDefinitionsTest.javasrc/test/java/de/tum/cit/ase/ares/api/aop/java/instrumentation/advice/JavaInstrumentationAdviceThreadSystemToolboxTest.java
**/*
⚙️ CodeRabbit configuration file
Dogmatically check all reviewed files for current British English in prose, comments, JavaDoc, documentation, workflow names, step names, issue/PR text, labels, user-facing messages, and review suggestions. Flag American spellings and grammar such as behavior, color, initialize, authorization, canceled, and program when they are natural-language text. Do not flag programming-language syntax, dependency coordinates, API names, class names, method names, package names, paths, URLs, quoted external identifiers, or other literals where American English is required by the technology.
Files:
src/test/java/de/tum/cit/ase/ares/api/aop/java/aspectj/adviceandpointcut/JavaAspectJThreadSystemAdviceDefinitionsTest.javasrc/test/java/de/tum/cit/ase/ares/integration/testuser/SecurityUser.javasrc/main/java/de/tum/cit/ase/ares/api/aop/java/instrumentation/advice/JavaInstrumentationAdviceThreadSystemToolbox.javasrc/test/java/de/tum/cit/ase/ares/integration/testuser/PackageAccessUser.javasrc/test/java/de/tum/cit/ase/ares/api/aop/java/instrumentation/advice/JavaInstrumentationAdviceThreadSystemToolboxTest.javasrc/main/java/de/tum/cit/ase/ares/api/aop/java/aspectj/adviceandpointcut/JavaAspectJThreadSystemAdviceDefinitions.ajsrc/test/resources/de/tum/cit/ase/ares/integration/testuser/securitypolicies/java/maven/archunit/aspectj/PolicySelfTestDefaultRestrictive.yaml
src/test/java/**/*.java
⚙️ CodeRabbit configuration file
Require tests to distinguish fixture failures from sandbox failures. Network tests must not start in-process listeners inside the sandbox; external fixtures may be skipped when absent, but explicit Ares SecurityException failures must propagate.
Files:
src/test/java/de/tum/cit/ase/ares/api/aop/java/aspectj/adviceandpointcut/JavaAspectJThreadSystemAdviceDefinitionsTest.javasrc/test/java/de/tum/cit/ase/ares/integration/testuser/SecurityUser.javasrc/test/java/de/tum/cit/ase/ares/integration/testuser/PackageAccessUser.javasrc/test/java/de/tum/cit/ase/ares/api/aop/java/instrumentation/advice/JavaInstrumentationAdviceThreadSystemToolboxTest.java
src/main/java/**/*.java
⚙️ CodeRabbit configuration file
Review as a Java 17 Maven security framework used to test untrusted student code in Artemis programming exercises. Prioritise sandbox escapes, fail-open behaviour, unsafe reflection, classloader/bootstrap boundary mistakes, global mutable state, concurrency races, insufficient canonicalisation, and changes that weaken file, command, thread, network, package, or class access restrictions. Treat unrecognised security-sensitive inputs as a potential fail-closed requirement. Prefer simple Java code and one field or method declaration per line.
Files:
src/main/java/de/tum/cit/ase/ares/api/aop/java/instrumentation/advice/JavaInstrumentationAdviceThreadSystemToolbox.java
src/main/java/de/tum/cit/ase/ares/api/aop/**/*.java
⚙️ CodeRabbit configuration file
Focus on runtime enforcement integrity: intercepted methods, argument extraction, null handling, recursive advice guards, bootstrap classloader interaction, and whether denied operations can reach the JVM or operating system before checks run.
Files:
src/main/java/de/tum/cit/ase/ares/api/aop/java/instrumentation/advice/JavaInstrumentationAdviceThreadSystemToolbox.java
🪛 YAMLlint (1.37.1)
src/test/resources/de/tum/cit/ase/ares/integration/testuser/securitypolicies/java/maven/archunit/aspectj/PolicySelfTestDefaultRestrictive.yaml
[error] 31-31: too many spaces inside empty brackets
(brackets)
[error] 32-32: too many spaces inside empty brackets
(brackets)
[error] 33-33: too many spaces inside empty brackets
(brackets)
[error] 34-34: too many spaces inside empty brackets
(brackets)
[error] 35-35: too many spaces inside empty brackets
(brackets)
🔇 Additional comments (7)
src/main/java/de/tum/cit/ase/ares/api/aop/java/instrumentation/advice/JavaInstrumentationAdviceThreadSystemToolbox.java (1)
125-138: LGTM!Also applies to: 148-161
src/main/java/de/tum/cit/ase/ares/api/aop/java/aspectj/adviceandpointcut/JavaAspectJThreadSystemAdviceDefinitions.aj (1)
134-166: LGTM!src/test/java/de/tum/cit/ase/ares/api/aop/java/aspectj/adviceandpointcut/JavaAspectJThreadSystemAdviceDefinitionsTest.java (1)
1-61: LGTM!src/test/java/de/tum/cit/ase/ares/api/aop/java/instrumentation/advice/JavaInstrumentationAdviceThreadSystemToolboxTest.java (1)
8-22: LGTM!Also applies to: 172-205
src/test/java/de/tum/cit/ase/ares/integration/testuser/PackageAccessUser.java (1)
19-28: LGTM!src/test/java/de/tum/cit/ase/ares/integration/testuser/SecurityUser.java (1)
28-33: LGTM!src/test/resources/de/tum/cit/ase/ares/integration/testuser/securitypolicies/java/maven/archunit/aspectj/PolicySelfTestDefaultRestrictive.yaml (1)
1-30: LGTM!Also applies to: 36-37
The manual still described scanForPackageName as reading the sources alone, so it neither mentioned the compiled-output step this branch adds nor said what the last resort actually promises. Spell out all three steps and state plainly that the default enforces nothing when the project does not contain it: the analysis path then resolves to a directory that does not exist, no class is imported, and the log warning is the only signal. Write the new policy's empty sequences as [], the form most of the existing policies already use.
Claudia-Anthropica
left a comment
There was a problem hiding this comment.
@MarkusPaulsen The exact head and all retained discussion were reviewed, and the captured CI checks are green. However, policy-free package discovery still permits a student-controlled or incomplete package scan to exclude the actual assignment package from enforcement, creating a high-severity false negative.
The Javadoc opened by claiming the method derives the package "never from a default that the project does not contain" and then returned exactly such a default fifty lines later. The manual went further and presented the reserved prefix filter as protection against a student flooding the project to choose the enforcement scope. It is not: it stops only a reserved namespace from winning, and every other namespace is available. Enough classes under a package of the submitter's choosing make that package the derived scope, and the five resource domains are then enforced over it instead of over the assignment. Counting compiled classes in step two changes the unit that is counted, not who controls it. Two further overclaims go with it. The compiled output is not authoritative "whatever the build descriptor says": step two reads the build tool's conventional directory, so a build that writes its output elsewhere is not followed there. And the warning said no compiled class declared a package, where what happened is that none declared an eligible one, the default package and reserved packages having been filtered out first. None of this changes behaviour. It stops the code and the manual from promising a boundary where there is a heuristic, and it names the one form of the scope that cannot be steered from the submission: pinning it in the policy. The one behavioural fix is in the same area. compiledPackageCounts discarded every class whose binary name contains a dollar sign, on the reasoning that the dollar marks a nested or anonymous class. It is also a legal identifier character, so a top-level Payload$Hidden was dropped and its package undercounted by however many classes an author chose to name that way. Nesting now comes from the class file through isTopLevelClass, with a fixture that fails against the old test and passes against this one.
Claudia-Anthropica
left a comment
There was a problem hiding this comment.
@MarkusPaulsen The exact head and retained discussions were re-reviewed; captured CI is green. Policy-free discovery still permits a student-controlled or incomplete package plurality to exclude the actual assignment package from enforcement, and the latest reply confirms this defect is deferred rather than fixed here. The widened srcDirs parser also still interprets inactive text as configuration and can misclassify a one-line test source set as production, as documented in the unresolved current-head threads.
Making srcDirs resolve for the first time exposed what the line-oriented scan
around it could not tell apart, and one of those cases turned a build that
worked into one that aborts.
An occurrence of srcDirs that Gradle never executes was read as a declaration.
Only // comments were stripped, so a commented-out srcDirs = ['old/path'] in a
block comment, or one inside a string as in println "srcDirs = ...", became a
candidate path, and a declared root that is not a directory is rejected. On
main the same file was harmless, because the srcDir alternative consumed the
prefix and the leftover token resolved to nothing.
The source set was tracked per line. A declaration sharing its line with the
block that opens it, as in sourceSets { test { java { srcDirs = ['t'] } } },
was attributed to whatever the previous line left behind, so a test root
counted as production and the test list came back empty. The set was also
decided by asking whether the line contained the text "test" anywhere, which
made main { java { srcDir 'contest' } } a test root. And nothing required the
declaration to be inside a java block, so resources { srcDir 'assets' } became
a Java source root, which matters because JavaProjectScanner reads every .java
file under a production root without asking Gradle whether it compiles them.
All three are older than this branch, but they could not fire for a project
declaring its sources with srcDirs, which is the form the Artemis template
uses, so this branch is what puts them in reach.
The descriptor is now masked, so comments and the contents of string literals
cannot be mistaken for code, and then walked once with a stack of the enclosing
blocks. Masking preserves every length, so an offset in the mask is the same
offset in the original: the mask decides what is code, the original supplies
the values. Property assignments are read from it too, so one inside a comment
no longer defines a property. A declaration counts only inside sourceSets, then
main or test, then java.
Gradle's own semantics now decide what the declarations add up to. A source set
starts at its conventional root, srcDir, srcDirs(...) and srcDirs += add to it,
and srcDirs = replaces what came before. srcDirs += ['generated'] therefore
means the conventional root and the generated one rather than the generated one
alone, and two assignments are the last one rather than both.
One existing expectation changes with that. An assignment whose value cannot be
resolved now leaves the source set empty instead of answering the conventional
root: the descriptor has replaced that root, so naming it is a guess against
what the build says, and the supervised package comes from the compiled output
instead.
Claudia-Anthropica
left a comment
There was a problem hiding this comment.
@MarkusPaulsen The exact head and all retained discussion were reviewed. The acknowledged high-severity policy-free scope defect remains: student-controlled or incomplete package counts can exclude the actual assignment package from enforcement, leaving forbidden operations unblocked; its fix is deferred to #189. The captured Run the gradle exercise check also fails on this head, with no repository evidence establishing that failure as unrelated.
Three findings, all in the code added by the previous commit and all of them fair. PMD's AvoidBranchingStatementAsLastInLoop fired twice, where a guard clause ended its loop body with a continue. Both read better as the positive condition: the source-set search returns on the name it was looking for, and the expression-position check returns on the first character that is not whitespace. SpotBugs pointed out that Path.getFileName can answer null, which it does for a root path, so asking a descriptor whether its name ends in .kts could have dereferenced it. Caught by the Examples workflow rather than locally, because mvn test does not run the gates that mvn install does.
Claudia-Anthropica
left a comment
There was a problem hiding this comment.
@MarkusPaulsen The exact head and all retained discussion were reviewed; the newest descriptor-structure changes address the two parser regressions raised in the prior threads. The acknowledged high-severity policy-free scope defect remains unchanged: scanForPackageName() still selects a student-influenceable plurality or an unverified default, so forbidden operations in the real assignment package can remain outside enforcement. Several captured checks are still in progress, although the snapshot identifies none as required.
|
@coderabbitai full review Context for the re-review: the last CodeRabbit review on this pull request is from 2026-08-10 and predates three commits. All threads from it are resolved. Since then the branch gained the |
Claudia-Anthropica
left a comment
There was a problem hiding this comment.
@MarkusPaulsen The new head carries the derived scope into both generated modes, but the generated runtime inventory is still not trustworthy. It can inspect a stale target/classes instead of the actual Gradle output and filters attacker-controlled production packages before validation; generated package-import rules also still persist the heuristic scope instead of runtime-declared packages. These are high-severity policy-free false negatives, so the generated path remains unsafe.
Three ways the runtime inventory could still be wrong, all of them mine, all introduced with the inventory itself. The output root was picked by trying target/classes and then build/classes/java/main and taking whichever existed. Those look equivalent and are not: a Gradle project that once built with Maven still has a target/classes, and the rules would then have been checked against a stale tree that need not resemble what runs. The build descriptor says which tool owns the project, so it decides, and a project carrying both descriptors is refused rather than guessed at. The import filtered the framework's own paths and, for a pinned scope, the scope subtree, before the coverage check ran. Filtering first decides what is worth checking using package names, which are exactly what whoever adds files to the project controls: a class the filter removed never reached the check and would have run unsupervised while the suite passed. Nothing is filtered now. What is imported is what is checked, and after a successful check everything imported lies within the scope anyway. The generated package-import rules still carried the scope itself as a permission. A permission is matched as a prefix, so that grant covered the whole namespace below the scope and outlived the coverage check that ran beside it. A derived scope now asks at runtime for the packages the compiled output actually declares, which cannot be broader than the output, and the creator no longer writes the heuristic fall-back into the file at all. A pinned scope keeps what the instructor declared, because that is a statement rather than a reading. The three assertions added with them are about what the generated file does, not how it is formatted: that it does not import nothing, that a derived scope asks for both its classes and its permissions at runtime, and that a pinned one is left as written.
Claudia-Anthropica
left a comment
There was a problem hiding this comment.
@MarkusPaulsen The runtime inventory closes the previously reported generated self-import path, so that thread can be resolved. However, the helper still accepts and grants reserved production packages, and its pinned branch analyses classes outside the instructor-selected scope; these cause a high-severity false negative and a medium-severity false positive. The snapshot has no failed checks, while Unit Tests and both WALA integration jobs were still in progress and were not listed as required.
Importing the output whole was the right answer to a filter that decided what was worth checking using package names the submitter controls. Removing every filter was not: it also removed the reserved-namespace check and the pinned scope's own boundary, so the helper accepted a class named into a trusted namespace and then granted that namespace as an import permission, and a pinned policy analysed the whole output instead of the part the instructor selected. Both are back, and both now sit after the import rather than before it, which is the distinction that was missed the first time. A reserved class is refused during validation and is never turned into a permission whatever else went wrong. A pinned scope narrows the imported inventory to its own subtree, so the rules see what the instructor selected while the check above still sees everything. Ares' own prefix is written in two concatenated pieces. Copying this class into a supervised project rewrites the framework's package token wherever it appears, string literals included, so spelling it out would have turned that entry into the copied API's prefix and stopped it naming Ares at all. The copied API is recognised separately, from where this class itself ended up rather than from a literal that would meet the same fate.
Claudia-Anthropica
left a comment
There was a problem hiding this comment.
@MarkusPaulsen The current head fixes the reserved-declaration and pinned-scope findings, and generated ArchUnit/WALA cases now validate compiled output at runtime. However, runtime-derived self-import permissions still accept a production package that is an ancestor of a trusted namespace, recreating a high-severity package-import bypass. The snapshot records no failed checks; CodeQL, Build, and both example jobs were still in progress but were not listed as required.
The Javadoc on isThreadCreationFromAresTimeout described the stack walk as prose about one frame reached before another, which left three of the four classification rules unnamed and never said what the returned value goes on to do. It now states the question the method answers, lists the four rules in the order they are applied, and records that the order of the first two is load-bearing: TimeoutUtils lies inside the Ares prefix itself, so testing the prefix first would skip the very frame the exemption depends on. Two claims are corrected rather than restated. Refusing every TimeoutUtils method other than executeWithTimeout was justified by saying those frames run the student's body, which holds for rethrowThrowableSafe and for no other: findTimeout, terminateTimedOutExecution and generateTimeoutFailure are Ares-internal and create no worker. And the exemption's absence was said to fail every @StrictTimeout test under a thread-forbidding policy, which overstates it, since a refusal only hands the operation to the ordinary check, and that check finds no restricted-package frame on the creating stack. Documentation only. No behaviour changes.
Claudia-Anthropica
left a comment
There was a problem hiding this comment.
@MarkusPaulsen The latest commit only revises the timeout-exemption Javadocs, so the existing generated-package finding remains. allowedPackages() still accepts a runtime-derived de.tum.cit permission because line 141 checks only whether the package lies inside a trusted prefix; validated() accepts that broad scope, and the resulting prefix grant covers de.tum.cit.ase.ares.api in both generated modes. Apply the ancestor-of-reserved and copied-API checks before adding each runtime-derived permission. No failed checks were captured; CodeQL, Build, and both example jobs were still running but are not listed as required.
The scope was assigned after construction, through a setter three builders and one direct construction site had to remember to call. Two of them did not: one was caught in review, and JavaArchitectureTestCase's own writeArchitectureTestCase was not. A test case carrying no scope refuses to write a file, so forgetting the second step produced an object that looks built and fails later, at the one moment where failing means no enforcement was generated. Both values now travel through every constructor of the three classes, the two fields are final, the setter is gone, and all three build() methods return their instance directly. What a reader had to notice is now what the compiler asks for. Fixing the forgotten branches surfaced an older defect underneath them. JavaArchunitTestCase.builder() and JavaWalaTestCase.builder() declare no such method: Java inherits statics by name, so both resolved to the one on JavaArchitectureTestCase and returned a base-class builder. Each branch of writeArchitectureTestCase therefore built another base-class instance and delegated to itself until the stack ran out. Neither mode has ever worked from this entry point. They now use archunitBuilder() and walaBuilder(), and carry allowedClasses along with the scope. Two tests pin both halves. Nothing changes on the production path, which converts through ArchitectureMode and never reached these branches.
Claudia-Anthropica
left a comment
There was a problem hiding this comment.
@MarkusPaulsen The runtime helper still turns a broad derived package such as de.tum.cit into a prefix permission, so the unresolved trusted-namespace bypass remains on this head. The latest signature refactor also removes released public entry points, breaking existing consumers; I left that as a new medium finding. The snapshot has no failed checks, while CodeQL analysis, Build, and both example jobs were still in progress and are not listed as required.
Every other entry in the reserved-package list names either the Java platform or a library that performs the supervision: declaring code into one of them would have it mistaken for the machinery meant to inspect it. That is the whole membership rule, and anonymous.toolclasses. and metatest. never met it. They are the test helpers of one downstream consumer, the reproducibility package, so every Ares user refused two ordinary package names on that consumer's behalf, with a diagnostic about trusted namespaces that means nothing in their project. Removed from the list, from its hand-kept copy in JavaArchunitSupervisedClasses and from the three build-boundary fixtures. They stay in INFRA_PREFIXES, which answers a different question: not what supervised code may be called, but which frames are the harness rather than the subject. RESERVED_PACKAGE_PREFIX_VERSION moves to 2, which is what that number is for. A version 1 snippet still in an exercise reserves more than this version does, so it refuses what is now allowed rather than allowing what is now refused; it is over-strict rather than bypassable, and no migration is urgent. The identifier must still change, because one version must not denote two lists.
An exercise that hands the student a package to fill has no compiled production class until they write one. Refusing there replaced their own failing tests with an Ares security error labelled Reason: Ares-Code, which is neither their fault nor Ares'. Nothing to supervise is not the same as a scope that supervises the wrong thing: where no class exists, none can escape a boundary, so enforcement is vacuous rather than mis-scoped and a warning is the honest response. An unreadable output stays a refusal, because not knowing what is there is not the same as knowing there is nothing. The same split closes the opposite hole, which was measurable: the emptiness was checked against the whole output and before the pinned scope narrowed it, so a scope matching no compiled class while classes existed returned an empty set and every rule passed against nothing. That is the very failure this class was written to stop, and it survived on the pinned path, where neither requireScopeToCover nor requireDerivedScopeToCoverTheProject runs. Deliberate narrowing means less than everything, never nothing, so it is now refused by name and with a count. The comment cleanup for these two files rides along, because the edits sit in the same methods and cannot be separated into their own commit.
The comments added by this pull request had grown past what they explained. Every newly added line comment is gone from the sixteen changed main files, and the Javadoc that remains is roughly a hundred lines shorter without dropping a claim: where a line comment carried a reason worth keeping, the reason moved into the Javadoc of the method it applied to, which is where the next reader will look for it rather than mid-expression. Three defects fell out of the pass. JavaCreator carried two adjacent comment blocks in the same ternary saying almost the same thing, one of them already stale. JavaProjectScanner carried two orphaned Javadoc blocks describing methods that no longer exist, so each documented whichever method happened to follow it. And the @throws on validated() and pinned() still promised a SecurityException when nothing is compiled, which stopped being true. Six line-comment lines remain in the two advice classes. They are a rewrap of a comment that predates this pull request, kept because the wording it replaced described a fall-back this pull request removed.
Three places deviated from a convention their own neighbours already follow. None changes behaviour; each is a thing a reader has to check twice. The AspectJ and instrumentation backends carry the same timeout exemption in two files kept in step by hand, and isThreadCreationFromAresTimeout had drifted in four spots: three comment wraps and a missing @nullable. The wraps are resolved towards the .java file, because Spotless formats *.java and not *.aj, so the reverse would be undone by the next spotless:apply. The two methods, javadoc included, are now identical character for character, which is the only state in which a divergence between them is visible as a diff. The missing @nullable was not merely an asymmetry across the two backends. It was the exception inside its own file: isImmediateCallerWithinRestrictedPackage sits beside it with the same parameter, the same type and the same meaning, annotated. The body already reads restrictedPackage != null, so null is a value the method expects rather than one it merely survives. The Builder in JavaArchitectureTestCase was missing @nullable on supervisedPackage and @nonnull on both of its setters. Every other nullable field in that builder carries the first, all seven of its other setters carry the second, and both subclass builders already spell their copies of these two setters exactly that way. The stray blank lines between the two fields go with them, so the block reads like its neighbours. Verified with spotless:check, a full compile including the AspectJ weave, and the five test classes that touch either area: 29 tests, no failures.
Claudia-Anthropica
left a comment
There was a problem hiding this comment.
@MarkusPaulsen The two previously open findings remain on this exact head: generated allowedPackages() still turns de.tum.cit into a prefix grant, and the former public constructors plus the Creator.createTestCases descriptor remain removed. The latest reserved-prefix change also permits metatest and anonymous.toolclasses declarations while WALA still treats those application packages as infrastructure, enabling an architecture-enforcement false negative; I left that as a new high finding. The captured snapshot has no failed checks, though Unit Tests, Static Code Analysis, Core Integration Tests, and all four mode integration jobs were still in progress and are not listed as required.
Two findings Claudia raised, both still open on the previous head. The first is that allowedPackages asked only half the question. It skipped a package inside a trusted namespace and granted everything else, but a package that sits above one carries that namespace with it, because a permission is matched as a prefix. Nothing upstream catches it: validation refuses a package inside a reserved prefix, and de.tum.cit lies inside none, while a scope broad enough to contain it covers every compiled class by construction and therefore passes coverage as well. The generated rule would then permit the supervised code every import from the framework supervising it. The direct path already asked the second question through ReservedPackageGuard.ancestorOfReservedPrefix and the generated one did not, so the two disagreed about the same package. The complement is written out locally rather than called, for the reason the class header gives: this file is copied into the supervised project, where only what the copy manifests list exists, and ReservedPackageGuard is not on them. It also has to consider the copied API prefix, which the direct path has no notion of. Reaching it refuses the run and names the package, rather than silently granting a narrower list, because the alternative is a project whose own classes cannot call each other and whose diagnostic says something else. The second is that the scope refactor replaced eight public constructors rather than adding to them. A client compiled against 2.1.2 links against those descriptors, so it would fail with NoSuchMethodError, and a source client could not recompile. All eight are back as delegating bridges, deprecated for removal, and Creator.createTestCases likewise as a default method. The bridge cannot know whether the scope was derived or pinned, so it reports derived, which is the strict reading of the two: a derived scope is checked against the whole compiled output before enforcement, so a bridge can only ever verify more than the caller asked for. An implementation written against the old Creator signature still has to be updated, since no bridge can supply a parameter its author never wrote. Making the scope a constructor parameter was what stopped a test case existing without one, and the bridges do not reopen that. A test case built through one carries no scope and refuses to write a generated file, with the diagnostic that names the missing declaration, rather than writing a file whose rules pass against an empty set of classes. Both are covered. JavaArchunitSupervisedClassesTest drives the new complement directly, including the broad ancestor Claudia named, the segment boundary that keeps de.tum.citadel out of it, and the arbitration with reservedPrefixOf so that one package cannot be refused twice for two reasons. ReleasedConstructorCompatibilityTest calls all eight signatures and asserts what they produce, which nothing else in the repository does: the production path builds through the builders, so a future removal would pass every other test in the suite. Verified with spotless:check, test-compile including the AspectJ weave, and the ten test classes touching either area: 99 tests, no failures.
Claudia-Anthropica
left a comment
There was a problem hiding this comment.
@MarkusPaulsen The latest commit closes the generated ancestor-package grant and restores the eight constructor descriptors. Two current-head defects remain: WALA still classifies newly permitted metatest.* and anonymous.toolclasses.* application classes as infrastructure, allowing forbidden paths with no student frame, and Creator still makes the new boolean overload abstract, so released third-party implementations no longer compile despite the old call descriptor being present as a default. The captured snapshot has no failed checks; Unit Tests and both WALA integration jobs were still in progress, but none is listed as required.
Claudia-Anthropica
left a comment
There was a problem hiding this comment.
@MarkusPaulsen The generated ancestor-package permission is fixed, but two blockers remain. [medium] Creator still makes the boolean overload abstract while the released overload is a default that immediately invokes it; old implementations therefore still fail to compile and old binaries can reach AbstractMethodError. [high] metatest.* and anonymous.toolclasses.* are permitted by the build boundary but remain WALA infrastructure prefixes, so evaluateSink() can traverse supervised frames in those packages as infrastructure and accept a forbidden path without evaluating a student approach. All captured checks succeed, but they do not cover these compatibility and namespace-collision cases.
Claudia's finding on the previous head, and it is correct. Reserving only what Ares can be confused with left anonymous.toolclasses. and metatest. nameable by supervised code while INFRA_PREFIXES still skips them when looking for the frame to attribute a forbidden call to. A supervised class declaring package metatest therefore put every frame of its own path into the skipped set, evaluateSink walked the predecessors, never found a frame to evaluate, and returned: the forbidden call was found and then reported to nobody. The drop is in evaluateSink rather than in evaluatePath, which is where the first attempt at this put the guard and where the new test showed it never runs. evaluateApproach is only ever reached once a non-infra predecessor has been found, so evaluatePath's own empty-student-frame branch cannot be taken in production. What tells the two cases apart is not the class loader. Ares' own api frames are application-loaded too, and a path made of them must stay a silent drop, which the existing infra-only test says out loud. It is whether supervised code was free to declare the namespace that was skipped: reserved means it could not have put itself there, unreserved means it could. So a sink that ends the walk having stepped over a reachable frame in an INFRA prefix that is not also a RESERVED prefix is refused, naming that frame and the call. Keying it to the gap between the two lists rather than to their contents keeps this true whatever they say next. An entry added to INFRA_PREFIXES without being reserved is refused rather than turned into a blind spot, and reserving one again makes the refusal disappear on its own. That is deliberately not a re-coupling of the lists: they answer different questions, and taking back the narrowing would restore the complaint it was made to remove, that every Ares user refused two ordinary package names on one consumer's behalf. Two tests, one for each shape. A path entered through metatest is refused and the message names the frame; a path of JDK frames alone is still dropped, or every synthetic WALA root would become a failure. Verified with spotless:check and 153 tests across the WALA, ArchUnit, scanner and reserved-package classes, including the infra-only path that the first version of this fix broke.
Claudia-Anthropica
left a comment
There was a problem hiding this comment.
@MarkusPaulsen The single-path WALA regression is fixed, but a parallel ordinary approach can still suppress refusal of an all-infrastructure student path. [medium] Creator also still makes the new boolean overload abstract while the released overload delegates to it, so existing third-party implementations fail to compile and can reach AbstractMethodError. No failed checks were captured; several checks remain pending but none is listed as required.
Claudia's finding on the previous head, and it is right. The refusal was decided per sink, on whether anything at all had been evaluated. A sink is commonly reached along several branches, and one of them ending in an ordinary caller says nothing about another: an exempt approach, or a false-positive transitive path, evaluates and reports nothing while still incrementing the counter. One such branch was therefore enough to excuse a parallel branch made entirely of frames supervised code may name itself into, which is the case the previous commit existed to refuse. The counter no longer decides. A branch is examined when it runs out of callers, which is the point at which it is known that no frame on it could be blamed, and it is examined on its own. Branches that continue are left to their extensions, and an approach evaluated on one has found its student frame already, so neither needs the check. The regression is written so that it fails against the previous commit rather than passing because the walk happened to reach the right branch first. The stack is LIFO, so the predecessor order decides which branch is examined first; listing the helper first makes the ordinary branch pop first and evaluate before the nameable-infrastructure branch is looked at, which is exactly the ordering the previous version got wrong. Reversed, the refusal would fire before anything evaluated and the test would prove nothing. Confirmed both ways: it fails on 8e82643 and passes here. Verified with spotless:check and 154 tests across the WALA, ArchUnit, scanner and reserved-package classes.
Claudia-Anthropica
left a comment
There was a problem hiding this comment.
@MarkusPaulsen The branch-local WALA walk fixes the mixed-branch false negative, and the complete snapshot has no failed checks. [medium] The released Creator interface remains source- and binary-incompatible because its new boolean overload is abstract while the former abstract overload is now default; old implementations lack the new descriptor, so current framework calls can reach AbstractMethodError. I also left a medium documentation finding because the manual promises empty-output refusal while both runtime validators now warn and continue.
Claudia's finding on the previous head, and the manual was wrong. It claimed that an output root holding nothing at all is refused by name. That stopped being true when letting an empty supervised package be an empty package turned those two abort paths into warnings, deliberately: an exercise whose package is still empty must not fail for being empty. The document was not brought along. Both places behave the same way and the manual now says so. requireDerivedScopeToCoverTheProject warns and returns where the output holds no class file, and again where it holds only package or module descriptors. JavaArchunitSupervisedClasses warns and analyses an empty set where the conventional output root is absent or yields nothing supervisable. What is still refused by name is listed as what it is: a class the scope leaves out, a class in the default package, a class in a reserved package, and an output root that exists but cannot be read. The claim about the last-resort default is qualified rather than removed, because it holds in one half of the case and not the other. Where anything at all is compiled, classes outside the default are caught and the run is refused; where nothing is compiled, nothing is caught. Saying it without that split was what made it read as a guarantee. The custom-output limitation was already documented as a fact about which directory is read. It is now stated as the consequence it has become: a project whose output goes elsewhere is indistinguishable from one that compiled nothing, so both pass with the same warning, and an exercise configured that way is not enforced by the derived path at all. That is the one case where a reader could take vacuous success for enforcement, so it says outright that such an exercise must declare its scope in a policy. Documentation only; no code changes. spotless:check is green.
Claudia-Anthropica
left a comment
There was a problem hiding this comment.
@MarkusPaulsen The documentation correction now matches the empty-output behaviour, and the prior generated-scope and WALA findings are resolved. [medium] Creator still makes the new boolean overload abstract while the released overload delegates to it, so existing third-party implementations cannot compile against this head and old binaries reach AbstractMethodError when the factory invokes the new descriptor. Keep the released overload abstract and make the boolean overload a default bridge, with JavaCreator implementing the scope-aware path.
Claudia raised this in four review bodies rather than on a line, so it carried no thread and my thread audit reported nothing open while the review still requested changes. The finding is right and my earlier answer to it was wrong. Restoring the released signature as a default that delegates to a new abstract one served callers and nobody else. A class written against 2.1.2 implemented the released method, so under that arrangement it implemented a default and left the abstract one unimplemented: it stopped compiling. One already compiled lacked the new descriptor entirely, and Ares' own call site always passes the scope, so it could be reached through AbstractMethodError. I had reasoned that no bridge can supply a parameter its author never wrote. That is true of the value and false of the conclusion. The parameter does not have to be supplied; which overload is abstract has to be chosen. The released signature is abstract again, so an implementation that predates the parameter is a complete implementation, and the scope-aware overload is the default, which drops the parameter and calls the released one. Dropping it is exactly what such an implementation did before the parameter existed. JavaCreator implements both: the scope-aware overload does the work, and the released one delegates to it reporting the scope as derived, which of the two readings is the strict one, since a derived scope is checked against the whole compiled output before enforcement. The cost is that a new implementation has to write both, one of them a one-liner. That is the price of not moving a released contract, and the alternative is silently dropping the parameter for an implementation that meant to act on it. ReleasedCreatorContractTest is the contract rather than a description of it. Its LegacyCreator implements only what 2.1.2 asked for, so this file compiling is half the assertion; the other half is that Ares' own call, which always passes the scope, arrives at the released method instead of failing. Verified with spotless:check and 185 tests across the creator, WALA, ArchUnit, scanner and reserved-package classes.
Claudia-Anthropica
left a comment
There was a problem hiding this comment.
@MarkusPaulsen The Creator compatibility blocker is fixed: the released overload is abstract again and the new overload is a default bridge covered by a legacy implementation test. Two medium issues remain in generated build-mode selection and the reserved-prefix migration material. Captured CI has no failures, although Unit Tests is still in progress and is not listed as required.
| boolean maven = Files.isRegularFile(Path.of("pom.xml")); | ||
| boolean gradle = Files.isRegularFile(Path.of("build.gradle")) | ||
| || Files.isRegularFile(Path.of("build.gradle.kts")); | ||
| if (maven && gradle) { |
There was a problem hiding this comment.
@MarkusPaulsen [medium] Generated tests still discard the build mode selected by the policy. ProjectSourcesFinder.discover(root, explicitlySelectedMode) deliberately supports a project containing both descriptors, but this helper treats that valid configuration as ambiguous and aborts instead of reading the selected mode's output. Carry the selected BuildMode into the generated validated/pinned calls and use it to choose exactly target/classes or build/classes/java/main.
🤖 Prompt for AI agents
In src/main/java/de/tum/cit/ase/ares/api/architecture/java/archunit/JavaArchunitSupervisedClasses.java, generated tests ignore the explicitly selected build mode and reject valid dual-descriptor projects. Thread the selected BuildMode through generated helper entry points and select only its production output root.
| * is urgent, but the identifier must still change: one version must not denote | ||
| * two different lists. | ||
| */ | ||
| public static final String RESERVED_PACKAGE_PREFIX_VERSION = "2"; |
There was a problem hiding this comment.
@MarkusPaulsen [medium] The prefix contract is now version 2 and no longer reserves anonymous.toolclasses or metatest, but the shipped Maven/Gradle examples and three setup/enforcement documents still declare prefix version 1 and include both removed prefixes. Users following those canonical examples therefore pin obsolete data and reject packages this version intentionally permits. Update both example snippets plus HowToMakeAProjectAnAresProject.md, HowToConvertAnAres1ProjectIntoAnAres2Project.md, and policy/EnforcementModel.md, and document the prefix-version migration in the PR body.
🤖 Prompt for AI agents
In src/main/java/de/tum/cit/ase/ares/api/architecture/java/wala/WalaPathClassification.java, the reserved-prefix version was raised to 2 without updating the repository's canonical examples and setup documentation. Synchronise all shipped snippets and version references with the version-2 prefix list and add the migration note.
Summary
Gradle source-root discovery silently dropped every
srcDirsdeclaration, so a project declaring its mainsources that way looked to Ares like a project with no production sources at all. Without a policy, the
supervised package then fell straight through to a configured default the project need not contain, and
enforcement was scoped to a directory that does not exist.
Linked issues
None. Found while migrating a set of Artemis ITP exercises from Ares 1 to Ares 2 and removing their
@Policyannotations to use the policy-free configuration.
1. Problem
Observed with Ares 2.1.2 on JDK 17, Gradle 9.0.0, ArchUnit and AspectJ, in an exercise whose
tests carry Ares annotations but no
@Policy, so the policy-free configuration applies. Setupmanual section 6 documents file system, network, command execution and thread creation as
denied there.
Supervised code performed all of them unhindered:
Files.readString, aProcessBuilderstartand a
Threadstart all ran, and the build stayed green.System.exit(3)was still blocked, soonly the five resource domains were unarmed.
The fault is in plugging Ares into the build.
ProjectSourcesFinder.SOURCE_DIRECTORYtriedsrcDirbeforesrcDirs, and the former is a prefix of the latter, so onsrcDirs = ["assignment/src"]the capture began at the leftover, the trailings, and yieldeds = ["assignment/src", whichresolveGradlePathcannot resolve. The entry was dropped withouta diagnostic and discovery returned no production roots.
Ares let forbidden student code through, silently.
2. Improvement from the user's perspective
Instructors using the policy-free configuration get the enforcement the manual promises. Before this change, an
exercise whose main source set is declared with
srcDirs, which includes Artemis Java exercises built from thestandard template, enforced nothing in the file, network, command and thread domains while reporting a green run.
Students see no change in a correctly configured exercise; in an affected one, code that should have been
rejected is now rejected.
The last-resort default now names the package the Artemis exercise templates are generated with,
de.tum.cit.aet, so an exercise that reaches step three lands on its own code rather than one letter beside it.The thread domain no longer depends on the supervised package being scoped correctly to work at all: a
mis-scoped exercise now narrows enforcement instead of disabling a whole domain.
3. Improvement from the maintainer's perspective
Discovery no longer collapses to a configured default in one step. A second detection step
reads the compiled production output, which is authoritative whatever the descriptor says,
because the build tool writes it to its own location, so an unparsed descriptor no longer costs
the supervised scope. Where nothing is detectable the default still applies, but now says so in
a warning naming the roots that were searched. The parser's remaining gaps are documented at
the pattern rather than implied.
The self-tests no longer inherit their supervised package from a production default.
SecurityUserandPackageAccessUserpinPolicySelfTestDefaultRestrictive.yaml, whichdeclares the same fully restrictive accesses the policy-free path derives, so what they assert
is unchanged while a future change to the TUM default cannot disarm them. The timeout exemption
is keyed to the method that creates the worker rather than to the class.
4. Testing manual
The parsing defect is Gradle-only, so the manual uses Gradle. The scanner change affects both build tools.
Prerequisites
mvn install -DskipTests.examples/ares-exercise-gradle.Steps
examples/ares-exercise-gradle/build.gradle, replace the main source-set declarationsrcDir 'src/main/java'with the list formsrcDirs = ['src/main/java'], and change nothing else. This isthe only edit needed to reproduce the defect, and it is what the Artemis exercise template does.
@Policyannotation and itsimportfrom the example's test class, so the policy-freeconfiguration applies. Leave the Ares test annotation in place: it is what activates Ares.
file:
./gradlew clean test.Expected result
forbiddenReadIsRejectedpasses, that is Ares threw. In thetest output, the line
Resolved Ares analysis/import path for GRADLE: .../build/classes/java/main/<your package path>names the package of the example's own classes.forbiddenReadIsRejectedfails, and its message showsjava.nio.file.NoSuchFileException: secret.txtrather than aSecurityException, that is the read reached thefile system. The same log line ends in
de/tum/cit/ase, a directory that does not exist underbuild/classes/java/main, which is the defect made visible.No supervised package could be detectedwarning must not appear in step 5. It appears only whenneither the sources nor the compiled output declares a package.
Negative case (what must still be rejected)
SecurityPolicy.yamland@Policyrestored, the same forbiddenread is still rejected, exactly as before.
package de.tum.cit.ase.ares.api;muststill fail the build with the reserved-package diagnostic.
build/classes/java/main/de/tum/cit/detected/Calculator.classpresent,scanForPackageName()answersde.tum.cit.detectedand not the configured default. Covered byJavaProjectScannerPackageFallbackTest, whosefixture package is deliberately distinct from the default so the assertion cannot hold through the fallback.
TimeoutUtils.executeWithTimeoutstays exempt, so that worker is never attributed to the student, while athread operation reached through
TimeoutUtils.rethrowThrowableSafe, that is from thestudent's test body, is rejected. Covered by
JavaAspectJThreadSystemAdviceDefinitionsTestandJavaInstrumentationAdviceThreadSystemToolboxTest, and end to end byde.tum.cit.ase.ares.integration.SecurityTest.test_useCommonPoolGood, which asserts that a benignCompletableFuture.supplyAsyncon the common pool is still blocked.PolicySelfTestDefaultRestrictive.yamldeclares the fullyrestrictive resource accesses the policy-free path derives, so pinning it changes what the fixtures depend on,
not what they prove. The full
SecurityTestandPackageAccessTestsuites still pass.Modes exercised
All four are green in CI. The manual walkthrough above was run in ArchUnit + AspectJ, which is the combination
the policy-free path fixes itself and therefore the only one reachable from a policy-free exercise. The
enforcement change touches both AOP variants, so it is applied symmetrically to
JavaAspectJThreadSystemAdviceDefinitionsandJavaInstrumentationAdviceThreadSystemToolboxand covered by atest for each.
5. Test case coverage regarding this PR
Read from
site/jacoco/jacoco.csvin thecoverage-reportartefact of the Coverage Report job forthis branch's current head
74361ebd(workflow run32113187323). Each cell is COVERED out ofMISSED plus COVERED, so a reviewer can recompute it. These replace figures taken at
18ce34ce,before the review findings were addressed.
ProjectSourcesFinderJavaProjectScannerJavaCreatorReservedPackageGuardBuildToolConfigurationJavaTestCaseFactoryAndBuilderJavaProgrammingExerciseProjectScannerJavaAspectJThreadSystemAdviceDefinitionsJavaInstrumentationAdviceThreadSystemToolboxThe descriptor masker is no longer the exception it was. An earlier revision of this section
admitted that
maskSlashyString,maskUntilandstartsAnExpressionwere at zero, so Groovy'sslashy and dollar-slashy strings were handled by code nothing had executed. They are now at 90%,
95% and 92%, and
maskInactiveRegionsat 100%, which moves the class from 85.6% instruction and70.6% branch to 93.4% and 78.3%, with every method executed.
maskQuotedStringat 52% andmaskBlockCommentat 62% are what remain, and their uncovered branches are triple-quoted stringsand nested block comments.
Those regressions are written so they would fail if the masking stopped working, rather than passing
because nothing happened: the decoy declaration sits inside the
javablock and the directory itnames exists, so an unmasked string would replace the assignment root and the assertion would say so.
The division case is the exception and its comment says so, since an unterminated slashy string
already stops at the line break.
The rest of the assertions are specific rather than incidental.
ProjectSourcesFinderTestandProjectSourcesFinderEdgeCaseTestassert the resolved roots and the rejections themselves, includingthe two ways the reader used to empty a source set silently, a comment inside a live list and a list
written over several lines, and the completeness flag that now records a declaration it cannot
resolve.
JavaProjectScannerScopeCoverageTestasserts every state of the output root against realclasses from
ToolProvider.getSystemJavaCompiler(), including the all-reserved output that used topass.
ReservedPackageGuardTestassertsancestorOfReservedPrefixon segment boundaries, ontrailing-dot normalisation, and that a package which is itself reserved is left to the other guard.
The two advice tests drive
isThreadCreationFromAresTimeoutdirectly and assert both stack shapes.The two advice classes are large and their figures are dominated by pointcut and message-building
code this pull request does not touch; the changed method itself is covered by the new tests.
Breaking changes and migration
The public API, the policy format, the generated tests and the minimum JDK, Maven and Gradle
versions are unchanged. Three behaviours change.
The point of the pull request. An exercise on the policy-free path declaring its main sources
with
srcDirsenforced nothing in the five resource domains and now enforces them. One needingany of them must declare it in a policy, as it always had to.
The TUM default package.
getDefaultPackage()returnsde.tum.cit.aetrather thande.tum.cit.ase, reached only when nothing declares a package. An exercise living underde.tum.cit.aseshould pintheSupervisedCodeUsesTheFollowingPackage.The thread domain under
@StrictTimeout. Under a policy forbidding thread creation, a threadstarted from a student's test body in a
@StrictTimeouttest is now rejected; a mis-scopedexercise let it through. Ares' own timeout worker stays exempt, so no such test breaks.
Checklist
docs/,README.adoc, Javadoc) was updated where the change is user-facing.Setup manual section 6.2 already states that the policy-free configuration denies file, network, command
and thread access, so that statement needed no edit; this change makes the implementation match it.
docs/securitytest/TestCaseFactoryAndBuilderManual.mdnow documents the three-step resolution order andstates plainly that the last-resort default enforces nothing when the project does not contain that
package. The resolution order, the reasons a default must not win over evidence, and why the timeout
exemption is keyed to the creating method are documented in the Javadoc of
scanForPackageName(),getDefaultPackage()andisThreadCreationFromAresTimeout(), and the parser's remaining known gaps atthe pattern in
ProjectSourcesFinder.Review progress