During an exam, students cannot see the results of penalty code checks or penalty meta tests. It would be nice if we could inform the student that they are getting such a penalty (when they click "Only tests").
For example, the description might say "do not use filter", but if the student misses this, they'll lose all points, without Andy telling them that this will happen.
We still need to determine whether we should make all penalties visible, or only a subset.
- For concrete penalties that are mentioned in the description, like "do not use
filter" (in a PBT assignment), it always makes sense to show them.
- For penalties like "
getTripById should not be verified" (in a mocking assignment), we probably do not want to show them. (They check whether students meet the learning objective of "being able to determine whether something should be a stub or a mock".)
- For penalties like "Meta test: finds bug where very specific indices are missing (fails when tests do not cover the input space well) (penalty: 50)" (in a PBT assignment), we still need to decide. (They check whether students follow the instruction to write "good" property-based tests, but they're not completely fool-proof, so they will cause problems because of false positives and false negatives.)
We probably want a mechanism where we can decide for each individual penalty whether it should be visible when clicking "Only tests".
We should be careful not to introduce vulnerabilities. If I remember correctly, the RunConfiguration is destroyed for "Only tests" so that students cannot access it. If we need to change this, we should be careful not to open up too much.
During an exam, students cannot see the results of penalty code checks or penalty meta tests. It would be nice if we could inform the student that they are getting such a penalty (when they click "Only tests").
For example, the description might say "do not use
filter", but if the student misses this, they'll lose all points, without Andy telling them that this will happen.We still need to determine whether we should make all penalties visible, or only a subset.
filter" (in a PBT assignment), it always makes sense to show them.getTripByIdshould not be verified" (in a mocking assignment), we probably do not want to show them. (They check whether students meet the learning objective of "being able to determine whether something should be a stub or a mock".)We probably want a mechanism where we can decide for each individual penalty whether it should be visible when clicking "Only tests".
We should be careful not to introduce vulnerabilities. If I remember correctly, the RunConfiguration is destroyed for "Only tests" so that students cannot access it. If we need to change this, we should be careful not to open up too much.