#130: Support benchmarks having @State annotations in parent classes#193
#130: Support benchmarks having @State annotations in parent classes#19307jasjeet wants to merge 3 commits intoKotlin:masterfrom
Conversation
Now, if a parent class is abstract, that child class will also be benchmarked, with annotations derived in order: child ?: parent.
1. Refactored code (cleanup) 2. All classes except final are considered when considering parent annotated class.
|
|
||
| val inheritableClassDescriptors = classDescriptors.filter { it.modality != Modality.FINAL } | ||
|
|
||
| // Abstract classes which are annotated directly. |
There was a problem hiding this comment.
It seems to me that directlyAnnotatedInheritableClassDescriptors includes more than just abstract classes.
There was a problem hiding this comment.
Oops yes old comment. Will fix!
| val classDescriptors = DescriptorUtils.getAllDescriptors(packageFragment.getMemberScope()) | ||
| .filterIsInstance<ClassDescriptor>() | ||
|
|
||
| val inheritableClassDescriptors = classDescriptors.filter { it.modality != Modality.FINAL } |
There was a problem hiding this comment.
WDYT about using "non-final" instead of "inheritable"?
The term "non-final" just sounds more familiar to me.
| }) { | ||
| add(AnnotatedClassDescriptor(it)) | ||
| } else { | ||
| val parent = it.getParentAnnotated(annotatedInheritableClassDescriptors) |
There was a problem hiding this comment.
Could directlyAnnotatedInheritableClassDescriptors be used here?
I am struggling to understand the need for annotatedInheritableClassDescriptors. Cloud you please clarify?
There was a problem hiding this comment.
Huh after another thought, I think directlyAnnotatedInheritableClassDescriptors should be used here. Will do!
| private fun ClassDescriptor.getParentAnnotated( | ||
| annotatedInheritableClassDescriptors: List<ClassDescriptor> | ||
| ): ClassDescriptor? { | ||
| annotatedInheritableClassDescriptors.forEach { annotatedAbstractClass -> |
There was a problem hiding this comment.
Could you please add some integration tests instead?
| annotatedInheritableClassDescriptors: List<ClassDescriptor> | ||
| ): ClassDescriptor? { | ||
| annotatedInheritableClassDescriptors.forEach { annotatedAbstractClass -> | ||
| if (this.isSubclassOf(annotatedAbstractClass)) { |
There was a problem hiding this comment.
Could it be that there are multiple superclasses for this class descriptor?
There was a problem hiding this comment.
I was expecting this! So as for phase-1 I was just looking out for bare bones implementation which means this here just a base setup. Currently this function picks the first superclass that is directly annotated. Now there are some cases:
- Simple straight forward: Only one annotated superclass.
- Multiple superclasses but only one is annotated.
- Multiple superclasses and intermediaries (super class of this but sub class of the top most annotated parent class) are annotated as well.
- Multiple superclasses and intermediaries with annotations E.x.,
@Warmup,@Measurementetc without@State.
What should be the route to follow here?
For 4th point, my suggestion is that independent annotations should be ignored. For the 3rd point, I would suggest nearest annotated parent. 1st and 2nd are easily dealt with.
There was a problem hiding this comment.
What should be the route to follow here?
It depends on how JMH handles this aspect. Being consistent with that library is important, since it runs the benchmarks on the JVM. Achieving consistent behavior across other platforms would be ideal. It is worth mentioning that we do not aim to replicate all of JMH's sophistication, but when something is not supported in non-JVM targets, we would like to issue a warning (or throw an exception) about it.
There was a problem hiding this comment.
I would greatly appreciate it if you could check how JMH handles this case. It would greatly help us decide which route we should take.
| @Fork(1) | ||
| @Warmup(iterations = 0) | ||
| @Measurement(iterations = 1, time = 1, timeUnit = TimeUnit.SECONDS) | ||
| abstract class BaseBenchmark { |
There was a problem hiding this comment.
Can the base class declare benchmark methods as well?
There was a problem hiding this comment.
No. As to why? -> It doesn't make sense to run the same benchmark again if two or more classes are inheriting from same non-final class.
There was a problem hiding this comment.
I can think of the following use case:
import kotlinx.benchmark.*
@State(Scope.Benchmark)
@Measurement(iterations = 7, time = 1, timeUnit = BenchmarkTimeUnit.SECONDS)
@Warmup(iterations = 7, time = 1, timeUnit = BenchmarkTimeUnit.SECONDS)
@OutputTimeUnit(BenchmarkTimeUnit.MILLISECONDS)
abstract class IterationBenchmarkBase {
abstract val list: List<String>
@Param("10", "100")
var size: Int = 0
@Benchmark
fun iterationBenchmark(bh: Blackhole) {
for (element in list) {
bh.consume(element)
}
}
}
class ArrayListIterationBenchmark : IterationBenchmarkBase() {
override val list: List<String>
get() = ArrayList<String>(size).apply { fill(size) }
}
class ArrayDequeIterationBenchmark : IterationBenchmarkBase() {
override val list: List<String>
get() = ArrayDeque<String>(size).apply { fill(size) }
}
private fun MutableList<String>.fill(size: Int) {
addAll((1..size).map { it.toString() })
}
| } | ||
| } | ||
|
|
||
| class TestInheritedBenchmark: BaseBenchmark() { |
There was a problem hiding this comment.
Can a TestInheritedBenchmark subclass declare benchmark methods as well?
There was a problem hiding this comment.
Could you expand a bit on your question?
There was a problem hiding this comment.
I am referring to cases of multi-hierarchy benchmark classes, within which multiple classes define benchmark methods.
I would appreciate it if you could clarify what exactly you think should be included in phase 1. |
2 participants
Targets #130
Changelog
@Stateannotation, our child class will also be considered when generating benchmarks.@Measurement@Warmup@OutputTime@BenchmarkModeare considered in this order: child -> top most annotated parentNote: This is a phase-1 implementation, with just the basic idea laid out.