Repro:
val l = (0..10).toDataFrame {
"id" from { it }
}
val r = (0..5).toDataFrame {
"id" from { it }
"frame" from {
(0..10).toDataFrame {
"col" from { it }
}
}
}
val res = l.leftJoinWith(r) { id == right.id }
res.schema().print()
res.frame[6].col
Actual:
id: Int
id1: Int?
frame: *
col: Int
Exception in thread "main" java.lang.IllegalStateException: Column not found exception in the generated DataFrame extension property 'col': Column not found: 'col'. See https://kotlin.github.io/dataframe/data-schemas-and-extension-properties-troubleshooting.html for more information.
at org.jetbrains.kotlinx.dataframe.exceptions.ExtensionPropertyExceptionKt.handleExtensionPropertyException(extensionPropertyException.kt:32)
at JoinFrameKt$main$res$1$DataFramePropertiesScope1.getCol(joinFrame.kt)
at JoinFrameKt.main(joinFrame.kt:18)
at JoinFrameKt.main(joinFrame.kt)
Both runtime and compile schemas tell us there's col in frame, but it's not always the case, and simply iterating over res can lead to an unexpected column not found exception.
Root cause:
Potential fix touches broader issue. joinWithImpl uses createByInference, which produces this invalid result.
|
else -> DataColumn.createByInference(generator.names[index], values) |
|
null -> DataFrame.empty() |
Smaller repro that would involve only createByInference will be
fun main() {
val res = dataFrameOf("frame" to columnOf(null, dataFrameOf("col" to columnOf(1))))
res.cast<DataFrameOf_42>().frame[0].col
}
@DataSchema
data class DataFrameOf_42(
val frame: List<Frame>
) {
@DataSchema
data class Frame(
val col: Int
)
}
Repro:
Actual:
Both runtime and compile schemas tell us there's col in frame, but it's not always the case, and simply iterating over
rescan lead to an unexpected column not found exception.Root cause:
Potential fix touches broader issue.
joinWithImpluses createByInference, which produces this invalid result.dataframe/core/src/main/kotlin/org/jetbrains/kotlinx/dataframe/impl/api/joinWith.kt
Line 114 in 8d7d9bd
dataframe/core/src/main/kotlin/org/jetbrains/kotlinx/dataframe/impl/columns/constructors.kt
Line 281 in 0d009c6
Smaller repro that would involve only
createByInferencewill be