Skip to content

Join operations that can introduce nulls (leftJoin, rightJoin, etc) lead to null values FrameColumn and NPE on basic operations #2039

Description

@koperagen

Repro:

val l = (0..10).toDataFrame {
    "id" from { it }
}
val r = (0..5).toDataFrame {
    "id" from { it }
    "frame" from {
        (0..10).toDataFrame {
            "col" from { it }
        }
    }
}
val res = l.leftJoin(r)
res.print()

Actual:
Throws exception at print.

Exception in thread "main" java.lang.NullPointerException: Parameter specified as non-null is null: method org.jetbrains.kotlinx.dataframe.DataFrameKt.getNrow, parameter <this>
	at org.jetbrains.kotlinx.dataframe.DataFrameKt.getNrow(DataFrame.kt)

Culprit is this unchecked cast that sneaks AnyFrame? values into FrameColumn (we expect no nulls there):

ColumnKind.Frame -> DataColumn.createFrameColumn(srcColumn.name, columnValues.asList() as List<AnyFrame>)

Expected:
At this point i'd expected frame column to have 5 DataFrames from r, and remaining 5 rows filled with empty dataframes with same schema: col: Int, but 0 rows
In that ColumnKind.Frame branch we should replace all nulls with "null object" dataframe:

public fun empty(schema: DataFrameSchema): AnyFrame = schema.createEmptyDataFrame()

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions