-
Notifications
You must be signed in to change notification settings - Fork 246
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Describe the bug
In the failed test case of #2444, I found that the struct type with duplicate keys will be deduplicated when converted to arrowType, which causes RowToColumanr
to lose some columns.
error:
Job aborted due to stage failure: Task 0 in stage 7.0 failed 1 times, most recent failure: Lost task 0.0 in stage 7.0 (TID 9) (10.5.155.10 executor driver): java.lang.IndexOutOfBoundsException: Index 2 out of bounds for length 2
at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:266)
at java.base/java.util.Objects.checkIndex(Objects.java:361)
at java.base/java.util.ArrayList.get(ArrayList.java:427)
at org.apache.comet.vector.CometStructVector.getChild(CometStructVector.java:55)
at org.apache.spark.sql.vectorized.ColumnarRow.isNullAt(ColumnarRow.java:101)
Steps to reproduce
No response
Expected behavior
No response
Additional context
No response
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working