Skip to content

Commit 031310b

Browse files
bersprocketsMaxGekk
authored andcommitted
[SPARK-52660][SQL] Add time type to CodeGenerator#javaClass
### What changes were proposed in this pull request? Update `CodeGenerator#javaClass` to map `TimeType` to `Long`. ### Why are the changes needed? The change is needed to fully support wholestage codegen for code using the time type. Without this change, the following query gets an error compiling the generated code (and falls back to non-wholestage codegen): ``` create or replace temp view v1(col1) as values (time'22:33:01'), (time'01:33:01'), (null); select max(col1), min(col1) from v1; ``` ### Does this PR introduce _any_ user-facing change? No. The time type is not released yet. ### How was this patch tested? New test. ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#51354 from bersprockets/time_agg_issue. Authored-by: Bruce Robbins <[email protected]> Signed-off-by: Max Gekk <[email protected]>
1 parent 2bc9dd3 commit 031310b

File tree

2 files changed

+17
-1
lines changed

2 files changed

+17
-1
lines changed

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1989,7 +1989,7 @@ object CodeGenerator extends Logging {
19891989
case ByteType => java.lang.Byte.TYPE
19901990
case ShortType => java.lang.Short.TYPE
19911991
case IntegerType | DateType | _: YearMonthIntervalType => java.lang.Integer.TYPE
1992-
case LongType | TimestampType | TimestampNTZType | _: DayTimeIntervalType =>
1992+
case LongType | TimestampType | TimestampNTZType | _: DayTimeIntervalType | _: TimeType =>
19931993
java.lang.Long.TYPE
19941994
case FloatType => java.lang.Float.TYPE
19951995
case DoubleType => java.lang.Double.TYPE

sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2915,6 +2915,22 @@ class DataFrameAggregateSuite extends QueryTest
29152915
assert (df.schema == expectedSchema)
29162916
checkAnswer(df, Seq(Row(LocalTime.parse(ts1), 2), Row(LocalTime.parse(ts2), 1)))
29172917
}
2918+
2919+
test("SPARK-52660: Support aggregation of Time column when codegen is split") {
2920+
val res = sql(
2921+
"SELECT max(expr), MIN(expr) " +
2922+
"FROM VALUES TIME'22:01:00', " +
2923+
"TIME'22:00:00', " +
2924+
"TIME'15:00:00', " +
2925+
"TIME'22:01:00', " +
2926+
"TIME'13:22:01', " +
2927+
"TIME'03:00:00', " +
2928+
"TIME'22:00:00', " +
2929+
"TIME'17:45:00' AS tab(expr);")
2930+
checkAnswer(
2931+
res,
2932+
Row(LocalTime.of(22, 1, 0), LocalTime.of(3, 0, 0)))
2933+
}
29182934
}
29192935

29202936
case class B(c: Option[Double])

0 commit comments

Comments
 (0)