-
Notifications
You must be signed in to change notification settings - Fork 257
Open
Labels
? - Needs TriageNeed team to review and classifyNeed team to review and classifybugSomething isn't workingSomething isn't working
Description
Given this part of the error, it looks to be we expect the CPU to throw an error and it is not (I think). It could be a "shape of data" problem as @firestarman mentions here: #12955.
[2025-07-24T11:27:04.911Z] ../../src/main/python/asserts.py:707: in assert_gpu_and_cpu_error
[2025-07-24T11:27:04.911Z] assert_spark_exception(lambda: with_cpu_session(df_fun, conf), error_message)
[2025-07-24T11:27:04.910Z] __________ test_array_slice_with_negative_length[Array(Integer)--5-b] __________
[2025-07-24T11:27:04.910Z] [gw1] linux -- Python 3.10.18 /opt/conda/bin/python
[2025-07-24T11:27:04.910Z]
[2025-07-24T11:27:04.910Z] data_gen = Array(Integer), valid_start = 'b', negative_length = -5
[2025-07-24T11:27:04.910Z]
[2025-07-24T11:27:04.910Z] @pytest.mark.parametrize('valid_start', [5, 'b'], ids=idfn)
[2025-07-24T11:27:04.910Z] @pytest.mark.parametrize('negative_length', [-5, 'c'], ids=idfn)
[2025-07-24T11:27:04.910Z] @pytest.mark.parametrize('data_gen', [ArrayGen(int_gen)], ids=idfn)
[2025-07-24T11:27:04.910Z] def test_array_slice_with_negative_length(data_gen, valid_start, negative_length):
[2025-07-24T11:27:04.910Z] negative_length_gen = IntegerGen(nullable=False, min_val=-25, max_val=-1, special_cases=[])
[2025-07-24T11:27:04.910Z] # When the list column is all null, the result is also all null regardless of the start and length
[2025-07-24T11:27:04.910Z] assert_gpu_and_cpu_are_equal_collect(
[2025-07-24T11:27:04.910Z] lambda spark: three_col_df(spark, array_all_null_gen, array_no_zero_index_gen, negative_length_gen, length=5).selectExpr(
[2025-07-24T11:27:04.910Z] f"slice(a, {valid_start}, {negative_length})"))
[2025-07-24T11:27:04.910Z] error = "The value of parameter(s) `length` in `slice` is invalid: Expects `length` greater than or equal to 0"\
[2025-07-24T11:27:04.910Z] if is_databricks143_or_later() or is_spark_400_or_later()\
[2025-07-24T11:27:04.910Z] else 'Unexpected value for length in function slice: length must be greater than or equal to 0.'
[2025-07-24T11:27:04.910Z] # length can not be negative
[2025-07-24T11:27:04.910Z] > assert_gpu_and_cpu_error(
[2025-07-24T11:27:04.910Z] lambda spark: three_col_df(spark, data_gen, array_no_zero_index_gen, negative_length_gen, length=5).selectExpr(
[2025-07-24T11:27:04.911Z] f"slice(a, {valid_start}, {negative_length})").collect(),
[2025-07-24T11:27:04.911Z] conf={},
[2025-07-24T11:27:04.911Z] error_message=error)
[2025-07-24T11:27:04.911Z]
[2025-07-24T11:27:04.911Z] ../../src/main/python/array_test.py:397:
[2025-07-24T11:27:04.911Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2025-07-24T11:27:04.911Z] ../../src/main/python/asserts.py:707: in assert_gpu_and_cpu_error
[2025-07-24T11:27:04.911Z] assert_spark_exception(lambda: with_cpu_session(df_fun, conf), error_message)
Metadata
Metadata
Assignees
Labels
? - Needs TriageNeed team to review and classifyNeed team to review and classifybugSomething isn't workingSomething isn't working