Skip to content

[BUG][DATAGEN_SEED=1753338612] test_array_slice_with_negative_length failed in scala 2.13 integration #13164

@abellina

Description

@abellina

Given this part of the error, it looks to be we expect the CPU to throw an error and it is not (I think). It could be a "shape of data" problem as @firestarman mentions here: #12955.

[2025-07-24T11:27:04.911Z] ../../src/main/python/asserts.py:707: in assert_gpu_and_cpu_error
[2025-07-24T11:27:04.911Z]     assert_spark_exception(lambda: with_cpu_session(df_fun, conf), error_message)
[2025-07-24T11:27:04.910Z] __________ test_array_slice_with_negative_length[Array(Integer)--5-b] __________

[2025-07-24T11:27:04.910Z] [gw1] linux -- Python 3.10.18 /opt/conda/bin/python

[2025-07-24T11:27:04.910Z] 

[2025-07-24T11:27:04.910Z] data_gen = Array(Integer), valid_start = 'b', negative_length = -5

[2025-07-24T11:27:04.910Z] 

[2025-07-24T11:27:04.910Z]     @pytest.mark.parametrize('valid_start', [5, 'b'], ids=idfn)

[2025-07-24T11:27:04.910Z]     @pytest.mark.parametrize('negative_length', [-5, 'c'], ids=idfn)

[2025-07-24T11:27:04.910Z]     @pytest.mark.parametrize('data_gen', [ArrayGen(int_gen)], ids=idfn)

[2025-07-24T11:27:04.910Z]     def test_array_slice_with_negative_length(data_gen, valid_start, negative_length):

[2025-07-24T11:27:04.910Z]         negative_length_gen = IntegerGen(nullable=False, min_val=-25, max_val=-1, special_cases=[])

[2025-07-24T11:27:04.910Z]         # When the list column is all null, the result is also all null regardless of the start and length

[2025-07-24T11:27:04.910Z]         assert_gpu_and_cpu_are_equal_collect(

[2025-07-24T11:27:04.910Z]             lambda spark: three_col_df(spark, array_all_null_gen, array_no_zero_index_gen, negative_length_gen, length=5).selectExpr(

[2025-07-24T11:27:04.910Z]                 f"slice(a, {valid_start}, {negative_length})"))

[2025-07-24T11:27:04.910Z]         error = "The value of parameter(s) `length` in `slice` is invalid: Expects `length` greater than or equal to 0"\

[2025-07-24T11:27:04.910Z]             if is_databricks143_or_later() or is_spark_400_or_later()\

[2025-07-24T11:27:04.910Z]             else 'Unexpected value for length in function slice: length must be greater than or equal to 0.'

[2025-07-24T11:27:04.910Z]         # length can not be negative

[2025-07-24T11:27:04.910Z] >       assert_gpu_and_cpu_error(

[2025-07-24T11:27:04.910Z]             lambda spark: three_col_df(spark, data_gen, array_no_zero_index_gen, negative_length_gen, length=5).selectExpr(

[2025-07-24T11:27:04.911Z]                 f"slice(a, {valid_start}, {negative_length})").collect(),

[2025-07-24T11:27:04.911Z]             conf={},

[2025-07-24T11:27:04.911Z]             error_message=error)

[2025-07-24T11:27:04.911Z] 

[2025-07-24T11:27:04.911Z] ../../src/main/python/array_test.py:397: 

[2025-07-24T11:27:04.911Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

[2025-07-24T11:27:04.911Z] ../../src/main/python/asserts.py:707: in assert_gpu_and_cpu_error

[2025-07-24T11:27:04.911Z]     assert_spark_exception(lambda: with_cpu_session(df_fun, conf), error_message)

Metadata

Metadata

Assignees

No one assigned

    Labels

    ? - Needs TriageNeed team to review and classifybugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions