Skip to content

GH-47125: [CI][Dev] Fix shellcheck errors in the ci/scripts/integration_hdfs.sh #47126

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

hiroyuki-sato
Copy link
Collaborator

@hiroyuki-sato hiroyuki-sato commented Jul 16, 2025

Rationale for this change

This is the sub issue #44748.

  • SC2034: source_dir appears unused. Verify use (or export if used externally).
  • SC2086: Double quote to prevent globbing and word splitting.
  • SC2155: Declare and assign separately to avoid masking return values.
shellcheck ci/scripts/integration_hdfs.sh

In ci/scripts/integration_hdfs.sh line 22:
source_dir=${1}/cpp
^--------^ SC2034 (warning): source_dir appears unused. Verify use (or export if used externally).


In ci/scripts/integration_hdfs.sh line 25:
export CLASSPATH=$($HADOOP_HOME/bin/hadoop classpath --glob)
       ^-------^ SC2155 (warning): Declare and assign separately to avoid masking return values.
                   ^----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean:
export CLASSPATH=$("$HADOOP_HOME"/bin/hadoop classpath --glob)


In ci/scripts/integration_hdfs.sh line 45:
pushd ${build_dir}
      ^----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean:
pushd "${build_dir}"

For more information:
  https://www.shellcheck.net/wiki/SC2034 -- source_dir appears unused. Verify...
  https://www.shellcheck.net/wiki/SC2155 -- Declare and assign separately to ...
  https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing ...

What changes are included in this PR?

  • SC2034: disable shellcheck
  • SC2086: Quote variables.
  • SC2155: separate variable declaration and export.

Are these changes tested?

Yes.

Are there any user-facing changes?

No.

@github-actions github-actions bot added the awaiting review Awaiting review label Jul 16, 2025
Copy link

⚠️ GitHub issue #47125 has been automatically assigned in GitHub to PR creator.

@hiroyuki-sato
Copy link
Collaborator Author

  • integration_hdfs.sh is only used by docker-compose.yml
  • conda-python-hdfs image is only used by test-conda-python-3.10-hdfs-XXXX
rg -B 40 'integration_hdfs.sh'

docker-compose.yml
1862-
1863-  conda-python-hdfs:
1864-    # Usage:
1865-    #   docker compose build conda-cpp
1866-    #   docker compose build conda-python
1867-    #   docker compose build conda-python-hdfs
1868-    #   docker compose run conda-python-hdfs
1869-    image: ${REPO}:${ARCH}-conda-python-${PYTHON}-hdfs-${HDFS}
1870-    build:
1871-      context: .
         # .. snip
1899-    command:
1900-      ["/arrow/ci/scripts/cpp_build.sh /arrow /build &&
1901-        /arrow/ci/scripts/python_build.sh /arrow /build &&
1902:        /arrow/ci/scripts/integration_hdfs.sh /arrow /build"]
rg -B 9 'conda-python-hdfs'  dev/tasks/tasks.yml
1178-
1179-{% for hdfs_version in ["2.9.2", "3.2.1"] %}
1180-  test-conda-python-3.10-hdfs-{{ hdfs_version }}:
1181-    ci: github
1182-    template: docker-tests/github.linux.yml
1183-    params:
1184-      env:
1185-        HDFS: "{{ hdfs_version }}"
1186-        PYTHON: "3.10"
1187:      image: conda-python-hdfs

@hiroyuki-sato
Copy link
Collaborator Author

@github-actions crossbow submit -g test-conda-python-3.10

Copy link

Invalid group(s) {'test-conda-python-3.10'}. Must be one of {'linux-amd64', 'cuda', 'packaging', 'nightly', 'vcpkg', 'c-glib', 'python', 'cpp', 'integration', 'fuzz', 'verify-rc', 'wheel', 'example', 'linux-arm64', 'r', 'example-python', 'nightly-tests', 'ruby', 'nightly-release', 'conan', 'verify-rc-source-macos', 'verify-rc-source', 'linux', 'example-cpp', 'verify-rc-source-linux', 'homebrew', 'test', 'nightly-packaging'}
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/16320210784

@hiroyuki-sato
Copy link
Collaborator Author

@github-actions crossbow submit test-conda-python-3.10-hdfs-2.9.2 test-conda-python-3.10-hdfs-3.2.1

Copy link

Revision: 22886af

Submitted crossbow builds: ursacomputing/crossbow @ actions-4e3cd10a5f

Task Status
test-conda-python-3.10-hdfs-2.9.2 GitHub Actions
test-conda-python-3.10-hdfs-3.2.1 GitHub Actions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting review Awaiting review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant