GH-20: Add release scripts (#496)

Fixes GH-20. This is based on apache/arrow-go#122 . Workflow: Cut a RC: 1. Bump version by `dev/release/bump_version.sh 19.0.0` 2. Run `dev/release/release_rc.sh` 1. `dev/release/release_rc.sh` pushes `v${version}-rc${rc}` tag 2. `.github/workflows/rc.yml` creates `apache-arrow-java-${version}.tar.gz{,.sha256,.sha512}` 3. `.github/workflows/rc.yml` uploads `apache-arrow-java-${version}.tar.gz{,.sha256,.sha512}` to GitHub Releases 4. `dev/release/release_rc.sh` downloads `apache-arrow-java-${version}.tar.gz` from GitHub Releases 5. `dev/release/release_rc.sh` signs `apache-arrow-java-${version}.tar.gz` as `apache-arrow-go-${version}.tar.gz.asc` 6. `dev/release/release_rc.sh` uploads `apache-arrow-java-${version}.tar.gz.asc` to GitHub Releases 3. Start a vote (GitHub Actions instead of https://dist.apache.org/repos/dist/dev/arrow/ is used like ADBC.) Verify a RC: 1. Run `dev/release/verify_rc.sh` Release an approved RC: 1. Run `dev/release/release.sh` 1. `dev/release/release.sh` pushes `v${version}` tag that refers that same commit ID as `v${version}-rc${rc}` 2. `.github/workflows/release.yml` create a GitHub Releases for `v${version}` tag by copying the RC GitHub Releases 3. `dev/release/release.sh` downloads `apache-arrow-java-${version}.tar.gz{,.asc,.sha256,.sha512}` from GitHub Releases 4. `dev/release/release.sh` uploads `apache-arrow-java-${version}.tar.gz{,.asc,.sha256,.sha512}` to https://dist.apache.org/repos/dist/release/arrow 5. `dev/release/release.sh` removes old releases from https://dist.apache.org/repos/dist/release/arrow 2. Add this release to ASF's report database: https://reporter.apache.org/addrelease.html?arrow Follow-up tasks: * Add support for building binary packages (.jar) * Add support for releasing binary packages (.jar) * Add support for running JNI test in the verification script * Add support for running integration test in the verification script * Add support for verifying binary packages (.jar) in the verification script --------- Co-authored-by: Raúl Cumplido <[email protected]>
apache · Jan 13, 2025 · 14dcd50 · 14dcd50
1 parent de80cef
commit 14dcd50
Show file tree

Hide file tree

Showing 7 changed files with 837 additions and 0 deletions.
diff --git a/.github/workflows/rc.yml b/.github/workflows/rc.yml
@@ -0,0 +1,141 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: RC
+on:
+  push:
+    branches:
+      - "**"
+      - "!dependabot/**"
+    tags:
+      - "*-rc*"
+  pull_request:
+  schedule:
+    - cron: "0 0 * * *"
+concurrency:
+  group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
+  cancel-in-progress: true
+permissions:
+  contents: read
+jobs:
+  archive:
+    name: Archive
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    steps:
+      - name: Checkout
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        with:
+          submodules: recursive
+      - name: Prepare for tag
+        if: github.ref_type == 'tag'
+        run: |
+          version=${GITHUB_REF_NAME%-rc*}
+          version=${version#v}
+          rc=${GITHUB_REF_NAME#*-rc}
+          echo "VERSION=${version}" >> ${GITHUB_ENV}
+          echo "RC=${rc}" >> ${GITHUB_ENV}
+      - name: Prepare for branch
+        if: github.ref_type == 'branch'
+        run: |
+          version=$(grep -o '^  <version>.*</version>' "pom.xml" |
+            sed \
+              -e 's,^  <version>,,' \
+              -e 's,</version>$,,')
+          rc=$(date +%Y%m%d)
+          echo "VERSION=${version}" >> ${GITHUB_ENV}
+          echo "RC=${rc}" >> ${GITHUB_ENV}
+      - name: Archive
+        run: |
+          id="apache-arrow-java-${VERSION}"
+          tar_gz="${id}.tar.gz"
+          echo "TAR_GZ=${tar_gz}" >> ${GITHUB_ENV}
+          git archive HEAD --prefix "${id}/" --output "${tar_gz}"
+          sha256sum "${tar_gz}" > "${tar_gz}.sha256"
+          sha512sum "${tar_gz}" > "${tar_gz}.sha512"
+      - name: Audit
+        run: |
+          dev/release/run_rat.sh "${TAR_GZ}"
+      - uses: actions/upload-artifact@6f51ac03b9356f520e9adb1b1b7802705f340c2b # v4.5.0
+        with:
+          name: archive
+          path: |
+            apache-arrow-java-*
+  verify:
+    name: Verify
+    needs:
+      - archive
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os:
+          - macos-latest
+          - ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        with:
+          submodules: recursive
+      - uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
+        with:
+          name: archive
+      - name: Verify
+        run: |
+          tar_gz=$(echo apache-arrow-java-*.tar.gz)
+          version=${tar_gz#apache-arrow-java-}
+          version=${version%.tar.gz}
+          # rc isn't used with VERIFY_DOWNLOAD=0
+          if [ "${GITHUB_REF_TYPE}" = "tag" ]; then
+            rc="${GITHUB_REF_NAME#*-rc}"
+          else
+            rc=$(date +%Y%m%d)
+          fi
+          VERIFY_DEFAULT=0 \
+            VERIFY_SOURCE=1 \
+            dev/release/verify_rc.sh "${version}" "${rc}"
+  upload:
+    name: Upload
+    if: github.ref_type == 'tag'
+    needs:
+      - verify
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    steps:
+      - name: Checkout
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        with:
+          submodules: recursive
+      - uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
+        with:
+          name: archive
+      - name: Upload
+        run: |
+          # GH-499: How to create release notes?
+          version=${GITHUB_REF_NAME%-rc*}
+          version=${version#v}
+          rc=${GITHUB_REF_NAME#*-rc}
+          gh release create ${GITHUB_REF_NAME} \
+            --generate-notes \
+            --prerelease \
+            --title "Apache Arrow Java ${version} RC${rc}" \
+            --verify-tag \
+            apache-arrow-java-*.tar.gz \
+            apache-arrow-java-*.tar.gz.sha*
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
@@ -0,0 +1,54 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Release
+on:
+  push:
+    tags:
+      - "*"
+      - "!*-rc*"
+concurrency:
+  group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
+  cancel-in-progress: true
+permissions:
+  contents: write
+env:
+  GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+jobs:
+  publish:
+    name: Publish
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    steps:
+      - name: Checkout
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - name: Download RC contents
+        run: |
+          set -x
+          latest_rc_tag=$(gh release list --json tagName --jq '.[].tagName' | \
+                            grep -F "${GITHUB_REF_NAME}-rc" | \
+                            head -n1)
+          gh release download ${latest_rc_tag} --dir dists
+      - name: Create GitHub Release
+        run: |
+          # GH-499: How to create release notes?
+          version=${GITHUB_REF_NAME#v}
+          gh release create ${GITHUB_REF_NAME} \
+            --generate-notes \
+            --title "Apache Arrow Java ${version}" \
+            --verify-tag \
+            dists/*
diff --git a/dev/release/README.md b/dev/release/README.md
@@ -0,0 +1,174 @@
+<!---
+  Licensed to the Apache Software Foundation (ASF) under one
+  or more contributor license agreements.  See the NOTICE file
+  distributed with this work for additional information
+  regarding copyright ownership.  The ASF licenses this file
+  to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance
+  with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing,
+  software distributed under the License is distributed on an
+  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  KIND, either express or implied.  See the License for the
+  specific language governing permissions and limitations
+  under the License.
+-->
+
+# Release
+
+## Overview
+
+  1. Test the revision to be released
+  2. Bump version for new release (detailed later)
+  3. Prepare RC and vote (detailed later)
+  4. Publish (detailed later)
+  5. Bump version for new development (detailed later)
+
+### Bump version for new release
+
+Run `dev/release/bump_version.sh` on a working copy of your fork not
+`[email protected]:apache/arrow-java`:
+
+```console
+$ git clone [email protected]:${YOUR_GITHUB_ACCOUNT}/arrow-java.git arrow-java.${YOUR_GITHUB_ACCOUNT}
+$ cd arrow-java.${YOUR_GITHUB_ACCOUNT}
+$ GH_TOKEN=${YOUR_GITHUB_TOKEN} dev/release/bump_version.sh ${NEW_VERSION}
+```
+
+Here is an example to bump version to 19.0.0:
+
+```
+$ GH_TOKEN=${YOUR_GITHUB_TOKEN} dev/release/bump_version.sh 19.0.0
+```
+
+It creates a feature branch and adds a commit that bumps version. This
+opens a pull request from the feature branch by `gh pr create`. So you
+need `gh` command and GitHub personal access token.
+
+See also:
+https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens
+
+We need to merge the pull request before we cut a RC. If we try cut a
+RC without merging the pull request, the script to cut a RC is failed.
+
+### Prepare RC and vote
+
+Run `dev/release/release_rc.sh` on a working copy of
+`[email protected]:apache/arrow-java` not your fork:
+
+```console
+$ git clone [email protected]:apache/arrow-java.git
+$ cd arrow-java
+$ GH_TOKEN=${YOUR_GITHUB_TOKEN} dev/release/release_rc.sh ${RC}
+(Send a vote email to [email protected].
+ You can use a draft shown by release_rc.sh for the email.)
+```
+
+Here is an example to release RC1:
+
+```console
+$ GH_TOKEN=${YOUR_GITHUB_TOKEN} dev/release/release_rc.sh 1
+```
+
+The argument of `release_rc.sh` is the RC number. If RC1 has a
+problem, we'll increment the RC number such as RC2, RC3 and so on.
+
+Requirements to run `release_rc.sh`:
+
+  * You must be an Apache Arrow committer or PMC member
+  * You must prepare your PGP key for signing
+
+If you don't have a PGP key,
+https://infra.apache.org/release-signing.html#generate may be helpful.
+
+Your PGP key must be registered to the followings:
+
+  * https://dist.apache.org/repos/dist/dev/arrow/KEYS
+  * https://dist.apache.org/repos/dist/release/arrow/KEYS
+
+See the header comment of them how to add a PGP key.
+
+Apache arrow committers can update them by Subversion client with
+their ASF account. e.g.:
+
+```console
+$ svn co https://dist.apache.org/repos/dist/dev/arrow
+$ cd arrow
+$ head KEYS
+(This shows how to update KEYS)
+$ svn ci KEYS
+```
+
+### Publish
+
+We need to do the followings to publish a new release:
+
+  * Publish to apache.org
+
+Run `dev/release/release.sh` on a working copy of
+`[email protected]:apache/arrow-java` not your fork to publish to
+apache.org:
+
+```console
+$ GH_TOKEN=${YOUR_GITHUB_TOKEN} dev/release/release.sh ${VERSION} ${RC}
+```
+
+Here is an example to release 19.0.0 RC1:
+
+```console
+$ GH_TOKEN=${YOUR_GITHUB_TOKEN} dev/release/release.sh 19.0.0 1
+```
+
+Add the release to ASF's report database via [Apache Committee Report
+Helper](https://reporter.apache.org/addrelease.html?arrow).
+
+### Bump version for new development
+
+We should bump version in the main branch for new development after we
+release a new version.
+
+Run `dev/release/bump_version.sh` on a working copy of your fork not
+`[email protected]:apache/arrow-java`:
+
+```console
+$ git clone [email protected]:${YOUR_GITHUB_ACCOUNT}/arrow-java.git arrow-java.${YOUR_GITHUB_ACCOUNT}
+$ cd arrow-java.${YOUR_GITHUB_ACCOUNT}
+$ GH_TOKEN=${YOUR_GITHUB_TOKEN} dev/release/bump_version.sh ${NEW_VERSION}-SNAPSHOT
+```
+
+Here is an example to bump version to 19.0.1-SNAPSHOT:
+
+```
+$ GH_TOKEN=${YOUR_GITHUB_TOKEN} dev/release/bump_version.sh 19.0.0-SNAPSHOT
+```
+
+It creates a feature branch and adds a commit that bumps version. This
+opens a pull request from the feature branch by `gh pr create`.
+
+## Verify
+
+We have a script to verify a RC.
+
+You must install the following commands to use the script:
+
+  * `curl`
+  * `gpg`
+  * `shasum` or `sha256sum`/`sha512sum`
+  * `tar`
+
+To verify a RC, run the following command line:
+
+```console
+$ dev/release/verify_rc.sh ${VERSION} ${RC}
+```
+
+Here is an example to verify the release 19.0.0 RC1:
+
+```console
+$ dev/release/verify_rc.sh 19.0.0 1
+```
+
+If the verification is successful, the message `RC looks good!` is shown.