Combine different release pipelines by the use of reusable workflows (onnx#6277)

### Description
<!-- - Describe your changes. -->

This pull request should be the start of a pipeline (current status of
the considerations under onnx#6246) that
automatically generates a whls after tagging, signs it and publishes it
directly to Pypi to improve supply chain security. As this is all too
much at once, the changes will be introduced and tested gradually.

Changes related to this PR:
Reusable Workflows are used to merge the individual OS release builds
together (we need one process as we want to have all artifacts in one
pipeline, so there is not need to download them manuelly). The workflows
are currently not really "reusable", as we more or less use pypi
creditionals in the workflow, among other things. I reused our testpypi
weekly repo as I didn't want to change our behavoir for pypi and
testpypi.

Maybe it would make sense to adapt the process right away so that you
can trigger our "Weekly" manually via the Github interface for testing
and don't have to wait a week ;-)

#### Key points for the review:
* How do we want to configure concurrency
* What concurrency-groups do we need?
* Where do we need "cancel-in-progress"?

#### Not part of the pull request:
* Extract publish whl to pypi/testpyp
* Introduce trusted publishing
* ... 

### Motivation and Context
* see onnx#6246

---------

Signed-off-by: Andreas Fehlner <[email protected]>
Co-authored-by: Justin Chu <[email protected]>

Author: andife

.github/workflows/create_release.yml

@@ -0,0 +1,86 @@
+# Copyright (c) ONNX Project Contributors
+#
+# SPDX-License-Identifier: Apache-2.0
+
+name: Create Releases
+on:
+  schedule:
+    # Run weekly on Monday 00:00
+    - cron:  '00 00 * * MON'
+  push:
+    branches: [main, rel-*]
+  pull_request:
+    branches: [main, rel-*]
+  workflow_dispatch:    
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event_name == 'workflow_dispatch' }}
+  cancel-in-progress: true
+
+jobs: 
+
+  call-workflow-ubuntu_x86:    
+    strategy:
+      matrix:
+        os: ['ubuntu-latest']    
+    uses: ./.github/workflows/release_linux_x86_64.yml
+    with:            
+      os: "linux_x86_64" 
+
+  call-workflow-ubuntu_aarch64:    
+    strategy:
+      matrix:
+        os: ['ubuntu-latest']    
+    uses: ./.github/workflows/release_linux_aarch64.yml
+    with:            
+      os: "linux_aarch64"      
+
+  call-workflow-win:    
+    strategy:
+      matrix:
+        os: ['windows-latest']    
+    uses: ./.github/workflows/release_win.yml
+    with:      
+      os: "win"      
+
+  call-workflow-mac:    
+    strategy:
+      matrix:
+        os: ['mac-latest']    
+    uses: ./.github/workflows/release_mac.yml
+    with:            
+      os: "macos"   
+
+
+
+  publish_to_testpypi: 
+    # TODO Add a deployment step for reviewing https://docs.github.com/en/actions/managing-workflow-runs/reviewing-deployments
+    name: Release (Publish to testpypi, onnxweekly)
+    runs-on: ubuntu-latest
+    needs: [call-workflow-ubuntu_x86, call-workflow-ubuntu_aarch64, call-workflow-mac, call-workflow-win]
+    if: (github.ref == 'refs/head/main') && (github.event_name != 'pull_request') && ((needs.call-workflow-mac.result == 'success') || (needs.call-workflow-ubuntu_x86.result == 'success') || (needs.call-workflow-ubuntu_aarch64.result == 'success') || (needs.call-workflow-win.result == 'success'))
+
+    environment:      
+      name: testpypi 
+      url: https://test.pypi.org/p/onnx
+
+    permissions:
+      contents: write  # IMPORTANT: mandatory for making GitHub Releases
+      id-token: write  # IMPORTANT: mandatory for sigstore     
+
+    steps:
+
+      - uses: actions/download-artifact@v4
+        with:
+          pattern: wheels* 
+          path: dist
+          merge-multiple: true
+
+      - name: Publish distribution to TestPyPI
+        if: (github.event_name == 'schedule') && (github.repository_owner == 'onnx')
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:   
+          repository-url: https://test.pypi.org/legacy/
+          verbose: true     
+          print-hash: true  
+

.github/workflows/release_linux_aarch64.yml

@@ -4,23 +4,16 @@
 
 name: LinuxRelease_aarch64
 
-on:
-  schedule:
-    # Run weekly on Monday 00:00
-    - cron:  '00 00 * * MON'
-  push:
-    branches: [main, rel-*]
-  pull_request:
-    branches: [main, rel-*]
-  workflow_dispatch:
+on:  # Specifies the event triggering the workflow
+  workflow_call:  # Indicates that this is a reusable workflow
+    inputs:      
+      os:
+        required: true
+        type: string
 
 permissions:  # set top-level default permissions as security best practice
   contents: read
 
-concurrency:
-  group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event_name == 'workflow_dispatch' }}
-  cancel-in-progress: true
-
 jobs:
   build:
     if: github.event_name != 'pull_request' || startsWith( github.base_ref, 'rel-') || contains( github.event.pull_request.labels.*.name, 'run release CIs')
@@ -83,11 +76,12 @@ jobs:
           pytest && \
           deactivate'
 
-    - uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
+    - uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b
       with:
-        name: wheels
-        path: dist
-
+        name: wheels-${{ inputs.os }}-${{ matrix.python-version }}
+        path: |
+            ./dist/*.whl
+            
     - name: Upload wheel to PyPI weekly
       if: (github.event_name == 'schedule') # Only triggered by weekly event
       run: |

.github/workflows/release_linux_x86_64.yml

@@ -4,23 +4,16 @@
 
 name: LinuxRelease_x86_64
 
-on:
-  schedule:
-    # Run weekly on Monday 00:00
-    - cron:  '00 00 * * MON'
-  push:
-    branches: [main, rel-*]
-  pull_request:
-    branches: [main, rel-*]
-  workflow_dispatch:
+on:  # Specifies the event triggering the workflow
+  workflow_call:  # Indicates that this is a reusable workflow
+    inputs:      
+      os:
+        required: true
+        type: string  
 
 permissions:  # set top-level default permissions as security best practice
   contents: read
 
-concurrency:
-  group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event_name == 'workflow_dispatch' }}
-  cancel-in-progress: true
-
 jobs:
   build:
     if: github.event_name != 'pull_request' || startsWith( github.base_ref, 'rel-') || contains( github.event.pull_request.labels.*.name, 'run release CIs')
@@ -65,10 +58,12 @@ jobs:
         python -m pip install dist/*manylinux2014_x86_64.whl
         pytest
 
-    - uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
+    - uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b
       with:
-        name: wheels
-        path: dist
+        name: wheels-${{ inputs.os }}-${{ matrix.python-version }}
+        path: |
+            ./dist/*.whl
+        
 
     - name: Upload wheel to PyPI weekly
       if: (github.event_name == 'schedule') # Only triggered by weekly event

.github/workflows/release_mac.yml

@@ -4,15 +4,12 @@
 
 name: MacRelease
 
-on:
-  schedule:
-    # Run weekly on Monday 00:00
-    - cron:  '00 00 * * MON'
-  push:
-    branches: [main, rel-*]
-  pull_request:
-    branches: [main, rel-*]
-  workflow_dispatch:
+on:  # Specifies the event triggering the workflow
+  workflow_call:  # Indicates that this is a reusable workflow
+    inputs:      
+      os:
+        required: true
+        type: string
 
 # Use MACOSX_DEPLOYMENT_TARGET=12.0 to produce compatible wheel
 env:
@@ -21,10 +18,6 @@ env:
 permissions:
   contents: read
 
-concurrency:
-  group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event_name == 'workflow_dispatch' }}
-  cancel-in-progress: true
-
 jobs:
   build:
     if: github.event_name != 'pull_request' || startsWith( github.base_ref, 'rel-') || contains( github.event.pull_request.labels.*.name, 'run release CIs')
@@ -68,10 +61,10 @@ jobs:
         fi
         python -m build --wheel
 
-    - uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
+    - uses: actions/upload-artifact@v4
       with:
-        name: macos-wheel-${{ matrix.python-version }}
-        path: dist
+        name: wheels-${{ inputs.os }}-${{ matrix.python-version }}
+        path: dist/*.whl
 
   test:
     needs: build
@@ -93,9 +86,9 @@ jobs:
         arch -${{ matrix.target-architecture }} python -m pip install -q --upgrade pip
         arch -${{ matrix.target-architecture }} python -m pip install -q -r requirements-release.txt
 
-    - uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
+    - uses: actions/download-artifact@v4
       with:
-        name: macos-wheel-${{ matrix.python-version }}
+        name:  wheels-${{ inputs.os }}-${{ matrix.python-version }}
         path: dist
 
     - name: Test the wheel

.github/workflows/release_win.yml

@@ -4,23 +4,16 @@
 
 name: WindowsRelease
 
-on:
-  schedule:
-    # Run weekly on Monday 00:00
-    - cron:  '00 00 * * MON'
-  push:
-    branches: [main, rel-*]
-  pull_request:
-    branches: [main, rel-*]
-  workflow_dispatch:
+on:  # Specifies the event triggering the workflow
+  workflow_call:  # Indicates that this is a reusable workflow
+    inputs:      
+      os:
+        required: true
+        type: string
 
 permissions:  # set top-level default permissions as security best practice
   contents: read
 
-concurrency:
-  group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event_name == 'workflow_dispatch' }}
-  cancel-in-progress: true
-
 jobs:
   build:
     if: github.event_name != 'pull_request' || startsWith( github.base_ref, 'rel-') || contains( github.event.pull_request.labels.*.name, 'run release CIs')
@@ -79,15 +72,14 @@ jobs:
         }
         python -m build --wheel
         Get-ChildItem -Path dist/*.whl | foreach {python -m pip install --upgrade $_.fullname}
-
     - name: Test the installed wheel
       run: |
         cd onnx
         pytest
 
-    - uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
+    - uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b
       with:
-        name: wheels
+        name: wheels-${{ inputs.os }}-${{ matrix.python-version }}-${{matrix.architecture}}
         path: ./onnx/dist
 
     - name: Upload onnx-weekly wheel to PyPI/PyPI weekly