Update to tensorflow-aarch64 and pytorch-aarch64 Docker builds

- Adds more verbose welcome message
- Enables support for v8a targets
  Default TensorFlow build is now 'generic' which will now support a greater range of targets
  including A72 and Neoverse cores.
- Updates TF build to use Compute Library 21.11
- Updates PyTorch build to use Compute Library 21.11
- Disables caching of ACL softmax primitives in TensorFlow

Co-authored-by: Luke Ireland <[email protected]>
Co-authored-by: Jonathan Louis Kaplan <[email protected]>

Author: 3 people

docker/pytorch-aarch64/Dockerfile

@@ -66,8 +66,9 @@ RUN apt-get -y update && \
       libncursesw5-dev \
       libpng-dev \
       libreadline-dev \
-      libssl-dev \
+      libsox-fmt-all \
       libsqlite3-dev \
+      libssl-dev \
       libxml2-dev \
       libxslt-dev \
       locales \
@@ -90,6 +91,7 @@ RUN apt-get -y update && \
       rsyslog \
       snapd \
       scons \
+      sox \
       ssh \
       sudo \
       time \
@@ -143,7 +145,7 @@ ENV NP_MAKE="${njobs}" \
     ACL_ARCH="${acl_arch}"
 
 # Key version numbers
-ENV ACL_VERSION="v21.08" \
+ENV ACL_VERSION="v21.11" \
     OPENBLAS_VERSION=0.3.10 \
     NINJA_VERSION=1.9.0
 
@@ -345,6 +347,10 @@ RUN pip install --no-cache-dir tqdm
 RUN pip install --no-cache-dir boto3
 RUN pip install --no-cache-dir future onnx==1.8.1
 RUN pip install --no-cache-dir iopath
+RUN pip install --no-cache-dir sox
+RUN pip install --no-cache-dir unidecode
+RUN pip install --no-cache-dir inflect
+RUN pip install --no-cache-dir 'librosa==0.8.0'
 
 # Copy examples
 ENV EXAMPLE_DIR=/home/$DOCKER_USER/examples

docker/pytorch-aarch64/examples/README.md

@@ -185,11 +185,10 @@ In order to reduce the runtime, for the purposes of confirming that it runs as e
 
 #### RNNT
 
-The speech recognition RNNT benchmark ([original paper available here](https://arxiv.org/pdf/1811.06621.pdf)) generates character transcriptions from raw audio samples. It is not built by default due to the time requirements for building its dependencies, and the size of the dataset (over 1000 hours of audio).
+The speech recognition RNNT benchmark ([original paper available here](https://arxiv.org/pdf/1811.06621.pdf)) generates character transcriptions from raw audio samples. The data and model parameters are not included in the Docker image by default due to their size (~1GB), but can be downloaded easily using the shell scripts described below.
 
-Three separate shell scripts for dependency build, model and data download, and running stages are generated for the built image using a patch file. These scripts can be found in `$HOME/examples/MLCommons/inference/speech_recognition/rnnt/` of the built image.
+Two shell scripts for model and data download, and running scenarios are generated for the built image using a patch file. These scripts can be found in `$HOME/examples/MLCommons/inference/speech_recognition/rnnt/` of the built image.
 
-  * `build-rnnt.sh` builds sox with flac support from source and installs the requisite Python packages
   * `download_dataset_model.sh` downloads the model and test dataset
   * `run.sh` runs the test, by default only the SingleStream scenario latency test
 

docker/pytorch-aarch64/patches/mlcommons_rnnt.patch

@@ -22,10 +22,10 @@
 #
 # *******************************************************************************
 diff --git a/speech_recognition/rnnt/README.md b/speech_recognition/rnnt/README.md
-index 27fbabd..6be2742 100644
+index 27fbabd..2794bed 100644
 --- a/speech_recognition/rnnt/README.md
 +++ b/speech_recognition/rnnt/README.md
-@@ -4,18 +4,13 @@ character transcription, without an external language model.
+@@ -4,18 +4,12 @@ character transcription, without an external language model.
 
  # 2. Directions
 
@@ -41,7 +41,6 @@ index 27fbabd..6be2742 100644
 -
 -As you complete individual stages, you can set the variable "stage" to
 -a higher number for restarting from a later stage.
-+To build and install necessary dependencies, run `build-rnnt.sh`.
 +To download the dataset and model, run `download_dataset_model.sh`.
 +To run the model, use `run.sh`; this shell script runs the SingleStream
 +scenario by default if no others are specified. Use `run.sh --help` to
@@ -51,72 +50,19 @@ index 27fbabd..6be2742 100644
 
  # 3. Dataset/Environment
  ### Publication/Attribution
-@@ -113,4 +108,4 @@ The differences are as follows:
+@@ -113,4 +107,4 @@ The differences are as follows:
  ### Quality metric
  7.452253714852645% Word Error Rate (WER) across all words in the output text of
  all samples less than 15 seconds in length in the dev-clean set, using a greedy
 -decoder and a fully FP32 model.
 \ No newline at end of file
 +decoder and a fully FP32 model.
-diff --git a/speech_recognition/rnnt/build-rnnt.sh b/speech_recognition/rnnt/build-rnnt.sh
-new file mode 100755
-index 0000000..c2563c3
---- /dev/null
-+++ b/speech_recognition/rnnt/build-rnnt.sh
-@@ -0,0 +1,47 @@
-+#!/bin/bash
-+# *******************************************************************************
-+# Copyright 2021 Arm Limited and affiliates.
-+# SPDX-License-Identifier: Apache-2.0
-+#
-+# Licensed under the Apache License, Version 2.0 (the "License");
-+# you may not use this file except in compliance with the License.
-+# You may obtain a copy of the License at
-+#
-+#     http://www.apache.org/licenses/LICENSE-2.0
-+#
-+# Unless required by applicable law or agreed to in writing, software
-+# distributed under the License is distributed on an "AS IS" BASIS,
-+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-+# See the License for the specific language governing permissions and
-+# limitations under the License.
-+# *******************************************************************************
-+set -euo pipefail
-+
-+base="$(dirname "${BASH_SOURCE[0]}")"
-+work_dir=$base/temp_work
-+local_data_dir=$work_dir/local_data
-+librispeech_download_dir=$local_data_dir/LibriSpeech
-+
-+mkdir -p $librispeech_download_dir
-+
-+install_dir=$base/third_party/install
-+mkdir -p $install_dir
-+install_dir=$(readlink -f $install_dir)
-+
-+# if venv not active, activate
-+if [[ $(python -c "import sys; print(sys.prefix)") != "/home/ubuntu/python3-venv" ]]; then
-+    source /home/ubuntu/python3-venv/bin/activate
-+fi
-+
-+# We need to convert .flac files to .wav files via sox. Not all sox installs have flac support, so we install from source.
-+wget https://ftp.osuosl.org/pub/xiph/releases/flac/flac-1.3.2.tar.xz -O third_party/flac-1.3.2.tar.xz
-+(cd third_party; tar xf flac-1.3.2.tar.xz; cd flac-1.3.2; CFLAGS="-I/home/ubuntu/inference/speech_recognition/rnnt/third_party/pybind/include"; ./configure --prefix=$install_dir && make && make install)
-+
-+wget https://sourceforge.net/projects/sox/files/sox/14.4.2/sox-14.4.2.tar.gz -O third_party/sox-14.4.2.tar.gz
-+(cd third_party; tar zxf sox-14.4.2.tar.gz; cd sox-14.4.2; LDFLAGS="-L${install_dir}/lib" CFLAGS="-I${install_dir}/include -I/home/ubuntu/inference/speech_recognition/rnnt/third_party/pybind/include" ./configure --prefix=$install_dir --with-flac && make && make install)
-+
-+(cd $(git rev-parse --show-toplevel)/loadgen; python setup.py install)
-+
-+pip install sox
-+pip install unidecode inflect
-+pip install 'librosa==0.8.0'
 diff --git a/speech_recognition/rnnt/download_dataset_model.sh b/speech_recognition/rnnt/download_dataset_model.sh
 new file mode 100755
-index 0000000..348cdb7
+index 0000000..8d4ac05
 --- /dev/null
 +++ b/speech_recognition/rnnt/download_dataset_model.sh
-@@ -0,0 +1,49 @@
+@@ -0,0 +1,46 @@
 +#!/bin/bash
 +# *******************************************************************************
 +# Copyright 2021 Arm Limited and affiliates.
@@ -141,16 +87,13 @@ index 0000000..348cdb7
 +local_data_dir=$work_dir/local_data
 +librispeech_download_dir=$local_data_dir/LibriSpeech
 +
-+install_dir=$base/third_party/install
-+install_dir=$(readlink -f $install_dir)
++mkdir -p $librispeech_download_dir
 +
 +# if venv not active, activate
 +if [[ $(python -c "import sys; print(sys.prefix)") != "/home/ubuntu/python3-venv" ]]; then
 +    source /home/ubuntu/python3-venv/bin/activate
 +fi
 +
-+export PATH="$install_dir/bin/:$PATH"
-+
 +# stage 0: download model. Check checksum to skip?
 +wget https://zenodo.org/record/3662521/files/DistributedDataParallel_1576581068.9962234-epoch-100.pt?download=1 -O $work_dir/rnnt.pt
 +
@@ -194,10 +137,10 @@ index 9bbea9c..39e2121 100644
 -        return torch.cat(seq, dim=2)[::self.factor, :, :], x_lens
 +        return rt, x_lens
 diff --git a/speech_recognition/rnnt/run.sh b/speech_recognition/rnnt/run.sh
-index 7538df9..c9ebe43 100755
+index 7538df9..d65a87c 100755
 --- a/speech_recognition/rnnt/run.sh
 +++ b/speech_recognition/rnnt/run.sh
-@@ -1,90 +1,114 @@
+@@ -1,90 +1,108 @@
 -#/bin/bash
 -
 +#!/bin/bash
@@ -339,9 +282,10 @@ index 7538df9..c9ebe43 100755
 -      --input_dir $librispeech_download_dir/dev-clean \
 -      --dest_dir $local_data_dir/dev-clean-wav \
 -      --output_json $local_data_dir/dev-clean-wav.json
--fi
-+install_dir=$base/third_party/install
-+install_dir=$(readlink -f $install_dir)
++# if venv not active, activate
++if [[ $(python -c "import sys; print(sys.prefix)") != "/home/ubuntu/python3-venv" ]]; then
++    source /home/ubuntu/python3-venv/bin/activate
+ fi
 
 -if [[ $stage -le 3 ]]; then
 -  for backend in pytorch; do
@@ -352,11 +296,7 @@ index 7538df9..c9ebe43 100755
 -          log_dir+=_accuracy
 -        fi
 -        log_dir+=rerun
-+# if venv not active, activate
-+if [[ $(python -c "import sys; print(sys.prefix)") != "/home/ubuntu/python3-venv" ]]; then
-+    source /home/ubuntu/python3-venv/bin/activate
-+fi
- 
+-
 -        python run.py --backend pytorch \
 -               --dataset_dir $local_data_dir \
 -               --manifest $local_data_dir/dev-clean-wav.json \
@@ -366,9 +306,7 @@ index 7538df9..c9ebe43 100755
 -               --backend ${backend} \
 -               --log_dir ${log_dir} \
 -               ${accuracy} &
-+# make sox executable findable by the Python package
-+export PATH="$install_dir/bin/:$PATH"
- 
+-
 -      done
 -    done
 -  done

docker/pytorch-aarch64/patches/welcome.txt

@@ -1,8 +1,26 @@
 
 ================================================================================
 PyTorch for AArch64
-Built using:
- https://github.com/ARM-software/Tool-Solutions
-For more details, see:
- https://github.com/ARM-software/Tool-Solutions/blob/master/docker/pytorch-aarch64/README.md
 ================================================================================
+
+Built using: https://github.com/ARM-software/Tool-Solutions
+For more details, see: https://git.io/JMESY
+
+--
+
+The default user's home (/home/ubuntu) contains:
+
+- examples/ - a selection of inference examples and benchmarks:
+-- Vision, Natural Language Processing (NLP) and speech recognition
+   inference examples
+-- MLCommons/ - sample inference benchmarks for Vision and NLP,
+   see https://github.com/mlcommons/inference
+
+  For more details, and instructions on how to run the benchmarks see:
+   examples/README.md (https://git.io/JMESs)
+
+- python3-venv/ - the complete python environment including PyTorch.
+
+- CK/ containing Collective Knowledge (https://cknowledge.org) repositories
+  used to manange datasets for the MLCommons benchmarks.
+=================================================================================

docker/tensorflow-aarch64/Dockerfile

@@ -294,6 +294,7 @@ ENV PATH=$PACKAGE_DIR/bazel:$PATH
 
 # Build TensorFlow
 COPY patches/tf_acl.patch $PACKAGE_DIR/.
+COPY patches/tf_softmax.patch $PACKAGE_DIR/.
 COPY patches/mkldnn_acl.patch $PACKAGE_DIR/.
 COPY patches/compute_library.patch $PACKAGE_DIR/.
 COPY scripts/build-tensorflow-io-gcs-filesystem.sh $PACKAGE_DIR/.

docker/tensorflow-aarch64/build.sh

@@ -40,10 +40,10 @@ function print_usage_and_exit {
   echo "                                 * tensorflow   - build image including TensorFlow build and benchmarks installed (default)."
   echo "                                 * full         - build all images."
   echo "      --build-target           AArch64 CPU target:"
-  echo "                                 * native       - optimize for the current host machine (default)."
+  echo "                                 * generic      - portable build suitable for any ARMv8-A target (default)."
+  echo "                                 * native       - optimize for the current host machine."
   echo "                                 * neoverse-n1  - optimize for Neoverse-N1"
   echo "                                 * thunderx2t99 - optimize for Marvell ThunderX2."
-  echo "                                 * generic      - generate portable build suitable for any Armv8a target."
   echo "                                 * custom       - use custom settings defined in cpu_info.sh"
   echo "                                 GCC provides support for additional target cpu's refer to the gcc manual for details."
   echo "      --no-cache / --clean     Pull a new base image and build without using any cached images."
@@ -85,7 +85,7 @@ nproc_build=
 bazel_mem=
 onednn=
 enable_onednn=0
-target="native"
+target="generic"
 clean_build=
 image_tag="latest"
 

docker/tensorflow-aarch64/cpu_info.sh

@@ -115,11 +115,12 @@ function set_target {
       eigen_l3_cache="512*1024"
     ;;
     generic )
+    # A portable build, suitible for all v8.x-a targets
       cpu="generic"
       tune="generic"
       arch="armv8-a"
       blas_cpu="ARMV8"
-      blas_ncores=
+      blas_ncores=64
       eigen_l1_cache=
       eigen_l2_cache=
       eigen_l3_cache=