From 0e31430a0af745ebd40a85620bb2a8ab16f9aa0b Mon Sep 17 00:00:00 2001 From: shresthamalik Date: Tue, 25 Sep 2018 17:13:58 -0700 Subject: [PATCH 01/11] Fixed OpExecutor.cpp: SRC and SINK nodes must only be linked via control slots and control slots must only be linked to control slots --- test/opexecuter.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/test/opexecuter.cpp b/test/opexecuter.cpp index 39d23ed1..af8bc766 100644 --- a/test/opexecuter.cpp +++ b/test/opexecuter.cpp @@ -260,7 +260,7 @@ void OpExecuter::ExecuteOnNGraph() { auto src_nodes_metadata = node_inedge_metadata[ip_node]; for (int j = 0; j < src_nodes_metadata.size(); j++) { graph.AddEdge(src_nodes_metadata[j].first, src_nodes_metadata[j].second, - arg_node, 0); + arg_node, Graph::kControlSlot); } // Adds an edge from arg_node to test_op graph.AddEdge(arg_node, 0, test_op, i); @@ -292,7 +292,7 @@ void OpExecuter::ExecuteOnNGraph() { // Add edges from _Retval to sink for (int j = 0; j < dest_nodes_metadata.size(); j++) { - graph.AddEdge(ret_node, 0, dest_nodes_metadata[j].first, + graph.AddEdge(ret_node, Graph::kControlSlot, dest_nodes_metadata[j].first, dest_nodes_metadata[j].second); } // Add edges from test_op to _Retval From 3c15179eb1b39b8d16346fefd8852b0473845ca2 Mon Sep 17 00:00:00 2001 From: shresthamalik Date: Thu, 27 Sep 2018 10:00:19 -0700 Subject: [PATCH 02/11] Make changes to Readme. Added build instructions to option 3. And few more changes --- README.md | 127 +++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 101 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index 5ff52521..c31c5769 100644 --- a/README.md +++ b/README.md @@ -1,22 +1,28 @@ -# Intel® nGraph™ Compiler and runtime for TensorFlow* +# nGraph-Tensorflow : Intel® nGraph™ Compiler and Runtime for TensorFlow -This repository contains the code needed to enable Intel® nGraph™ Compiler and -runtime engine for TensorFlow. Use it to speed up your TensorFlow training and -inference workloads. The nGraph Library and runtime suite can also be used to -customize and deploy Deep Learning inference models that will "just work" with -a variety of nGraph-enabled backends: CPU, GPU, and custom silicon like the -[Intel® Nervana™ NNP](https://itpeernetwork.intel.com/inteldcisummit-artificial-intelligence/). +nGraph-Tensorflow Bridge enables TensorFlow to run with [Intel® nGraph™](https://github.com/NervanaSystems/ngraph), Compiler and +Runtime engine, speeding up training and inference workloads on hardware supported by nGraph: CPU, GPU, and custom silicon like the [Intel® Nervana™ NNP](https://itpeernetwork.intel.com/inteldcisummit-artificial-intelligence/). It integrates seamlessly with Tensorflow, allowing developers the flexibility to switch amongst different hardwares, by making minimum to no change to their code. * [Build with Linux](#linux-instructions) * [Build using OS X](#using-os-x) -* Using the stable [upstreamed](#using-the-stable-upstreamed-version) version * [Debugging](#debugging) * [Support](#support) * [How to Contribute](#how-to-contribute) - ## Linux instructions +There are 3 ways to install nGraph enabled for tensorflow. + +| Option | Build TF | Build nGraph | +| | from Source | from Source | +|:---------------:|:-----------:|:------------:|:---------: +| [1](#Option-1:) | No | Yes | +| :--: | :--: | :--: | :----: +| [2](#Option-2:) | Yes | Yes | Builds unit tests. Recommended +| | | | if planning to contribute to +| | | | nGraph-Tensorflow +| :--: | :--: | :--: | :----: +| [3](#Option-3:) | Yes | No | nGraph enabled by default ### Option 1: Use an existing TensorFlow v1.11.0-rc2 (or greater) installation @@ -51,8 +57,10 @@ a variety of nGraph-enabled backends: CPU, GPU, and custom silicon like the make install pip install -U python/dist/ngraph-0.6.0-py2.py3-none-linux_x86_64.whl - -### Option 2: Build nGraph bridge from source using TensorFlow source +To enable nGraph in your python scripts + import ngraph + +### Option 2: Build nGraph using TensorFlow source To run unit tests, or if you are planning to contribute, install the nGraph bridge using the TensorFlow source tree as follows: @@ -99,7 +107,7 @@ The installation prerequisites are the same as described in the TensorFlow Note: Depending of specific version of the Python and components already installed on your system - the list of dependent Python components vary. Typically the following components are needed: `numpy mock keras keras_application`. - Install them if your Pythn environment doesn't have them already. + Install them if your Python environment doesn't have them already. 3. Now run `./configure` and choose `no` for the following when prompted to build TensorFlow. @@ -110,7 +118,7 @@ The installation prerequisites are the same as described in the TensorFlow nGraph support: - Do you wish to build TensorFlow with nGraph support? [y/N]: n + Do you wish to build TensorFlow with nGraph support? [y/N]: N No nGraph support will be enabled for TensorFlow. Since you are building nGraph using an existing TensorFlow build, you cannot respond with `y` @@ -118,7 +126,7 @@ The installation prerequisites are the same as described in the TensorFlow nGraph - one embedded within TensorFlow and the other you build and loaded. If you want to use the nGraph embedded within TensorFlow, see the - following section on how to use the upstream version. + [following section](#option-3). Note that if you are running TensorFlow on a Skylake family processor then select `-march=broadwell` when prompted to specify the optimization flags: @@ -165,25 +173,92 @@ the dependencies. Once the build and installation steps are complete, you can start using TensorFlow with nGraph backends. -Please add the following line to enable nGraph: `import ngraph` +To enable nGraph in your python scripts + import ngraph Note: The actual filename for the pip package may be different as it's version dependent. Please check the `build/python/dist` directory for the actual pip wheel. -## Using the stable upstreamed version +### Option 3: Build nGraph with TensorFlow Source + +nGraph is being added to the TensorFlow source tree. When built with this option, there is **no need to separately build `ngraph-tf` or use `pip` to install the ngraph module**. With this configuration, your TensorFlow model scripts will work with nGraph without any changes. -nGraph is being added to the TensorFlow source tree the using pull requests from -time to time. +#### Prepare the build environment + +The installation prerequisites are the same as described in the TensorFlow [prepare environment] for linux. + +1. We use the standard build process which is a system called "bazel". These + instructions were tested with [bazel version 0.16.0]. -In order to build that version of nGraph, download the source tree as mentioned -above and select `Y` when prompted to build with nGraph. + wget https://github.com/bazelbuild/bazel/releases/download/0.16.0/bazel-0.16.0-installer-linux-x86_64.sh + chmod +x bazel-0.16.0-installer-linux-x86_64.sh + ./bazel-0.16.0-installer-linux-x86_64.sh --user + +2. Add and source the ``bin`` path to your ``~/.bashrc`` file in order to be + able to call bazel from the user's installation we set up: + + export PATH=$PATH:~/bin + source ~/.bashrc -For this final option, there is **no need to separately build `ngraph-tf` or to -use `pip` to install the ngraph module**. With this configuration, your TensorFlow -model scripts will work without any changes. +#### Installation + +1. Once TensorFlow's dependencies are installed, clone the source of the + [tensorflow] repo to your machine. + + :warning: You need the following version of TensorFlow: `v1.11.0-rc2` + + git clone https://github.com/tensorflow/tensorflow.git + cd tensorflow + git checkout v1.11.0-rc2 + git status + HEAD detached at v1.11.0-rc2 + +2. You must instantiate a specific kind of `virtualenv` to be able to proceed + with the `ngraph-tf` bridge installation. For systems with Python 3.n or + Python 2.7, these commands are + + virtualenv --system-site-packages -p python3 your_virtualenv + virtualenv --system-site-packages -p /usr/bin/python2 your_virtualenv + source your_virtualenv/bin/activate # bash, sh, ksh, or zsh + + Note: Depending of specific version of the Python and components already + installed on your system - the list of dependent Python components vary. + Typically the following components are needed: `numpy mock keras keras_application`. + Install them if your Python environment doesn't have them already. + +3. Now run `./configure` and choose the following when prompted + + CUDA support: + + Do you wish to build TensorFlow with CUDA support? [y/N]: N + No CUDA support will be enabled for TensorFlow. + + nGraph support: + + Do you wish to build TensorFlow with nGraph support? [y/N]: y + No nGraph support will be enabled for TensorFlow. + + Note that if you are running TensorFlow on a Skylake family processor then select + `-march=broadwell` when prompted to specify the optimization flags: + + Please specify optimization flags to use during compilation + when bazel option "--config=opt" is specified + [Default is -march=native]: -march=broadwell + + This is due to an issue in TensorFlow which is being actively worked on: + https://github.com/tensorflow/tensorflow/issues/17273 + +4. Prepare the pip package and the TensorFlow C++ library: + + bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package + bazel-bin/tensorflow/tools/pip_package/build_pip_package ./ + +5. Install the pip package, replacing the `tensorflow-1.*` with your + version of TensorFlow: + + pip install -U ./tensorflow-1.*whl -Note: The version that is available in the upstreamed version of TensorFlow usually -lags the features and bug fixes available in the `master` branch of this repository. +Note: The version that is available with TensorFlow usually lags the features and bug fixes available in the `master` branch of this repository. ### Running tests @@ -212,7 +287,7 @@ Export the appropriate paths to your build location; OS X uses the `DYLD_` prefi export DYLD_LIBRARY_PATH=/bazel-out/darwin-py3-opt/bin/tensorflow:$DYLD_LIBRARY_PATH export DYLD_LIBRARY_PATH=/build/ngraph/ngraph_dist/lib:$DYLD_LIBRARY_PATH -Then follow "Running tests" on Linux as described above. +Then follow ["Running tests"](#Running-tests) on Linux as described above. ## Debugging From 245c4f379ab0d6c12a52cfbf0eba22e6dd49fd3f Mon Sep 17 00:00:00 2001 From: shresthamalik Date: Thu, 27 Sep 2018 10:00:19 -0700 Subject: [PATCH 03/11] Make changes to Readme. Added build instructions to option 3. And few more changes --- README.md | 127 +++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 101 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index 5ff52521..b2a13e60 100644 --- a/README.md +++ b/README.md @@ -1,22 +1,28 @@ -# Intel® nGraph™ Compiler and runtime for TensorFlow* +# nGraph-Tensorflow : Intel® nGraph™ Compiler and Runtime for TensorFlow -This repository contains the code needed to enable Intel® nGraph™ Compiler and -runtime engine for TensorFlow. Use it to speed up your TensorFlow training and -inference workloads. The nGraph Library and runtime suite can also be used to -customize and deploy Deep Learning inference models that will "just work" with -a variety of nGraph-enabled backends: CPU, GPU, and custom silicon like the -[Intel® Nervana™ NNP](https://itpeernetwork.intel.com/inteldcisummit-artificial-intelligence/). +nGraph-Tensorflow Bridge enables TensorFlow to run with [Intel® nGraph™](https://github.com/NervanaSystems/ngraph), Compiler and +Runtime engine, speeding up training and inference workloads on hardware supported by nGraph: CPU, GPU, and custom silicon like the [Intel® Nervana™ NNP](https://itpeernetwork.intel.com/inteldcisummit-artificial-intelligence/). It integrates seamlessly with Tensorflow, allowing developers the flexibility to switch amongst different hardwares, by making minimum to no change to their code. * [Build with Linux](#linux-instructions) * [Build using OS X](#using-os-x) -* Using the stable [upstreamed](#using-the-stable-upstreamed-version) version * [Debugging](#debugging) * [Support](#support) * [How to Contribute](#how-to-contribute) - ## Linux instructions +There are 3 ways to install nGraph enabled for tensorflow. + +| Option | Build TF | Build nGraph | +| | from Source | from Source | +|:---:|:---:|:---:|:---: +| [1](#Option-1:) | No | Yes | +| :---: | :---: | :---: | :---: +| [2](#Option-2:) | Yes | Yes | Builds unit tests. Recommended +| | | | if planning to contribute to +| | | | nGraph-Tensorflow +| :---: | :---: | :---: | :---: +| [3](#Option-3:) | Yes | No | nGraph enabled by default ### Option 1: Use an existing TensorFlow v1.11.0-rc2 (or greater) installation @@ -51,8 +57,10 @@ a variety of nGraph-enabled backends: CPU, GPU, and custom silicon like the make install pip install -U python/dist/ngraph-0.6.0-py2.py3-none-linux_x86_64.whl - -### Option 2: Build nGraph bridge from source using TensorFlow source +To enable nGraph in your python scripts + import ngraph + +### Option 2: Build nGraph using TensorFlow source To run unit tests, or if you are planning to contribute, install the nGraph bridge using the TensorFlow source tree as follows: @@ -99,7 +107,7 @@ The installation prerequisites are the same as described in the TensorFlow Note: Depending of specific version of the Python and components already installed on your system - the list of dependent Python components vary. Typically the following components are needed: `numpy mock keras keras_application`. - Install them if your Pythn environment doesn't have them already. + Install them if your Python environment doesn't have them already. 3. Now run `./configure` and choose `no` for the following when prompted to build TensorFlow. @@ -110,7 +118,7 @@ The installation prerequisites are the same as described in the TensorFlow nGraph support: - Do you wish to build TensorFlow with nGraph support? [y/N]: n + Do you wish to build TensorFlow with nGraph support? [y/N]: N No nGraph support will be enabled for TensorFlow. Since you are building nGraph using an existing TensorFlow build, you cannot respond with `y` @@ -118,7 +126,7 @@ The installation prerequisites are the same as described in the TensorFlow nGraph - one embedded within TensorFlow and the other you build and loaded. If you want to use the nGraph embedded within TensorFlow, see the - following section on how to use the upstream version. + [following section](#option-3). Note that if you are running TensorFlow on a Skylake family processor then select `-march=broadwell` when prompted to specify the optimization flags: @@ -165,25 +173,92 @@ the dependencies. Once the build and installation steps are complete, you can start using TensorFlow with nGraph backends. -Please add the following line to enable nGraph: `import ngraph` +To enable nGraph in your python scripts + import ngraph Note: The actual filename for the pip package may be different as it's version dependent. Please check the `build/python/dist` directory for the actual pip wheel. -## Using the stable upstreamed version +### Option 3: Build nGraph with TensorFlow Source + +nGraph is being added to the TensorFlow source tree. When built with this option, there is **no need to separately build `ngraph-tf` or use `pip` to install the ngraph module**. With this configuration, your TensorFlow model scripts will work with nGraph without any changes. -nGraph is being added to the TensorFlow source tree the using pull requests from -time to time. +#### Prepare the build environment + +The installation prerequisites are the same as described in the TensorFlow [prepare environment] for linux. + +1. We use the standard build process which is a system called "bazel". These + instructions were tested with [bazel version 0.16.0]. -In order to build that version of nGraph, download the source tree as mentioned -above and select `Y` when prompted to build with nGraph. + wget https://github.com/bazelbuild/bazel/releases/download/0.16.0/bazel-0.16.0-installer-linux-x86_64.sh + chmod +x bazel-0.16.0-installer-linux-x86_64.sh + ./bazel-0.16.0-installer-linux-x86_64.sh --user + +2. Add and source the ``bin`` path to your ``~/.bashrc`` file in order to be + able to call bazel from the user's installation we set up: + + export PATH=$PATH:~/bin + source ~/.bashrc -For this final option, there is **no need to separately build `ngraph-tf` or to -use `pip` to install the ngraph module**. With this configuration, your TensorFlow -model scripts will work without any changes. +#### Installation + +1. Once TensorFlow's dependencies are installed, clone the source of the + [tensorflow] repo to your machine. + + :warning: You need the following version of TensorFlow: `v1.11.0-rc2` + + git clone https://github.com/tensorflow/tensorflow.git + cd tensorflow + git checkout v1.11.0-rc2 + git status + HEAD detached at v1.11.0-rc2 + +2. You must instantiate a specific kind of `virtualenv` to be able to proceed + with the `ngraph-tf` bridge installation. For systems with Python 3.n or + Python 2.7, these commands are + + virtualenv --system-site-packages -p python3 your_virtualenv + virtualenv --system-site-packages -p /usr/bin/python2 your_virtualenv + source your_virtualenv/bin/activate # bash, sh, ksh, or zsh + + Note: Depending of specific version of the Python and components already + installed on your system - the list of dependent Python components vary. + Typically the following components are needed: `numpy mock keras keras_application`. + Install them if your Python environment doesn't have them already. + +3. Now run `./configure` and choose the following when prompted + + CUDA support: + + Do you wish to build TensorFlow with CUDA support? [y/N]: N + No CUDA support will be enabled for TensorFlow. + + nGraph support: + + Do you wish to build TensorFlow with nGraph support? [y/N]: y + No nGraph support will be enabled for TensorFlow. + + Note that if you are running TensorFlow on a Skylake family processor then select + `-march=broadwell` when prompted to specify the optimization flags: + + Please specify optimization flags to use during compilation + when bazel option "--config=opt" is specified + [Default is -march=native]: -march=broadwell + + This is due to an issue in TensorFlow which is being actively worked on: + https://github.com/tensorflow/tensorflow/issues/17273 + +4. Prepare the pip package and the TensorFlow C++ library: + + bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package + bazel-bin/tensorflow/tools/pip_package/build_pip_package ./ + +5. Install the pip package, replacing the `tensorflow-1.*` with your + version of TensorFlow: + + pip install -U ./tensorflow-1.*whl -Note: The version that is available in the upstreamed version of TensorFlow usually -lags the features and bug fixes available in the `master` branch of this repository. +Note: The version that is available with TensorFlow usually lags the features and bug fixes available in the `master` branch of this repository. ### Running tests @@ -212,7 +287,7 @@ Export the appropriate paths to your build location; OS X uses the `DYLD_` prefi export DYLD_LIBRARY_PATH=/bazel-out/darwin-py3-opt/bin/tensorflow:$DYLD_LIBRARY_PATH export DYLD_LIBRARY_PATH=/build/ngraph/ngraph_dist/lib:$DYLD_LIBRARY_PATH -Then follow "Running tests" on Linux as described above. +Then follow ["Running tests"](#Running-tests) on Linux as described above. ## Debugging From 4a3290085729f677202dccdf039eb886913e231a Mon Sep 17 00:00:00 2001 From: shresthamalik Date: Thu, 27 Sep 2018 12:02:06 -0700 Subject: [PATCH 04/11] format Table --- README.md | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index b2a13e60..c388b94a 100644 --- a/README.md +++ b/README.md @@ -13,16 +13,11 @@ Runtime engine, speeding up training and inference workloads on hardware support ## Linux instructions There are 3 ways to install nGraph enabled for tensorflow. -| Option | Build TF | Build nGraph | -| | from Source | from Source | -|:---:|:---:|:---:|:---: -| [1](#Option-1:) | No | Yes | -| :---: | :---: | :---: | :---: -| [2](#Option-2:) | Yes | Yes | Builds unit tests. Recommended -| | | | if planning to contribute to -| | | | nGraph-Tensorflow -| :---: | :---: | :---: | :---: -| [3](#Option-3:) | Yes | No | nGraph enabled by default +| Option | Build TF from Source | Build nGraph from Source | Notes | +|:---:|:---:|:---:|:---: | +| [1](#Option-1:) | No | Yes | | +| [2](#Option-2:) | Yes | Yes | Builds unit tests. Recommended for contributing to nGraph-Tensorflow | +| [3](#Option-3:) | Yes | No | nGraph enabled by default | ### Option 1: Use an existing TensorFlow v1.11.0-rc2 (or greater) installation From 6768017ca649a27c5824cb5cf4dca448f23e4ca6 Mon Sep 17 00:00:00 2001 From: shresthamalik Date: Thu, 27 Sep 2018 12:13:56 -0700 Subject: [PATCH 05/11] Check links --- README.md | 27 +++++++++++++-------------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index c388b94a..48556d35 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ # nGraph-Tensorflow : Intel® nGraph™ Compiler and Runtime for TensorFlow -nGraph-Tensorflow Bridge enables TensorFlow to run with [Intel® nGraph™](https://github.com/NervanaSystems/ngraph), Compiler and -Runtime engine, speeding up training and inference workloads on hardware supported by nGraph: CPU, GPU, and custom silicon like the [Intel® Nervana™ NNP](https://itpeernetwork.intel.com/inteldcisummit-artificial-intelligence/). It integrates seamlessly with Tensorflow, allowing developers the flexibility to switch amongst different hardwares, by making minimum to no change to their code. +nGraph-Tensorflow (ngraph-tf) enables TensorFlow to run with [Intel® nGraph™](https://github.com/NervanaSystems/ngraph), Compiler and +Runtime engine, speeding up training and inference workloads on supported hardware: CPU, GPU, and custom silicon like the [Intel® Nervana™ NNP](https://itpeernetwork.intel.com/inteldcisummit-artificial-intelligence/). It integrates seamlessly with Tensorflow, allowing developers the flexibility to switch amongst different hardwares, by making minimum to no change to their code. * [Build with Linux](#linux-instructions) * [Build using OS X](#using-os-x) @@ -15,15 +15,15 @@ There are 3 ways to install nGraph enabled for tensorflow. | Option | Build TF from Source | Build nGraph from Source | Notes | |:---:|:---:|:---:|:---: | -| [1](#Option-1:) | No | Yes | | +| [1](#Option-1:-Build-nGraph-using-an-existing-TensorFlow-v1.11.0-rc2-(or-greater) -installation:) | No | Yes | | | [2](#Option-2:) | Yes | Yes | Builds unit tests. Recommended for contributing to nGraph-Tensorflow | | [3](#Option-3:) | Yes | No | nGraph enabled by default | -### Option 1: Use an existing TensorFlow v1.11.0-rc2 (or greater) installation +### Option 1: Build nGraph using an existing TensorFlow v1.11.0-rc2 (or greater) installation 1. You need to instantiate a specific kind of `virtualenv` to - be able to proceed with the `ngraph-tf` bridge installation. For + be able to proceed with the `ngraph-tf` installation. For systems with Python 3.n or Python 2.7, these commands are virtualenv --system-site-packages -p python3 your_virtualenv @@ -39,8 +39,7 @@ There are 3 ways to install nGraph enabled for tensorflow. pip install -U tensorflow -3. Checkout `v0.6.0` from the `ngraph-tf` repo and build the bridge - as follows: +3. Checkout `v0.6.0` from the `ngraph-tf` repo and build it: git clone https://github.com/NervanaSystems/ngraph-tf.git cd ngraph-tf @@ -57,8 +56,8 @@ To enable nGraph in your python scripts ### Option 2: Build nGraph using TensorFlow source -To run unit tests, or if you are planning to contribute, install the nGraph -bridge using the TensorFlow source tree as follows: +To run unit tests, or if you are planning to contribute, install nGraph-tf +using the TensorFlow source tree as follows: #### Prepare the build environment @@ -92,7 +91,7 @@ The installation prerequisites are the same as described in the TensorFlow HEAD detached at v1.11.0-rc2 2. You must instantiate a specific kind of `virtualenv` to be able to proceed - with the `ngraph-tf` bridge installation. For systems with Python 3.n or + with the `ngraph-tf` installation. For systems with Python 3.n or Python 2.7, these commands are virtualenv --system-site-packages -p python3 your_virtualenv @@ -152,7 +151,7 @@ The installation prerequisites are the same as described in the TensorFlow git checkout v0.6.0 -7. Next, build and install nGraph bridge. +7. Next, build and install nGraph-tf. :warning: Run the ngraph-tf build from within the `virtualenv`. mkdir build @@ -174,7 +173,7 @@ To enable nGraph in your python scripts Note: The actual filename for the pip package may be different as it's version dependent. Please check the `build/python/dist` directory for the actual pip wheel. -### Option 3: Build nGraph with TensorFlow Source +### Option 3: Build nGraph within TensorFlow Source nGraph is being added to the TensorFlow source tree. When built with this option, there is **no need to separately build `ngraph-tf` or use `pip` to install the ngraph module**. With this configuration, your TensorFlow model scripts will work with nGraph without any changes. @@ -209,7 +208,7 @@ The installation prerequisites are the same as described in the TensorFlow [prep HEAD detached at v1.11.0-rc2 2. You must instantiate a specific kind of `virtualenv` to be able to proceed - with the `ngraph-tf` bridge installation. For systems with Python 3.n or + with the `ngraph-tf` installation. For systems with Python 3.n or Python 2.7, these commands are virtualenv --system-site-packages -p python3 your_virtualenv @@ -327,5 +326,5 @@ See the full documentation here: [diagnostics]:diagnostics/README.md [ops]:http://ngraph.nervanasys.com/docs/latest/ops/index.html [nGraph]:https://github.com/NervanaSystems/ngraph -[ngraph-tf bridge]:https://github.com/NervanaSystems/ngraph-tf +[ngraph-tf]:https://github.com/NervanaSystems/ngraph-tf From 71dd5bd1f508e2ac993c42bbf905f6049bbeb485 Mon Sep 17 00:00:00 2001 From: shresthamalik Date: Thu, 27 Sep 2018 12:13:56 -0700 Subject: [PATCH 06/11] Check links --- README.md | 27 +++++++++++++-------------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index c388b94a..3f762a70 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ # nGraph-Tensorflow : Intel® nGraph™ Compiler and Runtime for TensorFlow -nGraph-Tensorflow Bridge enables TensorFlow to run with [Intel® nGraph™](https://github.com/NervanaSystems/ngraph), Compiler and -Runtime engine, speeding up training and inference workloads on hardware supported by nGraph: CPU, GPU, and custom silicon like the [Intel® Nervana™ NNP](https://itpeernetwork.intel.com/inteldcisummit-artificial-intelligence/). It integrates seamlessly with Tensorflow, allowing developers the flexibility to switch amongst different hardwares, by making minimum to no change to their code. +nGraph-Tensorflow (ngraph-tf) enables TensorFlow to run with [Intel® nGraph™](https://github.com/NervanaSystems/ngraph), Compiler and +Runtime engine, speeding up training and inference workloads on supported hardware: CPU, GPU, and custom silicon like the [Intel® Nervana™ NNP](https://itpeernetwork.intel.com/inteldcisummit-artificial-intelligence/). It integrates seamlessly with Tensorflow, allowing developers the flexibility to switch amongst different hardwares, by making minimum to no change to their code. * [Build with Linux](#linux-instructions) * [Build using OS X](#using-os-x) @@ -15,15 +15,15 @@ There are 3 ways to install nGraph enabled for tensorflow. | Option | Build TF from Source | Build nGraph from Source | Notes | |:---:|:---:|:---:|:---: | -| [1](#Option-1:) | No | Yes | | +| [1](#Option-1:-Build-nGraph-using an-existing-TensorFlow-installation) | No | Yes | | | [2](#Option-2:) | Yes | Yes | Builds unit tests. Recommended for contributing to nGraph-Tensorflow | | [3](#Option-3:) | Yes | No | nGraph enabled by default | -### Option 1: Use an existing TensorFlow v1.11.0-rc2 (or greater) installation +### Option 1: Build nGraph using an existing TensorFlow installation 1. You need to instantiate a specific kind of `virtualenv` to - be able to proceed with the `ngraph-tf` bridge installation. For + be able to proceed with the `ngraph-tf` installation. For systems with Python 3.n or Python 2.7, these commands are virtualenv --system-site-packages -p python3 your_virtualenv @@ -39,8 +39,7 @@ There are 3 ways to install nGraph enabled for tensorflow. pip install -U tensorflow -3. Checkout `v0.6.0` from the `ngraph-tf` repo and build the bridge - as follows: +3. Checkout `v0.6.0` from the `ngraph-tf` repo and build it: git clone https://github.com/NervanaSystems/ngraph-tf.git cd ngraph-tf @@ -57,8 +56,8 @@ To enable nGraph in your python scripts ### Option 2: Build nGraph using TensorFlow source -To run unit tests, or if you are planning to contribute, install the nGraph -bridge using the TensorFlow source tree as follows: +To run unit tests, or if you are planning to contribute, install nGraph-tf +using the TensorFlow source tree as follows: #### Prepare the build environment @@ -92,7 +91,7 @@ The installation prerequisites are the same as described in the TensorFlow HEAD detached at v1.11.0-rc2 2. You must instantiate a specific kind of `virtualenv` to be able to proceed - with the `ngraph-tf` bridge installation. For systems with Python 3.n or + with the `ngraph-tf` installation. For systems with Python 3.n or Python 2.7, these commands are virtualenv --system-site-packages -p python3 your_virtualenv @@ -152,7 +151,7 @@ The installation prerequisites are the same as described in the TensorFlow git checkout v0.6.0 -7. Next, build and install nGraph bridge. +7. Next, build and install nGraph-tf. :warning: Run the ngraph-tf build from within the `virtualenv`. mkdir build @@ -174,7 +173,7 @@ To enable nGraph in your python scripts Note: The actual filename for the pip package may be different as it's version dependent. Please check the `build/python/dist` directory for the actual pip wheel. -### Option 3: Build nGraph with TensorFlow Source +### Option 3: Build nGraph within TensorFlow Source nGraph is being added to the TensorFlow source tree. When built with this option, there is **no need to separately build `ngraph-tf` or use `pip` to install the ngraph module**. With this configuration, your TensorFlow model scripts will work with nGraph without any changes. @@ -209,7 +208,7 @@ The installation prerequisites are the same as described in the TensorFlow [prep HEAD detached at v1.11.0-rc2 2. You must instantiate a specific kind of `virtualenv` to be able to proceed - with the `ngraph-tf` bridge installation. For systems with Python 3.n or + with the `ngraph-tf` installation. For systems with Python 3.n or Python 2.7, these commands are virtualenv --system-site-packages -p python3 your_virtualenv @@ -327,5 +326,5 @@ See the full documentation here: [diagnostics]:diagnostics/README.md [ops]:http://ngraph.nervanasys.com/docs/latest/ops/index.html [nGraph]:https://github.com/NervanaSystems/ngraph -[ngraph-tf bridge]:https://github.com/NervanaSystems/ngraph-tf +[ngraph-tf]:https://github.com/NervanaSystems/ngraph-tf From 46b7f244385cb6db80ab4c5445d99f8f84002df1 Mon Sep 17 00:00:00 2001 From: shresthamalik Date: Thu, 27 Sep 2018 13:56:59 -0700 Subject: [PATCH 07/11] Links and other refactoring --- README.md | 135 +++++++++++++++++++----------------------------------- 1 file changed, 48 insertions(+), 87 deletions(-) diff --git a/README.md b/README.md index a4b06bae..eaa2462d 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ # nGraph-Tensorflow : Intel® nGraph™ Compiler and Runtime for TensorFlow nGraph-Tensorflow (ngraph-tf) enables TensorFlow to run with [Intel® nGraph™](https://github.com/NervanaSystems/ngraph), Compiler and -Runtime engine, speeding up training and inference workloads on supported hardware: CPU, GPU, and custom silicon like the [Intel® Nervana™ NNP](https://itpeernetwork.intel.com/inteldcisummit-artificial-intelligence/). It integrates seamlessly with Tensorflow, allowing developers the flexibility to switch amongst different hardwares, by making minimum to no change to their code. +Runtime engine, speeding up training and inference workloads on supported hardware: CPU, GPU, and custom silicon like the [Intel® Nervana™ NNP](https://itpeernetwork.intel.com/inteldcisummit-artificial-intelligence/). It integrates seamlessly with Tensorflow, allowing developers the flexibility to switch amongst different hardware, by making minimum to no change to their code. * [Build with Linux](#linux-instructions) * [Build using OS X](#using-os-x) @@ -15,20 +15,42 @@ There are 3 ways to install nGraph enabled for tensorflow. | Option | Build TF from Source | Build nGraph from Source | Notes | |:---:|:---:|:---:|:---: | -| [1](#option-1:-build-ngraph-using-an-existing-tensorflow-installation) | No | Yes | | -| [2](#option-2:-build-ngraph-using-tensorFlow-source) | Yes | Yes | Builds unit tests. Recommended for contributing to nGraph-Tensorflow | -| [3](#option-3:-build-tensorFlow-source-with-ngraph) | Yes | No | nGraph enabled by default | +| [1](#option-1-build-ngraph-using-an-existing-tensorflow-installation) | No | Yes | | +| [2](#option-2-build-ngraph-using-tensorflow-source) | Yes | Yes | Builds unit tests. Recommended for contributing to nGraph-Tensorflow | +| [3](#option-3-build-tensorflow-source-with-ngraph) | Yes | No | nGraph enabled by default | +#### Create a python virtual environment -### Option 1: Build nGraph using an existing TensorFlow installation - -1. You need to instantiate a specific kind of `virtualenv` to - be able to proceed with the `ngraph-tf` installation. For - systems with Python 3.n or Python 2.7, these commands are +You need to instantiate a specific kind of `virtualenv` to be able to proceed with the `ngraph-tf` installation. For Python 3.n or Python 2.7, do virtualenv --system-site-packages -p python3 your_virtualenv virtualenv --system-site-packages -p /usr/bin/python2 your_virtualenv source your_virtualenv/bin/activate # bash, sh, ksh, or zsh + +Note: Depending on specific version of the Python and components already installed on your system - the list of dependent Python components vary. Typically the following components are needed: `numpy mock keras keras_applications`. + + pip install -U numpy mock keras keras_applications + +#### Install bazel for building TensorFlow + +The installation prerequisites are the same as described in the TensorFlow [prepare environment] for linux. + +1. We use the standard build process which is a system called "bazel". These + instructions were tested with [bazel version 0.16.0]. + + wget https://github.com/bazelbuild/bazel/releases/download/0.16.0/bazel-0.16.0-installer-linux-x86_64.sh + chmod +x bazel-0.16.0-installer-linux-x86_64.sh + ./bazel-0.16.0-installer-linux-x86_64.sh --user + +2. Add and source the ``bin`` path to your ``~/.bashrc`` file in order to be + able to call bazel from the user's installation we set up: + + export PATH=$PATH:~/bin + source ~/.bashrc + +### Option 1: Build nGraph using an existing TensorFlow installation + +1. Create a [python virtual environment] (#create-a-python-virtual-environment) 2. Install TensorFlow v1.11.0-rc2. Note that this is a pre-release so you need to use the following steps to install this: @@ -57,30 +79,11 @@ To enable nGraph in your python scripts ### Option 2: Build nGraph using TensorFlow source To run unit tests, or if you are planning to contribute, install nGraph-tf -using the TensorFlow source tree as follows: - -#### Prepare the build environment - -The installation prerequisites are the same as described in the TensorFlow -[prepare environment] for linux. - -1. We use the standard build process which is a system called "bazel". These - instructions were tested with [bazel version 0.16.0]. - - wget https://github.com/bazelbuild/bazel/releases/download/0.16.0/bazel-0.16.0-installer-linux-x86_64.sh - chmod +x bazel-0.16.0-installer-linux-x86_64.sh - ./bazel-0.16.0-installer-linux-x86_64.sh --user - -2. Add and source the ``bin`` path to your ``~/.bashrc`` file in order to be - able to call bazel from the user's installation we set up: - - export PATH=$PATH:~/bin - source ~/.bashrc +using the TensorFlow source tree as follows: #### Installation -1. Once TensorFlow's dependencies are installed, clone the source of the - [tensorflow] repo to your machine. +1. Install [bazel](#install-bazel-for-building-tensorFlow) and other TensorFlow dependencies. Now, clone the source of the [tensorflow] repo to your machine. :warning: You need the following version of TensorFlow: `v1.11.0-rc2` @@ -90,18 +93,7 @@ The installation prerequisites are the same as described in the TensorFlow git status HEAD detached at v1.11.0-rc2 -2. You must instantiate a specific kind of `virtualenv` to be able to proceed - with the `ngraph-tf` installation. For systems with Python 3.n or - Python 2.7, these commands are - - virtualenv --system-site-packages -p python3 your_virtualenv - virtualenv --system-site-packages -p /usr/bin/python2 your_virtualenv - source your_virtualenv/bin/activate # bash, sh, ksh, or zsh - - Note: Depending of specific version of the Python and components already - installed on your system - the list of dependent Python components vary. - Typically the following components are needed: `numpy mock keras keras_application`. - Install them if your Python environment doesn't have them already. +2. Create a [python virtual environment] (#create-a-python-virtual-environment) 3. Now run `./configure` and choose `no` for the following when prompted to build TensorFlow. @@ -156,7 +148,7 @@ The installation prerequisites are the same as described in the TensorFlow mkdir build cd build - cmake -DUNIT_TEST_ENABLE=TRUE -DTF_SRC_DIR= .. + cmake -DUNIT_TEST_ENABLE=TRUE -DTF_SRC_DIR= .. make -j make install pip install -U python/dist/ @@ -173,31 +165,15 @@ To enable nGraph in your python scripts Note: The actual filename for the pip package may be different as it's version dependent. Please check the `build/python/dist` directory for the actual pip wheel. +You can run tests following the instructions [here](#running-tests). + ### Option 3: Build TensorFlow Source with nGraph nGraph is being added to the TensorFlow source tree. When built with this option, there is **no need to separately build `ngraph-tf` or use `pip` to install the ngraph module**. With this configuration, your TensorFlow model scripts will work with nGraph without any changes. -#### Prepare the build environment - -The installation prerequisites are the same as described in the TensorFlow [prepare environment] for linux. - -1. We use the standard build process which is a system called "bazel". These - instructions were tested with [bazel version 0.16.0]. - - wget https://github.com/bazelbuild/bazel/releases/download/0.16.0/bazel-0.16.0-installer-linux-x86_64.sh - chmod +x bazel-0.16.0-installer-linux-x86_64.sh - ./bazel-0.16.0-installer-linux-x86_64.sh --user - -2. Add and source the ``bin`` path to your ``~/.bashrc`` file in order to be - able to call bazel from the user's installation we set up: - - export PATH=$PATH:~/bin - source ~/.bashrc - #### Installation -1. Once TensorFlow's dependencies are installed, clone the source of the - [tensorflow] repo to your machine. +1. Install [bazel](#install-bazel-for-building-tensorFlow) and other TensorFlow dependencies. Now, clone the source of the [tensorflow] repo to your machine. :warning: You need the following version of TensorFlow: `v1.11.0-rc2` @@ -207,18 +183,7 @@ The installation prerequisites are the same as described in the TensorFlow [prep git status HEAD detached at v1.11.0-rc2 -2. You must instantiate a specific kind of `virtualenv` to be able to proceed - with the `ngraph-tf` installation. For systems with Python 3.n or - Python 2.7, these commands are - - virtualenv --system-site-packages -p python3 your_virtualenv - virtualenv --system-site-packages -p /usr/bin/python2 your_virtualenv - source your_virtualenv/bin/activate # bash, sh, ksh, or zsh - - Note: Depending of specific version of the Python and components already - installed on your system - the list of dependent Python components vary. - Typically the following components are needed: `numpy mock keras keras_application`. - Install them if your Python environment doesn't have them already. +2. Create a [python virtual environment] (#create-a-python-virtual-environment) 3. Now run `./configure` and choose the following when prompted @@ -254,6 +219,15 @@ The installation prerequisites are the same as described in the TensorFlow [prep Note: The version that is available with TensorFlow usually lags the features and bug fixes available in the `master` branch of this repository. +## Using OS X + +The build and installation instructions are idential for Ubuntu 16.04 and OS X. + +To [run tests](#running-tests), export the appropriate paths to your build location; OS X uses the `DYLD_` prefix: + + export DYLD_LIBRARY_PATH=/bazel-out/darwin-py3-opt/bin/tensorflow:$DYLD_LIBRARY_PATH + export DYLD_LIBRARY_PATH=/build/ngraph/ngraph_dist/lib:$DYLD_LIBRARY_PATH + ### Running tests To run the C++ unit tests, @@ -270,19 +244,6 @@ run the following model with some MNIST data on your local machine: cd examples/mnist python mnist_fprop_only.py --data_dir -## Using OS X - -The build and installation instructions are idential for Ubuntu 16.04 and OS X. - -### Running tests - -Export the appropriate paths to your build location; OS X uses the `DYLD_` prefix: - - export DYLD_LIBRARY_PATH=/bazel-out/darwin-py3-opt/bin/tensorflow:$DYLD_LIBRARY_PATH - export DYLD_LIBRARY_PATH=/build/ngraph/ngraph_dist/lib:$DYLD_LIBRARY_PATH - -Then follow ["Running tests"](#Running-tests) on Linux as described above. - ## Debugging See the instructions provided in the [diagnostics] directory. From 2b1e95fbf058710bf7a4be3bbdc7f399ad22bf66 Mon Sep 17 00:00:00 2001 From: shresthamalik Date: Thu, 27 Sep 2018 13:56:59 -0700 Subject: [PATCH 08/11] Links and other refactoring --- README.md | 138 ++++++++++++++++++++---------------------------------- 1 file changed, 50 insertions(+), 88 deletions(-) diff --git a/README.md b/README.md index a4b06bae..10201dd3 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,8 @@ # nGraph-Tensorflow : Intel® nGraph™ Compiler and Runtime for TensorFlow -nGraph-Tensorflow (ngraph-tf) enables TensorFlow to run with [Intel® nGraph™](https://github.com/NervanaSystems/ngraph), Compiler and -Runtime engine, speeding up training and inference workloads on supported hardware: CPU, GPU, and custom silicon like the [Intel® Nervana™ NNP](https://itpeernetwork.intel.com/inteldcisummit-artificial-intelligence/). It integrates seamlessly with Tensorflow, allowing developers the flexibility to switch amongst different hardwares, by making minimum to no change to their code. +nGraph-Tensorflow (ngraph-tf) enables TensorFlow to run with [Intel® nGraph™](https://github.com/NervanaSystems/ngraph), compiler and +runtime engine, speeding up training and inference workloads on nGraph +supported hardware: CPU, GPU, and custom silicon like the [Intel® Nervana™ NNP](https://itpeernetwork.intel.com/inteldcisummit-artificial-intelligence/). It integrates seamlessly with Tensorflow, allowing developers the flexibility to switch amongst different hardware, by making minimum to no change to their code. * [Build with Linux](#linux-instructions) * [Build using OS X](#using-os-x) @@ -15,20 +16,42 @@ There are 3 ways to install nGraph enabled for tensorflow. | Option | Build TF from Source | Build nGraph from Source | Notes | |:---:|:---:|:---:|:---: | -| [1](#option-1:-build-ngraph-using-an-existing-tensorflow-installation) | No | Yes | | -| [2](#option-2:-build-ngraph-using-tensorFlow-source) | Yes | Yes | Builds unit tests. Recommended for contributing to nGraph-Tensorflow | -| [3](#option-3:-build-tensorFlow-source-with-ngraph) | Yes | No | nGraph enabled by default | +| [1](#option-1-build-ngraph-using-an-existing-tensorflow-installation) | No | Yes | | +| [2](#option-2-build-ngraph-using-tensorflow-source) | Yes | Yes | Builds unit tests. Recommended for contributing to nGraph-Tensorflow | +| [3](#option-3-build-tensorflow-source-with-ngraph) | Yes | No | nGraph enabled by default | +#### Create a python virtual environment -### Option 1: Build nGraph using an existing TensorFlow installation - -1. You need to instantiate a specific kind of `virtualenv` to - be able to proceed with the `ngraph-tf` installation. For - systems with Python 3.n or Python 2.7, these commands are +You need to instantiate a specific kind of `virtualenv` to be able to proceed with the `ngraph-tf` installation. For Python 3.n or Python 2.7, do virtualenv --system-site-packages -p python3 your_virtualenv virtualenv --system-site-packages -p /usr/bin/python2 your_virtualenv source your_virtualenv/bin/activate # bash, sh, ksh, or zsh + +Note: Depending on specific version of the Python and components already installed on your system - the list of dependent Python components vary. Typically the following components are needed: `numpy mock keras keras_applications`. + + pip install -U numpy mock keras keras_applications + +#### Install bazel for building TensorFlow + +The installation prerequisites are the same as described in the TensorFlow [prepare environment] for linux. + +1. We use the standard build process which is a system called "bazel". These + instructions were tested with [bazel version 0.16.0]. + + wget https://github.com/bazelbuild/bazel/releases/download/0.16.0/bazel-0.16.0-installer-linux-x86_64.sh + chmod +x bazel-0.16.0-installer-linux-x86_64.sh + ./bazel-0.16.0-installer-linux-x86_64.sh --user + +2. Add and source the ``bin`` path to your ``~/.bashrc`` file in order to be + able to call bazel from the user's installation we set up: + + export PATH=$PATH:~/bin + source ~/.bashrc + +### Option 1: Build nGraph using an existing TensorFlow installation + +1. Create a [python virtual environment](#create-a-python-virtual-environment) 2. Install TensorFlow v1.11.0-rc2. Note that this is a pre-release so you need to use the following steps to install this: @@ -57,30 +80,11 @@ To enable nGraph in your python scripts ### Option 2: Build nGraph using TensorFlow source To run unit tests, or if you are planning to contribute, install nGraph-tf -using the TensorFlow source tree as follows: - -#### Prepare the build environment - -The installation prerequisites are the same as described in the TensorFlow -[prepare environment] for linux. - -1. We use the standard build process which is a system called "bazel". These - instructions were tested with [bazel version 0.16.0]. - - wget https://github.com/bazelbuild/bazel/releases/download/0.16.0/bazel-0.16.0-installer-linux-x86_64.sh - chmod +x bazel-0.16.0-installer-linux-x86_64.sh - ./bazel-0.16.0-installer-linux-x86_64.sh --user - -2. Add and source the ``bin`` path to your ``~/.bashrc`` file in order to be - able to call bazel from the user's installation we set up: - - export PATH=$PATH:~/bin - source ~/.bashrc +using the TensorFlow source tree as follows: #### Installation -1. Once TensorFlow's dependencies are installed, clone the source of the - [tensorflow] repo to your machine. +1. Install [bazel](#install-bazel-for-building-tensorFlow) and other TensorFlow dependencies. Now, clone the source of the [tensorflow] repo to your machine. :warning: You need the following version of TensorFlow: `v1.11.0-rc2` @@ -90,18 +94,7 @@ The installation prerequisites are the same as described in the TensorFlow git status HEAD detached at v1.11.0-rc2 -2. You must instantiate a specific kind of `virtualenv` to be able to proceed - with the `ngraph-tf` installation. For systems with Python 3.n or - Python 2.7, these commands are - - virtualenv --system-site-packages -p python3 your_virtualenv - virtualenv --system-site-packages -p /usr/bin/python2 your_virtualenv - source your_virtualenv/bin/activate # bash, sh, ksh, or zsh - - Note: Depending of specific version of the Python and components already - installed on your system - the list of dependent Python components vary. - Typically the following components are needed: `numpy mock keras keras_application`. - Install them if your Python environment doesn't have them already. +2. Create a [python virtual environment](#create-a-python-virtual-environment) 3. Now run `./configure` and choose `no` for the following when prompted to build TensorFlow. @@ -156,7 +149,7 @@ The installation prerequisites are the same as described in the TensorFlow mkdir build cd build - cmake -DUNIT_TEST_ENABLE=TRUE -DTF_SRC_DIR= .. + cmake -DUNIT_TEST_ENABLE=TRUE -DTF_SRC_DIR= .. make -j make install pip install -U python/dist/ @@ -173,31 +166,15 @@ To enable nGraph in your python scripts Note: The actual filename for the pip package may be different as it's version dependent. Please check the `build/python/dist` directory for the actual pip wheel. +You can run tests following the instructions [here](#running-tests). + ### Option 3: Build TensorFlow Source with nGraph nGraph is being added to the TensorFlow source tree. When built with this option, there is **no need to separately build `ngraph-tf` or use `pip` to install the ngraph module**. With this configuration, your TensorFlow model scripts will work with nGraph without any changes. -#### Prepare the build environment - -The installation prerequisites are the same as described in the TensorFlow [prepare environment] for linux. - -1. We use the standard build process which is a system called "bazel". These - instructions were tested with [bazel version 0.16.0]. - - wget https://github.com/bazelbuild/bazel/releases/download/0.16.0/bazel-0.16.0-installer-linux-x86_64.sh - chmod +x bazel-0.16.0-installer-linux-x86_64.sh - ./bazel-0.16.0-installer-linux-x86_64.sh --user - -2. Add and source the ``bin`` path to your ``~/.bashrc`` file in order to be - able to call bazel from the user's installation we set up: - - export PATH=$PATH:~/bin - source ~/.bashrc - #### Installation -1. Once TensorFlow's dependencies are installed, clone the source of the - [tensorflow] repo to your machine. +1. Install [bazel](#install-bazel-for-building-tensorFlow) and other TensorFlow dependencies. Now, clone the source of the [tensorflow] repo to your machine. :warning: You need the following version of TensorFlow: `v1.11.0-rc2` @@ -207,18 +184,7 @@ The installation prerequisites are the same as described in the TensorFlow [prep git status HEAD detached at v1.11.0-rc2 -2. You must instantiate a specific kind of `virtualenv` to be able to proceed - with the `ngraph-tf` installation. For systems with Python 3.n or - Python 2.7, these commands are - - virtualenv --system-site-packages -p python3 your_virtualenv - virtualenv --system-site-packages -p /usr/bin/python2 your_virtualenv - source your_virtualenv/bin/activate # bash, sh, ksh, or zsh - - Note: Depending of specific version of the Python and components already - installed on your system - the list of dependent Python components vary. - Typically the following components are needed: `numpy mock keras keras_application`. - Install them if your Python environment doesn't have them already. +2. Create a [python virtual environment] (#create-a-python-virtual-environment) 3. Now run `./configure` and choose the following when prompted @@ -254,6 +220,15 @@ The installation prerequisites are the same as described in the TensorFlow [prep Note: The version that is available with TensorFlow usually lags the features and bug fixes available in the `master` branch of this repository. +## Using OS X + +The build and installation instructions are idential for Ubuntu 16.04 and OS X. + +To [run tests](#running-tests), export the appropriate paths to your build location; OS X uses the `DYLD_` prefix: + + export DYLD_LIBRARY_PATH=/bazel-out/darwin-py3-opt/bin/tensorflow:$DYLD_LIBRARY_PATH + export DYLD_LIBRARY_PATH=/build/ngraph/ngraph_dist/lib:$DYLD_LIBRARY_PATH + ### Running tests To run the C++ unit tests, @@ -270,19 +245,6 @@ run the following model with some MNIST data on your local machine: cd examples/mnist python mnist_fprop_only.py --data_dir -## Using OS X - -The build and installation instructions are idential for Ubuntu 16.04 and OS X. - -### Running tests - -Export the appropriate paths to your build location; OS X uses the `DYLD_` prefix: - - export DYLD_LIBRARY_PATH=/bazel-out/darwin-py3-opt/bin/tensorflow:$DYLD_LIBRARY_PATH - export DYLD_LIBRARY_PATH=/build/ngraph/ngraph_dist/lib:$DYLD_LIBRARY_PATH - -Then follow ["Running tests"](#Running-tests) on Linux as described above. - ## Debugging See the instructions provided in the [diagnostics] directory. From ee6f7bc41d04219891f1f3fe89554540888bb3d9 Mon Sep 17 00:00:00 2001 From: shresthamalik Date: Thu, 27 Sep 2018 14:22:57 -0700 Subject: [PATCH 09/11] Minor changes --- README.md | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 10201dd3..9b7d5add 100644 --- a/README.md +++ b/README.md @@ -28,30 +28,30 @@ You need to instantiate a specific kind of `virtualenv` to be able to proceed w virtualenv --system-site-packages -p /usr/bin/python2 your_virtualenv source your_virtualenv/bin/activate # bash, sh, ksh, or zsh -Note: Depending on specific version of the Python and components already installed on your system - the list of dependent Python components vary. Typically the following components are needed: `numpy mock keras keras_applications`. +Typically the following python packages are also needed `numpy mock keras keras_applications`. pip install -U numpy mock keras keras_applications -#### Install bazel for building TensorFlow +Note: Depending on the version of Python and the packages already installed on your system,the above list may vary. + +#### Install bazel for building TensorFlow Source The installation prerequisites are the same as described in the TensorFlow [prepare environment] for linux. -1. We use the standard build process which is a system called "bazel". These - instructions were tested with [bazel version 0.16.0]. +1. We use the standard build process called "bazel". The instructions were tested with [bazel version 0.16.0]. wget https://github.com/bazelbuild/bazel/releases/download/0.16.0/bazel-0.16.0-installer-linux-x86_64.sh chmod +x bazel-0.16.0-installer-linux-x86_64.sh ./bazel-0.16.0-installer-linux-x86_64.sh --user -2. Add and source the ``bin`` path to your ``~/.bashrc`` file in order to be - able to call bazel from the user's installation we set up: +2. Add and source the ``bin`` path to your ``~/.bashrc`` file in order to call bazel from the installation we set up: export PATH=$PATH:~/bin source ~/.bashrc ### Option 1: Build nGraph using an existing TensorFlow installation -1. Create a [python virtual environment](#create-a-python-virtual-environment) +1. Create and activate a [python virtual environment](#create-a-python-virtual-environment) 2. Install TensorFlow v1.11.0-rc2. Note that this is a pre-release so you need to use the following steps to install this: @@ -75,6 +75,7 @@ The installation prerequisites are the same as described in the TensorFlow [prep pip install -U python/dist/ngraph-0.6.0-py2.py3-none-linux_x86_64.whl To enable nGraph in your python scripts + import ngraph ### Option 2: Build nGraph using TensorFlow source @@ -84,7 +85,7 @@ using the TensorFlow source tree as follows: #### Installation -1. Install [bazel](#install-bazel-for-building-tensorFlow) and other TensorFlow dependencies. Now, clone the source of the [tensorflow] repo to your machine. +1. Install [bazel](#install-bazel-for-building-tensorflow-source) and other TensorFlow dependencies. Now, clone the source of the [tensorflow] repo to your machine. :warning: You need the following version of TensorFlow: `v1.11.0-rc2` @@ -94,7 +95,7 @@ using the TensorFlow source tree as follows: git status HEAD detached at v1.11.0-rc2 -2. Create a [python virtual environment](#create-a-python-virtual-environment) +2. Create and activate a [python virtual environment](#create-a-python-virtual-environment) 3. Now run `./configure` and choose `no` for the following when prompted to build TensorFlow. @@ -113,9 +114,9 @@ using the TensorFlow source tree as follows: nGraph - one embedded within TensorFlow and the other you build and loaded. If you want to use the nGraph embedded within TensorFlow, see the - [following section](#option-3). + [following section](#option-3-build-tensorflow-source-with-ngraph). - Note that if you are running TensorFlow on a Skylake family processor then select + Note: If you are running TensorFlow on a Skylake family processor then select `-march=broadwell` when prompted to specify the optimization flags: Please specify optimization flags to use during compilation @@ -161,6 +162,7 @@ Once the build and installation steps are complete, you can start using TensorFl with nGraph backends. To enable nGraph in your python scripts + import ngraph Note: The actual filename for the pip package may be different as it's version @@ -174,7 +176,7 @@ nGraph is being added to the TensorFlow source tree. When built with this option #### Installation -1. Install [bazel](#install-bazel-for-building-tensorFlow) and other TensorFlow dependencies. Now, clone the source of the [tensorflow] repo to your machine. +1. Install [bazel](#install-bazel-for-building-tensorflow-source) and other TensorFlow dependencies. Now, clone the source of the [tensorflow] repo to your machine. :warning: You need the following version of TensorFlow: `v1.11.0-rc2` @@ -184,7 +186,7 @@ nGraph is being added to the TensorFlow source tree. When built with this option git status HEAD detached at v1.11.0-rc2 -2. Create a [python virtual environment] (#create-a-python-virtual-environment) +2. Create and activate a [python virtual environment](#create-a-python-virtual-environment) 3. Now run `./configure` and choose the following when prompted From 57e84146fd67110cab7eba8aa06ecc1da1185e74 Mon Sep 17 00:00:00 2001 From: shresthamalik Date: Thu, 27 Sep 2018 14:39:59 -0700 Subject: [PATCH 10/11] Minor Changes --- README.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 9b7d5add..18a73e47 100644 --- a/README.md +++ b/README.md @@ -74,7 +74,7 @@ The installation prerequisites are the same as described in the TensorFlow [prep make install pip install -U python/dist/ngraph-0.6.0-py2.py3-none-linux_x86_64.whl -To enable nGraph in your python scripts +To enable nGraph, in your python scripts import ngraph @@ -161,7 +161,7 @@ the dependencies. Once the build and installation steps are complete, you can start using TensorFlow with nGraph backends. -To enable nGraph in your python scripts +To enable nGraph, in your python scripts import ngraph @@ -198,7 +198,7 @@ nGraph is being added to the TensorFlow source tree. When built with this option nGraph support: Do you wish to build TensorFlow with nGraph support? [y/N]: y - No nGraph support will be enabled for TensorFlow. + nGraph support will be enabled for TensorFlow. Note that if you are running TensorFlow on a Skylake family processor then select `-march=broadwell` when prompted to specify the optimization flags: @@ -249,10 +249,8 @@ run the following model with some MNIST data on your local machine: ## Debugging -See the instructions provided in the [diagnostics] directory. - -https://github.com/NervanaSystems/ngraph-tf/blob/master/diagnostics/README.md - +See the instructions provided in the [diagnostics](https://github.com/NervanaSystems/ngraph-tf/blob/master/diagnostics/README.md +) directory. ## Support From 97d8657fd084677c21891ce41e93aa5a1d21b131 Mon Sep 17 00:00:00 2001 From: shresthamalik Date: Thu, 18 Oct 2018 10:15:29 -0700 Subject: [PATCH 11/11] Added keras preprocessing to the list --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 2c0729b6..44b834cf 100644 --- a/README.md +++ b/README.md @@ -28,9 +28,9 @@ You need to instantiate a specific kind of `virtualenv` to be able to proceed w virtualenv --system-site-packages -p /usr/bin/python2 your_virtualenv source your_virtualenv/bin/activate # bash, sh, ksh, or zsh -Typically the following python packages are also needed `numpy mock keras keras_applications`. +Typically the following python packages are also needed `numpy mock keras keras_applications keras_preprocessing`. - pip install -U numpy mock keras keras_applications + pip install -U numpy mock keras keras_applications keras_preprocessing Note: Depending on the version of Python and the packages already installed on your system,the above list may vary.