tfc_end2end_example has pynq driver python dependency issues when trying to be deployed on FPGA

[Andy4Worc]

Prerequisites

Please make sure to check off these prerequisites before submitting a bug report.

• [ x] Test that the bug appears on the current version of the dev-branch. Make sure to include the commit hash of the commit you checked out.
• [ x] Check that the issue hasn't already been reported, by checking the currently open issues.
• [ x] If there are steps to reproduce the problem, make sure to write them down below.
• If relevant, please include the ONNX files, which were created directly before and/or after the bug.

Quick summary

tfc example deployment's execution on pynq is missing a lot of python dependencies and so the tutorial fails.

Details

Following the regular setup of FINN, I am able to run the FINN flow in the notebook in the docker container that gets launched.

Steps to Reproduce

1. Clone the FINN repository
2. Checkout the dev branch, with commit hash: 6f5aade
3. Start the docker container with the command: ./run-docker.sh notebook
4. Run full pynq "tfc" tutorial up till the point where it needs uploaded to the FPGA,

Note: this happens with either the "Z1" and the Z2" board selected (e.g. "pynq_board = "Pynq-Z2"").

Expected behavior

No errors or dependency issues and/or the "pynq-venv" having all dependencies.

Actual behavior

After copying the deployment file (.../notebooks/end2end_example/bnn-pynq/deploy-on-pynq-tfc.zip) file to a location in "\home\xilinx" on the FPGA board's memory, I run the commands provided in a terminal (via opening the terminal in the jupyter notebook main menu section). The first commands work fine. But when I try to run this command:
sudo python3 driver.py --exec_mode=execute --batchsize=1 --bitfile=resizer.bit --inputfile=input.npy

Then everything fails horribly.

First I notice that regardless of if I run the Z1 or Z2, the .zip deployment file is missing a lot of dependencies. It has, for example, a small portion of the qonnx repo, however the driver.py tries to import modelwrapper.py under qonnx/core which is a file that is missing in the driver created. So, I copied the qonnx directory from the git hub repo and copied it to the fpga to resolve that issue.

Other missing modules I've bee resolving one at a time:
onnx (I downloaded the src code onnx/onnx and copied it to be board). Resolved.
google (I did a pip install and also had to do: "pip3 install --upgrade google-api-python-client"). Resolved.
More... right now, I am getting an error: "ImportError: cannot import name 'ONNX_ML' from 'onnx.onnx_cpp2py_export' (unknown location)"

But I don't think this is the main issue. The environment should be set up already. I tried enabling the virtual environment - "/usr/local/share/pynq-venv" and sourcing the "xrt_setup.sh" script as well (google recommends) to try and help but it didn't.

Lastly, "sudo" isn't working directly. I have to do "sudo /usr/local/share/pynq-venv/bin/python3 driver.py ..." for it to work as well as non-sudo (either have the dependency issues).

[Andy4Worc]

The "cybersecurity" 3 notebook examples all run just fine for me with no failures or warnings. However, it also has a "sparse" deployment directory that would have the same dependency issues.

[Andy4Worc]

My bad, the cybersecurity did have some brevitas warnings during training with the first and second notebooks (shown below). However, I doubt these are relevant to the pynq driver issue.

UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at /pytorch/c10/core/TensorImpl.h:1935.)
  return super().rename(names)
/home/andy/Documents/FINN_dev/finn/deps/brevitas/src/brevitas/nn/quant_linear.py:69: UserWarning: Defining your `__torch_function__` as a plain method is deprecated and will be an error in future, please define it as a classmethod. 

[auphelia]

Thanks @Andy4Worc, we will have a look into that. It sounds like you have already a few fixes applied, would you like to create a PR for that to dev, following the contribution guidelines (https://github.com/Xilinx/finn/blob/dev/CONTRIBUTING.md)?

[Andy4Worc]

@auphelia , thank you for your reply.

I do not have a fix yet (although am working on one by manually altering the python venv and other packages). Before I make a PR though, I wonder if I am using the deployment wrong by copying it over to the FPGA manually, or if it needs to be run with a "remote" deployment. Or, should it work either way? I was thinking to try the remote deployment via ssh from my running container per the PYNQ board first time setup guidelines (found: https://finn.readthedocs.io/en/latest/getting_started.html). I also found other issues referencing "remote deployment" (e.g. #154 ).
I cam currently running into a "no name" issue in the docker cli when the raw "run-docker.sh" is launched. Once I get passed that, as mentioned, I was hoping that the deployment package launched remotely from my running docker container would have all the dependencies though.... we'll see I guess.

[Andy4Worc]

Not solved. Help appreciated.

I have been trying for a few days to pip install "onnx" on the pynq board (after installing various other dependencies as needed) to solve an error when trying to run the driver on it (the same onnx error I reported in the OP here)

The "remove deployment" (from the issue from 2020) must be an old thing that has been removed. Using the docker CLI to ssh into the pynq is the same environment as using a web-browser, so no help there either.