Vitis Unified Backend

[Tanawin1701d]

Description

VitisUnified backend

Motivation

• The current Vitis backend does not support a complete flow from HLS4ML to bitstream generation and driver deployment (zcu102 for my case)

Summarized features

• Automatic creation of AXI master reader/writer interfaces for HLS4ML kernels.
• Based on the v++ compiler and packaging flow.
• Configuration aligned with the Vitis Unified IDE project structure.
• Seamless integration with Xilinx hardware platforms:
  • the platform is the Xilinx package that contains the hardware structure such as axi interconnect, PS configuration, interrupt(except HLS4ML kernel)
  • Platforms encapsulate hardware structures such as AXI interconnects, PS configuration, and interrupts (excluding the HLS4ML kernel).
  • Xilinx provides platform for some boards integrated in Vitis /tools/Xilinx/Vitis/2023.2/base_platforms
  • Developers can also create custom platforms by following the official tutorial: https://github.com/Xilinx/Vitis-Tutorials/tree/2025.1/Vitis_Platform_Creation/Design_Tutorials/01-Edge-KV260
• Automatic PYNQ driver generation for streamlined deployment.

Type of change

For a new feature or function, please create an issue first to discuss it
with us before submitting a pull request.

Note: Please delete options that are not relevant.

• New feature (non-breaking change which adds functionality)

Tests

• we test with the tiny keras unet model @ test/pytest/test_backend/vitis_unified.py with 4 main aspect

bridge test

• Compare VitisUnified with Vitis
• we check predict file with 100% match

cosimulation

• we use two VitisUnified:
  • first one is used to generate bridge simulation
  • second one is used to generate start cosimulation and get the simulation result from cosimulator
• Compare with 1e-4 acceptable torelant (it comes from dat file rounding a bit)

fifo test optimization

• Procedure is similar to Co-simulation and inspect that there is fifo_depth.json exist

hardware test

• Stress test with 10,000 queries but have only 128 (input) + 128 (output) buffer size to make sure there is no deadlock from autogenerated xilinx platform axi-connection
• the tested was in function test_gen_unified in
  • the test was conducted in zcu102 with pynq framework

test reproduce

• Run pytest on test/pytest/test_backend/vitis_unified.py file
• for hardware test (test_gen_unified), you should specify XPFM_PATH(path to xpfm file) to the correct place.
• if LOG_STD == True, HLS4ML will give the HLS+linker compiling message @ console.
• if not, HLS4ML will give the messages @ <output_project_dir>/<prefix>_err.log or <output_project_dir>/<prefix>_out.log

Test Configuration:

Checklist

• I have read the guidelines for contributing.
• I have commented my code, particularly in hard-to-understand areas.
• I have made corresponding changes to the documentation.
• My changes generate no new warnings. (see the section below)
• I have installed and run pre-commit on the files I edited or added.
• I have added tests that prove my fix is effective or that my feature works.

implementation detail

• This flow of this backend to build the ready to ship file, we should do three flow things
  • file generation(HLS4ML generated file) prepare file for system Generation and pynq driver
  • synthesis Kernel (Synthesis Kernel (v++)) do c-synthesis for HLS4ML model
  • linker (Linker+vivado+Bitfile+hwh)

File structure

template structure

• the tree below is the template file allocate at hls4ml/templates/vitis_unified

├── build_lib_multigraph.sh
├── build_lib.sh
├── driver
│   └── pynq
│       └── pynq_driver.py.hls4ml (template for pynq driver)
├── hls_kernel_config.cfg         (config for HLS4ML model Synthesis)
├── myproject_bridge.cpp          (wrapper for C++ simulation using python .predict())
├── myproject_dm.cpp              (wrapper for HLS4ML model convert axi to axi stream)
├── myproject_dm.h   
├── myproject_test.cpp            (for cosimulation and fifo-optimization)
└── workspace
    ├── projectName
    │   └── vitis-comp.json       (project meta-data used for opening using vitis unified IDE)
    └── sysProj
        ├── buildAcc.sh           (script for linking the kernel with platform)
        └── buildConfig.cfg       (config file for linking progress)

output file structure

├ export                        (ready to ship file placed here!)
│   ├ pynq_driver.py
│   ├ system.bit
│   └ system.hwh
├ firmware
│   ├ <project_name>_dm.cpp     (wrapper for HLS4ML model convert axi to axi stream)
│   ├ <project_name>_dm.h       (the syntesizer not use this but required by cosim and bridge sim)
│   ├ <other files>             (other HLS src file generated from Vitis and vivado backend)
├ unifiedWorkspace              (folder for kernel synthesis and linking progress)
│   ├ linker                    (folder for platform linking project)
│   │   ├ buildAcc.sh           (build script for platform link)
│   │   ├ buildConfig.cfg       (config script for platform link)
│   │   └ <other files>         (file that v++ generated during link the platform)
│   └ <project_name>            (folder for HLS project from HLS4ML model)
│       ├ unifiedPrj            (folder for Vitis HLS internal file)
│       └ vitis-comp.json       (project meta-data used for opening using vitis unified IDE)
├ build_lib.sh                  (build script for bridge simulation)
├ hls_kernel_config_cosim.cfg   (config file for cosim and fifo depth optimization)
├ hls_kernel_config_csim.cfg    (config file for csim )
├ myproject_bridge.cpp          (wrapper for C++ simulation using python .predict())
└ myproject_test.cpp          (for cosimulation and fifo-optimization)

configuration

board='zcu102',
        part=None,
        clock_period=5,
        clock_uncertainty='12.5%',
        io_type='io_stream',
        driver='python',
        input_type='float',
        output_type='float',
        in_stream_buf_size=128,
        out_stream_buf_size=128,
        xpfmPath='/opt/Xilinx/Vitis/2023.2/base_platforms/' 'xilinx_zcu102_base_202320_1/xilinx_zcu102_base_202320_1.xpfm',
        **_,

• input_type and output_type are support only float and double. And it must be match
• {in/out}_stream_buf_size unit is in amount elements of the nnet::array
  xpfmPath

note to developer

• In case, you want to debug the generated HLS project using Vitis unified IDE, you can select the workspace folder at the program at unifiedWorkspace. The IDE will automatically detect your project
• For bridge simulation, if the configuration input_type/output_type was set to type x (double or float), you cannot predict with numpy array with different input/output type
• the depth argument @ axi_master write @ <project_name>_dm.cpp must be match of the array size generated the output array@ ````myproject_test.cpp``` for cosim and csim.
  • if the array allocation is larger than depth, the result will not correct
  • if the array allocation is lower than depth, the result is correct, but the system will throw segment falut error
  • the depth size will not impact the resource usage in hls generation
• The linked Vivado project are at <project_folder>/unifiedWorkspace/linker/_x/link/vivado/vpl/prj
• This backend will reject multigraph feature

note to tutorial

• we provide the tutorial at this repository
  https://github.com/Tanawin1701d/vitisUnifiedTutorial

generated warning

• warning in HLS4ML is only about the unet model that we use in pytest, I think it is not warning in the new backend

WARNING:absl:Skipping variable loading for optimizer 'Adam', because it has 17 variables whereas the saved optimizer has 1 variables. 
WARNING: Config parameter "algorithm" overwrites an existing attribute in layer "up_sampling2d" (Resize)

• for kernel synthesis with Vitis, I think it is general warning such as unused parameter, deprecated pragma, dataflow conflict

[nghielme]

is this file used?

[nghielme]

Here ap_axi_sdata.h is commented out, so probably is not necessary

[nghielme]

csim seems to be supported, see line 100

[nghielme]

If validation is not supported I think we should remove the parameter in the signature of the build() function

[nghielme]

I think export and package are the same thing

[nghielme]

I think Dfx term is referring to partial reconfiguration. I think we should rename this class. Also, why did you extend DefaultIP and not Overlay class?
See this one for example:

hls4ml/hls4ml/templates/vivado_accelerator/zcu102/python_drivers/axi_stream_driver.py

Line 7 in 744d7cc

class NeuralNetworkOverlay(Overlay):

[nghielme]

I think myproject is a variable, the user can change it to a different name. I think ${PROJECT} should be used instead

[Tanawin1701d]

Hi, I think myprojectBaseName and myprojectWrapName will be overwrited to user-defined project-name at hls4ml/writer/vitis_unified_writer/build_gen.py line 18 - 21.

[nghielme]

same here, as previously mentioned

[nghielme]

I think we should rename this file to myproject_axi.cpp, for consistency with VivadoAccelerator backend

[Tanawin1701d]

Since we support both AXI Master and AXI Stream, I think we may have separate files myproject_axim.cpp and myproject_axis.cpp—to clearly distinguish between the two wrapper implementations.

[nghielme]

I think we should rename this file to myproject_axi.cpp, for consistency with VivadoAccelerator backend