Add hardware IP

[btwbtw01]

Can I add hardware IP into FINN by using Vivado HLS? My understanding of Vivado HLS is writing C code and synthesis, then combine Vivado PS block to generate bistream. The tutorial seems like using python to generate custom IP, and there is limit for generating custom IP. Am I right? If I generate bistream, how could I deploy on FPGA by PYNQ

[maltanar]

Hi @btwbtw01 , this is indeed possible but is considered an advanced use-case and we don't have a tutorial for this just yet. There was a previous Gitter discussion on this here but I'm copying (with some edits) the key steps here, using the recently-added upsampling layer support as a running example:

• add new HLS class in finn-hlslib for new operator (example)+ HLS unit test (example)
• add new HLSCustomOp subclass in FINN compiler matching this HLS function (example)
• (only needed for some layers) additional transformations in the FINN compiler for data layout conversion (example)
• add new transformation convert_to_hls to FINN compiler for translating the relevant ONNX layer into the HLSCustomOp instance (example)
• add unit / integration test into FINN compiler to make sure everything works (example)

[btwbtw01]

Thanks for your response. The hardware IP is like compression and decompression. I just need to compress the activation to store in DRAM and decompress the activation to run convolution.

[btwbtw01]

@maltanar Can I just write HLS functions and wrap it up, so I don't have to add CustomOp subclass in FINN compiler?
All I nned to do is add new HLS class in finn-hlslib for new operator
Can I do this without adding additional transformations?

[heborras]

Hi,
the steps from Yaman are a bit more extensive, but in general they are the similar to the ones in the notebook.

[btwbtw01]

Hi, @HenniOVP
What is the function of custom op? Is setting the folding factors and specify properties of these nodes? How the example calls these python codes to create custom op?

I guess I have to add compression nodes between the hls layer after Creating a Dataflow Partition and during Adjusting the Parallelism

[btwbtw01]

According to notebooks/advanced/2_custom_op.ipynb, the most important function to add custom ops is get_nodeattr_types and execute_node. Am I right? In the model transform, we will first convert the graph to HLS, than execute custom ops. In Yaman example, it will execute transformation convert_to_hls first, then execute HLSCustomOp subclass. Is that correct?

[btwbtw01]

Hi, @HenniOVP
May I ask how the FINN compiler generate FIFO between two computing arrays? Can I have two FIFO between two computing arrays like this?

Thanks!!

[JPPalacios]

Hi @btwbtw01,

Were you able to create custom HLS Ops? If so, could you provide some details on how you went about it? I am looking for something similar but before/after.

Thanks

[auphelia]

Adding this link here, since the custom op integration has slightly changed in the latest release: #1026 (comment)

And the additional discussion here: #1160 (thanks @JPPalacios and @iksnagreb)