Why is there no more mem_mode?

[jurevreca12]

Hello,
I am trying to generate a set of quantized fully-connected, convolutional and maximum pooling layers in the most parallel setting possible. I would like to compare the designs to the those generated by chisel4ml (https://github.com/cs-jsi/chisel4ml), which creates direct circuit implementations of the before mentioned layers. I noticed in the documentation the setting for mem_mode, that I could use to either have weights in "decoupled" memory or embedded in the fabric as "const"ants. Or at least that is how I understood it.

My question is why was this setting removed? I feel like I need to use the "const" (or "internal_embedded") to achieve what I am after, but I do not know how to choose it.
More generally, am I driving FINN correctly to produce the maximum parallelism I desire? This is the code I am using to test FINN: https://github.com/jurevreca12/c4ml_test_runs/blob/phd-experiments/test_finn.py

I am using the latest version of FINN: 0.10.1

Thank you for the answers and best regards,
Jure

[fpjentzsch]

Hi,

do you try to set mem_mode globally on the DataflowBuildConfig level?
This was removed, as mentioned in #1020:

mem_mode is not exposed to the user via builder config anymore. This parameter needs to be deleted from the builder config and added per layer to the folding config json instead.

The mem_mode options have also been renamed, see ReadTheDocs for further information on the different modes.

Here is a simple mapping of the renaming:

const -> internal_embedded
decoupled -> internal_decoupled
external -> external (unchanged)

[jurevreca12]

Is there an example somewhere of how this settings can be applied? I am looking at the build_dataflow_config.py file and I do not see these settings. I tried to grep "internal_embedded" but most hits came up in the "custom_op" subfolder.

[fpjentzsch]

Here is the DataflowBuildConfig option to supply the .json file:

finn/src/finn/builder/build_dataflow_config.py

Line 183 in 76eede6

folding_config_file: Optional[str] = None

You could look at a finn-example like this: https://github.com/Xilinx/finn-examples/tree/main/build/mobilenet-v1
folding_config will contain fixed config files optimized to fit different boards.

[fpjentzsch]

There is also a specialize_layers_config_file that you might want to use to select if certain layers use the HLS or RTL backend. The RTL MVAU for example, can only implement the compute operations using DSPs, not using LUTs.

[jurevreca12]

Thanks for the tip. Just to be sure. If I don't explicitly select it, FINN will use the HLS backend. Is this correct?

Also, regarding the json file with folding_config_file.py; I find it kind of awkward to have to specify the name of the hardware module I am trying to configure, as I don't have access to the name (e.g. "MVAU_hls_0"). Is there any way I could get the names of these modules so that I could generate the JSON? Currently my best strategy is to just run the generation process once, and then just look at the generated json. However, currently I am testing only single layer "models". I will also be testing deep neural networks, which will use several layers, so I think generating the configuration json might get kind of tricky there.

Here is my code for generating the JSON. Currently the code assumes that qonnx_model_file is a model with only a single dense layer with fixed 32 outputs and variable number of inputs.

[fpjentzsch]

By default FINN will always try to use the RTL backend where possible (there are some hyperparameter restrictions), see

finn/src/finn/transformation/fpgadataflow/specialize_layers.py

Line 40 in 76eede6

def _determine_impl_style(node, fpgapart, model):

Normally you would examine the "modules" in the ONNX intermediate representation via ModelWrapper and getCustomOp. By default, FINN will save the ONNX IR after each step of the build flow in the intermediate_models directory of your build.

If you access a CustomOp instance this way (e.g., loop over all operators of a type via get_nodes_by_op_type) you can get/set its attributes directly.

For example for the MVAU:

finn/src/finn/custom_op/fpgadataflow/matrixvectoractivation.py

Line 60 in 76eede6

def get_nodeattr_types(self):

you could then set SIMD=MW and PE=MH, which are the respective maximum values. If you were to add this as a custom build step (which is described in this notebook) to your FINN build flow, you wouldn't need to apply this config via the folding_config.json