Multi-Layer Offload #1489
Open
Multi-Layer Offload #1489
+12,931
−400
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add loop_control and loop_control_wrapper
…ased on a lambda filter, updated the loop test to reflect the changes
Extended adjacency_list utility function to accept a lambda for filtering decisions.
Update loop tb
…4 mismatch issue)
… an MLO sim is being performed.
Refactor Loop Op flow
… when extracting folding config json
preusser
requested changes
Dec 10, 2025
Collaborator
preusser
left a comment
preusser
left a comment
There was a problem hiding this comment.
This is looking good functionally.
Please, fix licenses and clean up module interfaces though.
- Modified module parameters. - License header updates. - Refactoring RTL directories, getting rid of duplicates. - RTL internal changes.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Multilayer Offload (MLO) allows significantly larger neural networks (with repetitive blocks) to be deployed on FPGAs using FINN.
Instead of mapping every layer of a model onto hardware, MLO implements just one representative block of the repetitive structure in the network (for example, a single transformer encoder layer) and repeatedly reuses it. The weights for each iteration are streamed in from external memory such as DRAM or HBM, enabling models far larger than what could fit fully on-chip.
Importantly, the repeated block does not need to cover the entire network. Many architectures contain head and tail portions around a large repeated middle section. MLO can be applied selectively: fixed, non‑repetitive layers can be implemented as standalone hardware layers, while the repetitive core (e.g., the main stack of transformer layers) is handled by the MLO mechanism.
FINN continues to stream both the activations and the weight tensors as needed. The MLO control logic, which resides in the FPGA fabric next to the accelerator, operates fully automatically: it monitors the flowing data, manages iteration indices, selects the correct weight set, and loops outputs back into the computation when required. This removes the need for any manual scheduling or user‑driven orchestration of layer execution. The hardware autonomously steps through each repeated layer based purely on the data and iteration control embedded in the design.
The result is a practical trade‑off: slightly reduced throughput in exchange for the ability to deploy much larger models.