[WIP] TensorTiler2D

[hunhoffe]

Description

This PR introduces some classes to help with data transformations/tiling and data visualization. The main classes introduced are:

• TensorTile: A class representing information for specifying a 'chunk' of data, suitable for use in describing a npu_dma_memcpy_nd operation; also includes code for visualizing the TensorTile.
• TensorTile2DIter: An iterator class that produces TensorTiles based on an offset_fn
• TensorTiler2D: A class for representing data tiling. Provides a tile_iter() method to aid in generating iterative patterns of tiling over the tensor; includes code for visualizing the the transformation.

I also include:

• A (non-exhaustive, but also not completely trivial) set of tests in test/python/tensortiler
• A minor edit of a few programming examples (dma_transpose, matrix_scalar_add, row_wise_bias_add) to show how to use the TensorTiler2D
• Two new programming examples (in programming_examples/basic/tiling_exploration) that use the device to create an access order map that can be compared to the map provided by the TensorTiler2D. These examples are meant to verify the TensorTiler2D can produce valid access patterns but also illustrate how to use the tensor tiling code.
• A change to npu_dma_memcpy_nd to support taking a TensorTile directly instead of sizes/strides/offset

Some incidental things that got included in this PR:

• First programming example with no inputs, only an output which required some minor changes to support

TODO List

• Document tiling_exploration examples
• Support repeated patterns in TensorTiler2D
• Visualize repeated patterns (heatmap of # of accesses, integer count per element of # of accesses)
• Support for tile_repeat in tile_iter()
• Support for tile_stride (skips, for height and width) in tile_iter()
• Support for TensorTile2DIter and/or tiler_iter() where some iterations may be a 'partial' transformation (e.g., in some cases where factors do not divide evenly)
• Many sizes/strides generated by TensorTiler2D may not be valid due to hardware constraints but an equivalent representation of the data transformation may be vaild; explore how to transform sizes/strides to conform to hardware limitations.
• Port at least one of the MatMul examples to use the TensorTiler2D infrastructure (needs features above)

Wish List (probably not included in this PR)

• Visualization of transformation chaining (e.g., toStream AND fromStream w/ ObjectFIFOLink)
• Helper class for transformation chaining?
• Support for overlapping data patterns (e.g., image transforms)

Notes

Credit to Andre for his really excellent tool here: https://andreroesti.com/data-layout-viz/data_layout.html
This is my attempt at making something similar that can be integrated into a development tool directly, with the additional step of generating sizes/strides for you.

[github-actions]

Coverage Report

Created: 2024-10-25 14:52

Click here for information about interpreting this report.

Filename	Function Coverage	Line Coverage	Region Coverage	Branch Coverage
IR/AIEXDialect.cpp	100.00%	86.12%	88.69%	79.79%
Transforms/AIEDmaToNpu.cpp	100.00%	91.00%	85.61%	73.26%
Totals	100.00%	88.47%	87.68%	77.74%

Generated by llvm-cov -- llvm version 14.0.0

[fifield]

How hard is it to generalize from 2d?

[hunhoffe]

@fifield Great question but I'm not sure... This needs some work before it's ready for merge due to some complexities with translating between the sizes/strides generated by the TensorTiler2D and the limitations in sizes/sizes allowed by the hardware.

My plan right now is to get a bit more confident about the problem space in 2D (which is also easier to visualize) before moving on to something more complex.