"Discussion on GSoC Project 13: XLA Plugin for OpenVINO"

[Simonwzm]

Hi @rkazants, @mvafin

My name is Zimeng Wang, and I'm interested in project 13: "Implement XLA plugin to run OpenVINO applications on any XLA supported devices (NVIDIA GPUs, FPGA, TPUs)."

I've been reading materials about this project and would like to discuss my current understanding with you to make sure I'm heading in the right direction.

From what I've gathered so far, the project seems to involve two main parts:

1. Building a conversion method from OpenVINO IR to a format that XLA library can accept
2. Using XLA C++ API to make inference on the compiled model and adapting it to OpenVINO's plugin design pattern

For the first part, I'm currently looking into methods in pytorch-xla project, which can convert PyTorch fx graphs to StableHLO format. If OpenVINO IR has similarities with PyTorch fx graph format, maybe we could learn from their implementation? (I'm still trying to understand the details here.)

For the second part, I think we'd need to connect OpenVINO's abstractions and data structures with the XLA C++ API. After having a deep understanding of XLA's programming pattern, we can then follow other plugin's implementation pattern to design and build the xla plugin up. I'm also wondering what other goals the XLA plugin should achieve beyond basic functionality and correctness. If we also need to consider memory efficiency, concurrency, or asynchronous execution, for example, I imagine that would require a deeper understanding of both systems.

I'd really appreciate your guidance on whether I'm understanding the project correctly and any suggestions for areas I should focus on learning more about.

I currently don't the experience with XLA and StableHLO or other MLIR dialects, but I am familiar with OpenVINO code base and modern c++. If my skill set isn't best suit for the requirment of this task, please feel free to point out.

Lastly, I understand there's a prerequisite for this project requiring candidates to have a PR. I currently have an open PR #29224, which I hope satisfies this requirement.

Thanks for your time!

Best regards,
Zimeng Wang