Release Notes Template #9880

byjlw · 2025-04-03T19:37:45Z

byjlw
Apr 3, 2025
Collaborator

Sample format for creating release notes

ExecuTorch 0.X Release Notes

We're excited to announce the release of ExecuTorch 0.X! This release builds upon the foundation established in the [0.P release](https://github.com/pytorch/executorch/releases/tag/v0.5.0) (January 30, 2025) with significant improvements across various components of the framework. ExecuTorch continues to be a powerful solution for on-device AI, powering experiences across a wide range of platforms.

Note: The PR links in this document point to specific pull requests that contributed to each feature or improvement. You can click on them to see the detailed changes and discussions.

Highlights

[Improved Usability and Stability](#developer-experience): Significantly improved usability and stability fixes solving a lot of issues users were having when trying to install and use ExecuTorch
[Improved LLM Support](#llm-support): Significantly improved performance and capabilities for Large Language Models on edge devices, with optimizations for Llama model families
[New Hardware Accelerator Backends](#backend-delegates): Expanded hardware acceleration support with improved delegation interfaces and better hardware utilization
[Windows Support](#platform-support): Added Windows platform support for development and deployment workflows
[Developer Toolchain Improvements](#developer-experience): Improved integration between PyTorch model authoring, conversion, debugging and deployment workflows
[Performance Optimizations](#core-runtime-improvements): Substantial memory footprint reductions and faster inference across all supported devices
[Expanded Wearable & Microcontroller Support](#platform-support): Improved capabilities for highly constrained environments
[Serialization Improvements](#exir): More efficient model representation and loading for faster startup times

Core Runtime

Reduced memory usage during model initialization [#9120](Add buck to build binary for Android #9120)
Optimized tensor memory allocation for faster inference [#9023](Partitioner to not delegate mixed dtype operators #9023)
Added support for dynamic shapes in more operators [#8967](support #8967)
Improved error handling and reporting [#9145](Remove the ability to bring your own flatc #9145)

EXIR

Added new optimization passes for better model performance [#9002](Model File Size: Backend Weight Sharing via. Blob Storage Service #9002)
Enhanced graph partitioning for delegated execution [#8945](Use NDM in backend init context #8945)
Improved serialization format with reduced file sizes [#9087](Update shim submodule #9087)
Added type propagation for quantized operators [#8921](introduce file_data_sink #8921)

Backend Delegate

ARM

Added support for latest ARM Ethos-U NPU architecture [#9112](Use external hf_tokenizer in llama runner #9112)
Optimized convolution operations for ARM Mali GPUs [#9056](Split delinearize_index out of broadcast_util #9056)
Enhanced delegate partition selection heuristics [#8932](Parallelize portable ops if threadpool is available, with fallback to parallel_for-as-for-loop #8932)

NXP

Added i.MX 95 platform support [#9101](Fix static attention mask update #9101)
Optimized NPU utilization for transformer models [#8978](Update jni runner #8978)

XNNPACK

Optimized GEMM operations for latest ARM architectures [#9123](Add NamedData to flat_tensor schema #9123)
Added support for group convolutions with optimized kernels [#8989](fix mman header issues #8989)
Improved quantized operator performance [#9056](Split delinearize_index out of broadcast_util #9056)

MediaTek

Added support for latest APU architecture [#9045](Cleanup build from source page #9045)
Optimized vision model performance [#8967](support #8967)

OpenVino

Enhanced integration with latest OpenVino release [#9089](Add MPS Backend #9089)
Added support for more vision operators [#8978](Update jni runner #8978)

Qualcomm

Added support for Snapdragon 8 Gen 3 optimizations [#9134](Arm backend: Don't handle const placeholders in match_arg_ranks_pass #9134)
Improved HTP utilization for transformer models [#9001](Remove explicit descriptor #9001)
Enhanced delegate compatibility with more operators [#8934]([ExecuTorch][XNNPACK] Rename linear weight partitioning flag for clarity #8934)

Ops & Kernels

Added new operators for advanced NLP models [#9112](Use external hf_tokenizer in llama runner #9112)
Optimized matrix multiplication for various hardware targets [#9023](Partitioner to not delegate mixed dtype operators #9023)
Enhanced quantized operations for improved accuracy [#8978](Update jni runner #8978)
Added support for more activation functions [#9067](Enable aten.upsample_nearest2d.vec #9067)

Model Support

Added support for Llama 3.1 models [#9145](Remove the ability to bring your own flatc #9145)
Improved SDXL image generation performance [#9087](Update shim submodule #9087)
Enhanced support for Whisper speech recognition models [#8989](fix mman header issues #8989)
Added MoE (Mixture of Experts) support for LLMs [#9067](Enable aten.upsample_nearest2d.vec #9067)

Developer Experience

Improved error messages and debugging information [#9120](Add buck to build binary for Android #9120)
Enhanced documentation with more examples and tutorials [#8945](Use NDM in backend init context #8945)
Added new profiling tools for optimizing model performance [#9089](Add MPS Backend #9089)
Improved pybind interfaces for easier integration [#8921](introduce file_data_sink #8921)

Platform Support

Added support for new Android devices and architectures [#9101](Fix static attention mask update #9101)
Enhanced iOS compatibility with latest devices [#9045](Cleanup build from source page #9045)
Improved Linux embedded support for various platforms [#8998](Arm backend: Move ReplaceScalarTensorWithFullPass to transforms #8998)
Added new microcontroller targets [#8956]([XNNPACK Backend] Modified MobileBERT model outputs original shape on Android despite successful desktop inference #8956)

API Changes

Added new C++ APIs for advanced runtime configuration [#9123](Add NamedData to flat_tensor schema #9123)
Enhanced Python bindings for easier model export [#8989](fix mman header issues #8989)
Deprecated legacy initialization methods [#9056](Split delinearize_index out of broadcast_util #9056)
Added type-safe tensor access methods [#8967](support #8967)

Bug Fixes

Fixed memory leak in delegate handover process [#9134](Arm backend: Don't handle const placeholders in match_arg_ranks_pass #9134)
Resolved race condition in multi-threaded execution [#9001](Remove explicit descriptor #9001)
Fixed incorrect output shapes in dynamic execution [#8934]([ExecuTorch][XNNPACK] Rename linear weight partitioning flag for clarity #8934)
Addressed compatibility issues with latest PyTorch versions [#9112](Use external hf_tokenizer in llama runner #9112)

Getting Started

With ExecuTorch 0.6, you can now get started using pip:

pip install executorch

See the [documentation](https://pytorch.org/executorch/0.6/getting-started.html/) for detailed installation instructions and tutorials.

Community

We welcome feedback, suggestions, and contributions from the community! Join us on [Discord](https://discord.gg/Dh43CKSAdc) or check out our [Discussion Board](https://github.com/pytorch/executorch/discussions).

For more information on contributing, please review our [guidelines](https://github.com/pytorch/executorch/blob/main/CONTRIBUTING.md).

Contributors

We'd like to thank all the contributors who helped make this release possible:

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release Notes Template #9880

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 0 comments

Select a reply

Release Notes Template #9880

byjlw Apr 3, 2025 Collaborator

ExecuTorch 0.X Release Notes

Highlights

Core Runtime

EXIR

Backend Delegate

ARM

NXP

Apple

Vulkan

Cadence

XNNPACK

MediaTek

OpenVino

Qualcomm

Ops & Kernels

Model Support

Developer Experience

Platform Support

API Changes

Bug Fixes

Getting Started

Community

Contributors

Replies: 0 comments

byjlw
Apr 3, 2025
Collaborator