v0.3.0

[jakmro]

We're excited to announce react-native-executorch v0.3.0. Packed with powerful new features to enhance your on-device AI experience 🎉

What's New

🗣️ Speech to Text – Transcribe your audio using Whisper and Moonshine
🎮 Speech to Text example – See how to integrate Speech to Text into your app
📖 OCR – Optical Character Recognition for extracting text from images
🚀 MPS Support – Optimized execution on iOS using Metal Performance Shaders
⚡ Hookless API – Run models outside of React components for greater flexibility
🧠 LLM Message History – Provide your LLMs with some context
⬇️ Background Model Downloads – Fetch models seamlessly in the background
📊 Benchmarks – Gain performance insights into AI models
🧪 vertical OCR – Experimental Optical Character Recognition for extracting vertical text from images

What's Changed

• docs: fix typo by @NorbertKlockiewicz in docs: fix typo #70
• docs: correct title casing and minor typos by @chmjkb in docs: correct title casing and minor typos #71
• docs: Add matching details section in the docs by @jakmro in docs: Add matching details section in the docs #72
• fix: issue with isModelGenerating when switching between multiple models by @NorbertKlockiewicz in fix: issue with isModelGenerating when switching between multiple models #73
• docs: Add landing page by @jakmro in docs: Add landing page #78
• docs: Improvements by @jakmro in docs: Improvements #79
• Add video clip to README by @darthez in Add video clip to README #85
• Update admonitions colors by @jakmro in Update admonitions colors #86
• chore: Add lefthook back by @chmjkb in chore: Add lefthook back #88
• fix typo tokenizer -> tokenizerSource by @mkopcins in fix typo tokenizer -> tokenizerSource #89
• feat: Hookless API by @jakmro in feat: Hookless API #82
• docs: Add a quickstart guide for Llama by @chmjkb in docs: Add a quickstart guide for Llama #91
• docs: Benchmarks by @jakmro in docs: Benchmarks #92
• fix: Correct link in ExecuTorch bindings documentation by @jakmro in fix: Correct link in ExecuTorch bindings documentation #93
• feat: Add MPS support by @jakmro in feat: Add MPS support #102
• Use expo-file-system and expo-asset to fetch files by @jakmro in Use expo-file-system and expo-asset to fetch files #97
• feat: Make third-party/ExecuTorchLib's forward() accept multiple inputs by @chmjkb in feat: Make third-party/ExecuTorchLib's forward() accept multiple inputs #83
• feat: Whisper by @chmjkb in feat: Whisper #101
• fix: Remove duplicated native function in ArrayUtils.kt by @chmjkb in fix: Remove duplicated native function in ArrayUtils.kt #106
• feat: ocr(ios) by @NorbertKlockiewicz in feat: ocr(ios) #84
• feat: LLM message history by @jakmro in feat: LLM message history #104
• feat: implemented vertical ocr by @NorbertKlockiewicz in feat: implemented vertical ocr #109
• design: change design of displaying ocr results by @NorbertKlockiewicz in design: change design of displaying ocr results #112
• feat: moonshine and whisper streaming by @mkopcins in feat: moonshine and whisper streaming #110
• docs: Add v0.3.0 documentation by @jakmro in docs: Add v0.3.0 documentation #114
• changed isModelReady to isReady by @mkopcins in changed isModelReady to isReady #115

New Contributors

• @darthez made their first contribution in Add video clip to README #85

Full Changelog: v0.2.0...v0.3.0

[gabimoncha]

@jakmro Thanks a lot for this update! Is the Whisper model English only or multilingual?
If it's only English, do you have a link to docs or available script to export a custom whisper model?

[jakmro]

Hi @gabimoncha, the Whisper model is English only. Exporting Whisper to ExecuTorch required some changes and right now we do not have any export scripts in our docs. 😃

[jakmro]

This is the model that we have exported 👉 https://huggingface.co/openai/whisper-tiny.en

[gabimoncha]

thank you @jakmro. I would love to use a different model. Would be great in the future to have the possibility to choose between models. tiny / base / multi-lingual. also, since now executorch supports MPS, maybe even the possibility to choose with mps or coreml backend

[jakmro]

Yup, I strongly agree 👍