Whisper CPP

[Mr-7mdan]

Is your feature request related to a problem? Please describe.
As you know, finding a suitable subtitle file is always a hassle, especially with finding the right release properly synced to the current video file.

Describe the solution you'd like
I noticed that Whisper CCP was introduced and runs on tvOS, iOS, and MacOS. Which is the perfect fit for generating live captains and maybe live translation.
https://github.com/ggerganov/whisper.cpp/tree/master

Describe alternatives you've considered
Pls check AirCaptions app, as they already implemented this. The results are amazing
ggerganov/whisper.cpp#1065

It would amazing to add this to your player.

[kingslay]

我有用apple的 SFSpeechRecognizer来做实时的文字翻译。但是翻译效果很差。希望在iOS 18. apple能够实现利用ai的能力，让这个翻译效果变好。你说的Whisper CCP 我试下。看好不好接入。

[Mr-7mdan]

我有用apple的 SFSpeechRecognizer来做实时的文字翻译。但是翻译效果很差。希望在iOS 18. apple能够实现利用ai的能力，让这个翻译效果变好。你说的Whisper CCP 我试下。看好不好接入。

Whisper if fairly accurate especially with English content. It is a great addition when you cannot find a subtitle. Moreover, I am thinking that instead of getting the Audio track from the video and then converting it to wave and then to split it into smaller segments, the better implementation would be to listen to the audio driver and capture the audio in a live manner as if you are providing captions from audio coming from the microphone.

I really hope you an integrate this. Thanks in advance

[kingslay]

我初步试了下，实时增量的语音转字幕是不行的。无法输出文字。后来我换成自己把音频汇总起来，那就可以输出文字了。但是只能输出第一段，并且视频就会变成没有声音。所以目前这个功能实现卡住了

[kingslay]

在iOS 18，macOS15上 语音转字幕功能有比较大的提升，准确度高了很多。可以达到可用的状态了