Reuse vanilla YARP 3.9 interfaces

[PeterBowman]

PR robotology/yarp#2978 is scheduled for YARP 3.9 and introduces generic interfaces for speech transcription and synthesis. Once accepted, this could replace our IDL interfaces.

[PeterBowman]

The NWC device has been added in robotology/yarp#2986 and robotology/yarp#2988.

[PeterBowman]

This is live now. Docs:

• yarp::dev::ISpeechSynthesizer
• yarp::dev::ISpeechTranscription

[PeterBowman]

I have been playing with the YARP's audio recording/playing utilities described here. Free audio samples can be downloaded at this page, read from file using audioFromFileDevice and played with portaudioPlayer:

yarpdev --device deviceBundler --wrapper_device AudioRecorderWrapper --attached_device audioFromFileDevice \
        --start 1 --file_name shes-a-crazy-psycho-2.wav --reset_on_stop 1 --name /audiofile

yarpdev --device deviceBundler --wrapper_device AudioPlayerWrapper --attached_device portaudioPlayer \
        --start 1 --playback_network_buffer_size 0.1 --AUDIO_BASE::samples 320000 --name /player

yarp connect /audiofile/audio:o /player/audio:i

The --AUDIO_BASE::samples parameter to portaudioPlayer is quite relevant, picking a low value would result in stutter.

[PeterBowman]

The converse option, i.e. capturing audio via microphone and storing it as a .wav file, is also possible:

yarpdev --device deviceBundler --wrapper_device AudioRecorderWrapper --attached_device portaudioRecorder \
        --start 1 --min_samples_over_network 3200 --max_samples_over_network 3200 \
        --AUDIO_BASE::rate 16000 --AUDIO_BASE::samples 6400 --AUDIO_BASE::channels 1 --name /recorder

yarpdev --device deviceBundler --wrapper_device AudioPlayerWrapper --attached_device audioToFileDevice \
        --start 1 --file_name test.wav --name /audiofile

yarp connect /recorder/audio:o /audiofile/audio:i

# echo start | yarp rpc /recorder/rpc

The last RPC command initiates the recording if --start has not been provided to the first device.

[PeterBowman]

A working implementation is mostly ready: now we have two synthesizers (eSpeak and Piper) and one transcriber (Vosk) using the new YARP interfaces.

Regarding Piper, I had to introduce some CMake magic to extract the API into a shared library. The main repo has been forked, and the library is available at roboticslab-uc3m/piper (also, we depend on roboticslab-uc3m/piper-phonemize). Compilation is straightforward thanks to the adoption of CMake's ExternalProject, which pulls all necessary dependencies during the configure phase.

Regarding Vosk, I followed the instructions detailed here, which redirect to a sequence of commands listed in a Dockerfile. Some of those sed command invocations are actually obsolete now, and Kaldi's ./configure was unable to find the openfst dependency. This one is downloaded on runtime and extracted into openfst-1.8.0 at time of writing, therefore I just had to provide the --fst-root parameter pointing at said directory. Also, CUDA is not compatible with my setup (CUDA 12: gcc <= 12, but I have gcc 13), so I passed --use-cuda=no. GPU-enabled systems would need to build additional targets in the vosk-api step (ref). Besides, I followed the OPENBLAS_CLAPACK path instead of MKL.

I'd like to check whether it is easy to build Vosk/Kaldi from sources using CMake instead of autotools.

[PeterBowman]

Mostly implemented at 5d78e43. TODO:

• missing method implementations in PiperSynthesizer
• CI builds (pulling Piper and Vosk dependencies)
• tests
• perhaps more examples (C++/Python)
• docs