Create chat application for LLM agents

[PeterBowman]

This is the basis of an oncoming paper (to be published in RBVM 2025). Its main goal is designing a chat application that involves audio synthesis and recognition and an LLM-based completion device. It is meant to leverage new YARP interfaces (#31). Components:

• Speech synthesis (TTS): yarp::dev::ISpeechSynthesizer implemented with Piper
• Speech recognition (ASR): yarp::dev::ISpeechTranscription implemented with Vosk
  • includes wake word detection implemented with openWakeWord
• Large language model (LLM): yarp::dev::ILLM implemented with llama.cpp

A very similar idea has been presented in robotology/yarp-devices-llm involving the OpenAI API through Azure servers (perhaps TTS/ASR is also online). However, this app is meant to be run fully offline. As such, its main motivation is introducing the recent DeepSeek models in the LLM component.

[PeterBowman]

A working implementation is already available at the time of writing. Let's cite some sources:

• The LLM device re-uses this example from the llama.cpp repo.
• The wake word device re-uses this example from the openWakeWord repo.

Also, some useful webs found in the making of this app:

• https://github.com/dscripka/openWakeWord#training-new-models
  • Google Colab notebook
• https://mycroft-ai.gitbook.io/docs/mycroft-technologies/precise#training-custom-wake-words
• https://medium.com/@mckev/how-to-run-deepseek-r1-llm-in-c-a0b4f9f8dd2a
• https://medium.com/@isaakmwangi2018/a-simple-guide-to-deepseek-r1-architecture-training-local-deployment-and-hardware-requirements-300c87991126
• https://www.datacamp.com/blog/deepseek-r1-vs-v3
• https://www.bentoml.com/blog/the-complete-guide-to-deepseek-models-from-v3-to-r1-and-beyond
• https://github.com/andrewkchan/deepseek.cpp
• https://github.com/deepseek-ai/DeepSeek-R1
• https://github.com/deepseek-ai/DeepSeek-V3
• https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
• https://huggingface.co/deepseek-ai/DeepSeek-V3
• https://ollama.com/library/deepseek-r1:1.5b
• https://unsloth.ai/blog/deepseek-r1
• https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-1.5B-GGUF
• Issue converting DeepSeek-R1 to GGUF format. ggml-org/llama.cpp#11989

[PeterBowman]

Now some backups: hey-teo.zip. This includes:

• The Google Colab notebook linked above.
• The synthesized hey-teo.wav sentence using en_US-libritts_r-medium.
• A first hey-teo ONNX/TFLITE pair of models (currently only ONNX is supported) trained in <1 hour.
• A second and hopefully more accurate hey_tee_o ONNX/TFLITE pair of models trained in ~5 hours.

All relevant models are listed as parameters in this application file.

[PeterBowman]

Next steps:

• CI builds
• tests and examples
• docs
• migrate wakeWord from Python to C++ using rhasspy/openWakeWord-cpp

[PeterBowman]

Note there also exists yarp::dev::IChatBot. I don't see any use for it in this app, though, that would pose an advantage over yarp::dev::ILLM.