Plans and hurdles - Aug 2024 update

[tetele]

Hello, everyone! Sorry for what may have appeared as lack of attention in the past couple of months.

Apart from being on holiday for a while, I've been very caught up at work with a lot of things which kept me from dedicating enough time to my hobby projects (such as this one) and I've also tried spending a fair amount of time with my kid. The most amount of hobby project time I've spent with a new piece of hardware that I've received as an engineering sample and which I've tried to turn into a voice assistant. The most palpable result is this, which took much, much longer than anticipated.

Whatever time I've had to dedicate to the Onju Voice ESPHome config, I've dedicated without much success in these 3 directions:

Custom board definition

I've tried numerous times already to define a custom board such as this one in order to use custom pin names, pre-made buses (e.g. SPI), amount of PSRAM, amount of flash etc.

I haven't found too much documentation on the matter and had to quit about 3 times so far.

Using external flash

The Onju Voice board has a W25Q128JVSIQ 16MB QSPI NOR flash module on board, so I started investigating if this could be used to:

• store larger firmwares
• store local audio files to be used as confirmation beeps

Again, with lacking documentation, I was not able to progress much on this subject.

Improving audio quality and stability

Here I've been mostly testing stuff from gnumpi's amazing https://github.com/gnumpi/esphome_audio library, for use with the microWakeWord component. Some of the stuff I've tested was hindered by core ESPHome behavior and, in my experience, the lack of developer documentation there means that PRs won't be in line with intended (but undocumented) behavior. Plus, you don't know what you don't know :)

All these things are sort of intertwined

And not having much time, it always looked like even debugging meant pulling on what seemed to be a little piece of string, but which had 10 balls of yarn tied to it behind the scenes. And I simply lacked the concentration to pull on them.

I am sorry for not having been able to add more stuff for you all. This post is also a cry for help, though. If anyone has experience with the stuff I mentioned above, such as:

• defining custom boards for ESPHome
• using external flash memory in ESPHome
• ESP32 partitioning in ESPHome
• audio calibration, particularly using esphome_audio

then you are more than welcome to help out.

Here are a few things that I want to work on regarding the Onju Voice project:

• proper automatic testing and building
• firmware web installer
• easily storable and configurable audio confirmation of wake up/VAD end/command execution/timers
• more configurable UI (e.g. LED colors)

Hopefully, this autumn I'll get a lot more time to get these things going, but I will take all the help I can get if anyone's interested in helping out. Thank you all for understanding!

Cheers!
Tudor

[tbrasser]

I believe stuff is happening here:
https://github.com/esphome/voice-kit/tree/dev/esphome%2Fcomponents

And also it seems time to integrate the onju voice properly here:

https://github.com/esphome/firmware/tree/main

Also, my onju voice seems to work pretty good with some frankenstein yaml combo, also seen someone succesfully do 48kHz!

kutgw!

[jhbruhn]

Reg. External Flash: #69 (comment)

The Flash Module on the Onju Voice is an external flash, yes. But it is the flash that is used for the firmware already. The ESP32-S3R8 that is used on the Onju Voice does not have any internal flash. There is only one flash location, which is the external flash chip. The flash size configuration of the current Onju voice firmware just does not specify the correct size of the flash. Because of this, ESPHome defaults to 4MB. If it is specified correctly (see my comment referenced above), everything still works (incl. OTA Updates which indicates that the flash actually is that large), there is plenty of space for internal audio files.

While it would be nice to have a custom board definition, flash size can be defined without a custom board definition.

[jhbruhn]

Reg. Sound Quality:

I had success with a 48kHz audio setup, but I have not tested it extensively, as I am not using my Onju Voice devices for Music Playback. Maybe someone else can test it? Adding a resampler and using 48kHz instead of 44.1kHz seems to be the trick.

adf_pipeline:
  - platform: i2s_audio
    type: audio_out
    id: adf_i2s_out
    i2s_audio_id: i2s_shared
    i2s_dout_pin: GPIO12
    sample_rate: 48000
    adf_alc: true
    alc_max: .5
    bits_per_sample: 32bit
    fixed_settings: true
    channel: left

  - platform: i2s_audio
    type: audio_in
    id: adf_i2s_in
    i2s_audio_id: i2s_shared
    i2s_din_pin: GPIO17
    channel: left
    pdm: false
    sample_rate: 48000
    bits_per_sample: 32bit
    fixed_settings: true

microphone:
  - platform: adf_pipeline
    id: onju_microphone
    keep_pipeline_alive: true
    gain_log2: 3
    pipeline:
      - adf_i2s_in
      - resampler
      - self

media_player:
  - platform: adf_pipeline
    id: onju_out
    name: None
    internal: false
    keep_pipeline_alive: true
    pipeline:
      - self
      - resampler
      - adf_i2s_out