Backchannel based on VAD

Backchannel generation module based on VAD.

We implemented this module inspired by the previous backchannel module based on the following paper.

Elisabetta Bevacqua, Etienne de Sevin, Sylwia Julia Hyniewska, Catherine Pelachaud, A listener model: introducing personality traits. J. Multimodal User Interfaces 6(1-2): 27-38 (2012)

This module considers both of audio signal from the user and the agent.

Currently, this supports the following backchanels:

• Reactive backchannel: immediate backchannel when the agent recognize the start of the user's utterance
• Response backchannel: this is trrigered when the agent detect the end of user's utterance or inter-pausal unit (IPU)

Installation

Please follow common installation

Usage

1. In Modular.jar, add Microphone module from [Add -> Input -> Microphone]
2. In Modular.jar, add TurnManagement module from [Add -> Input -> Dialogue -> TurnManagement]
3. Create the following connections in Modular.jar: Feedback -> TurnManagement, TurnManagement -> BehaviorPlanner
4. When you are speaking something and Greta is not talking, the module randomly picks one XML file from bin/Examples/DemoEN/backchannel upon several rules written in /bin/Common/Data/TurnManagement/turnManager.py
5. Some rules were based on the previous backchannels module

Note

• For more details, please check the following sources
  • Microphone java project at auxiliary/Microphone
  • Python source code at bin/Common/Data/microphone

Screenshot