Turn management based on VAP

CAUTION: UNDER CONSTRUCTION!!

Turn management module based on voice activity projection (VAP) model.

This module considers both of signals from the user and the agent.

Currently, there are two mode

• audio VAP
• audio and face embed VAP

Installation

Please follow common installation

Usage

1. In Modular.jar, add Microphone module from [Add -> Input -> Microphone]
2. In Modular.jar, add TurnManagement module from [Add -> Input -> Dialogue -> TurnManagement]
3. Create the following connections in Modular.jar: Feedback -> TurnManagement, TurnManagement -> BehaviorPlanner
4. When you are speaking something and Greta is not talking, the module randomly picks one XML file from bin/Examples/DemoEN/backchannel upon several rules written in /bin/Common/Data/TurnManagement/turnManager.py
5. Some rules were based on the previous backchannels module

Note

• For more details, please check the following sources
  • Microphone java project at auxiliary/Microphone
  • Python source code at bin/Common/Data/microphone

Screenshot