-
Notifications
You must be signed in to change notification settings - Fork 17
DeepASR Module
Lucie Galland edited this page Jul 19, 2024
·
3 revisions
An ASR can be used to communicate orally with greta, the DeepASR module implements an interface between Greta and a python AST
Currently only DeepGram is implemented
To create a new DeepASR module,
- Create a class in the auxiliary project DeepASR that extends DeepASRFrame
- Create a new folder in Common/Data/DeepASR/{Yourmodelname} Copiyng the files in Common/Data/DeepASR/DeepGram
- Modify the copy of DeepGram.py with the code to access your LLM model, the final answer should be printed by your python module after "Speech Final:"
- Modify requirements.txt with your requirements
- Modify check_env.py and init_env.bat with a new conda environment name
- In the java class, initiate the strings DeepASR_python_env_checker_path, DeepASR_python_env_installer_path ,python_asr_path with the appropriate relative paths
- Add your new Module in Modular.xml
- Generating New Facial expressions
- Generating New Gestures
- Generating new Hand configurations
- Creating an Instance for Interaction
- Create a new virtual character
- Creating a Greta Module in Java
- Integration examples
- Development guideline
- Naming policy
- Github repository
- Technical showcase
- Python integration
- Known issues
- Technical Specifications
- Pitfalls for Greta development
-
FML/BML
-
Tools
-
AUs from external sources
-
Large language model (LLM)
-
Automatic speech recognition (ASR)
-
Extensions
Nothing to show :)
- Incrementality
- Motion Capture to BAP values
- Interruptions
- Back Channels
- Appreciation Generator
- SSI Greta Integration
- TopicPlanner
- Shore
- Disco
- Watson
- Object node controller
- OSC communication for Unity
- 3D pose estimation from RGB camera through VAE (PFE-OpenPose-to-VAE-to-BVH)
- HOW TO CREATE BEHAVIOR SET
- Projects
- Signals