This project is implemented as part of Human Robot Interaction coursework by Group 5. It is using 'Marty The Robot' for Language learning purpose and studying regarding the interaction between Human and Robot.
The project "Marty The Robot (EaRL) as a Language Learning Buddy - A Human-Robot Interaction Project" depicts a practical interactive system for beginners in the Spanish language (in this project, which can be generalised to any other languages) through human-robot interaction. Marty-a humanoid robot, which in this project is integrated with speech recognition, translation, computer vision, and feedback mechanisms is employed to create an encouraging learning environment for all learners. Marty works on the principle of two instructive modes: voice input and image input. In voice mode, Marty listens for the user's English phrases, runs those through a translation API to get the Spanish versions, and then checks the pronunciation against the user's spoken input. The user receives celebratory feedback and visual cues for correct attempts and is given corrective feedback and encouragement on incorrect attempts. In image mode, Marty identifies the objects using computer vision and machine learning libraries, thus providing the object names in Spanish and checking the user's pronunciation of those terms.
This project allows the reader to understand how socially interactive robots may influence language learning positively, caused by multimodal feedback, among other things, and supportive interaction, in which Marty is a responsive and empathetic learning partner. This can be measured by user engagement metrics, learning accuracy, and qualitative feedback in the context of Marty's impact on user motivation, confidence, and retention regarding Human-Robot Interaction. It also gives insight into the role that robotic feedback can play in enhancing language acquisition and the effectiveness of multimodal interactions involved in learning with an anthropomorphic robotic companion. Expected outcomes are to contribute to the body of knowledge in human-robot interaction and educational robotics with respect to design issues in developing socially interactive robots that foster desirable educational outcomes.
- Install the packages - MartyPy, SpeechRecognition, OpenCV, numpy, tensorflow.keras, googletrans, subprocess
- Keep the python files in 'Code' folder together in a place
- Replace the IP addresses in all three python files as of your Marty Robot
- Run the 'Master.py' file