Skip to main content

iTutor and iDemo: 3D Computer Vision/Graphics with Intelligent Avatar Interaction

Awarded Grant: $59,000

Principal Investigator:

Dr. Lijun Yin, Binghamton University

This project aims to develop an advanced teaching and learning technology by utilizing the existing computer vision technology and further develop our unique human behavior and facial expression analysis approaches to improve the experience of class teaching and learning. Specifically, two software systems will be developed: (1) An Idea Illustration and Demonstration (iDEMO) software tool will be created that provides instant visualization of instructor’s idea, theory, illustration, demonstration, and experiments that can be used by teachers in the classroom. Such a unique story-telling software can translate voice to text, then text to graphic scene, and allows a story presented by teachers and/or students to be visualized in an animated movie; (2) An Intelligent Virtual Tutor System (iTUTOR) using a synthesized individualized graphical avatar as a “virtual instructor” will be created for interacting with students. The virtual tutor is smart enough to understand students’ reaction through recognition of their expressions, voices, eye gazes, and hand gestures, and adjust the presentation materials accordingly at various levels. PI Dr. Yin has been working on developing virtual avatars and computer-based human behavior understanding and simulation for human computer/robot interaction for over 15 years. Such state-of-the-art software opens a new way for teaching and learning, potentially enhancing students’ creativity and critical thinking ability. We will apply our systems to five courses of computer science through the Fall semester 2013 and Spring 2014. The effectiveness of the new instruction system will be evaluated through the course instructors’ reports, students’ performance reports, and an external evaluator’s survey and comparison. The findings will be disseminated through technical publications, conferences, and project websites. The resulting software and data will be released to the public, including the research and educational community.

Co-PI’s and Key Partners:

  • Greta Mayers, Binghamton University

Reports and Resources:

Creative Commons License:

Attribution Non-Commercial Use