Measuring cognitive and emotional states through a commercial brain- computer interface to support multimedia design (2011)
While designing multimedia learning material, we select visual elements, audio and text, based on multimedia learning principles, instructional design guidelines and other considerations such as cognitive load or working memory limitations for processing information. Besides the complexity of the content, there are other factors that can create cognitive load and some of them are linked to the learner’s individual characteristics like previous knowledge or learning style.
Multimedia systems do not acknowledge if the student is attentive or experiences frustration, if content is complex or if an image or animation is too abstract or too simple. Learner’s response to stimuli is hard to detect, unless we use special devices. Cameras, haptic sensors, microphones, motion detectors and biometric sensors are now added to the traditional keyboard and mouse. However, they also bring new challenges to the selection of effective visual or audio elements, as in the case of augmented reality, speech recognition or non-speech sound recognition.
Commercial brain-computer interfaces were created for the video game industry. However they can be used as designing tools by developers and researchers. They can be programmed to control how multimedia content is displayed based on user’s response to stimuli, or be used to test how effective a multimedia content design is while it is displayed in different devices, like laptops or smartphones where screen size and interaction features can vary.
A maze controlled by two servos and an Arduino®board was designed to test attention, frustration, cognitive load and performance. Movement input commands are sent through serial communication from a brain-computer interface to a laptop's serial port. The research project works with a commercial brain-computer interface capable to recognize facial expressions, cognitive and emotional states. Its goal is to distinguish attention and frustration levels and their relation with cognitive overload. In addition, we aim to analyze brain’s activity while learner is solving simple and complex cognitive tasks in search of a distinctive brain activity we can relate to cognitive load.
Imelda Latapie is a freelance graphic designer and illustrator from Mexico City. She won a bronze award in the Eighth Annual3 Dimensional Art Directors & Illustrators Show (New,York), a mention in the 5th “Catálogo para ilustradores Infantiles y Juveniles” (Mexico), and her illustrations have been published in magazines, posters and children’s textbooks. She graduated with honorsas a Graphic Designer from Universidad Simón Bolívar in Mexico City;and received the “Medalla al Mérito Universitario” from Universidad Autonoma Metropolitana, campus Azcapotzalco in Mexico City, for her studies in Hipermedia and also for her Master’s degree studies, with the dissertation:“Método para el diseño de aplicaciones educativas: Una propuesta centrada en el aprendizaje e instrucción multimedia” (A method for designing educative applications: A proposal centered in multimedia learning and multimedia instructional design). At the moment she is working in her PhD dissertation about brain computer interfaces applied to cognitive load detection. She received a grant from the Instituto de Ciencia y Tecnología del Distrito Federal, (Mexico) to support her doctoral investigation from August 2010 to January 2011. She is studying to get her Bachelor of Computer Science at the University of the People; and she is interested in integrating new technologies to multimedia learning and multimedia instructional design.