On April 27, 2022, the Next Level Lab hosted a presentation by Dr. Pattie Maes, Professor of Media Technology at MIT’s Program in Media Arts and Sciences and the director of the MIT Media Lab’s Fluid Interfaces research group.
Devices put the world’s information and knowledge at our finger tips, but they do not help us with some of the skills that are needed for successful learning and performance, skills such as motivation, attention, creativity, effective communication, critical thinking and more. Inspired by insights from the cognitive sciences, Dr. Maes’ research aims to design systems that support and teach these skills in learners.
Dr. Pattie Maes’ presentation, Enhancing Human Cognition, explored whether knowledge and skills are sufficient for optimal performance and how to maximize human potential. Dr. Maes’ research focuses on how to leverage digital applications and devices to support cognitive functioning. The presentation offered some cutting-edge research and practices for educators to better leverage technology and devices and help learners develop various skills from spatial memory and motivation to creativity and confidence.
1) Project Nevermind: Using Augmented Reality (AR) to facilitate spatial memory
The Nevermind system uses Augmented Reality (AR) technology and devices to encode memory and effectively help people link factual learning to spatial memory to improve recall. By using subsequent winners of the Super Bowl starting in 1967, Dr. Maes’ team conducted an experiment asking people to learn who won the Super Bowl, either using Nevermind with a VR glass device or as a paper-based task. Specifically, participants in the Nevermind condition wore AR glasses and saw the superimposed winners of the Super bowl in sequence from a subway station to their offices. For instance, coming out of the subway station, participants saw a virtual person with tons of boxes reminding them that the Green Bay Packers won the game in 1967; continuing along the route, participants saw the winner of the next year such as the Kansas City Chiefs sitting on the steps of another building because they won 1970’s game, and so on.
In this way, researchers try to find how people link facts about who won the Super Bowl game to the spatial memory of the route from the subway station to their offices. The experiment found that both groups did equally well recalling the winners right after the experiment, but the group using Nevermind had better performance after 24 hours and 7 days afterward, because the participants encoded the knowledge in a particular way relating to their spatial memory.
This is one example that researchers are trying to prototype and evaluate through human subject experiments. Researchers are further exploring how to take advantage of how our brains work, what human behaviors are like, and how these findings on human cognition can help us in building applications and systems that support us in developing soft skills for learning.
2) Project AttentivU: Improving motivation for learning
Research revealed that some teachers are more motivating than others, not just because of what teachers say but also because how students relate to their teacher matters. To improve learners’ motivation, the research team explored how to learn from virtual avatars. The use of AI-generated characters could help enhance motivation in online learning environments. For example, using deep fake technology, the experimenters used an AI-generated version of Elon Musk and a generic person with similar age, gender, and other characteristics as Elon Musk, to provide the same content information as instructors. The results showed that depending on how much people admired a celebrity, people would be more willing to have advanced material from this person.
Moreover, the research team used AttentivU, a biofeedback system for real-time monitoring and improvement of attention, to improve motivation for learning. This glass has a built-in EEG, or brainwave sensing electrodes behind the ears and on the bridge of the nose that pick up movements of the eyes and measure the engagement of a learner. Researchers can determine if a learner is more attentive visually or more attentive to an audio source.
The experiment evaluated the performance of 3 groups of people by giving biofeedback, random feedback, and no feedback. Scientists found that people who are reminded every time their attention has drifted perform much better on the lecture tests presented by the teacher. Additionally, AttentivU is used to measure the fatigue level of drivers to take a break.
3) Exploiting the Protheus effect to increase creativity
To explore how to increase people’s creativity, researcher Joanne Leong used Snapchat filter technology and conducted a Zoom study with 21 participants in three conditions: without any filters, with a filter to make the person look like a child, and with a filter to make the person look like an inventor or crazy scientist. It turned out that the filters altered people’s behavior and even their cognitive performance: the study showed increased creativity in the child and inventor conditions as compared to the no-filter condition. People in the child and inventor conditions performed better in creativity tasks like Verb Generation Task (VGT) or Alternate Uses Task (AUT) than in normal conditions. For example, participants were asked to think of as many unique and unusual uses for objects, like a pizza box being used as a fire starter when cutting it into small pieces.
This project is inspired by the concept of the Proteus Effect by Nick Yee and Jeremy Bailenson in virtual reality. Proteus Effect refers to the concept that people’s behavior conforms to our digital self-representations and that self-representations alter how we make decisions in the environment. Another study shows that if people are virtually embodied with a different gender or race, they will reduce implicit bias toward that gender or race, and changing bodies ultimately leads to changing minds with personal growth and behavior change. In sum, to improve our abilities, we can repeat and practice, learn from mentors, and ultimately change our self-beliefs.
Furthermore, inspired by the study of how the assignment of the house at Harry Potter could potentially change people’s mindset and personality, a researcher Nataliya Kosmyna from Dr. Maes’ team conducted experiments by using the sorting hat from Harry Potter and EEG to track children’s brain activities and “read” children’s minds. The research team put a physical sorting hat with a built-in EEG headset on the participants’ heads and asked them to complete certain tasks such as making a robot ball move by using their brain activities. The researchers can read the brain activity and state and give feedback to participants to “build trust.” With the intervention of both wearing the sorting hat and EEG and by giving feedback to children, such as telling them they will do well in math, the participants did the best compared to other conditions of having either hat or EEG or the control group without intervention.
4) Enhancing confidence in public speaking
Dr. Maes also shared how researchers use technology to transform a participant’s photo and generate a video of that participant speaking confidently. The study found that particularly for female participants, watching the deep fake video of themselves speaking confidently increased their level of confidence and decreased their level of stress compared to female participants in other conditions (for example, watching a relaxing video). The researchers observed similar trends for male participants but the level of confidence increase is less pronounced than that of female participants.
5) Improve wellbeing: PsychicVR-VR&EEG(Muse) for real-time mindfulness training
Lastly, Dr. Maes demonstrated how VR devices can be used to improve people’s well-being by giving them biofeedback to keep emotions under control and stay more in the moment using EEG and Virtual Reality. For instance, using a VR device, participants will receive visual feedback of watching a forest and it will promote relaxation and wellbeing.
Dr. Maes concluded that wearable immersive devices can help us grow and thrive and realize our full potential. Sensor data and AI techniques can provide real-time support for important cognitive skills such as creativity, attention, memory & learning, communication, and emotional regulation. On the other hand, Dr. Maes mentioned the ethical considerations of using technology to enhance human cognition. As people shape technology and technology shapes people, we need to consider overdependence, understanding, control, equity, and privacy.
—
by Eiffy Luo, Ed.M. (NLL Research Assistant)