An experiment by University of Washington researchers is setting the stage for advances in mind reading technology. Using brain implants and sophisticated software, researchers can now predict what their subjects are seeing with startling speed and accuracy.
The ability to view a two-dimensional image on a page or computer screen, and then transform that image into something our minds can immediately recognize, is a neurological process that remains mysterious to scientists. To learn more about how our brains perform this task—and to see if computers can collect and predict what a person is seeing in real time—a research team led by University of Washington neuroscientist Rajesh Rao and neurosurgeon Jeff Ojermann demonstrated that it’s possible to decode human brain signals at nearly the speed of perception. The details of their work can be found in a new paper in PLOS Computational Biology.
The team sought the assistance of seven patients undergoing treatment for epilepsy. Medications weren’t helping alleviate their seizures, so these patients were given temporary brain implants, and electrodes were used to pinpoint the focal points of their seizures. The UW researchers saw this as an opportunity to perform their experiment. “They were going to get the electrodes no matter what,” noted Ojermann in a UW NewsBeat article. “We were just giving them additional tasks to do during their hospital stay while they are otherwise just waiting around.”
The patients were shown a random sequence of pictures—images of human faces, houses, and blank gray screens—on computer monitors in brief 400 millisecond intervals. Their specific task was to watch for an image of an upside-down house.
At the same time, the electrodes in their brain were connected to software that extracted two distinct brain signal properties, namely “event related potentials” (when massive batches of neurons simultaneously light up in response to an image) and “broadband spectral” changes (signals that linger after viewing an image).
As the images flickered on the screen, a computer sampled and digitized the incoming brain signals at a rate of 1,000 times per second. This resolution allowed the software to determine which combination of electrode locations and signals correlated best to what the patients were seeing. “We got different responses from different (electrode) locations; some were sensitive to faces and some were sensitive to houses,” Rao said.