Raphael Thys
In a 12-by-20-foot room at a skilled-nursing facility in Menlo Park, California, researchers are testing the next evolution of the computer interface inside the soft matter of Dennis DeGray’s motor cortex. DeGray is paralyzed from the neck down. He was hurt in a freak fall in his yard while taking out the trash and is, he says, “as laid up as a person can be.” He steers his wheelchair by puffing into a tube.
But DeGray is a virtuoso at using his brain to control a computer mouse. For the last five years, he has been a participant in BrainGate, a series of clinical trials in which surgeons have inserted silicon probes the size of a baby aspirin into the brains of more than 20 paralyzed people. Using these brain-computer interfaces, researchers can measure the firing of dozens of neurons as people think of moving their arms and hands. And by sending these signals to a computer, the scientists have enabled those with the implants to grasp objects with robot arms and steer planes around in flight simulators.
This story was part of our November 2021 issue
DeGray is the world’s fastest brain typist. He first established the mark four years ago, using his brain signals to roam over a virtual keyboard with a point-and-click cursor. Selecting letters on a screen, he reached a rate of eight correct words in a minute. Then, right before the covid-19 pandemic began, he demolished his own record, using a new technique where he imagined he was hand-writing letters on lined paper. With that approach, he managed 18 words per minute.
One of the people responsible for the studies with DeGray is Krishna Shenoy, a Stanford University neuroscientist and electrical engineer who is among the leaders of the BrainGate project. While other brain-interface researchers grabbed the limelight with more spectacular demonstrations, Shenoy’s group has stayed focused on creating a practical interface that paralyzed patients can use for everyday computer interactions. “We had to persevere in the early days, when people said Ah, it’s cooler to do a robotic arm—it makes a better movie,” says Shenoy. But “if you can click, then you can use Gmail, surf the Web, and play music.”
Shenoy says he is developing the technology for people with “the worst afflictions and the most need.” Those include patients who are utterly locked in and unable to speak, like those in the end stage of ALS.
But if the technology allows people like DeGray to link their brain directly to a computer, why not extend it to others? In 2016, Elon Musk started a company called Neuralink that began developing a neural “sewing machine” to implant a new type of threaded electrode. Musk said his goal was to establish a high-throughput connection to human brains so that society could keep pace with artificial intelligence.
Dennis DeGray uses brain signals to control a computer cursor. In a typing task, he managed 18 words per minute, about half as fast as the average able-bodied person texting from a smartphone.
The same month Neuralink went public with its plans, Facebook announced it would develop a “noninvasive” brain-reading helmet to translate thoughts into social media posts. What’s followed has been a huge influx of investment in brain interfaces of all kinds, including EEG readers, magnetic headbands, and new types of high-density implanted probes capable of measuring signals from tens of thousands of neurons at a time.
More than $300 million has been raised by such companies in the last 12 months, even though Facebook this year dropped its quest (it determined a brain-reading helmet won’t be a feasible way to send texts for years). “The field was un-investable until Elon entered. That is what sent shock waves through the venture capital world,” says Shenoy. “Now there are nearly infinite resources.”
The money comes with a catch, though. Medical researchers like Shenoy want to help desperate cases. But entrepreneurs want the next interface for everybody. Musk has said he is aiming for brain implants that would be available to any consumer who wants one—Neuralink even designed a sleek white surgical chair where he imagines people will sit for a routine 30-minute implant procedure.
Shenoy, who is a paid consultant to Neuralink, told me he’s living a scientific paradox. He is opposed to consumer brain implants; he worries about everything from their impact on inequality (what if only some people can afford one?) to the consequences of directly linking people’s brains to social media. But he has made a Faustian bargain in working with Neuralink, which is bringing much-needed resources to commercializing an interface that—at first, at least—promises benefits for paralyzed people.
“It’s not comfortable, but welcome to science,” says Shenoy. “Anything that is therapeutic and restorative, I am into. Anything that is elective, enhancement—I don’t want to work on that. But when the technology is so early, you can’t pursue the restorative stuff without being generally aligned with the people who want to take it beyond. We are on the early part of the same path.”
Monkey Pong
Neuralink is a secretive company that communicates with the public mostly via theatrical presentations. The latest, released in April 2021, showed a rhesus monkey named Pager playing the video game Pong with its mind. The demo led to an excited response on social media—as well as a lawsuit by animal rights activists—but mind Pong was not new. A BrainGate subject named Matt Nagle had played the game against a Wired editor in 2005.
The real advance made by Neuralink was something not visible in the video—the implant itself. Chip designers at the company have built a soda-cap-size disc, containing processors and a wireless radio, that connects to electrodes stitched into the monkey’s cortex. The disc lies flush with the monkey’s skull and is covered with skin—giving the implant a more practical footprint than the cables that protrude from DeGray’s head.
In a blog post, Neuralink said that Pong was just a demonstration—and also articulated for the first time what its implant would be used for, at least in the near term. It read, “Our first goal is to give people with paralysis their digital freedom back: to communicate more easily via text, to follow their curiosity on the web, to express their creativity through photography and art, and, yes, to play video games.” A Neuralink engineer later told IEEE Spectrum that the company had the specific aim of beating DeGray’s brain-communication record.
But Musk’s long-term plans are equally clear: he thinks human brains need to be directly connected to phones, computers, and applications. You could run Google searches directly from your brain. Or you can even imagine connecting to someone else’s mind, seeing and hearing what the other person is doing.
Musk says all this is part of a strategy to offset existential risks he thinks future artificial intelligence will pose to humankind—like a scenario in which an AI decides to wipe out humanity, Terminator style. His view is that to prevent such an outcome, humans should become cyborgs and merge with AI. “If you can’t beat em, join em,” Musk typed into Twitter in July 2020, describing the phrase as the “Neuralink mission statement.”
Shenoy says he is developing the technology to restore a digital existence to people with “the worst afflictions and the most need.”
Neuralink says its eventual goal is “creating a whole brain interface capable of more closely connecting biological and artificial intelligence.” Technologically, achieving that goal means developing a high-bandwidth brain-computer connection that can tap into thousands or millions of neurons at once.
The technology isn’t there yet. The system used on DeGray measures from around 100 electrodes at once. In general, brain implants use each electrode to listen to one neuron. Neuralink’s N1 implant measures from 1,024 electrodes that lie along thin metal threads; that means it’s listening to about a thousand neurons. And it has only been tested in monkeys and pigs so far.
When it comes to consumer implants installed via elective brain surgery, regulators, public opinion, and even the medical profession may also stand in the way. In 2016, a Pew Research poll found that 69% of Americans were either very or somewhat worried about the prospect of brain chips that offered an improved ability to concentrate or process information. According to Pew, this opposition was strongly related to a fear of “loss of human control.”
And brain surgeons will still need some convincing before they drill holes in the heads of healthy people. Jaimie Henderson, the Stanford neurosurgeon who put in DeGray’s implants and co-leads the project with Shenoy, says he thinks small implants done with minimal trauma are “fairly low risk,” with the main hazard being a 3% to 5% chance of infection—a risk that may be worth it to improve a severely disabled person’s life. The question will be whether healthy people gain enough from an implanted computer mouse to offset the dangers, even if they’re small.
“It’s unclear to me what benefits able-bodied people would be able to get from any current brain-computer interface system,” Henderson says. “Our goal has been to try to restore function for people who have lost it, as best we can—not to provide some sort of ‘superhuman’ capability.”
Still, Shenoy was one of several academic scientists who told me that, like it or not, they do think consumer brain implants are going to be possible. Enough subjects like DeGray have lived with implants for years, with few ill effects, and they’re achieving useful mastery of the brain mouse. “Technologically, I see no barrier. I would not have said that 10 years ago and might not have said it five years ago,” Shenoy says. “It’s basically electrodes, chips, and a radio.”
To some, such an interface is intriguing because of the sheer amount of time we now spend on phones, playing video games, listening to podcasts, or scrolling through social media. That’s propelling investments in new ways of interfacing with the brain, says Nita Farahany, a law professor at Duke University who is writing a book on consumer neurotechnology.
“The question of why seemingly disparate companies are investing is that if you could use your brain as the controller, instead of a mouse or joystick, it’s not so crazy to want to invest,” says Farahany. “This may be the next revolution in the computer interface.”
Nathan Copeland is another paralyzed person living with a brain implant—he’s part of a study in Pittsburgh. Last year he became the first to plug his head into a tablet computer at home, on his own time, not as part of scientific session (it normally takes a small team of medical workers in a clinical setting). Copeland told me at first he was using the device eight hours a day, playing video games and using drawing programs. He later tired of it—his tablet is a medical device that uses an older version of Windows, and its battery doesn’t last long.
Still, Copeland told me he believes paralyzed people are “test pilots” for future consumer brain interfaces. In his own case, he says, he’s mostly interested in being able to play more video games—one of his favorite pastimes—at a higher level.
Game changer
Of the 35 or so people who have received a long-term brain implant to interface with a computer, 29 of them, including DeGray, have electrode implants built by a company called Blackrock Neurotech, based in Salt Lake City. The implant, aptly called the Utah array, is a silicon square with 100 small needles, which is pushed into the surface of the brain. Blackrock mostly sells systems to researchers experimenting on animals, but as investors have flocked to implants, observers have sometimes called Blackrock and Neuralink the Lyft and Uber of brain interfaces.
The president of Blackrock, an electrical engineer named Florian Solzbacher, thinks the timing is right to take implants forward for paralyzed people. “People would say Oh my God, it’s brain surgery, but actually we haven’t seen any problems,” he says. Every time there is a video of someone controlling a robot or eating a Twinkie with a robotic hand, Solzbacher says, he gets calls from paralyzed people wondering when a commercial product could be available. It’s only recently that he’s been willing to say it could happen soon: “It’s always been 15 years away, and now what I can say for the first time is soon you will be able to take it home.”
The “mixed reality” experiments done in virtual space hint at how able-bodied people might experience computer worlds through brain interfaces.
That’s due to several factors, including the development of a wireless version of the BrainGate hardware. Instead of cables, subjects have a hockey-puck-size wireless transmitter screwed onto their brain ports. It’s nothing as compact and sleek as Neuralink’s electronics, but it works. Solzbacher says his company plans to seek approval to sell its own improved wireless system to people with ALS or severe paralysis.
Solzbacher says DeGray’s typing points to the potential of the technology—he can tap out words much faster than anyone using an EEG headband, for example. “That means you are 10 times faster than anything that is out there,” he says. “Now you can start being productive, and you have performance close to an able-bodied person.”
However, Solzbacher is being financed by people who aren’t only interested in helping paralyzed people. This year his company raised $10 million from investors including the German billionaire Christian Angermayer, who invests widely in psychedelics, longevity treatments, and mental health. In a tweet, Angermayer left no doubt he thinks a general-purpose brain mouse is the ultimate goal: “It’s fundamentally an input-output device for the brain, and it can benefit ALL. We can unlock truly astonishing use cases & I believe Blackrock will be the one to take us there. Ppl will communicate with each other, get work done + even create artwork, directly with their minds.”
Solzbacher says for now, none of Blackrock’s internal plans or projections involve consumer brain implants. Still, he acknowledges that could happen: “I expect there is part of society that may really want it, even though there is nothing wrong with them.”
I asked Solzbacher whether any able-bodied person had ever requested such a device. He says he’s hasn’t gotten such a request, yet.
Mixed reality
Robert “Buz” Chmielewski had his head down in concentration, and because of a screen, he couldn’t see which of two toy-size soccer balls had been placed in the robotic hand he was controlling. Using his thoughts, Chmielewski closed the plastic and metal hand and squeezed the ball. “Pink ball,” he called back. When the researcher swapped it for another, stiffer ball, Chmielewski could sense the change. “Black ball,” he said.
Chmielewski, 50, got his Utah arrays implanted in 2019, 30 years after a surfing accident in Ocean City, Maryland, left him in a wheelchair. During the two years the experiment lasted (it ended in September), he had more implants put in than any other patient—a total of six, in both hemispheres of his brain. Because of this, he was able to control two robot arms simultaneously. What’s more, three of the probes placed into his sensorimotor cortex sent signals back into his brain, allowing him to receive tactile information from the robots.
Robert “Buz” Chmielewski, 50, had implants in both hemispheres of his brain. When they were in place, he was able to control two robot arms simultaneously.
Chmielewski was part of a project at Johns Hopkins University’s Applied Physics Laboratory that’s testing new forms of perception. He also tried out the Microsoft HoloLens headset and used his sense of virtual touch to arrange blocks in virtual space. “If you would have told me three years ago I would be controlling things with my thoughts, I would have said you’re crazy,” Chmielewski said during a recent online presentation. “Some of the applications we are working on have blown my mind.”
The researchers at APL include Michael Wolmetz, manager of the Human and Machine Intelligence program. Wolmetz says the demonstrations are a glimpse of “fundamental” changes ahead in human-computer interaction, especially the concept of “mixed reality.” The experiments in virtual space hint at how able-bodied people might experience computer worlds through brain interfaces—making the APL project one of the most explicit explorations of how such technology might lead to human enhancement.
“For all of biological history, the only way we’ve interacted with the environment is with senses and motor function,” Wolmetz says. “We have, for the first time, the ability to go outside that paradigm. It’s the first time any biological organism has done that.”
Wolmetz doesn’t know whether surgically implanted brain interfaces will ever be widely used, but he says these devices are a “sneak peek” at how consumers might use future noninvasive systems like brain-reading helmets or headbands, should accurate ones be developed.
When I asked Wolmetz what he thought people might use such interfaces for in the future, he said it is hard to predict. “It’s like asking what the computer is going to be for,” he says. “I think that in our lifetimes it will be for everything. But in the next five years, it’s hard to answer.”
Some want not only the computer mouse but the entire interface—including the screen, or whatever replaces a screen—in the brain. One of them is Max Hodak, the former president of Neuralink. He was fired by Musk in March—it’s not clear why—but quickly formed a new company, called Science Corp., with financial backing from the cryptocurrency billionaire Jed McCaleb. Hodak says he plans to develop a new type of implant that rests on the retina and can send information to the visual cortex at the back of the brain.
Initially Hodak’s new company will be looking to help people, like his grandfather, who went blind from retinal diseases. But a medical product is a stalking horse for a bigger ambition, which is to create a device that can produce images in the eyes of healthy people as well.
“It could just be a computer screen that looks as solid as any ever has, and it’s just floating in front of you,” he says. “When your eyes are open, you would see the world of atoms. When you close your eyes, you see the world of bits.” Hodak thinks that in a generation, children will be “baffled when we tell them that there used to be just nothing there when we closed our eyes.”
Ethics questions
Before Musk and venture capitalists arrived on the scene, DARPA, an R&D agency of the US Department of Defense, was the world’s largest funder of brain-interface research.
Andy Schwartz, a researcher at the University of Pittsburgh, told me he is convinced the military’s fascination with the technology springs from a 1982 Clint Eastwood film, Firefox, whose plot involves an effort to steal a thought-controlled Soviet MiG jet. After the military had one of his research subjects fly a simulated war jet, Schwartz says, he stopped collaborating with the agency.
John Donoghue, a professor at Brown University and one of the founding scientists of BrainGate, is also concerned about a “circus-like atmosphere” around brain implants. He spent time in a wheelchair as a child, which is one reason he has pursued the goal of restoring movement to paralyzed people. But when he gave a talk at Google a few years ago, an engineer approached him and said he was an avid gamer. The engineer wanted to know if it would be possible to have a third thumb.
“That’s taking things to an extreme. I don’t want to implant electrodes into people so they can be better gamers,” says Donoghue. “I always challenge all of these ideas because I don’t see what it gets you. But I don’t dismiss it, either … that is what is driving people. It’s the cool factor, that you could have this new interface.”
Donoghue doubts that implants will provide superpowers, or that you’ll be able to download French for Beginners directly into your head anytime soon. The brain has evolved to receive and send information at the speed that it does—not at the rate of an Ethernet cable. “Have you listened to a podcast at 4x speed? It doesn’t work very well,” he says. “Our brains are made to make and intake speech at a level that we can use it.”
Shenoy says his concern is that putting computer interfaces into people’s minds will lead to inequality and the same sorts of information abuses seen on the internet.
But others believe that mind reading and mind control are rising dangers. In 2017, the same year that both Neuralink and Facebook’s brain-interface plans were unveiled, a group of researchers calling itself the Morningside Group published a manifesto in the journal Nature. It sounded alarm bells about a “convergence” between brain technology and AI advances.
The group formed at the urging of Rafael Yuste, a neuroscientist at Columbia University, who became alarmed over experiments in his own lab, in which he could not only read from the visual center of a mouse brain but also use a laser to make the animal perceive things that were not there. “We had control over the visual perceptions of the mice, and we could run them like puppets,” says Yuste.
Yuste keeps a list of experiments he thinks point to how neurotechnology could compromise human autonomy. For instance, there’s the work of Jack Gallant, in California, who has used MRI scanners to deduce what images people are seeing. Then there’s the scientist who wired one monkey’s brain to control the arm of different monkey, calling one the “master” and the other its “avatar.”
The fundamental fear is that everything bad about the internet—disinformation, malicious hackers, government control, corporate manipulation, endless harassment—could get much worse if technology were to breach what the Morningside Group calls “the last frontier of privacy” and know our thoughts. “There is a huge problem, and it’s the problem of mental privacy,” says Yuste.
In May, Yuste hosted a day-long online gathering of ethicists and neurotech entrepreneurs to discuss responsible neural-interface design.
Several participants said they believed there was a need to establish rules before it becomes possible to collect brain information easily. “We don’t want to go through this cycle of big corporations harvesting data to profit from and then, in the end, facing regulations and asking for forgiveness,” Ryan Field, CTO of Kernel, which is developing a noninvasive headset to read brain activity, said during the event.
Yuste wants far stricter privacy rules than those governing internet data or what’s on your iPhone. He would like to see brain data treated like transplant organs—carefully tracked and with a ban on any profit-making. At a minimum, he says, brain data should be protected like medical information. He also says the military should be forbidden from employing brain implants.
“I have to change me”
In certain ways, the field of brain-computer interfaces is already beginning to realize its loftiest goal and some people’s greatest fear: the merger of humans and AI.
That’s certainly the case with research volunteers like DeGray. The buzzing of his neurons is interpreted by AI software called a recurrent neural network. Each day that DeGray uses his implant, he starts by imagining some simple movements, like drawing a circle. The neural network listening to his neurons then calibrates the statistical map that relates each neuron’s activity to the movement. And most brain-computer interfaces won’t only use software to interpret brain signals, but also to improve on them—for example, programs might predict what word someone is trying to spell on the basis of the first few letters.
This results in what Blackrock’s Solzbacher calls “shared agency,” or outputs that are picked partly by a person and partly by a machine. “That is scientifically interesting but is also an ethics question,” he says. “Because who is actually making the decisions when the systems adapt?”
This is how your brain makes your mind
Currently, the closest thing there is to brain-interface experience design is the experiments being carried out with DeGray in California. Most recently, the team has been trying to get DeGray to try mental touch-typing. If software can track what movements he’s thinking of making with his fingers, that could increase his communication speed even more. The problem is that before his accident, DeGray was never more than a hunt-and-peck typist. He now has paper keyboards pasted on the ceiling above his bed so he can practice thinking about typing.
One thing I wanted to know from DeGray is what it feels like to operate a computer with his brain. He described what he calls a “meeting of the minds” with the cartful of machines and software reading his thoughts. This was particularly true when he was performing the imagined-handwriting task.
“It’s a very personal interaction. You have to feel for where the movements are in your own body,” he says. “You are trying to write the letters, and it is trying to understand you. I wouldn’t call it a relationship, but it’s close. It’s almost a conversation between the device and myself. Some days it’s a little bit surly at first—it’s hard to wake it up. Of course, the machine is perfectly constant. So I have to change me to get it to work.”
One day DeGray imagined writing 5,000 words. He worked so hard at it the researchers had to remind him to breathe. “I just pounded it out,” he says. “Over the course of doing so many words, you can get consistent. You quickly lapse into a pattern that the computer can recognize.”
