One implant enables parallel decoding of speech and gestures for natural multimodal communication in paralysis
Three paralyzed participants with a single high-density ECoG implant demonstrated simultaneous decoding of speech and upper-body gestures. By training decoders on both isolated and concurrent data and using cross-modality training to reduce opposite-modality false activations, the system achieves reliable parallel decoding and real-time avatar control, marking a step toward a unified, natural multimodal brain–computer interface for communication restoration.
