心智字幕：从人类大脑活动中演化出心理内容的描述性文本

心智字幕：从人类大脑活动中演化出心理内容的描述性文本

Mind captioning: Evolving descriptive text of mental content from human brain activity

——《科学进展》第11卷第45期，2025年11月7日——

Vol 11, Issue 45, 7 Nov 2025

【摘要】神经科学的核心挑战之一是解码大脑活动，以揭示包含多个组成部分及其相互作用的心理内容。尽管在从人脑活动中解码语言相关信息方面取得了进展，但生成与结构化视觉语义相关的复杂心理内容的全面描述仍然具有挑战性。我们提出了一种方法，该方法通过深度语言模型计算的语义特征生成反映大脑表征的描述性文本。我们构建了线性解码模型，将视频诱发的大脑活动转化为相应字幕的语义特征，并通过词语替换和插值将候选描述的特征与大脑解码的特征对齐，从而优化了这些描述。即使不依赖规范语言网络，该过程也能生成结构良好的描述，准确捕捉观看的内容。该方法还可以推广到将回忆的内容口头表达出来，作为心理表征和文本之间的解释性接口，同时展示了基于非语言思维的脑-文本交流的潜力，这可能为语言表达障碍（例如失语症）患者提供一种替代的沟通途径。 

[Abstract] A central challenge in neuroscience is decoding brain activity to uncover mental content comprising multiple components and their interactions. Despite progress in decoding language-related information from human brain activity, generating comprehensive descriptions of complex mental content associated with structured visual semantics remains challenging. We present a method that generates descriptive text mirroring brain representations via semantic features computed by a deep language model. Constructing linear decoding models to translate brain activity induced by videos into semantic features of corresponding captions, we optimized candidate descriptions by aligning their features with brain-decoded features through word replacement and interpolation. This process yielded well-structured descriptions that accurately capture viewed content, even without relying on the canonical language network. The method also generalized to verbalize recalled content, functioning as an interpretive interface between mental representations and text and simultaneously demonstrating the potential for nonverbal thought–based brain-to-text communication, which could provide an alternative communication pathway for individuals with language expression difficulties, such as aphasia. 

论文原文：Tomoyasu Horikawa (2025). Mind captioning: Evolving descriptive text of mental content from human brain activity. Science Advances, Volume 11, Issue 45, 7 Nov 2025

https://doi.org/10.1126/sciadv.adw1464

（需要英文原文的朋友，请联系微信：millerdeng95或iacmsp）