高级检索

    大语言模型的齿状回:基于参数化知识模式分离的场景认知能力诊断及增强方法

    The Dentate Gyrus of Large Language Models: A Parameterized Knowledge Pattern Separation Framework for Scenario Cognition Diagnosis and Enhancement

    • 摘要: 大语言模型在场景认知任务上表现出系统性的泛化局限,暴露了其知识记忆过程中可能存在的功能性缺陷。对此,本文受人脑海马体中齿状回结构的模式分离机制启发,提出了场景认知失败与模型内部模式分离能力局限之间的相关性假设。基于这一假设,本文首先设计了一种跨实体表示诊断(cross-entity representation diagnosis,CERD)方法,在无需额外训练的条件下量化模型各层的实体表示混淆程度,并观察到标准自回归训练并未使模型形成充分可用的模式分离能力,其内部仍存在跨实体的相关性混淆。在此基础上,本文进一步提出隐空间模式分离(latent-space pattern separation,LPS)方法,在实体混淆较为严重的目标层引入三元组对比正则化,将模式分离约束显式融入大语言模型的训练过程,以降低模型内部易混淆知识间的相关性。覆盖0.8B至9B参数规模的大量实验结果表明,该方法能够相对稳定地提升大语言模型的场景认知能力,相较于标准自回归训练,本方法在EM指标上获得了最高101.5%的相对提升,为理解与改进大语言模型的场景认知能力及知识记忆机制提供了新的视角。

       

      Abstract: Large language models (LLMs) exhibit systematic generalization limitations in scenario cognition tasks, revealing potential functional deficiencies in their knowledge memorization processes. Inspired by the pattern separation mechanism of the dentate gyrus in the human hippocampus, this paper hypothesizes that failures in scenario cognition are associated with limitations in the internal pattern separation capability of LLMs. Based on this hypothesis, we first design a cross-entity representation diagnosis (CERD) method, which quantitatively measures entity representation confusion across different layers without additional training. Our analysis reveals that standard autoregressive training fails to induce sufficiently effective pattern separation capabilities, leaving substantial cross-entity correlation confusion within LLM representations. Building upon this observation, we further propose latent-space pattern separation (LPS), which explicitly incorporates pattern separation constraints into the training process by introducing a triplet-based contrastive regularization at target layers with severe entity confusion, thereby reducing correlations among easily confused knowledge representations. Extensive experiments across LLMs ranging from 0.8B to 9B parameters demonstrate that LPS consistently improves scenario cognition capabilities. Compared with standard autoregressive training, our method achieves up to 101.5% relative improvement in exact match (EM), providing a new perspective for understanding and enhancing scenario cognition and knowledge memorization mechanisms in LLMs.

       

    /

    返回文章
    返回