高级检索

    基于图的同义词集自动获取方法

    Graph-Based Automatic Acquisition of Semantic Classes

    • 摘要: 同义词集是重要的语言基础知识,基于大规模语料库的同义词集自动获取是自然语言处理领域的一项基础性研究课题.从大规模语料中自动获取有并列结构关联的词语对,据此形成图,采用Newman算法对图进行划分而自动聚类相似词语.着重研究在Newman算法的基础上,充分挖掘和利用并列结构的特性和汉语的构词特点,采用6种方法对图中边的权值加以改进从而提升效果:分割语料、去除低频边、加重双向边、加重团、加重相同后字、惩罚音节不等.同义词集自动获取的准确率从初始的23.28%提升至53.12%,准确率提高了约30个百分点.

       

      Abstract: A semantic class is a collection of terms which share similar meaning. Knowing the semantic classes of words can be extremely valuable for many natural language processing tasks. This paper investigates the usage of linguistic knowledge on the graph-based acquisition of Chinese semantic classes, and demonstrates that linguistic knowledge can really improve the graph-based method. The used corpus is Xinhua News of LDC Chinese Gigaword. A graph is built by extracting word pairs with coordination structure from corpus, with the co-occurring words as nodes and the co-occurring frequency as edges weight between the two words. And then Newman algorithm is adopted to experiment word clustering in the graph. This paper focuses on transforming the edges weight, motivated by the properties of coordinate structure and Chinese language. We present six kinds of methods: divide the whole corpus to small parts, cut the low-frequency edges, enlarge the weight of bidirectional edges, enlarge the weight of edges within cliques, enlarge the weight of edges in which two nodes share the same last-character, and reduce the weight of edges in which two nodes have different number of characters. The experimental result with the six methods yields a promising precision of 53.12%, which outperform the baseline Newman algorithm by 29.84%.

       

    /

    返回文章
    返回