高级检索

    融合形态特征的最大熵蒙古文词性标注模型

    Fusion of Morphological Features for Mongolian Part of Speech Based on Maximum Entropy Model

    • 摘要: 最大熵模型以其能够较好地包容各种约束信息及与自然语言模型相适应等优点在词性标注研究中取得了良好的效果.因此,将其作为基本框架,提出了一种融合语言特征的最大熵蒙古文词性标注模型.首先,根据蒙古文构词特点及统计分析结果,定义并选取特征模板,利用训练语料提取了大量的候选特征集合,针对错误或者无效的特征通过设置一些规则筛选特征.然后,训练最大熵概率模型参数.实验结果表明,融合蒙古文形态特征的最大熵模型可以较好地标注蒙古文.

       

      Abstract: Part of speech tagging is one of the basic research for natural language processing fields, which plays an important role on the syntactic analysis, semantic analysis and machine translation, etc. Maximum entropy model is an outstanding statistical model for its good integration of various constraints and it has been favored in the part of speech tagging research. An approach combining linguistic morphological features for Mongolian part of speech tagging based on maximum entropy model is proposed in this paper. Mongolian has great and long history. Nonetheless, there is less research about Mongolian language processing. Mongolian is a typical agglutinative language that is characterized by rich morphology, with a high level of ambiguity. Firstly, based on the analysis of Mongolian scripts, the context feature and internal feature templates are defined and extracted from the training corpus. Then, various morphological features of words are integrated in the maximum entropy model and the IIS algorithm is employed to calculate the parameters of maximum entropy model. Experimental results on the close and open testing set prepared for Mongolian POS tagging task show that the integration of morphological features of the maximum entropy model outperforms the HMM model and can be fitful for Mongolian scripts.

       

    /

    返回文章
    返回