USING CHARACTER BIGRAM FOR AMBIGUITY RESOLUTION IN CHINESE WORD SEGMENTATION

This paper presents a method of using two kinds of statistical measures,mutual information and difference of t test of adjacent characters in sentences,to deal with ambiguities in Chinese word segmentation.A Chinese character bigram matrix,derived automatically from the raw corpus,serves as a basis for the related calculations.Preliminary experiments show that 90 3% correct rate is achieved for overlapped ambiguities.