Improving part-of-speech guessing of Chinese unknown words using hybrid models
This paper presents a hybrid model for part-of-speech (POS) guessing of Chinese unknown words. Most previous studies on this task have developed a unified statistical model for all Chinese unknown words and have rejected rule-based models without testing. We argue that models that use different sources of information about unknown words, both structural and contextual, can be effective for handling different types of unknown words. We propose a rule-based model that uses information about the type, length, and internal structure of unknown words and combine it with two existing statistical models that use information about the POS context and component characters of unknown words respectively for this task. By combining the complementary strengths of the three models that use different sources of information, the hybrid model achieves an accuracy of 89%, a significant improvement over the best result reported in previous studies.
Keywords: Chinese unknown words, POS tagging, rule-based models, hybrid models, corpus annotation, linguistic knowledge
Published online: 26 May 2008
Cited by 1 other publications
Lu, Xiaofei & Ben Pin-Yun Wang
This list is based on CrossRef data as of 19 may 2021. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.