Administer > Database administration > Data persistence > IR Expert > Chinese, Japanese, and Korean language analyzer

Chinese, Japanese, and Korean language analyzer

The language analyzer is a text tokenization analysis tool. It uses statistical-based, rule-based, and dictionary-based techniques to quickly determine the correct segmentation of Chinese, Japanese, or Korean text.

In addition to tokenization, the language analyzer offers normalization, part-of-speech information, morphological analysis, and reading information.