Lexical analysis

Symbol Description
Digits Numbers do not make good index terms, and are not usually included as tokens. In some instances, query statements consist of only digits, such as a record number. Therefore, IR Expert indexes digits along with alphabetic characters.
Hyphens Words broken at the end of lines, or including hyphens, can result in multiple word fragment tokens. IR Expert considers hyphenated terms as a single token and does not break them apart.
Other punctuation Other punctuation, including periods (.), commas (,), and underscores (_) are often used as parts of terms. IR Expert allows apostrophes (’), dashes (-), and periods (.) to appear within, but not at the beginning or end of a token.
Case Case distinctions are important in some cases, such as programming languages. IR Expert is case-insensitive. It converts all Service Manager database terms to lower case.

Related topics

Special considerations for using IR Expert
How IR Expert evaluates documents for relevance
IR Expert file descriptions
Updates to IR files
Customizing IR Expert
Find Solution
Creating an IR file