Lexical analysis

Symbol Description
Digits Numbers do not make good index terms, and are not usually included as tokens. In some instances, query statements consist of only digits, such as a record number. Therefore, IR Expert indexes digits along with alphabetic characters.
Hyphens Words broken at the end of lines, or including hyphens, can result in multiple word fragment tokens. IR Expert considers hyphenated terms as a single token and does not break them apart.
Other punctuation Other punctuation, including periods (.), commas (,), and underscores (_) are often used as parts of terms. IR Expert allows apostrophes (’), dashes (-), and periods (.) to appear within, but not at the beginning or end of a token.
Case Case distinctions are important in some cases, such as programming languages. IR Expert is case-insensitive. It converts all HPE Service Manager database terms to lower case.