Administer > Smart Analytics administration > Administrator tasks > Modify Stop Words for IDOL search engine

Modify Stop Words for IDOL search engine

User role: System Administrator

A stop-word list is a list of terms that can be ignored when the search engine is searching or indexing. Typically, stop-word lists include short and common words or prepositions, such as "a," "the," or "with" in English. However, they may also include longer words, such as long number strings, or words that are too common to be useful as search targets, such as the term "internet." Stop words are removed from words entered in the "Search for" box unless they are enclosed in double quotes (phrase search). They are not removed during indexing to allow for phrase searching.

Smart Search applies both the Smart Analytics stop words and the Service Manager stop words. However, some stop words used in Service Manager conflict with the stop words logic used in Smart Search. For example, "before" is a stop word defined in Service Manager, which means that "before" in a search string is ignored. However, in Smart Search, it supports the search string as "A BEFORE B", which means that Smart Search returns the results in which A comes before B. If "before" is not removed from the Service Manager stop words list, the returned results are not as expected. To avoid this problem, when you enable Smart Analytics, some of the English stop words used by the Solr search engine are removed. To obtain a list of the removed stop words, search for "stop_words_to_remove" in the "IDOL_EnableKM"script library.

If you want to keep all the Service Manager stop words when you upgrade your Smart Analytics server, back up the stop words list before upgrade and import it after upgrade.

When you search by the IDOL search engine:

  • If the search word is a stop word defined in the Solr search engine, no result is returned.
  • If the search word is a stop word defined in IDOL search engine only, a warning message is displayed.

Stop words are stored in Service Manager in lists by specific language. Not all languages support stop words (for example, Japanese and Chinese). Adjust the list of stop words by either adding or removing words from this list.

The stop-word list for your log-in language is used by default, and is loaded once when you first log in. Changing the query language parameter on the advanced search screen changes the stop-word list used. The new stop-word list is loaded each time you search in a language other than your log-in language. This may cause a delay in your search being submitted as the stop-word list is loaded. If you need to perform extensive searches in a language other than your log-in language, we recommend that you log out and then log back in with the other language to reduce this delay.

To modify Service Manager stop words:

  1. Click Smart Analytics > Smart Search > More > Stop Words.
  2. Click Search.
  3. Select the record for the language code you wish to change.
  4. Add a new word or modify an existing word.
  5. Click Save.

To modify Smart Analytics stop words:

  1. Open the <Smart Analytics Installation>/lanfiles folder.

    The stop words lists are saved as the <language name>.dat file in this folder.

  2. Open the language file that you wish to change.
  3. Add a new word or modify an existing word.
  4. Click Save.