Information Retrieval (IR) expert parameters: ir_term_drop_off

Parameter

ir_term_drop_off

Description

This parameter defines the maximum frequency (expressed as a percentage) with which search terms can appear in the Information Retrieval (IR) index before Service Manager excludes them from search results. Service Manager ranks each search term, based on how frequently it appears in the index. The less frequently that a term appears in the index, the more relevance Service Manager assigns to it in search results. If a term appears too frequently in the index then Service Manager ignores the search term as if it were in the stop word list.

Service Manager determines both a term search frequency and a relevance ranking for each search term. The search term frequency is a simple percentage that is calculated with the following formula:

<number of instances of search term in index>/terms in index * 100

The relevance ranking is determined by computing an IDF value. Service Manager uses the following formula to compute the IDF value of search terms:

[natural log (<terms in index>/<number of instances of search term in index>)]+1

For example, in an index of 1000 terms, a search term that appears 250 times in the index has a frequency percentage of 25% and an IDF value of 2.4. Since this is above the maximum frequency percentage value of 22%, Service Manager ignores the term as too frequent. A search term that appears only 10 times in a 1000 term index however has a frequency percentage of 1% and an IDF value of 5.6. Since this term is within the percentage frequency threshold, Service Manager includes it in the search results, although it will not be as relevant as search terms with a lower IDF value.

Valid if set from

Server's operating system command prompt

Initialization file (sm.ini)

Requires restart of the Service Manager server?

Yes

Default value

22

Possible values

Maximum percentage frequency

Example usage

Command line: sm -httpPort:13080 -ir_term_drop_off:25

Initialization file: ir_term_drop_off:25

Related topics

System parameters

Related topics

Enter a parameter in the sm.ini file

Related topics

Information Retrieval (IR) expert parameters