Improve Smart Analytics query performance

User Role: Administrator

Caching some match fields into memory can accelerate the query speed of Smart Analytics. To achieve this, you can modify the configuration files and corresponding sections of the content servers for which you want to accelerate the query speed.

To configure cache settings for match fields, follow these steps:

1. Open the <SmartAnalytics>/Content<x>/Content<x>.cfg files of the content servers for which you want to accelerate the query speed.

2. Configure the cache settings for match fields as follows:

   [server]
   NodeTableCacheFields=True
   NodeTablePrepopulateCache=True
   
   //--------------------------- Cache Settings ---------------------------------//
   
   //Maximum size of the field cache (in KB). Increasing the size of the cache can improve performance. However, you must ensure the host machine has sufficient available memory.
   [MemoryCache]
   EvictWhenFull=True
   MaxSize=1024000
   
   
   [FieldProcessing]
   23=SetCachedFields
   
   [SetCachedFields]
   Property=CachedFields
   PropertyFieldCSVs=*/*_MATCH,*/IMPORTMAGICEXTENSION
   
   
   //---------------------------Properties-------------------------------//
   
   [CachedFields]
   MemCachedType=True
   

3. Restart the content servers for which you modified the configuration files.

   Note It takes a while for the content servers to get restarted. For example, it takes about 35 minutes if you restart content servers on an environment with 1G memory cache. You can monitor the restart status by checking the <Smart Analytics>/Content<x>/logs/application.log file. If the caching is successful, you can see log information similar to the following:

   22/09/2016 16:45:50 [1] 30-Normal: Indexing nodetable memory information for 2268213 documents.
   22/09/2016 16:46:26 [1] 30-Normal: Indexing information from documents 10% complete.
   22/09/2016 16:47:35 [1] 30-Normal: Indexing information from documents 20% complete.
   22/09/2016 16:49:31 [1] 30-Normal: Indexing information from documents 30% complete.
   22/09/2016 16:53:39 [1] 30-Normal: Indexing information from documents 40% complete.
   22/09/2016 16:58:10 [1] 30-Normal: Indexing information from documents 50% complete.
   22/09/2016 17:02:05 [1] 30-Normal: Indexing information from documents 60% complete.
   22/09/2016 17:06:28 [1] 30-Normal: Indexing information from documents 70% complete.
   22/09/2016 17:10:51 [1] 30-Normal: Indexing information from documents 80% complete.
   22/09/2016 17:15:11 [1] 30-Normal: Indexing information from documents 90% complete.
   22/09/2016 17:19:34 [1] 30-Normal: Populated nodetable memory cache.
   22/09/2016 17:19:34 [1] 30-Normal: Writing out stateinfo
   22/09/2016 17:19:34 [1] 00-Always: Initialized memory. 
   

To improve the Smart Search response performance, you need to update the following configuration files in all content servers and the Smart Analytics Main server:

• <SmartAnalytics>/Content1/Content1.cfg
• <SmartAnalytics>/Content2/Content2.cfg
• <SmartAnalytics>/Content3/Content3.cfg
• <SmartAnalytics>/IDOL/IDOLServer.cfg
• <SmartAnalytics>/Content-SMSP/Content-SMSP
• Scaled content server

Follow these steps:

1. Update the existing parameters as highlighted in bold, or add the highlighted parameters if they do not exist:

   • IndexCacheMaxSize=512000

     Sample code:

     //--------------------------- Cache Settings -----------------------------//
     [IndexCache]
     IndexCacheMaxSize=512000

   • PropertyFieldCSVs=*/*_MATCH,*/EDK_*,*/*_MATCH_INDEX, */*_MATCH_WEIGHT*,*/*_BOOLEAN,*/IMPORTMAGICEXTENSION

     Sample code:

     [SetMatchFields]
     // Specify fields whose entire value is frequently used as a restriction in a search and are thus
     // stored for optimised matching.
     // NB Such fields cannot be numeric and can only be parametric if parametricnumericmapping=false.
     
     Property=MatchFields
     PropertyFieldCSVs=*/*_MATCH,*/EDK_*,*/*_MATCH_INDEX, */*_MATCH_WEIGHT*,*/*_BOOLEAN,*/IMPORTMAGICEXTENSION

2. Locate the [DateFields] section under the Properties separation line, and then add a new parameter Index=TRUE:

   //---------------------------Properties-------------------------------//
   
   [IndexFields]
   Index=TRUE
   
   [IndexWeightFields2]
   Index=TRUE
   Weight=2
   
   [IndexWeightFields4]
   Index=TRUE
   Weight=4
   
   [IndexWeightFields8]
   Index=TRUE
   Weight=8
   
   [IndexWeightFields16]
   Index=TRUE
   Weight=16
   
   [SectionFields]
   SectionBreakType=TRUE
   
   [DateFields]
   DateType=TRUE
   Index=TRUE
   

3. Restart the content server and the IDOL server.

4. Perform a full re-index for Smart Search.

To improve Smart Search performance, you need to update the following configuration files:

• <SmartAnalytics>/Content1/Content1.cfg
• <SmartAnalytics>/Content2/Content2.cfg
• <SmartAnalytics>/Content3/Content3.cfg
• <SmartAnalytics>/Content-SMSP/Content-SMSP
• Scaled content server

Follow these steps:

1. Update the Schedules section as follows:

   //--------------------------- Schedules ----------------------//
   //expire and compact data everyday
   [Schedule]
   Expire=TRUE
   ExpireTime=23:00
   ExpireInterval=24:00
   Compact=True
   CompactTime=00:00
   CompactInterval=24:00
   

2. Restart the content servers.