Improving accuracy for Smart Ticket

Smart Analytics provides several methods to help you to increase Smart Ticket accuracy according to different data sets. This section provides some best practices for you to improve the accuracy of Smart Ticket.

• Set up data cleansing rules

  Data cleansing can help you prepare the data that you want to send to Smart Analytics for indexing, training, and analyzing. By setting up proper data cleansing rules, you can have better data quality, which is critical to the best accuracy of auto-suggestion. To set up data cleansing rules, see Configure data cleansing.

• Choose best sample data

  In the definition for Smart Ticket or Hot Topic Analytics, you can specify a sample data query, through which you can decide what kind of data that you want to use as sample data to teach Smart Analytics to build the intelligence out of your large data volume.

  For example, you may have an "Other" category in your Service Manager implementation to accommodate the interactions for which the Service Desk agents cannot find a better, more accurate category. Normally, the interactions in this "Other" category are not considered as good sample data for Smart Ticket. We recommend that you add a filtering clause such as category~="Other" into the Training Sample Query field to exclude those records.

  Another example is that you can choose the records logged by Subject Matter Experts (SME) as the training samples for Smart Tickets. In this way, you can have sample data with better quality for categorization.

• Apply rule based training for Smart Ticket

  The basic training of Smart Ticket is meaning based training, which means Smart Analytics builds its intelligence based on the text information of your data. On top of meaning based intelligence, Smart Analytics also supports you to add rule based training to the Smart Ticket. Those rules will further increase the suggestion accuracy, especially in the case that multiple suggestion results have the same relevancy with the new record. The typical scenario is that if one particular record has the same relevancy within several categories, you can append a rule to one specific category to improve the categorization accuracy.

  For how to apply a rule to the Smart Ticket task definition, see Apply a rule-based training.

• Optimize your training for Smart Ticket

  Several advanced parameters defined in the Smart Ticket task definition are used to optimize the accuracy of auto suggestion. Note that these settings are tradeoffs between training time and accuracy, which means higher accuracy is achieved at the cost of longer training time. Listed below are some best practices for these optimization configurations.

  • Training by documents or training by terms

    Choose “best term” for a faster training process if you have huge data volume; choose "training documents" for a higher accuracy with a slower training process.

  • Training sample per category

    The maximum records for each category, normally more training sample per category leads to higher accuracy but longer training time.

  • Source data coverage

    The percentage of records out of the total source data that are used to create categories. Normally higher percentage means higher accuracy, but there is a threshold point. When the training source data percentage exceeds the threshold, the margin contribution will be lowered remarkably. The out-of-box value for this configuration is 90%, which is a best number tested in the lab. You can use the “source data coverage calculator” tool to find the best number for your data set.

  • Document weight and term weight

    Enable "Adjust term weight from test result" to automatically adjust the term weight for some terms in some categories based on the testing result, which may help improve accuracy.

    Enable "Remove low weight document" to help reduce the disturbance of low relevance training samples and improve accuracy.

    By default, these two parameters are disabled in the out-of-box environment.

    These advanced features need your experiment to get best results. You may enable either one or both.

• Perform tuning periodically

  Tuning the training result is a mechanism to continuously improve the accuracy of auto suggestion. For information about how to tune the training result, see Perform tuning in the Smart Ticket definition.