1 - Introduction

Classification of sentences is a task that seems relatively easy.

After a while, the AI model of your xBot has been fed by many new sentences that you have either classified in Intents, archived or deleted.

If your classified sentences have a low match, are too similar with sentences of other Intents, have low variability with other sentences classified within the same Intent, you may over time unbalance or desimprove the capacity of your xBot to recognize user questions correctly and provide the right answer.

Once an AI model is unbalanced, it is not easy to rebalance it. To mitigate and correct this, we have created a classification diagnostic tool, called the Optimization (formally called Classification helper), that can help you to optimize the current AI model of your xBot.

2 - the Optimization tool

Now, you can access the Optimization page by clicking on the icon on the vertical menu on the left side of the my.satisfaction.AI portal.

On this page, you get on the left a view on the Intents that do not have enough associated sentences.

If you want to get a diagnostic on the classification of your sentences and detect potential errors, you can click on the LAUNCH DIAGNOSTIC link on the right:

Once the Diagnostic process is completed (it can take several minutes), a new Tab opens with the results of the Optimization tool.

- NB: All diagnostic and optimizations are applied to your testing environment. In order for your users to benefit from these optimizations, you need to deploy the new model to your production environment.

A - Potential errors page

1 - Potential classification errors report

To help identify some potential errors in your classification, you can click on the Potential errors tab and discover if you have very similar sentences that belong to two different intents. This page will flag any potential errors, such as:

One of the sentences is not in the right Intent
The two intents are too similar and may cause some confusion for your xBot

2 - classification errors correction

For each sentence, you can:

change the associated Intent
or remove (delete) the sentence from the Intent

Once you are happy with your edits, click on the “Save” button on the top right.

B - High Similarity page

1 - Potential high similarity sentences report

In this tab, you can detect in your classification if there are a lot of similarities among sentences classified in the same intent.

If your model is experiencing some performance issues and you corrected all classification errors, then the problem could be due to some overfitting.

The information provided in this tab will help you to improve the understanding capacity of your xBot. With this report, you will be able to remove sentences that are very similar and do not add value to your model.

2 - High similarity sentences adjustments

For each Intent, you can remove (delete) sentences that have a high similarity score. Once you are satisfied with your edits, click on the “Save” button on the top right.

C - Low variability page

1 - Potential low variability report

For an AI model to perform well, each intent needs to include varied sentences so its cluster of knowledge is well defined and delimited.

With this report you can see at a glance the variability level of the intents defined for your xBot.

A horizontal bar with a :

grey color (with a “Variability level” set to “GOOD”) means that the variability is good enough,
green color (with a “Variability level” set to “GREAT”) means that the variability is excellent and no modification is needed,
red color (with a “Variability level” set to “LOW”) means the variability is low. In that case we recommend that you add more diverse sentences to that intent in order to increase the variability level for that intent.

D - Low match page

1 - Low match report

When a sentence matches an intent at a low rate it could mean one of two things:

This is a mistake and you need to reclassify this sentence

This is not a human classification error. In this case the AI model of your xBot will be improved if you add a few more sentences similar to this intent

2 - Sentences with Intent low match adjustments

This report lists the sentences that match an intent with a low rate. For each listed Intent, you can:

“Add” more similar sentences into the same Intent

&/or

“Remove” (delete) the reported sentence from the Intent

Once you are happy with your edits, click on the “Save” button. Do not forget to Re-Train your xBot (via the TRAINING Page of the portal) to notice your edits and improvements.

Documented/reviewed 03/08/2018 17:12:55