SAP Service Cloud Ticket Intelligence Machine Learning
This article describes the general features and functions of the Machine Learning scenario "Ticket Intelligence" as well as the necessary steps to implement the same.
Table of Contents
- Description of Features and Functions of Ticket Intelligence
- Prerequisites for the Implementation
Description of Features and Functions of Ticket Intelligence
Many businesses face the following problems related to ticket management:
- Service agents spend a large amount of time manually classifying tickets.
- No reference to best practices/accuracy based on historical data sets exists.
- Large customer service request queues exist.
The Ticket Intelligence feature automatically categorizes incoming customer inquiries to deliver best-in-class customer service.
Automatic categorization helps to:
- Improve service response times.
- Improve efficiency of service teams.
This is done by:
- Increasing agents productivity
- Better prioritization of incoming tickets
- Automatic classification based on model accuracy
But how does it work?
Historical data taken from existing tickets in SAP Service Cloud is used to train the Machine Learning model. As a result, you get a customer-specific predictive model which provides an automatic categorization of tickets.
Detailed Features and Functions
End User Experience
This category is one of the most important details contained in the ticket. Based on the right categorization of a ticket, SAP Service Cloud offers multiple standard features to streamline the ticket handling.
The main features are:
- Team Assignment
- Knowledge Base Article Determination
The Ticket Intelligence functionality automatically categorizes incoming tickets based on the subject and message body of an incoming e-mail (subject and description, if a ticket is manually created).
It is not mandatory to have both, subject and description, to get a prediction. However, to get an accurate description, both should be filled.
The main use case is when customers send e-mails to your company's support e-mail address and a ticket is automatically created in SAP Cloud for Customer. These tickets are categorized automatically.
When a ticket is submitted, the content of the ticket (subject and body) are reviewed and compared to historically submitted tickets that were used to train the model. If the confidence level is above the threshold set in the configuration of the model, then the predicted category or categories will be copied into the appropriate field(s) based on your configuration. All actions dependent on the category (workflow rules, routing and more) are then triggered.
If the confidence level is below the threshold, it will serve as a proposal and will not be copied.
Currently, the prediction is executed on every "save" of the ticket. However, if the standard category field is already filled, it will not be overwritten by the category which was determined on the "save". This new category is only visible in the "Solution Center". This means, if a ticket is updated manually or through e-mail, the standard category field will not be changed if it contains a value (filled in manually or automatically through the prediction). Also, manually created tickets are categorized automatically once they are saved.
In the above case, categories must not be mandatory in the Quick Create screen of the ticket.
Ticket Intelligence will be able to automatically predict:
- Service Category
- Incident Category
- Cause Category
- Object Category
As the functionality of category fields in the ticket are directly related to the setup of the category catalog, it is important to understand the impact of your system's setup on the prediction capabilities and possible supported use cases.
Currently, there is no authorization/user-specific configuration possible. Once Ticket Intelligence is switched on, it applies to all users who work with tickets and to all tickets which are created manually or automatically.
You can find a list of all supported languages here: Supported Languages
Catalog Structure and Impact on Ticket Intelligence
In order to understand the impact of the catalog structure, it is necessary to understand the basic catalog settings and how they relate to the category fields in the ticket.
The category catalog allows you to define the values that can be selected when filling the category fields in the ticket.
Each entry in the catalog is assigned to a category type which directly relates to the category fields in the ticket. A hierarchy can be defined in the catalog.
Therefore, an entry on a lower level in the catalog can only be chosen in the corresponding category field in the ticket, if in the category defined one level higher, fits the selection.
Based on the catalog structure, the values shown in the ticket for the lower level category fields are filtered out, and vice versa. If you enter a lower level category, automatically the higher level categories are derived.
In general, the catalog can be a flat structure:
- with no dependencies between the categories
- with a hierarchical structure with dependencies
- with a mixture of both
For example, the catalog can have two hierarchy levels but on the second level, different category types can be assigned.
To set up Ticket Intelligence, your catalog structure needs to be very clear.
The above information is required once you start to train your model as described later in the implementation chapter.
Prerequisites for the Implementation
To use Ticket Intelligence, you need to meet certain prerequisites in regards to your historical data. To find out if you meet the main criteria, run a check report in the predictive services work center view.
Once the report is executed, you can view the results and get a first indication on whether your data is sufficient or not.
Below is a detailed explanation of the relevant prerequisites. Note: Some of them are not checked in the report.
You must have an SAP Cloud for Customer Enterprise License.
The more historical data available, the better it is. To create the prediction model, we use the last 12 months of tickets. A minimum of 1000 tickets is required to create the prediction model. This number is heavily dependent on the factors that follow next.
The data history, from the change log on a ticket, is not required for the prediction of the category. In the future, other fields are planned to be predicted (for example, priority) in which case the historical data will become important.
The Machine Learning for tickets is based on the subject and description. Therefore, all tickets used for the model training should have the subject, the description, and the category value properly filled. The more structured the data in these fields are, the more accurate the prediction. For example, if the e-mail generated ticket is based on a web form template, the model accuracy will be higher compared to a similar scenario where the ticket generated is based on any free texts written by the sender.
Ticket categories need to correspond to the categories in the active catalog. Please make sure that all categories, which are in the tickets used for the model training, are part of the active category catalog. Otherwise, you will need a mapping to handle this and support from your implementation contact.
The data balance plays a significant role. If for example, 90% of the tickets have the same category, then Machine Learning might not make sense since it always determines the same category. However, this could still be beneficial since the user does not need to make this first decision. The more even the category assignments are, the greater the benefit of using Ticket Intelligence is. For example, let's say you have category A, B, C, D, and 10 000 tickets. An equal distribution of the categories is better than a distribution where one category dominates the others.
If you have custom development related to the ticket category, you need to check if it remains consistent once the Ticket Intelligence is switched on.
In general, the implementation can be divided into the following phases:
Check customer data with regards to the prerequisites listed above.
Activate and Configure Ticket Intelligence
Use the Existing Test Tenant
With the release of 19.02, a test process has been built in to the production tenant which allows for testing the results of the Machine Learning training in the production environment. The benefits here include the most up to date access of all tickets submitted, in real time. Also, this creates simplification for the training process as all data is already in production and will be trained as such. When training the model and testing the model in production, you have the ability to keep the model "disabled" until you are confident with the results. Only once you are confident with the results and once you enable the model, only then will it begin classifying tickets automatically.
Your users can use the below template to request the feature:
Dear SAP Support,
I am contacting you because we want to activate Machine Learning.
Could you please activate ML in the following tenant for the following language?
https://myXXXXXX.crm.ondemand.com/ (Production Tenant)
Language - "your target language" (for example, English)
Please activate ML in the following scenarios (Place an X next to the scenarios):
[ ] Deal Intelligence
[ ] Product Recommendation
[ ] Lead Intelligence
[ ] Account Intelligence
[x ] Ticket Intelligence – Categorization
[ ] Similar Tickets (Beta)
Once the feature is activated, you will see it in the predictive services work center. For your first step, run the check report as described in the previous section of this article.
Train Machine Learning Model
If you decide to follow option 1 or 2, as explained above regarding the tenant decision, you can request and train your model without the help of SAP.
To train your custom model, go to the administration work center and choose the view "Prediction Services" and then "Configure". The administrator user role needs to have the view assigned for it to be visible in the administrator work center.
The creation of the model is done in the following simple steps:
- Choose Ticket Intelligence.
- Add a new model. When defining the new model, it is crucial that you have a profound understanding of your catalog structure when requesting the model that best fits your catalog.
You can analyze the below examples of catalog structure to decide which model suits your situation better.
- Case 1 - The Catalog has a Flat Structure
You have no hierarchy but have independent category definitions for the four supported category fields. Select the categories which should be predicted and choose all or any combination of the four.
- Case 2 - The Catalog has a Hierarchical Structure
Each category describes one level of the hierarchy. All hierarchy levels always have only one category type assigned. Therefore, each single hierarchy level has one specific category type. This means that you can choose the lowest level category, the prediction would set this category, and the dependent higher levels would be derived. With this catalog structure it makes no sense to flag multiple categories when requesting the model.
The picture below represents a good example of this type of catalog structure.
Case 3 - The Catalog has a Hierarchical Structure that does not Follow the Same Patterns in all Hierarchy Levels or has Multiple Categories Assigned on One Sub-Level
In this case, you need to evaluate in detail the features of your catalog structure.
One example is when your hierarchy has two levels and this second level has three category types assigned (see picture below).
Your catalog is then a "mixture" between Case 1 and Case 2. Alternatively, if you flag an incident, cause or object category when requesting the model, you will only get a prediction for the flagged category and for the higher level of the service category. If instead, you flag more than one of the second level categories, the prediction would deliver the values. However, depending on the model accuracy and your threshold settings, you could get inconsistent values in the category fields with regards to the catalog definition.
Currently, there is no plausibility check performed in this scenario to validate if the categories belong to a first level category. This gap will be closed in a future release.
Another case where you need to carefully evaluate your options is where some hierarchy levels have more sub-levels than others.
Here, you can either choose the cause or incident category once you have requested your model. Since the cause category is not always available along the hierarchy, it only makes sense to choose the incident category. In this case, you will always get predictions along with the higher level derived categories.
If you choose the cause category, you get only predictions in the branch of your hierarchy where the cause category is assigned. Therefore, only part of your catalog can be covered.
The catalog allows for many additional combinations that could make sense for your business. The scope of this article is to focus on the most frequent ones. However, for all other setups, you need to carefully check if you can use the Machine Learning for ticket categorization.
Next, you need to:
- Mark your new model and choose "Train".
- As a result, the historical ticket data is sent to the Leonardo Machine Learning component and your customer specific model is trained/created based on your historical data.
- Wait until the status is "Finished". Note: This may take some time.
Validate Model Training Results
As a result of the model training, the SAP Leonardo component returns an accuracy score which tells you how good your model is.
Any accuracy score above 60 is considered as good and the higher the score, the better.
During validation, you can see the Model Performance Reports. By looking at the model performance reports, you can see where the predictions are accurate, and where they might be weaker. By understanding this, you will be able to make a decision if the model is working for the data supplied. These results will also give you information to better understand where there are discrepancies or deficiencies in the input data, and if there are opportunities to update the source data and retrain.
Once the validation is done and you are satisfied with the accuracy score, you need to activate the model.
Once the model is active, you can test new tickets through the Test Console. In the Test Console, you can either test sample inputs one at a time to determine what the predicted categories will be or you can upload a set of test data (recommended) to determine how the system would classify these incoming tickets. In the test console, you can see the predicted category and the confidence level.
Once testing is complete, you can enable the feature for all users.
This article introduced you to the Ticket Intelligence feature in SAP Service Cloud and its implementation guidelines. Now, you know how to empower your service teams and reduce their response time by leveraging accurate and automatic classification of tickets based on predictive models.
Through the power of Machine Learning, you can now ensure a world-class experience for your customers and an effortless resolution of all their inquiries.