Overview

Top 10 Recommendations

The promotions module of SAP Commerce Cloud internally uses a promotion engine that is entrusted to perform evaluations on cart and perform selected actions for eligible carts. The promotion engine is based on Drools engine. The recommendations in this article are aimed at helping you optimize the promotion engine performance through the use of certain tweaks or modifications.

Table of Contents

Tip #1: Gather Statistics on your Promotions

Before performing any optimization, it is important to identify the root cause of the problem at hand. A simple groovy script executed from the hybris Admin Console (hAC) can provide the information necessary to start the analysis. For example:

import de.hybris.platform.util.Config;
import de.hybris.platform.servicelayer.search.FlexibleSearchQuery;
import de.hybris.platform.servicelayer.search.SearchResult;
FlexibleSearchQuery flexibleSearchQuery=null;
String func_length, func_currDate;
if(Config.getParameter("db.driver").contains("Oracle")){//oracle
  println "processing for Oracle"
    func_length="length";
    func_currDate="sysdate";
} else if(Config.getParameter("db.driver").contains("sqlserver")){//azure
	println "processing for Azure"
    func_length="datalength";
    func_currDate="getDate()";
}
//Total number of promotions
query = "select count(*) from {PromotionSourceRule}";
flexibleSearchQuery = new FlexibleSearchQuery(query);
List l = new ArrayList();
l.add(Integer.class);
flexibleSearchQuery.setResultClassList(l);
SearchResult result = spring.getBean("flexibleSearchService").search(flexibleSearchQuery);
println "Total number of promotions: " + result.getResult().get(0);
//Total active promotions
query="select count(*) from {PromotionSourceRule as a join rulestatus as b on {a.status}={b.pk}} where {b.code}='PUBLISHED'";
flexibleSearchQuery = new FlexibleSearchQuery(query);
l = new ArrayList();
l.add(Integer.class);
flexibleSearchQuery.setResultClassList(l);
result = spring.getBean("flexibleSearchService").search(flexibleSearchQuery);
println "Total published promotions: " + result.getResult().get(0);
//Total published promotion but expired (date conditions)
query="select count(*) from {PromotionSourceRule as a join rulestatus as b on {a.status}={b.pk}} " +
            " where {b.code}='PUBLISHED'" +
            " and (" +
            "        ( ({a.endDate} is not null)  and   "+func_currDate+">{endDate}   )" +
            ")";
flexibleSearchQuery = new FlexibleSearchQuery(query);
l = new ArrayList();
l.add(Integer.class);
flexibleSearchQuery.setResultClassList(l);
result = spring.getBean("flexibleSearchService").search(flexibleSearchQuery);
println "Total published promotions but expired: " + result.getResult().get(0);
//Total active promotions per website
query="select {c.identifier},count({c.identifier}) as totalnum from {PromotionSourceRule as a join rulestatus as b on {a.status}={b.pk} join PromotionGroup as c on {a.website}={c.pk}} where {b.code}='PUBLISHED' group by {c.identifier} order by totalnum desc";
flexibleSearchQuery = new FlexibleSearchQuery(query);
l = new ArrayList();
l.add(String.class);
l.add(Integer.class);
flexibleSearchQuery.setResultClassList(l);
 result = spring.getBean("flexibleSearchService").search(flexibleSearchQuery);
println "Total published promotions per website: "
for(int i=0;i<result.getResult().size();i++){
    println result.getResult().get(i);
}
//Total number of active Drools rules
query = "select count(*) from {droolsrule} where {active}=1 and {currentversion}=1";
flexibleSearchQuery = new FlexibleSearchQuery(query);
l = new ArrayList();
l.add(Integer.class);
flexibleSearchQuery.setResultClassList(l);
result = spring.getBean("flexibleSearchService").search(flexibleSearchQuery);
println "Total number of active Drools rules: "
println result.getResult().get(0);
//Total number of redundant Drools rules
query = "select count(r.pk) from ({{ select {pk} as pk, {code} as code, {kieBase} as kieBase, {version} as version from {DroolsRule} }}) r join ({{ select max({version}) as version, {code} as code, {kieBase} as kieBase from {DroolsRule} group by {code}, {kieBase}}}) m on r.code = m.code and r.kieBase = m.kieBase and r.version <> m.version"
flexibleSearchQuery = new FlexibleSearchQuery(query);
l = new ArrayList();
l.add(Integer.class);
flexibleSearchQuery.setResultClassList(l);
result = spring.getBean("flexibleSearchService").search(flexibleSearchQuery);
println "Total number of redundant Drools rules: "
println result.getResult().get(0);
//Average content size of active Drools rules
query="select {b.name},count({b.name}) as totalnum, sum("+func_length+"({a.rulecontent}))/count({b.name}) as avg_size from {droolsrule as a join DroolsKIEBase as b on {a.kiebase}={b.pk}} where {a.active}=1 and {a.currentversion}=1 group by {b.name} order by totalnum  desc";
flexibleSearchQuery = new FlexibleSearchQuery(query);
l = new ArrayList();
l.add(String.class);
l.add(Integer.class);
l.add(Integer.class);
flexibleSearchQuery.setResultClassList(l);
 result = spring.getBean("flexibleSearchService").search(flexibleSearchQuery);
println "Average content size of active Drools rules:";
for(int i=0;i<result.getResult().size();i++){
    println result.getResult().get(i);
}
//Top 10 Drools rules per content size
query="select {a.code},"+func_length+"({a.rulecontent}) as rule_size from {droolsrule as a join DroolsKIEBase as b on {a.kiebase}={b.pk}} where {a.active}=1 and {a.currentversion}=1 and {b.name}='promotions-base' order by rule_size desc";
flexibleSearchQuery = new FlexibleSearchQuery(query);
l = new ArrayList();
l.add(String.class);
l.add(Integer.class);
flexibleSearchQuery.setResultClassList(l);
result = spring.getBean("flexibleSearchService").search(flexibleSearchQuery);
println "Top 10 Drools rules per content size: "
for(int i=0;i<(result.getResult().size()>=10?10:result.getResult().size());i++){
    println result.getResult().get(i);
}

Now, analyze the result. Here are some indicators to look for:

1. Is the number of active promotions in line with the expected published promotions per website?
2. Is the Total published promotions but expired greater than zero?
3. Is the Total number of redundant Drools rules greater than zero?
4. Look at the structure of the promotions which have their Drools rules content size in Top 10. Why do they have such large Drools rule content size? Are there lots of product, users or any similar entries added directly to the promotion, instead of adding them as categories or user groups?

Tip #2: Create a RETE Network Visualization of Your Promotions

Drools rule engine internally creates a network of conditions, which are evaluated when promotion calculations are triggered. This is called RETE network and there is an Eclipse plugin that can be used to create a visual structure of the conditions for a Rule. Drools rules can be analyzed using this plugin to look for complex evaluations and paths.

Tip #3: Ensure You're on the Latest SAP Commerce Core Patch

Patches are released each month for SAP Commerce, which have included a lot of promotion engine optimizations. If you are not on the latest patch for your version of SAP Commerce, there are chances that you are missing out on some of these easy wins.

A few of the major ones are mentioned below.

RRD Objects

This enhancement replaces the currently used RRD objects to track rule and rule group executions and replaces them with the usage of a newly introduced RuleAndRuleGroupExecutionTracker implementation which significantly decreases memory consumption.
This feature is disabled by default and can be enabled by setting the property ruleengineservices.use.deprecated.rrd.objects=false in your local.properties.

Rule Aware Objects (RAO) Model Simplifications

The RAO model has been greatly simplified, leading to less RAO objects included in a rule evaluation, thus improvement in both memory consumption as well as CPU usage.

ProductRAO, CategoryRAO and ProductConsumedRAO are no longer used. Instead all their attributes have been moved (and renamed) to OrderEntryRAO.

Drools Syntax Generation Improvements

Drools syntax is now generated to a more optimized structure, for faster evaluation. 

The code generated previously:

$var4 := ProductRAO(code in ("1382080", "1382080_"))

$v4 := ProductRAO(code == “1382080” || code == “1382080_“)

Tip #4: Product Promotions by Discount Row

This is a classical design mistake that may lead to an excess number of promotions in the system.

When a cart calculation triggers, it fires the drools engine that internally checks for applicable promotions. This evaluation happens in drools engine based on the various RAOs. Once a matching promotion is found, that has the top most priority, the corresponding action gets applied. These actions are typically a certain discount on the cart. Discounts can be a fixed amount or a percentage amount. Moreover, these can be on the total cart or on specific cart entries.

These discounts are attached to the cart/cart entries as discount rows.

For a simpler kind of promotion, where the condition only contains product, and the action contains a fixed / percentage discount, the same result can rather be achieved instead by directly creating discount rows against the products. What the business team called “product promotions” may be better implemented in SAP Commerce with discounts.

 Discount row is more performant, is simpler to import and works natively.

 Discount row is less flexible for complex conditions, the cart calculation logic may be different.

Tip #5: Use Virtual Categories and User Groups

Backoffice does not restrict the business users to associate products, users and other similar entries to a condition. Business Users may at times end up associating hundreds of such entries to the promotions, leading to an increased memory and CPU utilization. Over a period, the evaluation and publication time may increase substantially.

A better approach is to merge such large numbers of entries to their corresponding groups, and associate these groups to the promotion.

1. Create categories with these products/variants  (if it does not already exist). You may call them "Virtual categories", as these may just be used for promotion configuration.
2. Create user groups with users (if it does not already exist), and associate that in the promotion

Tip #6: Multi Code Coupon vs Single Code Coupon

SAP Commerce supports two types of coupons:

• Single code coupon: Typically shared with all customers or a group of customers and therefore can be redeemed multiple times.
• Multi code coupon: Generates multiple codes dynamically and each code is typically shared with one customer. These codes are unique and are for one time use only.

If multiple Single code coupons are created instead of Instead of creating Multi code coupons, you may end up with a very high number of coupons, thereby putting higher load on the promotion engine. Ensure you are using the appropriate type of coupon.

Tip #7: Clean-up Promotion with Maintenance Jobs

Over a period, promotion engine tends to accumulate a lot of unused drools rules and promotion source rules. These rules are typically not of much use functionally and therefore it is recommended that proper cleanups are done periodically.

Two types of cleanups are mainly needed.

Cleanup of Inactive Drools Rules: 

SAP Commerce comes with a maintenance job - DroolsRulesMaintenanceCleanupJob. This job improves Rule Engine speed and performance by deleting all inactive versions of every Drools rule in your system, leaving only the active drools rules versions. 

select count(*) from ({{select r.pk from ({{ select {pk} as pk, {code} as code, {kieBase} as kieBase, {version} as version from {DroolsRule} }}) r join ({{ select max({version}) as version, {code} as code, {kieBase} as kieBase from {DroolsRule} group by {code}, {kieBase}}}) m on r.code = m.code and r.kieBase = m.kieBase and r.version <> m.version}})

Refer Maintenance Cron Job for Drools Rules for the steps to enable this job.

Cleanup of Expired Promotion Source Rules:

A lot of promotions are created with end dates. These promotions cease to apply on the cart after the end date. This evaluation of end date constraint takes place in drools engine and is therefore executed every time a cart evaluation takes place. In other words, these promotions do not get removed automatically from Drools engine memory after they have expired.

Over a period, these promotions end up consuming a lot of CPU and memory without adding any functional value. Therefore, it is advised that these promotions are unpublished immediately when they have crossed their end date. Doing this manually can be a pain and the way to automate this using a cleanup job is quite simple. 

CleanupExpiredRulesStrategy.java

package <packagename>.ruleengine.cronjob;
import de.hybris.platform.cronjob.model.CronJobModel;
import de.hybris.platform.jobs.maintenance.MaintenanceCleanupStrategy;
import de.hybris.platform.ruleengineservices.jobs.RuleEngineCronJobLauncher;
import de.hybris.platform.ruleengineservices.model.RuleEngineCronJobModel;
import de.hybris.platform.ruleengineservices.model.SourceRuleModel;
import de.hybris.platform.servicelayer.internal.model.MaintenanceCleanupJobModel;
import de.hybris.platform.servicelayer.search.FlexibleSearchQuery;
import de.hybris.platform.servicelayer.session.SessionService;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Required;
import java.util.List;
public class cleanupExpiredRulesStrategy implements MaintenanceCleanupStrategy<SourceRuleModel, CronJobModel> {
    private static final String EXPIRED_RULES = "select {ar.pk} from {AbstractRule as ar}, {RuleStatus as rs} where {ar.enddate} < getDate() and {ar.status} = {rs.pk} and {rs.code} = 'PUBLISHED'";
    private RuleEngineCronJobLauncher ruleEngineCronJobLauncher;
    private SessionService sessionService;
    private final static Logger LOG = LoggerFactory.getLogger(cleanupExpiredRulesStrategy.class.getName());
    public cleanupExpiredRulesStrategycleanupExpiredRulesStrategy() {
    }
    public FlexibleSearchQuery createFetchQuery(CronJobModel cronJob) {
        if (!(cronJob.getJob() instanceof MaintenanceCleanupJobModel)) {
            throw new IllegalStateException("The job is not a MaintenanceCleanupJob");
        } else {
            return new FlexibleSearchQuery(EXPIRED_RULES);
        }
    }
    @Override
    public void process(List<SourceRuleModel> elements) {
        if (elements.size()>0){
            RuleEngineCronJobModel ruleEngineCronJobModel = getRuleEngineCronJobLauncher().triggerUndeployRules(elements, "promotions-module");
            LOG.info("Cron job created with code" + ruleEngineCronJobModel.getCode());
        }
        LOG.info("No expired promotion found for undeploy");
    }
    protected SessionService getSessionService() {
        return this.sessionService;
    }
    @Required
    public void setSessionService(SessionService sessionService) {
        this.sessionService = sessionService;
    }
    public RuleEngineCronJobLauncher getRuleEngineCronJobLauncher() {
        return ruleEngineCronJobLauncher;
    }
    public void setRuleEngineCronJobLauncher(RuleEngineCronJobLauncher ruleEngineCronJobLauncher) {
        this.ruleEngineCronJobLauncher = ruleEngineCronJobLauncher;
    }
}

In the above code, getDate() is used, which works on Microsoft Azure SQL but may need to be replaced with an equivalent code to get the current date/time.

<extension>-spring.xml

<bean id="expiredRulesMaintenanceCleanupJob" parent="abstractGenericMaintenanceJobPerformable" >
   <property name="maintenanceCleanupStrategy" ref="cleanupExpiredRulesStrategy"/>
</bean>
<alias alias="cleanupExpiredRulesStrategy" name="defaultCleanupExpiredRulesStrategy" />
<bean id="defaultCleanupExpiredRulesStrategy" class="<packagename>.ruleengine.cronjob.CleanupExpiredRulesStrategy">
   <property name="ruleEngineCronJobLauncher" ref="ruleEngineCronJobLauncher"/>
   <property name="sessionService" ref="sessionService"/>
</bean>

Impex

INSERT_UPDATE MaintenanceCleanupJob;code[unique=true];springId[unique=true];active[default=true]
;expiredRulesMaintenanceCleanupPerformable;expiredRulesMaintenanceCleanupJob;true
INSERT_UPDATE CronJob;code[unique=true];job(code);sessionLanguage(isoCode)[default=en]
;expiredRulesMaintenanceCleanupJob;expiredRulesMaintenanceCleanupPerformable
INSERT_UPDATE Trigger;cronJob(code)[unique=true];second;minute;hour;day;month;year;relative;active;maxAcceptableDelay
;expiredRulesMaintenanceCleanupJob;0;0;3;-1;-1;-1;false;true;-1

extensioninfo.xml

<requires-extension name="ruleengineservices"/>

Tip #8: Optimize the KIE Module

Create One KIE module per Website

If you have a setup with multiple sites and the cart is not shared between these different sites, it is advised that separate KIE Modules are created for each site. This will restrict the promotion calculation to the number of promotions associated to the individual websites only, instead of looking at all the promotions available across all the sites.

As mentioned, it works only if you do not mix products from different sites into the same cart.

Remove Preview KIE Modules

In cases where you are not using preview KIE modules, it is advised that you remove them so that it frees the memory used by them. Keep only those KIE modules which are being used.

Tip #9: Reduce Number of Promotion Calculation

Multiple Calls to Promotion Engine in Single Transactions

Are you aware of which activities in your project are triggering a promotion calculation? If not, this can be a good starting point and can lead to quick improvements. Developers tend to unknowingly add multiple cart recalculation calls within a single browser request and this leads to unnecessary load on the system. In these cases, it is usually possible with code optimization to achieve the same result by a single call to promotion engine.

For example, look at the methods invoked when a product is added to the cart. How many times does it call the DefaultPromotionEngineService.evaluate() method? If it is more than once, then possibly, the code can be optimized.

Promotion Engine Calls When Cart Need Not be Re-evaluated

Consider another scenario, where, in the checkout journey the end user is modifying the address parameters. In these cases, the cart total isn't expected to change, therefore a call to promotion engine is not expected. Do these kinds of scenarios invoke promotion engine method? If yes, then this should be reviewed and optimized.

Limit the Number of Cart Items

What is the maximum number of items, your customers are allowed to add into a single cart or wish list? What happens if a few of your customers or bots are adding hundreds of items into cart or wish list? Can you restrict the cart or size to a number that is sufficient enough for your customers? Carts are persisted in application and are retrieved every time the user comes back to a page. If a customer keeps adding items to the cart but does not checkout, this will put undue burden on the system. In such cases, it is advised that you restrict the cart size.

The maximum allowed size can vary depending on the business requirements and the type of products. For groceries and certain B2B sites, this may be a high, but for electronics and apparel, this may be low. You can even look at the historical orders to identify the maximum order size.

Tip#10: Garbage Collection Tuning

Drools engine evaluations happen in-memory and therefore the engine consumes a significant amount of Heap and Metaspace. Drool engine performs just-in-time compilation and therefore, the Metaspace utilization can also increase over a period. When there are high number of promotions, it is advised that the memory is monitored for any full GC in heap and Metaspace, and necessary corrective measures are taken if needed.

Remember that Drools engine, being in-memory, is usually one of the biggest contributor to the Heap and Metaspace, and keeping it in check is goes a long way in keeping the application memory healthy. See Using Dynatrace to Manage and Optimize the Performance of your SAP Commerce Cloud Solution for more.

Conclusion

The Promotion engine is an extremely powerful and agile module that supports creation of various kinds of promotions in-memory. A well managed promotion engine contributes immensely to the overall performance and stability of SAP Commerce application. This article covered some of the fundamental mistakes that are possible in any implementation and provides corrective and preventive measures. Some of these may or may not apply for you depending on the way you have configured promotions in your application.

In general, all the above recommendations to improve promotion engine performance can be grouped under one or more of these high level recommendations:

1. Reduce the number of Promotions and resulting Drools rules
2. Reduce the size of Drools Rules
   1. Reduce the number of conditions
   2. Reduce the number of items in each conditions
3. Reduce the number of hits to promotion engine

In your implementation, you can look for any other optimisation around these areas. The approach to run a performance test and optimize your solution is explained in the series of articles under Managing Performance in an SAP Commerce Cloud Project.