Set Up Profiling & Match-Merge Logic for SAP Marketing Cloud
24 min read
Understanding customers is important for marketing organizations. SAP Marketing Cloud lets you collect customer information from different sources and consolidate it into a single customer contact profile.
Table of Contents
SAP Marketing Cloud Profiling
This article will teach you how to build a single view of a customer (known as the Best Record on SAP Marketing Cloud). The contact's Best Record represents your customers most valuable and trusted set of contact information. It is usually enriched or created from different data sources, known as Origins. From a technical standpoint, logic is used to consolidate and evaluate master data, from different data sources, into a single source of truth for the contact master data. The Best Record is built when a contact record is imported from an external system or a scheduled application job is executed, on SAP Marketing Cloud. It is important that you understand what happens when a contact record is imported and which configurations are involved.
Match & Merge
Before importing contact data to SAP Marketing Cloud (especially from different sources), it is crucial to understand the Match & Merge procedure and the ID Origin configuration on SAP Marketing Cloud. This configuration is an essential part of any SAP Marketing Cloud configuration since it defines the build of the Best Record. The Best Record defines the collection of the best contact information imported and available on SAP Marketing Cloud. Although the processing logic and configurations are simple, they can become a source of confusion during a project. To avoid any confusion, first, you need to learn the difference between contact match and contact merge. The process is called Match & Merge logic, but it is a process with two different connected processes. Next, you’ll need to understand the difference between the Match & Merge process during data import and a scheduled marketing application job. Lastly, you need to know the implications of changing the configurations that are involved and how a Match & Merge logic is executed.
Match & Merge at Import
A match process is executed when a contact is imported to SAP Marketing Cloud. Depending on the patch executed in the matching procedure, a contact can be selected and flagged for the merge process. The merge process is a routine job which processes contact records that are selected to merge independently from the match process.
|SAP Customer Experience solutions Cloud||SAP Marketing Cloud||SAP Sales Cloud||SAP Commerce Cloud|
Match & Merge Based on a Unique ID
The most simple case is a Match & Merge based on a "For Identification" Source Contact ID or Additional ID. Let's assume the contacts are loaded sequentially and the SAP Sales Cloud contact is loaded first. Assuming that there is no existing contact record to trigger a successful match, the contact record imported from SAP Sales Cloud, creates a new contact Best Record on SAP Marketing Cloud.
When the SAP Commerce Cloud contact record is imported, the Match & Merge logic is executed as follows:
- Find contact with merging Origin ID: No (since the ID and ID Origin is different).
- Find contact with matching "For Identification" IDs: Multiple (both contact can match on the "For Identification" e-mail address, see ID Origin Configuration).
- Create new contact and set merge flag: 2 contact records with the same "For Identification" IDs should exist on the SAP Marketing Cloud, with the imported contact flagged for merge (as indicated on the contact record).
- Find contacts with matching "For Identification" IDs: Yes (both contacts share the same "For Identification" e-mail address).
- Check for One Per Contact conflict: No (There is no conflict with existing IDs. This check validates if there can be multiple occurrences of a contact origin data for a defined origin (Can a contact create only one SAP Commerce Cloud contact account or multiple?).).
- Merge contacts and remove merge flag: both contacts are merged into one Best Record. Only the merged information from both data sources is displayed in the contact profile.
After the merge, SAP Marketing Cloud keeps the original data in the system and does not delete the imported data. Therefore, when the source data is updated and the update is replicated to SAP Marketing Cloud, only the contact origin data is updated and the Best Record is “re-built" with the updated data.
The contact Best Record consolidates data from one or multiple contact records by priority. The higher the priority, the more trusted the source of data is. In our example, the contact record from SAP Sales Cloud has a higher priority. As a result, the contact Best Record build will consider SAP Sales Cloud as the most trusted data source.
A match on a "For Identification" source contact ID is the most reliable match criteria and should be used whenever possible. When no matching on "For Identification" source contact IDs is not possible, "For Identification" additional IDs are also a reliable matching criteria.
This is a sample configuration and it might vary from the actual system configuration. For example, a phone number (landline number) is considered a shareable ID for most use cases. However, we’ve changed this configuration for illustration. Keep in mind that changing the configuration on Contact Origin should not be taken lightly since this can have a huge impact on the Match & Merge process during import. A description of the configuration settings can be found in the Origins of Contact IDs documentation.
No Match & Merge Based on a Unique ID
In our second example, we’ve only changed one configuration setting in the ID Origin Configuration. We changed the setting EMAIL Additional ID to Not-"For Identification".
When importing the same contacts, as in our first example, the Match & Merge logic is executed as follows:
- Find contact with merging Origin ID: No (since the ID and ID Origin is different).
- Find contact with matching "For Identification" IDs: None (E-mail is now shareable , meaning multiple contact records on SAP Marketing Cloud could use the same Email address, and there is no other "For Identification" ID that exists on both contact record, see ID Origin Configuration).
- Find contact with matching shareable ID plus name: None. In our example, there is no match based on a non-identifying ID nor name since the name of the contact is different.
- Create new contact. With no matching "For Identification" and shareable IDs with name, two contact Best Records are created on SAP Marketing Cloud.
- Merge of contacts is not executed on either contacts since no merge flag is set. Changing only a small setting, in this case, completely changes the result of the contact Best Record. With the contact data available on both contact records, the system can't Match & Merge the contact into one contact Best Record.
Carefully review the Origin of Contact configuration not only from technical perspective but also from the functional and business perspective. Since the Match & Merge configuration can largely impact the outcome of the process, functional and business resources should be included in validating the configuration, the process and the expected outcome.
Match & Merge Based on a New Unique ID
Since we can't merge the contact record originating from SAP Sales Cloud and SAP Commerce Cloud with the existing data, let's see what happens when some information is changed or added. In this next example, we made the assumption that the contact records from example two is already in the system. Now, the SAP Sales Cloud record is updated with a phone number, matching the name from SAP Commerce Cloud.
When importing the updated contact, as in example two, the Match & Merge logic is executed as follows:
- Find contact with merging Origin ID: Yes (the contact already exists with ID Origin SALES_CLOUD and ID 12345).
- Find contact with matching "For Identification" IDs: Yes – another record with the same unique ID is found on SAP Marketing Cloud).
- Update existing contact and set merge flag.
- Process contacts flagged for merge.
- Find contacts with matching "For Identification" IDs: Yes – another contact record with the same "For Identification" ID (phone number) exists in the system.
- Check for One per Contact conflict: No.
- Merge contacts and remove merge flag: both contacts are merged and into one Best Record having the best set of information for both records.
Review the configuration of every ID on SAP Marketing Cloud. The configuration of a specific ID, such as E-Mail or Phone, can vary from the default configurations. Make sure all Contact Origins are set up properly to ensure correct Match & Merge processing.
In our examples, we have illustrated the one specific ID Origin configuration. The ID Origin configuration can be adapted and needs to be reviewed to fit your use cases and expectations on building a Best Record on SAP Marketing Cloud. The contact merge does not occur immediately at import. When there is a possible merge, the match process sets the selected for merge flag. This flag is visible on the contact record and indicates that the contact will be processed with the next scheduled run of the merge process.
SAP Marketing Cloud can update a contact record based on a "For Identification" ID and name. The attribute name is the full name which exist in the contact record on SAP Marketing Cloud.
Scheduled Match & Merge Process
While the Match & Merge process at import is executed during data imports, the scheduled Match & Merge process can be planned and executed from a marketing application job template.
The scheduled Match & Merge procedure follows a different Match & Merge logic than the one executed at contact import. This needs to be considered when running the Match and Merge contacts application job. The scheduled report is based on HANA Enterprise Information Management (EIM) capabilities and searches the system for possible duplicate contacts. Depending on the configuration, the system can merge high-confidence matches or keeps a list of identified contacts for review.
When you run the application job, you can view the Job Log to review the Match Groups and associated contacts in the Manage Contacts application.
Please refer also to the official documentation on the SAP Help Portal. Contacts: Match and Merge Contacts with EIM
In this example, high-confidence merge checkbox is not enabled. Instead, we are reviewing the contacts in the Manage Contacts application. When the merge high-confidence contact checkbox is enabled, Match Groups flagged as high-confidence will be merged automatically. Simply select a contact from the For Review list and select Review. The contacts of the Match Groups are displayed for review. You can then decide if this contact should be merged or not.
The Contacts: Match and Merge Contacts application job is to be used with caution. Keep in mind that this application job is independent to the Match & Merge logic at contact import and might lead to different results.
For every data source, we recommend you use a distinctive Origin on SAP Marketing Cloud. The Origin of data identifies the data source from where data originates. For example, when defining an SAP ERP as an Origin of data, the Origin SAP_ERP_CONTACT, the Origin Name in combination with the SAP ERP contact ID, is used to identify the contact record to SAP Marketing Cloud.
The configuration is simple and can be done in seconds. However, the decision to change a setting in the Origin of Contact configuration should not be made hastily. Changes can have major impact on the Best Record built on SAP Marketing Cloud.
We've imported two contact records with different data sources holding different sets of data. Based on the Origin of Contact configuration below, the contacts are merged on the e-mail address, which is configured as "For Identification" in our system. The full name exists in both contact records, but the Best Record pulls the full name from the most trusted Origin, which in our case, is SAP ERP. Since there is no address data available from the SAP ERP Origin, the Best Record pulls the address information from the next best Origin. The same goes for personal data. The Origin SAP_ODATA_IMPORT allows multiple contact records on one Best Record, but are defined as Technical. The technical setting defines if an Origin is displayed in the Origin Data of the Contact Profile.
Contact Records - Inspect Contact
Contact Best Record - Contact Profile
Origin of Contact Configuration
The Origin of Contact IDs configuration is one of the configuration steps for the item Contacts and Profiles.This available through the Manage your Solution App on SAP Marketing Cloud.
|Origin of Contact||SAP_ERP_CONTACT||SAP_HYBRIS CONSUMER||SAP_ODATA_IMPORT|
|Description||SAP ERP - Corporate Contact||SAP ERP - Corporate Contact||SAP OData Import Contact|
|One Per Contact||True||True||False||False|
This configuration is an example and might vary from the actual system configuration. A description of the configuration settings can be found in the Origins of Contact IDs documentation.
Use the Inspect Contact Application to view all Contact Origins of a contact's Best Record. You can also view the contact from the contact profile, but technical Contact Origins are not shown in the Origin Data tab.
Monitor and Review the Imported Contact Data
You can monitor and review the imported data in the SAP Marketing from the Marketing Data Stewardship application. The Marketing Data Stewardship provides an overview of relevant information, applications, and resources to monitor, maintain, and review contact data.
Data Load Monitor Application is another useful application for monitoring and reviewing data loads. This application gives you a single point of entry to monitor all import messages and status. It provides filters to quickly adapt the view to get a good overview on all import messages.
This application collects and organizes errors, warnings, and information messages for imported data to SAP Marketing Cloud, which can help identify most common import issues on your system and prioritize activities to improve data loads.
You can open and review the individual messages for each entry for a preview of the imported data set.
Data Aging and Contact Data Cleansing
After setting up integrations with SAP Marketing Cloud, configuring your requirements, and running thousands of campaigns, and so on, you might realize that some of your data is not worth keeping.
SAP Marketing Cloud does not automatically cleanse or delete data, but it provides you with the tools to plan and configure your data aging and cleansing strategy. A sales order, dating back ten years, might no longer be relevant for some people. For others, it might be necessary to keep that data. To manage your data on SAP Marketing Cloud, several Application Job templates are provided. They can be configured and scheduled to manage your data.
Extensions & Customization
In the Contact Match & Merge Process import diagram, extension points are marked with a rhombus. For any of the extension points, you can modify the standard procedure, creating custom logic in the Custom Fields and logic application. Enhancements are implemented with available business add-ins (BAdIs) on SAP Marketing Cloud. Each BAdI provides sample code and documentation.
For example, you can populate a field when empty or add additional logic and validation in the BAdI. Below, is an example of a code snippet from the BAdI Review Imported Interaction Contact Data. Here, the first and last names are concatenated when the contact record is created, and the contact type condition is true.
The implementation of custom logic to customize standard procedures on SAP Marketing Cloud needs to be carefully evaluated and tested. You can find the available BAdIs in the following SAP Help documentation: Custom Logic
In some cases, SAP Marketing Cloud can create so-called "Container Contacts" to hold interaction information that can't be assigned to a specific contact. Container contact is created when, for example, an interaction is imported with a shareable (non-identify-able) id where SAP Marketing Cloud can't clearly assign the interaction to one contact. A container contact is created to retain the imported interaction.
Please refer to the SAP Help documentation for additional information: Container Contacts
Q&A on Recommendations and Best Practices
|When should an ID Origin be created?||Best Practice is to create one ID Origin per data source.|
|What does "For Identification" mean in the contact of the ID configuration setting?||Shareable defines if an ID of an Origin can be shared among different contacts.|
|What is defined as a reliable match (hard match)?||Match of multiple Contact Origin records based on a "For Identification" ID.|
|What does the configuration setting “One per Contact” define?||This setting defines if a Best Record can have multiple data records of one Origin.|
|How do you define the priority of a data source?||00-99. The lower the number, the higher the priority of the data source (00 is default and counts as 99).|
|In what app can you get an overview of all Contact Origin data?||Browse Contact Origin Data.|
|What app enables you to check individual contact and their Contact Origin data?||Inspect Contact (also the Contact Profile).|
|Can one data Origin be use for multiple data sources?||No. The overwrite each other on SAP Marketing Cloud (different source – different Origin).|
|Can the same Origin be used for different types of data loads?||Yes. When the data is from the same source, the same Origin is used (same source – same Origin).|
|Is the middleware considered as a data source?||No. The middleware is not a source of data and therefore does not have an Origin.|
|One source has multiple entities, is one or multiple Origins created?||Different Entities count as different sources of data (different source (entities) – different Origins).|
|What is the most reliable matching criteria?||The ID of a "For Identification" Origin.|
|What happens when there is no match by a Origin ID?||The system looks for contacts with same "For Identification" IDs such as "For Identification" mobile number.|
|Can contacts be matched based on shareable ID?||Yes. Shareable ID with additional attributes (Name) can result in a match.|
|Can the import and matching logic be modified?||Yes. At certain extension points, BAdIs (Business Add-Ins) are available to modify the standard import procedure.|
|What is an additional ID?||An additional ID is an ID which is part of the contact data record such as e-mail, phone number, or Facebook ID.|
|Where can you find all relevant apps for monitoring and reviewing imported data?||The Marketing Data Stewardship group App.|
|Contact Records are processed immediately at import?||No. Imported contact data is loaded first in the staging area, from where the data is processed.|
|What App can be used to get detailed information on data imports?||The Import Monitoring App.|
|Does SAP Marketing Cloud clean up the data automatically?||No. SAP Marketing Cloud does not change or delete data automatically.|
|How can we cleanse outdated and irrelevant data?||Marketing Application Jobs can be defined and scheduled to cleanse data.|
|What are the steps for deleting a contact?||1.Flag for deletion (data is archived) 2.Delete flagged data (data is deleted).|
|Can data flagged for deletion still be used for campaigns?||No. Flagged data is archived and can’t be used anymore.|
Resources and Supporting Material
SAP Blog Posts
SAP Help Documentation
Profiling is an essential part of SAP Marketing Cloud that sets up profiling configuration and helps you understand your customers. Profiling should be addressed as early as possible, during a SAP Marketing Cloud implementation and it should align with your use cases. SAP Marketing Cloud provides an existing default configuration for standard contact origins. However, review the existing configurations to validate whether the default settings align with the data sources in scope.