CX Works

CX Works brings the most relevant leading practices to you.
It is a single portal of curated, field-tested and SAP-verified expertise for SAP Customer Experience solutions

Technical Implementation: Integrating SAP Customer Data Cloud and SAP Marketing Cloud

11 min read


Overview

This article is part of an article series about the coexistence of SAP Customer Data Cloud and SAP Marketing Cloud. To learn more about this topic, visit: Coexistence: SAP Customer Data Cloud and SAP Marketing Cloud


Table of Contents


Transfer Modes

Introduction

Two modes are available in SAP Marketing Cloud in order to synchronize contacts between SAP Customer Data Cloud and SAP Marketing Cloud:

  • Synchronous
  • Asynchronous

Before we can get into the details of these modes, you should have a high-level understanding of the components involved:

  • ETL (Extract, Transform, Load) solution (such as SAP Customer Data Cloud IdentitySync or SAP CPI) that offers an easy way to transfer data in bulk between platforms
  • SAP Gateway → SAP Marketing Cloud service which performs technical validation of the request such as field formatting
  • Staging Area → SAP Marketing Cloud service which stores imported data
  • Contact/Permission → SAP Marketing Cloud service which creates/updates the contact and permissions within SAP Marketing Cloud

Synchronous

When using the synchronous mode, the status of the transaction is received once the records are successfully transferred and processed by SAP Marketing Cloud. The below statements apply to this transfer mode:

  • Monitoring can be done from the requesting system (i.e. IdentitySync)
  • A record marked as successful in the requesting system represent a successful creation of the Contact in SAP Marketing Cloud however some preferences may have been rejected
  • In order to see the true state of the transaction, you should refer to the Import Monitor in SAP Marketing Cloud

Below is a simplified representation of the flow:

Cannot render diagram with PlantUML server

Asynchronous

When using the synchronous mode, the status of the transaction is received once the records are successfully processed by the Gateway. The below statements apply to this transfer mode:

    • A record marked as successful in the requesting system represent a successful transfer of the request rather than a successful contact creation.
    • In order to see the true state of the transaction, you should refer to the Import Monitor in SAP Marketing Cloud

Below is a simplified representation of the flow:

Cannot render diagram with PlantUML server

Recommended Approach

There are many factors which can lead to the decision of using asynchronous or synchronous transfer mode and we would advise you to get in touch with SAP Marketing Cloud experts to decide which suits best your scenario. However, based on our own experience, the asynchronous mode is more commonly used, especially during migration exercises or large(r) volumes for the following reasons:

  • Performance: The bottleneck of the integration is the actual contact and permission creation in SAP Marketing Cloud. The asynchronous mode allows to split the data transfer and the data processing therefore the data transfer completes very quickly and SAP Marketing Cloud can handle the data processing at best
  • Less processing errors: SAP Marketing Cloud locks contacts when they are being updated. The integration relies on multiple updates (contact, permission, subscription, ...) therefore locks may happen. The asynchronous flow handles the contact lock with a more robust retry mechanism as well as allowing a manual retry from the Load Monitor

Account Creation & Update

The following table highlights the main steps & transformations to include to the SAP Customer Data Cloud Identity Sync dataflow in order to have a successful integration. Important to note this is not an exhaustive list and client specific requirements may have an impact on the final version of the dataflow.

DataFlow Steps & Components

Datasource.read.gigya.account

Extract all new and modified records from emailAccounts. This selects all full and lite accounts

Datasource.lookup.gigya.account

Lookup records from accounts (full) where the emailAccount email equals either the verified or unverified emails.

The output to this step is as follows:

-       Record contains the full account record (if exists)

-       Record.emailAccount contains the lite account record

Field.evaluate (optional)

Same as above for the looked up full account record

Record.evaluate

Execute custom selection logic based on account and emailAccount records

Datasource.write.hybrismarketing

Define SAP Marketing Cloud connection details

Define subscriptions mapping

Data Mapping

Marketing OData Service Metadata contains the list of standard and custom data which can be transferred to SAP Marketing Cloud. The link to the metadata is https://<endpoint>/sap/opu/odata/sap/API_MKT_CONTACT_SRV;v=<versionNumber>/$metadata

Selection Lists

SAP Marketing Cloud offers the ability to create selection lists for the below attributes:

  • Country
  • Industry
  • Department
  • Function
  • Gender
  • Language
  • Marital status
  • Address region
  • Form of address

When building an integration with SAP Marketing Cloud, there are 2 options:

  1. Map to the code: Incorrect codes will result in import errors, indicating that the corresponding code is not valid
  2. Map to the free text property: This is used if you are not thoroughly familiar with the internal codes available in SAP Marketing Cloud and you should use properties that allow a free text. For example, if you do not know that DE is the country code for Germany, you can use Germany as the free text.

The benefit of option 2 is that the integration will never reject a value. However, the mapping of "new" free text properties would then need to be properly completed; this is done via the "Map Free Texts" application as per below screenshot:

Custom Field Mapping

When mapping from SAP Customer Data Cloud to SAP Marketing Cloud its recommended to use the field.rename component. You’ll need to reference the “_MPS” name.

Example: YY1_LiteFullRegistr_MPS

Tips:

  • It’s always recommended to use the core components when possible instead of scripting. The core components utilize the core java engine powering the platform.
  • For error handling purposes, it is also recommended to push synchronization errors to a remote location (sftp, S3, other)

Social Identities

Some customers are using social platforms to collect comments and likes/dislikes. This can be used to drive marketing campaigns however the information collected about the actual user is limited to what is known to the social provider (and agreed to be shared by the end user). Having the ability to understand who post comments/likes is critical to target the right campaigns and having the ability to synchronize the social IDs from SAP Customer Data Cloud to SAP Marketing Cloud can help reconcile the user data and behavior. Technically, the mapping of the SAP Customer Data Cloud social IDs to the SAP Marketing Cloud field(s) is as simple as mapping any other fields. However, in the event the customer is having many brands, it is likely for users to be connected to many Facebook IDs (one per brand) therefore the right synchronization needs to be defined.

If you are using CPI instead of IdentitySync you can also pass the social IDs directly as additional facets (one per social ID) as part of the API call. Use the entity 'AdditionalIDs' of API API_MKT_CONTACTS.

If you are using IdentitySync, you don't have the option to pass the list of social IDs directly as AdditionalIDs therefore you need to rely on the use of a custom field in SAP Marketing Cloud. In order to pass the full list of social IDs, the recommended approach is as follows:

  • Define a custom field (text) for a social provider in SAP Marketing Cloud to save a concatenation of all social ids (separated by ",")
  • Define a custom logic in SAP Marketing Cloud which parses the concatenated social ids in a list, as per below example
  • Concatenate social IDs during the transformation step in IdentitySync
***insert additonal facet for facebook apps ids maps
If contact_data-YY1_FB_MAPPPED_APP_IDS_MPS is not initial.
        SPLIT contact_data-YY1_FB_MAPPPED_APP_IDS_MPS AT ',' INTO TABLE lt_fb_apps_ids.
        if lt_fb_apps_ids is not initial.
            loop at lt_fb_apps_ids into lv_fb_apps_id.
                if lv_fb_apps_id is not initial or lv_fb_apps_id <> 'null'.
                    ls_additional_facet-id_origin = 'FB'.
                    ls_additional_facet-id = lv_fb_apps_id.
                    append ls_additional_facet to additional_facet.
                endif.
            endloop.
        endif.
endif.
If you want to use other social identities besides Twitter or Facebook you also need to set up the custom ID-Origins in SAP Marketing Cloud customizing accordingly.


Account Deletion

The dataflow below identifies the deleted account querying the auditLogEvent table with the audit.search API.

DataFlow Steps & Components

Datasource.read.gigya.audit

Extract all new records marked as “RightToBeForgotten” from the auditEventLog

Datasource.delete.hybrismarketing

Define SAP Marketing Cloud connection details for deletion

For error handling purposes, it is also recommended to push synchronization errors to a remote location (sftp, S3, other)

As of November 2019, all contacts deleted using the datasource.delete.hybrismarketing component are flagged with isEndOfPurposeBlocked set to True. As a result, the UID can no longer be used and the creation of another contact with the same UID would fail.

IdentitySync vs CPI – Pros & Cons

As of August 2020, IdentitySync is the only solution supported by SAP Customer Experience

System

Pros

Cons


CPI

  • Allows for greater level of scale compared to a point to point integration
  • Re-use of CPI as the central integration instance for SAP Marketing Cloud
  • No standard integration content (as of August 2020)

IdentitySync

  • Reduce implementation effort depending on architecture
  • Easy to maintain dataflows
  • Troubleshooting can be difficult at times
  • Unidirectional integration, no feed from SAP Marketing Cloud to SAP Customer Data Cloud
  • Lowest minimum frequency is 10 minutes

Conclusion

Continuing reading more on the topic of Coexistence: SAP Customer Data Cloud and SAP Marketing Cloud by accessing other articles within this series:




Overlay