CX Works

A single portal for curated, field-tested and SAP-verified expertise for your SAP C/4HANA suite. Whether it's a new implementation, adding new features, or getting additional value from an existing deployment, get it here, at CX Works.

Integrating Clickstream Data from Google Analytics


Integrating Clickstream Data from Google Analytics


Don't count the people that you reach, reach the people that count - David Ogilvy.

In this article you will learn more about the integration of SAP Marketing Cloud with Google Analytics. As a marketer, this can help you to leverage your consumers website behavior which ultimately gives you the chance to drive more personalized campaigns.

Table of Contents


Use Case and Business Questions

It's not news that you can integrate your clickstream data from Google Analytics into SAP Marketing Cloud. Clickstream data typically contains the information about which pages have been visited at which point in time and from which user or device. 

But clickstream data only becomes actionable if you also know who is behind the interaction. An anonymous cookie ID won't give you the ability to run personalized campaigns outside of Google Analytics.

But how do you identify a customer that doesn't login or what if your website doesn't offer any user identification/login functionality? Don't you need to have a profile management tool in place?

You don't. SAP Marketing Cloud acts as the profile management tool in this scenario. You can identify contacts through campaigns that are executed from SAP Marketing Cloud. By appending the SAP Outbound ID to links in your e-mail content, you can pass this alphanumeric ID to any web page. By using Google Analytics for tracking clickstream data based on the SAP Outbound ID, you are not revealing Personally Identifiable Information (PII) data.  You can then create interactions from Google Analytics by using the SAP Outbound ID as Interaction Contact ID.


image one:  High-level end-to-end process


To realize this process, several implementation steps have to be performed. For sending e-mails with links which have the SAP Outbound ID appended to it, find more information in the Application Help. No further configuration is required for this step. The required steps around tracking the customers on the websites through Google Analytics are described in the Preparation Phase section. The Configuration Phase section explains how to configure SAP Marketing Cloud and SAP Cloud Platform Integration to query the clickstream data from Google Analytics. 

Prerequisites and Setup

Let's set the stage by giving a high-level understanding of Google Analytics, the underlying terminology, and further topics that should get your attention.

Google Licenses

Google offers two versions of Google Analytics. The standard version of Google Analytics is free whereas Google Analytics 360 is the chargeable enterprise version of Google Analytics. Maybe you have also heard of BigQuery which is a Google Developers tool that lets you run fast queries of large datasets. You can export session and hit data from a Google Analytics 360 account to BigQuery, and then use a SQL-like syntax to query all of your analytics data.


While Google Analytics 360 generally offers more features around complex web tracking analytics compared to the standard version, you can set up and run this integration scenario with both versions.

  • With the standard version or with Google Analytics 360, you can run your queries from SAP Marketing Cloud directly against the Google Analytics API
    • The scenario can be implemented but multiple queries might have to be created
  • With Google Analytics 360, you can also export the data to Google BigQuery
    • The same functionality can be implemented in a more elegant way
    • This scenario can also be implemented with SAP Marketing Cloud (not highlighted in this blog)

Data Privacy & Google Analytics Terms of Service

Let's take a closer look at the Google Analytics Terms of Service and how it needs to be considered for this integration scenario.


Please be advised that nothing in this article constitutes legal advice. We strongly recommend that you involve your legal deputies to check and approve the concepts you want to implement.


When gathering data through Google Analytics and pushing it to SAP Marketing Cloud, your cookie and privacy policy should explain how you are using this data.

Maybe you are questioning why the 'SAP Outbound ID' parameter is used instead of just pulling the e-mail address or any other personal information that is given by the customer through a webform.

The background is that when using Google Analytics, you agree that you will not pass personally identifiable information (PII) to Google. The following extract is taken from the Google Analytics Terms of Service (Retrieved October 2019):

'You will not and will not assist or permit any third party to, pass information to Google that Google could use or recognize as personally identifiable information. You will have and abide by an appropriate Privacy Policy and will comply with all applicable laws, policies, and regulations relating to the collection of information from Users. You must post a Privacy Policy and that Privacy Policy must provide notice of Your use of cookies, identifiers for mobile devices (e.g., Android Advertising Identifier or Advertising Identifier for iOS) or similar technology used to collect data.'


Since alphanumeric database identifiers are not considered as PII data according to the Google Analytics Terms of Service, the SAP Outbound ID can be used for this scenario.


What Does Google Consider as PII?

PII is any data that could be used on its own to directly identify, contact or precisely locate an individual. This includes:

  • E-mail Addresses
  • Mailing Addresses
  • Phone Numbers
  • Precise Locations
  • Full Names or Usernames


When it comes to personally identifiable information (PII), it doesn't matter whether you are using the free Google Analytics Standard or the paid Google Analytics 360 solution.

Google Analytics Terminology

In order to set the baseline for the implementation, let's break down the Google Analytics terminology.

Google Analytics Account Structure



image two: Hierarchy of Organizations, Accounts, Users, Properties, and Views


Organizations

  • An organization represents a company and lets you access your company's product accounts, manage product users and permissions, and cross-product integrations.
  • You can group all of your Google Analytics accounts under an 'Organization' which is optional. This would allow you to manage all the accounts under one grouping.
  • Example: "ACME Cooperation" (organization)


Account

  • An account is your access point for Analytics. One or more Analytics accounts can belong to an organization. The Google Analytics account determines,how data is collected from your websites and manages who can access that data.
  • You can create separate Google Analytics accounts for distinct businesses or business units.
  • Example: "ACME Marketing" (account name) 


Property

  • A property is a website, mobile application, or device. An account can contain one ore more properties. Within a Google Analytics account, you add the properties from which you want to collect data.
  • You can have separate properties for different sales regions or different brands. This allows you to easily view the data for an individual part of your business, but keep in mind this won't allow you to see data from separate properties in aggregate.
  • Example: "ACME Webshop" (property name); "ACME Web App" (property name) 


Views

  • A reporting view is the level in an Analytics account where you can access reports and analysis tools. Google Analytics automatically creates one unfiltered view for every property in your account, but you can set up multiple views on a single property.
  • You can create one view that includes all of your global website data. But if you want to see data for individual regions, you could create separate views for North America, Europe, and Asia.
  • Example: "Webshop - All Data" (view name); "Webshop - Europe" (view name); "Webshop - Asia" (view name)


If you set up Google Analytics for the first time please consider your long-term reporting goals and how they might affect your account structure. Create an implementation plan before you set up new properties. For more information, click here.

By assigning Account Permission, Property Permission, or View Permission to your integration service account, you are defining from which account, property, or view to retrieve Google Analytics data into SAP Marketing Cloud. For details, please refer to the"Preparation Phase" section and the underlying task "Add Permissions to your Google Analytics Account".


It is useful to read through the Google documentation for more information around the Google Analytics account structure, where you can also find examples for account structures.

Google Analytics Data Model


image three: Google Analytics Data Model


In the basic data model used in Analytics, the user (visitor) interacts with your content over a period of time, and the engagement with your site or app is broken down into a hierarchy. Each of these three levels of interaction defines a specific scope of user engagement.

  • User (visitor): The client that visits the site such as the browser or mobile phone operated by a person.
  • Session (visit): The period of time during which the visitor is active on the site.
  • Interaction (hit): The individual activities that send a GIF request (hit) to the Google Analytics servers. These are typically characterized by a pageview, but can also include:
    • An event (for example, click on a motive button)
    • A transaction
    • A social interaction

For users, sessions, and interactions, a set of default dimensions exists. A dimension is a descriptive attribute or characteristic of data. Examples of standard dimensions in Google Analytics are Browser, Exit Page, and Session Duration. 

Additionally, you can also create custom dimensions for all three levels in order to collect and analyze data that Google Analytics isn't tracking automatically. 

In the end, any types of online interactions on the website can be tracked in SAP Marketing Cloud if they are in Google Analytics with a millisecond-level timestamp value and a user ID value. For more information check out the next section.

Implementation

When you are planning or currently implementing SAP Marketing Cloud, you might wonder when to integrate web tracking data from Google Analytics. Maybe you are already using Google Analytics for tracking and analyzing your websites. However, just using Google Analytics without adjusting it to this specific scenario is not enough. You need to do adjustments to the Google Analytics custom dimensions and the tracking code which you are using. Only after these adjustments are implemented, all necessary data is tracked at the granular level which you can use to create interactions in SAP Marketing Cloud.


Finish the preparation tasks for this scenario as soon as possible during your implementation. This is necessary to track your visitors at the granular level. The preparation tasks should be finished weeks or months earlier than the remaining steps since you can’t import historical data that was tracked before your adjustments have been implemented. The detailed preparation and implementation tasks are described later in this section.


Preparation Phase

Before you start with the implementation, you should discuss and decide from which properties you want to import the web tracking data as interactions into SAP Marketing Cloud. A simplified example could look like the following: Under the corporate Google Analytics account, there could be multiple properties such as shop sites, landing pages, corporate blogs, or further websites you might want to track.


image four: Google Analytics Exemplary Properties


We highly recommend starting with the properties which are most important for your marketing activities . This reduces the effort of setting up or adapting the web tracking code. Later, you should extend your scope to the less important properties.


After you have decided for which properties you want to import web tracking data, you should also consider the web tracking data in context of your overall data ingestion strategy. 

We generally don't recommend that you load all browsing events available in Google Analytics. You should focus on events which are required for profiling and campaigns.

At the same time, you should already have a concept ready about how long you want to keep the web tracking data. Usually the web tracking data gets less relevant and valuable over time.

Read the following article to find out more about how to define and implement your data retention policy for SAP Marketing Cloud: https://www.sap.com/cxworks/article/462209525/Implementing_Your_Data_Retention_Policy_for_SAP_Marketing_Cloud


As mentioned earlier, the tasks contained in the preparation phase should be done as soon as possible. Most of the tasks are well described in the integration guide for this scenario. Please follow the steps in the guide. Additional configuration and validation steps are described in this article. In the following table, we have summarized the tasks with the corresponding task description from the SAP Help Portal. 


Phase  

System  

Task  

Preparation  

-  

Get accesses and authorizations for all systems

Google Analytics  

 

Create your Service Account

Assign a Scope to your Service Account

Add Permissions to your Google Analytics Account  

Add Permissions to your Google BigQuery Account (only if Google BigQuery is used)

Set up Custom Dimension SAP Outbound ID  

Tracked Properties  

Set up or adapt Google Analytics tracking code (Please see below)

Google Analytics

Validate the query results with a custom report  (Please see below)

Google Query Explorer

Validate the query results  (Please see below)


Set Up and Adapt the Google Analytics Tracking Code

Another important task during this phase is the setup or adaptation of the tracking code. First, you will have to create two custom dimensions in Google Analytics. Then, you need to adapt the tracking code accordingly as described below. 


Custom Dimension Name

Scope

Required

SAP Outbound ID

User

Yes

yHitTimestamp

Hit

Yes



Make sure to use the exact spelling "SAP Outbound ID" for this custom dimension. In the most current version of SAP Marketing Cloud, this is required in the query configuration app.


The following adaption of your Google Analytics tracking code has to be done through Google Tag Manager or the global site tag (gtag.js). Both methods are interoperable and you are free to choose the solution that suits your needs. Google provides a help document to for Tag Manager and gtag.js. The following code example is for the widely used Google Tag Manager. The custom variable is of the type "Custom JavaScript".


yHitTimestamp
function() {
	return new Date().getTime();
}


Additionally, you need to fill the SAP Outbound ID custom dimension. 

Since the SAP Outbound ID is passed to the website through an URL parameter (for example: https://domain.com/site.html?sap-outbound-id=41BA458916AFF02AA504305B4F6C9FF1BB49A645 ), you need to push it into the JavaScript data layer. Below, you will find coding from the JavaServer pages of our internal development landscape:


SAP Outbound ID
<%@ tag body-content="empty" trimDirectiveWhitespaces="true" %>
<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>
<c:if test="${not empty request.getParameter('sap-outbound-id')}">
	window.dataLayer.push({
		'sap-outbound-id': '${request.getParameter("sap-outbound-id")}'
	});
</c:if>


After you have performed all previously described steps, you are able to track the clickstream data in Google Analytics.

To generate events which are tracked in Google Analytics, one way is to send an e-mail campaign from SAP Marketing Cloud to yourself. The links inside the e-mail should point to your tracked property and have the "Append Outbound ID" option enabled. 


Confirm that you are using the right SAP Marketing Cloud tenant for testing and validating the configuration. The "SAP Outbound IDs" sent in e-mails from the productive tenant are unknown in the quality tenant. Therefore, they can't be matched to contacts.


After you receive the e-mail and click on the corresponding link, the website visit(s) will be tracked in Google Analytics. 


Validate the Query Results with a Custom Google Analytics Report 

Now, you can already receive the web tracking data through a custom report in Google Analytics. The underlying idea is to evaluate if everything is well set up before pushing the data into SAP Marketing Cloud. You can find more information about how to create and manage custom reports here

In the custom report, specify the dimensions and metrics of your query. The screenshot below shows the typical dimensions, metrics, and filters. 


image five: Custom Google Analytics Report


Validate the Query Results with Google Query Explorer

In addition, you can also validate the results through the Google Query Explorer as shown in image six and seven. With this tool, you can build API queries to collect data from your Google Analytics views. You can use these queries in any of the client libraries to build your own tools.

After identifying your Google Analytics account, property, and view, you need to specify the dimensions and metrics for your query. The different query parameters are described in the Google Analytics Core Reporting APIs.


image six: Google Query Explorer


Please note, that the custom dimensions from Google Analytics will appear as ga:dimensionX (X = 1,2 ...) depending on how many custom dimensions you have.


In this integration scenario, default dimensions and custom dimensions are combined to create interactions in SAP Marketing Cloud. 


Please note, that not all dimensions and metrics can be queried together. Only certain dimensions and metrics can be used together to create valid combinations. You can use the Dimensions & Metrics Explorer to find out which combinations can work.


As an example, you can run the query and validate that the number of hits matches your expectations. 


image seven: Results in Google Query Explorer


Configuration Phase

Now that you have finished all preparation tasks, you can continue with the configuration tasks on SAP Marketing Cloud and SAP Cloud Platform Integration. 


You should consider first connecting your SAP Marketing Cloud quality tenant to Google Analytics. After you have validated that everything works as expected, you should change to your productive SAP Marketing Cloud tenant.


Again, most of the tasks are described in the integration guide for this scenario. Please follow the steps in the guide.



Below, you can find a configuration for the Google Query Configuration application. 

  • In the interactions fields section, you specify to create interactions of type WEBSITE_VISIT, communication medium WEB and contact origin SAP_TRACKING_ID
  • In the common query filter and dimensions section, you specify the filters and dimensions as shown below: 


image eight: Google Query Configuration Application in SAP Marketing Cloud


In the queries section, you have to map the dimensions from Google Analytics to interaction fields in SAP Marketing Cloud. In addition to mapping the dimensions, make sure to maintain ga:hits as Google Analytics Metric. The typical mapping should look like this: 


Google Analytics Dimension Interaction Field in SAP Marketing Cloud
ga:pageTitle InteractionContentSubject
ga:pagePath InteractionSourceDataURL
ga:deviceCategory DeviceType
ga:operatingSystem InteractionSourceObjectType
ga:channelGrouping SourceSystem
ga:source ReferralSource
ga:medium ReferralSourceCategory



Please be aware that custom fields for the interaction object in SAP Marketing Cloud can also be mapped against Google Analytics dimensions.


Test and Validation Phase

Phase System Task

Test and validation

SAP Marketing Cloud

Validating Query Configurations


From a technical integration perspective, you will see data coming in through the import monitor by filtering for the OData service "Web Tracking Mapping OData Service": 

image nine: Import Monitor in SAP Marketing Cloud


In a business validation step, you should cross-check the Google Query Configurations on SAP Marketing Cloud side as described in Validating Query Configurations .  

If you get different results for your queries, you should first compare the results from Google Query configuration with the results from Google Query Explorer. In some cases, errors in the JavaScript tracking code cause the differences. 


The system issues a warning if the query results contain sampling data. For more information, see https://support.google.com/analytics/answer/2637192?hl=en


Ultimately, you can see the interactions in the contact fact sheet, in the segmentation or in other apps. The screenshot below shows the website visit interaction in the timeline of the contact fact sheet:  


image ten: Contact app in SAP Marketing Cloud


Conclusion

This article introduced you to the integration of clickstream data from Google Analytics into SAP Marketing Cloud. The article explained how exactly this integration works and how you can implement and validate it.

By setting up the SAP Outbound ID as a custom dimension in Google Analytics, you are able to track website visit behavior of existing customers. With the Google Query configurations app, you are able to create the corresponding clickstream interactions in SAP Marketing Cloud. Ultimately, this helps you to better understand the online behavior of your customers which allows you to realize more targeted marketing campaigns.