CX Works

CX Works brings the most relevant leading practices to you.
It is a single portal of curated, field-tested and SAP-verified expertise for SAP Customer Experience solutions

SAP Marketing Cloud Regular Upgrade Testing

8 min read


SAP Marketing Cloud Regular Upgrade Testing

Table of Contents

Overview

SAP  executes  a comprehensive testing procedure before releasing  our software to customers in order to ensure  a high - quality standard. With each quarterly software release, customer  t esting is executed to validate that the functionality is working as expected; not to validate standard functionality per se, the emphasis is on customers' specific configuration, extensions and customization. The Onboarding Guide for SAP Marketing Cloud describes the concept of the regular upgrades of the solution, and includes details regarding regression testing the solution during the upgrade period. This article gives a high level overview of how to prepare, execute, and close your regular upgrade test cycles.  


The information in this article provides a high-level outline; however, each implementation is unique in configuration and/or customization and this must be  taken into account  during testing .


  The overall timeline of the upgrade process is outlined in the diagram below.


Prepare

Before the software upgrade is carried out, SAP informs customers that the upgrade will be taking place (about 6 weeks notice), updates help pages, produces webinars, and prepares detailed release notes.  You can register for and attend the webinars to get a sense of what's changed, deprecated, and new -  allowing you to plan your testing efficiently.  Test plans and scripts should be updated to reflect the updated feature set, and the individuals who will be involved in testing should be reminded of the test cycle to come, including the changes to the test phase.  The test planning should also consider the ‘Restrictions during the Upgrade Process’ to be prepared for system limitations during the process.

The table below outlines some fo the areas that should be considered when preparing your test plan.


Business Scenario or Area

Task/Exercise

 

Related Applications

Roles & Catalogs

Review relevant changes to Business Groups, Catalogs and apps according to the ‘What’s New’ section. For more details around Roles & Catalogs, see the Administration Guide, chapter Managing Business Users and Roles in Marketing.


  • Maintain Business Roles
  • Maintain Business Users
  • Business Roles Templates
  • Business Catalogs

Campaign Orchestration

Verify your Email Service Provider Connection. Monitor your first campaigns in order to track if emails are still being sent and bounce & complaint information collected.

  • Sender Profile
  • Content Studio
  • Communication Systems (e.g. SAP Digital Interconnect, Amazon)

Extensibility

Verify if implemented custom logic is still working as expected.

  • Custom Fields and Logic
  • Extensibility Inventory
  • Extensibility Settings

Integration 

Monitor your running integrations by checking if data is still loaded as expected. It is recommended to monitor SAP Cloud Platform Integration to see if any errors are coming up. For this it is helpful to use the Import Monitor to monitor the traffic.

  • Import Monitor
  • Marketing Application Log
  • Marketing Application Jobs

Operations

Verify if your existing & scheduled application jobs  are still running and set up correctly.

  • Marketing Application Log
  • Marketing Application Jobs


In the SAP Best Practice Explorer, there are detailed test scripts which have been prepared to support you and your team to build test cases that suit your context. for example,  Segmentation and Campaign Execution  Test Script.  At the end of this article you will also find an example of a test scenario that customers have used, which have been anonymized.  The right combination depends on your business scenarios and implemented features.


When the latest SAP Marketing Cloud software reaches RTC (Release To Customer) status, the  SAP Help Portal  is updated with “what’s new” content.  

Execute

The test case execution is performed in the Q environment using the identified functional test  cases  as well as end to end scenario s. The goal is to validate the  the functionality and business scenarios are working as expected.  Note that it's important to have confirmed the baseline behaviour before the upgrade so that you can be sure that any change in behaviour is due to the software update.

If issues are encountered, the  SAP ONE Support Launchpad  should be used to document incidents and to follow up on those incidents through to resolution.  

It is a best practice to document results of the testing as well as any other key performance indicators (KPIs) related to the testing such as planned vs executed test cases, passed test cases, failed test cases etc.

Ideally, testing related defects are reported early enough in the test cycle to allow for corrective actions to be implemented and tested behaviour re-validated before the P system (production tenant) upgrade.

Once the production tenant has been upgrade, a more limited set of test cases should be executed to verify that the upgrade didn't cause any adverse effects, and that the functionality is working as expected.   

Close

Once the test phase is completed, it is best practice to document lessons learned in order to better prepare to execute upgrade testing in the future.  It's also a good time to evaluate the expansion of your solution scope with new features by reviewing the ‘What’s New in SAP Marketing Cloud’ page on the SAP Help Portal.   






Conclusion

Testing of regular software upgrades requires some preparation, but allows you to ensure that the SAP Marketing Cloud solution is fulfilling the expected business outcomes with each iteration.  This guide provided some information to help you keep pace with the quarterly innovation cycle.

If you require assistance in planning, executing, or closing your test activities, SAP CX Services can support you through our Technical Design Assessment service offering.

Download the CX Marketing - Campaign Guide 

This document provides an end-to-end scenario test to create a trigger based campaign that can be used as the basis for regular regression testing.


Related CX Works Articles:


Overlay