CX Works

CX Works brings the most relevant leading practices to you.
It is a single portal of curated, field-tested and SAP-verified expertise for SAP Customer Experience solutions

Migrating to SAP Customer Data Cloud: Key phases

12 min read


Overview

From A to Z. How to neatly plan your migration project.

No matter what unexpected turn the implementation might have taken, every single SAP Customer Data Cloud Data migration performed by SAP Services has essentially followed the same tried and tested process to great success. This article aims to talk you through all of the steps with full context and expectations for each so you can actively integrate the user migration in your overall project plan.

This article is part of a wider series on SAP Customer Data Cloud migrations. See Migrating to SAP Customer Data Cloud for the full list.

Table of Contents


Phases of a migration project


Password Hashing Algorithm Validation

So… What is a password hashing algorithm, and why does it need validation?

Most systems storing user credentials nowadays use some hash function so that their users’ passwords are not stored in clear text in their database and are therefore unusable by hackers should there be any data leak.

Those hash functions, or hashing algorithms, are usually standard hashes (md5, sha1, etc…), some will make use of a salt, some won’t. Some other algorithms will be a bit… well, less standard. We’ve seen all sorts, and we can support all sorts.

Now the reason you will want to validate your password hashing algorithm as a first step is to ensure the migration to SAP Customer Data Cloud is a smooth process for your users and that they will be able to seamlessly login to your site once it is migrated, and while all kinds of hashing algorithms are supported out of the box, double checking does not hurt as your particular algorithm might require a bit of additional twiddling and potentially need an external hashing endpoint to be created. You can read more on this process in the separate article Migrating to SAP Customer Data Cloud: Mastering Password Hashing Algorithms.

Steps & Outcome

  • Data Team to provide details of the password hashing algorithm used
  • Data Team to provide 5 hashed passwords (and salt if applicable) and their clear text equivalent
  • SAP Services / System Integrator to validate password hashing algorithm and provide appropriate password format for import file
  • SAP Services / System Integrator to give instructions on / setup Custom Hashing Algorithm Endpoint if applicable

As a result of this password validation, the exact password format to follow for each user record you will be migrating should be communicated to the team in charge of extracting and formatting the data from your legacy system.


JSON Import File Format Validation

Now is the time for your data team to work on the export routine from your legacy system that will extract your users into a file format supported by SAP Customer Data Cloud. If you’ve paid attention to the title of the section, you will probably have worked out that we are expecting a JSON file. This file will have a specific structure too, which is detailed on our online documentation.

While it is always perfectly understandable and fully expected that all extract routines won't be able to produce a perfectly formulated JSON from the get go, keep in mind that any data transformation step will ultimately have to deal with extremely sensitive Production data containing Personally Identifiable Information. This is where your precious user data is going to be at its most vulnerable and all possible precautions, checks and controls should be taken to ensure that no data leak occurs at this stage. This includes - but is not limited to - proper file encryption, use of secure servers rather than personal laptops and anonymisation of test data until import of production data is absolutely needed for validation / UAT purposes.


Steps & Outcome

  • SAP Services / System Integrator and Client Data Team to go through file structure
  • Client Data Team / System Integrator to produce JSON sample
  • SAP Services / System Integrator to validate sample’s file format

As an outcome of this step, your data team should be comfortable with the SAP Customer Data Cloud Import File structure and should be able to produce a valid sample JSON extract with at least the bare minimum information to import a user in SAP Customer Data Cloud. This is more about the SAP Customer Data Cloud specific structure of the import file than about your own data schema, which we can focus on in the next step.


Small Subset Test Import

Prior to working out the exact structure of the file, one of the priority steps will be to define your SAP Customer Data Cloud Data Schema, which will be a key milestone in your overall project plan and a dependency for this next step.

While during the JSON Import File Format Validation step we focused on getting your teams familiar with the SAP Customer Data Cloud Import File structure, in this phase we will focus on populating the file with all the fields relevant to your now established data schema.

Of course, each SAP Customer Data Cloud implementation will be slightly different, and in many cases depending on where we are with the SAP Customer Data Cloud Schema definition, this step is combined with the previous one. Depending on the evolution of the data schema during the overall implementation project however, you might have to take another pass at this step should any significant changes occur (such as the addition of a mandatory field).

Steps & Outcome

  • SAP Services / System Integrator & Client Data Team to go through finalized Schema
  • Client Data Team / System integrator to produce file with small subset of users
  • SAP Services / System Integrator to import small subset of users
  • SAP Services / System Integrator to provide Data Migration report
  • When successful, Client Data & Business teams to validate imported data

The idea here is to get a small but representative sample of your users and go through a test run of the import. We are usually talking about 1000 users, or 1% of your migrated users (depending on the size of the data migration).

This step should help us weed out many issues ahead of the full data migration, and we are looking for a sign off from both the data team and the business team that the data imported is correct before we get started with bigger imports.


Large Subset Test Import

In some SAP Customer Data Cloud implementations, there might be a need for a larger subset of users to be imported; this could either be because the volume of imported users is so large we would rather go through another, larger subset import to try to catch more edge cases, or this could be because as part of your implementation an integration with a 3rd party system is being built and they require a large volume of data to test the integration.

The exact volume of users to be migrated (or whether this step is required at all) will be determined by the project team depending on the use case.

Again, sign-off from the various teams involved will be sought before we move on to the full staging import.


Full Staging Import

The full staging import is exactly what its name implies; a full migration of all your user data on a staging API key.

This step typically occurs around the same time as the start of the UAT phase and allows us to achieve a few things; highlight any issue with the migration that might not have been captured before, establish the expected success rate of the data migration as well as the anticipated time it will take to migrate your full user base.

This last one is quite important as it also allows you to fine tune the timing of the production import and draw red lines in the pre-go live calendar.

Steps & Outcome

  • Client Data Team to extract full legacy database in chunks of 300-500k users
  • SAP Services / System Integrator to import users on staging api key
  • SAP Services / System Integrator to provide updated Data Migration report
  • When successful, Client Data, Business and other applicable teams to validate imported data
  • Project Team to update Production Data Migration calendar

With the Full Staging import will come the final sign-off for the teams to proceed with the full production import.


Full Production Import

That’s it, your users have never been so close to enjoying the smooth new experience you’ve been meticulously building for them over the course of your SAP Customer Data Cloud implementation.

We usually recommend going about this step (timeline permitting) a couple of weeks before go live. This gives you this cosy safety net we all like to have during a project. This also lets your project team focus on other important matters related to the go live, knowing that your users are already there waiting for you.

Steps & Outcome

  • Client Data Team / System Integrator to extract full legacy database in chunks of 300-500k users
  • SAP Services / System Integrator to import users on production API key
  • SAP Services / System Integrator to provide updated Data Migration report
  • When successful, Client Data, Business and other applicable teams to validate imported data

At this point you might be wondering what is happening to the users that are still registering or updating their profiles on your legacy systems if we’ve completed the migration 2 weeks before your actual go live, wonder no more; this is where we proceed with the production delta import(s) to accommodate registrations that took place between the original import and launch.


Production Delta Import(s) & Completion of the Data Migration

Because of the reasons outlined in the previous section, we’ve migrated your entire user base prior to the go live. The problem we have now is that many of your most engaged (and most valuable) customers will keep on logging in and updating their user information, and new ones might also be registering; we wouldn’t want to lose those important updates and give those users a sub-par migration experience.

This is where the Production Delta Imports come in. Depending on the timing and volume, there may only be one such import; where your data team will provide an extract of all the users that will have been created/updated since the time you extracted your full user base (or your last delta should there be several).

This import typically happens during the switchover period; and there could be different strategies depending on the structure of your legacy platform, its capabilities or the volume of activity.

Most typically, our customers will opt to switch off the authentication & registration part of their legacy website, produce the delta file, import it then switch over to SAP Customer Data Cloud.

Steps & Outcome

  • Authentication & Registration switched off on legacy site
  • Client Data Team to produce delta extract
  • SAP Services / System Integrator to import delta on production API key
  • SAP Services / System Integrator to provide completed Data Migration report
  • Final check and sign-offs.
  • Switchover to SAP Customer Data Cloud
  • Project Team Celebrates


Conclusion

You should now have a keen understanding of the various steps you are going to go through during your migration project, and be able to see how this sub-project fits in your overall implementation of SAP Customer Data Cloud. Planning, Communication, Testing and Sign-off by both IT and business stakeholders at those various checkpoints will avoid any nasty surprise when the time for go live has come.

If you are part of the technical team, you should now have enough context to start diving a bit deeper into the topic and we would strongly recommend you study closely the very comprehensive User Import Guide on developers.gigya.com with a treasure trove of tips and gotchas.

Also be sure to check out the rest of the articles in this series: Migrating to SAP Customer Data Cloud.


Overlay