CX Works

CX Works brings the most relevant leading practices to you.
It is a single portal of curated, field-tested and SAP-verified expertise for SAP Customer Experience solutions

Migrate to SAP Commerce Cloud: Migrate Media with AzCopy

8 min read

AzCopy: A fast, simple way for SAP Commerce Media Migration

AzCopy  is a command-line utility that can be used to copy blobs or files to or from an Azure Blob Storage account. This article describes the migration for SAP Commerce media files to the SAP Commerce Cloud Media Storage. Before reading this article it is recommended that you go through the introduction on Migrating to SAP Commerce Cloud: Data and Media Migration.

Table of Contents

Migration Concepts

During a media migration your media are copied from the existing on-premise source to the target instance of SAP Commerce Cloud. The migration occurs in a like-for-like environment to ensure data segregation and security, that is, for example, Development to Development, Stage to Stage, and Production to Production. Please note that this process is not applicable for existing Customers of SAP Commerce Cloud on SAP Infrastructure.

Migration Phases

The following diagram outlines how you can go about migrating your media. More detailed explanations for each of the phases are covered in individual sections below.

Planning

This phase should defines which environments will be involved in the media migration: the typical setup involves three environments (Development, Stage, and Production), however, every customer is different and you may find you need to migrate fewer or additional environments. For example, some development environments might be re-initialized when deployed. Therefore, their data and media could be excluded from a particular migration project. The actual migration of media should be performed environment-by-environment, tweaking and improving along the way. 

Setup

Using a Self-Service VPN Tunnel

AzCopy allows transfer over HTTPS however it is strongly recommended to establish connectivity over VPN for added security during the transfer so no unauthorized person can hook, record, or steal proprietary data before, during, or after the migration. This is a self service process available directly in the SAP Commerce Cloud Portal and described in the documentation.

SAS Token

Shared Access Signature (SAS) Token is a single-use identity token URI that grants restricted access rights to Azure Storage resources. A client who is in possession of the SAS token can make a request against Azure Storage with just the SAS URI, and the information contained in the SAS token is used to authorize the request. In order to be able to copy the media files to the targeted SAP Commerce Cloud media blob storage, a temporary SAS Token will need to be created for each environment via a Support Ticket. One Ticket will be needed per environment. SAS Tokens are set to expire after 2 weeks. When SAS Token expires a new request will be needed. Please allow up to 1 week for ticket to be executed. The following example shows how that request would be made. Make sure that the environment name is mentioned, because multiple environments of the same type can exist. 

Please note that ALL the information in the ticket between square brackets are mandatory to be specified. The IP to be allowlisted is particularly important as it will be the only IP which is allowed communication to the media blob store in SAP Commerce Cloud. 

Media Migration to Commerce Cloud with AZCopy - SAS Tokens

Commerce Subscription ID: [link to your Cloud Portal subscription]

Storage Account Name: [Storage Account Name]

Cloud Environment: [Commerce Cloud Environment Name]

Purpose: On Prem to Commerce Cloud Media Migration

Incoming IP Address: [100.100.100.101]

Please create a SAS Token for the cloud environment mentioned above and send them. Thank you. 


Installation

Ideally, the you would want to run AzCopy on a system that has direct access to the source media files, such as an application server with a mounted NFS folder containing the sys_master folder of the SAP Commerce media. The following script shows how you can run AzCopy:

# Log in to the server as a regular user
# Download the tar.gz file to the home folder:
$ wget https://aka.ms/downloadazcopy-v10-linux
 
# Untar the archived files:
$ tar -xvf downloadazcopy-v10-linux
 
# Go into to the untar'ed folder to locate AzCopy executable:
$ cd azcopy_linux_amd64_10.0.4/
 
# Verify that the program can run:
$ ./azcopy

Testing

Once the allowlisting is complete and you're in possession of the needed SAS tokens, a browser can be used to test access using a URL similar to the example below. The Storage Account Name is set in the media.globalSettings.cloudAzureBlobStorageStrategy.connection property that can be read in the hybris Admin Console (hAC) of one of your environment endpoints.

https://[storage account name].blob.core.windows.net/[container name]?[SAS Token]&comp=list&restype=container

Once the AzCopy installation is complete and library is accessible, test connectivity by listing the contents of a container from the SAP Commerce Cloud's media storage.

#list destination to see if connected and accessible
./azcopy list "https://[storage account name].blob.core.windows.net/[container name]?[SAS Token]"

Executing

For media storage SAP Commerce provides different Strategies, which define how and where medias are stored - see Media Storage Overview for more details. When a build is deployed to an SAP Commerce Cloud environment with the initialization option, by default, SAP Commerce will create the following media types.

Media Type Source Location

Source Folder

Target Container

Target Folder

root .../sys_master / sys-master-root /
images .../sys_master /images sys-master-images /images
impex .../sys_master /impex sys-master-impex /impex
cronjob .../sys_master /cronjob sys-master-cronjob /cronjob
catalogsync .../sys_master /catalogsync sys-master-catalogsync /catalogsync
jasperreports .../sys_master /jasperreports sys-master-jasperreports /jasperreports


For each Media type, an AzCopy command is needed to copy the data to the correct container. In the following script example, it will copy the catalogsync from the source and fully copy that folder and all its content into the target container. With this block of code, scripting the migration to handle all media types should be straight forward.

#catalogsync example
./azcopy copy "../sys_master/catalogsync" "https://[storage account name].blob.core.windows.net/sys-master-catalogsync/catalogsync?[SAS Token]" --recursive=true --log-level ERROR


Some customers might have other media type created from customization utilizing the Commerce Media Storage Strategy. 

Media Type Source Location

Source Folder

Target Container

Target Folder

root .../sys_master /customMediaType sys-master-customMediaType /customMediaType


The strategy to migrate those custom media types is the same with the additional operation to create the target container that will ultimately get the migrated files.

#customMediaType example
./azcopy make "https://[storage account name].blob.core.windows.net/sys-master-customMediaType?[SAS Token]
./azcopy copy "../sys_master/customMediaType" "https://[storage account name].blob.core.windows.net/sys-master-customMediaType/customMediaType?[SAS Token]" --recursive=true --log-level ERROR

Delta execution

AzCopy can also support delta migration using the sync option to facilitate pre-cutover and cutover activities reducing furthermore migration time. This approach is meant to be an extension to the current process, which will give the customers a different option to use in case of a large amount of media to be migrated. 

Conclusion

You should now have an understanding of the fundamental steps of migrating media into your SAP Commerce Cloud in the Public Cloud environment(s). For more information on migration please see the rest of the articles in the Migrate to SAP Commerce Cloud series.

Overlay