Migrate to SAP Commerce Cloud: Migrate Media with AzCopy
8 min read
AzCopy is a command-line utility that can be used to copy blobs or files to
or from an Azure Blob Storage account. This article describes the migration for SAP Commerce media files to
the SAP Commerce Cloud Media Storage. Before reading this article it is recommended that you go through the
introduction on Migrating
to SAP Commerce Cloud: Data and Media Migration.
Table of Contents
During a media migration your media are copied from the existing on-premise source to the target instance of SAP Commerce Cloud. The migration occurs in a like-for-like environment to ensure data segregation and security, that is, for example, Development to Development, Stage to Stage, and Production to Production. Please note that this process is not applicable for existing Customers of SAP Commerce Cloud on SAP Infrastructure.
The following diagram
outlines how you can go about migrating your media. More detailed explanations for each of the phases are
covered in individual sections below.
This phase should defines which environments will be involved in the media migration: the typical setup involves three environments (Development, Stage, and Production), however, every customer is different and you may find you need to migrate fewer or additional environments. For example, some development environments might be re-initialized when deployed. Therefore, their data and media could be excluded from a particular migration project. The actual migration of media should be performed environment-by-environment, tweaking and improving along the way.
Using a Self-Service VPN Tunnel
AzCopy allows transfer over HTTPS however it is strongly recommended to establish connectivity over VPN for added security during the transfer so no unauthorized person can hook, record, or steal proprietary data before, during, or after the migration. This is a self service process available directly in the SAP Commerce Cloud Portal and described in the documentation.
A Shared Access Signature (SAS) Token is a single-use identity token URI that grants restricted access rights to Azure Storage resources. A client who is in possession of the SAS token can make a request against Azure Storage with just the SAS URI, and the information contained in the SAS token is used to authorize the request. In order to be able to copy the media files to the targeted SAP Commerce Cloud media blob storage, a temporary SAS Token will need to be created for each environment via a Support Ticket. One Ticket will be needed per environment. SAS Tokens are set to expire after 2 weeks. When SAS Token expires a new request will be needed. Please allow up to 1 week for ticket to be executed. The following example shows how that request would be made. Make sure that the environment name is mentioned, because multiple environments of the same type can exist.
Please note that ALL the information in the ticket between square brackets are mandatory to be specified. The IP to be allowlisted is particularly important as it will be the only IP which is allowed communication to the media blob store in SAP Commerce Cloud.
Ideally, the you would want to run AzCopy on a system that has direct access to the source media files, such as an application server with a mounted NFS folder containing the sys_master folder of the SAP Commerce media. The following script shows how you can run AzCopy:
Once the allowlisting is complete and you're in possession of the needed SAS tokens, a browser can be used to test access using a URL similar to the example below. The Storage Account Name is set in the media.globalSettings.cloudAzureBlobStorageStrategy.connection property that can be read in the hybris Admin Console (hAC) of one of your environment endpoints.
Once the AzCopy installation is complete and library is accessible, test connectivity by listing the contents of a container from the SAP Commerce Cloud's media storage.
For media storage SAP Commerce provides different Strategies, which define how and where medias are stored - see Media Storage Overview for more details. When a build is deployed to an SAP Commerce Cloud environment with the initialization option, by default, SAP Commerce will create the following media types.
|Media Type||Source Location||
For each Media type, an AzCopy command is needed to copy the data to the correct container. In the following script example, it will copy the catalogsync from the source and fully copy that folder and all its content into the target container. With this block of code, scripting the migration to handle all media types should be straight forward.
Some customers might have other media type created from customization utilizing the Commerce Media Storage Strategy.
|Media Type||Source Location||
The strategy to migrate those custom media types is the same with the additional operation to create the target container that will ultimately get the migrated files.
AzCopy can also support delta migration using the sync option to facilitate pre-cutover and cutover activities reducing furthermore migration time. This approach is meant to be an extension to the current process, which will give the customers a different option to use in case of a large amount of media to be migrated.
You should now have an understanding of the fundamental steps of
migrating media into your SAP Commerce Cloud in the Public Cloud environment(s). For more information on
migration please see the rest of the articles in the Migrate to SAP Commerce