Integration Options for SAP Commerce Cloud
Your SAP Commerce Cloud solution does not live in a vacuum. It will always need to integrate with one or more external services in order to deliver a great customer experience. The types of integrations can vary such as payment, shipping, reviews or even connecting to other SAP Customer Experience solutions like SAP Marketing Cloud or SAP Sales Cloud. Each solution you wish to integrate can have one or more ways to create that connection. This article will use some of the key aspects covered in Overview of SAP Customer Experience solutions Integration and Extension Options to guide you in your decision on which integration option(s) might be best.
Table of Contents
How to Choose?
With so many integration options, it can be tough to choose which option to move forward with. The first step is to quickly eliminate those options that will not work for you. The flow chart below can help you to understand which options to pursue. You should also be consulting the Roadmap to understand if (or when) a pre-built integration that you require, might be available. If you're looking to integrate another SAP product with SAP Commerce Cloud, you should also consult the Architecture of SAP Integrations page.
'Customize' is used here as a catch-all for options such as:
- Greenfield: Build your integration from scratch. This would be a custom solution created by (and supported by) you.
- Integration API: Build your own OData based integration APIs that can be used with SAP Cloud Platform Integration (SCPI), SAP Process Integration (PI), and SAP Process Orchestration (PO).
- SCPI: Customize an existing SCPI flow that utilizes existing integration APIs.
- Extract Transform Load (ETL): Use a different ETL tool to translate content into ImpEx.
- SAP Cloud Platform Extension Factory (CPEF) also known as project "Kyma": Build or leverage existing microservices built on top of the "Kyma" project.
SAP App Center
The SAP App Center is a fully digital, enterprise marketplace where you can discover, try, buy, manage, and deploy trusted partner applications that extend your existing SAP technology and solutions. By leveraging these existing applications, you can quickly integrate with the vendor without the need to start from scratch. In some cases, you can extend the applications to align with your business requirements, as well as gain additional support. The App Center is typically used for common integrations such as payment, tax and enriching the customer experience.
SAP Commerce, data hub
The SAP Commerce, data hub ("Data Hub") was first introduced in SAP Commerce 5.2 to provide a flexible, scalable, and service-oriented solution to simplify and speed up data integration efforts (import and export) and Master Data Management between SAP Commerce and external data storage solutions and systems, such as Enterprise Resource Planning (ERP), Product Lifecycle Management (PLM), Customer Relationship Management (CRM), and more. To learn how to set up Data Hub for your SAP Commerce Cloud solution, consult this page.
While the use of integration APIs with SAP Cloud Platform Integration (see below) has been announced as the strategic way forward, the SAP Data Hub still plays a very important role, especially since it has many pre-built integrations with many SAP applications. Existing customers and those just starting on their project should be confident that the SAP Data Hub will still receive support. We recommend that you minimize extensive customizations and monitor the progress of the integration APIs on the Roadmap. When they meet your needs, you should include in your solution roadmap migrating over to the integration APIs.
For how-to videos on how to use the SAP Data Hub, please consult the following Enable.cx playlist.
The SAP Commerce, data hub is extensible. You're free to create a new extension for your new items. The biggest challenge typically comes when you need to manipulate existing items, like the target system. An example would be if you would like to change the exportcodes for the ImpEx header or the ImpEx column name (target item exportcode).
Vertical scaling of the SAP Data Hub can be completed by creating additional pools. Horizontal scaling is possible by setting up a cluster. Each instance would have its own data partition in the database. All data loaded into a pool must be directed to the same node. Furthermore, only one node can be active for a pool. If you have multiple nodes for the SAP Data Hub, you would need to ensure that all of the same data types go to the same node. For more details, consult Scaling and High Availability.
Concurrency is only available when there are no dependencies between the data items across the data pools or instances.
Audit and Control
The SAP Data Hub provides complete transparency as well as a full history of data transformations, which are stored in the database attached to the SAP Data Hub instance. Aside from the history, there is a pre-built XML file that allows you to have control over what is being logged. See Configuring Logging in the SAP Data Hub for more information.
When it comes to using the main features of the SAP Data Hub, including analyzing errors, we recommend using the Backoffice Data Hub perspective. Within the user interface, there is a section that allows you to analyze the errors and failures, split by load, composition, and publication. See SAP Data Hub UI - Errors and Failures for more information.
A set of unit tests come with the SAP Data Hub code, as well as an example of an integration test. The scripts and steps to confirm your instance is working, as part of testing your local installation, can be used as a foundation for developing your own tests, and to confirm your instance is working correctly.
Ease of Deployment
The SAP Data Hub is deployed and monitored as part of your SAP Commerce Cloud subscription. Consult this page and its child pages to learn how to configure your code repository and manifest files to ensure your SAP Data Hub instances get deployed with your customizations.
Calls are made asynchronously. Therefore, front-end performance won't be affected if the SAP Data Hub node(s) go down. The integration is decoupled from the front-end.
The configuration of raw, canonical and target items cannot be done at runtime. Modification of the XML should trigger a redeployment and restart of the SAP Data Hub instance(s). We recommend to turn off the processing as part of the deployment process, as any in-progress actions would need to be manually restarted post-deployment. You can perform many different tasks in the Backoffice Perspective, including loading, composing and publishing data, as well as creating new pools and feeds.
Integration API and SAP Cloud Platform Integration
As mentioned above, integration APIs are the strategic way forward for the whole SAP Customer Experience solutions portfolio. With each new release of the SAP Commerce Cloud, additional APIs are being added. The SAP Integration APIs expose a set of interfaces used for data integration with your SAP Commerce Cloud solution with data being passed as Integration Objects. These APIs are built by using and extending the OData protocol. These APIs can be used along by SAP Cloud Platform Integration (SCPI) to connect to other solutions, including SAP Solutions like SAP S/4HANA. You can check the SAP API Business Hub, SAP Customer Experience solutions for the latest APIs available to be used by SCPI. It is important to remember for an integration to work both the inbound and outbound services need to exist. An integration flow may be available in SCPI, but the corresponding inbound or outbound service might not be delivered in SAP Commerce Cloud until the following release.
You can create your own custom Integration Objects as per of the product documentation tutorials. You can then use these custom Integration Objects by making them available as part of an integration API as outlined here.
The use of Integration Objects and integration APIs combined with SCPI allows for a fully scalable integration solution.
Audit and Control
Monitoring and control of the services can be done in SAP Commerce Cloud (for example, Inbound). In addition, the logging of messages can be configured in SCPI. Authentication and authorization to both SAP Commerce Cloud and SCPI can be configured through multiple ways. Please see the respective product documentation.
You can easily test the various flows created in SCPI as well as through the SAP Commerce Cloud back-office application. If you're just looking at testing and monitoring the inbound-outbound services, you can look at some examples here.
Ease of Deployment
The foundations for creating Integration Objects and integrations APIs exist in SAP Commerce Cloud 1808 and need to be enabled. Specific services are being developed for integration with many other SAP products. Consult this section to understand which services are available.
The integration is decoupled from the front-end, so if your processing nodes or SCPI instances go down, your site should remain unaffected.
Unexpected results are logged. You can review both the integration flows in SCPI and the messages that are being sent back and forth, to determine where the issue occurs. You can also use the SAP Commerce Cloud Backoffice UI Tool to monitor and troubleshoot issues as shown in this example.
Cloud Platform Extension Factory (project "Kyma")
The SAP Cloud Platform Extension Factory was announced in October 2018 as the key extensibility layer for the SAP Customer Experience solutions. It leverages the concepts and innovations from project "Kyma" and will be an integral part of SAP Cloud Platform itself, facilitating the management of connected systems and their respective extensions. Although focused mostly on using scalable microservices for extending the SAP Customer Experience solutions, there is nothing to stop these services from being used to integrate with other products. If you're looking to get started, you will need to be on version 1811+. Please see the below video as well as the product documentation:
Although SAP Cloud Platform Extension Factory has been announced, it is still in preview. If you're running an SAP Commerce on your own infrastructure and are looking to get started, you can also download the project "Kyma" source and deploy it on whichever infrastructure you are using for hosting your applications. You can then create your own services, and when SAP Cloud Platform Extension Factory is commercially available, you can decide whether to port your services over or keep supporting your project "Kyma" instance.
The SAP Cloud Platform Extension Factory is set up to be fully extensible. You will be able to create and subscribe to services through the Service Catalog, including SAP Cloud Platform and other third-party services such as AWS, GCP or Azure. Commerce extensibility can be achieved through the Application Connector which provides an event model and API access for connected applications.
The SAP Cloud Platform Extension Factory will be an integral part of the SAP Cloud Platform. Therefore, it is fully scalable to meet your demands. The platform is based on Kubernetes which supports a wide variety of configurations to scale your extensions and services.
Audit and Control
The SAP Cloud Platform Extension Factory leverages the monitoring, tracing and logging features of project "Kyma". Security is also built-in, leveraging the Kubernetes Role Based Authentication and Dex, an open-source OpenID identity provider.
The SAP Cloud Platform Extension Factory leverages the testing mechanisms of the underlying Kubernetes and Helm components. In addition, namespaces (based on Kubernetes namespaces) allow you to partition your runtime cluster in a flexible way to define distinct environments such as development, test, performance, and production.
Ease of Deployment
As stated above, the SAP Cloud Platform Extension Factory will be part of the SAP Cloud Platform, so you'll just be subscribing to services through a service catalog. If you do not want to use the SAP Cloud Platform Extension Factory and all the support and improvements that come with it, you're still free to deploy the open-source project "Kyma" code on your own infrastructure. The connections to the SAP Cloud Platform Extension Factory/project "Kyma" are part of SAP Commerce and need to be enabled and configured to point to the right instances as per the documentation.
Reliability and Stability
When the SAP Cloud Platform Extension Factory is commercially available, it will be governed by the SAP Cloud Platform Services Level Agreement. There may be additional items that are specific to the SAP Cloud Platform Extension Factory, and those would be listed as part of a services description.
Import and export scripts (ImpEx), provided by the ImpEx module, can be used to get data in and out of your SAP Commerce Cloud solution. Although ImpEx was originally implemented to simplify the data setup in the system during development, it has grown into a module that can be used to integrate data.
ImpEx can be executed in a variety of ways:
- Admin Console (HAC)
- ImpEx API
- Service Layer
- Import/Export ImpEx User Guide) and Backoffice Wizard (see
Custom generated ImpEx is typically run manually but sometimes it is run as part of a script or through Hot Folders. More often, the ImpEx is generated and executed as part of a process (see SAP Data Hub example above) or as part of data migration.
ImpEx is one of the core foundations of data import-export in the SAP Commerce Cloud. There is extensive documentation on how to use ImpEx and many options to extend it. The syntax is based on comma-separated values (CSV). Aside from changing the "csv.fieldseparator" property, there isn't a lot that can be changed. That said, your ability to import or export is limited by what you've defined in your type system. Depending on what you put in your header and value rows will determine what gets imported.
The execution of ImpEx can be distributed in many cases. Consult this page to understand how to configure and run distributed ImpEx.
Audit and Control
Running an import or export ImpEx creates a cronjob. Any errors will be captured against this job, as well as in the server logs.
Changes to data from ImpEx execution will not be recorded for audit purposes, as they are typically run using the administrator user. It is important to lock down access to the sections within the Backoffice and Admin Console (HAC) that allow a user to execute an ImpEx, as it can be used to alter any data in the system.
When executing an ImpEx file manually, you can validate the scripts thought the HAC. Be aware that the validation only confirms that things are correct from a syntactical point of view. Whether or not your data is correct or would be accepted (you may have some validation in place) won't be known until the ImpEx is executed. This is the same when executing through the API. You can write tests when using the API, but you would need to ensure the test data being passed in (or mocked) is valid unless you're testing a failure scenario.
Ease of Deployment
Since ImpEx is part of the platform code, it is deployed to all servers as part of your standard deployment. If you've implemented some code that uses the ImpEx API, then you would need to ensure that code is also included in your deployment process.
Executing an ImpEx file manually doesn't require any additional configuration or code changes and is a good option for administrators or those who have access to run them without requiring a code deployment or a restart of the server. This is not true when using ImpEx API for your custom code, as any changes to the execution flow would need a deployment.
A correct ImpEx file will run successfully each time. If a server goes down during the ImpEx process, the job can be restarted, or a new import (or export) can be kicked off manually on a different server.
Any errors in the ImpEx will be logged to the server logs and the cronjob. If using the ImpEx API, you can add any exception handling to react as per your requirements.
When executing manually, there is no way to abort the cronjob. Therefore, if the ImpEx cronjob is taking a long time to complete, the only way to stop it would be to stop the server.
Having exhausted all the options above, you may find that you need to create a custom extension or extend some existing code that is part of your SAP Commerce solution. Although there is nothing inherently wrong with customizations, we often see projects that don't take into account the full cost of a customization. This is why we recommend all the options above to be considered first. They provide simpler forms of extensibility and align with the vision for integrating with SAP Commerce. A direct customization in code not only requires the time for the developer to code, but also ongoing costs such as:
- Bug fixes and support
- Testing (including performance and regression tests with each release)
- Additional infrastructure (if required)
- Impact on upgrades, if not extended in line with recommended practices
When a greenfield customization is required, it may be worth it to re-evaluate the business requirements driving the need for a customization to determine if they can be altered in such a way that any of the other options above can be used.
When it comes to integrating with SAP Commerce Cloud, there are many options, each with its own pros and cons. With each integration, it is best to go through your requirements and establish, using the guidance above, which integration option might work best to achieve your integration.