Go Live Readiness for SAP Commerce Cloud
Being ready to take your site live, or even subsequent releases, is not just about ensuring you can deploy your code into production. Although deploying the code is important, it is also just as important to address the readiness of your team to support all aspects of deploying the code, as well as being prepared to address the new changes to your business that come from the deployment.
Table of contents
Deployment is an important operational capability. You will need to iteratively test and improve upon deployed code and configuration from the initial stages throughout the launch and post-launch maintenance phases. Ideally, the configuration and deployment activities should be automated. Manual deployment tasks can introduce unknown risks and variability.
The most important consideration for go-live readiness is that you have deployment procedure, and an equivalent staging environment (same code version, catalog size, customer size, etc.) that you can use to test the exact steps in deployment plan that you will be using in production. Once all deployment steps have been executed, you should run through as many tests as possible (automated and manual) to ensure there are no blocking defects/bugs found. If a re-run is required, you should reset everything to be as equivalent to production as possible.
In addition, your deployment processes should consider:
- Think about after launch day. Try to design a good release plan and branching strategies. The deployments and release processes are simpler when there is no production traffic; you should consider these strategies ahead of time.
- Determine the build and deployment workflow. For best results, the deployment workflow should be continuous. It should be able to deliver new code and configuration to a testing environment faster than development teams can provide them, so that this process is not a productivity bottleneck. The builds to development may be based on commits into the version control system, or on a timer basis. However, the builds for QA may need to be manually promoted by development, and deploy jobs may need to be manually triggered by the QA team. A similar approach could be made for deployments to the stage and production environments. The QA team promotes the build, and the Operations team can manually trigger the deploy job. Usually during development phases, before go live, not much emphasis is put on processes on how to deploy to production. You will now need to have a reliable workflow for production releases and deployment. A good article describing the pros and cons of a few options can be found here: http://www.infoq.com/articles/Continous-Delivery-Patterns
- Builds should be promoted from one environment to the next only after all the unit, integration and any automated functional tests have passed.
- For minor bug fixes, changes that do not affect the data model, and do not require system updates, consider implementing a rolling deployment for production. There should be no downtime, deploy one server at a time and restart. Please note, this type of deployment should be proven in a staging environment before executing it on production.
- Major release processes, such as implementing major functionality or upgrading the SAP Commerce platform, should have more structure besides the automated tests in development and manual functional tests. Consider also performing load tests in your staging environment. Refresh your staging environment with data from production before performing these types of load tests so that the test is as realistic as possible.
- You should have a third party run a security/penetration test against your site to ensure any customization you have done has not exposed your site to security vulnerabilities.
- If you have not implemented a zero downtime strategy (see Rolling Updates), make sure you account for a scheduled downtime.
- Maintenance windows and maintenance pages ("Pardon our dust...") should be configured. If you have a Content Delivery Network, you should ensure it is configured there. Enable full redirects to the "Site Down Page" during ‘long’ maintenance. Please note, these procedures should be fully operationalized, and the Operations teams should take part in requirements understanding, testing, and acceptance.
- The development team should prepare and review the deployment procedure. This is where they specify exactly what additional tasks are needed for deployment. Comprehensive documentation should be prepared for each deployment that the QA and Operations teams can easily follow. The documentation can contain, but is not limited to, the following:
- Running specific ImpEx files.
- The options that should be selected in the update running system page.
- Manual changes required.
- Manually running a CronJob or executing a full Solr index.
- Validation tests, after the deployment has completed, should include smoke tests, full transactions, search process, submit returns, third-party integrations, etc.
- Any new purge processes to be set up.
- Database index modifications.
- A careful Rollback strategy should be defined and tested:
- Ensure you have taken a snapshot of your production system prior to deployment.
- For code-only changes, these can be done via the current code deployment process – symbolic link (current v/s release folders), build and deploy.
- Type system rollbacks are more difficult unless you can return to the original type system, assuming you used the Rolling Update On the Cluster process. Type system changes should be carefully vetted out in lower environments (QA). Load testing can be used to mitigate finding new issues.
- In situations where a type system issue makes it to production, a fix will (likely) need to be produced and deployed to production to resolve the issue, as opposed to a Rollback.
- Doing restores of a snapshot can take some time. Ideally an issue would be found in lower environments using good QA and load testing processes. However, each case will need to be examined carefully.
- Solr configurations should be part of the code in your source control.
- The deployed application and its extensions should be the same across all nodes.
- Ensure you have defined the correct IP filter sets and certificates to ensure the right people have access to the right endpoints.
- If you are replacing an existing site with SAP Commerce Cloud, and you plan to use the same domain name, verify that the DNS TTL of the site being replaced is < 5 min (at least at launch).
- Make sure you have an SEO strategy in place. You should have a plan of what will happen with the old sites' SEO indexed URLs. If those URLs patterns need to be redirected, make sure you are not overlapping with existing URLs. You should test these prior to go live. Ideally, your new site will not have overlapping patterns with the old site. One way to do this would be to have the new site in a different context path.
- Make sure you have you tested user migrations from the old to the new site(s). In many cases, the users will need to change the passwords upon the first login. Make sure the logon form has the correct message, so that the user is not confused about this. A blast email to ask the users to change passwords on the launch day may not be a good idea either, as you do not want to draw more traffic to the site than necessary at launch.
- Ensure that monitoring is enabled, and that the right people on your teams are set up to receive the right alerts.
- After deployment to production, validate your business transactions defined in your monitoring system, to make sure that they meet the required SLAs.
- Have a formalized ‘After Incident Report’ if there are issues with deployments, builds, or if outages occur.
Considerations for an SAP Commerce On-Prem Launch
If you are a customer that is running an on-prem version of SAP Commerce, there are additional items that should be accounted for (as you are responsible for maintaining the infrastructure):
- Ensure all important/relevant patches have been applied before launch for all servers in your SAP Commerce stack: app server, web server, SOLR and database.
- Confirm the operating systems your servers run on are supported versions for your SAP Commerce version.
- Consider using configuration management tools to facilitate deployments, such as Salt Stack, Chef, Ansible or Puppet.
- Solr and Web server configurations should be included in your source control, and your deployment process should provide the ability to automatically point to the correct sources.
- If the build process includes a customize ant task, these changes should be reviewed carefully and updated on any patching or upgrading of the SAP Commerce platform.
- Store your build and deploy scripts in source control.
Each node within a cluster has some unique values in your configuration. For example, Node ID’s, database connection strings and tomcat startup parameters may be different per environment. The deploy process should modify these specific configuration files (per node) in each environment (via token replacement for cluster ID, possibly using different property files per environment).
Please review the following wiki pages for more information Cluster Improvements & ID Auto Autodiscovery, Task Service, SAP Commerce Runtime and Optional Configuration, Assigning a Cron Job to a Group of Nodes
More detailed infrastructure considerations are covered in the Infrastructure Considerations for On-Prem SAP Commerce.
Just like pilot's have a flight check to ensure everything is set before the plane takes off, you should also have a checklist to ensure you are ready for go-live. Whether you are using SAP Commerce Cloud, or an on-prem version of SAP Commerce, we have put together an example Go-Live Checklist that outlines many of the recommended checks that should be done prior to launch. These are common areas where we sometimes see projects trip-up. Ideally you will use it throughout your project. At the very least it should be looked at prior to a go/no-go decision for launching your site.
Going live is the culmination of the work your team has done for an individual release. It is a critical step in the process, because it is when your customers start interacting with the work you have done. In this article we covered some of the common recommendations for being prepared for go-live. We also shared a sample Go-Live Checklist that you can use, as well as a sample Deployment Procedure to help you be better prepared for when your site goes live.