
SAP Standard Application Benchmarks
Benchmarking Procedure
The benchmarking procedure is standardized and well defined. It is monitored by the SAP benchmark council made up of representatives of SAP and technology partners involved in benchmarking.
Originally introduced to strengthen quality assurance, the SAP Standard Application Benchmarks can also be used to test and verify scalability, concurrency, power efficiency, and multi-user behavior of system software components, RDBMS, and business applications.
Over the years, specific sets of configurations have become the de facto standard, independent of the hardware vendor, operating system, or database version. To simplify certification, these configuration examples are used as a default. Any deviation from this default needs to be stated explicitly.
However, as hardware and technology architectures evolve, the defaults are also subject to changes. The default configurations and major changes are described below.
Codepage
To come in line with the overall move of SAP applications to Unicode, all SAP Standard Application Benchmarks use Unicode from June 1, 2010 onwards. Prior to that, some of the benchmarks used the Latin-1 code page, and if Unicode was used in these benchmarks, a special mention was made on the benchmark certificate. For other benchmarks, e.g. the Enterprise Portal benchmarks, there was no need to mention Unicode explicitly on the certificate as no other option was possible.
Note that for the ERP benchmarks, in particular the SD benchmark, Unicode was explicitly mentioned on the certificate for a transition period of one year, i.e. throughout 2009.
32-bit/64-bit software version
Early SAP software releases supported only 32-bit versions of operating systems and databases. With improved hardware capabilities 64-bit versions of OS and DB software are supported and thus the SAP software can run on 64-bit versions of OS and DB as well. SAP software on 64-bit Unix versions has been available since 1999, and became default in 2001. 64-bit Windows and Linux versions became available in 2002.
In the benchmark environment whether 32-bit or 64-bit versions are used, strongly depends on the SAP release and the processor type. As a general rule, you could say that as of mySAP ERP 2005 (ECC 6.0) and SAP NetWeaver 2004s the default used is 64-bit. Before that time, roughly since 1999 there was a coexistence of benchmarks run on either version, predominantly depending on processor type. In this interim period there were processors that used only 32-bit, only 64-bit, and some that supported both. As a consequence, the benchmark certificates detailed the use of 32-bit and 64-bit, respectively, if the processor supported both versions. When a benchmark was run on a processor that supported only one of the versions, it was not specifically mentioned, as there was no option for the submitter.
Hardware configuration
The benchmark certificates describe the total server configuration of a benchmark setup. This includes all active servers, and for each active server the total number of processors, cores, threads as well as the total amount of memory, no matter whether this was used during the actual benchmark or not.
Also, in the benchmark certificates, "segmentation" refers to the kind of partitioning, virtualization, or other technology that may have been used to run more than one operating system image on a physical server.
The details of the actual benchmark configuration can be obtained from SAP or the respective technology partner.
Number of processors
With the development of new processor architectures using multi-core and multi-threading technology, the n-way SMP description was no longer sufficient to describe the processing units. Since August 2005 the certificate states the total number of processors / cores / threads. These numbers describe the maximum of the hardware no matter whether all processors / cores / threads are being used or not. The actual configuration during the benchmark may be different.
Note: The terms CPU, processors, etc. might be used differently by the individual vendor.
Virtualized platforms
When a benchmark was run on a virtualized environment the certificate can be extended to include the virtualization technology. The extended certificate states the virtualization technology, the entitlement or the number of assigned processing units to the virtualized environment, and the CPU utilization within the virtualized environment.
Note: The server configuration on the benchmark certificate describes the complete hardware, whether or not it is used. For example, the actual configuration used by the virtualized environment during the benchmark may be different from the complete hardware.
Identical Servers
In cases where vendors use one certification for identical servers these servers are detailed on the benchmark certificate. In addition see the statements on the respective hardware vendor’s Web page:
2007017, 2007021
2007030, 2007031
2007041, 2007043
2007075, 2008022
2007026, 2007051
2007059, 2008021
2008040, 2008042
2008058, 2008062
2008075, 2009026
2009038, 2009046
2021001, 2021002
2022020, 2022021
2023043, 2023049
2023044, 2023050
2023045, 2023051
2023046, 2023052
2023047, 2023053
2023048, 2023054
2024004, 2024007
2024005, 2024008
2024006, 2024009
2024015, 2024016
2024019, 2024020
2024023, 2024024
2024026, 2024027
2024029, 2024030
2024032, 2024033
2024035, 2024036
2024038, 2024039
Obtain more information
According to the full disclosure policy, you can obtain more information about a particular benchmark from SAP and the respective technology partner.