SAP Standard Application Benchmarks
Benefits of Benchmarking
SAP Standard Application Benchmarks help customers and partners find the appropriate hardware configuration for their IT solutions. Working in concert, SAP and our hardware partners developed SAP Standard Application Benchmarks to test the hardware and database performance of SAP applications and components.
Released for technology partners, benchmarks provide basic sizing recommendations to customers by placing a substantial load upon a system during the testing of new hardware, system software components, and relational database management systems (RDBMS). All performance data relevant to system, user, and business applications are monitored during a benchmark run and can be used to compare platforms.
List of Benchmark Benefits
Running a SAP Standard Application Benchmark
During the benchmark run, all relevant performance parameters of an SAP system are monitored, for example, CPU utilization, memory consumption, the I/O system, network load, as well as functional errors and system availability. As a result, "special tunings" to achieve better results are impossible.
The output files are thoroughly analyzed for any divergence from the expected behavior. These technology checks result in a benchmark certification. The benchmark run takes at least 15 minutes. The throughput results are then extrapolated to one hour. Each SAP Standard Application Benchmark consists of a number of script files that simulate typical and popular transactions and workflow in a particular business scenario. The benchmark also has a predefined SAP client database that contains sample company data against which the benchmark is run.
Benchmark Characteristics
Customers can benefit from the benchmark results in various ways. For example, benchmark results illuminate the scalability and manageability of large installations. The benchmark results:
- Allow users to compare different platforms)
- Enable Proof-of-concepts scenarios)
- Provide an outlook for future performance levels (new platforms, new servers, and so on)
- Provide basic information to configure and size SAP Business Suite
Possible Configurations
In general, SAP's technology partners – particularly hardware vendors – run the benchmarks, testing different business application scenarios on a specific hardware. The primary objective is to use the results to determine an optimal hardware configuration for a customer system. In addition, the marketing departments use benchmark results to support sales.
SAP Benchmark Council
Established in 1995, the SAP Benchmark Council monitors all benchmarking activities and provides standards for publishing benchmark results in the SAP environment. Representatives of SAP and technology partners, such as hardware and database vendors involved in benchmarking, make up the SAP Benchmark Council.
On behalf of the SAP Benchmark Council, SAP defines and controls the content of the benchmarks and establishes rules that determine the testing procedure during benchmarking. Technology partners submit the benchmark results to SAP for certification. There are a number of rules and guidelines that must be followed to obtain a certification. The certificate is the basis for any benchmark publication by the technology partners.
Some of the benchmark rules include:
- The average dialog response time must be below one second
- Each Benchmark User must have a fixed think time of ten seconds
- The tested hardware and the release combination of operating system, database, and SAP releases must be generally available for customers within six months after the certification of the benchmark
- The following SAP software settings can be changed during benchmarking:
- The number of instances
- The number of dialog work processes
- The number of update processes
- The size of SAP buffers
In general, only those tuning features are allowed that can also be used and applied in a productive customer installation.
Publications, Policy and Violations
Read the SAP Standard Application Benchmark Publication Policy to learn about practices for the publication of information related to SAP Standard Application Benchmarks.
Publication violations recognized by the SAP Benchmark Council are listed in the table below. The table includes a high-level description of the violation, corrections suggested by the workgroup, and reasons for accepting the ruling.
| Date of Decision | Violating Company Name | Description of the Violations | Suggested Correction Actions | Clarification of the Workgroup |
|---|---|---|---|---|
| Currently no violations | n/a | n/a | n/a | n/a |