Comprehensive Software Reviews to make better IT decisions

Sr hero 001 Sr hero 002 Sr hero 003 Sr hero 004

SAP S4/HANA and HANA Licensing Series – Part II: HANA DB Hype?

HANA is SAP’s first serious entry into the database market segment. HANA is touted as being superior in performance when working with SAP applications vs. alternate database solutions, but objective benchmarks are lacking. Additionally, the adoption of S4/HANA necessitates adoption of the HANA DB as it is the only certified solution for S4/HANA ERP.

It didn’t take a lot of digging, even for a non-technical analyst in the Vendor Practice such as myself, to uncover significant performance characteristics between a columnar database and a row-oriented database. HANA is a columnar database. In short, columnar databases generally provide performance gains when used for analytics based processing tasks such as those conducted in a data warehouse environment. Whereas row-oriented databases will perform at a higher level when used for the processing of transactional data.

Since ERP systems are first and foremost transactional systems, it would be reasonable to question SAP’s claims that HANA will outperform competitor database products. In these cases, an objective set of benchmarks should set the record straight in quick order. And this is where the problem lies as there simply seems to be a dearth of third-party data comparing HANA database performance vs. the likes of Oracle’s 12c or Microsoft’s SQL Server Enterprise offerings.

SAP’s current lack of transparency into transactional system benchmarking has me looking deeper into how they do benchmark HANA. Of initial interest is that the only benchmark for HANA published by SAP is the BWAML (previously BW-EML) which are related to the BW application. Fitting that the benchmark published matches the strengths of a columnar database, especially in light of the fact that no benchmarks are published for any of the applications running on databases:

It’s easy to see that for SD – Sales and Distribution modules, benchmarks are available on all manner of databases, all except HANA.

Let’s briefly cover some other key deficiencies in SAP’s HANA database benchmarking methodology:

  1. Where are the benchmarks outside of BW? SAP’s ecosystem of consulting partners is legendary in size and scale; they essentially operate as an extension of SAP and frequently recommend SAP solutions to their client. This clearly allows SAP to exert significant control on the types of information, benchmark and otherwise, that may reflect poorly on SAP.
  2. Benchmark objectivity is non-existent when judging yourself. In SAP’s own words:
    "SAP Standard Application Benchmarks help customers and partners find the appropriate hardware configuration for their IT solutions. Working in concert, SAP and our hardware partners developed SAP Standard Application Benchmarks to test the hardware and database performance of SAP applications and components."
    It should be a foregone conclusion that without third-party benchmarks provided by SAP or encouraged by SAP, SAP’s results should be taken with a grain of salt.
  3. BWAML benchmark is based on outdated Info-Cubes. Presumably with a columnar database the need for BW Info-Cubes is eliminated. Why then would SAP run a HANA DB benchmark predicated on Info-Cubes, which are a set of devices purpose built to perform best upon a row-oriented DB? Most organizations have a lot of overhead and refinement built into their Info-Cubes and may not be ready to throw all that work aside quite yet.

Our Take

Remember, SAP has a hard “no discount” policy on its HANA database product licenses, making this one expensive line item. It is no secret that SAP and all other enterprise software vendors are publishing benchmarks for the sole reason to promote and increase sales of their products.

As such, it is critical to look beneath the surface of all the glamorous and successful benchmark results to see what was not measured and ask why. The lack of independent or third-party benchmarks should be another red flag that sews legitimate doubt and compels further scrutiny by the customer. Buyer beware! IT buyers should think twice prior to taking SAP’s or its consulting partners’ word for HANA DB performance. Request the benchmarks comparing HANA with other vendor database solutions running on the same application with like parameters. If you can’t obtain this data, proceed at your own risk.

Want to Know More?

SAP S4/HANA and HANA Licensing Series – Part I: The Forced “Upgrade"