OBAS: An OLAP Benchmark for Analysis Services

Abstract

Some benchmarks have been proposed for helping in the design of data warehouse performance tests, such as TPC-H, which was extended by SSB (Star Schema Benchmark). However, we identified the lack of a benchmark for analysis services (OLAP). In this article, we specify a benchmark for analysis services, called OBAS that extends SSB's multidimensional schema and creates varying-sized data cubes. It enables the evaluation of analysis services through the specification of running configuration parameters, a set of MDX queries and a set of service performance metrics, i.e. response time, rate of execution and service reliability. Using OBAS, we compared two analysis services that adopt XMLA as communication interface: the SQL Server Analysis Services (SSAS) and the Pentaho Mondrian. The results showed that OBAS helped in the comparison of these services with respect to performance and reliability regarding the number of threads.