Towards a Statistical Evaluation of PigLatin Joins

  • Renato Javier M. Mogrovejo
  • José Maria Monteiro
  • Javam C. Machado
  • Carlos Juliano M. Viana
  • Sérgio Lifschitz PUC-Rio
Keywords: Cloud databases, MapReduce, PigLatin, Hadoop, Join Evaluation, Multiple Regression

Abstract

There are different scalable data management solutions which can take advantage of cloud features making them more attractive for a deployment in such environments. One of the most critical operations in data processing is joining large data sets. This is one of the most expensive and hardest operations to optimize. We are mainly  concerned here with join operations expressed in PigLatin, an abstract query language for a high-level platform, called Pig, which creates MapReduce programs with Hadoop. In this work we explore statistical methods, namely multiple regression analysis, in order to predict three distinct join types execution times, comparing them with actual running times.
Published
2013-09-17
Section
SBBD Articles