\section{Introduction}
\label{sec:Intr}


Workflows have long been used to meet the needs of IT support for business processes. They are compositions of tasks by means of causal or data dependencies that are carried out on a computer using a workflow management system (WfMS) \cite{ProductionWorkflows}. Recently, the workflow technology has found application in the area of scientific computing and simulations for implementing complex scientific applications and the term \h{scientific workflow} has been coined \cite{Workflows4e-Science}. %Simulations, as a subset of scientific applications, are typically compositions of complex calculations and data management tasks, which makes them good candidates for the realization as workflows. For instance, partial differential equations have to be solved to determine temporal or spatial changes of simulated objects, \eg of the structure of a car in a crash test.
Many of these scientific workflows treat data and their processing as first-class citizens. Such data-intensive workflows typically process huge amounts of possibly distributed and heterogeneous data and carry out a multiplicity of complex data processing steps that may be directly reflected in the workflow definitions \cite{DM_Challenges_SW}. %The data may be processed on external resources or even within the workflow.
Examples are workflows for analyzing previously generated data, for reducing models of a simulation problem, and for protein modeling and analysis.
%Examples are workflows for analyzing previously generated data \cite{SciDataAnalysis}, for reducing models of a simulation problem \cite{ModelReduction}, and for protein or genome modeling and analysis \cite{GenomeAnalysis}.

Traditionally, the descriptions of data-intensive workflows focus on their data flow instead of the control flow due to several benefits of this kind of workflow descriptions. This particularly includes optimization opportunities for massive data processing \cite{Lud_SDM}, \cite{VDAL4MapReduce}, \cite{HPC4ScientificWorkflows}. Nevertheless, control-flow-oriented workflow languages are increasingly adapted to the needs of data-intensive workflows since they have some advantages for modeling and executing such workflows as well \cite{Boehm}, \cite{AdaptBPEL2ScWF}. They provide a common level of abstraction for both data-intensive workflows and classical orchestration workflows such as business workflows. This enables a comprehensive re-engineering or optimization across all kinds of workflows \cite{BIA}, \cite{PGM/F_Approach}, \cite{Boehm}. Furthermore, it results in a generic solution, \ie being applicable in multiple domains of scientific or business applications and also for workflows combining multiple domains \cite{SIMPL}.%, \cite{CDML}.% However, the problem still remains that workflows described in control-flow-oriented languages tend to be less efficient for data-intensive processes compared to specialized data-flow-oriented approaches.

%This includes optimization opportunities, \eg pipeline parallelism between individual workflow tasks \cite{Lud_SDM} and the usage of parallel data processing infrastructures \cite{VDAL4MapReduce}, \cite{HPC4ScientificWorkflows}. These opportunities are particularly relevant when workflows locally process huge amounts of data within the workflow tasks.

\begin{figure}[tb]
	\centering
		\includegraphics[width=0.73\textwidth]{Introduction/HighLevelDataProcessingArchitecture.pdf}
	\caption{Data processing during workflow execution phase.}
	\label{fig:DataProcessingInWorkflows}
\end{figure}

The life cycle of a scientific workflow typically consists of the phases workflow design, workflow preparation or deployment, workflow execution, and post-execution analysis where result data, provenance data, and other metadata are archived and analyzed \cite{Lud_BPM2009}, \cite{DM_Challenges_SW}. Figure \ref{fig:DataProcessingInWorkflows} presents a high-level view of a workflow processing architecture including a workflow execution environment, external resources and applications, and a provenance framework to capture, manage, and store provenance data \cite{Provenance}.  In this paper, we focus on the workflow execution phase, in particular on the processing of application data during this phase. For some data sets, it may be appropriate or even inevitable to process them directly in the workflow execution environment. %For smaller data sets, \eg up to a few hundred MBs or a few GBs, data may be processed directly within the workflow execution environment and within the workflow context whenever this is appropriate or inevitable.
The other option, which is particularly interesting for larger data sets, is to outsource data processing to external resources or applications. Here, many approaches exist that deal with access mechanisms for proprietary data sources or legacy applications \cite{SIMPL}, \cite{SimTech-sWfMS} or focus on specific optimization opportunities, \eg late binding of resources or re-engineering of workflow models \cite{PGM/F_Approach}. However, optimization opportunities for processing high amounts of data directly in the workflow context, in particular for control-flow-oriented workflow languages, have largely been neglected in previous work.

%However, processing huge amounts of data within the workflow context is especially important for data-intensive workflows that are modeled using control-flow-oriented workflow languages, but specific optimization opportunities for this kind of data processing have largely been neglected in previous work.

In this paper, we focus exactly on this setting and show how to extend the system architecture to transparently improve the local data processing in a workflow execution environment. We introduce various techniques to partition the local data processing tasks in an improved way. These data processing tasks are either assigned to the workflow runtime or to a tightly integrated local database engine thus exploiting its data processing capabilities. We evaluate the effectiveness of these techniques \mbox{by means} of various test scenarios and come up with possible indicators when to use either the workflow runtime or the local database engine. Altogether, this offers a great potential for improved performance and reliability of local data processing in data-intensive workflows and it broadens the set of such workflows that may be described in terms of control flow. It forms the desired generic solution for all kinds of workflows in multiple domains. Furthermore, it increases the extensibility of current workflow execution environments by additional data management functionalities, \eg the usage of a geographic information system may support operations tailor-made to spatial problems.

%In this paper, we provide a new kind of optimization targeted at this setting. We show how to extend a typical architecture of control-flow-oriented workflow execution environments to transparently optimize the local data processing in workflows. Furthermore, we introduce various techniques that partition the local data processing tasks to be performed during workflow execution in an improved way. These data processing tasks are either assigned to the workflow engine or to a tightly integrated local database engine thus exploiting its data processing capabilities. We evaluate the effectiveness of the optimization techniques by means of various test scenarios and state suggestions when to use the workflow engine or the local database engine. Altogether, this offers a great potential for improved performance and reliability of the local data processing in data-intensive workflows and broadens the set of such workflows that may be described in terms of control flow. It forms the desired generic solution for all kinds of workflows in multiple domains. Furthermore, it increases the extensibility of current workflow execution environments by additional data management functionalities, \eg the usage of a geographic information system may support operations tailor-made to spatial problems.

% \eg the usage of data integration solutions to describe external and local data in a transparent way.

The rest of this paper is organized as follows: \mbox{Section \ref{sec:prel}} deals with the state of the art of data processing in workflows. In \mbox{Section \ref{sec:OptDP}}, we introduce our approach and discuss its benefits in \mbox{Section \ref{sec:Discussion}}. Afterwards, \mbox{Section \ref{sec:Evaluation}} shows the results of our evaluation. We highlight major related work in \mbox{Section \ref{sec:RW}}, and \mbox{Section \ref{sec:Conclusion}} concludes and gives a brief outlook for future research.


%% old Version %%

%In this paper, we address this problem by providing a new kind of optimization targeted at data-intensive workflows that are described in control-flow-oriented languages. Firstly, Section \ref{sec:prel} deals with the state of the art of data processing in such workflows. In Section \ref{sec:OptDP}, we show how to extend a typical architecture of control-flow-oriented workflow execution environments to transparently optimize the local data processing in workflows. Furthermore, we introduce various techniques that partition the data processing tasks to be performed during workflow execution in an improved way. These data processing tasks are either assigned to the workflow engine or to a tightly integrated local database engine thus exploiting its data processing capabilities. After discussing the benefits of our optimization approach in \mbox{Section \ref{sec:Discussion}}, we evaluate the effectiveness of the optimization techniques by means of various test scenarios and state suggestions when to use the workflow engine or the local database engine in \mbox{Section \ref{sec:Evaluation}}. All this will entail that an increasing number of data-intensive workflows may be defined in terms of control flow. Subsequently, we highlight major previous related work in Section \ref{sec:RW}. Finally, \mbox{Section \ref{sec:Conclusion}} concludes and gives a brief outlook for future research.