\section{Benefits of the Approach}
\label{sec:Discussion}


In this section, we discuss the benefits of our approach. This covers its optimization potential for workflow-internal data processing, generality issues, and additional opportunities to extend current workflow execution environments by further data management functionalities.



\subsection{Optimization Potential for Workflow-Internal Data Processing}
\label{sec:Discussion:Efficiency}


In previous control-flow-oriented workflow environments, the workflow-internal data processing could only be executed by the workflow runtime. The data processing optimizer provides a new option to shift data processing to the integrated DBS. This offers a great potential for improved performance and reliability of the workflow-internal data processing in data-intensive workflows. The main reason is that the mature database technology can more efficiently and reliably deal with large volumes of data and the complex as well as frequent data processing operations that are involved in such workflows. Furthermore, we can reduce the amount of data transfered from the runtime to the local DBS because all data may be directly stored in the DBS without the indirection over the runtime.

When multiple workflow instances run concurrently, the execution of their local data processing within the workflow runtime typically leads to an increased main memory consumption of this runtime. It may get too busy or even overloaded. In such a case, the runtime pauses some of the workflow instances and stores all relevant data in the DBS in order to resume the paused instances at a later point in time. This leads to an increased execution time for some workflow instances and to reduced overall throughput. The database technology is a mature technology regarding concurrent execution of processes and regarding an efficient main memory allocation. Following our approach, we can push parts of the workflow processing to the local DBS. This may lead to a more balanced utilization of all involved system components, in particular to a more efficient overall main memory utilization, \mbox{and thus} to a faster execution and a higher throughput of concurrent workflow instances. Furthermore, the more efficient main memory utilization may reduce the number of main memory overloads in the workflow system and thus enhance the reliability of workflow execution.

On the other hand, processing data in the main memory of the workflow runtime leads to less costs of data access and the runtime may exploit customized internal data structures that are tailor-made to the workflow-internal data processing. For less data-intensive workflows as well as for small data sizes, \eg for orchestration workflows, the data processing optimizer may decide to leave the responsibility for data processing to the workflow runtime. To avoid the additional overhead for this optimization decision at workflow execution time, this decision may be made during deployment time. In \mbox{Section \ref{sec:Evaluation}}, we evaluate some of the arguments discussed in this section.



\subsection{Generic Workflow Management}
\label{sec:Discussion:Generality}


Due to the above-mentioned optimization potential, our approach broadens the set of data-intensive workflows that may be described in terms of control flow. Since this approach is also applicable to small or less data-intensive workflows as well as for small data sizes and since we extend a common architecture for control-flow-oriented workflow environments, it is suitable for orchestration workflows as well. This results in a common level of abstraction, \ie the control flow, for all kinds of workflows, which entails several benefits. The common level of abstraction enables further optimizations over the whole spectrum from the workflow level to the data level and across all workflow classes \cite{PGM/F_Approach}, \cite{Boehm}. Furthermore, it leads to an integrated tool support, \ie all kinds of processes can be modeled using standard languages such as BPEL and executed within common tools or environments. Altogether, this forms a generic solution being applicable to all workflow classes depicted in \mbox{Figure \ref{fig:WorkflowClassification}}, in multiple domains of scientific or business applications, and even in the combination of scientific and business applications \cite{SIMPL}, \cite{CDML}.

%In particular for BPEL, there exist plenty of open-source or free tools as well as de-facto standards for service technologies, \eg Web Services. These de-facto standards are key enablers to integrate heterogeneous applications and other resources such as data sources.

%To enable the integration of business workflows with simulation workflows, \eg a crash test simulation embedded in a vehicle development and testing process \cite{CDML}, companies are interested in using the same language for both kinds of processes.

%In particular for BPEL, there exist plenty of open-source or free tools as well as de-facto standards for service technologies, \eg Web Services. These de-facto standards are key enablers to integrate heterogeneous applications and other resources such as data sources.


\subsection{Increased Extensibility of Workflow Execution Environments}
\label{sec:Discussion:Exentesibility}

%Our approach enables a highly increased extensibility of current workflow execution environments by additional data management functionalities.

The new possibility to intensively use the local database system in our new architecture of workflow execution environments enables a highly increased extensibility of such environments by additional data management functionalities. For example, current database systems offer useful capabilities for workflows that a workflow runtime does not offer or for which it would have to be extended in a complicated way. %The logging information of a database system, for example, may help to provide an improved workflow auditing or provenance support to enable the traceability and reproducibility of workflow executions and their applications. Furthermore, some database systems offer specific functions suitable for certain domains or problems. 
A geographic information system may support operations tailor-made to spatial problems. Such operations can be used in a simulation that determines spatial changes of the structure of a car in a crash test \cite{CDML}. %Using our extended architecture, we can more easily exploit such capabilities of current database systems.
Furthermore, we can easily extend the new architecture to support global data structures that may serve as a cache for data needed by several workflow instances or for data that is used for synchronization purposes. Traditional approaches define such global data structures in the local DBS that workflows access following the separate or hybrid approaches depicted in \mbox{Table \ref{table:DM4Workflows:Dimensions}}. In contrast, the extended architecture allows to provide some kind of shared variables. The main advantage is that these shared variables are modeled and used transparently, \ie in the same way as conventional process variables. The local DBS already offers functionality needed for the management of shared variables. This includes guaranteed persistence and life-cycle management of variables and their contents as well as concurrency control mechanisms that are needed when multiple workflow instances access the same shared variables.


%: (1) functions for the life-cycle management of variables, \eg data definition and manipulation queries that enable to create or drop variable schemas and to insert or delete new or existing variable instances, (2) guaranteed persistence of the variable contents to ensure that these contents are valid during the execution of several workflow instances, and \mbox{(3) concurrency} control mechanisms that are needed when multiple workflow instances access the same shared variables.

Many data-intensive workflows integrate highly heterogeneous external data and load parts of them into their workflow context. Our approach serves as starting point to employ different kinds of data processing facilities for the workflow-internal data processing that are tailor-made to specific workflows. For example, some workflows might benefit from facilities for semi-structured or unstructured data such as key-value stores. For data or workflows not having high persistence requirements, we may also employ main-memory-based database systems to further improve performance. Furthermore, we can use data integration solutions to describe external and local data in a transparent way.


%Furthermore, we can more easily extend the new architecture by some kind of shared variables that several workflow instances may access. Such shared variables may serve as a cache for data needed by these workflow instances or for data that is used to synchronize several instances. Traditional approaches define global data structures or containers in the DBS that workflows access via the separate or hybrid approaches depicted in \mbox{Table \ref{table:DM4Workflows:Dimensions}}. In contrast, these shared variables can be modeled and used transparently, \ie in the same way as conventional process variables. The local DBS already offers functionality needed for the management of shared variables: (1) functions for the life-cycle management of variables, \eg data definition and manipulation queries that enable to create or drop variable schemas and to insert or delete new or existing variable instances, (2) guaranteed persistence of the variable contents to ensure that these contents are valid during the execution of several workflow instances, and \mbox{(3) concurrency} control mechanisms that are needed when multiple workflow instances access the same shared variables.

%Compared to the previous architecture for workflow environments, we can more easily extend the new architecture by some kind of shared variables that several workflow instances may access. Such shared variables may serve as a cache for data needed by these workflow instances or for data that is used to synchronize several instances. Traditional approaches define global data structures or containers in the DBS that workflows access via the separate or hybrid approaches depicted in \mbox{Table \ref{table:DM4Workflows:Dimensions}}. In contrast, these shared variables can be modeled and used transparently, \ie in the same way as conventional process variables. The local DBS already offers functionality needed for the management of shared variables: (1) functions for the life-cycle management of variables, \eg data definition and manipulation queries that enable to create or drop variable schemas and to insert or delete new or existing variable instances, (2) guaranteed persistence of the variable contents to ensure that these contents are valid during the execution of several workflow instances, and \mbox{(3) concurrency} control mechanisms that are needed when multiple workflow instances access the same shared variables.

%Many data-intensive workflows integrate highly heterogeneous external data and load them into their workflow context. Our approach serves as starting point to employ different kinds of data processing facilities for the workflow-internal data processing that are tailor-made to specific external data sources. For example, traditional BPEL-based workflow environments define all their process variables in an XML-based format. We now can extend these environments by variables that are organized in a relational format, and not in XML anymore. This is beneficial for data-intensive workflows that integrate only set-oriented data, \eg coming from SQL database tables or from files based on comma-separated values, and that carry out set-oriented operations on these data \cite{SIMPL}. %The major benefits are that (1) we do not need to convert the external data into an XML-based format or vice versa and (2) that the relational database facilities execute typical set-oriented operations more efficiently when the data are described in a relational format
%Other workflows might benefit from facilities for semi-structured or unstructured data that go beyond traditional SQL or XML processing such as key-value stores. For data or workflows not having high persistence requirements, \eg for very short-running workflows, we may also employ main-memory-based database systems to further improve performance. Furthermore, we can use federated information systems or other data integration solutions to distribute data among different data sources and to describe external and local data in a transparent way. In the same sense, a data warehouse system might be integrated into the workflow environment to enable data analysis workflows accessing the data warehouse data like every other data within their workflow context. Altogether, our approach enables a highly increased extensibility of current workflow execution environments by additional data management functionalities.