Task Planning for Hybrid Clouds
Hybrid clouds may enable a more cost effective and faster data processing in scenarios with a constant base load accompanied by periodical peaks. While cloud computing resources offer a greater flexibility and are thereby well suited to handle peak loads, on-premise resources tend to be more cost-effective in constant load situations. Thus for the described data processing scenario, a combination of both worlds appears to be desirable.
Optimizing data processing in a hybrid cloud with respect to cost efficiency and processing time requires a strategy to provision the resources and to schedule the data processing tasks across the resources. Strategies for provisioning resources for web services already exist: They observe the software usage and the resource consumption and dynamically adapt the resources based on the observations. For example, the CPU utilization is watched in order to spawn more resources when a threshold is reached. In the web service situation the load much depends on the usage of the service, which may not be known in advance. In difference to that, in the data processing scenario the given input data provides knowledge (e.g., size of the data to be processed) that is relevant for the resource and task allocation. Further information may be available if there are repeating patterns in the data processing tasks. Ideally, one can use this knowledge to plan most of the resource and task allocation in advance.
In this research project we are investigating the following research questions:
- What is the optimal on-premise resource setup for data processing tasks with respect to the criteria cost efficiency and processing time (time to market) considering available information about the tasks and the data processing history?
- How to optimally schedule tasks on on-premise and cloud computing resources with respect to the criteria cost efficiency and processing time (time to market) considering available information about the data processing tasks and the data processing history?
For the evaluation we may do simulations based on data provided by our industry partner Traveltainment.