gt;gt; Chapter One gt; gt;Introductiongt; gt; gt; This book presents a novel probabilistic temporal framework to address the limitations of conventional temporal research and the new challenges for lifecycle support of time-constrained e-science applications in cloud workflow systems. The novel research reported in this book is concerned with the investigation of how to deliver high temporal Quality of Service (QoS) from the perspective of cost-effectiveness, especially at workflow run-time. A set of new concepts, innovative strategies and algorithms are designed and developed to support temporal QoS over the whole lifecycles of scientific cloud workflow applications. Case study, comparisons, quantitative evaluations and/or theoretical proofs are conducted for each component of the temporal framework. This would demonstrate that with our new concepts, innovative strategies and algorithms, we can significantly improve the overall temporal QoS in scientific cloud workflow systems. gt; This chapter introduces the background, motivations and key issues of this research. It is organised as follows. gt;Section 1.1gt; gives a brief introduction to temporal QoS in cloud workflow systems. gt;Section 1.2gt; presents a motivating example from a scientific application area. gt;Section 1.3gt; outlines the key issues of this research. Finally, gt;Section 1.4gt; presents an overview of the remainder of this book. gt; gt; gt;1.1 Temporal QoS in Scientific Cloud Workflow Systemsgt; gt; Cloud computing is a latest market-oriented computing paradigm. Gartner estimated the revenue of the Worldwide cloud services is $58.6 billion in 2009, and it is forecast to reach $68.3 billion in 2010, and projected to reach $148.8 billion in 2014. International governments such as the United States, the United Kingdom, Canada, Australian and New Zealand governments take cloud services as an opportunity to improve business outcomes through eliminating redundancy, increasing agility and providing ICT services at a potentially cheaper cost. Cloud refers to a variety of services available on the Internet that deliver computing functionality on the service provider's infrastructure. A cloud is a pool of virtualised computer resources and may actually be hosted on such as grid or utility computing environments. It has many potential advantages which include the ability to scale to meet changing user demands quickly; separation of infrastructure maintenance duties from users; location of infrastructure in areas with lower costs for real estate and electricity; sharing of peak-load capacity among a large pool of users and so on. Given the recent popularity of cloud computing, and more importantly the appealing applicability of cloud computing to the scenario of data and computation intensive scientific workflow applications, there is an increasing demand to investigate scientific cloud workflow systems. For example, scientific cloud workflow systems can support many complex e-science applications such as climate modelling, earthquake modelling, weather forecasting, disaster recovery simulation, astrophysics and high energy physics. These scientific processes can be modelled or redesigned as scientific cloud workflow specifications (consisting of such things as workflow task definitions, process structures and QoS constraints) at the build-time modelling stage. The specifications may contain a large number of computation and data-intensive activities and their non-functional requirements such as QoS constraints on time and cost. Then, at the run-time execution stage, with the support of cloud workflow execution functionalities, such as workflow scheduling, load balancing and temporal verification, cloud workflow instances are executed by employing the supercomputing and data-sharing ability of the underlying cloud computing infrastructures with satisfactory QoS. gt; One of the research issues for cloud workflow systems is how to deliver high QoS. QoS is of great significance to stakeholders, namely service users and providers. On one hand, low QoS may result in dissatisfaction and even investment loss of service users; on the other hand, low QoS may risk the service providers of out-of-business since it decreases the loyalty of service users. QoS requirements are usually specified as quantitative or qualitative QoS constraints in cloud workflow specifications. Generally speaking, the major workflow QoS constraints include five dimensions, namely time, cost, fidelity, reliability and security. Among them, time, as one of the most general QoS constraints and basic measurements for system performance, attracts many researchers and practitioners. For example, a daily weather forecast scientific cloud workflow, which deals with the collection and processing of large volumes of meteorological data, has to be finished before the broadcasting of a weather forecast programme every day at, for instance, 6:00 p.m. Clearly, if the execution time of workflow applications exceeds their temporal constraints, the consequence will usually be unacceptable to all stakeholders. To ensure on-time completion of these workflow applications, sophisticated strategies need to be designed to support high temporal QoS in scientific cloud workflow systems. gt; At present, the main tools for workflow temporal QoS support are temporal checkpoint selection and temporal verification which deal with the monitoring of workflow execution against specific temporal constraints and the detection of temporal violations. However, to deliver high temporal QoS in scientific cloud workflow systems, a comprehensive temporal framework which can support the whole lifecycles, viz. from build-time modelling stage to run-time execution stage, of time-constrained scientific cloud workflow applications needs to be fully investigated. gt; gt; gt;1.2 Motivating Example and Problem Analysisgt; gt; gt; In this section, we present an example in Astrophysics to analyse the problem for temporal QoS support in scientific cloud workflow systems. gt; gt; gt;gt;1.2.1 Motivating Examplegt;gt; gt; The Parkes Radio Telescope (gt;http://www.parkes.atnf.csiro.au/gt;), one of the most famous radio telescopes in the world, serves institutions around the world. The Swinburne Astrophysics group (gt;http://astronomy.swinburne.edu.au/gt;) has been conducting pulsar searching surveys (gt;http://astronomy.swin.edu.au/pulsar/gt;) based on the observation data from the Parkes Radio Telescope. The Parkes Multibeam Pulsar Survey is one of the most successful pulsar surveys to date. The pulsar searching process is a typical scientific workflow which involves a large number of data- and computation-intensive activities. For a single searching process, the average data volume (not including the raw stream data from the telescope) is over 4 terabytes and the average execution time is about 23 hours on Swinburne's high-performance supercomputing facility (gt;http://astronomy.swinburne.edu.au/ supercomputing/gt;). gt; For the convenience of discussion, as depicted in gt;Figure 1.1gt;, we illustrate only the high-level workflow structure and focus on one path out of the total of 13 parallel paths for different beams (the other parallel paths are of similar nature and denoted by cloud symbols). The average durations (normally with large variances) for high-level activities (those with sub-processes underneath) and three temporal constraints are also presented for illustration. Given the running schedules of Swinburne's supercomputers and the observing schedules for the Parkes Radio Telescope (gt;http://www.parkes.atnf.csiro.au/observing/schedules/gt;), an entire pulsar searching process, i.e. a workflow instance, should normally be completed in 1 day, i.e. an overall temporal constraint of 24 hours, denoted as gt;U(SW)gt; in gt;Figure 1.1.gt; gt; Generally speaking, there are three main steps in the pulsar searching workflow. The first step is data collection (about 1 hour), data extraction and transfer (about 1.5 hours). Data from the Parkes Radio Telescope streams at a rate of 1 gigabit per second and different beam files are extracted and transferred via gigabit optical fibre. The second step is data pre-processing. The beam files contain the pulsar signals which are dispersed by the interstellar medium. gt;De-dispersiongt; is used to counteract this effect. A large number of de-dispersion files are generated according to different choices of trial dispersions. In this scenario, 1,200 is the minimum number of dispersion trials and normally takes 13 hours to complete. For more dispersion trials, such as 2,400 and 3,600, either longer execution time is required or more computing resources need to be allocated. Furthermore, for binary pulsar searching, every de-dispersion file needs to undergo an gt;Accelerategt; process. Each de-dispersion file generates several accelerated de-dispersion files and the whole process takes around 1.5 hours. For instance, if we assume that the path with 1,200 de-dispersion files is chosen, then a temporal constraint of 15.25 hours, denoted as gt;Ugt;(gt;SWgt;gt;1gt;), usually should be assigned for the data pre-processing step. The third step is pulsar seeking. Given the generated de-dispersion files, different seeking algorithms can be applied to search for pulsar candidates, such as FFT Seek, FFA Seek and single Pulse Seek. For the instance of 1,200 de-dispersion files, it takes around 1 hour for FFT seeking to seek all the candidates. Furthermore, by comparing the candidates generated from different beam files in the same time session (around 20 minutes), some interference may be detected and some candidates may be eliminated (around 10 minutes). With the final pulsar candidates, the original beam files are retrieved to find their feature signals and fold them to XML files. The gt;Fold to XMLgt; activity usually takes around 4 hours. Here, a temporal constraint of 5.75 hours, denoted as gt;Ugt;(gt;SWgt;gt;2gt;), usually should be assigned. Finally, the XML files will be inspected manually (by human experts) or automatically (by software) to facilitate the decision making on whether a possible pulsar has been found or not (around 20 minutes). gt; gt; gt;gt;1.2.2 Problem Analysisgt;gt; gt; Based on the above example, we can see that to guarantee on-time completion of the entire workflow process, the following problems need to be addressed. gt; gt;1.gt; Setting temporal constraints. Given the running schedules, a global temporal constraint, i.e. the deadline gt;Ugt;(gt;SWgt;) of 24 hours, needs to be assigned. Based on that, with the estimated durations of workflow activities, two other coarse-grained temporal constraints gt;Ugt;(gt;SWgt;gt;1gt; and gt;Ugt;(gt;SWgt;gt;2gt;) are assigned as 15.25 and 5.75 hours, respectively. These coarse-grained temporal constraints for local workflow segments can be defined based on the experiences of service providers (e.g. the estimated activity durations) or the requirements of service users (e.g. QoS requirements). However, fine-grained temporal constraints for individual workflow activities, especially those with long durations such as gt;De-dispersiongt; and seeking algorithms, are also required in order to support fine-grained control of workflow executions. Meanwhile, the relationship between the high-level coarse-grained temporal constraints and their low-level fine-grained temporal constraints should be investigated so as to keep the consistency between them. gt; Therefore, a temporal constraint setting component should be able to facilitate the setting of both coarse-grained and fine-grained temporal constraints in scientific cloud workflow systems. Meanwhile, as the prerequisite, an effective forecasting strategy for scientific cloud workflow activity durations is also required to ensure the accuracy of the setting results. gt; gt;(Continues...)gt; gt; gt; gt; gt; gt;gt; gt;gt;gt; Excerpted from gt;Temporal QoS Management in Scientific Cloud Workflow Systemsgt; by gt;Xiao Liu Yun Yang Jinjun Chengt; Copyright © 2012 by Elsevier Inc.. Excerpted by permission of ELSEVIER. All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.gt;Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.