Wildfire Distributed, GridEnabled Workflow Construction and野火分布式网格功能的工作流建设和.pptVIP

WildfireDistributed, Grid-Enabled Workflow Construction and Execution Arun Krishnan, PhD Assistant Professor, Institute for Advanced Biosciences, Keio University, Tsuruoka, Japan Two Trends Bioinformatics analysis Increasingly complex analyses Several bioinformatics applications assembled into workflows Affordable HPC Commodity hardware assembled into Beowulf clusters Pooled hardware in Grids Parallel by design The Problem We need Tool for construction and execution of workflows on supercomputers User-interface must be intuitive for non-HPC-specialists Execution must support different supercomputing platforms Solution Objectives: Coarse-grained parallel programming for Grid Exploit heterogeneity of Grid (s/w licences, data, h/w) Approach: An expressive workflow description language, GEL Sequential and parallel composition Conditional execution (if-then-else) Sequential iteration (while loop) Parameterised parallel composition (parameter sweeps) Parameterised sequential composition GEL: An OverviewSemantics A workflow has One input directory One or more output directories Workflow cannot modify its input directory GEL: An OverviewSemantics (Job) Job (atomic workflow subunit) Characteristics Executable name Resource/system/software/data requirements One input directory One output directory Semantics Stage files into input directory Run executable Present output directory as result GEL: An OverviewSemantics (Conditional) Conditional (if E then A else B) E is a job for which we ignore the output files A and B are workflows Executing (if E then A else B) entails Execute E and observe stdout If stdout is non-empty then execute A If stdout is empty then execute B GEL: An OverviewSemantics (seq, par compn) Sequential composition (A;B) Execute A Copy files from all output directories of A into input directory of B Execute B Note: implicit merge of output directories of A Parallel composition (A||B) Execute A and B from input directories populated from the same