Bioinformatics Seminar
Time:
Venue: Na
14 May 2019
NaPortable Pipelines Project: Developing reproducible bioinformatics pipelines with standardised workflow languages
Richard LupatPeterMac
There are ongoing efforts to improve the reproducibility and portability of bioinformatics pipelines. Projects such as Common Workflow Language (CWL) standardise how workflows are specified ; while workflow execution engines such as Toil and Cromwell provide many useful features to run these workflows across HPC and cloud infrastructures. However ; there is often debate on whether to adopt CWL ; Workflow Definition Language (WDL) or other competing standards. CWL provides rigid ; easy-to-parse specifications and is supported by variety of engines ; but is more difficult to write ; while alternative standards such as WDL offer more features and an easier learning curve but are tightly coupled to a specific engine such as Cromwell.
We present Janis ; an open source tool to machine generate CWL/WDL workflows. This tool is designed to build standardised workflows via a translation mechanism that generates validated workflow specifications (CWL ; WDL or others in future) as output. These translated workflows act as a transport layer that can be shared ; published and executed using any workflow execution engine that supports the selected specifications. Combining this approach with containerisation technology such as Docker and Singularity ; any bioinformatics pipelines generated by this tool would be capable of running across various computing infrastructures.
This project is a collaborative effort between Peter Mac ; WEHI and University of Melbourne (Melbourne Bioinformatics). Through Janis ; we have developed exemplar translated cancer variant-calling pipelines that are functional across these three institutes' HPC and commercial cloud infrastructures.;;;;