Hard to understand
Dependencies and data flow are implied by command order, filenames, and local conventions.
Workflow Description Language
WDL is an openly governed language for describing tasks, inputs, dependencies, and runtime requirements. Different execution engines can interpret the same description on laptops, clusters, and cloud platforms.
workflow example {
input {
Array[File] inputs
}
scatter (file in inputs) {
call process { file }
}
output {
Array[File] results = process.result
}
}The problem
Scripts are a natural place to begin. As an analysis grows, its dependencies, data flow, parallel work, and resource needs become harder to see, while the details needed to run it spread across scripts, inputs, and platform configuration.
Dependencies and data flow are implied by command order, filenames, and local conventions.
The next person needs assumptions and context that the scripts do not carry with them.
More inputs require custom loops, batching, and bookkeeping around each step.
Paths, queues, and installed software tie the analysis to one environment.
The WDL approach
WDL is designed for the people who author analyses, the systems that execute them, and the community that evolves the standard. It describes workflow structure clearly, provides abstractions for common execution patterns, and leaves concrete run values and platform configuration to other tools.
A concise, declarative grammar helps software engineers, domain experts, and platform operators reason from the same workflow description.
Typed inputs and outputs, explicit data dependencies, conditionals, scatter-gather, runtime requirements, containers, and imports express common workflow patterns directly.
A task or workflow that conforms to the WDL specification can run on any platform supported by its execution engine. Editor integrations and portability lints help authors avoid environment-specific assumptions.
A public specification and open governance process let users and implementers inspect, discuss, and contribute to how the language evolves.
A short history
Early workflow-description tooling work begins at the Broad Institute.
Cromwell and evolving WDL drafts establish the task-and-workflow model.
OpenWDL forms to steward WDL as an open, community-governed standard.
WDL 1.0 becomes the official specification.
WDL 1.1 adds standard runtime attributes, JSON I/O, and struct literals.
WDL 1.2 adds directories, multi-line strings, requirements, and hints.
WDL 1.3 adds enums and improves type safety, retries, and cross-engine consistency.
Where it is today
WDL is a community-governed language with a formal specification, conformance-tested execution engines, and an ecosystem spanning local, HPC, and cloud execution. WDL 1.3.0 is the latest stable specification release.