Commit fc535dca authored by aknecht2's avatar aknecht2
Browse files

Finished updating docs for workflow_job to describe all the new stuff.

parent 404e9d87
......@@ -594,7 +594,8 @@ class WorkflowJob(object):
:param param_name: The name of the input/output to get information for.
:type param_name: str
Returns the information associated with a particular input/output.
Returns the information associated with a particular input/output as a
dictionary: the type & file_type if applicable.
"""
for param_type in ["inputs", "outputs"]:
if self.job_data[param_type] is not None:
......
......@@ -231,41 +231,149 @@ a value provided. Here's a snippet from bwa_align_single.yaml:
has_value: true
default: 2
Each of these arguments gets added as a key and a value i.e. "-q 2", "-l 32".
Each individual argument has four properties that must be defined, and a few
optional properties (all properties are in `chipathlon.conf.argument_keys`).
Required properties: type, changeable, required, has_value.
Optional properties: default, path, file_type, separator.
| **type** defines what the argument is -- a string, a number, or a file.
| **changeable** is a boolean property that defines whether or not the value can be changed in the user provided param file.
| **required** is a boolean property that defines whether or not the argument needs to be included.
| **has_value** is a boolean property that defines whether or not an argument is a positional argument or a key-value argument.
| **default** defines the default value an argument has if it is not passed through the param file.
| **path** defines the folder in which a default config file is contained.
| **separator** is specifically for list arguments. Some commands don't do a great job of parsing list inputs and require some extra help.
|
These properties come with a few implication rules:
1. changeable & no default => must be in params to be included.
If the user does not provide a value for this argument it will not be
added to the job.
2. not changeable => required.
If the argument can't be changed in the param file then it must have a
value to be included.
3. not changeable & has_value => required and default.
Similar to the previous one, the argument must have a value defined if it
is not positional.
Finally, inputs & outputs can be referenced in arguments by referencing their
defined name. Again looking at the cp.yaml file, we see that the values of the
two arguments are "$input_file" and "$output_file". At runtime, this will
adjust the values to the files that were passed into the workflow_job. This
interpolation can also be done in the default property for files passed by
keyword arguments, including in lists. Here's the yaml for idr:
.. code-block:: yaml
idr:
inputs:
sample_1:
type: file
file_type: bed
sample_2:
type: file
file_type: bed
outputs:
idr_bed:
type: file
file_type: bed
command: idr
arguments:
- "--samples":
type: file_list
changeable: false
required: true
has_value: true
default:
- $sample_1
- $sample_2
- "--rank":
type: numeric
changeable: true
required: true
has_value: true
default: 5
- "--input-file-type":
type: string
changeable: false
required: true
has_value: true
default: bed
- "--output-file":
type: file
changeable: false
required: true
has_value: true
default: $idr_bed
walltime: 600
memory: 8000
cores: 1
nodes: 1
Here we include both samples in a list of arguments passed into the --samples
argument. If required=false arguments will not be added unless they have a
value. Here's download_from_gridfs.yaml:
.. code-block:: yaml
The cp takes two
arguments, and input file and output location which is why the cp.yaml only
has two arguments specified. For each argument, you should define four
properties: type, changeable, required, and has_value, and can optionally
include a fifth property: default. Type specifies the type of the argument
and can be numeric, string, file or list. Changeable specifies whether or not
the argument can be overriden by the passed in param file. Required specifies
whether or not the argument must be provided. Has_value defines if the
argument has a value, if not than it is a positional argument. Finally,
default defines the default value if not overriden.
These properties come with a few implication rules:
download_from_gridfs:
inputs:
host:
type: string
username:
type: string
password:
type: string
gfs_id:
type: string
outputs:
downloaded_result:
type: file
file_type: any
command: chip-job-download-gridfs
arguments:
- "-H":
type: string
changeable: false
required: true
has_value: true
default: $host
- "-u":
type: string
changeable: false
required: false
has_value: true
default: $username
- "-p":
type: string
changeable: false
required: false
has_value: true
default: $password
- "-i":
type: string
changeable: false
required: false
has_value: true
default: $gfs_id
- "-d":
type: file
changeable: false
required: true
has_value: true
default: $downloaded_result
walltime: 60
memory: 2000
cores: 1
nodes: 1
1. changeable & has_value & no default => must be in params.
You should try to avoid this case when possible, unless manual user input
is absolutely necessary.
2. not changeable => required.
If the argument can't be changed in the param file then it must have a static
value to be included.
3. not changeable & has_value => required and default.
Similar to the previous one, the argument must have a value defined if it is
not positional.
Finally, inputs & outputs can be referenced in arguments by referencing their
index. Again looking at the cp.yaml file, we see that the values of the two
arguments are "$inputs.0" and "$outputs.0". At runtime, this will adjust the
values to the 0th input and the 0th output respectively. This interpolation
can also be done in the default property for files passed by keyword arguments.
The authentication parameters here "-u" for username and "-p" for password
are not required. This means that if a value is not passed in for these
inputs, these arguments will not be added on the command line.
Resources
^^^^^^^^^
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment