|
|
*(AKA: Stuff that broke when I tried to use Bridges)*
|
|
|
|
|
|
- Trying to automagically generate the Pegasus `sites.xml` and `conf.rc` files probably isn't going to be feasible due to the variety of run-time environments people might be using. Probably better is to add command-line flags to the `chip-gen` script to specify paths to existing files which are then just copied to the workflow directory. We can provide examples in the docs for a few types of environments to help people create the files for their particular setup.
|
|
|
- Trying to automagically generate the Pegasus `sites.xml` and `conf.rc` files probably isn't going to be feasible due to the variety of run-time environments people might be using. Probably better is to add command-line flags to the `chip-gen` script to specify paths to existing files which are then just copied to the workflow directory. We can provide examples in the docs for a few types of environments to help people create the files for their particular setup. (Adam will do)
|
|
|
|
|
|
- DB files must exist locally or the code bails. This is fine when running directly, but when using BOSCO this means the DB will get staged again for every run via SSH. Would be better to allow specifying a location on the remote resource. This would require people to manually copy the files initially, but then Pegasus would just do symlinking for each run.
|
|
|
- DB files must exist locally or the code bails. This is fine when running directly, but when using BOSCO this means the DB will get staged again for every run via SSH. Would be better to allow specifying a location on the remote resource. This would require people to manually copy the files initially, but then Pegasus would just do symlinking for each run. (Adam will do)
|
|
|
|
|
|
- All the executables are added using `site="local"`. That's probably ok for things in `jobs/scripts` and `jobs/wrappers`, but the system commands (mv, cp, etc.) should be added using the remote site. If added using local, Pegasus will copy the binaries, which might cause issues if the submitting machine is a different enough version of Linux (or OS X even). Adding using site='local' does require people create a chipathlon and idr conda environment manually, and then adding the bin PATH to the sites.xml. Adding the remote site executables would be nice; that would still require the user to create the two conda envs, and then pass the bin paths to the `chip-gen` script.
|
|
|
- All the executables are added using `site="local"`. That's probably ok for things in `jobs/scripts` and `jobs/wrappers`, but the system commands (mv, cp, etc.) should be added using the remote site. If added using local, Pegasus will copy the binaries, which might cause issues if the submitting machine is a different enough version of Linux (or OS X even). Adding using site='local' does require people create a chipathlon and idr conda environment manually, and then adding the bin PATH to the sites.xml. Adding the remote site executables would be nice; that would still require the user to create the two conda envs, and then pass the bin paths to the `chip-gen` script. (Adam will do)
|
|
|
|
|
|
- In order for the above three things to work, the remote site name will also need to be given as an argument to `chip-gen`, so that when adding remote input files or executables to the DAX the location is correctly specified.
|
|
|
- In order for the above three things to work, the remote site name will also need to be given as an argument to `chip-gen`, so that when adding remote input files or executables to the DAX the location is correctly specified. (Adam will do)
|
|
|
|
|
|
- Default resource requirements are way off. (Avi mostly fixed)
|
|
|
|
... | ... | |