Fix unstage controls in command.run when using storeDir #6364
Conversation
Signed-off-by: jorgee <[email protected]>
✅ Deploy Preview for nextflow-docs-staging canceled.
|
Signed-off-by: jorgee <[email protected]>
| @@ -160,6 +160,10 @@ class BashWrapperBuilder { | |||
| return targetDir && workDir!=targetDir | |||
| } | |||
|
|
|||
| protected boolean shouldUnstageControls() { | |||
| return false | |||
There was a problem hiding this comment.
I wonder if this should not default to shouldUnstageOutputs()
There was a problem hiding this comment.
No, the issue was caused because it was using shouldUnstageOutputs in both outputs and controls. If we default to shouldUnstageOutputs we will not solve the issue.
There was a problem hiding this comment.
Not sure Jorge, especially this
However, control logs (.command.err, .command.out, etc.) are always copied to the working directory.
Only the .command.log and .command.begin and .exitcode are created directly in the (target) work dir. The stage controls are these below and follow the usual logic as for output files
|
|
||
| @Override | ||
| protected boolean shouldUnstageControls() { | ||
| return true |
There was a problem hiding this comment.
why is this only azure specific ?
There was a problem hiding this comment.
The shouldUnstageOutputs is also azure specific. It was always unstaging outputs and controls. In a first implementation, I removed the specific Azure behaviour for controls but tests were failing, so I decided to keep it.
close #6311
unstage_controlswas set in .command.runusing the same condition as the unstage outputs. It was producing an error message in.command.logwhen using thestoreDir. However, control logs (.command.err, .command.out, etc.) are always copied to the working directory.This PR changes the condition when the
unstage_controlsis set. A new methodshouldUnstageControlsis created to differentiate the condition when checking the outputs or controls. By default, it is set to false, the unstage controls will be applied if scratch is defined. There is a special case for Azure, where boths controls and outputs are forced to be unstaged