You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
IceCube would like to switch from GridFTP and staging out within their job wrapper scripts to Pelican and having HTCSS manage the stageout.
One feature of their existing stageout framework is the ability to have output files be marked as optional: that is, depending on the physics process, the job may-or-may-not produce a given output file. They (a) do not want to rewrite how their physics software works to transition to Pelican and (b) do not want to do the old trick of creating zero-length files to make HTCSS happy.
Accordingly, to smooth the transition, we'd like to make some file transfers optional. That is (only when run in the file transfer plugin mode):
Check to see if the PelicanOptionalTransfers attribute exists in the job ClassAd for the transfer; if so, parse it as a string of comma-separated filenames.
For each file in the list, if the file does not exist locally, do not attempt its transfer as part of the stageout (simply don't add it to the list of things to do).
The text was updated successfully, but these errors were encountered:
IceCube would like to switch from GridFTP and staging out within their job wrapper scripts to Pelican and having HTCSS manage the stageout.
One feature of their existing stageout framework is the ability to have output files be marked as optional: that is, depending on the physics process, the job may-or-may-not produce a given output file. They (a) do not want to rewrite how their physics software works to transition to Pelican and (b) do not want to do the old trick of creating zero-length files to make HTCSS happy.
Accordingly, to smooth the transition, we'd like to make some file transfers optional. That is (only when run in the file transfer plugin mode):
PelicanOptionalTransfers
attribute exists in the job ClassAd for the transfer; if so, parse it as a string of comma-separated filenames.The text was updated successfully, but these errors were encountered: