You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In October/November 2024, there were several PDS calibration runs that did not end gracefully, and this resulted in raw data files that were not cleanly closed.
A utility was developed to recover missing file-level HDF5 Attributes in these files and thereby allow successful creation of the file-transfer metadata and successful transfer of the raw data files to offline storage. That utility is described here.
As part of reviewing that PR, Wes noticed that there may be additional recovery steps that we can take if/when we see similar un-closed raw data files in the future. We may be able to recover TriggerRecord-level HDF5 Attributes for the last TriggerRecord in un-closed files, and thereby make those records read-able offline.
For reference, the hdf5_dump.py utility was successfully run on all of the TRs in one of the sample unclosed PDS raw data files. This seems to imply that all of the data is available for the last TR in the file, and it may just be missing record-level Attributes that prevents the last TR from being examined by more sophisticated tools.
This Issue suggests that we look into all of this at some point in time.
The text was updated successfully, but these errors were encountered:
In October/November 2024, there were several PDS calibration runs that did not end gracefully, and this resulted in raw data files that were not cleanly closed.
A utility was developed to recover missing file-level HDF5 Attributes in these files and thereby allow successful creation of the file-transfer metadata and successful transfer of the raw data files to offline storage. That utility is described here.
As part of reviewing that PR, Wes noticed that there may be additional recovery steps that we can take if/when we see similar un-closed raw data files in the future. We may be able to recover TriggerRecord-level HDF5 Attributes for the last TriggerRecord in un-closed files, and thereby make those records read-able offline.
For reference, the
hdf5_dump.py
utility was successfully run on all of the TRs in one of the sample unclosed PDS raw data files. This seems to imply that all of the data is available for the last TR in the file, and it may just be missing record-level Attributes that prevents the last TR from being examined by more sophisticated tools.This Issue suggests that we look into all of this at some point in time.
The text was updated successfully, but these errors were encountered: