The Image De-identification ETL is a WIP tool to assist with reading DICOM images from Orthanc, conversion to anonymized NIfTI images, and uploading to Flywheel.
First, copy the following file, renaming it to .env
in the process:
$ cp .env.sample .env
Then, customize its contents with a text editor:
- For
AWS_PROFILE
, follow these instructions to configure a named profile for yourself. Use https://d-906762f877.awsapps.com/start as the SSO start URL andus-east-1
as the SSO region. Assign the name of the profile you create to the value forAWS_PROFILE
. - For
FLYWHEEL_API_KEY
, generate an API key from your Flywheel user profile. - For
FLYWHEEL_GROUP
, specify eitherd3b
or an alternative group created for testing (e.g., your name). - For
ORTHANC_CREDENTIALS
, use your Orthanc username and password specified likeusername:password
. - For
ORTHANC_HOST
, specify the hostname (minushttp(s)://
) that you use to access Orthanc. - For
PHI_DATA_BUCKET_NAME
, specify the bucket name where the ETL should backup NIfTI files. - For
SUBJECT_ID_MAPPING_PATH
, specify the path to the CSV file containing subject ID mappings.
Next, run update
to build the container image and initialize the database:
$ ./scripts/update
Finally, use console
to enter a shell environment within the container that contains all the CLI's dependencies:
$ ./scripts/console
And, voilà, the CLI should fully functional ✨:
$ image-deid-etl -h
usage: image-deid-etl [-h] [--program [{cbtn,corsica}]] [--site [SITE]] {initdb,importuuids,check,validate,run,upload2fw,add-fw-metadata,s3-backup-niftis} ...
A WIP tool to assist with reading DICOM images from Orthanc, conversion to anonymized NIfTI images, and uploading to Flywheel.
positional arguments:
{initdb,importuuids,check,validate,run,upload2fw,add-fw-metadata,s3-backup-niftis}
validate check sub/ses mapping
run download images and run deidentification
upload2fw upload results to Flywheel, when complete
add-fw-metadata add metadata in JSON sidecars to NIfTIs on Flywheel
s3-backup-niftis copies NIfTIs to S3
optional arguments:
-h, --help show this help message and exit
--program [{cbtn,corsica}]
program namespace (default: cbtn)
--site [SITE] site namespace (default: chop)
To check Orthanc for and process N number of new studies, pipe the output of check
into the run
command:
$ image-deid-etl check --limit N --raw | xargs image-deid-etl run
To process an individual study, specify an Orthanc UUID after the run
command:
$ image-deid-etl run UUID
CHOP's AWS SSO configuration will automatically sign you out of your session every 1 to 2 hours. If you encounter any permission errors, your session has likely expired. Follow these instructions to refresh your login, using the aws sso login
command.
If you do not have the AWS CLI v2 installed locally, you will need to execute these steps from within the container image.