-
Notifications
You must be signed in to change notification settings - Fork 27
Guide to Bonnie Rake Tasks
Rails rake tasks are by default run against the development database; to run a rake task against a different environment such as production add RAILS_ENV=<environment>, for example
bundle exec rake bonnie:db:resave_measures RAILS_ENV=production
Rake tasks can be used to grant and revoke access to the admin role (used for administering the site), the portfolio role (used for creating sets of test patients across measures, such as for Cypress test decks), the dashboard role (which makes the complexity and change dashboard available to a user) and the dashboard_set role (which makes all measures loaded in a user account available on the complexity and change dashboard). The various tasks include
bundle exec rake bonnie:users:grant_admin EMAIL="<user-email>"
bundle exec rake bonnie:users:revoke_admin EMAIL="<user-email>"
bundle exec rake bonnie:users:grant_portfolio EMAIL="<user-email>"
bundle exec rake bonnie:users:revoke_portfolio EMAIL="<user-email>"
bundle exec rake bonnie:users:grant_dashboard EMAIL="<user-email>"
bundle exec rake bonnie:users:revoke_dashboard EMAIL="<user-email>"
bundle exec rake bonnie:users:grant_dashboard_set EMAIL="<user-email>"
bundle exec rake bonnie:users:revoke_dashboard_set EMAIL="<user-email>"
It's possible to move a measure from one user account to another, moving all test patient records and copying all value sets:
bundle exec rake bonnie:users:move_measure SOURCE_EMAIL="<source-user-email>" DEST_EMAIL="<destination-user-email>" CMS_ID="<measure-cms-id>"
Copying Patients between Measures and (optionally) Accounts (Currently only available on the cql4bonnie branch)
This will copy (not move) patients from one measure on one user account to the same (or another) measure on another account (optionally the same account).
bundle exec rake bonnie:users:copy_measure_patients SOURCE_EMAIL="<source-user-email>" DEST_EMAIL="<destination-user-email>" SOURCE_CMS_ID="<source-measure-cms-id>" DEST_CMS_ID="<destination-measure-cms-id>"
-
First export the patients from production, something like:
bundle exec rake bonnie:users:export_patients [email protected] HQMF_SET_ID=93F3479F-75D8-4731-9A3F-B7749D8BCD37 FILENAME=CMS72_patients.jsonSubstitute in the appropriate email, HQMF set id, and a unique filename. Note: on the AHRQ-hosted sites you must also include the flag
RAILS_ENV=production -
Test locally, and after there are no issues, import patients into alpha, something like:
bundle exec rake bonnie:users:import_patients [email protected] HQMF_SET_ID=93F3479F-75D8-4731-9A3F-B7749D8BCD37 FILENAME=CMS72_patients.json MEASURE_TYPE=CQL
Users can export their test patient data with corresponding calculation results in excel format using the web interface; there's a rake task that allows the same spreadsheets to be generated for a user account, one spreadsheet per measure:
bundle exec rake bonnie:users:export_spreadsheets USER_EMAIL="<user-email>"
The measures are exported into files with the name <measure-cms-id>.xlsx
For development or demonstration purposes it can be useful to reset a Bonnie database and populate it with some basic data:
bundle exec rake bonnie:db:reset
This can be limited to a smaller set of data by using the DEMO flag
bundle exec rake bonnie:db:reset DEMO=true
This task clears all existing data and populates the database with a single user account with some measures loaded:
Username: [email protected]
Password: b0nn13
When measures are saved, some actions are completed triggered by the save just prior to writing the measure to the database, such as calculating measure complexity. This rake task re-saves all measures in the database:
bundle exec rake bonnie:db:resave_measures
Note that this does not regenerate the measure calculation JavaScript, to do that see the reset_js task.
The contents of a Bonnie MongoDB database can be dumped to a backup file using
bundle exec rake bonnie:db:dump
This can be used for creating snapshots for, say, debugging or as part of a regular backup process. If used as part of a regular backup process there's a companion rake task that prunes the database dumps, keeping daily snapshots for the most recent month but deleting older files so as to only keep weekly snapshots for older backups:
bundle exec rake bonnie:db:prune_dumps
QDM logic based measures are calculated in Bonnie by generating executable JavaScript based on the measure logic, which then gets run against patient data. This JavaScript is automatically created when a Measure is loaded into Bonnie, and is cached on the Measure object, since once generated it does not normally need to change. There are some circumstances where the measure logic JavaScript needs to be regenerated, such as when a bug is fixed in the actual generation code. To regenerate the JavaScript for all measures in the database, run
bundle exec rake bonnie:measures:reset_js
The reset_js rake task combines two other tasks for clearing the JS and generating the JS, which can be invoked individually if desired:
bundle exec rake bonnie:measures:clear_js
bundle exec rake bonnie:measures:pregenerate_js
Note that if a measure does not have JS, it will be created and cached the next time that measure is used within Bonnie.
For calculation, Bonnie makes use of a patient model where the important parts look something like
{
birthdate: ...
first: ...
last: ...
conditions: [...]
encounters: [...]
procedures: [...]
}
For patient creation and editing, the important parts of the patient model look more like
{
birthdate: ...
first: ...
last: ...
source_data_criteria: [...]
}
Where the source_data_criteria contains information on all the conditions, encounters, procedures, etc that are part of the patient history. Before calculation can happen, the data in source_data_criteria needs to be translated into data for populating the conditions, encounters, etc fields, including the selection of appropriate codes (ie ICD-10, SNOMED, etc). That step is called "materialization". All test patient records in Bonnie can be "re-materialized" via a rake task:
NOTE: This rake task has a significant issue, and should not be run until it's addressed; notably, when codes are selected during materialization, previously selected codes are preserved on the source_data_criteria and are re-used; this happens correctly for primary codes for data criteria (ie Encounter, Performed: Office Visit), but not for embedded field values (ie a Diagnosis embedded within an Encounter, Performed). This may cause some codes to change unexpectedly, changing calculation results.
bundle exec rake bonnie:patients:materialize_all
Bonnie provides the same measure period for all users; historically this has been set to the full year of 2012, and typically isn't changed on deployed servers because changing it would change the results of most calculations. When using Bonnie to generate patient data for specific purposes, such as Cypress test decks, it can be useful to change the measure period. When making that change, it can be necessary to shift the date information within patient records. This can be done via a rake task:
bundle exec rake bonnie:patients:date_shift EMAIL="<user-email>" DIR=<forward|backward> YEARS=<number> MONTHS=<number> WEEKS=<number> DAYS=<number> HOURS=<number> MINUTES=<number> SECONDS=<number>
Value sets are sets of codes (ICD-9, ICD-10, SNOMED, LOINC, etc) grouped together to represent a single concept for use in the context of a measure, and represented by an OID (Object IDentifier). Bonnie loads value set information from VSAC (the NLM Value Set Authority Center, at https://vsac.nlm.nih.gov/). There are two rake tasks for working with value sets. The first allows the contents of a value set to be looked up given the OID:
bundle exec rake bonnie:vsac:lookup_oid OID=<oid> USERNAME=<vsac-username> PASSWORD=<vsac-password>
The second retrieves the current listing of VSAC expansion profiles from VSAC. Expansion profiles (ie "MU2 Update 2015-05-01") are a type of versioning, needed to allow different sets of codes to be present in value sets at different times. Retrieve the list of these using
bundle exec rake bonnie:vsac:get_profiles USERNAME=<vsac-username> PASSWORD=<vsac-password>
The complexity and change dashboard allows users with dashboard access to compare sets of measures. The way Bonnie handles these sets of measures is via special purpose user accounts, not intended for login but just used as a way of aggregating measures. Sets of measures can be loaded into these accounts via a rake task:
bundle exec rake bonnie:dashboard:load_measures FILE=<zip-of-zipped-measures-file> EMAIL=<user-account> VSAC_USERNAME=<vsac-username> VSAC_PASSWORD=<vsac-password>
This task takes the zip files that CMS uses to distribute sets of measures (ie the 2016 EP measures available at https://ecqi.healthit.gov/system/files/ecqm/2016/EP/2014_eCQM_EligibleProfessional_April2016.zip) which contain further zip files, one for each measure. The task will create a new user account with the email provided if necessary (otherwise using an existing account), generate a random password for that account (the password is not needed since the account does not need to be logged into), set the account to be a dashboard set account, and load each measure into the account.
The eCQM calculation engine used by Bonnie, https://github.com/projecttacoma/hqmf2js, is an extremely complex piece of software. When changes need to be made, it is very useful to be able to perform regression testing to ensure that only the desired targeted behavior changes occur. There are some rake tasks in Bonnie that allow all the patient records loaded into a Bonnie instance to be used as a regression test.
There are two sets of regression tests, which operate in a very similar fashion but target slightly different parts of the calculation process. The first (Calculation Tests) targets the simple end results of the calculation on a population level (ie IPP, NUMER, DENOM, etc), while the second (Colorization Tests) targets the calculation rationale, which includes detailed information on intermediate calculation results, and the specificsRationale, which factors the results of specific occurrence calculations into the intermediate calculation results and is used when highlighting the logic when displayed in Bonnie.
To run the calculation tests, first set up the development environment to a state before any code changes are present, then run
bundle exec rake bonnie:calculation_tests:prime_regression
This calculates every patient record in the Bonnie DB against the measure it's associated with and stores the calculation result in the database. After the prime step completes, update the development environment to a state that includes the code changes to be tested and run
bundle exec rake bonnie:calculation_tests:test_regression
This will compare the calculation results to the stored calculation results and display any discrepancies.
If the goal is to just run all the calculations and see if there are execution errors, without actually considering results, run the following task:
bundle exec rake bonnie:calculation_tests:calculate_all
To run the colorization tests, first set up the development environment to a state before any code changes are present, then run
bundle exec rake bonnie:colorization_tests:prime_regression
This calculates every patient record in the Bonnie DB against the measure it's associated with and stores the rationale and specificsRationale in the database. After the prime step completes, update the development environment to a state that includes the code changes to be tested and run
bundle exec rake bonnie:colorization_tests:test_regression
This will compare the calculation results to the stored calculation results and display any discrepancies.
Note: These tasks have mostly been superseded by tests in HealthDataStandards which compare models generated from HQMF with SimpleXML, but may still be useful.
QDM logic based eCQMs are represented using the HQMF standard. The MAT also exports eCQMs in a format called SimpleXML, which while not a standard is still a useful representation of eCQMs since it can be compared to the HQMF format. Bonnie can import measures in SimpleXML format because support was implemented during a period when the MAT was not exporting any HQMF, only SimpleXML. Two rake tasks allow some comparisons to be made between SimpleXML and HQMF measures. The first creates an HTML file that, when opened, displays the human readable QDM logic derived from each measure format:
bundle exec rake bonnie:parser_comparison:create_html FILE=<measure-zip-file> VSAC_USERNAME=<vsac-username> VSAC_PASSWORD=<vsac-password>
The <measure-zip-file> is a MAT file that contains both the HQMF and SimpleXML XML files. The second task takes the two measure formats and finds any existing appropriate patients created for these measures (based on the hqmf_set_id) and calculates each patient against the model generated from each format, and reports on any discrepancies:
bundle exec rake bonnie:parser_comparison:compare_calculation FILE=<measure-zip-file> VSAC_USERNAME=<vsac-username> VSAC_PASSWORD=<vsac-password>
QDM logic measures packaged using HQMF are parsed using the Health Data Standards library (https://github.com/projectcypress/health-data-standards/). When debugging measure parsing issues, it can be useful to parse a measure and examine the resulting data model, or have a simple command line way to start parsing in order to interact with the parsing code using a debugger.
Note: this rake task is available in the Health Data Standards library, not in Bonnie.
bundle exec rake hqmf:parse[<hqmf-file-path>,<hqmf-version>]
The <hqmf-file-path> should refer to the HQMF XML file, and the <hqmf-version> should be either 1 or 2. The version number changes which parser is invoked; all recent measures use version 2. The output of parsing is written as a JSON representation of the measure model, in a file in tmp/json/ (the rake task displays the name and location of the output file).
In addition to HQMF, eCQMs from the MAT can also be represented in SimpleXML. While not a formal standard, measures in this format can be useful, mainly as a comparison point with the HQMF version. SimpleXML parsing support is provided to Bonnie via the simplexml_parser library (https://github.com/projecttacoma/simplexml_parser/). SimpleXML parsing can be run via the command line similarly to HQMF parsing.
Note: this rake task is available in the SimpleXML library, not in Bonnie.
bundle exec rake simplexml:parse[<simple-xml-file-path>]
The <simple-xml-file-path> should refer to the SimpleXML XML file. The output of parsing is written as a JSON representation of the measure model, in a file in tmp/parsed/ (the rake task displays the name and location of the output file).
The fixture export tool will create test-ready fixtures from an extant database. Before using this tool, make sure to load a copy of the desired database and set the database property in config/mongoid.yml
The fixture export task can be run from the command line as follows:
bundle exec rake bonnie:fixtures:generate_backend_fixtures
This command takes the following arguments:
cms_hqmf: Use cms or hqmf id.
values: cms, hqmf
path: Fixture path.
user_email: email of user to export
measure_id: id of measuer to export
Ex: bundle exec rake bonnie:fixtures:generate_backend_fixtures[cms,test/CMS55v5,fake@fake,CMS55v5]
It is also possible to generate frontend fixtures using this tool using:
bundle exec rake HDS:test:generate_frontend_fixtures
With the arguments
cms_hqmf: Use cms or hqmf id.
values: cms, hqmf
path: Fixture path.
user_email: email of user to export
measure_id: id of measure to export
Ex: bundle exec rake bonnie:fixtures:generate_frontend_fixtures[cms,test/CMS55v5,initialLoad,fake@fake,CMS55v5]
On the cql4bonnie branch, cql measures can be exported in a similar manner with one notable exception: Frontend fixtures add two fields to allow the user to define a single patient to export. The cql frontend export arguments are:
cms_hqmf: Use cms or hqmf id.
values: cms, hqmf
path: Fixture path.
user_email: email of user to export
measure_id: id of measure to export
patient_first_name: first name of patient to export. Do not include this argument if you want to export all patients.
patient_last_name: last name of the patient to export. Do not include this argument if you want to export all patients.
Ex: Export with all patients
bonnie:fixtures:generate_frontend_cql_fixtures[cms,test/CMS55v5,fake@fake,CMS55v5]
Ex: Export with specific patient
bonnie:fixtures:generate_frontend_cql_fixtures[cms,test/CMS55v5,fake@fake,CMS55v5,fakeFirst,fakeLast]