Walkthrough
This Walkthrough introduces Inferno by demonstrating its use as an automated testing tool for the Draft Test Method of the 21st Century Cures Act: Interoperability, Information Blocking, and the ONC Health IT Certification Program. It uses the ONC-hosted Inferno reference instance to test against a publicly available SMART-on-FHIR sandbox.
At the end of this walkthrough, you will be able to use Inferno's draft ONC Program Certification tests to evaluate APIs for conformance to the Proposed ONC certification criteria. If you are interested in how to use Inferno to test other FHIR-based data exchanges that fall outside the scope of ONC's proposed rule, please visit Extending Inferno in this wiki.
Please note that Inferno is undergoing active development and is expected to change before the ONC rule is finalized. Updates to Inferno can be followed on the Inferno Release Page.
- Step 1: Build a SMART-on-FHIR Sandbox
- Step 2: Open Inferno
- Step 3: Select FHIR Version and Enter FHIR Endpoint
- Step 4: Perform Discovery and Registration Tests
- Step 5: Perform Standalone Patient App Tests
- Step 6: Perform EHR Launch App Tests
- Step 7: Perform Data Access Tests
- Step 8: Review Results
Before you begin using Inferno, you need a FHIR server to test. Many vendors (including, Epic, Cerner, AllScripts, etc.) have released public-facing FHIR server sandboxes that developers can use to test out FHIR applications. For the purposes of this guide, you will be using a sandbox provided by the Health Services Platform Consortium (HSPC).
Details of how each vendor system handles these steps differ slightly, but they generally follow the same general workflow. So while this is HSPC Sandbox-specific, you can expect the same concepts to be found elsewhere.
Note that test failures are expected in this walkthrough, as we are not preloading the HSPC Sandbox with fully profile-conformant data, and the HSPC Sandbox does not currently support all proposed certification criteria. Some standards represented in these tests are very new and are only starting to be implemented in public systems.
- Go to https://sandbox.hspconsortium.org/#/start and click the 'Sign Up' button on the welcome page
- Follow the instructions to create an HSPC account
- Create a new DSTU2 Sandbox
- Locate the SMART DSTU2 FHIR Endpoint found under Settings and copy it to a file for later reference
Note that we will be alternating between the HSPC Sandbox system and Inferno through the course of this walkthrough. Demonstration of the requirements of a SMART on FHIR server require a number of user-driven steps, including client registration and app launch, some of which require initiation from within the Sandbox system.
Now that you have a sandbox FHIR server set up, you can start using Inferno:
- Go to http://inferno.healthit.gov
- Select 'Try It Here' ONC Program Edition', which is an instance of Inferno configured to specifically test the requirements of the criteria in the ONC proposed rule.
The ONC proposed rule currently specifies DSTU2 as the required version of FHIR. Inferno provides a reference set of tests for FHIR R4 in the case that the final rule requires FHIR R4. This walkthrough will demonstrate the DSTU2 set of tests.
- Select "Proposed ONC Health IT Certification Tests (FHIR DSTU2)"
- Enter the FHIR URI that was retrieved from the HSPC Sandbox site in Step 1
- Click 'Begin'
Inferno's tests for the proposed ONC certification criteria are organized into four steps. This allows the tester to walk through the requirements in an order similar to what would be done in a real world situation, while limiting redundant testing.
The first step, 'Discovery and Registration', verifies that the system is capable of performing manual client registration and supports endpoint discovery using service metadata provided by the FHIR and SMART on FHIR specifications.
- Click 'Run Tests.' You will presented with a modal that provides necessary registration information for Inferno for use in HSPC, and accepts a 'Client ID' field.
- Note the 'Launch URI' and 'Redirect URI' fields
- In a separate window, manually register Inferno as an app in your HSPC Sandbox by clicking on the '+' icon in the 'Apps' section of the sandbox. Enter in the 'Launch URI' and 'Redirect URI' and give the app a reasonable name (e.g. Inferno). You can choose either a Confidential or Public Client, for the purpose of this demo choose Public.
- At the time of the writing of this guide, HSPC did not fully support the scope requirements of the SMART App Launch Guide, and therefore fails if default scopes are used. For the purposes of this walkthrough, set the scope to
launch launch/patient offline_access openid profile user/*.* patient/*.* - Click save to finish registering Inferno as an app.
- You will be given a 'Client ID', copy this and in the Inferno browser window and click 'Execute'.
- The tests will execute, and an overall test result for the entire step will be presented, along with results for each component of the test. Click on 'details' for more information about any component of the test.
- Inferno provides in-depth information about what occurred during the course of the test to help debug any possible errors. This includes pass/fail status on any given test, a list of errors, HTTP requests made during the course of the test, and a detailed test description.
- Click 'Show Details' to provide specifics of the test.
- For even more information on any individual test step, click 'results'.
- You have now completed your first test.
After you have finished reviewing the results from the Discovery and Registration tests, click 'Next' or click on the 'Standalone Patient App' tab to progress to the next step in the test procedure. This test demonstrates that a server is capable of handling a standalone launch from a patient-centric application; e.g. a mobile device.
- Click on the 'Run Tests' button to begin.
- The HSPC Sandbox does not fully support the requirements of the SMART App Launch guide, and therefore rejects the pre-filled set of scopes required by default. Enter in the following scopes instead:
launch launch/patient offline_access openid profile user/*.* patient/*.*.
- Once you click 'Execute', Inferno will notify you that it is redirecting you to the HSPC Authorization server as part of the SMART on FHIR / OAuth launch sequence.
- Follow the instructions on the screen, including selecting a patient to authorize, and which services to authorize.
- Follow the instructions on the screen, including selecting a patient to authorize, and which services to authorize.
Continue on to the 'EHR Launch' set of tests. This set of tests requires the user to initiate an app launch outside of Inferno in order to fully demonstrate the ability of the server to support the EHR Launch flow as described in the SMART App Launch Guide. Inferno tests this by pausing this set of tests mid-execution, and waits at the specified launch point for the user to initiate the launch sequence from the EHR. This action will then inform Inferno that the test may continue running, with information provided during the launch.
- The HSPC Sandbox does not fully support the requirements of the SMART App Launch guide, and therefore rejects the pre-filled set of scopes required by default. Enter in the following scopes instead:
launch launch/patient offline_access openid profile user/*.* patient/*.*.
- Click 'Execute' to begin the tests
- The tests will begin executing and immediately the interface will notify the user that Inferno needs to receive an external action in order to continue.
- Navigate back to the HSPC interface and locate the Inferno app that you created earlier. Click on the 'Launch' button in the app. This should be configured to send your browser to Inferno, which will then resume the test execution.
- From this point on, the tests will execute in a similar manner to the Standalone Launch sequence provided earlier.
- And finally, results will be displayed in a similar manner to the previous test groups.
At this point, the user should have received a Patient ID and be authorized to perform the required FHIR queries on the FHIR server. Click 'Next' or on the 'Data Access' tab to begin testing that capability.
- Before starting, the user will be shown the Bearer Token collected earlier, as well as the Patient ID returned on the most recent SMART Launch. This may have been either the Standalone Launch or Patient launch -- this set of tests currently does not require users to demonstrate all of these queries in both situations.
- After running these tests, you will be presented with the test results. These tests typically follow this pattern:
- Ensure that the user does not have access to searching without the appropriate authorization header
- Perform a FHIR search for all resources of a certain type that are associated with the relevant patient
- For each of the filtered searches required by US Core / Argonaut, generate search queries that should return at least one result based on data that has already been seen, and verify that all data returned falls within the search criteria.
- Validate all resources returned against the relevant profile. This includes validating that codes are within required ValueSets
- Ensure that all references contained within the resource can be retrieved
- Note: if the selected patient does not include all required resources, then some tests will be marked as 'SKIP'. The tester can then execute one of the Launch Sequence tests and authorize another patient, and only execute the tests that were previously skipped. This allows the test system to have the flexibility to demonstrate that all data can be returned, without requiring a single patient to have all required data elements
Finally, a report that includes a summary and details of each test is provided. The user can print this report and save it as a file for their records.
This was a short walkthrough of testing a system against the Inferno set of draft ONC certification tests. You are encouraged to try this against an R4 set of servers.