CDA Content Evaluation

From IHEWiki

Jump to: navigation, search

Contents

Notes

  • The CDA Content Evaluation tool has been reconfigured to provide support for both the North American and European Gazelle instances. Access the North American instance [here] and the European instance [there]. The old link will be deactivated soon. The top banner of the tool will indicate which instance you are using, to help avoid confusion.
  • A wiki icon link to this wiki page has been added to the top banner for your convenience.

Overview

The Clinical Document Architecture (CDA) Content Evaluation application is a web based tool which is intended to allow developers to compare the content of their CDA documents with the content of standard documents that we have defined. The goal is to provide developers with clinical content for specific sections of CDA documents. This tool will then compare the content in the developer's document with the same content from "gold standard" documents. Comparisons are made by applying a series of assertions to the "gold standard" document and a document supplied by the developer, then comparing the results. Some results may be definitive, for example, a given node must exist, or the text value of the node in the developer's document must match the value in the standard document. In other cases, the results must be visually inspected for similarity.

In addition to this main function, the test tool has options to view or download "gold standard" documents and the test assertions which are used in the evaluation. They will also be able to view or download actual vendor sample documents which have been "curated" by Connectathon monitors.

This tool is not intended to replace the NIST tools that test conformance to CDA and specific CDA document types. The NIST tool does not test for content. This tool is used to test that content.

Getting an Account

Some tasks of the CDA Content Evaluation tool require you to log in with a Gazelle user id and password for the appropriate system. You can get a North American Gazelle account here, and a European account there. You will see the "Create an account" link near the bottom of the page. When creating a new account, be sure to select the correct company!

Accessing the CDA Content Evaluation Tool

Except for certain tasks (which will be mentioned below) no login is required to access the tool. Simply surf to the [North American] or the [European] instance to access the application window, shown below:

CDA2.png

Note: The indication of which instance of the CDA tool you are using, and the wiki link, will appear on the banner on every screen view. I have not taken the time to update every screen view in the wiki to reflect this change.

There are four tasks which can be addressed by the tool. Two of them are available to all users, and correspond to the tabs just below the page header. To select either of these tasks, click on the appropriate tab. Then see the section below with the same name as the tab for instructions.

The third task is reserved for users logged in with their Gazelle id. The task permits viewing, and optionally downloading, sample CDA documents which have been saved at a Connectathon by the monitors. Documentation for this task is here

The fourth task is reserved for Connectathon monitors. When a monitor determines that a CDA document submitted by a participant passes all conformance tests, the monitor can store the document, which then can be retrieved by logged in users. Documentation for this task is here

Evaluate CDA Document content

The Evaluate CDA Document content tab has four main panels, each of which has a horizontal "show/hide" bar at the top. In the initial screen shown above, the first of these panels is in 'show' mode and the other three are in 'hide' mode. Clicking on one of the "show/hide" bars will toggle the corresponding panel between the show and hide state. The four panels are:

  • Document being Evaluated which contains a text box for the document being tested (that is, your document), along with control buttons for loading a test document, evaluating it, and clearing the screen.
  • Evaluation Results which contains the results of the evaluation, if one has been done.
  • Test Assertions which contains the assertions used to evaluate the particular document type selected. This section is for reference.
  • Document Standard which contains the "gold standard" document for this document type, which will be compared with your test document. This section is for reference.

Directly under the banner area, above the first "show/hide" panel, is a menu box which allows you to select which CDA document type you wish to have evaluated.

How to Evaluate a Document

1. Select the appropriate document type from the "CDA document to evaluate" menu box just under the banner. Each time this document type is changed, the application will clear any previous test document and evaluation results, making it ready for a new document evaluation.

A. Not all CDA documents type have defined content for testing at this time. It is quite possible we have not published the content and testable assertions for your document type.

2. Load the document you want evaluated into the text window in the Document being Evaluated panel. There are two ways to do this:

A. Copy your document into the clipboard on your computer (e.g., from Oxygen, XML Spy, Notepad, vi), then paste it into the text window. If you use this method, be sure to delete the prompt message that is in the window.
B. Upload a file containing the document you want to test to the text window. To do this:
1) Click on the "Browse..." button on the right side of the Document being Evaluated panel. Note: A "Browse..." button appears on Firefox and IE. On Chrome the button reads "Choose File". A file selection box will open for you to select the document file to upload. This file selection window is generated using Firefox and Ubuntu. Your window may be different. Select the file and click "Open".


CDAfileSelection.png


2) The file path and name will be (partially) displayed next to the "Choose..." button. Click on "Load document file".


LoadFile.png


The file will be uploaded and displayed in the text window.


LoadedFile.png


3. When the document to be tested is displayed in the text window, click the "Evaluate Document" button. This will cause the document content to be evaluated. The results of the evaluation will be displayed in the Evaluation Results panel, which will be set to "show" so that you can read the results.

As mentioned above, Test Assertions panel and the Document Standard panel are solely for reference purpose. Click on the show / hide bar at the top of either of these panels to open them, and again to close them.

When you are finished with the results, you can click the "Reset" button in the Document being Evaluated panel, or select another document type in the "CDA document to evaluate" menu box to clear the results.

Interpreting Evaluation Results

The assertions which are used in CDA Content Evaluation are of the form:

COMMAND-CODE Document-Node Comment

There are three Command codes: PRESENT, SIMILAR, and EQ (Equal).

The Document node is a description of a node in the CDA document, given in XPath format.

The Comment is human readable text which helps identify the assertion which is being made.

Once a document type has been selected in the "CDA Document to Evaluate" menu, the assertions for that document type will be displayed in the Test Assertions panel. This panel is in "hide" mode by default, but you can switch it to "show" mode by clicking on its header bar if you want to see what assertions will be applied to your test document. There may also be comments which help to understand the purpose of particular assertions. Here is an example assertion list:

Assertions.png

PRESENT assertions

PRESENT assertions require that the specified node is present in the test document. When the document is evaluated, an entry will appear in the Evaluation Results panel:

SUCCESS: Node found or,
ERROR: Node not found

depending on whether or not the specified node is present in your test document. The assertion will also be shown for reference. All nodes referred to in PRESENT assertions are present in the standard documents for that document type.

SIMILAR assertions

SIMILAR assertions require that the specified node be present in the test document, and that its text value be similar to, but not necessarily the same as, the text value in the same node of the standard document. When the document is evaluated, an entry will appear in the Evaluation Results panel:

ERROR: Node not found (if the specified node was not present in the test document),
SUCCESS: Node values equal (if the node was present and the values happened to be the same), or
WARNING: Review - node values should be similar (if the node was present but the values were not the same)
In the SUCCESS and WARNING messages, the text values found will listed on the following line in the format: "Expected: [value from the standard document] found: [value from the test document].

If the WARNING result is received, the two text values must be visually compared to determine if they are "SIMILAR" enough.

EQ (Equal) assertions

EQ (Equal) assertions require that the specified node be present in the test document, and that its text value be identical to the text value in the same node of the standard document. When the document is evaluated, an entry will appear in the Evaluation Results panel:

ERROR: Node not found (if the specified node was not present in the test document),
SUCCESS: Node values equal (if the node was present and the values happened to be the same), or
ERROR: Node values not equal (if the node was present but the values were not the same)
In the SUCCESS and WARNING messages, the text values found will listed on the following line in the format: "Expected: [value from the standard document] found: [value from the test document].

Document Types

The following document types are currently available for evaluation using the CDA Content Evaluation tool:

  1. PCC Immunization Content
  2. PCC ED Referral
  3. PCC XDSMS Referral
  4. PCC XDSMS Discharge
  5. PCC XPHR Extract

Document types are followed by version numbers to track changes to the "gold standard" document and evaluation assertions.

View / Retrieve 'Gold Standard' documents

This task allows you to select document types you are interested in from the list of document types currently supported on the test tool, view the "gold standard" document and/or the assertions which are used to evaluated content for that document, or download this information for later use. Clicking on the "View / Retrieve 'Gold Standard' documents" tab will make that function current, and show this screen:

CDA VR gold.png

The "View / Retrieve 'Gold Standard' documents" tab has three main panels, each of which has a horizontal "show/hide" bar at the top. In the initial screen shown above the first of these panels is open and the others closed. Clicking on one of the "show/hide" bars will toggle the corresponding panel between the show and hide states. The three panels are:

  • "Gold Standard" Document List which displays a list of the document types currently supported by the CDA tool. It also contains controls for viewing and/or retrieving selected document type data.
  • "Gold Standard" Document Display which displays a selected the "gold standard" document for a selected document type.
  • "Gold Standard" Assertions Display which displays the assertions used by the tool to evaluated documents of a selected type.

How to View the "gold standard" document and/or assertions for a particular document type

1. In the "Gold Standard" Document List panel select one of the document types in the list. They can be selected by clicking on them. When one document is selected, the command buttons next to the list will activate. 2. Click on the "Display selected Document" command button to display the "gold standard" document for the selected document type. The "Gold Standard" Document Display panel will change to "show" mode, displaying the gold standard document.

A. This document can be copied to your computer's clip board to be saved.
B. The "Gold Standard" Document List panel can be hidden by clicking on its "show/hide" bar to make more room for the document.

CDAGoldMsg.png

To view the assertions used by the testing tool to evaluated CDA document of a document type, follow the same rules as above, but click on the "Display selected Document Assertions" command button. The "Gold Standard" Assertions Display panel will open to display the assertions.

How to Retrieve 'Gold standard' Document and assertions

1. Click on the Document Type or types in the list that you want to retrieve. If one or more document types are selected, the "Download Selected Document(s)" Command button will be activated. 2. Click on the Download Selected Document(s) Command button. When you do this, a zip file named goldCDADoc.zip will be generated, and the 'Gold Standard' Document List panel will be changed to show a download link for that file:

CDAgoldZip.png

Note: The appearance of the download link may vary depending on your operating system and browser.

3. Click the "Download zip file of selected document(s)" link to download the zip file. A download dialog window will appear to let you save the zip file:

CDAGoldZip.png

Note: The appearance of the download dialog may vary depending on your operating system and browser.

4. Click on the "Reset" Command button to reset the tab to its default values.

View / Retrieve vendor sample documents (Logged in users only)

Note to European users: Vendor sample documents will be added to this database by Connectathon monitors at the Connectathon. There will be nothing to retrieve until then.

This task allows you to select documents you are interested in from a database of actual vendor documents which have been curated (to some extent) by Connectathon Monitors and stored for use as samples. The selection is made using one or more of four simple filters. Setting the filters and clicking the "Query" button will open a list of documents which match the filter settings (if any). Once a query has been made, you can select one document from the list and display it on the web page, or download a zip file containing one or more selected documents from the list.

If you have not logged in, you will not see the tab for this task. To make it visible, click on the "Log In" button in the header. This will display a Gazelle log in screen:

CDALogIn.png

Log in using your Gazelle user id and password. If you need a Gazelle account, see #Getting an Account. When you are logged in, you will the "View / Retrieve vendor sample documents" tab. Clicking on it will make that function current, and show this screen:

CDA VR sample.png

The "View / Retrieve vendor sample documents" tab has three main panel, each of which has a horizontal "show/hide" bar at the top. In the initial screen shown above the first of these panels are open and the others closed. Clicking on one of the "show/hide" bars will toggle the corresponding panel between the show and hide states. The three panels are:

  • Document Filters which contains controls for setting the various document filters and retrieving documents which match the filter settings.
  • Document List which shows summary information on documents when a retrieval is done, allows the selection of document, and the display or download of selected documents.
  • Document Display which displays a selected document from the Document List.

How to Retrieve Vendor Sample Documents

1. Use the filters in the 'Document Filters' panel to select documents that are specific to your needs. There are four filters:

A. The Organization Code filter limits documents to those from the selected organization. Only organizations registered in Gazelle for CDA testing will appear in the drop down; the entries in the drop down are their gazelle organization codes. Use the "Clear Organization Code" command button to reset the Organization code filter to "Any".

CDASamplefilterOrg.jpg

B. The System Code filter limits documents to those from a particular (gazelle) system code within a selected organization. If the Organization Code filter is set to "Any", the System Code filter will automatically be set to "Any" also, and be disabled. When an organization code is selected, the System Code filter will be enabled and allow you to select one of the System Codes for that Organization, or "Any" to select any system code for that organization. Use the "Clear System Code" to reset the System Code to "Any".

CDASamplefilterSys.png

C. The Document Type filter limits documents to those of a particular document type. It can be set to "Any" or to one of the document types Shown in the drop down list. Use the "Clear Document Type" command button to reset the Document Type to "Any".

CDASamplefilterMsg.png

C. The Max rows filter is used to limit the number of documents retrieve in cases where there may be a large number of documents available, but only few samples are needed. The value of this filter defaults to 10. It will be a long time before this will be needed. Use the "Reset max rows" command button to reset the value to 10.
D. The Reset All Filters command button can be used to reset all filters to their default values.

2. When the filters are set properly, click the "Query using these settings" command button. If any documents match your filter settings, the 'Document List' panel will open to display information about those documents in a table.

How to use the Document List

This screen shot shows the 'Document List' panel, with five retrieved documents:

CDASampleMsgList.png

One more more documents in the list can be selected simply by clicking their line in the table. Ctrl-click and Shift-click will also work in the usual way. In addition there are four command buttons:

  • Select all Documents, which can be clicked for that purpose, unless all documents are already selected.
  • De-select all Documents, which can be clicked for that purpose, unless all documents are already de-selected.
  • Display Selected Document, which can be clicked to display the selected document in the Document Display panel. It is active only when one and only one document is selected.
  • Download Selected Document(s), which can be clicked to generate a zip file of the currently selected document or documents for downloading to your system. It is active only when one or more documents are selected.
  • A fifth command button, Delete selected Document(s) , will appear if you are logged in as a Monitor. It is active only when one or more documents are selected, and can be used by Connectathon monitors to delete sample documents which are not wanted in the archive.

How to Display a Document

1. In the Document List, select the document you want to display. Be sure no other documents are selected. 2. Click the Display Selected Document command button. The Document List panel will shift to "hide" and the Document Display panel will shift to "show", displaying the document you selected:

CDASampleMsgDisplay.png

It should be possible to select and the displayed document and copy it to your clipboard if you want to save it for use later.

How to Download Sample Document(s)

1. In the Document List, select the document(s) you want to download. 2. Click the Download Selected Document(s) command button. When you do this, a zip file named sampleCDADoc .zip will be generated, and the Document List panel will change to show a download link for that file:

CDASampleZip.png

Note: The appearance of the download link may vary depending on your operating system and browser.

3. Click the "Download sample CDA document zip file" link to download the zip file. A download dialog window will appear to let you save the zip file:

CDASampleDownload.png

Note: The appearance of the download dialog may vary depending on your operating system and browser.

4. Click the "Reset" command button to reset the tab to its default values.

How to delete sample document(s) (monitors only)

1. In the Document List, select the document(s) you want to download. 2. Click the Delete selected Document(s) command button. The selected documents will be deleted from the archive, and the screen will reset.

Store sample document (Monitors only)

Overview

It is expected that vendors would benefit from seeing sample CDA documents generated by other vendors, but that these sample documents should be somewhat "curated" by a Connectathon monitor beforehand. Thus the function of storing a vendor sample document in our collection is a "Monitors only" task. It is intended that:

1. The monitor extracts the CDA document from some other source. This is most likely from the Gazelle tool.

2. The monitor tests the CDA for conformance using the NIST tool.

3. If there is content defined for this document type, the monitor uses this tool to validate the document for content.

4. If the monitor decided that the document was appropriate to store as a sample, they would:

A. Log in as a monitor directly from the vendor screen.
B. Save the sample document with appropriate parameters.

How to log in as a monitor

Click on the "Log In" command button in the header area. Log in using the standard gazelle log in screen, which I assume you already know how to do. When you are logged in, the screen will have an extra tab on the right, Store sample documents, which is described below.

How to store a vendor sample document

1. Evaluation:

A. Extract the CDA document(from Gazelle)
B. Validate with the NIST tool: http://xreg2.nist.gov/cda-validation
C. Load the document into this tool.
D. If there is NO content defined for this document type, skip directly to step 2.
E. If there is content defined for this document type, validate the content with this tool.

At that point, the CDA tool screen should look something like this:

CDAMonitorStart.png

2. If you have not logged in already, log in now. 3. Click on the Store sample documents tab. You should see a screen something like this:

CDAMonitorLoggedIn.png

If the message was just validated using this tool (see step 1.E above) the message you entered there will appear in the Document to Store panel, and its background color will be white. Otherwise, the prompt Paste document to store here will appear, and the background color will be pink.

4. Use the Organization dropdown to select the correct vendor code from the list. All vendors signed up for CDA at the Connectathon will be on the list.

5. After you have selected the Vendor, use the System dropdown to select the correct system. All systems for the selected vendor will be on the list. Note: If the selected vendor has only one system signed up for CDA at the Connectathon, the tool will simply display that system code, there will be no dropdown.

6. Enter whatever commentary you think might be valuable to others looking at the sample document in the "Notes,comments" text area. Notes are not mandatory, but are recommended.

7. If the document to be stored was not copied in already, paste that document in the Document to store panel. Be sure to replace the "Paste document..." prompt. Note: The application does not validate the document you store in any way, other than to check that you put something in. Be sure your cut and paste included the entire xml document.

CDAMonitorReady.png

8. To cancel the store operation, click on the Cancel button. The computer will clear and reset the tab without storing the document.

9. Click on the "Store document" command button to store the document. If there are no errors, the computer will store the message and display this screen:

CDAMonitorStored.png

10. Click on the Return to Store Task button to return to the Store sample documents tab, clear it, and prepare for another document store task.

11. Click on the Return to Evaluate Task button to return to the Evaluate CDA document content tab, clear it, and prepare for another store task.

When you are finished processing, click on the Log Out button in the header. This will clear your gazelle login session and return you to the starting screen.

Personal tools