XDS Test Kit 2007-2008 Test Descriptions

From IHEWiki

Jump to: navigation, search

Contents

[edit]

How to Report Results

[edit]

Uploading test log reference

To evaluate many tests it is necessary to review the contents of the Test Log associated with the Public Registry server. When this is called for, identify your results in the Test Log using the Test Log Browser. Then submit the reference to your test instance in a file named testlog.txt.

[edit]

Log Results of Provide and Register or Register transaction tested against the Public Registry Repository

When testing the Provide & Register transaction or the Register transaction against the Public Registry/Repository, the test directions either directed you to use

the test tool xdstest2 or
to configure your Document Source actor or
to configure your Combined Document Source / Document Repository

to produce a submission. If the test was started by your

  • Document Source or Combined Document Source / Document Repository actor implementation then Log the XDSSubmissionSet.uniqueId attribute of the submission then use one of the three following options:
  1. Log the message id (or message link) via the directions here
  2. Or logging by uniqueId. This is done by:
      1. Create a text file named uniqueid.txt.
      2. In this file put the value of the XDSSubmissionSet.uniqueId attribute.
      3. Upload this file to Kudu.
  • If the test was started by xdstest2, submit the resulting log.xml file.

Instructions on how to log test results to Kudu can be found here

[edit]

Log Results of Query or Stored Query transactions tested against the Public Registry

When testing the Query Registry transaction or Stored Query transaction against the Public Registry:

  1. Perform the query using your Document Consumer
  2. Use the XDS Test Log Browser to find the log event describing your query.
  3. Select the log event, displaying the detail on the right side of the screen.
  4. In the upper right corner of the browser page, select the link next to the Message ID: label.
  5. Copy this url and place it in a file named url.txt
  6. Submit this file to Kudu as evidence of your successful test
[edit]

Log Results of Register transaction against your Registry

The following details shall be logged to the test management tool:

Instructions on how to log test results can be found here

[edit]

Log Results of Query or Stored Query transaction against your Registry

This test will be run using the command line tool xdstest2 which produces a log file log.xml.

  • The file log.xml must be logged into Kudu as evidence of successful test completion

Instructions on how to log test results can be found here

[edit]

Result Reporting - Server Testing using xdstest

This is to be used when using xdstest to test your Registry actor implementation.

The following details shall be logged to the test management tool:

  • file log.msg generated by xdstest, it will contain:
    • Patient ID used in testing. It must be unique for the test. You must edit this field (patientid=) into test.properties
    • Request sent to the server.
    • RegistryResponse XML response from the server.

If xdstest generates the files request.msg and result.msg instead of log.msg, you need to download the newer version of xdstest from here

  • If the test being reported is a submission then include:
    • Response from a GetAll query ( 11806)(returnType="LeafClass") issued against the patientId used.
    • In this query, you must use $status = ('Approved', 'Deprecated')

You can use xdstest and test 11806 (modifying the patient ID used in the test) to retrieve this data.

In situations where you must upload more that one file to the test management tool, each file must have a unique name or it will not upload. If you are uploading 2 log.msg files, just give them multiple names like log1.msg, log2.msg.

[edit]

Result Reporting - Server Testing using xdstest and the public registry

This is to be used when using xdstest to test your Repository actor implementation.

The following details shall be logged to the test management tool:

  • file log.msg generated by xdstest, it will contain:
    • Patient ID used in testing. It must be unique for the test. You must edit this field (patientid=) into test.properties
    • Request sent to the server.
    • RegistryResponse XML response from the server.

If xdstest generates the files request.msg and result.msg instead of log.msg, you need to download the newer version of xdstest from here

[edit]

The Test Management Tool - how to report results to the Test Managers

The Test Management Tool, called Kudu, is a web site that you will use to record the results of your testing. Each required test for each company is presented as a separate web page inside the tool. In this area a vendor can upload log files to document the results of a test. In the XDS test procedures, when asked to 'log the following information' we are refering to uploading files into Kudu.

[edit]

Configuring and running the TestPlan

Most tests to be run against Registry, Repository, and Receiving Gateway actors use the xdstest2 tool. Most tests require that you configure the test plan, execute it, and evalute the resulting log file. Each test is defined in a directory of the download-able test kit. Tests are found under the directory tests. Each test has its own directory named with the test number. To configure the test, edit the testplan.xml file in the test directory. Change the value of the RegistryEndpoint element to point to your Repository, Registry, or Receiving Gateway, depending on the test being run. Then use the xdstest2 command to run the test. The results are placed in the file log.xml. The file log.xml must be submitted to evaluation by the test monitors. Documentation for the xdstest2 tool can be found here.

Typical configuration changes are:

  • Endpoint of your actor implementation
  • repositoryUniqueId (for XDS.b Retrieve tests)
  • Patient Id to use for the test. For most tests this is controlled from the mgmt sub-directory of the xdstest2tool installation. The file patientid_base.txt holds the patient-specific part of the Patient ID and the file assigning_authority.txt holds the domain-specific part of the Patient ID. A Patient Id is constructed by concatenating the contents of patientid_base.txt followed by the string ^^^ followed by the contents of assigning_authority.txt.
  • HomeCommunityId of the Receiving Gateway (Receiving Gateway tests only). The file mgmt/default.xml of the xdstest2tool installation holds the homeCommunityId of the Receiving Gateway being tested. This must be updated to match your local configuration. The default contents of the file look like: 
<Configuration>
  <HomeCommunityId>urn:oid:1.19.6.24.109.42.1.3</HomeCommunityId>
</Configuration>

You must edit in the real value of your HomeCommunityId to pass any Receiving Gateway tests.

[edit]

Evaluating the results of the TestPlan

The results of the test plan are placed in the file log.xml in the test directory. The test status is placed in the status attribute of the TestResults element (top level element). All tests, when successful, end with a status of Pass. 

<TestResults status="Pass">
 <RegistryEndpoint>http://localhost:9080/axis2/services/test11966</RegistryEndpoint>
 <TestStep id="submit" status="Pass">

If status="Fail" then one or more of the TestStep sections will have status="Fail" OR there will be a FatalError element at the bottom of the file indicating a large failure.

[edit]

Patient IDs and uniqueIds in test material

All test and example metadata provided in the testkit has the patientId and uniqueIds coded as variables. These variables look like

  • $PatientId
  • $uniqueId01, $uniqueId02 ...

To use, real patientIDs and uniqueIds must be allocated and be inserted in metadata replacing the variables.

[edit]

Evaluations

[edit]

Schema

Metadata validated against appropriate Schema(s)

[edit]

Metadata Validator

Tool that checks for metadata structure required by the standard but beyond the capability of XML Schema. Also checks for structural and coding rules imposed by the XDS profile.

[edit]

Using the Public Registry

The Public Registry is used to support XDS testing. There are several rules to follow when testing against this combined Document Registry and Document Repository actor implementation.

  1. All patient IDs must be registered. Many tests require the use of a unique Patient ID for the test. XDSSubmissionSet, XDSDocuentEntry, and XDSFolder all have attributes for patientID.
  2. All XDSSubmissionSet, XDSDocumentEntry, and XDSFolder objects must contain uniqueId attributes that are unique in this registry.

The following sections document how to follow these rules.

[edit]

Allocating new Patient IDs for Testing

The Registry actor, per the XDS Profile specification, validates patient IDs against against those received in the Patient Identify Feed. The Public Registry will reject submissions if they reference a non-registered Patient ID.

To generate patient ID for a testing against the Public Registry, go to the XDS Tools Page and select Allocate Patient ID. A new Patient ID will be returned. You may use this Patient ID in as many tests as you like.

[edit]

Allocating uniqueIDs for testing

The public registry implementation, according to the XDS Profile, validates the uniqueness of the uniqueId attributes of Documents, Folders, and Submission Sets. Contact your regional Test Manager to be assigned an OID base to be used for constructing uniqueIds. If OID bases are not being managed for this testing event, you still need to generate unique OIDs for unique ID attributes in XDS. You can use what ever method you like but the Public Registry will validate

  1. unique IDs are formatted as OIDs
  2. The unique IDs are unique as required by the XDS profile

When using the test tool xdstest2, it will automatically generate unique IDs for you.

[edit]

Test Data Description

[edit]

From test 11870

This test set has been loaded into the Public Registry multiple times. The original, shown below, has two flaws: other users submitted data to this same patient ID and the authorPerson attribute is in the wrong format. The second submission, shown second below, fixes the authorPerson attribute and is submitted for a Patient ID which I have locked so others cannot accidentally submit additional material.

Note that these are for Query testing only. There are no documents behind this metadata. Having no idea what folks are doing with content, I will submit and post in this section content that you send me. -bill

[edit]

Submission 1

The table below briefly describes the test data loaded into the Public Registry for testing. The same data is available in test #11870 for use in vendor registries.

All patientIds are shown without their ^^^&1.3.6.1.4.1.21367.2005.3.7&ISO suffix.

All uniqueIds are shown without their 129.6.58.91. prefix.

PatientId SS uniqueId Doc uniqueId Fol uniqueId Description
428db2e57eaf4aa 4895 One document submission
4894
428db2e57eaf4aa 4897 Single document in a folder submission
4896
4898
428db2e57eaf4aa 4901 One folder and two documents submission
4899
4900
4902
[edit]

Submission 2

The table below briefly describes the test data loaded into the Public Registry for testing. The same data is available in test #11870 for use in vendor registries. This is an update to Submission 1 when issues were found with its metadata content.

All patientIds are shown without their ^^^&1.3.6.1.4.1.21367.2005.3.7&ISO suffix.

All uniqueIds are shown without their 129.6.58.91. prefix.

PatientId SS uniqueId Doc uniqueId Fol uniqueId Description
306268c3314d406 9737 One document submission
9736
306268c3314d406 9739 Single document in a folder submission
9738
9740
306268c3314d406 9743 One folder and two documents submission
9741
9742
9744
[edit]

Submission 3

The table below briefly describes the test data loaded into the Public Registry for testing. The same data is available in test #11870 for use in vendor registries. This is an update to Submission 1 when issues were found with its metadata content.

All patientIds are shown without their ^^^&1.3.6.1.4.1.21367.2005.3.7&ISO suffix.

All uniqueIds are shown without their 129.6.58.91. prefix.

This version fixed the following bugs:

Attribute URI missing - it now points to a small txt document, size and hash match
two versions of testplan.xml are provided in the test kit, one coded for XDS.a, the other for XDS.b
PatientId SS uniqueId Doc uniqueId Fol uniqueId Description
270a59d7a8b145b 12408 One document submission
12407
270a59d7a8b145b 12410 Single document in a folder submission
12409
12411
270a59d7a8b145b 12414 One folder and two documents submission
12412
12413
12415
[edit]

Contributed Content

[edit]

Scan Document (CDA wrapped PDF)

In the Public Registry:

XDSDocumentEntry.uniqueId = 1.2.3.4.100000022002209036.1196211173506.1

URI = http://129.6.24.109:9080/Repository/1.2.3.4.100000022002209036.1196211173506.1.xml

[edit]

Individual Tests

[edit]

11710

Connection + IP Registration

The primary purpose of this test is to help manage the users of the Public Registry by associating a Vendor Name with an IP address. This allows the test managers to find your test data based on our displays. This is occasionally necessary to help answer questions. If you are testing a Document Repository then you need to download the XDS testkit to run all your tests and so you have the choice of running test 11710 or registering via the Test Log Browser. The instructions for registering via test 11710 are described below under Test Procedure. If you are testing a Document Registry then you have no interaction with the Public Registry and this test is not necessary. If you are testing a Document Source or Document Consumer actor then a new facility built into the Test Log Browser can be used instead of this test which gets you out of downloading and installing the xdstest2 testing tool. You can find documentation on using the Test Log Browser to register here.


References

None

Actors Tested

None

Test Procedure

  1. Edit the file tests/11710/testplan.xml
  2. Edit the following fields:
    • <Your Company Name> (include product name is testing more than one).
    • <Your Name>
    • <Your Email>
  3. Send this metadata as a Submission Request by executing the following from a command prompt:
    • Change directory to tests/11710
    • Run the test plan
    • Verify the test was successful
  4. This test does not need to be reported to the Test Manager or logged to Kudu.
  5. If you test from more than one machine and therefore run this multiple times, please spell the company name the same each time.

To verify this test was successful, look in log.xml. You should see:

/TestResults[@status="Pass"] and
/TestResults/TestStep[@status="Pass"]
/TestResults/TestStep/SimpleTransaction/Result/rs:RegistryResponse[@status="urn:oasis:names:tc:ebxml-regrep:ResponseS

tatusType:Success"]

[edit]

11720

Metadata Validation Tool

XDS Schema and XDS Metadata Validator engine available as a web portal to support testing of registry metadata. This is not a reportable test but instead a reference to a development/debug tool.

References

Tables 3.14.4.1-5 through 3.14.4.1-7 of ITI TF-2

Actors Tested

None

Test Procedure

  1. Assemble Repository Submission Metadata or Registry Submission Metadata in a file.
  2. Bring up Validator in your web browser.
  3. Select type of submission (Registry or Repository)
  4. Select submission file
  5. Submit button

The web portal validates your metadata with the XDS/ebXML Registry v2.1 Schema and XDS Metadata Validator and return results. This Schema/Validator pair is also installed on the Public Registry. This validation is run on every submission.

[edit]

11721

Basic Transport

This test can be used to test any SOAP transport. It supports tests labeled 11721 and 11814.

Test basic transport mechanisms, HTTP and SOAP, for transferring metadata without mutual TLS authentication. Document Repository makes an HTTP connection to Document Registry. A simple SOAP package is transferred containing simple non-metadata XML. The Registry may return an error message because the payload is not metadata. This is acceptable.

References

ITI TF-2 3.14

Actors Tested

Document Registry OR Document Repository

Testing Document Registry Actor

  1. Edit test/11721/test.properties
  2. Change the URL parameter to point to your registry or repository
  3. Run the test using xdstest
  4. Do not log the results. This test is for debugging SOAP implementations only.
[edit]

11728

Submit a Folder via XDS.a

Verify that the XDS.a Document Source can submit a Folder via Provide and Register Document Set-a transaction.

References

ITI TF-2 3.15

Actors Tested

Document Source

Dependencies

Test 11710 must be run before this test

You will need to

Allocate a Patient ID
Allocate unique IDs for Submission Set and Document objects

Resources

examples/11728 section of test kit
XDS Test Log Browser

Testing Document Source Actor

  1. Configure your Document Source actor to submit to the Public Repository at EndPoint http://129.6.24.109:9080/axis2/services/test11728
  2. Submit a Submission Set containing a single Folder using the Provide and Register Document Set-a transaction
  3. Log the results
[edit]

11729

Submit a Folder with an initial document via XDS.a

Verify that the XDS.a Document Source can submit a Folder and initial document via Provide and Register Document Set transaction.

References

ITI TF-2 3.15

Actors Tested

Document Source

Dependencies

Test 11710 must be run before this test

You will need to

Allocate a Patient ID
Allocate unique IDs for Submission Set and Document objects

Resources

examples/11729 section of test kit
XDS Test Log Browser

Testing Document Source Actor

  1. Configure your Document Source actor to submit to the Public Repository at EndPoint http://129.6.24.109:9080/axis2/services/test11729
  2. Submit a Submission Set containing a Single Folder with initial Document using the Provide and Register Document Set transaction
  3. Log the results
[edit]

11730

Add Document to Folder

Add a Document to an existing Folder. The Folder is identified by its UUID which is obtained from a query of your choice.

This submission consists of:

  • an XDSSubmissionSet object
  • an XDSDocumentEntry object
  • a HasMember Association object linking the Submission Set to the Document
  • a HasMember Association object linking the existing Folder (in Registry) with Document in submission
    • An ObjectRef identifying the Folder as an object already in the registry

References

ITI TF-2 3.15

Actors Tested

Document Source

Dependencies

Creation of a Folder to add the document to. Test 11728 can be used for this purpose.

Testing Document Source Actor

  1. Allocate a new patient ID and uniqueIDs (see #Using_the_Public_Registry)
  2. Create a Folder per test 11728.
  3. Configure the Document Source under test to send to the URL
    • http://hcxw2k1.nist.gov:8080/xdsServices2/registry/soap/portals/yr3a/repository?testeval=11730
  4. Identify the UUID of the Folder of interest with a query of your choosing
  5. Formulate a submission as described above or use the example metadata found in examples/11730
    • If you use this example, the Folder is referenced symbolically as "$Folder01". This must be replaced with the actual UUID of the folder object in the registry. Addionally, an ObjectRef with its id set to this UUID must be present in the submission.
  6. Have your Document Source send the submission
  7. Verify that you get a successful RegistryResponse message in return.
  8. Log the results: #Log_Results_of_Provide_and_Register_or_Register_transaction_tested_against_the_Public_Registry_Repository
[edit]

11731

Register One Document

Verify that the Document Repository can generate a valid SOAP over HTTP message containing valid metadata representing a single document. The metadata will be validated against both the ebXML Registry Schema, XDS metadata rules and the ebXML Registry metadata requirements. The metadata must contain a Submission Set and a single XDSDocumentEntry.

References

ITI TF-2 3.14

Actors Tested

Document Repository

Testing Document Repository Actor and Integrated Document Source/ Document Repository actor

  1. Allocate a new patient ID and uniqueIDs (see #Using_the_Public_Registry)
  2. Configure the Repository under test to send to the URL
    • http://hcxw2k1.nist.gov:8080/xdsServices2/registry/soap/portals/yr3a/registry?testeval=11731
  3. For Repository actors and Integrated Document Source/ Document Repository actors that offer the External Document Source option, use test/11731:
  4. For Integrated Document Source/ Document Repository actors that do not offer the External Document Source option:
  5. Verify that you get a successful RegistryResponse message in return.
[edit]

11732

Register Two Documents

Verify that the Document Repository can generate a valid SOAP over HTTP message containing valid metadata representing a Submission Set and two documents. The metadata will be validated against both the ebXML Registry Schema, XDS metadata rules and the ebXML Registry metadata requirements. The metadata must contain a Submission Set and two XDSDocumentEntry objects.

References

ITI TF-2 3.14

Actors Tested

Document Repository

Testing Document Repository Actor and Integrated Document Source/ Document Repository actor

  1. Allocate a new patient ID and uniqueIDs (see #Using_the_Public_Registry)
  2. Configure the Repository under test to send to the URL
    • http://hcxw2k1.nist.gov:8080/xdsServices2/registry/soap/portals/yr3a/registry?testeval=11732
  3. For Repository actors and Integrated Document Source/ Document Repository actors that offer the External Document Source option, use test/11732:
  4. For Integrated Document Source/ Document Repository actors that do not offer the External Document Source option:
  5. Verify that you get a successful RegistryResponse message in return.
[edit]

11733

Accept Register One Document via XDS.a

Verify that the XDS.a Document Registry can accept a Register Document Set-a transaction containing metadata for a Submission Set containing a single Document.

References

ITI TF-2 3.14

Actors Tested

Document Registry

Dependencies

Your registry must support the GetSubmissionSetAndContents Stored Query to pass the evaluation step of this test

You will need to

None

Resources

tests/11733 section of test kit


Testing Document Registry Actor Submit a Submission Set containing a single Document using the Register Document Set-b transaction to your Registry. Then validate the state of your Registry using the GetSubmissionSetAndContents Stored Query.

  1. Configure and run the test plan in the submit sub-directory to perform the submission to your Registry
  2. Evaluate the results.
  3. Configure and run the test plan in the eval sub-directory to perform the evaluation (query) against your Registry
  4. Evaluate the results.
  5. Log the results. You need to return the log.xml file from both the submit and eval parts of the test.
[edit]

11734

Retrieve Document

Verify that Document Consumer can retrieve a document via HTTP Get.

References

ITI TF-2 3.17

Actors Tested

Document Consumer

Testing Document Consumer Actor

Using any available test data:

  1. Perform queries as is appropriate for your document Consumer leading up to the retrieval of the document
  2. Retrieve the document
  3. Log the results: Log, into Kudu, the time and date of the successful Retrieve (EST please)
[edit]

11735

Accept Register Two Documents

Verify that the Document Registry can accept a valid SOAP over HTTP message containing valid metadata representing two documents. The metadata will contain a Submission Set and two XDSDocumentEntry objects.

References

ITI TF-2 3.14

Actors Tested

Document Registry

Testing Document Registry Actor

  1. Edit test/11735
    • Edit the patient ID attributes and uniqueId attributes for your registry
  2. Edit test/11735/test.properties
    • Change the URL parameter to point to your registry.
  3. Run the test using xdstest.
  4. Log the results: #Log_Results_of_Register_transaction_against_your_Registry
[edit]

11736

Submit Replace/Transformation for Existing Document

It is not necessary for Document Sources or Document Repositories to demonstrate this test. Once test #11748 has been successfully passed, this test will automatically be passed as well.


[edit]

11737

Submit Add By-Reference to Submission Set

It is possible to add a document to a submission set that has a different patient ID. First, the document itself must be added to the registry by submitting it in a Submission Set. An example of this is shown in examples/11746. Then to add it to another Submission Set, one that possibly has a different patient ID, submit together or separately an association as follows: 

<Association associationType="HasMember" 
	sourceObject="the submission set UUID" 
	targetObject="symbolic name or UUID of document">
    <Slot name="SubmissionSetStatus">
        <ValueList>
            <Value>Reference</Value>
        </ValueList>
    </Slot>
</Association>

where the SubmissionSetStatus is set to Reference. The new document and the association can be submitted together or separately.

Adding a Document to an existing Submission Set can be done in two ways. Given an existing Submission Set in the Registry:

  1. Submit the new Document (of course submission must include Submission Set) and the Association linking it to the existing Submission Set
  2. Submit the new Document. Submit the Assocation linking it to the existing Submission Set in a separate submission

References

ITI TF-2 3.14

Actors Tested

Document Source

Dependencies

None

Testing Document Source Actor

  1. Allocate a new patient ID and uniqueIDs (see #Using_the_Public_Registry)
  2. Compose Submission Set as described above
    • Also, see examples/11737/metadata.xml
    • where "Another_SubmissionSet" is replaced with the UUID of the target Submission Set (already present in the Registry).
  3. Configure the Document Source to submit to
    • http://hcxw2k1.nist.gov:8080/xdsServices2/registry/soap/portals/yr3a/repository?testeval=11737
  4. Do the Submit.
  5. Verify that you get a successful RegistryResponse message in return
  6. Log the results: #Log_Results_of_Provide_and_Register_or_Register_transaction_tested_against_the_Public_Registry_Repository
[edit]

11738

Submit via Offline Mode

Generate a Provide and Register transaction via Offline Mode.

References

ITI TF-2 3.14

Actors Tested

Document Source
Document Repository

Dependencies

None

Testing Document Source and Document Repository Actors No support facilities are available to test this option. The exception is that Eric provides an email server to use to relay offline message. Contact Eric for details. To demonstrate success, again, contact Eric.

[edit]

11739

Retrieve with mutual TLS

Repeat test #11734 with mutual TLS enabled.

[edit]

11740

Register with mutual TLS

Repeat any Register test over TLS

[edit]

11741

Query with mutual TLS

Repeat any Query test with mutual TLS enabled.

References

ITI TF-2 3.15

Actors Tested

Document Registry
Document Consumer

Dependencies

None

Testing Document Consumer Actor

  1. Rerun any Query test with mutual TLS enabled
  2. Log the results: #Result_Reporting_-_Client_Testing_against_server

Testing Document Registry Actor

  1. Rerun any Register test with mutual TLS enabled
  2. Log the results: #Result_Reporting_-_Server_Testing_using_xdstest



[edit]

11744

Accept Basic Transport - Document Repository

Test basic transport mechanisms, HTTP and SOAP, for transferring metadata without mutual TLS authentication between the test tool xdstest and a Document Repository Actor. xdstest makes an HTTP connection to Document Repository. A simple SOAP package is transferred containing simple non-metadata XML.

References

ITI TF-2 3.14

Actors Tested

Document Repository

Testing Document Repository Actor

  1. Edit test/11744/test.properties
    • Change the URL parameter to point to your repository
  2. Run the test using xdstest
  3. Do not log the results, this is for testing/debug only.





[edit]

11749

Life-cycle Management

Verify that Document Source can issue the three variations on life-cycle management: transformation, addendum, replace with transformation. For each variation, an original document must be discovered through a query. Its UUID must be extracted from the query results. The new document is composed along with an APND, XFRM, or XFRM_RPLC Association to link the new document to the old. The registry adaptor takes responsibility for deprecating the old document for the XFRM_RPLC (replace with transform) type. More specifically, the following must be composed in a submission:

  • Submission Set
  • New Document Metadata
  • A HasMember Association linking Submission Set to the XDSDocumentEntry
  • APND, XFRM, or XFRM_RPLC Association linking the new document to the existing document (in registry, discovered by query) being replaced
  • An ObjectRef pointing to the document being replaced

There must also be a single document included in the submission as an attachment.

References

ITI TF-2 3.14

Actors Tested

Document Source

Dependencies

Must be able to run 11807 - GetDocument query (or equivalent).

Testing Document Source Actor

  1. Allocate new patient IDs as needed (see #Allocating_new_Patient_IDs_for_Testing)
  2. Rerun test 11746 with a new patientId for this test. This creates a document to be manipulated via life-cycle management.
  3. Use the GetDocument query ( test 11807), retrieve the UUID of the document just submitted by 11746
  4. Compose metadata to perform the replacement. An example can be found in 11752/metadata.xml
    • Set the patientId on the Submission Set and document
    • Add an APND Association linking the new document to the one discovered via query.
      • Set the targetObject attribute on the APND Association object to the UUID of the document submitted by 11746
      • Add an ObjectRef for the UUID of the document to indicate that an object already in the registry is being referenced
  5. Configure the Document Source under test to send to the URL
    • http://hcxw2k1.nist.gov:8080/xdsServices2/registry/soap/portals/repository
  6. Have your Document Source send the metadata and the APND document
  7. Verify that you get a successful RegistryResponse message in return
  8. Log the results: #Result_Reporting_-_Client_Testing_against_server


[edit]

11751

Accept two documents via XDS.a

Verify that the XDS.a Document Repository can accept a submission containing two documents via the Provide and Register Document Set-a transaction and forward the updated metadata to the Public Registry.

References

ITI TF-2 3.15
ITI TF-2 3.14

Actors Tested

Document Repository

Dependencies

Test 11710 must be run before this test

You will need to

Allocate a Patient ID

Resources

tests/11751 section of test kit which contains submit and eval sub-directories

Testing Document Repository Actor

Submit a Submission Set containing two Documents using the Provide and Register Document Set transaction to your Repository. Your Repository forwards the metadata on to the Public Registry via the Register Document Set transaction

  1. Configure your Document Repository actor to forward metadata to the Public Registry at EndPoint http://129.6.24.109:9080/axis2/services/registryA2doc
  2. Configure and run the test plan in the submit sub-directory to perform the submission to your Repository
  3. Evaluate the results.
  4. Configure and run the test plan in the eval sub-directory to query and evaluate your Repository's submission to the Public Registry
  5. Evaluate the results.
  6. Log the results of both submit and eval parts of the test

Evaluation by the Public Registry

  1. Schema
  2. Metadata Validator
  3. Correct content (Submission Set containing two Documents)
  4. Correct hash and size computed by Repository actor and placed in metadata.

Evaluation by xdstest2

  1. Evaluation query returns correct metadata
[edit]

11763

Submit Example HL7 Lab

Submit an example of your HL7 Lab message for review.

References

None

Actors Tested

Document Source

Dependencies

None

Testing Document Consumer Source

  1. Submit, as a log entry, an example of your HL7 Lab
[edit]

11764

Submit Example PDF

Submit an example of your PDF Document for review.

References

None

Actors Tested

Document Source

Dependencies

None

Testing Document Consumer Source

  1. Submit, as a log, an example of your PDF document for review.
[edit]

11765

Submit Example Wrapped PDF

Submit an example of your PDF Document wrapped in a CDA R2 wrapper for review.

References

None

Actors Tested

Document Source

Dependencies

None

Testing Document Consumer Source

  1. Submit example document as a log to this test
[edit]

11766

Submit Example XDS-MS/CDA R2

Submit an example of your XDS-MS/CDA R2 for review.

References

None

Actors Tested

Document Source

Dependencies

None

Testing Document Consumer Source

  1. Submit your example as a log for this test.
[edit]

11801

Query for ObjectRefs

Verify that Document Consumer can query the Registry for ObjectRefs and that a Document Registry can service such a query.

Why does this exist? Any potential document listing can be very long. To use LeafClass queries to generate listing pages (pick lists) would be very slow as this requires the Registry to assemble metadata XML for each document. Starting with an ObjectRef query that returns just the list of identifiers for each document can be used to generate 'page at a time' listing. For each page, a full LeafClass query can be issued to get the content for display. 11801 (and 11802) are intended for debug usage only.

References

ITI TF-2 3.14

Actors Tested

Document Consumer
Document Registry

Dependencies

Test #11870 must be run before this test. It loads necessary test data.

Testing Document Consumer Actor

  1. Configure the Document Consumer under test to send to the URL
    • http://hcxw2k1.nist.gov:8080/xdsServices2/registry/soap/portals/query
  2. Have your Document Consumer send the query in test/11801/query.xml.
  3. Verify that you get a successful RegistryResponse message in return containing 1 ObjectRef.
  4. Log the results: #Result_Reporting_-_Client_Testing_against_server

Testing Document Registry Actor

  1. Edit test/11801/test.properties
  2. Change the URL parameter to point to your registry.
  3. Run the test using xdstest.
  4. Log the results: #Result_Reporting_-_Server_Testing_using_xdstest
[edit]

11802

Query for LeafClass

Verify that Document Consumer can query the Registry for LeafClass and that a Document Registry can service such a query. This is intended for debug purposes only.

References

ITI TF-2 3.14

Actors Tested

Document Consumer
Document Registry

Dependencies

Test 11870 must be run before this test

Testing Document Consumer Actor

  1. Configure the Document Consumer under test to send to the URL
    • http://hcxw2k1.nist.gov:8080/xdsServices2/registry/soap/portals/query
  2. Have your Document Consumer send the query in test/11802/query.xml.
  3. Verify that you get a successful RegistryResponse message in return containing 1 ExtrinsicObject.
  4. Log the results: #Result_Reporting_-_Client_Testing_against_server

Testing Document Registry Actor

  1. Edit test/11802/test.properties
  2. Change the URL parameter to point to your registry.
  3. Run the test using xdstest.
  4. Log the results: #Result_Reporting_-_Server_Testing_using_xdstest
[edit]

11803

FindDocuments Query

Verify that Document Consumer can use the FindDocuments query and that a Document Registry can service such a query.

References

ITI TF-2 3.16

Actors Tested

Document Consumer
Document Registry

Dependencies

When testing Document Registry actor, test 11870 must be run before this test to establish test data

Testing Document Consumer Actor

  1. Configure the Document Consumer under test to send to the URL
    • http://hcxw2k1.nist.gov:8080/xdsServices2/registry/soap/portals/query
  2. Have your Document Consumer issue the query in
    • 11803/find_documents.xml.
  1. Verify that you get successful RegistryResponse messages in return containing ExtrinsicObject(s).
  2. Log the results: #Result_Reporting_-_Client_Testing_against_server

Testing Document Registry Actor

  1. Edit the test.properties file in 11803.
    • Change the URL parameter to point to your registry.
  2. Run the tests using xdstest.
  3. Verify that you get a successful RegistryResponse messages in return containing ExtrinsicObject(s)
  4. Log the results: #Result_Reporting_-_Server_Testing_using_xdstest
[edit]

11804

FindSubmissionSets Query

Verify that Document Consumer can use the FindSubmissionSets query and that a Document Registry can service such a query.

References

ITI TF-2 3.16

Actors Tested

Document Consumer
Document Registry

Dependencies

When testing Document Registry actor, test 11870 must be run before this test to establish test data

Testing Document Consumer Actor

  1. Configure the Document Consumer under test to send to the URL
    • http://hcxw2k1.nist.gov:8080/xdsServices2/registry/soap/portals/query
  2. Have your Document Consumer issue the query in
    • 11804/find_submission_sets.xml.
  1. Verify that you get successful RegistryResponse messages in return containing RegistryPackages(s).
  2. Log the results: #Result_Reporting_-_Client_Testing_against_server

Testing Document Registry Actor

  1. Edit the test.properties file in 11804.
    • Change the URL parameter to point to your registry.
  2. Run the tests using xdstest.
  3. Verify that you get a successful RegistryResponse messages in return containing RegistryPackages(s)
  4. Log the results: #Result_Reporting_-_Server_Testing_using_xdstest


[edit]

11805

FindFolders Query

Verify that Document Consumer can use the FindFolders query and that a Document Registry can service such a query.

References

ITI TF-2 3.16

Actors Tested

Document Consumer
Document Registry

Dependencies

When testing Document Registry actor, test 11870 must be run before this test to establish test data

Testing Document Consumer Actor

  1. Configure the Document Consumer under test to send to the URL
    • http://hcxw2k1.nist.gov:8080/xdsServices2/registry/soap/portals/query
  2. Have your Document Consumer issue the query in
    • 11805/find_folders.xml.
  1. Verify that you get successful RegistryResponse messages in return containing RegistryPackages(s).
  2. Log the results: #Result_Reporting_-_Client_Testing_against_server

Testing Document Registry Actor

  1. Edit the test.properties file in 11805.
    • Change the URL parameter to point to your registry.
  2. Run the tests using xdstest.
  3. Verify that you get a successful RegistryResponse messages in return containing RegistryPackages(s)
  4. Log the results: #Result_Reporting_-_Server_Testing_using_xdstest
[edit]

11806

GetAll Query

Verify that Document Consumer can use the GetAll query and that a Document Registry can service such a query.

To run efficiently, this query must be broken down into 3 queries:

  1. Query for ExtrinsicObjects (XDSDocumentEntry)
  2. Query for RegistryPackages (XDSSubmissionSet and XDSFolder)
  3. Query for Associations linking the objects

References

ITI TF-2 3.16

Actors Tested

Document Consumer
Document Registry

Dependencies

When testing Document Registry actor, test 11870 must be run before this test to establish test data

Testing Document Consumer Actor

  1. Configure the Document Consumer under test to send to the URL
    • http://hcxw2k1.nist.gov:8080/xdsServices2/registry/soap/portals/query
  2. Have your Document Consumer issue the queries in
    • 11806a/get_all_1.xml.
    • 11806b/get_all_2.xml.
    • 11806c/get_all_3.xml.
  3. Verify that you get successful RegistryResponse messages in return containing, in total, 1 ExtrinsicObject, 1 Submission Set, and 1 Association.
  4. Log the results: #Result_Reporting_-_Client_Testing_against_server

Testing Document Registry Actor

  1. Edit the test.properties file in 11806a 11806b and 11806c.
    • Change the URL parameter to point to your registry.
  2. Run the tests using xdstest.
  3. Verify that you get a successful RegistryResponse messages in return containing, in total, 1 ExtrinsicObject, 1 Submission Set, and 1 Association.
  4. Log the results: #Result_Reporting_-_Server_Testing_using_xdstest
[edit]

11807

GetDocument Query

Verify that Document Consumer can use the GetDocument query and that a Document Registry can service such a query.

References

ITI TF-2 3.14

Actors Tested

Document Consumer
Document Registry

Dependencies

When testing Document Registry actor, test #11870 must be run before this test to establish test data

Testing Document Consumer Actor

  1. Configure the Document Consumer under test to send to the URL
    • http://hcxw2k1.nist.gov:8080/xdsServices2/registry/soap/portals/query
  2. Have your Document Consumer send the query in test/11807/query_by_uniqueid.xml.
  3. Verify that you get a successful RegistryResponse message in return containing 1 ExtrinsicObject.
  4. Log the results: #Result_Reporting_-_Client_Testing_against_server

Testing Document Registry Actor

  1. Edit test/11807/test.properties
  2. Change the URL parameter to point to your registry.
  3. Run the test using xdstest.
  4. Log the results: #Result_Reporting_-_Server_Testing_using_xdstest
[edit]

11808

GetSubmissionSetContents Query


Verify that Document Consumer can use the GetSubmissionSetContents query and that a Document Registry can service such a query.

This query allows the Document Consumer to specify the documents by UUID (native to Registry standard) or uniqueId (native to health care environment). This Query test offers opportunity to test either.

To run efficiently, this query must be broken down into 3 queries:

  1. Query for XDSSubmissionSet
  2. Query for Documents
  3. Query for XDSFolders

References

ITI TF-2 3.16

Actors Tested

Document Consumer
Document Registry

Dependencies

When testing Document Registry actor, test 11870 must be run before this test to establish test data

Testing Document Consumer Actor

  1. Configure the Document Consumer under test to send to the URL
    • http://hcxw2k1.nist.gov:8080/xdsServices2/registry/soap/portals/query
  2. Have your Document Consumer issue the queries in
    • uuid/11808a/get_submission_set_contents_1.xml
    • uuid/11808b/get_submission_set_contents_2.xml
    • uuid/11808c/get_submission_set_contents_3.xml

To use these queries, the parameter $uniqueId must be changed to a real value

AND

  • uniqueId/11808a/get_submission_set_contents_1.xml
  • uniqueId/11808b/get_submission_set_contents_2.xml
  • uniqueId/11808c/get_submission_set_contents_3.xml

To use these queries, the parameter $uuid must be changed to a real value

  1. Verify that you get successful RegistryResponse messages in return
  2. containing XDSSubmissionSet, Document and XDSFolder.
  3. Log the results: #Result_Reporting_-_Client_Testing_against_server

Testing Document Registry Actor

  1. Edit the test.properties file in 11808a 11808b and 11808c.
    • Change the URL parameter to point to your registry.
  2. Run the tests using xdstest.
  3. Verify that you get a successful RegistryResponse messages in return
  4. containing XDSSubmissionSet, Document and XDSFolder.
  5. Log the results: #Result_Reporting_-_Server_Testing_using_xdstest
[edit]

11809

GetFolderContents Query


Verify that Document Consumer can use the GetFolderContents query and that a Document Registry can service such a query.

To run efficiently, this query must be broken down into 3 queries:

  1. Query for XDSFolder
  2. Query for XDSSubmissionSet
  3. Query for Documents

References

ITI TF-2 3.16

Actors Tested

Document Consumer
Document Registry

Dependencies

When testing Document Registry actor, test 11870 must be run before this test to establish test data

Testing Document Consumer Actor

  1. Configure the Document Consumer under test to send to the URL
    • http://hcxw2k1.nist.gov:8080/xdsServices2/registry/soap/portals/query
  2. Have your Document Consumer issue the queries in
    • uuid/11809a/get_folder_contents_1.xml
    • uuid/11809b/get_folder_contents_2.xml
    • uuid/11809c/get_folder_contents_3.xml

AND

  • uniqueId/11809a/get_folder_contents_1.xml
  • uniqueId/11809b/get_folder_contents_2.xml
  • uniqueId/11809c/get_folder_contents_3.xml
  1. Verify that you get successful RegistryResponse messages in return
  2. containing XDSSubmissionSet, Document and XDSFolder.
  3. Log the results: #Result_Reporting_-_Client_Testing_against_server

Testing Document Registry Actor

  1. Edit the test.properties file in 11809a 11809b and 11809c.
    • Change the URL parameter to point to your registry.
  2. Run the tests using xdstest.
  3. Verify that you get a successful RegistryResponse messages in return
  4. containing XDSSubmissionSet, Document and XDSFolder.
  5. Log the results: #Result_Reporting_-_Server_Testing_using_xdstest


[edit]

11810

GetFoldersForDocument Query


Verify that Document Consumer can use the GetFolderForDocument query and that a Document Registry can service such a query.

References

ITI TF-2 3.16

Actors Tested

Document Consumer
Document Registry

Dependencies

When testing Document Registry actor, test 11870 must be run before this test to establish test data

Testing Document Consumer Actor

  1. Configure the Document Consumer under test to send to the URL
    • http://hcxw2k1.nist.gov:8080/xdsServices2/registry/soap/portals/query
  2. Have your Document Consumer issue the query in
    • uuid/11810/get_folders_for_document.xml

AND

  • uniqueId/11810/get_folders_for_document.xml
  1. Verify that you get successful RegistryResponse messages in return
  2. containing XDSFolder(s).
  3. Log the results: #Result_Reporting_-_Client_Testing_against_server

Testing Document Registry Actor

  1. Edit the test.properties file in 11810.
    • Change the URL parameter to point to your registry.
  2. Run the tests using xdstest.
  3. Verify that you get a successful RegistryResponse messages in return
  4. containing XDSFolder(s).
  5. Log the results: #Result_Reporting_-_Server_Testing_using_xdstest


[edit]

11811

GetAddendums Query


Verify that Document Consumer can use the GetAddendums query and that a Document Registry can service such a query.

References

ITI TF-2 3.16

Actors Tested

Document Consumer
Document Registry

Dependencies

When testing Document Registry actor, test 11870 must be run before this test to establish test data

Testing Document Consumer Actor

  1. Configure the Document Consumer under test to send to the URL
    • http://hcxw2k1.nist.gov:8080/xdsServices2/registry/soap/portals/query
  2. Have your Document Consumer issue the query in
    • uuid/11811/get_addendums.xml

AND

  • uniqueId/11811/get_addendums.xml
  1. Verify that you get successful RegistryResponse messages in return
  2. containing ExtrinsicObject(s).
  3. Log the results: #Result_Reporting_-_Client_Testing_against_server

Testing Document Registry Actor

  1. Edit the test.properties file in 11811.
    • Change the URL parameter to point to your registry.
  2. Run the tests using xdstest.
  3. Verify that you get a successful RegistryResponse messages in return
  4. containing ExtrinsicObject(s).
  5. Log the results: #Result_Reporting_-_Server_Testing_using_xdstest


[edit]

11812

GetHistory Query


Verify that Document Consumer can use the GetHistory query and that a Document Registry can service such a query.

References

ITI TF-2 3.16

Actors Tested

Document Consumer
Document Registry

Dependencies

When testing Document Registry actor, test 11870 must be run before this test to establish test data

Testing Document Consumer Actor

  1. Configure the Document Consumer under test to send to the URL
    • http://hcxw2k1.nist.gov:8080/xdsServices2/registry/soap/portals/query
  2. Have your Document Consumer issue the query in
  • uuid/11812/get_history.xml

AND

  • uniqueId/11812/get_history.xml
  1. Verify that you get successful RegistryResponse messages in return
  2. containing ExtrinsicObject(s).
  3. Log the results: #Result_Reporting_-_Client_Testing_against_server

Testing Document Registry Actor

  1. Edit the test.properties file in 11812.
    • Change the URL parameter to point to your registry.
  2. Run the tests using xdstest.
  3. Verify that you get a successful RegistryResponse messages in return
  4. containing ExtrinsicObject(s).
  5. Log the results: #Result_Reporting_-_Server_Testing_using_xdstest
[edit]

11813

GetTransformations Query


Verify that Document Consumer can use the GetTransformations query and that a Document Registry can service such a query.

References

ITI TF-2 3.16

Actors Tested

Document Consumer
Document Registry

Dependencies

When testing Document Registry actor, test 11870 must be run before this test to establish test data

Testing Document Consumer Actor

  1. Configure the Document Consumer under test to send to the URL
    • http://hcxw2k1.nist.gov:8080/xdsServices2/registry/soap/portals/query
  2. Have your Document Consumer issue the query in
  • uuid/11813/get_transformations.xml

AND

  • uniqueId/11813/get_transformations.xml
  1. Verify that you get successful RegistryResponse messages in return
  2. containing ExtrinsicObject(s).
  3. Log the results: #Result_Reporting_-_Client_Testing_against_server

Testing Document Registry Actor

  1. Edit the test.properties file in 11813.
    • Change the URL parameter to point to your registry.
  2. Run the tests using xdstest.
  3. Verify that you get a successful RegistryResponse messages in return
  4. containing ExtrinsicObject(s).
  5. Log the results: #Result_Reporting_-_Server_Testing_using_xdstest


[edit]

11814

Basic Transport

Test basic transport mechanisms, HTTP and SOAP, for transferring metadata without mutual TLS authentication. Document Consumer makes an HTTP connection to Document Registry. A simple SOAP package is transferred containing simple non-metadata XML. The Registry may return an error message because the payload is not metadata. This is acceptable.

References

ITI TF-2 3.14

Actors Tested

Document Consumer
Document Registry

Testing Document Consumer Actor

  1. Configure the Document Source under test to send to the URL
    • http://hcxw2k1.nist.gov:8080/xdsServices2/registry/soap/test?testid=11814
  2. Have your Document Source send a SOAP over HTTP message with arbitrary content
  3. Verify that you get a valid SOAP message back, it may contain an error message from the repository. If you using a SOAP parser, errors in parsing are likely. This test is intended to support folks using simpler tools. The parsing error is acceptable.
  4. Log the results: #Result_Reporting_-_Client_Testing_against_server

Testing Document Repository Actor

  1. Edit test/11814/test.properties
  2. Change the URL parameter to point to your repository
  3. Run the test using xdstest
  4. Log the results: #Result_Reporting_-_Server_Testing_using_xdstest
[edit]

11827

Accept One Document via XDS.a

Verify that the XDS.a Document Repository can accept a SOAP with Attachments message containing metadata representing a Submission Set and a single document. The content of the document are include as an attachment to the message according to the SOAP with Attachments standard. Specifically, the metadata shall be augmented with the following attributes to match the included document:

  • XDSDocumentEntry.size
  • XDSDocumentEntry.hash
  • XDSDocumentEntry.URI

and then forwarded on to the Public Registry in a XDS.a Register transaction. These attributes must be calculated and added correctly. The URI installed into XDSDocumentEntry.URI need not be publicly available on the Internet. See tests/11733 for an example of how to add these metadata attributes.

Warning: some testers have run the utility unix2dos on the entire test kit and it will break this test. The file attachment, my_document.txt, when modified by this utility will no longer match in size or hash.

References

ITI TF-2 3.14
ITI TF-2 3.15

Actors Tested

XDS.a Document Repository

Dependencies

Test 11710 must be run before this test

You will need to

Allocate a Patient ID

Resources

tests/11827 section of test kit which has two sections, submit and eval
XDS Test Log Browser

Testing Document Repository Actor

Submit a Submission Set containing a single Document using the Provide and Register Document Set transaction to your XDS.a Repository. Your Repository forwards the metadata on to the Public Registry via the Register Document Set transaction

  1. Configure your Document Repository actor to forward metadata to the Public Registry at EndPoint http://129.6.24.109:9080/axis2/services/test11827
  2. Configure and run the test plan in the submit sub-directory to perform the submission to your Repository
  3. Evaluate the results.
  4. Configure and run the test plan in the eval sub-directory to query and evaluate your Repository's submission to the Public Registry
  5. Evaluate the results.
  6. Log the results of both submit and eval parts of the test

Evaluation by the Public Registry

  1. Schema
  2. Metadata Validator
  3. Correct content (Submission Set containing a single Document)
  4. Correct hash and size computed by Repository actor and placed in metadata.


Evaluation by xdstest2

  1. Evaluation query returns correct metadata
[edit]

11870

Load test data set 1

Load test data into Registry to support query testing.

References

ITI TF-2 3.14

Actors Tested

None

Dependencies

None

Testing Document Registry Actor

  1. Edit testdatga/11870/test.properties
  2. Change the URL parameter to point to your registry
  3. Edit testdata/11870/metadata.xml
    • Change $PatientId to a real patient ID globally
    • Change all uniqueIds (start with $uniqueId) to real uniqueIDs in your system
  4. Run the test using xdstest.
  5. Log the results: no logging required

This submission contains a single Submission Set, 3 Documents, and Associations to make the documents part of the Submission Set.

[edit]

11871

Accept Document Replace via XDS.a

Verify that the XDS.a Document Registry can accept a Document Replace via the Register Document Set transaction. No transformations or addendum are present.

References

ITI TF-2 3.14

Actors Tested

Document Registry

Dependencies

None

You will need to

Testing Document Registry Actor

Submit a Document Replace using the Register Document Set transaction to your Registry. Then validate the state of your Registry using the GetSubmissionSetAndContents Stored Query.

  1. Configure and run the test plan in the submit sub-directory to perform the submission to your Registry. This will submit a submission set containing a single document. This document will be the target of the replace operation in the next step.
  2. Evaluate the results.
  3. Configure and run the test plan in the rplc sub-directory to perform the replacement. This will submit a submission set containing a single document, the replacement document, and the RPLC association which will trigger the registry to deprecate the original document (submitted in the last step).
  4. Evaluate the results.
  5. Configure and run the test plan in the eval sub-directory to perform the evaluation (query) against your Registry. This step has two parts. First a test step named validate_deprecate will test that the orginal document is deprecated. The second test step, validate_new, will test that the replacement document is present.
  6. Evaluate the results.
  7. Log the results. You need to return the log.xml files from the submit, rplc, and eval parts of the test.

Evaluation

  1. Submit, rplc, eval steps must be successful
  2. Within the eval step, the validate_deprecate test step must show that the original document is deprecated and the validate_new must show that the replacement document is approved (status = Approved).
[edit]

11873

Accept Document Addendum via XDS.a

Verify that the XDS.a Document Registry can accept a Document Addendum via the Register Document Set transaction.

References

ITI TF-2 3.14

Actors Tested

Document Registry

Dependencies

None

You will need to

None

Testing Document Registry Actor

Submit a Document Addendum using the Register Document Set transaction to your Registry. Then validate the state of your Registry using the GetSubmissionSetAndContents Stored Query. An overview of this test is:

  • Submit a submission set containing a single document. Do this twice.
  • Replace (issue RPLC) for one of the documents
  • Attempt addendum on both original documents. Creating an addendum on the replaced document must fail. Creating an addendum on the non-replaced document must succeed.
  • Issue GetSubmissionSetandContents Stored Query on both addendum attempts. The plain addendum must succeed and the one involving the replace must fail. In this case succeed and fail simply refers to whether content is returned and not the status on the Stored Query.

The detailed steps are:

  1. Configu