XD Async Testing

From IHEWiki

Revision as of 16:44, 1 November 2013 by BillM (Talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Async Testing has been removed from the XDS Toolkit. We are working to restore this functionality but for now there is no support for this functionality.

Contents

Overview

This toolkit supports asynchronous web services testing for XDS.a and XCA profiles. This page describes the testing approach, tools available, and specific MESA tests used for testing asynchronous web services. As with other XD* related testing, the Public Registry server serves as a message recipient and xdstest (command line tool) serves as a message initiator. A new element is introduced, Apache Synapse, which acts as a sync-async converter. Synapse is an Enterprise Service Bus (ESB) engine that can be downloaded freely and used for testing.

Server Testing

This covers testing of Document Repository, Document Registry, and Receiving Gateway actors.

The xdstest tool has been upgraded to generate asynchronous web service requests. A new instruction <ASync/> has been added to the language. This instruction must be placed inside a <*Transaction/> instruction. When this instruction is used:

  • The .as transaction is used from actors.xml (Provide and Register will take the pr.as endpoint from actors.xml)
  • The generated SOAP Header will include the MessageID and ReplyTo headers
  • The endpoint contained in the ReplyTo header will be opened and will accept a reply message
  • An HTTP 202 message is expected as the immediate reply to the request (same connection as the request)
  • The response is expected, in a separate connection, to the ReplyTo endpoint

All other aspects of the web service interaction are the same as when using synchronous interactions. Since you are testing your server in your development environment, it is expected that there is no firewall present to prevent asynchronous message interaction with the xdstest tool.

Client testing

This covers testing of Document Source, Document Repository, Document Consumer, and Initiating Gateway actors.

Two testing environments are anticipated, one where your client sits behind a firewall and is incapable of receiving asynchronous messages (response is carried in a separate connection initiated outside your firewall which the firewall blocks) and the other where your client is not behind a firewall and is capable of receiving asynchronous messages (no firewall to block the response connection back to your client). With synchronous messages, the request and response are carried in the same connection initiated from your client behind your firewall.

Direct testing

In the no-firewall (or outside the firewall) situation, the Public Registry will respond to asynchronous web service requests.

Indirect testing with Synapse

When you must operate behind a firewall that will block incoming messages (asynchronous responses), you must install and use Apache Synapse to mediate your web service requests. In this case, Synapse becomes a local extension to the Public Registry. It accepts asynchronous requests from your client, forwards the request to the Public Registry server over synchronous web services, and finally replies to your client via asynchronous web services.

Getting Synapse

Synapse (version 1.2) can be downloaded directly from the Apache site:

http://synapse.apache.org/download/1.2/download.cgi

Configuring and running Synapse

Once downloaded and unpacked, you must:

  • Install the configuration file for xdstest by copying xdstoolkit/xdstest/synapse.xml to the repository/conf directory of the synapse installation, overwriting the default file.
  • In the synapse installation, move to the directory bin and execute either synapse.bat or synapse.sh depending on your system type (Windows vs Unix-like)

Synapse should now start and listen on its default port 8280. Its log output looks like:

bmajur:bin bill$ synapse 
 Using Bouncy castle JAR for Java 1.5
Starting Synapse/Java ...
Using SYNAPSE_HOME:    /opt/local/synapse-1.2
Using JAVA_HOME:       /System/Library/Frameworks/JavaVM.Framework/Home
Using SYNAPSE_XML:     /opt/local/synapse-1.2/repository/conf/synapse.xml
2009-01-09 15:41:54,027 [-] [main]  INFO ServerManager Using Synapse home as : /opt/local/synapse-1.2
2009-01-09 15:41:54,029 [-] [main]  INFO ServerManager Using the Axis2 Repository : /opt/local/synapse-1.2/repository
2009-01-09 15:41:54,029 [-] [main]  INFO ServerManager Using the axis2.xml : /opt/local/synapse-1.2/repository/conf/axis2.xml
2009-01-09 15:41:54,030 [-] [main]  INFO ServerManager Using server name : localhost
2009-01-09 15:41:54,030 [-] [main]  INFO ServerManager The timeout handler will run every : 15s
2009-01-09 15:41:54,699 [-] [main]  INFO SynapseInitializationModule Initializing Synapse at : Fri Jan 09 15:41:54 EST 2009
2009-01-09 15:41:54,700 [192.168.254.153-bmajur.local] [main]  INFO SynapseInitializationModule Loading mediator extensions...
2009-01-09 15:41:54,700 [192.168.254.153-bmajur.local] [main]  INFO SynapseInitializationModule Initializing the Synapse configuration ...
2009-01-09 15:41:54,702 [192.168.254.153-bmajur.local] [main] DEBUG SynapseConfigurationBuilder synapse.properties file is loading from classpath
2009-01-09 15:41:54,706 [192.168.254.153-bmajur.local] [main] DEBUG DataSourceRegistrar No DataSources defined for initialization..
2009-01-09 15:41:54,708 [192.168.254.153-bmajur.local] [main]  INFO XMLConfigurationBuilder Generating the Synapse configuration model by parsing the XML configuration
2009-01-09 15:41:54,719 [192.168.254.153-bmajur.local] [main] DEBUG ConfigurationFactoryAndSerializerFinder getConfiguration({http://ws.apache.org/ns/synapse}definitions)
2009-01-09 15:41:54,756 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder Added MediatorFactory class org.apache.synapse.mediators.spring.SpringMediatorFactory to handle {http://ws.apache.org/ns/synapse/spring}spring
2009-01-09 15:41:54,758 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder Added MediatorFactory class org.apache.synapse.mediators.bsf.ScriptMediatorFactory to handle {http://ws.apache.org/ns/synapse}script
2009-01-09 15:41:54,759 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder Added MediatorFactory class org.apache.synapse.mediators.throttle.ThrottleMediatorFactory to handle {http://ws.apache.org/ns/synapse}throttle
2009-01-09 15:41:54,762 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder Added MediatorFactory class org.apache.synapse.mediators.xquery.XQueryMediatorFactory to handle {http://ws.apache.org/ns/synapse}xquery
2009-01-09 15:41:54,762 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}log)
2009-01-09 15:41:54,765 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}in)
2009-01-09 15:41:54,766 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}log)
2009-01-09 15:41:54,766 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}filter)
2009-01-09 15:41:54,801 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}header)
2009-01-09 15:41:54,803 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}send)
2009-01-09 15:41:54,814 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}drop)
2009-01-09 15:41:54,815 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}filter)
2009-01-09 15:41:54,815 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}header)
2009-01-09 15:41:54,816 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}send)
2009-01-09 15:41:54,816 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}drop)
2009-01-09 15:41:54,816 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}filter)
2009-01-09 15:41:54,819 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}header)
2009-01-09 15:41:54,819 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}send)
2009-01-09 15:41:54,820 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}drop)
2009-01-09 15:41:54,820 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}filter)
2009-01-09 15:41:54,820 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}header)
2009-01-09 15:41:54,821 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}send)
2009-01-09 15:41:54,821 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}drop)
2009-01-09 15:41:54,821 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}filter)
2009-01-09 15:41:54,822 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}header)
2009-01-09 15:41:54,822 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}send)
2009-01-09 15:41:54,822 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}drop)
2009-01-09 15:41:54,823 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}filter)
2009-01-09 15:41:54,823 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}header)
2009-01-09 15:41:54,824 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}send)
2009-01-09 15:41:54,824 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}drop)
2009-01-09 15:41:54,824 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}out)
2009-01-09 15:41:54,825 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}log)
2009-01-09 15:41:54,825 [192.168.254.153-bmajur.local] [main] DEBUG MediatorFactoryFinder getMediator({http://ws.apache.org/ns/synapse}send)
2009-01-09 15:41:54,828 [192.168.254.153-bmajur.local] [main]  INFO SynapseConfigurationBuilder Loaded Synapse configuration from : /opt/local/synapse-1.2/repository/conf/synapse.xml
2009-01-09 15:41:54,843 [192.168.254.153-bmajur.local] [main] DEBUG SynapseConfiguration Initializing the Synapse Configuration
2009-01-09 15:41:54,843 [192.168.254.153-bmajur.local] [main] DEBUG SequenceMediator Initializing child mediators
2009-01-09 15:41:54,844 [192.168.254.153-bmajur.local] [main] DEBUG InMediator Initializing child mediators
2009-01-09 15:41:54,844 [192.168.254.153-bmajur.local] [main] DEBUG FilterMediator Initializing child mediators
2009-01-09 15:41:54,844 [192.168.254.153-bmajur.local] [main] DEBUG FilterMediator Initializing child mediators
2009-01-09 15:41:54,844 [192.168.254.153-bmajur.local] [main] DEBUG FilterMediator Initializing child mediators
2009-01-09 15:41:54,845 [192.168.254.153-bmajur.local] [main] DEBUG FilterMediator Initializing child mediators
2009-01-09 15:41:54,845 [192.168.254.153-bmajur.local] [main] DEBUG FilterMediator Initializing child mediators
2009-01-09 15:41:54,845 [192.168.254.153-bmajur.local] [main] DEBUG FilterMediator Initializing child mediators
2009-01-09 15:41:54,845 [192.168.254.153-bmajur.local] [main] DEBUG OutMediator Initializing child mediators
2009-01-09 15:41:54,846 [192.168.254.153-bmajur.local] [main] DEBUG SequenceMediator Initializing child mediators
2009-01-09 15:41:54,846 [192.168.254.153-bmajur.local] [main]  INFO SynapseInitializationModule Deploying the Synapse service..
2009-01-09 15:41:54,872 [192.168.254.153-bmajur.local] [main]  INFO SynapseInitializationModule Synapse server name : localhost
2009-01-09 15:41:54,872 [192.168.254.153-bmajur.local] [main]  INFO SynapseInitializationModule Deploying Proxy services...
2009-01-09 15:41:54,872 [192.168.254.153-bmajur.local] [main]  INFO SynapseInitializationModule Synapse initialized successfully...!
2009-01-09 15:41:54,903 [192.168.254.153-bmajur.local] [main]  INFO HttpCoreNIOSSLSender Loading Identity Keystore from : lib/identity.jks
2009-01-09 15:41:55,039 [192.168.254.153-bmajur.local] [main]  INFO HttpCoreNIOSSLSender Loading Trust Keystore from : lib/trust.jks
2009-01-09 15:41:55,062 [192.168.254.153-bmajur.local] [main]  INFO HttpCoreNIOSender HTTPS Sender starting
2009-01-09 15:41:55,206 [192.168.254.153-bmajur.local] [main]  INFO HttpCoreNIOSender HTTP Sender starting
2009-01-09 15:41:55,209 [192.168.254.153-bmajur.local] [main]  INFO VFSTransportSender VFS Sender started
2009-01-09 15:41:55,312 [192.168.254.153-bmajur.local] [main]  INFO JMSSender JMS Sender started
2009-01-09 15:41:55,320 [192.168.254.153-bmajur.local] [main]  INFO ServerManager Starting transport https on port 8243
2009-01-09 15:41:55,321 [192.168.254.153-bmajur.local] [main]  INFO HttpCoreNIOSSLListener Loading Identity Keystore from : lib/identity.jks
2009-01-09 15:41:55,322 [192.168.254.153-bmajur.local] [main]  INFO HttpCoreNIOSSLListener Loading Trust Keystore from : lib/trust.jks
2009-01-09 15:41:55,324 [192.168.254.153-bmajur.local] [main]  INFO HttpCoreNIOListener HTTPS Listener starting on port : 8243
2009-01-09 15:41:55,325 [192.168.254.153-bmajur.local] [main]  INFO ServerManager Starting transport http on port 8280
2009-01-09 15:41:55,325 [192.168.254.153-bmajur.local] [main]  INFO HttpCoreNIOListener HTTP Listener starting on port : 8280
2009-01-09 15:41:55,325 [192.168.254.153-bmajur.local] [main]  INFO ServerManager Starting transport vfs
2009-01-09 15:41:55,376 [192.168.254.153-bmajur.local] [main]  INFO VFSTransportListener VFS Listener started
2009-01-09 15:41:55,403 [192.168.254.153-bmajur.local] [main]  INFO ServerManager Starting transport mailto
2009-01-09 15:41:55,404 [192.168.254.153-bmajur.local] [main]  INFO MailTransportListener MAILTO Listener started
2009-01-09 15:41:55,405 [192.168.254.153-bmajur.local] [main]  INFO ServerManager Ready for processing

Converting Public Registry Endpoints to Synapse

Synapse is configured to accept your asynchronous inputs, convert them to synchronous mode for communication through your firewall to the Public Registry and respond to your client via asynchronous. The last thing to do is configure your Document Source, Document Repository, Document Consumer or Initiating Gateway actor implementation to send to Synapse.

The rules for generating the proper WS Endpoint are as follows:

  • hostname - the host running Synapse
  • port - 8280
  • service name - anything you like.

The Synapse configuration delivered with xdstoolkit, maps WS:Actions to Public Registry endpoints. For example, messages with WS:Action urn:ihe:iti:2007:RegistryStoredQueryAsync are forwarded to http://ihexds.nist.gov:9080/tf5/services/xdsregistryb. The config file synapse.xml is fairly easy to read if you want to see the rest of the WS:Action to Endpoint mappings.

Testing the installation

In my development, I always test xdstest against the Public Registry to see that the two tools are consistent. First I test against a local copy of the Public Registry running on my development machine. This involves no firewall and does not use Synapse. To test Provide and Register (test 12333) over asynchronous web services:

xdstest -Dsite=dev -err -t 12333 submit

To repeat that test against the Public Registry server (and through the firewall) I use a version of xdstest that is proxy-aware, xdstestp:

xdstestp -Dsite=pub -err -t 12333 submit


The key part of this script is
java gov.nist.registry.xdstest2.XdsTest -Dhttp.proxyHost=localhost -Dhttp.proxyPort=8280 $params

which shows Java using an HTTP proxy on localhost:8280 which is where I run Synapse. The communication between xdstestp and Synapse uses asynchronous web services and the communications between Synapse and the Public Registry server uses synchronous web services.

Asynchronous web services over TLS

The Synapse configuration supplied is not capable of running asynchronous web service over TLS.

Synapse Quirks

Synapse has a bug that conflicts with IHE web service requirements. All IHE defined web services specify the web service action to be used for both the request and response message. The action is reported in both the SOAP Header and as the top level element in the SOAP Body. Response messages produced by Synapse always have the value of urn:mediateResponse in the SOAP Header. Please ignore this mistake in your testing. A bug report has been filed with the Synapse development team and I will update these tools as soon as a patch is available. The xdstest tool has been modified to accept this value without reporting an error.

Message Exchange Patterns

HTTP messaging is fundamentally a synchronous service, a reply is returned in the same connection as the request. When web services are using with a HTTP binding the web service reply is returned in the HTTP reply message, usually with a status of 200. When asynchronous web services are used over HTTP, there is still a synchronous response to complete the HTTP interaction. Then, later, a new HTTP interaction is initiated to carry the web service response message. This interaction is also synchronous from the point of view of HTTP. When bound to HTTP, asynchronous web services take on their 'asynchronous' behavior by using two different HTTP request/response interactions to carry the web service request and response. Obviously more complicated patterns are possible. The tests documented in this section do not cover more complicated message exchange patterns. A single out message is responded to with a single in message.

Testing at Connectathons

All vendors testing the ASync profile at Connectathon are expected to package their request as a single HTTP connection and package their response as a single HTTP connection. If you wish to test more advanced patterns such as a single out followed by multiple ins, this testing must be done privately.

Example HTTP 202 message

This toolkit, either from the Public Registry or Synapse returns an immediate acknowledgment that looks like

HTTP/1.1 202 Accepted
Server: Apache-Coyote/1.1
Content-Type: text/xml;charset=UTF-8
Content-Length: 0
Date: Tue, 30 Dec 2008 14:22:51 GMT

HTTP proxies

It is common to view an ESB such as Synapse as an HTTP proxy from the point of view of the client. At the HTTP message level, this means there is an apparent inconsistency between where the header says the message is going and where the connection says it is going. When a proxy is not used, the message request header on an HTTP message is formatted as:

POST /axis2/services/xdsrepositoryb HTTP/1.1

Where

POST is the HTTP verb
/axis2/services/xdsrepositoryb is the service being addressed
HTTP/1.1 is the HTTP version used

The machine address and port number portion of the service address are not represented in the request header because they are not needed, the socket connection is made directly to that machine and port. When a proxy is used, the situation is slightly more complicated. When a proxy is used the request header on the HTTP message is formatted as:

POST http://ihexds.nist.gov:9080/axis2/services/xdsrepositoryb HTTP/1.1

Notice that the service address includes the hostname and port number. Your local connection (connection made by the client) is to the proxy hostname and port. The proxy must determine where to send the request. It expects the full service address in the request header.

When using Synapse as a local front end to the Public Registry, it is technically being used as a Reverse Proxy which means that it directs all incoming traffic to a predetermined location, the Public Registry server.

Personal tools