Monday, November 8, 2010

Griffon: Signing an Application

Last week, I tried using Griffon to help investigate the possibility of creating a Java Web Start application. Everything went smoothly with the exception of some trouble I had signing the application for the "production" environment. To help those in a similar situation, I'll summarize the steps I needed to get things singed below signed below.

Note:I used this thread on Markmail to and the Griffon Quick Start guide to help me out.

I created my application as the guide suggests with:

  $ griffon create-app DemoConsole

Now the guide offers some suggestions for making the resulting application more useful, but here I'm just going to do the bare minimum I know to get the application deployed.

Next, in order to sign the jars in the application for production use, we need to create a keystore:

  $ keytool -genkey -alias GriffonKey

Enter keystore password:
Re-enter new password:
What is your first and last name?
[Unknown]: John Prystash
...
Enter key password for <GriffonKey>
(RETURN if same as keystore password):

I used prystasj for both the keystore and key password, this being information we'll need to know later. The resulting keystore file was created at: /home/prystasj/.keystore

The signing and key information is determined from the file griffon-app/conf/BuildConfig.groovy. Below is the relevant file information in its original form:

// key signing information
environments {
//...
production {
signingkey {
params {
sigfile = 'GRIFFON'
keystore = 'CHANGE ME'
alias = 'CHANGE ME'
// NOTE: for production keys it is more secure to rely on key prompting
// no value means we will prompt //storepass = 'BadStorePassword'
// no value means we will prompt //keypass = 'BadKeyPassword'
lazy = false // sign, regardless of existing signatures
}
}

griffon {
jars {
sign = true
pack = true
destDir = "${basedir}/staging"
}
webstart {
codebase = 'CHANGE ME'
}
}
}
}

We'll need to point the production configuration at the keystore we created earlier and set the alias to GriffonKey. We'll also add the password we gave to the keytool program, which is prystasj in both cases. Finally, we'll be publishing the app to a webserver, whose location we set in the codebase property:

// key signing information
environments {
//...
production {
signingkey {
params {
sigfile = 'GRIFFON'
keystore = '/home/prystasj/.keystore'
alias = 'GriffonKey'
storepass = 'prystasj'
keypass = 'prystasj'
lazy = false // sign, regardless of existing signatures
}
}

griffon {
jars {
sign = true
pack = true
destDir = "${basedir}/staging"
}
webstart {
codebase = 'http://myhost.org/prystasj/democonsole/'
}
}
}
}

Now we can build the app with:

  $ griffon prod package webstart

The result is a zip file at dist/webstart. Taking this zip we can unpack on the webserver at the location we set in the codebase property.

Next, we can download the application and run it from: http://myhost.org/prystasj/democonsole/application.jnlp.

Hope this helps anyone else looking to get a Griffon application signed.

Wednesday, October 20, 2010

Groovy: Retrieving the Value of Multiple XML Elements

Yesterday, I ran into an interesting case at work where some code was parsing XML using Groovy's XmlSlurper to retrieve the value of an element and treat it as a String. Something along the likes of:

  def xml = "<xml><character>a</character></xml>"
def node = new XmlSlurper().parseText(xml)
String result = node.character
println result

Which simply prints out a. The code was expected to only return value of the first element found, but when a second element is added:

  def xml = "<xml><character>a</character><character>b</character></xml>"
def node = new XmlSlurper().parseText(xml)
String result = node.character
println result

The resulting output is ab, which broke a new test case.

One solution to the issue is to grab the first element with:

  String result = node.character[0]

Another interesting point is that the result of node.character[0] is a NodeChild, not a String. Since the type of the result variable is declared, the right side of the assignment was coerced to a String. If that were not the case, and we had:

  def result = node.character[0]
println result.getClass().getName()
println result

The output would be:

  groovy.util.slurpersupport.NodeChildren
a

Note:We need to use getClass() as result.class would return the class node and not the Class object itself.

Alternatively, we can use the text() method of NodeChildren (which also exists for NodeChild) to ensure we get a String:

  def result = node.character[0].text()
println result.getClass().getName()
println result

Giving us:

  java.lang.String
a

To summarize, putting all the methods discussed so far to use with:

  def xml = "<xml><character>a</character><character>b</character></xml>"
def node = new XmlSlurper().parseText(xml)

println node.character
println node.character.getClass().getName()
println()

println node.character.text()
println node.character.text().getClass().getName()
println()

println node.character[0]
println node.character[0].getClass().getName()
println()

println node.character[0].text()
println node.character[0].text().getClass().getName()
println()

We get:

  ab
groovy.util.slurpersupport.NodeChildren

ab
java.lang.String

a
groovy.util.slurpersupport.NodeChild

a
java.lang.String

While the original gotcha might not be all that hard to resolve, hopefully this gives a little insight to those who might explore things a little bit further.

Tuesday, October 19, 2010

Bash: Learning about Arrays

Last year, I wrote about a little Groovy script to help me validate XML documents against a schema. A ended up with a little script to call to help me from having to recall how to run it using Maven. The script takes a schema location and an example instance document as arguments:

  #!/bin/bash
mvn -o exec:java -Dexec.mainClass=Validator -Dexec.args="$1 $2"

Which I can call with:

  $ ./validate.sh order.xsd order.xml

We store example XML documents along with our internal schemas, and sometimes I find myself running the same script multiple times, once for each document in the project, so I thought having a script to loop through the documents in a directory and report the results would also be helpful.

In order to report both the 'good' instances and the 'bad' instances at the end of the scripts run, I needed to learn a little about bash arrays.

In found that in bash, there are multiple ways to create an array. You can start by simply assigning a value to a yet unused array:

  #!/bin/bash
good[0]="my.xml"
echo ${good[0]}
Or you can declare an array using:
  #!/bin/bash
declare -a good = ("my.xml", "your.xml")
echo ${good[0]}
echo ${good[1]}

In my case, I will be iterating through the files and adding them to the appropriate array, so I won't be able to declare the array or its size up front. I used this method to first check if the array is empty in an if statement, and if so declare and initialize it. If the array does in fact exist, I append an element to in the else clause, by using @ to get the length of the array:

  #!/bin/bash
if [ ${#bad[0]} -eq 0 ]; then
declare -a bad=("$i")
else
bad=("${bad[@]}", $i)
fi

To roll everything up, my new scripts takes the schema location and a directory as arguments. For every, directory listing in the directory, I run the Groovy code against it. If the validation failed, setting $? to 1, I add the file to the 'bad' list. Otherwise, it goes to the good list:

  #!/bin/bash
xsd=$1
dir=$2

for i in `ls $dir`
do
echo -n "$i..."
mvn -o exec:java -Dexec.mainClass=Validator -Dexec.args="$1 $2/$i"

if [ $? -eq 1 ]; then
if [ ${#bad[0]} -eq 0 ]; then
declare -a bad=("$i")
else
bad=("${bad[@]}", $i)
fi
else
if [ ${#good[0]} -eq 0 ]; then
declare -a good=("$i")
else
good=("${good[@]}", $i)
fi
fi
done

echo
echo "Good files: "
echo "${good[@]}"
echo
echo "Failed files: "
echo ${bad[@]}
echo

At the end of script, I simply echo both lists. An example of the output for the report:

  Good files: 
good1.xml good2.xml good3.xml

Failed files:
bad1.xml bad2.xml bad3.xml

I'm sure I could remove some of the duplication, but I since I have only two arrays and this is just a helper, I think I'll leave things be for me. If you have any more array advice, please leave a comment!

Friday, October 1, 2010

Trang: Creating Schemas from XML

Does anyone like writing XML schemas? Sometimes they can be frustrating, and yet always ends up feeling simple when you're done. When given the choice, it always feels good to me to start writing a schema from an example instance document, and of course there are plenty of tools to help.

While a lot of tools are available for a price, Trang however is free, and helps me with the writer's block I tend to get when I'm handed an XML document and asked to make a schema from scratch.

Trang is a Java app that can be downloaded in zip form. Luckily (for me at least), it was available in the Ubuntu package repositores:

$ sudo apt-get install trang

Now that we have Trang installed, let's generate a schema from a simple XML document, languages.xml:

<?xml version='1.0' encoding='UTF-8'?>
<languages>
<language>
<name>Groovy</name>
<platform>JVM</platform>
<appeared>2003</appeared>
</language>
<language>
<name>Scala</name>
<platform>JVM</platform>
<appeared>2003</appeared>
</language>
<language>
<name>Boo</name>
<platform>CLR</platform>
<appeared>2003</appeared>
</language>
</languages>

Now let's tell Trang, gratefully, to make us a schema:

$ trang languages.xml languages.xsd

And a schema is generated for us:

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified">
<xs:element name="languages">
<xs:complexType>
<xs:sequence>
<xs:element maxOccurs="unbounded" ref="language"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="language">
<xs:complexType>
<xs:sequence>
<xs:element ref="name"/>
<xs:element ref="platform"/>
<xs:element ref="appeared"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="name" type="xs:NCName"/>
<xs:element name="platform" type="xs:NCName"/>
<xs:element name="appeared" type="xs:integer"/>
</xs:schema>

This is a real good start, but I feel like I should make a couple changes. To make things a little easier to understand for consumers, I think I'll change the uses of NCName to string:

<xs:element name="name" type="xs:NCName"/>
<xs:element name="platform" type="xs:NCName"/>

And require at least one language element for the document to be valid:

<xs:element minOccurs="1" maxOccurs="unbounded" ref="language"/>

My edited schema becomes:

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified">
<xs:element name="languages">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="1" maxOccurs="unbounded" ref="language"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="language">
<xs:complexType>
<xs:sequence>
<xs:element ref="name"/>
<xs:element ref="platform"/>
<xs:element ref="appeared"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="name" type="xs:string"/>
<xs:element name="platform" type="xs:string"/>
<xs:element name="appeared" type="xs:integer"/>
</xs:schema>

Even with this simple example, Trang has saved me a lot of typing. Trang also has the options to also create RELAX NG and DTD documents if you need them.

Tuesday, September 21, 2010

Perl & Bash: Mass Substitutions from the Command Line

Today I found myself in a situation where I had to do the same global substitution for every file in a directory. I considered writing a script to handle it, but I wanted to try using a Perl one-liner to get the job done.

Given a directory containing a file with the following contents:

  This is a XML file
This is a XML file
I can substitute every occurence of XML with text using:
  $ perl -i -pe 's/XML/text/g' file1.xml
Afterwards, the file now reads:
  This is a text file
This is a text file
If I wanted to do the substitution for every file in the current directory, I can wrap the command in a bash for loop:
  $ for i in `ls`; do perl -i -pe 's/XML/text/g' $i; done

This isn't revolutionary by any means, but its harder for me to forget this technique if I post it here.

Saturday, August 14, 2010

Maven & soapUI: Testing a Service

In a previous post, I demonstrated the creation of the business logic for an adding web service, that when given a list of numbers, returns their sum. The goal for me was to create a simple service so I can try out automated testing of the service using soapUI from Maven.

In this post, I'll show how I can run the test suite for the service. In the next post, hopefully I'll demonstrate how we can expose the service in Tomcat using Mule.

The created a test suite, when run, produces the following results as demonstrated by the service's log:

INFO  [http-8080-1][2010-07-07 11:23:34,981] - Sum of: [2, 2] = 4
INFO [http-8080-1][2010-07-07 11:23:35,109] - Sum of: [1, 2, 3] = 6
INFO [http-8080-1][2010-07-07 11:23:35,133] - Sum of: [1, 2, 3, 4] = 10
INFO [http-8080-1][2010-07-07 11:23:35,148] - Sum of: [0, 1] = 1
INFO [http-8080-1][2010-07-07 11:23:35,167] - Sum of: [0, 0] = 0
INFO [http-8080-1][2010-07-07 11:23:35,221] - Sum of: [] = 0

In the adder service project, I exported the project containing a test suite to a new integration testing module, adder-it, as:

src/test/resources/adder-soapui-project.xml

The POM for the adder-it module contains the following:

<?xml version="1.0" encoding="utf-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.prystasj.adder</groupId>
<artifactId>adder</artifactId>
<version>1.0-SNAPSHOT</version>
</parent>
<artifactId>adder-it</artifactId>
<name>adder Integration Testing</name>
<description>Integration testing with soapUI.</description>
<build>
<plugins>
<plugin>
<groupId>eviware</groupId>
<artifactId>maven-soapui-plugin</artifactId>
<version>3.5.1</version>
<configuration>
<projectFile>src/test/resources/adder-soapui-project.xml</projectFile>
<host>localhost</host>
<port>8080</port>
</configuration>
<executions>
<execution>
<id>soap-integration-test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>

To run the tests, with the service up and running locally on port 8080, invoke Maven with:

$ mvn integration-test

Below is some of the build output statements from a test run:

[INFO] [soapui:test {execution: soap-integration-test}]
soapUI 3.5.1 Maven2 TestCase Runner
19:31:31,085 INFO [WsdlProject] Loaded project from [file:/home/prystasj/workspace/prystasj/mule/adder/adder-it/src/test/resources/adder-soapui-project.xml]
19:31:31,713 INFO [SoapUITestCaseRunner] Running soapUI tests in project [adder]
...
19:31:31,734 INFO [SoapUITestCaseRunner] Running soapUI testcase [Add_2_to_2]
19:31:31,750 INFO [SoapUITestCaseRunner] running step [Add_2_to_2]
19:31:33,305 INFO [SoapUITestCaseRunner] Assertion [SOAP Response] has status VALID
19:31:33,306 INFO [SoapUITestCaseRunner] Assertion [Contains] has status VALID
19:31:33,306 INFO [SoapUITestCaseRunner] Finished running soapUI testcase [Add_2_to_2], time taken: 1543ms, status: FINISHED
...
19:31:33,307 INFO [SoapUITestCaseRunner] Running soapUI testcase [Add_1_to_2_to_3]
...
19:31:33,385 INFO [SoapUITestCaseRunner] Project [adder] finished with status [FINISHED] in 1664ms

To access the soapUI Maven plugin, add the following plugin repository to your settings.xml:

<pluginRepository>
<id>eviwarePluginRepository</id>
<url>http://www.eviware.com/repository/maven2/</url>
</pluginRepository>

Tuesday, July 6, 2010

Groovy & Spock: Adding Numbers Together

Lately, I've been playing around with embedding Mule in Tomcat. To get going, I wanted to start with creating an extermely simple service, and along the way I decided to try out the Spock specification framework to test the logic I'd be using in my service. I liked what I found so much, I thought I'd take a little detour and write about my first use of the framework, comparing it to more a common test class, and save the Tomcat posting for another day.

The simple service that will eventually be developed will take a list of numbers and add them up. Here is the class that will ultimately do the math:

class Adder {
Integer add(List<Integer> numbers) {
println "adding: $numbers"
numbers.inject(0) { sum, item -> sum + item }
}
}

The add method takes a list of numbers and returns their sum. I'm using Integer as the return type and List on the parameter declaration instead of the def keyword to aid in the generation of the service WSDL down the line. While the println won't make into the final version of the class, it might help us out demonstrating what is being sent to the method during testing.

With Spock we'll be writing a specification that describes the behavior of the Adder class:

  • Given a list of numbers, they should be added together to produce a sum.
  • Given a list containg no numbers, zero should be returned.

The framework's documentation does a great job explaining how to write a spec, so I will not to try and reproduce much of it here. The spec is pretty much self-explanatory, each of the two requirements manifests itself in a method:

import spock.lang.*

class AdderSpec extends Specification {

def adder = new Adder()

def "numbers should be added together to produce a sum"() {
expect:
sum == adder.add(numbers)
where:
sum | numbers
4 | [2, 2]
6 | [1, 2, 3]
10 | [1, 2, 3, 4]
1 | [0, 1]
0 | [0, 0]
}

def "summing no numbers at all should return zero"() {
expect:
sum == adder.add(numbers)
where:
sum | numbers
0 | []
0 | null
}
}

The where clauses resemble a table, with each entry describing the input to the add() method on the right, and the expected result on the left. The first entry in the first method would have the values for sum and numbers substituted into the expect clause so that it would read:

4 == adder.add([2, 2])

The output from the test run helps illustrate how things are added during a run of the spec:

Running org.prystasj.services.adder.AdderSpec
adding: [2, 2]
adding: [1, 2, 3]
adding: [0, 1]
adding: []
adding: null
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.091 sec

If the value for numbers instead read [2, 3], we would get the following failure when we run the test as 2+3 does not equal 4:

Condition not satisfied:

sum == adder.add(numbers)
| | | | |
4 | | 5 [2, 3]
| org.prystasj.services.adder.Adder@969c29
false

While I could of used a helper method to remove the redunduncy between the two expect clauses, I liked this output such much, I decided to keep it, as using a method here would modify the resulting failure description. For example:

check(sum, numbers)
| | |
false 4 [2, 3]

Compare the above spec class to a more traditional unit test and see what you think:

import org.junit.Before
import org.junit.Test
import static org.junit.Assert.assertEquals

class AdderTest {

def underTest

@Before
void setUp() {
underTest = new Adder()
}

def verifySumsFor(data) {
data.each { sum, numbers ->
assertEquals "$sum <= $numbers", sum, underTest.add(numbers)
}
}

@Test
void test_numbers_added_together_produce_a_sum() {
def data = [
4 : [2, 2],
6 : [1, 2, 3],
10 : [1, 2, 3, 4],
1 : [0, 1],
0 : [0, 0],
]
verifySumsFor(data)
}

@Test
void test_summing_no_numbers_at_all_should_return_zero() {
def data = [
0 : [],
0 : null,
]
verifySumsFor(data)
}
}

Which version do you like better? Thanks for reading.

Saturday, June 26, 2010

Mule: Transport of Larger Responses in CXF

Upon calling a remote web service via a CXF endpoint in Mule I was presented with the following error:
Caused by: javax.xml.bind.UnmarshalException
- with linked exception:
[com.ctc.wstx.exc.WstxIOException: Connection reset]
at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.handleStreamException(UnmarshallerImpl.java:426)
at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:362)
at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:339)
at org.apache.cxf.jaxb.JAXBEncoderDecoder.unmarshall(JAXBEncoderDecoder.java:537)
... 96 more

At first, I thought either my request or the response from the remote server was malformed XML since the exception class and stack trace seems to suggest a marshalling problem. On the other hand, the exception mentions a connection reset.

To help spare you kind readers the details of my investigation, it turns out the above problem only manifested itself when a large amount of data was being transferred between the services.

The Mule CXF Transport documentation lists an attribute, mtomEnabled, that can be enabled on the definition of the outbound endpoint for the remote service:

    <cxf:endpoint name="remoteService"
address="http://constanza.com/service/architectureService"
clientClass="com.costanza.ArchitectureService"
wsdlPort="ArchitectureServicePort"
wsdlLocation="http://constanza.com/service/architectureService?wsdl"
operation="getBlueprint"
mtomEnabled="true"/>

The attribute turns on the SOAP Message Transmission Optimization Mechanism which encodes the response payload for travel between the services. The Mule documentation refers to this allowing for data to be sent as an attachment. At the very least it tells CXF to be on the look out or to handle a potentially large response.

Tuesday, June 22, 2010

WSDLs and Message Parts not recognized

I recently ran into an interesting problem consuming a web service where a call to the service produced the following error message:

Message part {http://myservice.com/}getSets was not recognized.  (Does it exist in service WSDL?)

I originally tested the service using soapUI with no problem, but when I tried to invoke the service from elsewhere I was presented with the error.

My service WSDL was auto-generated by CXF. Here I found I had to tweak it a little to help it better conform to the doc/literal style. The message part declaration for the getSets operation originally looked something like this (nothing special):

<wsdl:message name="getSets">
<wsdl:part element="tns:getSets" name="parameters">
</wsdl:part>
</wsdl:message>

The definition of the referenced getSets element is where the problem was hiding. The element declaration contained a reference to a complexType instead of having the complexType defined within, or as part of, the element declaration.

The original element defintion:

<xs:element name="getSets" type="tns:getSets"/>
<xs:complexType name="getSets">
<xs:sequence>
<xs:element minOccurs="0" name="shipment" type="xs:string"/>
</xs:sequence>
</xs:complexType>

The new definition that allows calls from both soapUI and another client to work:

<xs:element name="getSets">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="shipment" type="xs:string" />
</xs:sequence>
</xs:complexType>
</xs:element>

Hopefully this can help someone out who runs into something similar.

Thursday, May 6, 2010

Mule: Defining Properties at Runtime

I've had the desire for sometime to investigate telling Mule at run (or start-up) time what set of properties to use for a particular environment as in development, QA, or production, instead of deciding at build or deploy time what the target environment will be. So below I've come up with a rather simplistic service that uses a system property to define which set of properties to use at startup.

We will use the following Mule configuration element to define the environment properties file (explained in more detail here).

  <context:property-placeholder location="classpath:default.properties,classpath:${env}.properties" />

With the above line included in our Mule configuration, the default.properties file will be loaded first, followed by the environment-specific properties file. The latter will have the opportunity to override anything defined by the former.

The service will report the current season. A greeter bean configured in the Mule context will have two properties to be set via injection:

  • greetee - who the greeting will be addressed to
  • season - the current season (a true property of the environment)

The greeting will take the form of:

  "Hello ${greetee}, it is ${season}"

A default properties file will be used with the hope that we can eliminate having to repeat the property definitions
whose values would be common for every environment. We'll also investigate the ability to override this property in
case a particular environment would need to do so.

The default properties file will hold the 'greetee'.

    # default.properties
greetee = Friend
We'll have two season-defining properties files, one for spring and one for winter. The winter properties file will simply define the season:
    # winter.properties
season = winter
The spring properties file will define the season as well as override the greetee property set in default.properties:
    # spring.properties
greetee = Chap
season = spring

Since I'm using Maven 2 to build my service, we'll place the properties file in the src/main/resources directory. By convention, Maven will place all files found in this directory in the JAR file produced by the build. To make the files available to Mule on the classpath, we can place the JAR in hte lib/user directory of our Mule deployment so that they are available on the classpath as requested by the property-placeholder element.

The SeasonReporter class will have the two properties injected and will be used as a service component in our Mule model:

class SeasonReporter {
def greetee
def season
def reportSeason() { "Hello $greetee, it is $season" }
}

Our Mule configuration will have an inbound HTTP endpoint that will reply synchronously with the environment-specific greeting. The component that is invoked will be implemented by a SeasonReporter class that takes the two properties used to create the seasonal greeting.

We'll direct Mule to use the reportSeason method with no arguments (by default the HTTP transport will provide a payload of one argument, /season here):

<mule>
<context:property-placeholder location="classpath:default.properties,classpath:${env}.properties" />

<model name="seasonModel">
<service name="seasonService">
<inbound>
<http:inbound-endpoint address="http://localhost:8080/season"/>
</inbound>
<component>
<no-arguments-entry-point-resolver>
<include-entry-point method="reportSeason"/>
</no-arguments-entry-point-resolver>
<spring-object bean="seasonReporter"/>
</component>
</service>
</model>

<spring:bean name="seasonReporter" class="SeasonReporter" scope="prototype">
<spring:property name="greetee" value="${greetee}"/>
<spring:property name="season" value="${season}"/>
</spring:bean>
</mule>

To start Mule using the winter properties file, we can pass in the env property when starting Mule on the command line with:

  $ mule/bin/mule start -M-Denv=winter

After starting Mule and hitting URL http://localhost:8080/season in a browser, we are presented with:

  Hello Friend, it is winter

Now we can try starting Mule up for season spring. The greeting should now include the overridden greetee property of 'Chap':

  $ mule/bin/mule start -M-Denv=spring
Giving us:
  Hello Chap, it is spring

Thanks for reading!

Tuesday, April 20, 2010

Mule & CXF: Multiple Operations and Groovy Components

When I picture a web service in Mule, there is usually a component backing the endpoint. Here I have a case where I would want the service fielding the web service request to delegate its work to another service. To the client, the fact that Mule will pass the actual workload for a request to another service will be hidden.

In this example, we have a mock "warehouse", where a client can store and retrieve a box. A box is uniquely identified by an ID (a String). Each operation will result in the invocation of an additional, but separate, service to do the actual work.

Here's the interface defined for our warehouse:

package prystasj.warehouse;

import javax.jws.WebParam;
import javax.jws.WebResult;
import javax.jws.WebService;

@WebService
public interface Warehouse {

@WebResult(name="receipt")
List<String> retrieve(@WebParam(name="boxId") String boxId);

@WebResult(name="box")
String store(@WebParam(name="boxId") String boxId);
}

The inteface defines both the operations. To determine which operation the client wishes to invoke, we can inspect the cxf_property of the Mule message created by the CXF Transport.

A request indicating the store operation will be routed to a storageService, while a retrieve operation will be routed to a retrievalService:

<model name="warehouse">
<service name="warehouseService">
<inbound>
<cxf:inbound-endpoint address="http://localhost:8080/warehouse"
serviceClass="prystasj.warehouse.Warehouse"/>
<inbound>
<outbound>
<filtering-router>
<vm:outbound-endpoint path="retrieval" synchronous="true"/>
<expression-filter evaluator="groovy"
expression="message.getProperty('cxf_operation').getLocalPart() == 'retrieve'"/>
<filtering-router>
<filtering-router>
<vm:outbound-endpoint path="storage" synchronous="true"/>
<expression-filter evaluator="groovy"
expression="message.getProperty('cxf_operation').getLocalPart() == 'store'"/>
<filtering-router>
<outbound>
<service>
</model>

To test the processing flow out, we can script a couple of components with Groovy. A couple of println's will help us pick out that the operations were invoked when viewing the Mule log. With a scripting component, we'll have access to a variable payload, which in each operation will contain the web parameters (a boxId in both cases) as defined in the Warehouse interface.

    <service name="retrievalService">
<inbound>
<vm:inbound-endpoint path="retrieval"/>
</inbound>
<script:component>
<script:script engine="groovy">
println "Operation: retrieve; Box ID: $payload"
new Box(payload)
<script:script>
<script:component>
<service>
<service name="storageService">
<inbound>
<vm:inbound-endpoint path="storage"/>
</inbound>
<script:component>
<script:script engine="groovy">
println "Operation: store; Box ID: $payload"
new Receipt(payload)
<script:script>
<script:component>
<service>

The store operation returns an instance of a Box and the retrieve operation returns a Receipt. As both services are invoked synchronously, the result of the component invocations will be returned to the calling Warehouse service, which will pass them through to the client.

Monday, March 29, 2010

XMLUnit: Simpler Asserts

Quite a while ago, I wrote about using XMLUnit in my unit tests to compare XML results.

The method used to create Diffs has been useful, but the project offers a simpler way to compare XML:

import static org.custommonkey.xmlunit.XMLAssert.assertXMLEqual

class AssertXmlEqualTest {

@Test void test_xml_is_similar() {
def expected = "<hi>bye</hi>"
def actual = "<hi>bye</hi>"
assertXMLEqual expected, actual
}
}
The test passes. If I change the value of actual to something different, say:

    def actual = "<hi>goodbye</hi>"

The test fails as the XML documents are no longer similar:

junit.framework.AssertionFailedError: org.custommonkey.xmlunit.Diff
[different] Expected text value 'bye' but was 'goodbye' -
comparing <hi ...>bye<hi> at /hi[1]/text()[1] to <hi ...>goodbye<hi> at /hi[1]/text()[1]
Since I have no reason in this simple case to be inspecting the Diff object created by XMLUnit as described in the previous posting, this method should serve as a nice substitute.

Sunday, March 7, 2010

Mule: Service Composition with Dependencies

Last time around, I did a little work with service composition in hopes to come up with an approach I could use in a real-world project. The approach I came up with last time appeared to fit the bill, until another requirement was added: The response from the first service would be needed to make the request to the second.

The new requirement definitely forced me to rethink my previous approach, as I could no longer simply send the intial request off to be routed to the two services that needed to be consumed, aggregating the responses for a request to the business logic.

For this example, I'll try to come up with a similar problem for a service to solve that follows a similar processing flow: The goal of this new service, given an order number, is to respond with a report about the customer containing their information and order history.

In order to accomplish this task, this new service will need to make use of two remote services:

  1. A customer service: Returns the information for a customer who placed a particular order.
  2. An order history service: Returns the order history for a particular customer.

The information contained in both service responses is used in the business logic contained within this composite service to produce a report, let's say to relate the ordering of certain products to different demographic profiles.

Now at this point, one might say: "Why can't the order history service return the customer information its provided in its request in its response?".

A good question, but some would argue that a service should not respond with data the caller already has or knows. My job here would be a lot of easier if the order history service could be changed to do just that, I could just pass the order history response to my business logic and not need to keep the customer service response around. On the other hand, the order history service should just be charged with returning the information it was requested to produce and not concern itself with how its response will ultimately be used, as long as that response is usable and correct. Not all clients might make use of the customer information returned in the order history response, and we should not request that type of overhead.

Ok, enough talking and more Mule-ing. For now I can start with the two things I know for certain, I will need to make requests to both the customer service and order history service, and they need to happen in order, so let's start there. For the sake of brevity, I'll omit any needed transformations to make the processing paths easier to follow.

<service name="orderService">
<inbound>
<jms:inbound-endpoint queue="order.queue" synchronous="true"/>
</inbound>
<outbound>
<chaining-router>
<vm:outbound-endpoint path="customer.queue" synchronous="true"/>
<vm:outbound-endpoint path="history.queue" synchronous="true"/>
</chaining-router>
</outbound>
</service>

The incoming request will be routed to a VM queue where a listening internal service will route it to the remote customer service:

<service name="customerService">
<inbound>
<vm:inbound-endpoint path="customer.queue"/>
</inbound>
<outbound>
<pass-through-router>
<jms:outbound-endpoint queue="customer.service.request.queue" synchronous="true"/>
</pass-through-router>
</outbound>
</service>

The response from the customer service will be provided to the order history service, which is defined similarly.

Now the next problem here is getting both responses to the same place so a request for my business logic can be created and the request carried out. From here, I'll add the internal service containing the business logic:

<service name="reportService">
<inbound>
<vm:inbound-endpoint path="responses.queue" synchronous="true"/>
<custom-inbound-router class="ResponseAggregator"/>
</inbound>
<component>
<spring-object bean="reportingComponent"/>
</component>
</service>

The inbound router is an extension of Mule's AbstractEventAggregator class that waits for a group of 2 responses (to avoid a context switch I'll omit the implementation details for now). It's job is to take both service responses and create a request for the reportingComponent. Now that we have that defined, how do we get the service responses to its VM queue?

Getting the order history response to the queue is easy, we just add another outbound endpoint to the chaining router:

<service name="orderService">
<inbound>
<jms:inbound-endpoint queue="order.queue" synchronous="true"/>
</inbound>
<outbound>
<chaining-router>
<vm:outbound-endpoint path="customer.queue" synchronous="true"/>
<vm:outbound-endpoint path="history.queue" synchronous="true"/>
<vm:outbound-endpoint path="responses.queue" synchronous="true"/>
</chaining-router>
</outbound>
</service>

Now we need to get the customer service response to the order history service and to the same response.queue. A "customer response routing service" might work here. This new service would be added as an endpoint in the chaining router between the routings to the customer and history services. This new internal service needs to route its request to the VM queue for the report service and return the request to the chaining router:

<service name="customerRoutingService">
<inbound>
<vm:inbound-endpoint path="customer.routing.queue" synchronous="false"/>
</inbound>
<outbound>
<multicasting-router>
<vm:outbound-endpoint path="responses.queue" synchronous="false"/>
<vm:outbound-endpoint path="async.queue" synchronous="false"/>
</multicasting-router>
</outbound>
<async-reply>
<vm:inbound-endpoint path="async.queue"/>
<single-async-reply-router>
<!-- we want to return the message that maps to the correlation ID found on the original request -->
<expression-message-info-mapping messageIdExpression="#[header:JMSCorrelationID]"
correlationIdExpression="#[header:JMSCorrelationID]"/>
</single-async-reply-router>
</async-reply>
</service>

Unlike the other internal services that respond synchronously, this service is asynchronous. To the caller, the chaining router, the request will have been handled synchronously, following the Async Request Response style.

The request received (the customer information), will be sent to the desired VM queue and to the async.queue where a reply router will return the customer information response it recevied on its inbound endpoint to the chaining router.

Now we place the routing to the customerRoutingService into the chain:

<service name="orderService">
<inbound>
<jms:inbound-endpoint queue="order.queue" synchronous="true"/>
</inbound>
<outbound>
<chaining-router>
<vm:outbound-endpoint path="customer.queue" synchronous="true"/>
<vm:outbound-endpoint path="customer.routing.queue" synchronous="true"/>
<vm:outbound-endpoint path="history.queue" synchronous="true"/>
<vm:outbound-endpoint path="responses.queue" synchronous="true"/>
</chaining-router>
</outbound>
</service>

Since the last endpoint in the chain will be the second message placed on responses.queue, the inbound router for the reportService will be invoked, in turn invoking the component. The response from the component will be returned to the chain, completing it. The chaining router will then return the response to the caller of the orderService.

To sum up the processing flow:

  1. A request is sent to the order service.
  2. The request is forwarded to the remote customer service.
  3. The customer service response is sent to the customer response routing service.
  4. The routing service sends the request to the responses queue (event #1) and to an async reply queue.
  5. A listener on the async reply queue returns the customer service response to the chaining router
  6. The customer service response is forwarded to the remote order history service.
  7. The order history response is sent to the responses queue (event #2).
  8. The aggreagation router listening to the responses queue, having both events, is invoked, creating the reporting request.
  9. The reporting component is invoked creating the service response.
  10. The response is returned to the chaining router, completing the chain.
  11. The response is returned to the caller.

This could one of several approaches to the stated problem, alternatives are welcome!