Tuesday, December 22, 2009

SFTP Message: Received message too long

Using sftp to access a remote server I had access to, I was presented with the below error message:
  prystasj@prystasj:~/workspace$ sftp somehost
Received message too long 1231953966
prystasj@prystasj:~/workspace$
However, I was able to ssh just fine:
  prystasj@prystasj:~/workspace$ ssh somehost
In .bashrc
In .bash_profile
prystasj@somehost:~$
The key was that my login scripts, .bashrc and .bash_profile produced output as shown above, which was intentional as I was trying to once demonstrate the difference between a login and non-login shell to myself to see which files were sourced. After removing the output-producing commands from the files, I was able to sftp like normal.

Sunday, December 6, 2009

Answering Yes to Prompts Automatically

The yes command can be used to pipe a series of y's to another command, so that the user does not have to wait to be prompted during the command's execution.

For example, after a Linux install, I went to use CPAN for the first time to install the File::Find::Rule module.

  prystasj@prystasj:~$ cpan -i "File::Find::Rule"

CPAN is the world-wide archive of perl resources.
...
Would you like me to configure as much as possible automatically? [yes]
While answering yes above will configure CPAN automaticaly, I'll still be prompted to answer questions regarding the module's dependencies:
  Writing Makefile for File::Find::Rule
Falling back to other methods to determine prerequisites
---- Unsatisfied dependencies detected during ----
---- RCLAMP/File-Find-Rule-0.32.tar.gz ----
Number::Compare [requires]
Text::Glob [requires]
Shall I follow them and prepend them to the queue of modules
we are processing right now? [yes]
To automatically answer yes to every question, you can pipe the output of the yes command to the prompting command. For example:
  prystasj@prystasj:~$ yes | cpan -i "File::Find::Rule"
Now I can install the module without watching the command's progress.

Tuesday, December 1, 2009

Groovy: Retrieving XML Element Attributes in a Map

The Groovy script below is a little test I came up to help me remember how to grab element attributes from an XML document. In the example message, I want the attributes from the request element in a Map:
  def message = """<message>
<request file="/home/prystasj/file.txt" type="txt" />
<message>"""

def xml = new XmlParser().parseText(message)
def attrs = xml.request[0].attributes()

println "Class: ${attrs.getClass().getName()}"
println "Size: ${attrs.size()}"
println "Attributes: $attrs"
println "File: ${attrs.file}"
println "Type: ${attrs.type}"

The key that tripped me up was that I needed to grab the first request element (indexed at 0) in the assignment to attrs.

The output from the script:

  Class: java.util.LinkedHashMap
Size: 2
Attributes: [file:/home/prystasj/file.txt, type:txt]
File: /home/prystasj/file.txt
Type: txt

Sunday, November 29, 2009

Groovy: Switching on Enumerations

Recently, in Groovy, I had a need to switch on a variable that contained an enumerated value and was wondering what I should use in my case clauses. For example given the following:

enum OS { LINUX, MAC, WINDOWS }
Should (or could) I switch on OS.LINUX or "LINUX", or even "${OS.LINUX}"? I wrote a little script to help me decide:

enum OS { LINUX, MAC, WINDOWS }

[OS.LINUX, "LINUX", OS.MAC, "MAC", OS.WINDOWS, "WINDOWS"].each { os ->
switch(os) {
case OS.LINUX:
println "Enum: OS.LINUX"
break
case "${OS.LINUX}":
println "String: LINUX"
break
case OS.MAC:
println "Enum: OS.MAC"
break
case "WINDOWS":
println "String: WINDOWS"
break
default:
println "Unknown OS: '$os'"
}
}

The output from the above:

  LINUX => Enum: OS.LINUX
LINUX => String: LINUX
MAC => Enum: OS.MAC
MAC => Unknown OS: 'MAC'
WINDOWS => String: WINDOWS
WINDOWS => String: WINDOWS

Looking at the results, I'm not sure I really have an answer. Using the "WINDOWS" case as an example, the first two cases could be replaced with "LINUX". But then, am I really switching on an enumerated value or just taking advantage of the fact that enumerations are easily converted to strings?

At the very least, this shows there are options depending on how strict I might want to be with the typing of the variable to be switched on.

Saturday, November 14, 2009

Parameterized Testing with JUnit 4

I have seen several examples in the past of using parameterization with JUnit 4, but always felt like I didn't completely grasp what was going on. So if I don't understand something, I should try it out right?

My example class to test with parameterization will be a nice car with one state-changing method that will add a number of miles to the total traveled, whose value will be stored in a odometer property:

    package prystasj.junit

class Lambo {

int odometer

def drive(miles) {
if (miles > 0) odometer += miles
}

@Override String toString() {
"${getClass().getName()}[odometer=$odometer]"
}
}

To test the odometer, I will "drive" distances of 5, 10, then 20 miles, checking the odometer along the way. At the end, my low-use Lambo should have traveled a total of 35 miles. To do so, I will use the same Lambo instance throughout the tests.

The @BeforeClass annotation will run the annotated static method once per test run, so I will initialize my Lambo, which also must be static, there:

    static Lambo lambo

@BeforeClass static void createLambo() {
lambo = new Lambo()
println "Created Lambo: $lambo"
}

For every run of my test method, I want to receive a number miles to drive and the expected value of the odometer afterwards. Therefore, I need a way to set these values in my test class via a constructor:

    def total
def miles

LamboMileageTest(expectedTotal, milesToDrive) {
this.total = expectedTotal
this.miles = milesToDrive
}

The test parameters come from a static data method that returns a collection of arrays. Each entry in the collection will have its values passed as parameters to my constructor above:

    @Parameters static def data() {
def data = []
def expectedTotal = 0
[5, 10, 20].each { miles ->
expectedTotal += miles
def group = [miles, expectedTotal] as Integer[]
data << group
}
data
}

Without the conversion via as Integer[], the following exception was thrown:

  Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.038 sec <<< FAILURE!
initializationError0(prystasj.junit.LamboMileageTest) Time elapsed: 0.007 sec <<< ERROR!
java.lang.Exception: prystasj.junit.LamboMileageTest.data() must return a Collection of arrays.
Additionally, if I tried to append to data in one statement without the intermediate group variable, a GroovyCastException was thrown. I could wrap the right side of the append with parentheses, but I think this way is easier to read.

Each parameter grouping will be passed to each test method in my test class. Here I will have only one test method, but if I had two, each would be run with [5, 5], and then each would be run with [10, 15], and so on.

The test method:

    @Test void driveSomeDistance() {
println "Parmeters: [miles=$miles, total=$total]"
lambo.drive(miles)
assertEquals "Odomoter reads '$total'", total, lambo.odometer
}

The output from a successful test run:

  Running prystasj.junit.LamboMileageTest
Created Lambo: prystasj.junit.Lambo[odometer=0]
Parmeters: [miles=5, total=5]
Parmeters: [miles=10, total=15]
Parmeters: [miles=20, total=35]
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.173 sec

The entire test class:

    package prystasj.junit

import org.junit.BeforeClass
import org.junit.Test
import org.junit.runner.RunWith
import org.junit.runners.Parameterized
import org.junit.runners.Parameterized.Parameters
import static org.junit.Assert.assertEquals

@RunWith(value=Parameterized)
public class LamboMileageTest {

static Lambo lambo
def total
def miles

LamboMileageTest(milesToDrive, expectedTotal) {
this.miles = milesToDrive
this.total = expectedTotal
}

@BeforeClass static void createLambo() {
lambo = new Lambo()
println "Created Lambo: $lambo"
}

@Parameters static def data() {
def data = []
def expectedTotal = 0
[5, 10, 20].each { miles ->
expectedTotal += miles
def group = [miles, expectedTotal] as Integer[]
data << group
}
data
}

@Test void driveSomeDistance() {
println "Parmeters: [miles=$miles, total=$total]"
lambo.drive(miles)
assertEquals "Odomoter reads '$total'", total, lambo.odometer
}

}

Looking at the nature of a parameterized test class, and at the goal of my tests, I would recommend that this test class only be used for testing the accumulation of miles. If I wanted to test other Lambo functionality or usage, such as a Lambo not being able to drive negative miles, I would create another test class.

Without making use of parameterization, I would of wrote a test class like this:

    public class AlternateLamboMileageTest {

Lambo lambo

@Before void setUp() {
lambo = new Lambo()
}

@Test void starting_with_zero_miles_drive_5() {
lambo.odometer = 0
lambo.drive(5)
assertEquals 5, lambo.odometer
}

@Test void starting_with_five_miles_drive_10() {
lambo.odometer = 5
lambo.drive(10)
assertEquals 15, lambo.odometer
}

@Test void starting_with_fifteen_miles_drive_20() {
lambo.odometer = 15
lambo.drive(20)
assertEquals 35, lambo.odometer
}
}

This test class might be easier to follow though, or at least would of been before I started writing. With this approach however, I need to set the odometer before testing the result of a drive as I can't guarantee the order in which the tests are run. If I could, I could also use a static Lambo instance as before and avoid the odometer settings, or JExample to define test dependencies (which I wrote a bit about here).

This new test class also helps expose a design issue with my Lambo modeling, as unless I was some sort of shady used car salesman, I shouldn't be able to set the odometer once its initialized.

Then again, maybe if I was a shady salesman, I could find my way into a real Lambo... :)

Friday, November 13, 2009

Subversion: Changing Log/Commit Message

I'm posting this one because I only find myself looking it up whenever I need it, but to change the commit message for a particular revision for a Subversion repository, run:
  $ svn propset --revprop svn:log --revision 100 \
"New message, last was wrong." http://svn.mycompany.org/MyProject
This requires that the pre-revprop-change hook script is enabled. More on how to do that can be found here: Implementing Repository Hooks

Bash: Default Variable Values from Command Line Arguments

I wanted to have a bash script that takes one command line argument, but would default to a given value if the argument was ommitted. Here's a simple test script that demonstrates how I can do it:
  #!/bin/bash
VAR=default
if [ -n "$1" ]; then
VAR=$1
fi
echo $VAR
The -n operator tests that a string is not null and is quoted which is considered a good practice.

If I run the script, with no arguments, default is output to the screen, otherwise the argument is displayed.
  prystasj:~$ ./test.sh
default
prystasj:~$ ./test.sh hello
hello

Saturday, October 24, 2009

JExample: Defining Dependencies Between Unit Tests

I can't remember how I found it, but I've been meaning to investigate JExample for some time. To quote the project page, "JExample is a small framework to write unit tests that depend on each other."

I've read a lot about whether or not this is a good idea, and a lot of people say your unit tests should be able to run independent of one another. If one test fails, preventing the others to run, you might not catch all the failures in a complete run of your test suite. One test will fail, you'll fix it, and then you will be presented with a failure in another test.

After reading the project page, I can see where the project authors are coming from. They use a Stack as an example. Why you should run a test that adds three items to a Stack, when adding one item fails, or adding an item to an empty stack would fail? Maybe there's something to having dependencies between your tests. I'm not going to recommend either approach, for fear of starting another debate, but I would saying having the dependent approach in your toolbox could be useful depending on your particular testing situation.

The jar can be downloaded here: JExample Latest. Since I'm a Maven fan, I'll include how I added the jar to my Maven repository at the end of this post.

I do not want to reproduce the example found on the project page here, so I'll add a twist by writing my sample project in Groovy to see if it plays nicely (as I type, I haven't started).

My test project will be very similar though, a Rolodex class that keeps a list of Contacts

The Rolodex and Contact classes:

  class Contact {
def name
def phone
def email
}

class Rolodex {
def contacts = []
def add(contact) { contacts << contact }
def remove(contact) { contacts >> contact }
def size() { contacts.size() }
}
The start of my test class without anything from JExample:
  import org.apache.commons.logging.Log
import org.apache.commons.logging.LogFactory

class RolodexTest extends GroovyTestCase {

Log logger = LogFactory.getLog(getClass())
def rolodex

void setUp() { rolodex = new Rolodex() }

void test_initialized_rolodex() {
logger.info "Test initial Rolodex."
assertEquals "rolodex empty", 0, rolodex.size()
}

void test_adding_first_contact() {
logger.info "Adding Cosmo."
def contact = new Contact(name:"Cosmo", phone:"x6469", email:"cosmo@mail.next")
rolodex.add contact
assertEquals "contact added", 1, rolodex.size()
}

void test_removing_first_contact() {
logger.info "Removing Cosmo."
def contact = new Contact(name:"Cosmo", phone:"x6469", email:"cosmo@mail.next")
rolodex.add contact
assertEquals "contact added", 1, rolodex.size()
rolodex.remove contact
assertEquals "rolodex now empty", 0, rolodex.size()
}
}

In every test, I have to start with a newly constructed Rolodex. The first test may seem trivial. Do I really need to test a List starts empty? I say why not, but at the very least it will help demonstrate adding dependencies between the tests. The key here is the third test, which essentially duplicates the second by requiring that I add a Contact in order to remove it.

The dependency chain flows naturally out of the tests in the order they are written. I can't create a Rolodex with one contact unless I start with an empty one, and I can't remove a Contact unless I have one to remove.

Ok, I'm ready to try some JExample. After a little trial and error, it turns out I need JUnit 4 so I can use the @RunWith annotation. Since I'm using JUnit 4, I no longer need to extend GroovyTestCase. At this point, it seems I'm forced to use the @Test annotation as well since I'm using the @RunWith annotation.

I also added the @Before annotation to my setUp() method. I won't reproduce the code here to save space, but with or without the @Before annotation, the setUp() method isn't being called. To keep going, I'll add a call to each test method for now, as I think after I define my dependencies the need for the method will go away anyway.

The test class after these changes looks like:

  import org.apache.commons.logging.Log
import org.apache.commons.logging.LogFactory
import org.junit.Test
import org.junit.runner.RunWith
import static org.junit.Assert.assertEquals
import ch.unibe.jexample.JExample
import ch.unibe.jexample.Given

@RunWith(JExample)
class RolodexTest extends GroovyTestCase {

Log logger = LogFactory.getLog(getClass())

def rolodex

void setUp() { rolodex = new Rolodex() }

@Test void test_initialized_rolodex() {
logger.info "Test initial Rolodex."
setUp()
assertEquals "rolodex empty", 0, rolodex.size()
}

@Test void test_adding_first_contact() {
logger.info "Adding Cosmo."
setUp()
def contact = new Contact(name:"Cosmo", phone:"x6469", email:"cosmo@mail.next")
rolodex.add contact
assertEquals "contact added", 1, rolodex.size()
}

@Test void test_removing_first_contact() {
logger.info "Removing Cosmo."
setUp()
def contact = new Contact(name:"Cosmo", phone:"x6469", email:"cosmo@mail.next")
rolodex.add contact
assertEquals "contact added", 1, rolodex.size()
rolodex.remove contact
assertEquals "rolodex now empty", 0, rolodex.size()
}
}

And the tests pass. Note that the third test for removing a Contact is run second. As JUnit does not guarantee tests are run in the order they are defined, this should not come as a surprise:

  Running prystasj.jexample.RolodexTest
INFO: Test initial Rolodex.
INFO: Removing Cosmo.
INFO: Adding Cosmo.
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.981 sec

Now to try and reach for my goal here by adding dependencies between my tests. The @Given annotation is used to define dependencies. To make things more readable within the annotations, I'm also going to remove the test prefix at the start of my methods.

Skipping a bit ahead in my discovery, I was hoping the definition of the dependency chain would allow me to remove the addition of the Contact in the third test. Before I get to that, here's the new test class:

  @RunWith(JExample)
class RolodexTest {

Log logger = LogFactory.getLog(getClass())

def rolodex
def contact = new Contact(name:"Cosmo", phone:"x6469", email:"cosmo@mail.next")

void setUpEmptyRolodex() { rolodex = new Rolodex() }

@Test
void initialized_rolodex() {
logger.info "Test initial Rolodex."
setUpEmptyRolodex()
def rolodex = new Rolodex()
assertEquals "rolodex empty", 0, rolodex.size()
}

@Given("initialized_rolodex")
void add_first_contact () {
logger.info "Adding Cosmo."
rolodex.add contact
assertEquals "contact added", 1, rolodex.size()
}

@Given("add_first_contact")
void remove_first_contact() {
logger.info "Removing Cosmo."
logger.info "Rolodex size: ${rolodex.size()}"
rolodex.remove contact
assertEquals "rolodex now empty", 0, rolodex.size()
}
}

My tests pass without the addition of good ol' Cosmo in remove_first_contact. It does appear however that the dependencies are executed again for every test which can be seen in the output below. I added an additional logging statement to help verify the Rolodex has a size of 1 at the start of remove_first_contact:

  Running prystasj.jexample.RolodexTest
INFO: Test initial Rolodex.
INFO: Test initial Rolodex.
INFO: Adding Cosmo.
INFO: Test initial Rolodex.
INFO: Adding Cosmo.
INFO: Removing Cosmo.
INFO: Rolodex size: 1
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.016 sec

Method initialize_rolodex is run three times. While this may be considered a turn off, but if you're tests are fast as they should be, it shouldn't pose a problem. This might be working to my benefit here as I'm storing my Rolodex as a property of my test class. Before I get to that, let's fail the addition test and verify the removal test is not run by changing Rolodex.add(contact) to do nothing:

    def add(contact) { // do nothing }

After add_first_contact fails, the dependant, remove_first_contact, is skipped:

  Running prystasj.jexample.RolodexTest
INFO: Test initial Rolodex.
INFO: Test initial Rolodex.
INFO: Adding Cosmo.
Tests run: 3, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 1.014 sec <<< FAILURE!

Results :

Tests in error:
add_first_contact(prystasj.jexample.RolodexTest)

Tests run: 3, Failures: 0, Errors: 1, Skipped: 1
It looks I accomplished what I was after. There's no need to test removal, if addition doesn't work.

Back to storing my Rolodex as a property. I originally decided that was necessary so I could ensure the same Rolodex was being used in each test, but I have not made use of one of the features of JExample: The result of a test method can be injected into a method dependent on it as a parameter.

Making use of this, I can remove the rolodex property, but then I also have to make the tests in which others depend on return the altered Rolodex:

  @RunWith(JExample)
class RolodexTest {

Log logger = LogFactory.getLog(getClass())

def contact = new Contact(name:"Cosmo", phone:"x6469", email:"cosmo@mail.next")

@Test
Rolodex initialized_rolodex() {
logger.info "Test initial Rolodex."
def rolodex = new Rolodex()
assertEquals "rolodex empty", 0, rolodex.size()
rolodex
}

@Given("initialized_rolodex")
Rolodex add_first_contact(Rolodex rolodex) {
logger.info "Adding Cosmo."
rolodex.add contact
assertEquals "contact added", 1, rolodex.size()
rolodex
}

@Given("add_first_contact")
def remove_first_contact(Rolodex rolodex) {
logger.info "Removing Cosmo."
logger.info "Rolodex size: ${rolodex.size()}"
rolodex.remove contact
assertEquals "rolodex now empty", 0, rolodex.size()
}
}

There are a couple things to note here, although I'm using Groovy here, I have to explicitly add the return type of Rolodex to my test methods to get things to play nicely with JExample. Methods initialize_rolodex and add_first_contact return the Rolodex for use in the next test, but I've omitted the optional return keyword.


Maven & JExample

I was unable to find JExample in remote Maven repository, so I needed to add it manually to my repository. This makes sense as the project appears to come from a research group and there doesn't appear to be an official release of sorts (at the time of this writing the latest jar is titled r378). I'll use r378 as my version number for my Maven dependency instead of a more conventional release or snapshot version number.

The coordinates I'll use:

  <dependency>
<groupId>jexample</groupId>
<artifactId>jexample</artifactId>
<version>r378</version>
</dependency>

I considered using a more descriptive groupId that reflected the Software Composition Group, but I decided to keep things simple.

Install the jar to your local repository with:

  $ mvn install:install-file \
-DgroupId=jexample \
-DartifactId=jexample \
-Dversion=r378 \
-Dpackaging=jar \
-Dfile=jexample-r378.jar \
-DgeneratePom=true

You then should be able to include the jar as a dependency in your project using the dependency coordinates found above.


Enlightenment: Terminal Transparency Workaround

With my recent e17 install, I was unable to set a transparent background in any of terminal programs (xfce terminal, gnome-terminal, mrxvt, etc.).

Well, not exactly, I was able to set a transparent background, but the effect didn't take.

I was able to correct it using Esetroot which is part of eterm-utils:

  $ Esetroot ~/Pictures/02043_conch_1280x960.jpg

This is more of a pseudo-transparency I believe. The picture you choose can be different than that used for your wallpaper, which I think is pretty cool.

Enlightenment e17 Install on Ubuntu 9.04

My desktop at home is not getting any younger, so I've been experimenting with some more lightweight desktops and window managers. I've tried (and recommend) e16, fvwm, and fvwm-crystal. My current Linux distrubtion is Xubuntu 9.04 and I also like the xfce desktop it provides. Maybe I just like everything.

After reading an article on Enlightenment in Linux Journal, I decided to give e17 a go.

First add the Enlightenment Repository to your sources list by opening your repository sources file with:

  $ sudo vi /etc/apt/sources.list
Add the following line:
  deb http://packages.enlightenment.org/ubuntu jaunty main extras

After adding the repository to your sources list, grab and add the repository key:

  $ wget http://packages.enlightenment.org/repo.key
$ sudo apt-key add repo.key
$ sudo apt-get update
And then install e17:
  $ sudo apt-get install e17 emodule-places \
emodule-tclock emodule-calendar

Once everything's installed, log out and at the log in screen click Session and select Enlightenment. Log back in and you'll be presented with a few setup pages (I selected the Standard setup and added Firefox and some more friends to application list). After setup, you should be good to go.

Tuesday, October 20, 2009

State vs. Method Parameters Lesson: IllegalStateExcpetions in a Mule Transformer

Proper use of state versus passing values around as parameters seems like a topic one should have down pretty well by some point in their programming careers, and therefore would not be the focus of too much attention as time goes on. I, however, have run into a situation recently where not paying enough attention to it has caused me some problems, which in hindsight make me feel like I was asking for them. To summarize, storing a variable as a property in class when I had no real reason to, even when it felt like it could be justified, proved to be troublesome.

The context of the problem involved my implementation of a Mule transformer, in which I had to implement a doTransform() method to transform a response from my service if the incoming service request had a specific property. The MuleMessage containing the property I'm interested in can be retrieved statically via the RequestContext and can be thought of to live through the lifetime of the request (i.e. until a response is sent).

Here's the code conceptually:
import org.mule.RequestContext
import org.mule.transformer.AbstractTransformer

class ResponseTransformer extends AbstractTransformer {

def muleMessage

@Override
protected Object doTransform(Object response, String encoding) {
try {
muleMessage = retrieveMuleMessage()
if (isSpecialResponseExpected()) {
def attributes = retrieveAttributesFromMessage()
response = makeMessageSpecial(response, attributes)
}
} catch(e) {
throw new TransformerException(this, e)
}
return response
}

def retrieveMuleMessage() { RequestContext.event.message }
boolean isSpecialResponseExpected() { muleMessage.getProperty("special") }
def retrieveAttributesFromMessage() { // do something with muleMessage property.. }
def makeResponseSpecial(response, attributes) { // transform response here ... }

}

There were no problems with the code as written when I was sending only one message at a time to my service. When the service had an opportunity to grab and process more than one message concurrently, however, I would consistently run into the following excpetion:

Caused by: java.lang.IllegalStateException: Only owner thread can write to message: Thread[asyncDelivery4,5,main]/Thread[asyncDelivery2,5,main]
at org.mule.transport.AbstractMessageAdapter.newException(AbstractMessageAdapter.java:619)
at org.mule.transport.AbstractMessageAdapter.checkMutable(AbstractMessageAdapter.java:605)
at org.mule.transport.AbstractMessageAdapter.assertAccess(AbstractMessageAdapter.java:546)
at org.mule.transport.AbstractMessageAdapter.setProperty(AbstractMessageAdapter.java:244)

Obviously Mule must not be very thread safe, right? I'm obviously not doing anything bad in my code to deserve this...

It's always easier to make something you had no hand in writing the first target for all your problems, but the truth of the matter is, by saving the retrieved Mule message in the muleMessage property, I am giving my transformer state.

Thinking functionally, the same request object and message properties should always map to or produce the same result. Therefore, the transformer could be implemented with one big method where all my code would reside, the doTransform() method here. On the other hand, with one conceptual way in and one way out, it couldn't hurt to just store the Mule message in a property instead of passing it around, right?

It seems however that transformers are instantiated in Mule as singletons (don't quote me on that), or at the very least I was not doing anything to ensure or verify they weren't. So by storing the message as a property, concurrent message threads were trying to access the same Mule message. The lesson here, (for me at least), is that I probably shouldn't store variables as properties unless I have a good reason to, when having state makes sense.

The revised code:

class ResponseTransformer extends AbstractTransformer {

@Override
protected Object doTransform(Object response, String encoding) {
try {
def muleMessage = retrieveMuleMessage()
if (isSpecialResponseExpected(muleMessage)) {
def attributes = retrieveAttributesFromMessage(muleMessage)
response = makeMessageSpecial(response, attributes)
}
} catch(e) {
throw new TransformerException(this, e)
}
return response
}

def retrieveMuleMessage() { RequestContext.event.message }

boolean isSpecialResponseExpected(muleMessage) { muleMessage.getProperty("special") }
def retrieveAttributesFromMessage(muleMessage) { // do something with muleMessage property.. }
def makeResponseSpecial(response, attributes) { // transform response here ... }
}

Passing the Mule message around is more functional, and could at the very least help me with my unit tests, where I wouldn't have to set the "state" of the transformer to test the methods that could have just as easily have another parameter in their signatures.

Saturday, October 3, 2009

XML Schema Validation with a Simple Groovy Script

I'm sure there are many methods for validating XML documents against schemas out there in Java-land. I decided I want a relatively simple way to do it using the command line method using a Groovy script. Since I'd also need to leverage some libraries to do the job, I thought I'd also try running my script from Maven, so once I defined the libraries I needed, I wouldn't have to worry about having to have them set up.

To start, I generated a simple project, selecting the basic GMaven archetype:

  $ mvn archetype:generate
...
45: internal -> gmaven-archetype-basic (Groovy basic archetype)
...
Which is located in the Codehaus repository at: http://repository.codehaus.org.

Next, I edited the POM to include dom4j, the final version:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>prystasj</groupId>
<artifactId>schema-validation</artifactId>
<name>Schema Validation</name>
<version>1.0-SNAPSHOT</version>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.groovy.maven</groupId>
<artifactId>gmaven-plugin</artifactId>
<version>1.0-rc-5</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>org.codehaus.groovy.maven.runtime</groupId>
<artifactId>gmaven-runtime-1.6</artifactId>
<version>1.0-rc-5</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>dom4j</groupId>
<artifactId>dom4j</artifactId>
<version>1.6.1</version>
</dependency>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy</artifactId>
<version>1.6.4</version>
</dependency>
</dependencies>
</project>

I wrote up this little script, src/main/groovy/Validator.groovy:
import org.dom4j.io.SAXReader

def (schema, document) = args
def schemaStream = new File(schema).newInputStream()
def documentReader = new File(document).newReader()
SAXReader reader = new SAXReader()
setupSaxReader(reader, schemaStream)
reader.read(documentReader)
println "Document valid.\n"

def setupSaxReader(reader, stream) {
reader.setValidation(true)
reader.setFeature("http://apache.org/xml/features/validation/schema", true)
reader.setFeature("http://apache.org/xml/features/validation/schema-full-checking", true)
reader.setProperty("http://java.sun.com/xml/jaxp/properties/schemaLanguage", "http://www.w3.org/2001/XMLSchema")
reader.setProperty("http://java.sun.com/xml/jaxp/properties/schemaSource", stream)
}

As you can see, the script takes two arguments, the first being a schema to validate against, and the second is a XML instance docuemnt. I found a test schema from another test project and copied it over to my project directory along with a instance document.

I can run the script by first compiling:

  $ mvn compile
And then using the exec plugin in offline mode (-o) to speed things up as I already have all the jars I need so there's no need for Maven to check:
  $ mvn -o exec:java -Dexec.mainClass=Validator -Dexec.args="order.xsd order.xml"
If the validation succeeds, I'm given the normal BUILD SUCCESSFUL output. If not, the error is presented on the screen, for example:
  Error on line 7 of document  : cvc-complex-type.2.4.b: The content of element 'customer' is not complete.
One of '{country}' is expected. Nested exception: cvc-complex-type.2.4.b:
The content of element 'customer' is not complete.
One of '{country}' is expected.
To finish up, remembering the mvn command is a little rough for me, so I wrapped it into a shell script:
  #!/bin/bash
mvn -o exec:java -Dexec.mainClass=Validator -Dexec.args="$1 $2"
Which I can call with:
  $ ./validate.sh order.xsd order.xml

This little project has proved pretty useful as I can edit the schema or XML instance and run it though rather quickly. I'm sure there are plenty of other methods out there.

Saturday, September 26, 2009

VPNC & Linux: "vpnc: no response from target" and host lookup

In trying to use VPNC on my desktop, running Arch Linx at home to VPN to work, I encountered two problems. The first is the popular vpnc: no response from target problem and the second involved determining the name of my host once I was connected.

I installed VPNC using Arch's package manager:

$ pacman -Sy vpnc
Then I copied a configuration file from my laptop running Ubuntu 9.04 to /etc/vpnc/mycompany.conf (I'm omitting the gateway address of course and replacing it with a series of X's):
IPSec ID 3des
IPSec gateway XXX.XXX.XX.X
IPSec secret 3des
Xauth username prystasj
NAT Traversal Mode cisco-udp

This worked fine for a while, but eventually I was unable to connect with the popular vpnc: no response from target exception. After some googling, I was added the following two IKE directives and a Local directive for the port:

IPSec ID 3des
IPSec gateway XXX.XXX.XX.X
IPSec secret 3des
Xauth username prystasj
IKE Authmode psk
IKE DH Group dh2
NAT Traversal Mode cisco-udp
Local Port 10000
Then I was back in business When viewing similar postings about the no response error, it was noted the NAT directive was one that was also commonly needed.

Hopefully the above can help some with similar connection issues. Now for the second problem. After my VPN connection was created, a Java application I needed to run required to lookup the hostname, was failing with a java.net.UnknownHostException.

To confirm, I ran nslookup with my hostname and received an error at the end of the output:

$ nslookup myhost
Server: 123.456.78.999
Address: 123.456.78.999#11

** server can't find myhost: NXDOMAIN
The answer it turns out (for me at least) lied in /etc/resolv.conf. When running VPNC the file was replaced with the contents similar to the following:
#@VPNC_GENERATED@ -- this file is generated by vpnc
# and will be overwritten by vpnc
# as long as the above mark is intact
# Generated by dhcpcd from eth0
# /etc/resolv.conf.head can replace this line
domain gateway.2wire.net
nameserver 132.174.47.100
nameserver 132.174.47.101
# /etc/resolv.conf.tail can replace this line
I needed to add the address of my server as a nameserver taking the address from the output of nslookup:
domain gateway.2wire.net
nameserver 123.456.78.999
nameserver 132.174.47.100
nameserver 132.174.47.101
Now both nslookup, and therefore my Java application, can resolve the hostname:
$ nslookup myhost
Server: 123.456.78.999
Address: 123.456.78.999#11

Name: myhost.somegateway.someprovider.net
Address: 321.654.87.999

Unfortunately, I can no longer reach any hosts on the network I'm now connected to. I need to do some more research on resolving it seems. I also tried a search directive. I followed the advice here for dnsmasq: Setup of OpenDNS and dnsmasq (on Arch Linux).

Now the Java application cited it could not find a host with name D5BMHS91, so I had to edit my /etc/hosts as such:

# /etc/hosts: static lookup table for host names
127.0.0.1 localhost myhost D5BMHS91

Notes: This was all done on Arch Linx 2009.08. In order to get nslookup to work, I had to install dnsutils.

$ sudo pacman -Sy dnsutils

Sunday, September 6, 2009

Groovy & XML: Retrieve the Namespace of Nested Elements

Recently, I need to retrieve the default namespace from an incoming message such as:

<message xmlns="Message" a="1" b="2">
<request>
<text xmlns="Text" a="1">ok</text>
</request>
</message>

With a message like the above, I want to retrieve the namspace Text from the text element.

One method I was able to leverage was using Groovy's DOMBuilder and DOMCategory to read the message and retrieve the namespace:

import groovy.xml.DOMBuilder
import groovy.xml.dom.DOMCategory

def xml = """<message xmlns="Message" a="1" b="2">
<request>
<text xmlns="Text" a="1">ok</text>
</request>
</message>"""

def reader = new StringReader(xml)
def doc = DOMBuilder.parse(reader)
def root = doc.documentElement
def ns

use(DOMCategory) {
ns = root['request']['text'].'@xmlns'[0]
}
println "ns=$ns"

Output:

ns=Text

Omitting the lookup of the first element, we would get a list:

ns = root['request']['text'].'@xmlns'

Output:

ns=["Text"]

This could be used to get a list of all attribute value for an element.

Another method is to use XMLParser:

def doc = new XmlParser(false, false).parseText(xml)
def ns = doc.'request'.'text'.'@xmlns'
Or XMLSlurper:
def doc = new XmlSlurper(false, false).parseText(xml)
def ns = doc.'request'.'text'.'@xmlns'

Both methods requiring making the XML reader namespace unaware with the following constructors:

XmlSlurper(boolean validating, boolean namespaceAware)
XmlParser(boolean validating, boolean namespaceAware)

I'm sure there are plenty of other methods to do the same, if you have one please feel free to leave a comment.

Tuesday, September 1, 2009

Mule: Using JiBX in Mule

Below I'd like to share my attempt to use JiBX inside of Mule to transform incoming XML message into a domain object and back again on the way out. In short, I created a request and response transformer that extended AbstractTransformer.

To start, I had the following domain objects:

   class Person {
String firstName
String lastName
Address addresss
}

class Address {
String city
String state
}

And a sample XML message where a Person has an Address:

  <person>
<firstname>George</firstname>
<lastname>Costanza</lastname>
<address>
<city>Queens</city>
<state>NY</state>
</address>
</person>

The binding file I used to test out my transformers:

  <?xml version="1.0" encoding="UTF-8"?>
<binding>
<mapping name="person" class="Person">
<value name="firstname" field="firstName"/>
<value name="lastname" field="lastName"/>
<structure name="address" field="address" class="Address">
<value name="city" field="city" />
<value name="state" field="state" />
</structure>
</mapping>
</binding>

Ok, now to something more interesting with that out of the way... the transformer to turn an incoming XML message to a Person object to use in my service component. Below is one representation of the transformer in Groovy with the types releated to JiBX declared:

class JibxXmlToObject extends AbstractJibxTransformer {
protected Object doTransform(Object src, String encoding) throws TransformerException {
def transformedObject
try {
Reader reader = new StringReader((String) src)
IBindingFactory factory = BindingDirectory.getFactory(Class.forName(getTargetClassName()))
IUnmarshallingContext context = factory.createUnmarshallingContext()
transformedObject = context.unmarshalDocument(reader)
}
catch(e) {
throw new TransformerException(this, e)
}

return transformedObject
}
}

To test out my transformer originally, I originally hard-coded the target class name:

  IBindingFactory factory = BindingDirectory.getFactory(Class.forName("org.prystasj.Person"))

In my Mule configuration, I will be declaring it however. If the transformer was a Java class, I might have to have getters and setters to allow for the target class name to be injected. Below is my declaration of the transformer as a custom transformer:

  <custom-transformer name="XmlToPerson" class="org.prystasj.transformers.JibxXmlToObject">
<spring:property name="targetClass" value="org.prystasj.Person"/>
</custom-transformer>
  <file:endpoint name="XmlInboundEndpoint"
  path="/tmp/input"
  pollingFrequency="3000"
  transformer-refs="XmlToPerson"/>

And here are the outgoing side of things, transforming a Person back to XML:
public class JibxObjectToXml extends AbstractJibxTransformer {
protected Object doTransform(Object src, String encoding) throws TransformerException {
String xml
try {
def writer = new StringWriter()
IBindingFactory factory = BindingDirectory.getFactory(Class.forName(getTargetClassName()))
IMarshallingContext context = factory.createMarshallingContext()
context.marshalDocument(src, encoding, null, writer)
xml = writer.toString()
} catch(e) {
throw new TransformerException(this, e)
}

return xml
}
}

And the corresponding Mule configuration:

  <custom-transformer name="PersonToXml" class="org.prystasj.transformers.JibxObjectToXml">
<spring:property name="targetClass" value="org.prystasj.Person"/>
</custom-transformer>
  <file:endpoint name="XmlOutboundEndpoint"
path="/tmp/output"
pollingFrequency="3000"
responseTransformer-refs="PersonToXml"/>

Putting the pieces together with a service definition, this example would take a Person in XML and echo it back out in XML after performing both transforms:

  <model name="XmlPersonModel">
<service name="XmlInputService">
<inbound>
<file:inbound-endpoint ref="XmlInboundEndpoint"/>
</inbound>
<echo-component/>
<outbound>
<pass-through-router>
<file:outbound-endpoint ref="XmlOutboundEndpoint"/>
</pass-through-router>
</outbound>
</service>
</model>

A more useful service component could be used to manipulate the incoming person in some way.