Saturday, June 28, 2014

Sending data to a server with Groovy

The Groovy language provides a lot of additions to existing Java classes that can make our jobs easier.

A couple of them we were able to take advantage of lately that use closures to make our coding simpler were:

Our use case: We have a service whose responsibility is to send "records" over a TCP/IP connection to a remote server.

Of course we want to include a test for this in our integration tests, but how can we do it? We cannot expect a client to leave a server available open to receive records sent by our tests. We could create our test server to accept and output the sent record.

We'll write a test server using Groovy that can be started at the beginning of the test run and shutdown at the end:

class TestServer {

    static defaultPort = 6666

    static ServerSocket start_on_default_port() {
        start_on defaultPort
    }

    static ServerSocket start_on(port) {

        def server = new ServerSocket(port)
        String receivedRecord ''
        def done = false

        Thread.start { t ->
            while (!done) {
                server.accept { socket ->
                    socket.withStreams { input, output ->
                        done = false

                        while (!done) {
                            def block = new byte[128]
                            int size = input.read block

                            if (size == -1) done = true // there is nothing left to have been received

                            block = trim(block as List)
                            receivedRecord += new String(block)
                        }
                    }
                }
            }
        }
        server
    }

    /** Remove any empty or non-set bytes at the end of the array. */
    private static def trim(List block) {
        block.reverse().findAll { b -> b != 0 }.reverse() as byte[]
    }
}

In our integration tests class, written with Spock, we start and stop the server at the beginning of the run:

@ContextConfiguration
class RecordDeliveryServiceIntegrationSpec extends Specification {

    @Autowired
    RecordDeliveryService service

    static def server

    def setupSpec() {
        server = TestServer.start_on_default_port()
    }

    def cleanupSpec() {
        server.close()
    }

    def 'delivers a record to a server'() {
        expect:
        expectedResponse == service.send 'localhost', TestServier.defaultPort, record
    }

    //...
}

For reference to complete the example, here's what a simplified service implementation could look like, using a methodology similar to that the test server uses to accept the record:

dclass RecordDeliveryService implements DeliveryService {

    public def send(String host, long port, Record record) {

        def socket = new Socket(host, port)

        socket.withStreams { input, output ->
            output << record.toString()
            output.flush()
            output.close()
        }

        socket.close()
    }
}   //...
}

If you can think of any improvements, or a better approach to the problem, I would be glad to hear from you!

Maven: Failing the Build After Running All Modules

Maven: Failing the Build After Running All Modules

Here's a quick one to help me feel a better about not posting anything for quite a while.

In our large project at work that contains several deployed services, we have several parent integration testing Maven modules that run after our nightly development install. Each has several child modules that each run tests against a particular endpoint in each service.

So naturally, we want our build server to kick off each parent testing module after the install. The problem we ran into is when that one of the earlier child modules failed, the build will stop, without running the rest of the modules (Maven's default behavior).

This would often lead to us making changes to get the tests in the failing module to pass, then re-running the testing build, only to have a later child module to fail. It would be nice to have had all the modules to run, reporting all failures, so that all the required changes and fixes could be made up front, saving us time.

Fortunately, Maven provides an option we learned about do just that:

-fae,--fail-at-end     Only fail the build afterwards;
                       allow all non-impacted builds to continue

There are also a couple more failure related options as well, but we have yet to find any use for them (yet):

-ff,--fail-fast        Stop at first failure in reactorized builds
-fn,--fail-never       NEVER fail the build, regardless of project result

When working on the tests locally, we can pick up the build from within the parent from the point of the failed chilid module after making changes to our running service to fix the test with:

-rf,--resume-from      Resume reactor from specified project

For example:

$ mvn verify -rf :service-integration-testing-records

For more on working with multiple modules, see: Guide to Working with Multiple Modules.

Saturday, May 11, 2013

Spring AOP: Ordering Aspects

Sometimes we may want to apply multiple aspects to the same pointcut when using Spring AOP, keeping each aspect responsible for a single activity or concern. Below we'll discuss a business example of why we may want to order more than one aspects and how to ensure that ordering using Spring. Again, we'll be writing our code in Groovy to keep things simple.

To start our example, let's say we have a service method that takes a Purchase and returns a Shipment:

interface PurchaseProcessor {
    Shipment process(Purchase)
}

Our first use of AOP may be recording some information about the Purchase for statistics purpose. Since statistics keeping is a separate concern from actually processing the Purchase, it serves as a perfect candidate for an aspect. For the purposes of this example, we will use an aspect that provides around advice instead of an alternative like before advice:

class StatRecordingAspect {

    StatRecorder statRecorder

    Object around(ProceedingJoinPoint joinPoint) {
        Purchase purchase = joinPoint.args[0]
        statRecorder.record purchase
        joinPoint.proceed()
    }
}

Later on, we decide we want to inform an internal process via JMS when the processed Shipment is on its way out the door. As the sending of the information could be considered another concern separate from the actual processing of the Purchase we decide it to should be done in an aspect. Additionally, this notification requires information from the statistics from the Purchase:

class NotificationAspect {

    JmsShipmentNotifier notifier
    StatRecorder statRecorder

    Object around(ProceedingJoinPoint joinPoint) {
        Purchase purchase = joinPoint.args[0]
        Shipment shipment = (Shipment)joinPoint.proceed()
        notifier.notifyAbout shipment, statRecord.statsFor(purchase)
        shipment
    }
}

The above business case requires that the stats about the Purchase be recorded before the notification. If we target both aspects at the PurchaseProcessor method via the same pointcut, how do we guarantee that the stats are recorded first?

We could opt to combine the two aspects into one, but it could be argued then the combined aspect would have one more than responsibility, record stats and sending out notifications, and have more than one concern.

A solution provided by Spring AOP is to have each aspect implement the Ordered interface provided by spring-core (Spring also provides an @Order annotation as an alternative to implemented the Ordered interface.):

.
class StatRecordingAspect implements Ordered { //...

    Object around(ProceedingJoinPoint joinPoint) { //... }

    @Override
    int getOrder() {
        0
    }
}

class NotificationAspect implements Ordered { //...

    Object around(ProceedingJoinPoint joinPoint) { //... }

    @Override
    int getOrder() {
        1
    }
}

With the above implementations, the StatRecordingAspect would be executed first as it is the aspect with the lowest order.

If the above makes you a little uneasy because the values returned are not linked to each other in any way, the values seem arbitrary, you could relate the orders to each other with an enumeration:

enum AspectOrder {

    STAT_RECORDING(0),
    NOTIFICATION(1)

    final int value

    AspectOrder(int value) {
        this.value = value
    }

    int value() {
        value
    }
}


class StatRecordingAspect implements Ordered { //...

    Object around(ProceedingJoinPoint joinPoint) { //... }

    @Override
    int getOrder() {
        AspectOrder.STAT_RECORDING.value
    }
}

class NotificationAspect implements Ordered { //...

    Object around(ProceedingJoinPoint joinPoint) { //... }

    @Override
    int getOrder() {
        AspectOrder.NOTIFICATION.value
    }
}

Now in your unit tests you can compare the order value of one aspect to the others when testing the getOrder() method, ensuring they would in practice be ordered as you would expect.

Loading Classpath Resources from a Static Context

Recently, I had to figure out how to load classpath resources from within a static method in a test class. I've been doing it from a non-static method forever, but had to do some searching to figure out how to do it from a static method.

For example to build a scenario where one might want to do this, let's take a generic Record class that is used by many of our classes under test. As JSON is the input method for records from the user of our system, it might make sense to store our records to use during our tests as JSON test resources.

During our tests, we can load the record JSON from a test resource on the classpath with a test helper class like the below. The Groovy test class uses the ObjectMapper class from Jackson:

class RecordBuilder {
    def recordFor(resource) {
        new ObjectMapper().readValue textOf(resource), Record.class
    }
    def textOf(resource) {
        getClass().getClassLoader().getResourceAsStream(resource).text
    }
}

One annoyance with using the above class in our test classes is that each test class would need to create or be given an instance of RecordBuilder to do its work. An alternative is to make the record-building method static, which should be ok since the normal concerns regarding static methods, pro or con, should not apply in a testing scenario as tests are usually executed one at at time.

The problem with making the recordFor method static is the approach to reading the resource from the classpath no longer works, and the solution may not be apparent to those who have not tried to do something similar before. Below includes the answer I was able to find, referring to the RecordBuilder class itself:

class RecordBuilder {
    static def recordFor(resource) {
        new ObjectMapper().readValue textOf(resource), Record.class
    }
    static def textOf(resource) {
        RecordBuilder.class.getClassLoader().getResourceAsStream(resource).text
    }
}

I guess this was a rather long-winded way to show how to load classpath resources from a static context, but I hope it will do. Thanks.

Wednesday, January 23, 2013

Spring & RestTemplate: Dealing with Double Escaped Entities

We've been using Spring's RestTemplate pretty extensively to perform various operations. Several have escaped entities in the query. For example, one query requires its parameters be surrounded by quotes:

http://myhost.com/records?query=no+"1"+owner="John"

We could inject that URL into a bean containing a RestTemplate instance, representing the URL with a String and escaping the quotes with %22 to make the parsing of the bean file easier:

    <bean id="recordUrl" class="java.lang.String">
        <constructor-arg value="http://myhost.com/records?query=no+%221%22+owner=%22John%22"/>
    </bean>

Then executing the query:

  restTemplate.getForObject(recordUrl, String.class, urlVariables)

However, our query would not work as each %22 would be escaped again as %2522. For example:

  DEBUG main org.springframework.web.client.RestTemplate - Created GET request for: "http://myhost.com/records?query=no+%25221%2522+owner=%2522John%2522"

How do we avoid the double escaping?. The RestTemplate Javadoc contains the answer. To avoid the double-escaping, we need to use the version of the RestTemplate method that takes a java.net.URI as the first parameters instead of a String representing the URL:

  restTemplate.getForObject(new URI(recordUrl), String.class, urlVariables)

Or instead of creating a URI, inject an instance of one like what was done with the String originally.

Sunday, October 28, 2012

Lombok and Spring: IntelliJ Plugin

As indicated in my last post, I've been playing around with Project Lombok a little. One of the easier to absorb features is the @Getter/@Setter annotations, which removes the need to explicitly define getters and setters in your Java classes.

I use IntelliJ as my main IDE. However, when I started to use Lombok, I would get error messages from the IDE when viewing a Spring configuration file that used setter injection.

The answer to this problem was to simply install the lombok-intellij-plugin, which fixed the error rather nicely.

We'll setup the scenario withe simplest of objects that I can think of. If you now have the answer you're looking for (and you probably do), feel free to skip the example below:

    public class Person {

        private String name;

        public Person() {}

        public Person(String name) {
            this.name = name;
        }

        public String getName() {
            return name;
        }

        public void setName(String name) {
            this.name = name;
        }
    }

A very simplisitc Spock specification that uses Spring testing to inject an instance of a Person:

    @ContextConfiguration
    class PersonSpec extends Specification {

        @Autowired
        Person person

        def "has a name"() {
            expect: person.name == 'Rudiger'
        }
    }

And the bean that is autowired in from PersonSpec-context.xml:

    <bean id="rudiger" class="prystasj.lombok.plugin.example.Person">
        <property name="name" value="Rudiger"/>
    </bean>

Here the test passes and there is no "red" for errors in our IDE.

Now if we replace the getter and setter in the Person with annotations from Lombok:

    public class Person {

        @Getter @Setter
        private String name;

        public Person() {}

        public Person(String name) {
            this.name = name;
        }
    }

The tests still passes, but our IDE is showing us some red when viewing the Spring configuration file:

At this point, installing the lombok-intellij-plugin sends the error away.

Wednesday, October 24, 2012

Lombok: Using @Getter/@Setter and @Delegate

I've been working on a project for the majority of the past year that's part of a larger architecture where the powers that be have not approved the use of a language like Groovy due to strong feelings about things like type safety, among other fears. I don't want to get into a debate about whether such fears have any foundation (maybe at a later date). I do however want to try to introduce Project Lombok here, which has helped simplify the Java we've been writing, helping to remove some of the boilerplate code that we've been writing, bringing our code a little closer to what it might look like with Groovy.

Lombok has many features to help simplify your Java. For now, I'll present just @Getter/@Setter and @Delegate. We'll compare a plain Java version of some object and present both a Lombok and Groovy version for comparison.

Our example is pretty straightforward, a nameable Rocket that uses a Booster to take off and record if the rocket has been started:

    public class Rocket {

        private final Booster booster;

        private String name;

        public Rocket(String name, Booster booster) {
            this.name = name;
            this.booster = booster;
        }

        public void takeoff() {
            booster.takeoff();
        }

        public boolean isStarted() {
            return booster.isStarted();
        }

        public String getName() {
            return name;
        }

        public void setName(String name) {
            this.name = name;
        }
    }

    public class Booster {

        private boolean started = false;

        public void takeoff() {
            System.out.println("3.. 2.. 1.. Java Rocket go now!");
            started = true;
        }

        public boolean isStarted() {
            return started;
        }
    }

The above code has some classic Java boilerplate in getters and setters, along with a couple one line methods that acts as simple pass-throughs from Rocket to its Booster collaborator.

Now we can introduce the @Getter/@Setter and @Delegate annotations to simplify the code (with the import statements ommitted for brevity):

    public class Rocket {

        @Delegate private final Booster booster;

        @Getter @Setter private String name;

        public Rocket(String name, Booster booster) {
            this.name = name;
            this.booster = booster;
        }
    }

    public class Booster {

        @Getter private boolean started = false;

        public void takeoff() {
            System.out.println("3.. 2.. 1.. Lombok Rocket go now!");
            started = true;
        }
    }

This code is simpler as the getters and setters have been removed, along with the pass-throughs. What would a more Groovy example look like?

    class Rocket {
        @Delegate private Booster booster
        String name
    }

    class Booster {

        boolean started = false;

        def takeoff() {
            println "3.. 2.. 1.. Groovy Rocket go now!"
            started = true
        }

    }

This code is extremely similar to the Lombok example. The @Getter and @Setter annotations are missing because with Groovy we get them by default, however, now we could conceivably find a way to directly set the started property of the Booster class. Another somewhat significant difference is the Rocket constructor is missing, which we would have to reinstate if we were using this code from Java.

The same Spock specification can be used to test all 3 classes to verify they provide the same behavior:

    class RocketSpec extends Specification {

        def name = "LombokRocket"
        def booster = new Booster()

        def rocket = new Rocket(name, booster)

        def "initially has not taken off"() {
            expect: !rocket.isStarted()
        }

        def "can takeoff"() {
            when: rocket.takeoff()
            then: rocket.isStarted()
        }

        def "has a name that can be changed"() {
            when: rocket.name =  name.reverse()
            then: rocket.name == name.reverse()
        }

    }

When looking at the Lombok version, it is definitely closer to the Groovy version, showing that if you are stuck using Java, there are some features you can still have with a little work.

Another point of interest is getting both Lomboked and Groovy code to compile alongside our Java code using Maven. I've uploaded this project to GitHub if you would like to view the POM and the rest of the source:

I'll look to add to the above project in the future to demonstrate some more Lombok features.

Comment away, and thanks!