Sunday, September 21, 2014

Groovy: Import Aliases

A feature I like in Groovy is the ability to import class with aliases. I had forgotten about this feature, so I'm writing about it here to help get it to stick with me a bit.

Take this Spock test class tha has to define a couple several values of a couple types:

    import org.company.id.CorporateSymbol
    import org.company.id.CorporateName

    class CorporationMapperSpec extends Specification {

        @Subject
        CorporationMapper mapper = new CorporationMapper()

        def 'maps corporate symbols to names'() {
            expect:
            symbolsToNames == mapper.map(symbols)
        }

        def symbol1 = new CorporateSymbol('KRA')
        def symbol2 = new CorporateSymbol('VLY'),
        def symbll3 = new CorporateSymbol('KRS'),

        def symbolsToNames = [
            (symbol1): new CorporateName('Kramerica')
            (symbol2): new CorporateName('Vandalay')
            (symbol3): new CorporateName('Kruger')
        }
    }

We can use aliases with the import statements to make the code easier to work with:

    import org.company.id.CorporateSymbol as CS
    import org.company.id.CorporateName as CN

    class CorporationMapperSpec extends Specification {

        @Subject
        CorporationMapper mapper = new CorporationMapper()

        def 'maps corporate symbols to names'() {
            expect:
            symbolsToNames == mapper.map(symbols)
        }

        def symbol1 = new CS('KRA')
        def symbol2 = new CS('VLY'),
        def symbll3 = new CS('KRS'),

        def symbolsToNames = [
            (symbol1): new CN('Kramerica')
            (symbol2): new CN('Vandalay')
            (symbol3): new CN('Kruger')
        }
    }

I would be a little wary about using it in main source, as it does add a little bit of indirection to the readability of the code, but for test source, I think it can definitely help making the code easier to read and work with.

Sunday, September 7, 2014

Jackrabbit and XPath Queries: Escaping Paths

Jackrabbit and XPath Queries: Escaping Paths

We have a service that manages a Apache Jackrabbit repository. The main client of the service builds lists of records of the form: ///, where user's are represented by a UUID. For example:

  /JCP/feeadeaf-1dae-427f-bf4e-842b07965a93/label/

Now we started to build a web endpoint to the service that can be used to view the contents of the repository at any time. This endpoint may want to create a query into the repository along the lines of "show me all lists for institution JCP and user feeadeaf-1dae-427f-bf4e-842b07965a93". When we create lists in the repository, we increment a property named sequence (more on this on a later date), so an XPath query that proved to work for us given the above example proved to be:

  /*/JCP/feeadeaf-1dae-427f-bf4e-842b07965a93/label//*[@sequence]

This was working well at first until we started to execute queries where the UUID had a leading digit. We would see an exception in our logs of the form:

  Encountered "-" at line 1, column 26.
  Was expecting one of:
   ...
   ...
   ...
   ...
  ...
     for statement: for $v in /*/JCP/2eeadeaf-1dae-427f-bf4e-842b07965a93/label//*[@sequence] return $v

Since the hypen was indicated as the culprit by the exception message, and given the fact we only ran into this when the UUID node began with a digit, we thought that both conditions were required to create the invalid query. Our first attempt at a solution simply invovled prefixing all nodes that fit the pattern of a UUID with a string like uuid_.

In reality, it was simply the leading digit that caused the problem. The above solution would definitely not work for all future use cases of the service. The problem can simply be stated that the query was invalid as XML nodes cannot start with digits. We could still however create paths into the repository in the above manner as long as we escaped the leading digit in the query:

  /*/JCP/_x0032_eeadeaf-1dae-427f-bf4e-842b07965a93//*[@sequence]

The code below demonstrates our modeling of a path into the repository and the method by which we perform the escaping. Before that though it is worth noting that we had to encode the individual steps when building the path included in the query. If we included the whole path in the escaping logic, the delimiting slashes would be escaped, and the query would not work. If we included the query as a whole into the escaping logic, the asterisks would be escaped, and the query would not also work.

    import org.apache.commons.lang3.StringUtils;
    import org.apache.jackrabbit.util.ISO9075;

    class Path {
        List<String> steps; //...

        public String asQuery() {
            return steps.size() > 0 ? "/*" + asPathString(encodedSteps()) + "//*" : "//*";
        }

        private String asPathString(List<String> steps) {
            return '/' + StringUtils.join(steps, '/');
        }

        private List<String> encodedSteps() {
            List<String> encodedSteps = new ArrayList<>();
            for (String step : steps) {
                encodedSteps.add(ISO9075.encode(step));
            }
            return encodedSteps;
        }
    }

The ISO9075 class can be found in org.apache.jackrabbit:jackrabbit-jcr-commons.

My initial search for answers started on Stack Overflow, and of course the good folks answering questions there didn't let me down!

Saturday, August 30, 2014

Using Maven to Detect Duplicate Classes

Using Maven to Detect Duplicate Classes

As the number of dependencies from some our corporate project grew, we ran into a couple cases where we would be transitively pulling into different versions of the same classes. For a long time, we were lucky, as this did not cause any issues, until someday a developer new to our team introduced trying to run our application using a different operating system. The JVM on his machine loaded up the classes in a different order, and his application instance used an older version of a class that resulted in errors running the application.

This developer thankfully pointed out we can detect when we were pulling in different versions of classes using the Maven Enforcer plugin with some help of some extra rules. We added something like this to our parent POM:

    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-enforcer-plugin</artifactId>
        <version>1.3.1</version>
        <executions>
            <execution>
                <id>enforce</id>
                <phase>validate</phase>
                <configuration>
                    <rules>
                        <banDuplicateClasses>
                            <ignoreClasses>
                                <ignoreClass>javax.*</ignoreClass>
                                <ignoreClass>org.junit.*</ignoreClass>
                                <ignoreClass>org.hamcrest.*</ignoreClass>
                                <ignoreClass>org.slf4j.*</ignoreClass>
                            </ignoreClasses>
                            <findAllDuplicates>true</findAllDuplicates>
                        </banDuplicateClasses>
                    </rules>
                    <fail>false</fail>
                </configuration>
                <goals>
                    <goal>enforce</goal>
                </goals>
            </execution>
        </executions>
        <dependencies>
            <dependency>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>extra-enforcer-rules</artifactId>
                <version>1.0-beta-2</version>
            </dependency>
        </dependencies>
    </plugin>

Example output from the validate phase of a build:

    [INFO] --- maven-enforcer-plugin:1.1:enforce (enforce) @ cataloging-api ---
    [INFO] Adding ignore: javax.*
    [INFO] Adding ignore: org.junit.*
    [INFO] Adding ignore: org.hamcrest.*
    [INFO] Adding ignore: org.slf4j.*
    [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BanDuplicateClasses failed with message:
    Duplicate classes found:

      Found in:
        org.company.id.api:id-api:jar:1.28.0:compile
        org.compnay.legacy.id-api:id-common-api:jar:2.8.0:compile
      Duplicate classes:
        org/company/service/id/SAMLEntityId.class
        org/company/service/id/crypto/ServiceKey.class

We were surprised at the amount of duplicate classes we were bringing in. We thought our dependency management was in pretty good shape, but adding the new rules showed we still had some work to do.

It is also worth noting, while our project is large, adding the plugin execution to the build added an insignifcat amount of time to the build overall, so there should be no performance concerns if you were considering adding something similar to your project.

Saturday, August 9, 2014

Spring AOP: Excluding Methods in Pointcuts

We use Spring AOP to make use of aspects in our applications for the purpose of recording statistics about requests they receive. This approach has worked out well because it helps separate the concern of tracking statistics from our business logic. Previous iterations of our applications mixed the two, leaing to code that was generally harder to maintain.

Below is an example of how we might configure an aspect to target all entry points into our business application with the same aspect:

    <aop:config>
        <aop:pointcut id="requestPointcut"
                      expression="execution(public * org.company.service.*.*(..))"/>
        <aop:aspect id="requestAspect" ref="requestStatAspect">
            <aop:around method="around" pointcut-ref="requestPointcut"/>
        </aop:aspect>
    </aop:config>

    <bean id="requestAspect" class="org.company.service.aop.RequestIdentifierAspect" lazy-init="true"/>

With the above, we are saying we want request identifiers managed for every request to our interfaces. With the wildcards in the expression, we are saying target every class in the org.mycompany.myapp.service package and every method in those classes.

This approach worked greatly until we added an interface that provided health reporting for system monitoring purposes. This method lived in a new interface. Let's suppose added the interface method is defined as:

  • public boolean org.company.HealthService.isHealthy()
    

Since the method signature is different (it takes no arguments), running the advice in the aspect would lead to errors as the aspect evaluated the method arguments. We could change the code in the aspect to avoid processing if the arguments contained within the join point do not match what we are expecting. This approach is not that attractive because it one, involves changing code for special cases, and two, could lead to the aspect siltently not invoking its main logic at times we might not expect. If we target a method with the aspect unintentionally, we want to be informed by errors in our log.

An alternative we found was to exclude the new method from the pointcut expression:

  • execution(public * org.company.service.*.*(..)) &amp;&amp; !execution(* org.company.service.HealthService.*(..))
    

Just with adding special case logic to an aspect or code in general to avoid exception cases, we have to be careful that the exceptional cases we avoid targeting with the aspect does not get out of hand as we are tempted to add more in the future.

Saturday, June 28, 2014

Sending data to a server with Groovy

The Groovy language provides a lot of additions to existing Java classes that can make our jobs easier.

A couple of them we were able to take advantage of lately that use closures to make our coding simpler were:

Our use case: We have a service whose responsibility is to send "records" over a TCP/IP connection to a remote server.

Of course we want to include a test for this in our integration tests, but how can we do it? We cannot expect a client to leave a server available open to receive records sent by our tests. We could create our test server to accept and output the sent record.

We'll write a test server using Groovy that can be started at the beginning of the test run and shutdown at the end:

class TestServer {

    static defaultPort = 6666

    static ServerSocket start_on_default_port() {
        start_on defaultPort
    }

    static ServerSocket start_on(port) {

        def server = new ServerSocket(port)
        String receivedRecord ''
        def done = false

        Thread.start { t ->
            while (!done) {
                server.accept { socket ->
                    socket.withStreams { input, output ->
                        done = false

                        while (!done) {
                            def block = new byte[128]
                            int size = input.read block

                            if (size == -1) done = true // there is nothing left to have been received

                            block = trim(block as List)
                            receivedRecord += new String(block)
                        }
                    }
                }
            }
        }
        server
    }

    /** Remove any empty or non-set bytes at the end of the array. */
    private static def trim(List block) {
        block.reverse().findAll { b -> b != 0 }.reverse() as byte[]
    }
}

In our integration tests class, written with Spock, we start and stop the server at the beginning of the run:

@ContextConfiguration
class RecordDeliveryServiceIntegrationSpec extends Specification {

    @Autowired
    RecordDeliveryService service

    static def server

    def setupSpec() {
        server = TestServer.start_on_default_port()
    }

    def cleanupSpec() {
        server.close()
    }

    def 'delivers a record to a server'() {
        expect:
        expectedResponse == service.send 'localhost', TestServier.defaultPort, record
    }

    //...
}

For reference to complete the example, here's what a simplified service implementation could look like, using a methodology similar to that the test server uses to accept the record:

dclass RecordDeliveryService implements DeliveryService {

    public def send(String host, long port, Record record) {

        def socket = new Socket(host, port)

        socket.withStreams { input, output -&gt;
            output &lt;&lt; record.toString()
            output.flush()
            output.close()
        }

        socket.close()
    }
}   //...
}

If you can think of any improvements, or a better approach to the problem, I would be glad to hear from you!

Maven: Failing the Build After Running All Modules

Maven: Failing the Build After Running All Modules

Here's a quick one to help me feel a better about not posting anything for quite a while.

In our large project at work that contains several deployed services, we have several parent integration testing Maven modules that run after our nightly development install. Each has several child modules that each run tests against a particular endpoint in each service.

So naturally, we want our build server to kick off each parent testing module after the install. The problem we ran into is when that one of the earlier child modules failed, the build will stop, without running the rest of the modules (Maven's default behavior).

This would often lead to us making changes to get the tests in the failing module to pass, then re-running the testing build, only to have a later child module to fail. It would be nice to have had all the modules to run, reporting all failures, so that all the required changes and fixes could be made up front, saving us time.

Fortunately, Maven provides an option we learned about do just that:

-fae,--fail-at-end     Only fail the build afterwards;
                       allow all non-impacted builds to continue

There are also a couple more failure related options as well, but we have yet to find any use for them (yet):

-ff,--fail-fast        Stop at first failure in reactorized builds
-fn,--fail-never       NEVER fail the build, regardless of project result

When working on the tests locally, we can pick up the build from within the parent from the point of the failed chilid module after making changes to our running service to fix the test with:

-rf,--resume-from      Resume reactor from specified project

For example:

$ mvn verify -rf :service-integration-testing-records

For more on working with multiple modules, see: Guide to Working with Multiple Modules.

Saturday, May 11, 2013

Spring AOP: Ordering Aspects

Sometimes we may want to apply multiple aspects to the same pointcut when using Spring AOP, keeping each aspect responsible for a single activity or concern. Below we'll discuss a business example of why we may want to order more than one aspects and how to ensure that ordering using Spring. Again, we'll be writing our code in Groovy to keep things simple.

To start our example, let's say we have a service method that takes a Purchase and returns a Shipment:

interface PurchaseProcessor {
    Shipment process(Purchase)
}

Our first use of AOP may be recording some information about the Purchase for statistics purpose. Since statistics keeping is a separate concern from actually processing the Purchase, it serves as a perfect candidate for an aspect. For the purposes of this example, we will use an aspect that provides around advice instead of an alternative like before advice:

class StatRecordingAspect {

    StatRecorder statRecorder

    Object around(ProceedingJoinPoint joinPoint) {
        Purchase purchase = joinPoint.args[0]
        statRecorder.record purchase
        joinPoint.proceed()
    }
}

Later on, we decide we want to inform an internal process via JMS when the processed Shipment is on its way out the door. As the sending of the information could be considered another concern separate from the actual processing of the Purchase we decide it to should be done in an aspect. Additionally, this notification requires information from the statistics from the Purchase:

class NotificationAspect {

    JmsShipmentNotifier notifier
    StatRecorder statRecorder

    Object around(ProceedingJoinPoint joinPoint) {
        Purchase purchase = joinPoint.args[0]
        Shipment shipment = (Shipment)joinPoint.proceed()
        notifier.notifyAbout shipment, statRecord.statsFor(purchase)
        shipment
    }
}

The above business case requires that the stats about the Purchase be recorded before the notification. If we target both aspects at the PurchaseProcessor method via the same pointcut, how do we guarantee that the stats are recorded first?

We could opt to combine the two aspects into one, but it could be argued then the combined aspect would have one more than responsibility, record stats and sending out notifications, and have more than one concern.

A solution provided by Spring AOP is to have each aspect implement the Ordered interface provided by spring-core (Spring also provides an @Order annotation as an alternative to implemented the Ordered interface.):

.
class StatRecordingAspect implements Ordered { //...

    Object around(ProceedingJoinPoint joinPoint) { //... }

    @Override
    int getOrder() {
        0
    }
}

class NotificationAspect implements Ordered { //...

    Object around(ProceedingJoinPoint joinPoint) { //... }

    @Override
    int getOrder() {
        1
    }
}

With the above implementations, the StatRecordingAspect would be executed first as it is the aspect with the lowest order.

If the above makes you a little uneasy because the values returned are not linked to each other in any way, the values seem arbitrary, you could relate the orders to each other with an enumeration:

enum AspectOrder {

    STAT_RECORDING(0),
    NOTIFICATION(1)

    final int value

    AspectOrder(int value) {
        this.value = value
    }

    int value() {
        value
    }
}


class StatRecordingAspect implements Ordered { //...

    Object around(ProceedingJoinPoint joinPoint) { //... }

    @Override
    int getOrder() {
        AspectOrder.STAT_RECORDING.value
    }
}

class NotificationAspect implements Ordered { //...

    Object around(ProceedingJoinPoint joinPoint) { //... }

    @Override
    int getOrder() {
        AspectOrder.NOTIFICATION.value
    }
}

Now in your unit tests you can compare the order value of one aspect to the others when testing the getOrder() method, ensuring they would in practice be ordered as you would expect.