Sunday, November 30, 2014

Subversion: Recovering Password from Local Cache

Subversion: Recovering Password from Local Cache

If you've forgotten your Subversion password, you should be able to retrieve it from another machine where you've succesfully made a commit before.

If you look in the local cache for your server's authentication type, you should see a series a files with hexadecimal names.

For example, for a server using simple or basic authentication:

  $ ls ~/.subversion/auth/svn.simple
  2af94d688c4073220b0a6af1b5884861  887652dff29c33e3f09394ea7379fac9

If you then look in one of the file, you should see your password:

  $ cat ~/.subversion/auth/svn.simple/2af94d688c4073220b0a6af1b5884861 | grep -A 2 password
    password
    V 9
    mypassword

The above commands and locations are for a Linux box, but something similar should work on other OSes.

Wednesday, November 5, 2014

Spock and Objenesis: Resolving IllegalAccessError

Spock and Objenesis: Resolving IllegalAccessError

When mocking some classes with Spock in one of the usual ways, we often may need to include cglib. We also have to add objensis as well.

In one our modules, the latter was removed, leading to the below type of error that did not directly point out the cause:

java.lang.IllegalAccessError: tried to access method org.company.service.export.ExportRequest.<init>(Lorg/company/service/export/ExportRequest$Builder;)V from class org.company.service.export.ExportRequest$$EnhancerByCGLIB$$1969beb7
 at org.spockframework.mock.runtime.MockInstantiator.instantiate(MockInstantiator.java:33)
 at org.spockframework.mock.runtime.ProxyBasedMockFactory$CglibMockFactory.createMock(ProxyBasedMockFactory.java:92)
 at org.spockframework.mock.runtime.ProxyBasedMockFactory.create(ProxyBasedMockFactory.java:49)
 at org.spockframework.mock.runtime.JavaMockFactory.create(JavaMockFactory.java:51)
 at org.spockframework.mock.runtime.CompositeMockFactory.create(CompositeMockFactory.java:44)
 at org.spockframework.lang.SpecInternals.createMock(SpecInternals.java:47)
 at org.spockframework.lang.SpecInternals.createMockImpl(SpecInternals.java:282)
 at org.spockframework.lang.SpecInternals.MockImpl(SpecInternals.java:83)
 at org.company.service.export.RecordExporterSpec.$spock_initializeFields(RecordExporterSpec.groovy:141)

For our mocking, we would define the mock at the field level like:

    ExportRequest request = Mock()

After adding objenesis, the error was resolved:

    <dependency>
        <groupId>cglib</groupId>
        <artifactId>cglib-nodep</artifactId>
        <version>2.2</version>
    </dependency>
    <dependency>
        <groupId>org.objenesis</groupId>
        <artifactId>objenesis</artifactId>
        <version>1.2</version>
    </dependency>

Saturday, November 1, 2014

Sending Email with Spring

Sending Email with Spring

At work, we were using an in-house dependnecy to send text-based internal emails out the door of one our webapps for notifications. The solution worked will for us, but did more than just send emails. As a result, depending on it introduced a large dependency tree to our application.

Since the artifact simply appeared to be a wrapper around Spring, which we were using directly anyway, we thought it might be better to write our own little solution instead, and simplify our application's dependency graph.

Below I'll describe some "highlights". A full project example can be found here: spring-mail-example

In our business processes, we send notifications to same email address and from the same sender. The only thing that may change is the subject of the email (success vs. failure) and the contents of the email. So we wanted a simple interface that just took both these parameters:

interface Sender {
    void send(String subject, String text)
}

Below is an implementation of the interface that:

  • Sends a simple MimeMessage with type text/html.
  • Expects the sender and recipient email address to be injected.
  • Expects the name and port of the mail host to use to be injected.
  • Creates an instance of JavaMailSender to help simplify configuration for the user.

The code:

class EmailSender implements Sender, InitializingBean {

    String mailHost
    int mailPort
    String fromAddress
    String toAddress

    protected JavaMailSender mailSender

    @Override
    void send(String subject, String text) {
        try {
            mailSender.send htmlMessageWith(subject, text)
        }
        catch (e) {
            logFailure e
            throw new MessageException(e)
        }

        logSent toAddress, subject, text
    }

    private MimeMessage htmlMessageWith(subject, text) {
        def mimeMessage = new MimeMessage(null)
        def messageHelper = new MimeMessageHelper(mimeMessage)

        messageHelper.with {
            addTo toAddress
            setFrom fromAddress
            setSubject subject
            setText text, true
        }

        messageHelper.mimeMessage
    }

    @Override
    void afterPropertiesSet() throws Exception {
        mailSender = new JavaMailSenderImpl()
        mailSender.setHost mailHost
        mailSender.setPort mailPort
    }
}

An example of how to configure the implementation with Spring:

    <bean class="prystasj.spring.mail.EmailSender">
       <property name="fromAddress" value="prystasj@company.org"/>
       <property name="toAddress" value="friend@company.org"/>
       <property name="mailHost" value="mailhost.company.org"/>
       <property name="mailPort" value="25"/>
   </bean>

In the end, we decided to go with this solution for our emails. While we now have code of our own to maintain, the resulting dependency tree is smaller, and we have control over adding any features we want, so for now the trade-off is worth it.

Sunday, September 21, 2014

Groovy: Import Aliases

A feature I like in Groovy is the ability to import class with aliases. I had forgotten about this feature, so I'm writing about it here to help get it to stick with me a bit.

Take this Spock test class tha has to define a couple several values of a couple types:

    import org.company.id.CorporateSymbol
    import org.company.id.CorporateName

    class CorporationMapperSpec extends Specification {

        @Subject
        CorporationMapper mapper = new CorporationMapper()

        def 'maps corporate symbols to names'() {
            expect:
            symbolsToNames == mapper.map(symbols)
        }

        def symbol1 = new CorporateSymbol('KRA')
        def symbol2 = new CorporateSymbol('VLY'),
        def symbll3 = new CorporateSymbol('KRS'),

        def symbolsToNames = [
            (symbol1): new CorporateName('Kramerica')
            (symbol2): new CorporateName('Vandalay')
            (symbol3): new CorporateName('Kruger')
        }
    }

We can use aliases with the import statements to make the code easier to work with:

    import org.company.id.CorporateSymbol as CS
    import org.company.id.CorporateName as CN

    class CorporationMapperSpec extends Specification {

        @Subject
        CorporationMapper mapper = new CorporationMapper()

        def 'maps corporate symbols to names'() {
            expect:
            symbolsToNames == mapper.map(symbols)
        }

        def symbol1 = new CS('KRA')
        def symbol2 = new CS('VLY'),
        def symbll3 = new CS('KRS'),

        def symbolsToNames = [
            (symbol1): new CN('Kramerica')
            (symbol2): new CN('Vandalay')
            (symbol3): new CN('Kruger')
        }
    }

I would be a little wary about using it in main source, as it does add a little bit of indirection to the readability of the code, but for test source, I think it can definitely help making the code easier to read and work with.

Sunday, September 7, 2014

Jackrabbit and XPath Queries: Escaping Paths

Jackrabbit and XPath Queries: Escaping Paths

We have a service that manages a Apache Jackrabbit repository. The main client of the service builds lists of records of the form: ///, where user's are represented by a UUID. For example:

  /JCP/feeadeaf-1dae-427f-bf4e-842b07965a93/label/

Now we started to build a web endpoint to the service that can be used to view the contents of the repository at any time. This endpoint may want to create a query into the repository along the lines of "show me all lists for institution JCP and user feeadeaf-1dae-427f-bf4e-842b07965a93". When we create lists in the repository, we increment a property named sequence (more on this on a later date), so an XPath query that proved to work for us given the above example proved to be:

  /*/JCP/feeadeaf-1dae-427f-bf4e-842b07965a93/label//*[@sequence]

This was working well at first until we started to execute queries where the UUID had a leading digit. We would see an exception in our logs of the form:

  Encountered "-" at line 1, column 26.
  Was expecting one of:
   ...
   ...
   ...
   ...
  ...
     for statement: for $v in /*/JCP/2eeadeaf-1dae-427f-bf4e-842b07965a93/label//*[@sequence] return $v

Since the hypen was indicated as the culprit by the exception message, and given the fact we only ran into this when the UUID node began with a digit, we thought that both conditions were required to create the invalid query. Our first attempt at a solution simply invovled prefixing all nodes that fit the pattern of a UUID with a string like uuid_.

In reality, it was simply the leading digit that caused the problem. The above solution would definitely not work for all future use cases of the service. The problem can simply be stated that the query was invalid as XML nodes cannot start with digits. We could still however create paths into the repository in the above manner as long as we escaped the leading digit in the query:

  /*/JCP/_x0032_eeadeaf-1dae-427f-bf4e-842b07965a93//*[@sequence]

The code below demonstrates our modeling of a path into the repository and the method by which we perform the escaping. Before that though it is worth noting that we had to encode the individual steps when building the path included in the query. If we included the whole path in the escaping logic, the delimiting slashes would be escaped, and the query would not work. If we included the query as a whole into the escaping logic, the asterisks would be escaped, and the query would not also work.

    import org.apache.commons.lang3.StringUtils;
    import org.apache.jackrabbit.util.ISO9075;

    class Path {
        List<String> steps; //...

        public String asQuery() {
            return steps.size() > 0 ? "/*" + asPathString(encodedSteps()) + "//*" : "//*";
        }

        private String asPathString(List<String> steps) {
            return '/' + StringUtils.join(steps, '/');
        }

        private List<String> encodedSteps() {
            List<String> encodedSteps = new ArrayList<>();
            for (String step : steps) {
                encodedSteps.add(ISO9075.encode(step));
            }
            return encodedSteps;
        }
    }

The ISO9075 class can be found in org.apache.jackrabbit:jackrabbit-jcr-commons.

My initial search for answers started on Stack Overflow, and of course the good folks answering questions there didn't let me down!

Saturday, August 30, 2014

Using Maven to Detect Duplicate Classes

Using Maven to Detect Duplicate Classes

As the number of dependencies from some our corporate project grew, we ran into a couple cases where we would be transitively pulling into different versions of the same classes. For a long time, we were lucky, as this did not cause any issues, until someday a developer new to our team introduced trying to run our application using a different operating system. The JVM on his machine loaded up the classes in a different order, and his application instance used an older version of a class that resulted in errors running the application.

This developer thankfully pointed out we can detect when we were pulling in different versions of classes using the Maven Enforcer plugin with some help of some extra rules. We added something like this to our parent POM:

    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-enforcer-plugin</artifactId>
        <version>1.3.1</version>
        <executions>
            <execution>
                <id>enforce</id>
                <phase>validate</phase>
                <configuration>
                    <rules>
                        <banDuplicateClasses>
                            <ignoreClasses>
                                <ignoreClass>javax.*</ignoreClass>
                                <ignoreClass>org.junit.*</ignoreClass>
                                <ignoreClass>org.hamcrest.*</ignoreClass>
                                <ignoreClass>org.slf4j.*</ignoreClass>
                            </ignoreClasses>
                            <findAllDuplicates>true</findAllDuplicates>
                        </banDuplicateClasses>
                    </rules>
                    <fail>false</fail>
                </configuration>
                <goals>
                    <goal>enforce</goal>
                </goals>
            </execution>
        </executions>
        <dependencies>
            <dependency>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>extra-enforcer-rules</artifactId>
                <version>1.0-beta-2</version>
            </dependency>
        </dependencies>
    </plugin>

Example output from the validate phase of a build:

    [INFO] --- maven-enforcer-plugin:1.1:enforce (enforce) @ cataloging-api ---
    [INFO] Adding ignore: javax.*
    [INFO] Adding ignore: org.junit.*
    [INFO] Adding ignore: org.hamcrest.*
    [INFO] Adding ignore: org.slf4j.*
    [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BanDuplicateClasses failed with message:
    Duplicate classes found:

      Found in:
        org.company.id.api:id-api:jar:1.28.0:compile
        org.compnay.legacy.id-api:id-common-api:jar:2.8.0:compile
      Duplicate classes:
        org/company/service/id/SAMLEntityId.class
        org/company/service/id/crypto/ServiceKey.class

We were surprised at the amount of duplicate classes we were bringing in. We thought our dependency management was in pretty good shape, but adding the new rules showed we still had some work to do.

It is also worth noting, while our project is large, adding the plugin execution to the build added an insignifcat amount of time to the build overall, so there should be no performance concerns if you were considering adding something similar to your project.

Saturday, August 9, 2014

Spring AOP: Excluding Methods in Pointcuts

We use Spring AOP to make use of aspects in our applications for the purpose of recording statistics about requests they receive. This approach has worked out well because it helps separate the concern of tracking statistics from our business logic. Previous iterations of our applications mixed the two, leaing to code that was generally harder to maintain.

Below is an example of how we might configure an aspect to target all entry points into our business application with the same aspect:

    <aop:config>
        <aop:pointcut id="requestPointcut"
                      expression="execution(public * org.company.service.*.*(..))"/>
        <aop:aspect id="requestAspect" ref="requestStatAspect">
            <aop:around method="around" pointcut-ref="requestPointcut"/>
        </aop:aspect>
    </aop:config>

    <bean id="requestAspect" class="org.company.service.aop.RequestIdentifierAspect" lazy-init="true"/>

With the above, we are saying we want request identifiers managed for every request to our interfaces. With the wildcards in the expression, we are saying target every class in the org.mycompany.myapp.service package and every method in those classes.

This approach worked greatly until we added an interface that provided health reporting for system monitoring purposes. This method lived in a new interface. Let's suppose added the interface method is defined as:

  • public boolean org.company.HealthService.isHealthy()
    

Since the method signature is different (it takes no arguments), running the advice in the aspect would lead to errors as the aspect evaluated the method arguments. We could change the code in the aspect to avoid processing if the arguments contained within the join point do not match what we are expecting. This approach is not that attractive because it one, involves changing code for special cases, and two, could lead to the aspect siltently not invoking its main logic at times we might not expect. If we target a method with the aspect unintentionally, we want to be informed by errors in our log.

An alternative we found was to exclude the new method from the pointcut expression:

  • execution(public * org.company.service.*.*(..)) &amp;&amp; !execution(* org.company.service.HealthService.*(..))
    

Just with adding special case logic to an aspect or code in general to avoid exception cases, we have to be careful that the exceptional cases we avoid targeting with the aspect does not get out of hand as we are tempted to add more in the future.

Saturday, June 28, 2014

Sending data to a server with Groovy

The Groovy language provides a lot of additions to existing Java classes that can make our jobs easier.

A couple of them we were able to take advantage of lately that use closures to make our coding simpler were:

Our use case: We have a service whose responsibility is to send "records" over a TCP/IP connection to a remote server.

Of course we want to include a test for this in our integration tests, but how can we do it? We cannot expect a client to leave a server available open to receive records sent by our tests. We could create our test server to accept and output the sent record.

We'll write a test server using Groovy that can be started at the beginning of the test run and shutdown at the end:

class TestServer {

    static defaultPort = 6666

    static ServerSocket start_on_default_port() {
        start_on defaultPort
    }

    static ServerSocket start_on(port) {

        def server = new ServerSocket(port)
        String receivedRecord ''
        def done = false

        Thread.start { t ->
            while (!done) {
                server.accept { socket ->
                    socket.withStreams { input, output ->
                        done = false

                        while (!done) {
                            def block = new byte[128]
                            int size = input.read block

                            if (size == -1) done = true // there is nothing left to have been received

                            block = trim(block as List)
                            receivedRecord += new String(block)
                        }
                    }
                }
            }
        }
        server
    }

    /** Remove any empty or non-set bytes at the end of the array. */
    private static def trim(List block) {
        block.reverse().findAll { b -> b != 0 }.reverse() as byte[]
    }
}

In our integration tests class, written with Spock, we start and stop the server at the beginning of the run:

@ContextConfiguration
class RecordDeliveryServiceIntegrationSpec extends Specification {

    @Autowired
    RecordDeliveryService service

    static def server

    def setupSpec() {
        server = TestServer.start_on_default_port()
    }

    def cleanupSpec() {
        server.close()
    }

    def 'delivers a record to a server'() {
        expect:
        expectedResponse == service.send 'localhost', TestServier.defaultPort, record
    }

    //...
}

For reference to complete the example, here's what a simplified service implementation could look like, using a methodology similar to that the test server uses to accept the record:

dclass RecordDeliveryService implements DeliveryService {

    public def send(String host, long port, Record record) {

        def socket = new Socket(host, port)

        socket.withStreams { input, output -&gt;
            output &lt;&lt; record.toString()
            output.flush()
            output.close()
        }

        socket.close()
    }
}   //...
}

If you can think of any improvements, or a better approach to the problem, I would be glad to hear from you!

Maven: Failing the Build After Running All Modules

Maven: Failing the Build After Running All Modules

Here's a quick one to help me feel a better about not posting anything for quite a while.

In our large project at work that contains several deployed services, we have several parent integration testing Maven modules that run after our nightly development install. Each has several child modules that each run tests against a particular endpoint in each service.

So naturally, we want our build server to kick off each parent testing module after the install. The problem we ran into is when that one of the earlier child modules failed, the build will stop, without running the rest of the modules (Maven's default behavior).

This would often lead to us making changes to get the tests in the failing module to pass, then re-running the testing build, only to have a later child module to fail. It would be nice to have had all the modules to run, reporting all failures, so that all the required changes and fixes could be made up front, saving us time.

Fortunately, Maven provides an option we learned about do just that:

-fae,--fail-at-end     Only fail the build afterwards;
                       allow all non-impacted builds to continue

There are also a couple more failure related options as well, but we have yet to find any use for them (yet):

-ff,--fail-fast        Stop at first failure in reactorized builds
-fn,--fail-never       NEVER fail the build, regardless of project result

When working on the tests locally, we can pick up the build from within the parent from the point of the failed chilid module after making changes to our running service to fix the test with:

-rf,--resume-from      Resume reactor from specified project

For example:

$ mvn verify -rf :service-integration-testing-records

For more on working with multiple modules, see: Guide to Working with Multiple Modules.