Since I wrote my previous post on how to use Play! Scala with Jerkson, I was asked a number of times how to make use of custom de/serialization for classes. This happens when e.g. the case class you want to de/serialize has a POJO or another complex type that you’d like to have more control over, such as e.g. the BSON ObjectId or Play’s Anorm PK.

Here’s a wrap-up on how you can easily create custom de/serializers and use them within your Play! Scala application. Let’s take the example of the BSON ObjectId, which you will most likely run into if you use MongoDB in the backend.

First, we need to create a Jackon serializer and de-serializer for the org.bson.types.ObjectId type:

import org.codehaus.jackson.map.annotate.JsonCachable
import org.bson.types.ObjectId
import org.codehaus.jackson.map.{DeserializationContext, JsonDeserializer, SerializerProvider, JsonSerializer}
import org.codehaus.jackson.{Version, JsonParser, JsonGenerator}

@JsonCachable
class ObjectIdSerializer extends JsonSerializer[ObjectId] {
  def serialize(id: ObjectId, json: JsonGenerator, provider: SerializerProvider) {
    json.writeString(id.toString)
  }
}

class ObjectIdDeserializer extends JsonDeserializer[ObjectId] {
  def deserialize(jp: JsonParser, context: DeserializationContext) = {
    if (!ObjectId.isValid(jp.getText)) throw context.mappingException("invalid ObjectId " + jp.getText)
    new ObjectId(jp.getText)
  }
}

This is pretty straightforward: when serializing, we just print out the string representation of the ObjectId via toString(), and when deserializing we check if we have a valid ObjectId and then create an instance given the text representation (or throw an exception if we don’t get a valid one).

Now all what is left to do is to register this de/serializer couple. For that purpose we can extend Jerkson’s Json trait that gives us access to the mapper and register a new Jackson SimpleModule:

import org.codehaus.jackson.map.module.SimpleModule

object CustomJson extends com.codahale.jerkson.Json {
  val module = new SimpleModule("CustomJson", Version.unknownVersion())
  module.addSerializer(classOf[ObjectId], new ObjectIdSerializer)
  module.addDeserializer(classOf[ObjectId], new ObjectIdDeserializer)
  mapper.registerModule(module)
}

And that’s all that is needed. If you are using a custom action in Play! in order to generate Json, make sure to update it so that it uses the CustomJson object we just created:

trait JsonAction { self: Controller =>
  def Json(data: AnyRef): Result = new Result() {
    def apply(request: Request, response: Response) {
      val encoding = getEncoding
      setContentTypeIfNotSet(response, "application/json; charset=" + encoding)
      response.out.write(CustomJson.generate(data).getBytes(encoding))
    }
  }
}

This morning I spent some time looking for information on how to deploy a legacy webservice on Apache Karaf.

This post isn’t about Karaf and OSGi and webservices though, but a reflection on how to efficiently find information that can lead towards a specific technical problem on the Internet. For quite some time now I feel that this task has become much more complicated in the past 2-3 years than it used to.

We are now in 2011 and at least in my perception, Google isn’t anymore a simple, efficient nor even effective way of finding information regarding specific problems. Which is an euphemism for “Google really sucks these days”. Type in a query, click on a link, search again for something rather different and you’ll see one of the results you clicked earlier appear in the results of your new query (which is related to the previous query by maybe only one keyword, two at most. What’s worse is that in many cases, the page that Google lists as result don’t even contain one of the keywords entered in the query, but have something to do with sites I browsed in the past. Seriously Google, why should I care about things I searched for in the past in a new query?

And yet, especially when it comes to software development, finding sound information about a problem is critical. Way too often I find myself rushing to Google and searching for a solution, then finding some random piece of information and trying to get something out of it – i.e. starting to work on a solution on a shaky ground, even though I perfectly know I’m on shaky grounds (intuition is very helpful there).

So there’s this one issue of finding high-quality information. There’s another, much bigger problem though: understanding the problem to look for the right solution. Oftentimes do I rush into looking for a solution when I haven’t yet understood the problem (or think I do, but really am just headed in the plain wrong direction).

At this point, I shall point to The Five Orders of Ignorance by Phillip G. Armour. If you haven’t read it and are in the business of writing software, do so. Now.

For the sake of the completeness of this post, I’ll list the Five Orders anyhow. They go like this:

• 0th Order – Lack of Ignorance: “I have 0OI when I provably know something”
• 1st Order – Lack of Knowledge: “I have 1OI when I don’t know something”
• 2nd Order – Lack of Awareness: “I have 2OI when I don’t know that I don’t know something”
• 3rd Order – Lack of a Suitably Efficient Process: “I have 3OI when I don’t have a suitably efficient way to find out that I don’t know that I don’t know something”
• 4th Order – Meta Ignorance: “I have 4OI when I don’t know about the Five Orders of Ignorance”

So how does that relate to the information search issue from before? Well, as I pointed out, oftentimes a problem when looking for information is the lack of clear understanding of the problem I try to solve. So this is – as far as I understood the 5 Orders – a 2OI: I lack the awareness that I do not understand (know) the problem for which I am looking for a solution.

So how to get from 2OI to 1OI? Of course this is situation dependent, but I would say that when you have some domain knowledge things are much easier. In software development, there’s one type of problems which I call “configuration problems” (although they could probably also be called “integration problems”) and that go like this:

• how do I deploy…
• how can I configure…
• I need to integrate…

and involve one or more technologies of the many, many technologies that are available in the open-source space.

Whenever I get to one of these questions (which appear to be 1OI-type-of-questions), I try to step back and ask myself: does that even make sense? Isn’t there a deeper problem I actually want / need to solve? I say that I “try to” step back because in some cases I either forget and rush into finding a solution to the wrong problem, or I know or feel that this is not the right question, but have a personal itch to get things to work anyway (“I really want Spring DM 2 to work on Karaf! It just has to work!”).

Now let’s go back to the initial issue described in this post (Google sucks): how to find answers to technical questions on the Internet these days?

When I think of not using Google, I am now trying the following:

• if the problem relates to a specific technology, I try to find a mailing-list and its archive and look there first. It may just be a POMO (Plain Old Mailing-List) but I found that oftentimes, you can get good answers just by looking at the archives (and by archives I do not mean Googling for the archives but browsing through them or searching only the archive site with Google’s “site” keyword)
• search and ask on Stackoverflow
• subscribe to a technology mailing-list and send the question there, to the developers and experienced users. I used to avoid this step for some time, wanting to solve things all by myself…but really it is not worth the trouble.
• check out the damn code. There is a lot in the code, much more than you’d think. And looking into the internals can sometimes be much more efficient than searching for a sample or documentation – let’s be honest, not too many projects have always up-to-date documentation and samples.

Allright, that’s it. Let’s hope Google fixes their search engine, or offers a paying high-quality service for search (I would pay for that).

In a project I’m currently working on we use the OSGi platform in order to achieve higher modularity (the project is about building a framework for massive data ingestion and processing for a high-availability service, and the processing is dealt with via workflows that are composed of a number of plugins that we don’t develop, hence the need for modularity).

We use the Apache Karaf framework, which offers neat extension points and means to customize it. One of the nice features is the WebConsole that can also be extended, which would provide the means to add a GUI for the users. However, this happens through providing a Servlet implementation, which isn’t really handy these days when wanting to develop a bit more complicated GUIs.

Since I’ve got some experience with Google Web Toolkit I thought of using this instead for coding the GUI (and eventually integrate it with the WebConsole later on, not sure at which level of depth yet). And the first thing necessary in order to do this is to get the GWT application to run on the Karaf platform.

Here’s how I managed to get it work:

Step 1: create a vanilla GWT application with the gwt-maven-plugin

I used the gwt-maven-plugin archetype in order to create a new GWT module. At the time at which I am writing this post, I had some issues with the unit tests that are generated by default, so I just trashed them for the moment.

Step 2: make the GWT application ready for deployment via Pax Exam Runner

Karaf can leverage Pax Runner in order to deploy OSGified WARs. For that, you need to have a MANIFEST.MF generated and attached to the WAR maven generates. There’s two things required for this.

First, configure the Apache Felix maven-bundle-plugin to generate the manifest for you (you could do it by hand, but why bother):

            <plugin>
                <groupId>org.apache.felix</groupId>
                <artifactId>maven-bundle-plugin</artifactId>
                <version>2.1.0</version>
                <extensions>true</extensions>
                <executions>
                    <execution>
                        <id>bundle-manifest</id>
                        <phase>process-classes</phase>
                        <goals>
                            <goal>manifest</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <supportedProjectTypes>
                        <supportedProjectType>jar</supportedProjectType>
                        <supportedProjectType>bundle</supportedProjectType>
                        <supportedProjectType>war</supportedProjectType>
                    </supportedProjectTypes>
                    <instructions>
                        <Bundle-ClassPath>.,WEB-INF/classes</Bundle-ClassPath>
                        <Embed-Directory>WEB-INF/lib</Embed-Directory>
                        <Embed-Dependency>*;scope=compile|runtime</Embed-Dependency>
                        <Embed-Transitive>true</Embed-Transitive>
                        <Import-Package>*;resolution:=optional</Import-Package>
                    </instructions>
                </configuration>
            </plugin>

This will generate a manifest which takes into account the required dependencies that come with the WAR, so that it can be deployed by Pax Runner. Note that this isn’t just a configuration for GWT itself, but in theory for any kind of WAR.

Update: I figured out how to customize the webapp name, by adding the following in the manifest instructions:

                        <!-- OSGi spec config option -->
                        <Web-ContextPath>gui</Web-ContextPath>
                        <!-- Pax Runner knows this option -->
                        <Webapp-Context>gui</Webapp-Context>

Pax Runner does not understand the OSGi spec <Web-ContextPath> instruction, so it needs to be given the <Webapp-Context> instruction.

Next, you need to tell the maven-war-plugin to add the generated manifest file to the WAR:

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-war-plugin</artifactId>
                ...
                <configuration>
                    ...
                    <archive>
                        <!-- add the generated manifest to the war -->
                        <manifestFile>${project.build.outputDirectory}/META-INF/MANIFEST.MF</manifestFile>
                    </archive>
                </configuration>
                ...
            </plugin>

Step 3: Activate war deployment support on Karaf

Fire up the Karaf console and type:

features:install war

This will activate everything needed in order to deploy WARs on the platform.

Step 4: Deploy the WAR

Now you just need to deploy the WAR on Karaf. You can do this in several ways:

• by copying the WAR file to the deploy/ directory of the Karaf platform
• by defining a feature descriptor and adding the war artifact to it

Since we already had a feature descriptor (because we have many bundles that need to be deployed, I just added the artifact to the feature descriptor XML like this:

<?xml version="1.0" encoding="UTF-8"?>
<features name="uim-features">
    <feature name="uim-core" version="1.0.0.SNAPSHOT">
    ...
               <bundle>mvn:eu.europeana/europeana-uim-gui-gwt/1.0.0-SNAPSHOT/war</bundle>
    ...
    </feature>
</features>

Note that you need to explicitly reference the “war” classifier, or this won’t work. Now, re-install the feature in order for the changes to take effect.

Step 5: Enjoy

You should see something like this (amongst others) in the log:

14:42:50,487 | INFO  | l Console Thread | WebXmlObserver
| nder.war.internal.WebXmlObserver  128
| 581 - org.ops4j.pax.web.pax-web-extender-war - 0.7.3
| Using [gui] as web application context name

That’s it, you can access your GWT module at http://localhost:8181/<web app name>

I recently started using IntelliJ IDEA in addition to Eclipse after seeing it being used by some dutch colleagues of mine. And I must say that I do like it more and more every day — it just has all these many little things that fill so well to Java.

One thing I do miss though is the ability to debug ANT tasks. In Eclipse, you can make an external launcher, select one or more targets, and run them in debug mode. With IDEA, that just doesn’t seem to work. I did install the ANT debugger plugin which makes it possible to debug through the build file, but that’s not still what I wanted.

So here is a workaround I found, until the folks at JetBrains or HandyEdit will figure it out: make a launch configuration that directly invokes ANT:

• Main class:

  org.apache.tools.ant.launch.Launcher

• Program parameters:

  -buildfile your-build-file.xml your.target

Et voilà, now you should be able to debug your ANT task execution.