Monday, May 16, 2011

Using a Tomcat Engine JMX Proxy Bean With Spring

Recently, I needed to determine which Tomcat server in a load balanced environment an application was running on. One way to to do this is to look at the jvmRoute property of the running Tomcat Engine. If you're in a load balanced environment you'll probably see the jvmRoute appended to the session id. What I wanted to do was to get that value directly, without any reference to a session. Here's how I did it.

One quick note before I begin - obviously, this ties your application to Tomcat so you might want to look at ways of abstracting this out if server portability is a concern.

The first thing you need to do is make sure that a jvmRoute value is being set. This is as simple as adding something like -DjvmRoute=jvm1 to your VM arguments.

Next up is adding a MBeanProxyFactoryBean to your application context. To inject a proxy for the Engine, you need


<bean id="tomcatEngineProxy" class="org.springframework.jmx.access.MBeanProxyFactoryBean">
<property name="objectName" value="Catalina:type=Engine" />
<property name="proxyInterface" value="org.apache.catalina.Engine" />
<property name="useStrictCasing" value="false" />
</bean>


You may or may not need to play about with the objectName value depending on how you have configured your Engine in server.xml - have a look here.

Setting useStrictCasing to false is especially important. For some reason Spring's default behaviour is to set useStrictCasing to true. What this means is that when you access a property via a getter (in our case, getJvmRoute) causing the proxy to ask for the property value over JMX, by default it ignores normal Javabean style conventions and asks for the "JvmRoute" property. Apache is expecting to be asked for the "jvmRoute" property and you'll be on the receiving end of a "cannot find property" error. Setting useStrictCasing to false makes Spring ask for "jvmRoute" instead. I'd really like to know why useStrictCasing is set to true by default, as it seems to violate the principle of least surprise.

Now that we have a proxy for the Tomcat Engine in the context, we can autowire it where it's needed -

@Autowired private Engine engine;


and getting the jvmRoute is as simple as

engine.getJvmRoute()


Thanks go out to Joachim Sauer on stackoverflow for pointing me in the right direction.

Thursday, August 26, 2010

Google App Inventor and node.js

My invite to Google's new, easy to use App Inventor for Android came through recently and within minutes I had hello world running. Even better, after a few more minutes of playing with the programming toybox I had an easy to use tool.

For awhile now I've wanted a simple tool that would let me go around scanning the barcodes of books that I own and saving them into a text file for importing into LibraryThing but I haven't had the spare time to learn enough Android programming to build one. A combination of a simple client built with App Inventor and a simple server written with Node and I had the tool I've been wanting in a matter of minutes.

The AppInventor Client

AppInventor comes in two parts - an "Interface Builder" website and a visual coding environment where you connect up blocks of code (that sits on top of the Kawa dialect of Scheme - which makes App Inventor coding a lot like a combintation of Lego and Lisp). I quickly added a label (to display the scanned ISBN number), a button (to start the scan) and then two non-visible components: a barcode scanner and a TinyWebDB. Once the UI was done it was off to the coding environment to wire everything up.

The only not quite straightforward part of all this is the TinyWebDB component as it's App Inventor's only built in way of accesing a web service (other than Twitter, which has it's own component). However, TinyWebDB isn't a generic web service client, it's meant for storing and retreiving simple key/value pairs so I needed to write my server with that in mind.

As the coding environment is visual, it really doesn't lend itself to a blog post. Fortunately, you can download the app source code in a zip file and marvel at the underlying code. The meat of the app is:


;;; Screen1
(do-after-form-creation (set-and-coerce-property! Screen1 'Title "Screen1" 'text)
)
;;; Label1
(add-component Screen1 Label Label1 (set-and-coerce-property! Label1 'Text "Text for Label1" 'text)
)
;;; Button1
(add-component Screen1 Button Button1 (set-and-coerce-property! Button1 'Text "Scan" 'text)
)
(define-event Button1 Click()
(call-component-method 'BarcodeScanner1 'DoScan (list)
*no-coercion*)

)
;;; BarcodeScanner1
(add-component Screen1 BarcodeScanner BarcodeScanner1 )
(define-event BarcodeScanner1 AfterScan( result )
(call-component-method 'TinyWebDB1 'StoreValue (list "isbn" (get-property BarcodeScanner1 Result)
)
'( text any)
)

(set-and-coerce-property! Label1 'Text (get-property BarcodeScanner1 Result)
'text)

)
;;; TinyWebDB1
(add-component Screen1 TinyWebDB TinyWebDB1 (set-and-coerce-property! TinyWebDB1 'ServiceURL "http://192.168.1.101:8765" 'text)
)
(init-runtime #f)

While that's not quite the entire source, it does give some idea of what goes on underneath.

Node(.js)

TinyWebDB will post a key/value pair to whatever server you want. So I needed a simple server that would sit there, extract the value and output to stdout (which will then get piped into a file). This seemed like the perfect opportunity to try Node (I can't tell if it's supposed to be called node or node.js), which is a Javascript execution environment based on the V8 engine for writing network programs. A quick and dirty Javascript hack produced:

var http = require('http');
http.createServer(function (request, response) {
request.on('data', function(chunk) {
var b = chunk.toString('ascii', 0, chunk.length);
var v = b.split("&")[1];
var vv = v.split("=")[1];
var vvv = vv.replace("%22", "");
var vvvv = vvv.replace("%22", "");
console.log(vvvv);
});
response.writeHead(200, {'Content-Type': 'text/plain'});
response.end('Hello, World\n');
}).listen(8765);


Undoubtedly a regexp could have been used, but would have required more mental effort than I really wanted. In case you're wondering about the response, TinyWebDB apparently essentially disregards it, so sending back Hello, World doesn't hurt.

Conclusion

In a matter of minutes I had a useful (at least to me) tool. I have a feeling that this is what App Inventor will mostly be used for - simple apps that satisfy a single person's needs. It's not going cause the "Animated Cat App Apocalypse" in the Android Market that some commentators have worried about. I have no idea if it will be of any use as the "Logo for the modern age" that was the original reason for it's existence (there are still references to "Young Android" in the generated code and the application icons are of an adult and child robot). I would call it the "Visual Basic for the modern age" except that it feels wrong comparing something that "is part of an ongoing movement in computers and education that began with the work of Seymour Papert and the MIT Logo Group in the 1960s" with Visual Basic.

Tuesday, February 9, 2010

Using Scala To Configure Spring's DispatcherServlet

After reading Francois Armand's post about "Scala and Spring 3 JavaConfig", I was inspired to try and use Scala to replace some servletname-servlet.xml Spring config files.

I have a small Scala/Spring MVC project that at the moment uses two DispatcherServlets, one for each controller. There was a historical reason for this (I needed two different Velocity configurations) and while I can now probably get away with a single DispatcherServlet, what I'm going to try to do is move one of the servlets over to a Scala based configuration and leave one using xml based configuration.

Of the two configurations, I'm going to move the simplest one over. It does nothing more than define a root controller bean. So the Scala config should be as simple as


@Configuration
class CalendarConfig {
@Bean def rootController = new CalendarController()
}


And a quick change to my web.xml

...
<servlet>
<servlet-name>calendars</servlet-name>
<servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
<load-on-startup>1</load-on-startup>
<!-- Configure DispatcherServlet to use JavaConfigWebApplicationContext
instead of the default XmlWebApplicationContext -->
<init-param>
<param-name>contextClass</param-name>
<param-value>
org.springframework.web.context.support.AnnotationConfigWebApplicationContext
</param-value>
</init-param>
<!-- Again, config locations must consist of one or more comma- or space-delimited
and fully-qualified @Configuration classes -->
<init-param>
<param-name>contextConfigLocation</param-name>
<param-value>com.maethorechannen.calendars.CalendarConfig</param-value>
</init-param>
</servlet>
...



I started my app up in my local development Google App Engine server and it failed to start up, thanks to "java.lang.IllegalStateException: CGLIB is required to process @Configuration classes. Either add CGLIB to the classpath or remove the following @Configuration bean definitions: [calendarConfig]". I added cglib-nodep-2.2 (available from here if you're not using something like maven or ivy) to my library of jars and it started to work.

In case anyone's interested - I pushed it to App Engine and it still worked (occasionally, what works locally doesn't work when on the Big G's servers). One thing I'm really interested in seeing is what (if any) sort of impact on startup times programmatic configuration has compared to XML configuration. When using GAE every millisecond of startup time counts and I would think Scala based configuration should be quicker.


What's interesting to note (at least to me) is that I didn't need to create a global WebApplicationContext or move everything over to Java(Scala) based configuration, which is something I wasn't sure about after reading the documentation from SpringSource.

Scala + Spring + Templates - Velocity

Next up on my list of template engines to try with Scala & Spring - Velocity. Unlike FreeMarker, I had actual success in getting it to work. Unsurprisingly, "Native" Scala object support needed to be added in - fortunately the way Velocity was designed makes that easy.

Velocity uses introspection/reflection to access properties and methods on an object and this is where the Scala customizations fit in. The first thing to create is a subclass of UberspectImpl which does two things - return a java.util.Iterator for iterable objects and return a VelPropertyGet object for getting access to an object's properties. There is a third thing that could also be done - returning a VelPropertySet object to support setting properties from Velocity but I haven't implemented this as I don't need that functionality at the moment.

Getting a Java Iterator is fairly straightforward (with Scala 2.7.7 - I'm not sure what, if any, changes will be needed with 2.8)

override def getIterator(obj: java.lang.Object, i: Info): JavaIterator[_] = {
def makeJavaIterator(iter: Iterator[_]) = new JavaIterator[AnyRef] {
override def hasNext() = iter.hasNext
override def next() = iter.next().asInstanceOf[AnyRef]
override def remove() = throw new java.lang.UnsupportedOperationException("Remove not supported")
}

obj match {
case i: Iterable[_] => makeJavaIterator(i.elements)
case i: Iterator[_] => makeJavaIterator(i)
case _ => super.getIterator(obj, i)
}
}


The method to return a VelPropertyGet object is almost as simple - but it also shows that we need two more classes - one for accessing Scala properties and one for accessing Scala Map members like they're properties.


override def getPropertyGet(obj: java.lang.Object, identifier: String, i: Info): VelPropertyGet = {
if (obj != null) {
val claz = obj.getClass()

val executor = obj match {
case m: Map[_, _] => new ScalaMapGetExecutor(log, claz, identifier)
case _ => new ScalaPropertyExecutor(log, introspector, claz, identifier)

}

if (executor.isAlive) {
new VelGetterImpl(executor)
} else {
super.getPropertyGet(obj, identifier, i)
}
} else {
null
}
}

(You can get the full source code here)

To access Scala style properties I'll need a Scala specific subclass of PropertyExecutor. The code to access properties from Scala objects is once again trivial. In fact, because of the way Scala generates it's getter methods the code to access Scala properties is simpler than the code to access JavaBean style properties. (source)


override def discover(clazz: java.lang.Class[_], property: String) = {
val params = Array[java.lang.Object]()
setMethod(introspector.getMethod(clazz, property, params))
if(!isAlive()) {
super.discover(clazz, property)
}
}



Compare that to the code in the parent class


protected void discover(final Class clazz, final String property)
{
/*
* this is gross and linear, but it keeps it straightforward.
*/

try
{
Object [] params = {};

StringBuffer sb = new StringBuffer("get");
sb.append(property);

setMethod(introspector.getMethod(clazz, sb.toString(), params));

if (!isAlive())
{
/*
* now the convenience, flip the 1st character
*/

char c = sb.charAt(3);

if (Character.isLowerCase(c))
{
sb.setCharAt(3, Character.toUpperCase(c));
}
else
{
sb.setCharAt(3, Character.toLowerCase(c));
}

setMethod(introspector.getMethod(clazz, sb.toString(), params));
}
}
/**
* pass through application level runtime exceptions
*/
catch( RuntimeException e )
{
throw e;
}
catch(Exception e)
{
String msg = "Exception while looking for property getter for '" + property;
log.error(msg, e);
throw new VelocityException(msg, e);
}
}



It took me a little while to get Maps working (even though there's not much code to show for it).


class ScalaMapGetExecutor(val llog: Log, val clazz: java.lang.Class[_], val property: String) extends MapGetExecutor(llog, clazz, property) {
override def isAlive = true

override def execute(o: AnyRef) = o.asInstanceOf[Map[String, AnyRef]]
.getOrElse[AnyRef](property, null).asInstanceOf[java.lang.Object]

}


All that's left to do now is configure Velocity from Spring so that it uses the Scala specific Uberspector instead of the default. So, in my servletname-servlet.xml file I needed to add the following


<bean id="velocityConfig" class="org.springframework.web.servlet.view.velocity.VelocityConfigurer">
<property name="resourceLoaderPath" value="/WEB-INF/views/"/>
<property name="velocityProperties">
<props>
<prop key="runtime.introspector.uberspect">info.threebelow.verla.mvc.view.velocity.ScalaUberspect</prop>
</props>
</property>
</bean>

Saturday, January 23, 2010

Scala + Spring + Templates - the story so far

I've been writing a fairly simple web app with Scala and Spring that will be running on Google's App Engine. I've been trying to decide which template language (or even languages) to write my views in. I'll be rendering a lot of non-HTML and non-XML and over the years I've noticed some template engines are better at this than others. Also, I'm looking for first class Scala support (so anything you can do with a Java Bean you can do with a Scala object) - ideally built in and at worst trivial to add. Here's what I've come across so far:

Closure Templates

I like Google's Closure Template, but there's no "out of the box" support for it in Spring. If I can't find another template engine I like then I will come back and look at how to add some support.

FreeMarker

I will start at the point where my experiments ended - apparently FreeMarker doesn't currently work on GAE, as FreeMarker uses some swing classes that aren't available in GAE. This is a show stopper. I didn't run in to this during my experiments as it was working on the local dev server and I wasn't aware of this problem until I was submitting a bug for another problem I found.

During my experiments, I found several other issues, one of which calls into question using FreeMarker with Spring and Java, let alone Scala.

First, there isn't built in object wrapping for Scala objects. By default, FreeMarker will treat Scala objects as Java objects, but as plain Scala objects don't follow JavaBeans conventions access to getters ceases to be trivial. I was able to whip up a Scala specific FreeMarker Model and ObjectWrapper with relative ease, but then I ran into another problem - there doesn't appear to be an easy way in Spring to configure FreeMarker with an alternative ObjectWrapper.

The answer was to create a subclass of org.springframework.web.servlet.view.freemarker.FreeMarkerConfigurer and modifying one of the Configuration related methods to return a Configuration with an ObjectWrapper property set to my Scala specific ObjectWrapper. However, I then ran into a problem that I eventually traced to what appears to be a bug in FreeMarker. When Spring sends a model to FreeMarker it uses FreeMarker's AllHttpScopesHashModel - unfortunately AllHttpScopesHashModel's constructor doesn't let it's super class know about the ObjectWrapper that was passed in (it just stores it in a final variable that is private to AllHttpScopesHashModel). This means inherited methods fall back to using Fremarker's default ObjectWrapper - so Scala objects don't work. While working with the FreeMarker source code to make sure this is what was going wrong (it was), I ran into two more issues with FreeMarker.

As I was using the trunk from FreeMarker's SVN repository I assume I was using what will be the 2.4 release of FreeMarker. The first problem I ran into with this was that Spring's FreeMarkerView imports freemarker.core.ParseException, unfortunately ParseException has been moved to freemarker.core.parser.ParseException which obviously breaks Spring 3. As I said above, I have doubts about about using FreeMarker with Spring and any language - this is the reason why. At some point this backwards incompatible API change is going cause someone, somewhere a lot of pain and I don't want to be that person.

While mucking about with FreeMarker's source, I ran into another problem. FreeMarker 2.4 will have some new security features which seemed to cause no end of trouble with GAE. While you should be able turn these features off in the Configuration object, setting the secure property to false had no impact on getting FreeMarker to work with GAE. While I had pretty much given up on FreeMarker by this point, I still wanted to make sure that the reason why my ObjectWrapper wasn't working was because of AllHttpScopesHashModel's constructor so I commented out some security related lines in FreeMarker and eventually got it all working.


Summary

I won't be using FreeMarker. There are still a few more template engines to look at before I'll fall back to forcing Closure Templates to work with Spring (I would be astounded if it doesn't work GAE). So, I'll carry on my search (Velocity is probably next).

Tuesday, December 22, 2009

A Basic Ant+Ivy+Scala Setup

I've decided that for my latest project I'd give Ant with Ivy a go as my build system. To start with, I'm going to try for the simplest possible setup necessary in order to compile my project (so at this stage, it's not even rolling the output up into a jar - I can fix that later). At it's most basic level, this means that I'll need two XML files to get everything to build:

ivy.xml

ivy.xml contains information about the dependencies needed to build the project. To compile scala source, I'll need to add two dependencies - scala-library and scala-compiler.

<ivy-module version="2.0">
<info organisation="com.maethorechannen" module="snow-core"/>
<dependencies>
<dependency org="org.scala-lang" name="scala-library" rev="2.7.7"/>
<dependency org="org.scala-lang" name="scala-compiler" rev="2.7.7"/>
<!-- other dependecies -->
</dependencies>
</ivy-module>


If you've used Maven then this will look vaguely familiar yet more compact - groupId becomes org, artifactId becomes name and version becomes rev. As by default Ivy uses Maven's repositories, you can use the same values as you would in Maven.

build.xml

My build.xml will need to do 3 things:

  1. download ivy

  2. setup scala

  3. compile



Strictly speaking, the first item is not technically required if I manually install Ivy into Ant. For the moment, it seems easier if I just set it up so that Ivy gets automagically installed if need be. Fortunatly, the ivy-go example build.xml file shows us how to do this.

The Scala side of things is also fairly straightforward. The scala-lang.org page on Scala Ant Tasks gave me a fairly good idea of what needed to be done, however I had to make a few changes:

  • make the init target depend on the install-ivy target

  • modify init so that it executes ivy:retrieve (which downloads the dependencies) and modify the way init generates the build classpath so that it includes jars from the lib directory

  • modify the taskdef that imports the scala tasks (also in the init target) so that it finds scala-compiler and scala-library in the lib directory and not in scala.home

  • modify the build target, so that it compiles all scala source files in the src/main/scala directory and subdirectories



As the build.xml file is a bit big for a blog post, you can find it here

Friday, December 18, 2009

Accessing BBC Listen Again data with YQL

The BBC have had for a while now XML feeds detailing their "Listen Again" content. For some reason, today I decided it would be fun to access that data from YQL.

The quickest way is to just use

select * from xml where url='http://www.bbc.co.uk/radio/aod/availability/radio4.xml'


However, I wanted to try out Open Data Tables, and this seemed like a good a time as any. Describing an Open Data Table is surprisingly easy. It's just an XML file, with some metadata about the table and then some binding information that tells YQL where to get the repeating data from.


<table xmlns="http://query.yahooapis.com/v1/schema/table.xsd">
<meta>
<author>Scot McSweeney-Roberts</author>
<documentationURL>http://froth-and-java.blogspot.com/accessing-bbc-listen-again-data-with-yql</documentationURL>
<sampleQuery>select entry from {table} where station="radio4" and entry.parents.parent.pid="b006qfvv"</sampleQuery>
</meta>
<bindings>
<select itemPath="schedule.entry" produces="XML">
<urls>
<url env="all">http://www.bbc.co.uk/radio/aod/availability/{station}.xml</url>
</urls>
<inputs>
<key id="station" type="xs:string" paramType="path" default="radio4" />
</inputs>
</select>
</bindings>
</table>



You'll have noticed the "station" key - the BBC has a separate feed for each service, so Radio 1's feed is at radio1.xml, Radio 4's is at radio4.xml, Radio Scotland is at radioscotland.xml, etc. By using a key and inserting its value into the select URL I can use a single Open Data Table definition and access whichever feed I want. The reason why I called it station and not service is that service is a child element of each entry element, so to keep things simple I called the key "station".

One reason for using an Open Data Table is that makes it slightly easier to query - for example, a query to select all Shipping Forecasts on Radio 4 without using an Open Data Table is


select * from xml where url="http://www.bbc.co.uk/radio/aod/availability/radio4.xml" and itemPath="schedule.entry" and parents.parent.pid="b006qfvv"


With an Open Data Table, it's


USE "http://sites.google.com/site/frothandjava/home/code/bbc_aod_open_table.xml"AS aod;
SELECT * FROM aod where station="radio4" and parents.parent.pid="b006qfvv"


(It's not really much of a difference though - I think table definitions come into their own when you use them to insert data in to a web service)

One of the more useful elements in the feed are the "mediaselector" links. A mediaselector link is a link to yet another XML file, but this file contains links to the available media for a particular programme. Some of the links aren't particularly useful, as they're internal iPlayer links, but some are useful as they're the links to WMA and Real Audio streams (though the BBC are dropping support for Real Audio at some point in the future).

Again, as it's XML it's easy to access from YQL

select * from xml where url='http://www.bbc.co.uk/mediaselector/4/mtis/stream/b00p6w2c'

(note, that query will probably not work after lunchtime on the 19th December, 2009 as that particular episode falls out of it's listen again window)

However, there is a bit of a problem at the moment for UK users of this data while accessing it through YQL - the BBC are currently sending a slightly different mediaselector file depending on where the request came from. As Yahoo's YQL servers are located outside the UK, we get the "international" version of the file and not the one containing UK data. Not only does this mean that we can't get access to the higher quality UK only versions of the streams, but I've noticed that the international WMA streams don't seem to be working in the UK (but the RA ones do). Apparently, this will be solved "at some point in the new year".