Froth & Java

Using a Tomcat Engine JMX Proxy Bean With Spring

2011-05-16T09:06:00.015Z

Recently, I needed to determine which Tomcat server in a load balanced environment an application was running on. One way to to do this is to look at the jvmRoute property of the running Tomcat Engine. If you're in a load balanced environment you'll probably see the jvmRoute appended to the session id. What I wanted to do was to get that value directly, without any reference to a session. Here's how I did it.

One quick note before I begin - obviously, this ties your application to Tomcat so you might want to look at ways of abstracting this out if server portability is a concern.

The first thing you need to do is make sure that a jvmRoute value is being set. This is as simple as adding something like -DjvmRoute=jvm1 to your VM arguments.

Next up is adding a MBeanProxyFactoryBean to your application context. To inject a proxy for the Engine, you need


<bean id="tomcatEngineProxy" class="org.springframework.jmx.access.MBeanProxyFactoryBean">
  <property name="objectName" value="Catalina:type=Engine" />
  <property name="proxyInterface" value="org.apache.catalina.Engine" />
  <property name="useStrictCasing" value="false" />
 </bean>

You may or may not need to play about with the objectName value depending on how you have configured your Engine in server.xml - have a look here.

Setting useStrictCasing to false is especially important. For some reason Spring's default behaviour is to set useStrictCasing to true. What this means is that when you access a property via a getter (in our case, getJvmRoute) causing the proxy to ask for the property value over JMX, by default it ignores normal Javabean style conventions and asks for the "JvmRoute" property. Apache is expecting to be asked for the "jvmRoute" property and you'll be on the receiving end of a "cannot find property" error. Setting useStrictCasing to false makes Spring ask for "jvmRoute" instead. I'd really like to know why useStrictCasing is set to true by default, as it seems to violate the principle of least surprise.

Now that we have a proxy for the Tomcat Engine in the context, we can autowire it where it's needed -

@Autowired private Engine engine;

and getting the jvmRoute is as simple as

engine.getJvmRoute()

Thanks go out to Joachim Sauer on stackoverflow for pointing me in the right direction.

Google App Inventor and node.js

2010-08-26T21:48:00.006Z

My invite to Google's new, easy to use App Inventor for Android came through recently and within minutes I had hello world running. Even better, after a few more minutes of playing with the programming toybox I had an easy to use tool.

For awhile now I've wanted a simple tool that would let me go around scanning the barcodes of books that I own and saving them into a text file for importing into LibraryThing but I haven't had the spare time to learn enough Android programming to build one. A combination of a simple client built with App Inventor and a simple server written with Node and I had the tool I've been wanting in a matter of minutes.

The AppInventor Client

AppInventor comes in two parts - an "Interface Builder" website and a visual coding environment where you connect up blocks of code (that sits on top of the Kawa dialect of Scheme - which makes App Inventor coding a lot like a combintation of Lego and Lisp). I quickly added a label (to display the scanned ISBN number), a button (to start the scan) and then two non-visible components: a barcode scanner and a TinyWebDB. Once the UI was done it was off to the coding environment to wire everything up.

The only not quite straightforward part of all this is the TinyWebDB component as it's App Inventor's only built in way of accesing a web service (other than Twitter, which has it's own component). However, TinyWebDB isn't a generic web service client, it's meant for storing and retreiving simple key/value pairs so I needed to write my server with that in mind.

As the coding environment is visual, it really doesn't lend itself to a blog post. Fortunately, you can download the app source code in a zip file and marvel at the underlying code. The meat of the app is:


;;; Screen1
(do-after-form-creation (set-and-coerce-property! Screen1 'Title "Screen1" 'text)
)
;;; Label1
(add-component Screen1 Label Label1 (set-and-coerce-property! Label1 'Text "Text for Label1" 'text)
)
;;; Button1
(add-component Screen1 Button Button1 (set-and-coerce-property! Button1 'Text "Scan" 'text)
)
(define-event Button1 Click()
(call-component-method 'BarcodeScanner1 'DoScan (list)
*no-coercion*)

)
;;; BarcodeScanner1
(add-component Screen1 BarcodeScanner BarcodeScanner1 )
(define-event BarcodeScanner1 AfterScan( result )
(call-component-method 'TinyWebDB1 'StoreValue (list "isbn" (get-property BarcodeScanner1 Result)
)
'( text any)
)

(set-and-coerce-property! Label1 'Text (get-property BarcodeScanner1 Result)
'text)

)
;;; TinyWebDB1
(add-component Screen1 TinyWebDB TinyWebDB1 (set-and-coerce-property! TinyWebDB1 'ServiceURL "http://192.168.1.101:8765" 'text)
)
(init-runtime  #f)

While that's not quite the entire source, it does give some idea of what goes on underneath.

Node(.js)

TinyWebDB will post a key/value pair to whatever server you want. So I needed a simple server that would sit there, extract the value and output to stdout (which will then get piped into a file). This seemed like the perfect opportunity to try Node (I can't tell if it's supposed to be called node or node.js), which is a Javascript execution environment based on the V8 engine for writing network programs. A quick and dirty Javascript hack produced:


var http = require('http');
http.createServer(function (request, response) {
request.on('data', function(chunk) {
    var b = chunk.toString('ascii', 0, chunk.length);
    var v = b.split("&")[1];
    var vv = v.split("=")[1];
    var vvv = vv.replace("%22", "");
    var vvvv = vvv.replace("%22", "");
    console.log(vvvv);
});
response.writeHead(200, {'Content-Type': 'text/plain'});
response.end('Hello, World\n');
}).listen(8765);

Undoubtedly a regexp could have been used, but would have required more mental effort than I really wanted. In case you're wondering about the response, TinyWebDB apparently essentially disregards it, so sending back Hello, World doesn't hurt.

Conclusion

In a matter of minutes I had a useful (at least to me) tool. I have a feeling that this is what App Inventor will mostly be used for - simple apps that satisfy a single person's needs. It's not going cause the "Animated Cat App Apocalypse" in the Android Market that some commentators have worried about. I have no idea if it will be of any use as the "Logo for the modern age" that was the original reason for it's existence (there are still references to "Young Android" in the generated code and the application icons are of an adult and child robot). I would call it the "Visual Basic for the modern age" except that it feels wrong comparing something that "is part of an ongoing movement in computers and education that began with the work of Seymour Papert and the MIT Logo Group in the 1960s" with Visual Basic.

Using Scala To Configure Spring's DispatcherServlet

2010-02-09T14:45:00.007Z

After reading Francois Armand's post about "Scala and Spring 3 JavaConfig", I was inspired to try and use Scala to replace some servletname-servlet.xml Spring config files.

I have a small Scala/Spring MVC project that at the moment uses two DispatcherServlets, one for each controller. There was a historical reason for this (I needed two different Velocity configurations) and while I can now probably get away with a single DispatcherServlet, what I'm going to try to do is move one of the servlets over to a Scala based configuration and leave one using xml based configuration.

Of the two configurations, I'm going to move the simplest one over. It does nothing more than define a root controller bean. So the Scala config should be as simple as


@Configuration
class CalendarConfig {
 @Bean def rootController = new CalendarController()
}

And a quick change to my web.xml


...
    <servlet>
        <servlet-name>calendars</servlet-name>
        <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
        <!-- Configure DispatcherServlet to use JavaConfigWebApplicationContext
             instead of the default XmlWebApplicationContext -->
        <init-param>
            <param-name>contextClass</param-name>
            <param-value>
                org.springframework.web.context.support.AnnotationConfigWebApplicationContext
            </param-value>
        </init-param>
        <!-- Again, config locations must consist of one or more comma- or space-delimited
             and fully-qualified @Configuration classes -->
        <init-param>
            <param-name>contextConfigLocation</param-name>
            <param-value>com.maethorechannen.calendars.CalendarConfig</param-value>
        </init-param>
    </servlet>
...

I started my app up in my local development Google App Engine server and it failed to start up, thanks to "java.lang.IllegalStateException: CGLIB is required to process @Configuration classes. Either add CGLIB to the classpath or remove the following @Configuration bean definitions: [calendarConfig]". I added cglib-nodep-2.2 (available from here if you're not using something like maven or ivy) to my library of jars and it started to work.

In case anyone's interested - I pushed it to App Engine and it still worked (occasionally, what works locally doesn't work when on the Big G's servers). One thing I'm really interested in seeing is what (if any) sort of impact on startup times programmatic configuration has compared to XML configuration. When using GAE every millisecond of startup time counts and I would think Scala based configuration should be quicker.

What's interesting to note (at least to me) is that I didn't need to create a global WebApplicationContext or move everything over to Java(Scala) based configuration, which is something I wasn't sure about after reading the documentation from SpringSource.

Scala + Spring + Templates - Velocity

2010-02-09T11:55:00.006Z

Next up on my list of template engines to try with Scala & Spring - Velocity. Unlike FreeMarker, I had actual success in getting it to work. Unsurprisingly, "Native" Scala object support needed to be added in - fortunately the way Velocity was designed makes that easy.

Velocity uses introspection/reflection to access properties and methods on an object and this is where the Scala customizations fit in. The first thing to create is a subclass of UberspectImpl which does two things - return a java.util.Iterator for iterable objects and return a VelPropertyGet object for getting access to an object's properties. There is a third thing that could also be done - returning a VelPropertySet object to support setting properties from Velocity but I haven't implemented this as I don't need that functionality at the moment.

Getting a Java Iterator is fairly straightforward (with Scala 2.7.7 - I'm not sure what, if any, changes will be needed with 2.8)


   override def getIterator(obj: java.lang.Object, i: Info): JavaIterator[_] = {
     def makeJavaIterator(iter: Iterator[_]) =  new JavaIterator[AnyRef] {
        override def hasNext() = iter.hasNext
        override def next() = iter.next().asInstanceOf[AnyRef]
           override def remove() = throw new java.lang.UnsupportedOperationException("Remove not supported")
       }
    
     obj match {
       case i: Iterable[_] => makeJavaIterator(i.elements)
       case i: Iterator[_] => makeJavaIterator(i)
       case _ => super.getIterator(obj, i)
     }
   }

The method to return a VelPropertyGet object is almost as simple - but it also shows that we need two more classes - one for accessing Scala properties and one for accessing Scala Map members like they're properties.


 override def getPropertyGet(obj: java.lang.Object, identifier: String, i: Info): VelPropertyGet = {
   if (obj != null) {
     val claz = obj.getClass()

        val executor = obj match {
       case m: Map[_, _] => new ScalaMapGetExecutor(log, claz, identifier)        
       case _ => new ScalaPropertyExecutor(log, introspector, claz, identifier)
         
     }

        if (executor.isAlive) {
          new VelGetterImpl(executor)
        } else {
          super.getPropertyGet(obj, identifier, i)
        }
   } else {
     null
   }
 }

(You can get the full source code here)

To access Scala style properties I'll need a Scala specific subclass of PropertyExecutor. The code to access properties from Scala objects is once again trivial. In fact, because of the way Scala generates it's getter methods the code to access Scala properties is simpler than the code to access JavaBean style properties. (source)


override def discover(clazz: java.lang.Class[_], property: String) = {
     val params = Array[java.lang.Object]()
  setMethod(introspector.getMethod(clazz, property, params))
  if(!isAlive()) {
    super.discover(clazz, property)
  }
 }

Compare that to the code in the parent class


 protected void discover(final Class clazz, final String property)
    {
        /*
         *  this is gross and linear, but it keeps it straightforward.
         */

        try
        {
            Object [] params = {};

            StringBuffer sb = new StringBuffer("get");
            sb.append(property);

            setMethod(introspector.getMethod(clazz, sb.toString(), params));

            if (!isAlive())
            {
                /*
                 *  now the convenience, flip the 1st character
                 */

                char c = sb.charAt(3);

                if (Character.isLowerCase(c))
                {
                    sb.setCharAt(3, Character.toUpperCase(c));
                }
                else
                {
                    sb.setCharAt(3, Character.toLowerCase(c));
                }

                setMethod(introspector.getMethod(clazz, sb.toString(), params));
            }
        }
        /**
         * pass through application level runtime exceptions
         */
        catch( RuntimeException e )
        {
            throw e;
        }
        catch(Exception e)
        {
            String msg = "Exception while looking for property getter for '" + property;
            log.error(msg, e);
            throw new VelocityException(msg, e);
        }
    }

It took me a little while to get Maps working (even though there's not much code to show for it).


class ScalaMapGetExecutor(val llog: Log, val clazz: java.lang.Class[_], val property: String) extends MapGetExecutor(llog, clazz, property) {
 override def isAlive = true

 override def execute(o: AnyRef) = o.asInstanceOf[Map[String, AnyRef]]
                                 .getOrElse[AnyRef](property, null).asInstanceOf[java.lang.Object]
      
}

All that's left to do now is configure Velocity from Spring so that it uses the Scala specific Uberspector instead of the default. So, in my servletname-servlet.xml file I needed to add the following


<bean id="velocityConfig" class="org.springframework.web.servlet.view.velocity.VelocityConfigurer">
  <property name="resourceLoaderPath" value="/WEB-INF/views/"/>
    <property name="velocityProperties">
    <props>
      <prop key="runtime.introspector.uberspect">info.threebelow.verla.mvc.view.velocity.ScalaUberspect</prop>
    </props>
  </property>
</bean>

Scala + Spring + Templates - the story so far

2010-01-23T18:09:00.003Z

I've been writing a fairly simple web app with Scala and Spring that will be running on Google's App Engine. I've been trying to decide which template language (or even languages) to write my views in. I'll be rendering a lot of non-HTML and non-XML and over the years I've noticed some template engines are better at this than others. Also, I'm looking for first class Scala support (so anything you can do with a Java Bean you can do with a Scala object) - ideally built in and at worst trivial to add. Here's what I've come across so far:

Closure Templates

I like Google's Closure Template, but there's no "out of the box" support for it in Spring. If I can't find another template engine I like then I will come back and look at how to add some support.

FreeMarker

I will start at the point where my experiments ended - apparently FreeMarker doesn't currently work on GAE, as FreeMarker uses some swing classes that aren't available in GAE. This is a show stopper. I didn't run in to this during my experiments as it was working on the local dev server and I wasn't aware of this problem until I was submitting a bug for another problem I found.

During my experiments, I found several other issues, one of which calls into question using FreeMarker with Spring and Java, let alone Scala.

First, there isn't built in object wrapping for Scala objects. By default, FreeMarker will treat Scala objects as Java objects, but as plain Scala objects don't follow JavaBeans conventions access to getters ceases to be trivial. I was able to whip up a Scala specific FreeMarker Model and ObjectWrapper with relative ease, but then I ran into another problem - there doesn't appear to be an easy way in Spring to configure FreeMarker with an alternative ObjectWrapper.

The answer was to create a subclass of org.springframework.web.servlet.view.freemarker.FreeMarkerConfigurer and modifying one of the Configuration related methods to return a Configuration with an ObjectWrapper property set to my Scala specific ObjectWrapper. However, I then ran into a problem that I eventually traced to what appears to be a bug in FreeMarker. When Spring sends a model to FreeMarker it uses FreeMarker's AllHttpScopesHashModel - unfortunately AllHttpScopesHashModel's constructor doesn't let it's super class know about the ObjectWrapper that was passed in (it just stores it in a final variable that is private to AllHttpScopesHashModel). This means inherited methods fall back to using Fremarker's default ObjectWrapper - so Scala objects don't work. While working with the FreeMarker source code to make sure this is what was going wrong (it was), I ran into two more issues with FreeMarker.

As I was using the trunk from FreeMarker's SVN repository I assume I was using what will be the 2.4 release of FreeMarker. The first problem I ran into with this was that Spring's FreeMarkerView imports freemarker.core.ParseException, unfortunately ParseException has been moved to freemarker.core.parser.ParseException which obviously breaks Spring 3. As I said above, I have doubts about about using FreeMarker with Spring and any language - this is the reason why. At some point this backwards incompatible API change is going cause someone, somewhere a lot of pain and I don't want to be that person.

While mucking about with FreeMarker's source, I ran into another problem. FreeMarker 2.4 will have some new security features which seemed to cause no end of trouble with GAE. While you should be able turn these features off in the Configuration object, setting the secure property to false had no impact on getting FreeMarker to work with GAE. While I had pretty much given up on FreeMarker by this point, I still wanted to make sure that the reason why my ObjectWrapper wasn't working was because of AllHttpScopesHashModel's constructor so I commented out some security related lines in FreeMarker and eventually got it all working.

Summary

I won't be using FreeMarker. There are still a few more template engines to look at before I'll fall back to forcing Closure Templates to work with Spring (I would be astounded if it doesn't work GAE). So, I'll carry on my search (Velocity is probably next).

A Basic Ant+Ivy+Scala Setup

2009-12-22T21:44:00.004Z

I've decided that for my latest project I'd give Ant with Ivy a go as my build system. To start with, I'm going to try for the simplest possible setup necessary in order to compile my project (so at this stage, it's not even rolling the output up into a jar - I can fix that later). At it's most basic level, this means that I'll need two XML files to get everything to build:

ivy.xml

ivy.xml contains information about the dependencies needed to build the project. To compile scala source, I'll need to add two dependencies - scala-library and scala-compiler.


<ivy-module version="2.0">
    <info organisation="com.maethorechannen" module="snow-core"/>
    <dependencies>
  <dependency org="org.scala-lang" name="scala-library" rev="2.7.7"/>
  <dependency org="org.scala-lang" name="scala-compiler" rev="2.7.7"/>
         <!-- other dependecies -->
    </dependencies>
</ivy-module>

If you've used Maven then this will look vaguely familiar yet more compact - groupId becomes org, artifactId becomes name and version becomes rev. As by default Ivy uses Maven's repositories, you can use the same values as you would in Maven.

build.xml

My build.xml will need to do 3 things:

download ivy

setup scala

compile

Strictly speaking, the first item is not technically required if I manually install Ivy into Ant. For the moment, it seems easier if I just set it up so that Ivy gets automagically installed if need be. Fortunatly, the ivy-go example build.xml file shows us how to do this.

The Scala side of things is also fairly straightforward. The scala-lang.org page on Scala Ant Tasks gave me a fairly good idea of what needed to be done, however I had to make a few changes:

make the init target depend on the install-ivy target

modify init so that it executes ivy:retrieve (which downloads the dependencies) and modify the way init generates the build classpath so that it includes jars from the lib directory

modify the taskdef that imports the scala tasks (also in the init target) so that it finds scala-compiler and scala-library in the lib directory and not in scala.home

modify the build target, so that it compiles all scala source files in the src/main/scala directory and subdirectories

As the build.xml file is a bit big for a blog post, you can find it here

Accessing BBC Listen Again data with YQL

2009-12-18T18:49:00.006Z

The BBC have had for a while now XML feeds detailing their "Listen Again" content. For some reason, today I decided it would be fun to access that data from YQL.

The quickest way is to just use


select * from xml where url='http://www.bbc.co.uk/radio/aod/availability/radio4.xml'

However, I wanted to try out Open Data Tables, and this seemed like a good a time as any. Describing an Open Data Table is surprisingly easy. It's just an XML file, with some metadata about the table and then some binding information that tells YQL where to get the repeating data from.


<table xmlns="http://query.yahooapis.com/v1/schema/table.xsd">
  <meta>
    <author>Scot McSweeney-Roberts</author>
    <documentationURL>http://froth-and-java.blogspot.com/accessing-bbc-listen-again-data-with-yql</documentationURL>
    <sampleQuery>select entry from {table} where station="radio4" and entry.parents.parent.pid="b006qfvv"</sampleQuery>
  </meta>
  <bindings>
   <select itemPath="schedule.entry" produces="XML">
 <urls>
         <url env="all">http://www.bbc.co.uk/radio/aod/availability/{station}.xml</url>
 </urls>
 <inputs>
  <key id="station" type="xs:string" paramType="path" default="radio4" />
 </inputs> 
    </select>
  </bindings>
</table>

You'll have noticed the "station" key - the BBC has a separate feed for each service, so Radio 1's feed is at radio1.xml, Radio 4's is at radio4.xml, Radio Scotland is at radioscotland.xml, etc. By using a key and inserting its value into the select URL I can use a single Open Data Table definition and access whichever feed I want. The reason why I called it station and not service is that service is a child element of each entry element, so to keep things simple I called the key "station".

One reason for using an Open Data Table is that makes it slightly easier to query - for example, a query to select all Shipping Forecasts on Radio 4 without using an Open Data Table is


select * from xml where url="http://www.bbc.co.uk/radio/aod/availability/radio4.xml" and itemPath="schedule.entry" and parents.parent.pid="b006qfvv"

With an Open Data Table, it's


USE "http://sites.google.com/site/frothandjava/home/code/bbc_aod_open_table.xml"AS aod;
SELECT * FROM aod where station="radio4" and parents.parent.pid="b006qfvv"

(It's not really much of a difference though - I think table definitions come into their own when you use them to insert data in to a web service)

One of the more useful elements in the feed are the "mediaselector" links. A mediaselector link is a link to yet another XML file, but this file contains links to the available media for a particular programme. Some of the links aren't particularly useful, as they're internal iPlayer links, but some are useful as they're the links to WMA and Real Audio streams (though the BBC are dropping support for Real Audio at some point in the future).

Again, as it's XML it's easy to access from YQL


select * from xml where url='http://www.bbc.co.uk/mediaselector/4/mtis/stream/b00p6w2c'

(note, that query will probably not work after lunchtime on the 19th December, 2009 as that particular episode falls out of it's listen again window)

However, there is a bit of a problem at the moment for UK users of this data while accessing it through YQL - the BBC are currently sending a slightly different mediaselector file depending on where the request came from. As Yahoo's YQL servers are located outside the UK, we get the "international" version of the file and not the one containing UK data. Not only does this mean that we can't get access to the higher quality UK only versions of the streams, but I've noticed that the international WMA streams don't seem to be working in the UK (but the RA ones do). Apparently, this will be solved "at some point in the new year".

Extending Soy Templates

2009-11-23T18:24:00.011Z

One of the more interesting features of Soy templates is that you can extend the templating system. While working with Soy from JavaScript, I found that I needed to do exactly that.

In order to avoid some confusion as to how you can extend the template system in Scala, even though you will be using the templates from JavaScript, I'll quickly explain how Soy templates work when you're using JavaScript. Essentially, you precompile your .soy file into a .js file using a template compiler (which is written in Java). It's the compiler that you're extending and it's output is what you use in your web app. So you don't directly use the template from JavaScript.

What I wanted to do was something like this:

{switch typeof($msgparm)}
{case 'number'}
<i4>{$msgparm}</i4>
{case 'string'}
<string>{$msgparm}</string>
{/switch}

Unfortunatly, there is no typeof function "out of the box" so I needed to write my own in a plugin. As I was only interested in using this function in templates that I'm going to use in JavaScript, I needed to write a class that implements a single interface - SoyJsSrcFunction. If I wanted to use this function from Java, I'd need to implement SoyToFuFunction instead. There are only 3 methods to implement, 2 of which are one-liners in Scala. In this instance, all three are one liners, but computeForJsSrc could be more complicated, depending on what you're doing.


@Singleton
class TypeOfFunction @Inject() extends SoyJsSrcFunction {
   def getName = "typeof"
 
   def getValidArgsSizes: java.util.Set[Integer] = ImmutableSet.of[Integer](1);

   def computeForJsSrc(args: java.util.List[JsExpr]): JsExpr = new JsExpr("typeof(" + args.get(0).getText() + ")", Integer.MAX_VALUE)
 
}

Scala Source Code

As you can probably tell, all this does is use JavaScript's typeof operator to implement the functionality I was looking for.

You might notice the @Singleton and @Inject annotations - Soy uses Google Guice for dependency injection. This means that we're not done yet - we also need to write a subclass of Guice's AbstractModule so that our extension can be injected into Soy. Fortunately it's trivial and XML-free:


class ExtendedFunctionsModule extends AbstractModule {
  def configure() = {
   val soyFunctionsSetBinder = Multibinder.newSetBinder(binder(), classOf[SoyFunction])
   soyFunctionsSetBinder.addBinding().to(classOf[TypeOfFunction])
 }

}

Scala Source Code

To use the extension all I need to do is add my Module class to the template compiler's pluginModules option:


java -cp typeof.jar:scala-library.jar:SoyToJsSrcCompiler.jar com.google.template.soy.SoyToJsSrcCompiler --pluginModules com.maethorechannen.widgets.ExtendedFunctionsModule --outputPathFormat rpc.js rpc.soy

Consice Java Maps in Scala

2009-11-23T18:00:00.007Z

This post is a follow up to my previous post on using Closure Templates in Scala.

One of the things that bugged me in the Scala example was how verbose it was compared to the Groovy version. So I had an idea - use YAML and the snakeyaml library to generate a Java Map with my data. While it's still a couple lines longer than the Groovy version, it's a lot simpler than the Scala original.

The biggest change is this:


val y = """widget: {
name: Widget Name,
display: {
  width: 240,
  height: 320
}
}"""

val yaml = new Yaml()
val map = yaml.load(y).asInstanceOf[java.util.Map[String, _]]

Scala Source Code

Using Google's Closure Templates in Scala and Groovy

2009-11-23T15:26:00.013Z

Google recently released it's Closure Tools - which is made up of a JavaScript compiler, a JavaScript framework and a templating system which can be used from both Java and JavaScript. I'm guessing from the package names that Closure Templates are actually called Soy templates internally at Google, and that's what I'm going to call them for the rest of this post.

I'm not going to go into the syntax of Soy templates - if you've ever used a templating system like FreeMarker or Django Templates then you shouldn't have any trouble using Soy templates (though you should be aware that Soy eats white space, so your output might not be what you expect if you don't use special character commands to put whitespace into the output).

Here's an example template (that I saved as "widget.soy"):


{namespace com.maethorechannen.widgets}

/** Widget template
* @param widget Widget Data
*/
{template .widget}
<?xml version="1.0" encoding="utf-8"?>{\n}
<widget>{\n}
{\t}<widgetname>{$widget.name}</widgetname>{\n}
{\t}<width>{$widget.display.width}</width>{\n}
{\t}<height>{$widget.display.height}</height>{\n}
</widget>
{/template}

Using the templates from Scala:

At a minimum, you'll need the following imports:


import com.google.template.soy.SoyFileSet
import com.google.template.soy.data.SoyMapData
import com.google.template.soy.tofu.SoyTofu
import java.io.File

To pass your data to the ToFu rendering engine, you'll need to have your data in either a SoyMapData object or in a plain old Java Map object. As I wanted to use Scala's sugar for quickly building up a Map, I needed to write up an implicit def from Scala's Map to a SoyMapData (quick aside - the sort of thing that stops me from loving Scala is having to do something like this - the impedence mismatch between Java collections and Scala collections is still too big). This function isn't complete, but it will convert the values I was using:


implicit def mapToSoy(m: Map[String, _]): SoyMapData = {
val sm = new SoyMapData()
m.keys.foreach {k =>
val v: Any = m(k)
v match {
   case s: String => sm.put(k,s)
   case mm: Map[String, _] => sm.put(k, mapToSoy(mm))
   case i: Int => sm.put(k, i)
}
}
return sm
}

Here's the Map I'm passing in:


val map = Map[String, Any](
"widget" -> Map[String, Any](
"name" -> "Scala Test",
"display" -> Map[String, Any](
    "width"-> 240,
    "height" -> 320)))

To actually use the template, I need to load the template into a SoyFileSet, and then compile the template to a rendering object


val sfs = (new SoyFileSet.Builder()).add(new File("widget.soy")).build();
val tofu = sfs.compileToJavaObj()

Finally, I render the template with my data


println(tofu.render("com.maethorechannen.widgets.widget", map, null));

Which renders the following:


<?xml version="1.0" encoding="utf-8"?>
<widget>
     <widgetname>Scala Test</widgetname>
     <width>240</width>
     <height>320</height>
</widget>

Scala Source Code

Using the templates from Groovy:

Using Soy from Groovy is almost identical to using Soy from Scala, but without the need for the implicit def (as Groovy Map sugar returns a Java Map).


import com.google.template.soy.SoyFileSet
import com.google.template.soy.data.SoyMapData
import com.google.template.soy.tofu.SoyTofu

def map = [widget: [
name: "Widget Name",
display: [
width: 240,
height: 320
]
]]

def sfs = (new SoyFileSet.Builder()).add(new File("widget.soy")).build();
def tofu = sfs.compileToJavaObj()
println(tofu.render("com.maethorechannen.widgets.widget", map, null));

Groovy Source Code

Musical Fun With YQL

2009-07-15T15:27:00.005Z

Vernian Process have released a large amount of their back catalog available for free download from last.fm (63 mp3 files worth)¹. As someone who never says no to free dark ambient, I thought I would quickly screen scrape the free download urls from the 6 month chart page and feed them into wget instead of manually downloading them².

For the screen scraping, I turned to Yahoo's YQL service. In effect, YQL makes the web "programmable" with a SQL-like language. One of YQL's features is that it can use any webpage as a data source (it also supports various web APIs, like flickr's, but all I need for this is it's html ability). Here's the YQL query:


select * from html 
where url="http://www.last.fm/music/Vernian+Process/+charts?rangetype=6month&subtype=tracks"
and xpath="//a[starts-with(@href, 'http://freedownloads.last.fm')]"

That will return all the free download anchors contained on the page.

It was then a single command to download all the tracks (thanks to curl and wget)


curl "http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%22http%3A%2F%2Fwww.last.fm%2Fmusic%2FVernian%2BProcess%2F%2Bcharts%3Frangetype%3D6month%26subtype%3Dtracks%22%20and%20xpath%3D%22%2F%2Fa%5Bstarts-with(%40href%2C%20'http%3A%2F%2Ffreedownloads.last.fm')%5D%22&format=xml" | wget --force-html -i -

1. They've done it because they've ~~improved~~ changed styles and have gone more Industrial.

2. In case you're wondering, I used the Chart and not the Free Tracks page as the Free Tracks page only had 4 of the freely downloadable tracks.

Sieve of Eratosthenes

2009-06-29T14:28:00.004Z

This is my response to the Scala slide in the "JavaOne Talk: Performance Comparisons of Dynamic Languages on the Java Virtual Machine" presentation. In particular, the Scala example seemed quite verbose and "unscalalike", so I thought I would see what I could quickly knock up. It's the Sieve of Eratosthenes (sort of - it's recursive and it doesn't bother with Step 5 of the algorithm on the Wikipedia page, so it does more work than is needed).

It's quick and dirty and could stand to be improved - my main motivation was to show that it could be done quickly and in a lot fewer lines in Scala than was shown on the presentation slide so it's not the greatest example of code ever written. I've put it up here because blogger's commenting system nukes source code layout.


def primes(n: Int): List[Int] = {
        def nomults(s: Int, xs: Seq[Int]): List[Int] = (for (x <- xs if x % s != 0) yield x).toList
        def sieve(xs: List[Int]): List[Int] = {
                if (xs.isEmpty) xs
                else {
                        val p = xs.first
                        val nxs = nomults(p, xs drop 1)
                        p :: sieve(nxs)
                }
        }

        val odds = nomults(2, 2 to n)
        2 :: sieve(odds)
}

println( primes(100) )

Oracle To Buy Sun

2009-04-20T14:08:00.002Z

Article at the NY Times

Sun's Press Release

My first impression is that this should be good for Java, less good for MySQL.

Scala on the Google AppEngine

2009-04-08T21:00:00.007Z

Good News - It was fairly trivial to use Scala with the new Java support in AppEngine.

I used a text editor and ant instead of mucking about with Eclipse. It was pretty much a case of following the docs for Java, just using Scala instead of Java for writing the servlet. The main difference between using Java and Scala is that you need to merge some Scala specific items into the build.xml file. Here's my build.xml


<project>
  <property name="scala.home" location="../scala-2.7.3.final" />
  <property name="sdk.dir" location="../appengine-java-sdk-1.2.0" />
  <property name="sources.dir" value="${base.dir}/src" />
  <property name="build.dir" value="${base.dir}/build"/>

  <import file="${sdk.dir}/config/user/ant-macros.xml" />

  <path id="project.classpath">
    <pathelement path="war/WEB-INF/classes" />
    <fileset dir="war/WEB-INF/lib">
      <include name="**/*.jar" />
    </fileset>
    <fileset dir="${sdk.dir}/lib">
      <include name="shared/**/*.jar" />
    </fileset>
  </path>

  <target name="init">
    <property
      name="scala-library.jar"
      value="${scala.home}/lib/scala-library.jar"
       />
    <path id="build.classpath">
      <pathelement location="${scala-library.jar}"   />
 
      <pathelement location="${build.dir}"   />
    </path>
    <taskdef resource="scala/tools/ant/antlib.xml">
      <classpath>
        <pathelement location="${scala.home}/lib/scala-compiler.jar"   />
        <pathelement location="${scala-library.jar}"   />
      </classpath>
    </taskdef>
  </target>
  
  
    <target name="copyscala"
      description="Copies the App Engine JARs to the WAR.">
    <copy
        todir="war/WEB-INF/lib"
        flatten="true">
      <fileset dir="${scala.home}/lib">
        <include name="**/scala-library.jar" />
      </fileset>
    </copy>
  </target>

    <target name="copyjars"
      description="Copies the Scala JARs to the WAR.">
    <copy
        todir="war/WEB-INF/lib"
        flatten="true">
      <fileset dir="${sdk.dir}/lib/user">
        <include name="**/*.jar" />
      </fileset>
    </copy>
  </target>
  
    <target name="compile" depends="copyscala, copyjars, init"
      description="Compiles Scala source and copies other source files to the WAR.">
    <mkdir dir="war/WEB-INF/classes" />
    <copy todir="war/WEB-INF/classes">
      <fileset dir="src">
        <exclude name="**/*.scala" />
      </fileset>
    </copy>
    <scalac
        srcdir="src"
        destdir="war/WEB-INF/classes"
        classpathref="project.classpath"
         />
  </target>

  
  <target name="runserver" depends="compile"
      description="Starts the development server.">
    <dev_appserver war="war" />
  </target>

</project>

Java Coming To Google AppEngine

2009-04-08T09:21:00.002Z

The docs are up on Google's AppEngine site!

I've been playing with the python version (though I haven't released anything) - now it's time to look at using Java (and hopefully Scala and Groovy).

Starbucks (UK) Vs Linux

2009-03-03T23:17:00.008Z

I can't believe how braindead Starbucks' UK website is. If you're running Linux and trying to register your Starbucks card you'll get this message:

Browser error

We're sorry, your browser doesn't support the Starbucks Card registration pages of this website.

Please update your browser to Internet Explorer version 5.0 or above, or Netscape Navigator version 6.0 or above.

Thank you.

Here I was thinking Firefox 3 was above Netscape 6.

Of course, if you switch the User Agent to say that you're running Windows, all works well. 1999 called - they want thier browser detection back.

Maybe it has something to do with having headquaters in Seattle.

Stupid Scala Trick: Injecting Methods With Spring

2008-07-31T12:32:00.013Z

I'm not sure why I came up with this idea, but it occurred to me that since functions are first class objects in Scala then I should be able to inject them as dependencies using the Spring framework. So I decided to give it a try (using constructor injection to make life simple).

First, I need a class that will have a method injected -


class User(val f: String => String) {

def b() = {f("hello")}

}

So, this class will have a method named f (which takes a String and returns a String) injected into it. It also has a method called b, which will use f.

Next up, a function that will be injected.


object func1 extends Function1[String, String] {
        def apply(s: String): String = {s.toUpperCase()}
}

This is essentially the same as


def func1(s: String): String = {s.toUpperCase()}

Now, the Spring bean definition xml (which I'll save in a file called app.xml)


<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
           http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">


  <bean id="user" class="User">
        <constructor-arg>
          <bean class="func1$"/>
        </constructor-arg>
  </bean>
</beans>

Finally, a small application to fire up spring and test that injection worked


import org.springframework.context.support._

object SpringTest {

        def main(args: Array[String]) = {
                
                val context  = new ClassPathXmlApplicationContext(Array[String]("app.xml"));
                val user = context.getBean("user").asInstanceOf[User]
                println(user.b())
                println(user.f("world"))
        }
}

And here's the output (minus some log4j warnings)


HELLO
WORLD

maven & build-helper

2008-07-18T16:55:00.006Z

I ran into a little issue with maven and the build-helper plugin today.

The background is as follows - I was part-mavenizing a project (long, boring work related story) and I needed to generate some source code using JAXB (which was a trial in and of itself - we're compiling on Java 1.4 and JAXB hates Java 1.4 - in the end I needed to use antrun to run the 1.0.0 version of xjc) and then have it compile as part of the main compile process. So build-helper to the rescue. The documentation states that you just add the plugin to the build-


...
     <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>build-helper-maven-plugin</artifactId>
        <version>1.1</version>
        <executions>
          <execution>
            <id>add-source</id>
            <phase>generate-sources</phase>
            <goals>
              <goal>add-source</goal>
            </goals>
            <configuration>
              <sources>
                <source>some directory</source>
                ...
              </sources>
            </configuration>
          </execution>
        </executions>
      </plugin>
...

Note the "some directory" for the source. I was trying to place the generated code into /target/generated-code/java, seeing as the generated code was output and output ends up in /target. It wouldn't work. I tried several variations on a theme, but as long as I kept trying to compile source that lived in target, maven would not see it. In the end, I placed the code in {$basedir}/src/generated/java (ie, in src, not target), pointed build-helper to it and it started to work.

While it works, it does mean I have more work ahead in order to get clean to work as expected.

Processing/Scala Improvement

2008-07-17T19:46:00.008Z

I've come up with an improvement to my Processing/Scala combination. Instead of using appletviewer, I've created an application object that fires up Swing and embeds the PApplet.

I first turned to the latest preprint of Programming in Scala which has a chapter dedicated to Scala Swing applications and then rapidly ran into issues. First, it doesn't mention that scala.swing isn't in the standard library (at least, not yet), so I had to track down a jar of version 0.3 and then add it to the classpath. Second, the example code in the book still doesn't work (it complains that += is not a member of Seq[Component] on the code that adds items to the main frame). So I gave up on using scala.swing and turned to the Scala wiki to see how to drive Swing directly. The result is


import javax.swing._

object TestRunner extends JFrame( "Test" ) 
{    
    def main( args: Array[String] ) = 
    {
        setDefaultCloseOperation( JFrame.EXIT_ON_CLOSE );
        val t = new Test()
        add( t );
        t.init()
        pack()
        setVisible( true );
    }
}

Combining Scala With Processing

2008-07-16T21:24:00.007Z

I recently bought a copy of Ben Fry's "Visualizing Data", which explores data visualization (unsurprising, considering the title) using Processing. So, I got the idea of combining Processing with Scala, in order to get Scala's more advanced language features with Processing's easy to use graphics.

The key to combining the two is page 249 of Visualizing Data, which describes how to use Java instead of Processing's own Java like language. It was fairly trivial to convert the Java to Scala -


import processing.core._

class Test extends PApplet {
 val bg = 0xFFEECC00
 
 override def setup = {
 
  size(400,400)
 }

 
 override def draw = {
 
  background(bg)
  line(mouseX, mouseY, width/2.0f, height/2.0f)
 }
}

When you compile the code with scalac, you'll need to include Processing's core.jar on the classpath.

As you can probably tell from the processing base class, you end up with a good old fashioned Java applet. As it's been over 10 years since the last time I played with applets, it took a few minutes to get it to work. To cut a long story short, I did the following:

1) jar Test.class
2) copy core.jar from Processing into the current directory
3) copy scala-library.jar from Scala into the current directory
4) create a HTML file with the following:


<APPLET CODE=Test.class WIDTH=500 HEIGHT=500 ARCHIVE="Test.jar, core.jar, scala-library.jar">
</APPLET>

5) finally, start appletviewer:

appletviewer test.html

I'm certain there's a better way to package it all up, but like I said, it's been years since I played with applets.

A TCP/IP Echo Server in Scala

2008-02-22T16:24:00.004Z

I thought I'd see what low level (for Java) network programming would be like in Scala and at the same time get used to using Scala's Actors for concurrency.

Starting with the basics, I've implemented an Echo server (which seems to be the "hello, world" of network programming). It's quite simple - wait for a connection, read a single line of text, send the text straight back and close the connection. Here's what I've come up with (I'm certain it could be made more concise, but then in Scala "there's always a more consise way to do it")


import java.net._
import java.io._

import scala.actors._
import scala.actors.Actor._

case class Echo(socket: Socket)

object Service extends Actor {

 implicit def inputStreamWrapper(in: InputStream) =
  new BufferedReader(new InputStreamReader(in))

 implicit def outputStreamWrapper(out: OutputStream) =
  new PrintWriter(new OutputStreamWriter(out))

 def echo(in: BufferedReader, out: PrintWriter) {
   val line = in.readLine()
   out.println(line)
   out.flush()
 }

 def act() {
  loop {
   receive {
    case Echo(socket) =>
     actor {
      echo(socket.getInputStream(), socket.getOutputStream())
      socket.close
     }
   }
  }
 }

}

object EchoServer {
 Service.start

 val serverSocket = new ServerSocket(12111)
 
 def start() {
  while(true) {
   println("about to block")
      val clientSocket = serverSocket.accept()
   Service ! Echo(clientSocket)
   println("back from actor")
  }
 }
}

EchoServer.start

New Toy - Asus Eee PC

2008-02-12T13:55:00.000Z

Sorry for the lull in my Scala vs Groovy posts - I bought a new toy a couple weeks back and it's been sucking up my free time. I've got three posts mostly finished and they should be out soon. The funny thing is that I bought the thing to become more productive.

As for the device itself - it's fantastic. I ended up buying the 2G Surf model (512 meg of RAM, 2 gig SSD), basically because it was available and I wanted one right then and there (unsurprisingly the 4Gs were few and far between). While I'd recommend the 4G version to anyone buying one (as there's at least a chance that you can upgrade the memory in the 4G, it's got a better battery and isn't as underclocked), the 2G is fine for what I wanted (as I was going to install Xubuntu to a SDHC card no matter what model I was going to get).

Xubuntu installed fine (though a very small amount of extra work was needed to get WiFi and sound working), Java's running fine (though I've yet to test Scala and Groovy), mplayer's runs (though I think it'll run better with the underclocking switched off, yet another item on the to do list). Firefox runs and Google Reader's offline mode makes for a great way to pass the time on the tube to work.

Scala vs Groovy: Lists (Part 1)

2008-01-15T19:59:00.000Z

Before looking at the syntactic sugar Scala and Groovy have for strings, I'm going to have a quick look at how they handle lists. Collection classes are fairly important in both languages, so I'll have a more in depth look later.

Lists in Groovy

In Groovy, Lists come straight out of the Java collection classes but with added syntactic sugar.

The easiest way to create a list is:


def myList = [1, 2, 3]

An empty list can be created by using []


def myEmptyList = []

As Groovy's List is a java.util.List, you can use all the methods on the java.util.List interface. An example would be add:


myList.add(4)

which will add 4 to the end of the list.

You can add an element (or even the contents of another list) to the end of a list with the += operator


myList += 5
myList += [6,7]

will result in myList containing [1, 2, 3, 4, 5, 6, 7]. The + operator works much as you would expect:


def anotherList = [1,2,3] + [4,5,6]

results in [1, 2, 3, 4, 5, 6]. One interesting thing to note is that:


def yetAnotherList = [1, 2, 3]
yetAnotherList.add([4, 5, 6])

results in [1, 2, 3, [4, 5, 6]] and not [1, 2, 3, 4, 5, 6]

To access a particular element of a list, you use square brackets:


println(myList[0])

will print the number 1.

You can an also slice a list using a range:


def subList = myList[2..4]

results in subList containing [3,4,5].

If you create two different lists and they both contain the same elements, == works as you expect:


def list1 = [1,2,3]
def list2 = [1,2,3]

list1 == list2

returns true.

As you have seen, lists in Groovy are mutable - you can add, remove and change the elements to your hearts content. Scala is different.

Lists (and Arrays) in Scala

Unlike Groovy, Scala has both Lists and Arrays. The main difference between them (at least for the purposes of this post) is mutability - Arrays are mutable, Lists are not.

Scala doesn't have a special syntax for either lists or arrays (though it does use a language feature so that you don't need to use new). So, to create a list (or an array) in Scala, you can use:


val myList = List(1, 2, 3)
val myArray = Array(4, 5, 6)

As arrays are mutable, you may wish to create an empty array and populate later:


val myEmptyArray = new Array[Int](3)

which creates an array of size 3 that contains integers. There are a couple of ways to set the values in this array:


myEmptyArray.update(0, 10)
myEmptyArray(1) = 11

myEmptyArray will now contain [10, 11, 0] - when the array was created, each value was initialized to the default "empty" value for that type, which in this case is 0 for Ints.

With lists in Scala, you can't add elements to the list (or delete or change elements either), in the same way that you can't add, delete or change characters in a string. But just like with strings, you can still perform operations on lists, it's just that you get an entirely new list as a result of the operation. We'll start by looking at adding elements:


var aList = List(1, 2, 3)
aList = 0 :: aList

results in aList containing [0, 1, 2, 3]. You'll note how I used var and not val - val would not have worked as we are creating a new list and changing aList to reference the new list. You'll also notice the :: "operator"¹ , which is called "cons". Cons creates a new list with the the object on the left hand side of cons as the first element of the new list and the elements of the list on the right hand side making up the rest of the new list.

Scala has a special "shorthand" for creating an empty list, "Nil"

You can use Nil to build up lists. For example:


var bList = 0 :: 1 :: 2 :: 3 :: Nil

results in bList containing [0, 1, 2, 3]. One interesting thing to note is that


aList == bList

will return true, as both lists contain [0, 1, 2, 3].

There is also a ::: operator, which will combine 2 lists into a single new list


var cList = List(1, 2, 3)
var dList = 4 :: 5 :: 6 :: Nil
var longList = cList ::: dList

will result in a list containing [1, 2, 3, 4, 5, 6]

If we wanted to slice longList and get a sub list of elements:


var sliced = longList.slice(2, 4)

results in a list contain [3, 4] and not [3, 4, 5] that the Groovy slice above (using (2..4)) returns.

Conclusion

I've left a lot (and I mean a lot) of info about Lists in both Groovy and Scala out in this post. The reason why I'm introducing them now is that both languages give Strings list-like syntactic sugar. As I've barely scratched the surface of Lists (let alone the rest of the collection classes) I will get back to them in greater detail later.

Footnotes:
¹Why operator is in double quotes will be the subject of a future comparison of operators and operator overloading

Scala vs Groovy: Strings (Part 1)

2008-01-11T22:54:00.000Z

Single Line Strings

I'm going to start the comparison of string handling in Scala and Groovy with a look at single line strings - the sort of string found in Java.


String s = "Hello, Java!";

Scala's basic string handling is just like Java (but with Scala's syntax and type inference):


var s: String = "Hello, Scala!"
var s2 = "Hello Type Inferer!"

Groovy has both Java-esque strings that are enclosed in double quotes as well as strings that are enclosed in single quotes.


def s = "Groovy's double quoted String"
def s2 = 'A "single quoted" Groovy String'

Groovy also has a third syntax for strings, using forward slashes


def s3 = /Slashy String/

All of the above are java.lang.String - however in Groovy, double quoted strings can become "GStrings". GStrings contain an expression contained in a ${ } block (something that will be familiar to anyone who has written JSP pages). For example:


def x = 4
def gs = "the value of x is: ${x}"
println gs

will print:

the value of x is: 4

Double quoted strings only get promoted to GStrings when they contain ${some expression}. If they don't, then they're plain old java.lang.String strings.

Multi-line Strings

Both Scala and Groovy support multi-line strings. Both use a similar syntax - the triple quote:


var ms = """This is
a Scala multi-line 
String"""


def ms = """This is
a Groovy ${lines}
String"""

You'll notice that in Groovy, multi-line strings get promoted to GStrings as well.

Character

While looking at Strings, we should also look at Character. In a manner not dissimilar to Java, a Character in Scala is defined by a single character in single quotes:


val c = 'd'

In Groovy, Characters can't be created directly. This:


def c = 'd'

creates a String, not a Character. Instead, you need to write this:


def c = 'd' as char

which will create a String and then use some dynamic magic to convert it to a Character

End of Part 1

That's it for the basic comparison. However, both languages have syntactic sugar that gets layered onto strings, but that really should be a post on it's own.

Scala vs Groovy: Variable Definintion & Optional Semicolons

2008-01-11T12:39:00.000Z

Because Groovy is a dynamic language, when you declare a variable you don't have to specify a type for the variable because the run time will work out what the type is for you. To declare such an untyped variable, you use:

def s = "Hello, World!"

However, if you feel like declaring the type, you can:

String s = "Hello, World!"

One thing to note is that declaring the type might actually slow your code down (at least with the version of Groovy current at the time of writing).

Unlike Groovy, Scala is a statically typed language, just like Java. Unlike Java, it can often work out what the type is supposed to be at compile time, and when it can you don't have to explicitly declare the type. In Scala, you use either the "var" keyword or the "val" keyword when you define a variable (I'll discuss the difference between var and val below)

var s = "Hello, World!"

val t = "Hello, World!"

Just like in Groovy, you can also declare the variables type. Unlike Groovy, the way you do this does not look like Java:

var s: String = "Hello World"

You'll notice that you still need the var (or val) keyword and that the type comes after the variable name.

Vars and Vals

What's the difference between var and val? Vars are mutable, vals are not. What this means is that you can change the value of var, but the value of a val is constant. Effectively:

val t = "Hi There!"

is the same as this in Java:

static final String t = "Hi There!";

One thing to be aware of with vals is that if they reference an object, the properties of that object may be mutable - the val is really only a reference to the object, so it's only that reference that's guaranteed to never change.

What all this means is that while you can do this:


var s = "Hello, World!"
s = "Dominate, World!"

You can't do this:


val t = "Hello, World!"
t = "Dominate, World!"

I've found myself thinking of vals as "values" instead of variables because they don't vary.

Optional semicolons

One thing to quickly point out, both Groovy and Scala have optional semicolons. You can still use them if you want to, but most of the time they can be left out.

Comparing Scala And Groovy

2008-01-11T12:22:00.000Z

As I've started to learn Scala, I've been thinking that it would be a good idea to compare Scala and Groovy. I'm going to assume that both are correctly installed and running.

I'll start by comparing comments:


//This is a Groovy comment

/*
It shows how I'll be using green to indicate Groovy code
*/


//This is a Scala comment

/*
It shows how I'll be using blue for Scala code
*/


//This is a Java comment

/*
It shows how I'll be using red if I ever need to show some Java code as well
*/

As you can see Groovy, Scala and Java all use the same style comments.

Playing With Scala

2008-01-07T17:42:00.000Z

Recently I've started playing with Scala, a mixed OO/functional language for the JVM. It's biggest selling point is that it looks and feels a lot like a dynamic (ie scripting) language, but it's statically typed and therefore about as fast as Java code.

My biggest issue has been that a lot of the documentation about it has a very academic feel. One doc I read jumped almost straight into "Actors" (a rather advanced topic, especially for people who have never used Erlang) instead of introducing you to the language. I've been hearing that 2008 is the year Scala will take off - but I think that will only happen if there is "documentation for the rest of us".

Fortunately, that does appear to be coming. I broke down and bought the preprint of Programming in Scala, which goes some way towards decent documentation (though some of the examples do stray into academic territory). Even though it's still in an early form, I highly recommend it to anyone who might be interested in looking into Scala and who doesn't want to feel that they're back on a Computer Science course. My first look at Scala was a couple months back and the lack of decent docs put me off, I thought I'd give it another shot now that this book is out and I haven't dropped Scala yet.

Fixing Photo Dates

2007-06-28T15:44:00.001Z

I've recently realized that I've been taking photographs with the right day and month, but the wrong year (2006, not 2007). This means that some of the EXIF photo metadata is wrong and I want to correct it for each file.

Reading EXIF appears easy under Java with Drew Noakes’s metadata-extractor but that library doesn't write EXIF data. There's the javax.imageio package, but that seems to be low level (ie, hard to use). I want a quick and dirty hack. So I'll use metadata-extractor to get the dates and call out to exiv2 to set the dates.

Of course, Java often brings up things you can stumble over, and this was getting the exiv2 command to run. I decided to use ProcessBuilder from java.lang (which, surprisingly, Groovy didn't seem to auto import), but it took a while to work out how to get it to launch an exiv2 command correctly (if at all). At a unix command line, in order to set the Original Date Time, you'd use something like:


exiv2 -M"set Exif.Photo.DateTimeDigitized String 2007:03:27 07:53:16" ./photos/HPIM0059.JPG

It took ages to work out that I had to split the -M from the set command and that I should not quote the set command, so that the call to ProcessBuilder is along the lines of


Process p = new java.lang.ProcessBuilder("exiv2", "-M", "set Exif.Photo.DateTimeOriginal String 2007:03:27 07:53:16", "./photos/HPIM0059.JPG").start()

Here's the code. Use at your own risk (remember, this script modifies files, so it could destroy something), and you'll proabably need to modify it to get it to work anyway.


import com.drew.metadata.*
import com.drew.imaging.jpeg.*
import com.drew.metadata.exif.ExifDirectory
import org.joda.time.*
import org.joda.time.format.*

DateTimeFormatter fmt = DateTimeFormat.forPattern("yyyy:MM:dd HH:mm:ss");

photoDir = new File("photos")

photoDir.eachFile { file ->

 Metadata metadata = JpegMetadataReader.readMetadata(file);

 com.drew.metadata.exif.ExifDirectory exifDir = metadata.getDirectory(com.drew.metadata.exif.ExifDirectory)

 DateTime original = new org.joda.time.DateTime(exifDir.getDate(ExifDirectory.TAG_DATETIME_ORIGINAL))

 String newOriginal = fmt.print(original.plusYears(1))

 DateTime digitized = new org.joda.time.DateTime(exifDir.getDate(ExifDirectory.TAG_DATETIME_DIGITIZED))
 String newDigitized = fmt.print(digitized.plusYears(1))

 String fn = "~/projects/photo/fixdate/photos/${file.name}"

 String exiv2OCmd = "set Exif.Photo.DateTimeOriginal String ${newOriginal}"

 String exiv2DCmd = "set Exif.Photo.DateTimeDigitized String ${newDigitized}"

 println exiv2OCmd
 Process p = new java.lang.ProcessBuilder("exiv2", "-M", exiv2OCmd, fn).start()
 p.errorStream.eachLine {  println it }

 Process q = new java.lang.ProcessBuilder("exiv2", "-M", exiv2DCmd, fn).start()
 q.errorStream.eachLine {  println it }


//might as well set the last modified date
 file.setLastModified(original.plusYears(1).toDate().time)

}

Groovy and GNOME, Part 2

2007-06-12T00:14:00.000Z

I've got an app up and running, though it doesn't do much.

I created a simple UI with Glade. There's a built in "Gnome App" widget, which contains the beginnings as a main application window - menu bar, status bar and an area to place more widgets.

With the glade file in hand I created 1 class and 1 script. The class is the backing for glade - it loads the glade file and handles it's events. I also added a show method to make the app display. At the moment, it only handles 2 events - the event from the File -> Close menu item and the delete-event, which is sent when you click on the X button on the right hand corner.


import org.gnu.glade.LibGlade;
import org.gnu.gtk.Gtk;
import org.gnu.gtk.event.GtkEvent;


public class TestApp {

 private static final String GLADE_FILE = "TestApp.glade"
 private LibGlade libglade;

 public TestApp() {
  libglade = new LibGlade(GLADE_FILE, this);
 }

 public void show() {
  libglade.getWidget("gnomeapp1").show()
 }

 private quitApp(int res) {
  Gtk.mainQuit()
  System.exit(res)  
 }
 
 public on_FileQuit(GtkEvent event) {
  this.quitApp(0)
 }

 public on_DeleteEvent(GtkEvent event) {
  this.quitApp(0)
 }
}

The script is a couple lines, enough to initialize Gnome (and, by extenision GTK), create an instance of the main application class, cause the main window to be shown and then enter the Gtk main loop


import org.gnu.gtk.Gtk;
import org.gnu.gnome.Program;

Program.initGnomeUI("TestApp", "0.1", args);
TestApp gui = new TestApp()
gui.show()

Gtk.main()

Groovy and GNOME, Part 1

2007-06-11T23:46:00.000Z

I've decided to write a small app in Groovy, but I plan to use Gtk/GNOME/Glade for the UI side of things.

So far, I've installed Glade (version 3) and libglade-java. It looks like libglade-java depends on libgnome-java, so I'm hoping Synaptic has pulled in all the needed dependencies to get up and running.

On Ubuntu, the jar files needed to start developing Gtk/GNOME apps live in /usr/share/java and need to get appended to the classpath. You also need to set LD_LIBRARY_PATH to include /usr/lib/jni as this is where the jni portion of the bindings live.

I just wrap all of this, plus the setting up of GROOVY_HOME and such like, into a bash script.

As I get on with the project, I'll post more

The GData API and "Web Content"

2007-06-08T23:29:00.000Z

As mentioned previously, recently I wrote a script to scrape the UV Index forecast for London off of the TEMIS website and then upload that data into Google Calendar as a bunch of events that have associated "web content", that way there will be a little coloured box on the top of each day, and hovering over the box will give the numerical value for the forecasted UV Index for that day.

It was, for the most part, fairly straight forward (even though I used the "raw" Google Data Java API and not the sugared API that's available for groovy). The biggest issue I ran into was that the API doesn't like WebContent that doesn't have a content URL set. According to the documentation at Google's Help Center, a content URL is optional. A content URL is the URL for content that will be displayed when the user clicks on the icon for the event, which isn't always necessary (such as with the weather forecast). However, trying to insert an event with WebContent using the Java API that doesn't have a content URL always resulted in 404 ResourceNotFound errors. Putting in a URL was the only way to get it working, so I quickly created an about.html that displays a copyright notice indicating that the data is from TEMIS (which is a good thing to have anyway).

The coloured boxes were whipped up with ImageMagick, the colours are based on the colours used in the wikipedia article on the UV Index. The coloured box that gets displayed is dependent on the forecasted UV Index.

The calendar can be found here. If you add it to your own set of calendars, you'll probably have to hit the refresh button to get the boxes to display.

One other thing, that I'm not sure is mentioned elsewhere, but the API needed the Java Mail API (javx.mail) on the classpath. So I had to download it from Sun.

HTML Screen Scraping With Groovy

2007-06-08T22:18:00.000Z

Recently, I wrote a script to scrape UV Index data from the Tropospheric Emission Monitoring Internet Service and then upload that data to Google Calendar as "Web Content". Here's how I went about scraping the data

I used Groovy's XmlSlurper to do the heavy lifting. XmlSlurper uses SAX underneath and, importantly, lets you choose a different SAXParser. As I wanted to parse HTML and not XML, I used TagSoup as my SAXParser.


slurper = new XmlSlurper(new org.ccil.cowan.tagsoup.Parser())

Everytime I run the script (with cron, overnight), I'm going to want to use the latest data, so I created a URL object that had the URL for the UV Index data for London, and then used groovy's "withReader" enhancement to read and parse the data


url = new URL("http://blah/blah?blah")

url.withReader { reader ->

    html = slurper.parse(reader)

       //we should now have a parsed file

    ...scrapeing code...

}

As the data I was looking for was conviently located in a table, all I had to do was find the path to the table (firebug comes in real handy here)


  tbl = html.body.table.tr.td.dl.dd.table

That gives a table and we can use a closure to iterate over the rows



  tbl.tr.list().each { row ->
  
   ... row parsing code ...

  }

Each row has a td list, so any particular cell of a row can then be accessed as row.td[X]. In order to get a row as a string, you'll need to use toString (so, to get the data of the first cell as a string, it's row.td[0].toString()).

I came across an interesting issue with the trim function when I was trying to parse the first column into a DateTime (using the Joda Time library. There were some non-breaking spaces in the String, and trim doesn't trim non breaking spaces, so I had to run a quick regular expression on the String to get rid of them


  ds = row.td[0].toString().replaceAll(/\xA0/ , {""})

So putting it all together (though without the Google API code to do the uploading)



slurper = new XmlSlurper(new org.ccil.cowan.tagsoup.Parser());

url = new URL('http://www.temis.nl/uvradiation/nrt/uvindex.php?lon=-0.07&lat=51.30')

url.withReader { reader ->
 html = slurper.parse(reader);

 tbl = html.body.table.tr.td.dl.dd.table

 tbl.tr.list().each { row ->

  if (row.td.size() == 3) {
   //trim doesn't work on a non breaking space
   ds = row.td[0].toString().replaceAll(/\xA0/, {""}).trim()
   uvi = row.td[1].toString().toFloat()

   //now do something with the date and uv index

  }
 }
}

There is one thing to note here -it does a quick check to make sure that there a 3 columns (date, uv index and ozone column), this is because there will be an extra row at the start of the table that contains the city name, if TEMIS know what the city name is for a set of co-ordinates.

I ran into a couple of niggles with the Google side of things, but that's probably best left to a different post

Welcome to Froth & Java

2007-06-08T11:07:00.000Z

Where I'm currently working, the catering company has recently changed, and the new company has brought in coffee cups with "Froth & Java" written on them. I though that would be a great name for a blog, and as I was already thinking of having a separate blog for Java and Groovy related things I thought I might as well start now. So here it is.

I might bring a few posts across from some of my old blogs, so if you see a post that's older than this welcome, then it's because that post pre-dates this blog.