Fixing Photo Dates

I've recently realized that I've been taking photographs with the right day and month, but the wrong year (2006, not 2007). This means that some of the EXIF photo metadata is wrong and I want to correct it for each file.

Reading EXIF appears easy under Java with Drew Noakes’s metadata-extractor but that library doesn't write EXIF data. There's the javax.imageio package, but that seems to be low level (ie, hard to use). I want a quick and dirty hack. So I'll use metadata-extractor to get the dates and call out to exiv2 to set the dates.

Of course, Java often brings up things you can stumble over, and this was getting the exiv2 command to run. I decided to use ProcessBuilder from java.lang (which, surprisingly, Groovy didn't seem to auto import), but it took a while to work out how to get it to launch an exiv2 command correctly (if at all). At a unix command line, in order to set the Original Date Time, you'd use something like:

exiv2 -M"set Exif.Photo.DateTimeDigitized String 2007:03:27 07:53:16" ./photos/HPIM0059.JPG

It took ages to work out that I had to split the -M from the set command and that I should not quote the set command, so that the call to ProcessBuilder is along the lines of

Process p = new java.lang.ProcessBuilder("exiv2", "-M", "set Exif.Photo.DateTimeOriginal String 2007:03:27 07:53:16", "./photos/HPIM0059.JPG").start()


Here's the code. Use at your own risk (remember, this script modifies files, so it could destroy something), and you'll proabably need to modify it to get it to work anyway.


import com.drew.metadata.*
import com.drew.imaging.jpeg.*
import com.drew.metadata.exif.ExifDirectory
import org.joda.time.*
import org.joda.time.format.*

DateTimeFormatter fmt = DateTimeFormat.forPattern("yyyy:MM:dd HH:mm:ss");

photoDir = new File("photos")

photoDir.eachFile { file ->

Metadata metadata = JpegMetadataReader.readMetadata(file);

com.drew.metadata.exif.ExifDirectory exifDir = metadata.getDirectory(com.drew.metadata.exif.ExifDirectory)

DateTime original = new org.joda.time.DateTime(exifDir.getDate(ExifDirectory.TAG_DATETIME_ORIGINAL))

String newOriginal = fmt.print(original.plusYears(1))

DateTime digitized = new org.joda.time.DateTime(exifDir.getDate(ExifDirectory.TAG_DATETIME_DIGITIZED))
String newDigitized = fmt.print(digitized.plusYears(1))

String fn = "~/projects/photo/fixdate/photos/${file.name}"

String exiv2OCmd = "set Exif.Photo.DateTimeOriginal String ${newOriginal}"

String exiv2DCmd = "set Exif.Photo.DateTimeDigitized String ${newDigitized}"

println exiv2OCmd
Process p = new java.lang.ProcessBuilder("exiv2", "-M", exiv2OCmd, fn).start()
p.errorStream.eachLine { println it }

Process q = new java.lang.ProcessBuilder("exiv2", "-M", exiv2DCmd, fn).start()
q.errorStream.eachLine { println it }


//might as well set the last modified date
file.setLastModified(original.plusYears(1).toDate().time)

}




Groovy and GNOME, Part 2

I've got an app up and running, though it doesn't do much.

I created a simple UI with Glade. There's a built in "Gnome App" widget, which contains the beginnings as a main application window - menu bar, status bar and an area to place more widgets.

With the glade file in hand I created 1 class and 1 script. The class is the backing for glade - it loads the glade file and handles it's events. I also added a show method to make the app display. At the moment, it only handles 2 events - the event from the File -> Close menu item and the delete-event, which is sent when you click on the X button on the right hand corner.


import org.gnu.glade.LibGlade;
import org.gnu.gtk.Gtk;
import org.gnu.gtk.event.GtkEvent;


public class TestApp {

private static final String GLADE_FILE = "TestApp.glade"
private LibGlade libglade;

public TestApp() {
libglade = new LibGlade(GLADE_FILE, this);
}

public void show() {
libglade.getWidget("gnomeapp1").show()
}

private quitApp(int res) {
Gtk.mainQuit()
System.exit(res)
}

public on_FileQuit(GtkEvent event) {
this.quitApp(0)
}

public on_DeleteEvent(GtkEvent event) {
this.quitApp(0)
}
}


The script is a couple lines, enough to initialize Gnome (and, by extenision GTK), create an instance of the main application class, cause the main window to be shown and then enter the Gtk main loop


import org.gnu.gtk.Gtk;
import org.gnu.gnome.Program;

Program.initGnomeUI("TestApp", "0.1", args);
TestApp gui = new TestApp()
gui.show()

Gtk.main()


Groovy and GNOME, Part 1

I've decided to write a small app in Groovy, but I plan to use Gtk/GNOME/Glade for the UI side of things.

So far, I've installed Glade (version 3) and libglade-java. It looks like libglade-java depends on libgnome-java, so I'm hoping Synaptic has pulled in all the needed dependencies to get up and running.

On Ubuntu, the jar files needed to start developing Gtk/GNOME apps live in /usr/share/java and need to get appended to the classpath. You also need to set LD_LIBRARY_PATH to include /usr/lib/jni as this is where the jni portion of the bindings live.

I just wrap all of this, plus the setting up of GROOVY_HOME and such like, into a bash script.

As I get on with the project, I'll post more

The GData API and "Web Content"

As mentioned previously, recently I wrote a script to scrape the UV Index forecast for London off of the TEMIS website and then upload that data into Google Calendar as a bunch of events that have associated "web content", that way there will be a little coloured box on the top of each day, and hovering over the box will give the numerical value for the forecasted UV Index for that day.

It was, for the most part, fairly straight forward (even though I used the "raw" Google Data Java API and not the sugared API that's available for groovy). The biggest issue I ran into was that the API doesn't like WebContent that doesn't have a content URL set. According to the documentation at Google's Help Center, a content URL is optional. A content URL is the URL for content that will be displayed when the user clicks on the icon for the event, which isn't always necessary (such as with the weather forecast). However, trying to insert an event with WebContent using the Java API that doesn't have a content URL always resulted in 404 ResourceNotFound errors. Putting in a URL was the only way to get it working, so I quickly created an about.html that displays a copyright notice indicating that the data is from TEMIS (which is a good thing to have anyway).

The coloured boxes were whipped up with ImageMagick, the colours are based on the colours used in the wikipedia article on the UV Index. The coloured box that gets displayed is dependent on the forecasted UV Index.

The calendar can be found here. If you add it to your own set of calendars, you'll probably have to hit the refresh button to get the boxes to display.

One other thing, that I'm not sure is mentioned elsewhere, but the API needed the Java Mail API (javx.mail) on the classpath. So I had to download it from Sun.

HTML Screen Scraping With Groovy

Recently, I wrote a script to scrape UV Index data from the Tropospheric Emission Monitoring Internet Service and then upload that data to Google Calendar as "Web Content". Here's how I went about scraping the data

I used Groovy's XmlSlurper to do the heavy lifting. XmlSlurper uses SAX underneath and, importantly, lets you choose a different SAXParser. As I wanted to parse HTML and not XML, I used TagSoup as my SAXParser.


slurper = new XmlSlurper(new org.ccil.cowan.tagsoup.Parser())


Everytime I run the script (with cron, overnight), I'm going to want to use the latest data, so I created a URL object that had the URL for the UV Index data for London, and then used groovy's "withReader" enhancement to read and parse the data


url = new URL("http://blah/blah?blah")

url.withReader { reader ->

html = slurper.parse(reader)

//we should now have a parsed file

...scrapeing code...

}


As the data I was looking for was conviently located in a table, all I had to do was find the path to the table (firebug comes in real handy here)


tbl = html.body.table.tr.td.dl.dd.table


That gives a table and we can use a closure to iterate over the rows



tbl.tr.list().each { row ->

... row parsing code ...

}



Each row has a td list, so any particular cell of a row can then be accessed as row.td[X]. In order to get a row as a string, you'll need to use toString (so, to get the data of the first cell as a string, it's row.td[0].toString()).

I came across an interesting issue with the trim function when I was trying to parse the first column into a DateTime (using the Joda Time library. There were some non-breaking spaces in the String, and trim doesn't trim non breaking spaces, so I had to run a quick regular expression on the String to get rid of them


ds = row.td[0].toString().replaceAll(/\xA0/ , {""})


So putting it all together (though without the Google API code to do the uploading)



slurper = new XmlSlurper(new org.ccil.cowan.tagsoup.Parser());

url = new URL('http://www.temis.nl/uvradiation/nrt/uvindex.php?lon=-0.07&lat=51.30')

url.withReader { reader ->
html = slurper.parse(reader);

tbl = html.body.table.tr.td.dl.dd.table

tbl.tr.list().each { row ->

if (row.td.size() == 3) {
//trim doesn't work on a non breaking space
ds = row.td[0].toString().replaceAll(/\xA0/, {""}).trim()
uvi = row.td[1].toString().toFloat()

//now do something with the date and uv index

}
}
}



There is one thing to note here -it does a quick check to make sure that there a 3 columns (date, uv index and ozone column), this is because there will be an extra row at the start of the table that contains the city name, if TEMIS know what the city name is for a set of co-ordinates.


I ran into a couple of niggles with the Google side of things, but that's probably best left to a different post

Welcome to Froth & Java

Where I'm currently working, the catering company has recently changed, and the new company has brought in coffee cups with "Froth & Java" written on them. I though that would be a great name for a blog, and as I was already thinking of having a separate blog for Java and Groovy related things I thought I might as well start now. So here it is.

I might bring a few posts across from some of my old blogs, so if you see a post that's older than this welcome, then it's because that post pre-dates this blog.