Accessing BBC Listen Again data with YQL

The BBC have had for a while now XML feeds detailing their "Listen Again" content. For some reason, today I decided it would be fun to access that data from YQL.

The quickest way is to just use

select * from xml where url='http://www.bbc.co.uk/radio/aod/availability/radio4.xml'

However, I wanted to try out Open Data Tables, and this seemed like a good a time as any. Describing an Open Data Table is surprisingly easy. It's just an XML file, with some metadata about the table and then some binding information that tells YQL where to get the repeating data from.

<table xmlns="http://query.yahooapis.com/v1/schema/table.xsd">
  <meta>
    <author>Scot McSweeney-Roberts</author>
    <documentationURL>http://froth-and-java.blogspot.com/accessing-bbc-listen-again-data-with-yql</documentationURL>
    <sampleQuery>select entry from {table} where station="radio4" and entry.parents.parent.pid="b006qfvv"</sampleQuery>
  </meta>
  <bindings>
   <select itemPath="schedule.entry" produces="XML">
 <urls>
         <url env="all">http://www.bbc.co.uk/radio/aod/availability/{station}.xml</url>
 </urls>
 <inputs>
  <key id="station" type="xs:string" paramType="path" default="radio4" />
 </inputs> 
    </select>
  </bindings>
</table>

You'll have noticed the "station" key - the BBC has a separate feed for each service, so Radio 1's feed is at radio1.xml, Radio 4's is at radio4.xml, Radio Scotland is at radioscotland.xml, etc. By using a key and inserting its value into the select URL I can use a single Open Data Table definition and access whichever feed I want. The reason why I called it station and not service is that service is a child element of each entry element, so to keep things simple I called the key "station".

One reason for using an Open Data Table is that makes it slightly easier to query - for example, a query to select all Shipping Forecasts on Radio 4 without using an Open Data Table is

select * from xml where url="http://www.bbc.co.uk/radio/aod/availability/radio4.xml" and itemPath="schedule.entry" and parents.parent.pid="b006qfvv"

With an Open Data Table, it's

USE "http://sites.google.com/site/frothandjava/home/code/bbc_aod_open_table.xml"AS aod;
SELECT * FROM aod where station="radio4" and parents.parent.pid="b006qfvv"

(It's not really much of a difference though - I think table definitions come into their own when you use them to insert data in to a web service)

One of the more useful elements in the feed are the "mediaselector" links. A mediaselector link is a link to yet another XML file, but this file contains links to the available media for a particular programme. Some of the links aren't particularly useful, as they're internal iPlayer links, but some are useful as they're the links to WMA and Real Audio streams (though the BBC are dropping support for Real Audio at some point in the future).

Again, as it's XML it's easy to access from YQL

select * from xml where url='http://www.bbc.co.uk/mediaselector/4/mtis/stream/b00p6w2c'

(note, that query will probably not work after lunchtime on the 19th December, 2009 as that particular episode falls out of it's listen again window)

However, there is a bit of a problem at the moment for UK users of this data while accessing it through YQL - the BBC are currently sending a slightly different mediaselector file depending on where the request came from. As Yahoo's YQL servers are located outside the UK, we get the "international" version of the file and not the one containing UK data. Not only does this mean that we can't get access to the higher quality UK only versions of the streams, but I've noticed that the international WMA streams don't seem to be working in the UK (but the RA ones do). Apparently, this will be solved "at some point in the new year".