Blog CS A Repository of Things

Scraping Weather Data

We recently had a requirement for UK weather data. A little digging around revealed that hourly weather observation data is available, free of charge from the UK met office, via the uk gov open data site

However, this link above only gives one hour or results at a time, which is a little restrictive...

undefined

Then I took a look at the met office's own API (Datapoint), which is supposed to provide data for a given datetime for it's weather stations.Unfortunately, although I could get xml/json data feeds from the api, every time I polled for data before today, it returned 'no matching records'.

In the end I decided to write the following python script to loop through every day and hour I was interested in on the data gov site, and pull back the csvs into one folder so I could get the full dataset. 

Click here to see the code in gist or else click permalink below, where I have included the code.

Global Address Book Visualisation (Part I)

Ever wanted to mine the entire global address book and then present it as an interactive visualisation?

Me too!

First off, we need the data. The following VBA code can be run in Outlook to iterate through the whole address book and pull the data. The key fields here are really the alias and the manager - that's how we're going to build our hierarchy.

Click permalink below to see more

Home