Blog CS A Repository of Things

Scraping Weather Data

We recently had a requirement for UK weather data. A little digging around revealed that hourly weather observation data is available, free of charge from the UK met office, via the uk gov open data site

However, this link above only gives one hour or results at a time, which is a little restrictive...

undefined

Then I took a look at the met office's own API (Datapoint), which is supposed to provide data for a given datetime for it's weather stations.Unfortunately, although I could get xml/json data feeds from the api, every time I polled for data before today, it returned 'no matching records'.

In the end I decided to write the following python script to loop through every day and hour I was interested in on the data gov site, and pull back the csvs into one folder so I could get the full dataset. 

Click here to see the code in gist or else click permalink below, where I have included the code.

SonArtPi (Part II: The Application)

Sonart is my project to control Sonos / Spotify directly via a camera and album covers.

I got the application up and running last night, and it works a charm. I have just pushed the initial code to github, and you can find it here:

https://github.com/willycs40/sonart/tree/master

I plan to make some updates to clean up the code, and also add some scripts for viewing and updating the record library. But for now, it works a charm, and I put on a playlist this morning using it!

SonArtPi (Part I: The Setup)

Sonart is my project to control Sonos / Spotify directly via a camera and album covers.

Since building the POC (see my last post above), I wanted to make this a permanent capability for my Sonos set up. I decided to build a 'production' version, on top of Raspberry Pi hardware. This post is about how I got the pi fully set up for computer vision.

I chose a Raspberry Pi 2 (as I'll need as much processing power as possible to process the images quickly), and also bought the official Pi Camera, and a case that could hold the whole thing. After having assembled the Pi, camera and case, and plugging in all the relevant cables, I powered it up and installed Raspbian through Noobs, making sure to enable the camera module in raspi-config.

From there I started to follow various guides to install and test the camera, get SimpleCV installed and deploy my Sonart application. I'd like to say things worked perfectly, but unsurprisingly I came up against a number of issues! In this post I've detailed those issues, and how I managed to fix them. I've also included the happy-path code for anyone who wants to avoid the problems!

Click permalink below to see the rest!

Sonart (controlling Sonos and Spotify using album art)

I have Sonos speakers in my flat. In combination with Spotify, this allows me to play just about whatever music I want, in whichever rooms of the flat, simply by opening the Sonos app on my phone and searching for the artist. The biggest challenge is deciding what to listen to.

I also have a record player in my flat, and a number of vinyl records, in nice sleeves with nice cover art. Even though putting on a record is much less convenient than opening Sonos on my phone, I still get a lot from having them around - primarily the ability to wonder over and flick through a big pile of album art to help decide what I want to listen to: something which is lost by music streaming services.

I had an idea for how I could combine the two; keep the convenience and low cost of music streaming, but also retain a physical presence for my music collection (in my case a pile of records).

The solution is built using a webcam and some Python code (using SimpleCV, SoCo libraries) - take a look at it in action:

Click permalink below to get to the code...

GDAL steps to convert Northings and Eastings to WGS84

I have written before about geospatial data types and the open source libraries which can be used to manipulate them.

Today, I needed to take some geospatial data which was in a database with northing and easting fields (basically ordinance survey gridpoints), and convert it to a WGS-84 coordinate system (latitude, longitude used by most GPS systems these days). This is so that I could spatially match it to another data-set. 

The conversion took a few steps and had a couple of 'gotchas', so I've documented it here for prosperity. Although I'm converting from SRID 27700 to 4326 here, you can of course use the same steps to convert between other types.

Click permalink to see the code...

Home ← Older posts