TEDx Boulder 2011

TEDxBoulder is returning tomorrow night with a brand new theme: Time and Change. This year, a few of us at Quick Left were privileged enough to hack on a cutting edge data visualization and progressive website, showcasing the event.

Go To TEDxBoulder Website »

(Data) Change Over Time

Good Apples presented the idea of visualizing user interaction with the web site over an interval of time. We were able to create some interesting graphics using cutting edge technology, great image manipulation libraries, and a bit of math. Here’s the breakdown:

Website: Sam set up a Node.js application to serve the web site on Heroku. He implemented JavaScript that builds up user data and caches it into a Redis store over time. Rylan took Good Apple’s awesome site design and breathed life into it using CSS3 transitions, JavaScript optimizations, and advanced fluid layout techniques.

Data: After data collection began, I created some code that renders SVG images representing averaged data outputs using key attributes from each user’s interaction (details below). Using Mike Bostock’s d3 JavaScript library, we were able to add rich animations that showed visual changes in the data set.

Result: One month passed, and our data set grew to over one million tiny pieces of info. I’ve always been a huge nerd for data manipulation, so I decided to take things a bit further and analyze the data at one hour intervals from the beginning to the end. I stumbled upon an interesting thing while playing with SVG’s: they are saved as text in XML format. This allowed the rendering script to output raw SVG files.

The idea was to extract one hour segments of the data set and output frames for each stage of the experiment. The code worked great, but V8 just couldn’t keep up with the amount of data being processed. It was time to move to Ruby. All the rendering code, along with some attribute adjustments was ported to ruby and data was pumped out into over 650 SVG files. The final SVGs were quite large, and we needed them in a static format to create a video, so I moved everything to my home server and let ImageMagick work overnight. A little bit of ffmpeg magic, and we ended up with this:

TEDx Boulder Change and Time from Quick Left on Vimeo.

Data Rendering

Here’s a breakdown of what each point actually represents:

radius = width / 10 * ( page + 1 ) theta = - ( 15 * time + 135 ) * toradians depth = ( datalength > 50 ? 50 : data_length ) / 30 point = { :d => depth, :x => radius * Math.cos( theta ) + width/2, :y => radius * Math.sin( theta ) + height/2 }

1. Radius from the origin reflects the portion of the application in which the user is interacting with. This computes the width divided by two, multiplied by one of 5 major portions and normalized to fit the canvas.

2. Theta represents the polar angular coordinate. One full rotation ( 360 degrees ) is split into 24 sections at 15 degrees each to represent each hour of the day. The computation is shifted so that midnight is at the bottom left position, then converted to radians in order to convert to cartesian coordinates.

3. Depth shows the user's interaction frequency. The amount of interaction captured is capped and normalized at 50 data points per day then scaled to fit on the canvas properly.

4. Calculus is used to convert each point from polar to cartesian coordinates and scaled properly to fit the canvas.

I’d like to give a huge shout out to Andrew Hyde for organizing this incredible event as well as Good Apples and the other teammates in our TEDxBoulder web crew: Sam Breed, Rylan Bowers, and Jessica Dillon.