Posts from the Visualization Category

+ high-res version

Visualizing Follower Growth at Twitter

I worked with the Visual Insights team of Data Science and Analytics over at Twitter this year as a data visualization scientist. My work here largely involved analyzing and narrating large and interesting data sets (or colloquially ‘big data’) in an understandable manner to general audience visually.

I worked on two projects there –

World Cup Team followers

This was a project was shipped on the Twitter Interactive website. I made this because I was working at Twitter during the 2014 World Cup. This project involved doing research on what forms of visualization strategies work for large datasets which ended up being the choices that needed to be made. e.g. binning vs overlaying; saturation vs multiplicity etc. Follow the link to see the project –

VIT (Verified accounts) followers

This was a similar project to the one above but focused primarily on Verified users and their follower growth. I made an internal tool to help other teams work with this data both for online as well as print media.

Some of the images are obfuscated and taken out of context for privacy concerns.

Oakwood Beach Returns to the Wild

http://www.oakwoodbeachreturnstothewild.com/

At its peak, Hurricane Sandy, the largest Atlantic hurricane on record, spanned over 1,150 miles – roughly the driving distance between New York and Miami. As it began nearing the coastline, pouring down rain and rattling windows, schools closed, public transit shut down, and flood-zone residents evacuated their homes. The full force of the storm hit New York City on October 29, 2012, with winds gusting to 80 miles per hour.

Oakwood Beach, a low-lying hamlet on the south shore of Staten Island, was no match for the superstorm. By the time the winds and waters receded, leaving mud-filled homes in their wake, three residents were dead. Dozens more would be homeless for months.

During the year that followed, some residents chose to return and rebuild. But now, two years since the storm, the state is in the midst of implementing a buy-out program designed to convince all residents to leave their homes. Officials have closed on 276 buy-out sales in Oakwood Beach and two nearby towns, at a cost of $112 million; another 200 applications are pending.

In Oakwood Beach, once an oceanfront sanctuary for working class families, just a handful of stubborn holdouts remain. They live among empty lots and boarded windows; 47 neighborhood houses have already been demolished.

We visited Oakwood Beach around Sandy’s anniversary to talk to the remaining residents. They are surrounded: by nature, as animals and wetlands reclaim the land, and by government, as officials prepare for a world in which Oakwood Beach no longer exists. For now, the town sits in limbo.

Footage shot with DJI Phantom 2 and a GoPro Hero4 with a gimbal attachment.

Space Exploration on NYTimes graphic. + high-res version

Abstract ‘Space Exploration’ visualization created with data from NYTimes.

I love space exploration and our universe. So I created this quick graphic which is an abstract portrait of all the articles in The New York Times about ‘space explorations’. I also love typography so the occurrence lines are translated over the glyphs of Helvetica.

Background Image: The Milky Way; courtesy: NASA.

Vehicle Collision over time vs Instantaneous Traffic speeds.

This was just a small visualization I was experimenting with to show two contrasting data-sets on the same screen. The blue lines show the traffic speeds information (at the instant of 9/21/14 9:07pm) captured by various traffic speed detectors installed by the New York City DOT. In contrast, the red dots show the vehicle collisions that have happened in NYC over the last two years (2012-2014).

Data Sources:

1. data.cityofnewyork.us/Transportation/Real-Time-Traffic-Speed-Data/xsat-x5sa
2. data.cityofnewyork.us/NYC-BigApps/NYPD-Motor-Vehicle-Collisions/h9gi-nx95

about:memory for real people aims to convey memory consumption statistics to web-developers as well as normal users. This was a project I did for Mozilla but never ended up submitting it for the summer of code.

The current about:memory page is primarily used by Firefox developers for various debugging purposes. Though the information provided on the page is exhaustive and full of tech jargon (which is awesome if you understand it), but its hard to interpret for everyday users. The plan was to use various forms of data visualizations for expressing the exhaustive details in a concise manner to users, while giving power users the option to view all the unabridged data.

The primary objectives in this project were to –

  1. Express information and functionalities in the easiest possible manner.
  2. Keep the data unabridged.
  3. Make the new page itself consume as less memory as possible. (this was a bummer considering visualizations are very memory-philic).

Visualizing the memory tree

This is the current about:memory page –

Previous about:memory

Nick Nethercote and Felipe Gomes from Mozilla helped me understand the structuring of the page.

The new design uses 3 primary forms of data visualizations – donut charts, line charts and sunburst charts to account for both basic as well as verbose views. It is kept visually in sync with the about:healthreport page.

Features of the new design

HEADER

  • The new header allows users to switch between the Raw Data view and the new Memory Report view. The Raw data section would display a similar text based memory tree like the one on the current about:memory page. The Memory Report section is illustrated below.
  • The three buttons on the header in the top right corner, will account for GC (garbage collection), CC (cycle collection) and minimize memory usage functionalities.

KEY STATS AND FUNCTIONS

Refer to the left sidebar shown below.

  • Functionalities to load and save reports.
  • Combined statistics about memory consumption by Tabs, Add Ons and System. “System” here will be a bucket for all the memory compartments, allocations and listings that do not contribute to either Tabs or Add-Ons.
  • Functionalities to close inactive tabs (i.e. tabs that are still about:blank in the memory because of the “Do not load tabs until selected” feature).
  • Functionalities to disable add-ons, extensions and plugins.

THE GROUPED VIEW

Three main views – Grouped, Live and Verbose will be used to display the various visualizations.

In the Grouped view, users can see two major categories namely – the tabs and the Add-Ons. Donut charts are used to illustrate the same because they give an abstract view of the proportions of memory used by each tabs just like pie-charts. Hovering over a partition of the donut chart will reveal more information about the consumption statistics by that partition. In the case of a large number of tabs or add-ons, people can alternatively switch to a list-view just in order to view precise data related to all the tabs in a more convenient manner. The list view will also allow users to sort tabs in ascending or descending orders and also to selectively close tabs.

Grouped View for about:memory

Grouped view for about:memory

THE VERBOSE VIEW

The Verbose view is essentially a sunburst chart showing the details of the verbose memory allocation tree. A sunburst chart is a hierarchical form of a donut chart that allows users to drill down to see metadata about each partition. A sunburst chart is a visualization that allows comprehensive details to be accommodated in abstract forms in order to reduce the so-called cognitive friction . As above, hovering over the partitions of the chart will reveal more information about the consumption statistics.

Verbose view for about:memory

Verbose view for about:memory

THE LIVE VIEW

The Live View enables people to view real-time information about Firefox’s overall memory consumption. Line charts as shown below would be used for the same. The update frequency is arbitrarily chosen to be 1.5s.

Live view for about:memory

Live view for about:memory

From several posts on the MemShrink project blog, I observed that developers were keen to find out memory leaks due to pages/add-ons/extensions from the memory tree on about:memory. Though this project focuses more on everyday users, power users and developers can always benefit if there is an option of an advanced view that can point out possible memory leaks, identify and aggregate extensions’ compartments or highlight entities that consume disproportionately large amounts of memory or portray abnormal behavior.

Mike Bostock‘s D3.js was chosen to be used because of my previous experience with the same. Some of the graphs above are from the D3.js samples.