Happy Dashboarding

Sprinkle some Magic in your Klipfolio dashboard

Nowadays, the world of analytics and visualizations has evolved into a complex mesh of API’s and dispersed data sources. In the age of Big Data and Internet of Things, everything and every piece of data is susceptible of being reused and repackaged in new, creative ways.

Thanks to import.io you can create a custom API to easily extract data from (almost) any page or site1.

Use Case

As a seasoned analyst and co-moderator of the largest Google Analytics user community on Google Plus, I can’t live without data. I grew more frustrated that Google “forgot” to provide reports for popular posts, top contributors and the like, or an API to allow us to built reports on our own.

A perfect case for import.io and Klipfolio!

The Magic wand

Let’s start with a simple example. Suppose you want to discover the beautiful Québec-city region. You could browse TripAdvisor… or magically retrieve the data. Creating a new extractor with the new import.io Magic is as easy as counting to three two:

  1. Head over to https://magic.import.io/
  2. Enter the URL where the data is, for example http://www.tripadvisor.ca/Attractions-g155033-Activities-Quebec_City_Quebec.html

It works like a charm! You now have details about top things to do in Québec-city, and you can click to the next page to get more. Import.io automatically detects data organized into tables and attemps to retrieve all the relevant information “automagically”. Note this feature is still in beta so in some cases, you might have to use their regular extractor in order to get what you want.

Import.io

Bring the Magic Home

Back to my use case - getting some data from our GA G+ community. Since it’s a little more advanced, we will switch to the Magic API and head over to Klipfolio to create a new Data Source.

All we have to do is specify the base URL of the Magic API (https://api.import.io/store/data/_magic), the URL we want to retrieve (the GA G+ Community at https://plus.google.com/u/0/communities/114481059214254340537). By default, the page will show only the latest 20 results, but we want to get at least 30 days back, which, based on the current volume of posts, would be about 10 pages of 20 posts. In order to do so, we add the infiniteScrollPages=10 and enable JavaScript (js=true) querystring parameters. When creating an import.io account you also get assigned a user and API key which needs to be added to the request.

Klipfolio supports several authentication methods, but in this case, it’s just a normal HTTP GET request which returns JSON data.

Klipfolio JSON format

Click on “Get Resource”, let import.io stir the magic potion for a few seconds. Voilà! You got your data served fresh!

Klipfolio JSON verification

Next steps

We now have access to the data we want. As I mentioned, import.io Magic is still in beta and at the time of writing, there’s no way to rename the attributes or edit an existing extractor (but word is this will be possible shortly). Under the hood, G+ community structure is very complex and therefore, the returned data is accordingly complex - but trust me, I tried other means and gave up, so I’m very impressed by what import.io have done!

In my next article I will share some tips & tricks I used to create the Klips that make good use of this data.


1 There are some exceptions, for example, LinkedIn decided to shut out developers by locking down its APIs and block all crawlers and tools such as import.io. Facebook and Twitter also impose severe restrictions.

About Stéphane Hamel

Innovator, speaker and renowned digital analytics consultant, Stéphane Hamel was named "Most Influential Industry Contributor" by the Digital Analytics Association in 2013. Strong of twenty-five years’ experience empowering organizations to analyze and optimize their online channels, he has cemented his position as a leading voice for digital analytics and online optimization.

Share