Consumer products often have great data that can be used to inform a perspective on trends and market sizing. Two basic scripts I have written for this purpose are outlined below.

Twitter

I frequently want to know how often a specific @handle is mentioned, or term is used, on Twitter. There are existing tools like Topsy that help answer this question, but I have never particularly liked any of them.

I ended up writing my own simple Ruby script to query the Twitter API for a given term. I then combined this with an R script so that I could graph the number of mentions a day for several terms1.

You can find the Ruby and bash scripts on Github

Yelp

Occasionally I want to know how many of a certain type of SMB there are in particular US cities. Yelp is one of the best free source of this data2, and using their API it is fairly trivial to set up a basic Ruby script to programatically get a count.

You can do this for a single search term in a single city using this script I wrote.

You could query for a variety of terms across many different US cities using a different script.

Footnotes

  1. It would be awesome if Twitter actually had Twitter Trends like Google Trends

  2. I haven’t used Factual, though I think it may be a more accurate data set, but you have to pay for it

Scripts for Research Using Twitter and Yelp was published on (revised: ) Miles Grimshaw

A Personal Map of NYC

Our digital activities create a personal data exhaust that traces a unique map of the world.Here's my personal trace of NYC based on [GPS...… Continue reading

MindBody's Market Penetration

Published on March 17, 2015

Replacing Email With Slack

Published on March 08, 2015