Skip to content

API Codex (previously API Alchemy)

One of my first projects at 3scale is API Codex – a centralized portal for great information about APIs around the web.  There are many fantastic articles, presentations and books available for people to learn about APIs, but nowhere for folks to find them all – until now!

I’m very excited about this, not just as a learning tool for the community, but also as a technical project – I used semantic mediawiki to create a site where you can explore the data space by tag, category or author, and we can add more ways to learn about the space based on date or other factors.  We’re looking forward to seeing how people use the site and adapting it to what people want and need.

We’ve got links to articles by industry leaders on API Design, hypermedia, business cases and tutorials. We’re always looking for more suggestions as well. If you’re wanting a quick intro to the API world, you can go directly to our API 101 page, which is a great jumping-off point to learning about the system.

Since we’re focused on APIs, I’m looking forward to adding sentiment analysis about the articles, using a link shortener API, and adding other ways to discover new great information in the space.

Take a look, contact us at @apicodex on twitter or

If you’re interested in talking to me about the semantic mediawiki experience and setup, let me know.  It’s a fantastic system and really allows for the development of an interesting semantic system.

Posted in API, Geek Stuff.


At the API Strategy and Practice conference, I gave a workshop on API 101 (slides).  I was lucky to have my actual mother in the audience so I could make “Even your mom can understand” jokes, which was great (I got her permission first, and kudos to her for letting us gently tease her in front of 150 strangers).  But also it gave me the opportunity to give a talk that even my (very smart, artistic, non-technical) mother could understand.  As I was working on the more technical pieces I realized that we have a self-referential problem frequently when trying to describe API Transactions.  We talk about parameters, headers, meta-information, requests and responses as if these terms were meaningful to a real person. Not really, no.  So, I submit the following thoughts for your consideration.

So first – what is an API? What is it like?  The easiest comparison I could come up with was the old time switchboard operator.  Your phone didn’t have to know how to get ahold of Aunt Mae, it just needed to know how to talk to the operator, and he or she would do what was necessary to connect you two.  An API is a computer version of this abstraction.

Now, for the juicy bits.  Transactions are conversations between computers.  This conversation can be boiled down to something that should amuse people who know me at all. Follow along while I describe the pieces of the HTTP Transaction like ordering an iced tea from Starbucks.

  • Resource ID: “Iced tea”
  • HTTP Verbs
    • POST: “I’d like to order an iced tea”
    • GET: “Can you read my order back to me?”
    • PUT: “Actually, make it larger”
    • DELETE: “Oh no, I left my wallet at home! Never mind”
  • Parameters: “Extra ice, unsweetened”
  • Headers: “For here or to go?”

Once you’ve walked through this conversation example you can get a little more technical – your browser uses headers to tell the server what language you speak.  Changes and substitutions are added to the name of the item.

I challenge my geeky friends to think of other ways to describe technical things without using technical terms that are meaningless to the general public.  What other examples can you think of?


Posted in Uncategorized.

Quantifying your Fitness

At OSCON last week, I gave a presentation on Quantifying your Fitness. It was well received, and most people were quite amused to see me wandering the conference with 5 fitness devices (6 if you count my iPhone). You’re welcome to check the slides out for more detailed information, but as I suspect most readers are in a hurry this post will be a summary of my findings. In some cases I don’t have all the data, so I’ll note that where appropriate and ask you, dear reader, to pass along the information if you have it. Please keep in mind that while some of these devices provide more or less information, any device that you will use and wear regularly will work great, even if it’s a simpler model. These summaries are just that, summaries. You can check the slides for more detailed information on any of these.

Ok, ready? Let’s go.


Bodymedia Fit

This is something of the gold standard of fitness devices at this time. The calorie estimates track very closely to chest-strap heart rate monitors and this device is approved by the FDA for use in clinical trials of various types. The skin temperature, moisture and three motion sensors collect 5,000 data points per minute, which they combine with some amazing backend algorithms to create a fantastic product. Price wise it’s reasonable, although the subscription price is a little daunting. The main drawback of this device is that it’s just really ugly – it must be worn on the upper arm and looks somewhat like an insulin pump. They are coming out with a new model this month (August 2013) which will look much better and also have an optional heart rate monitor – and it will be usable during swimming, which is a huge boost of coolness for this product. The website is somewhat limited – it will report your nutritional information, for instance, if you use a different application to monitor it; but it isn’t designed to be the input/tracking application for things other than the information from the device itself.


This is the one most people know – it’s a cute little device that you wander around with and it uses motion sensors to track your activity. The website is pretty darned good – you can log food, activities, mood, etc. and it will keep track of those items along with the sensor data. The API is quite strong, and has good developer support – so if you’re looking for a device/API to integrate with your cool game/application/motivational website this is a great choice. The device is easy to use, it syncs automatically when it sees the base or your phone, and it has a cute flower when you exercise. On the other hand, clipping it to your bra means you’ll wash it and have to buy a new one, so I’m on my 5th Fitbit at this point.

Jawbone Up

This device is a wristband which comes in funky colors. The headphone jack connector is supposed to be covered with a matching plastic cover, but I have kittens so as soon as I took the cover off it became one of their toys and I never saw it again. In any case, the motion sensors seem to do a decent job, but syncing by plugging the thing into the headphone jack of your phone just seems like too much effort to me (I’m lazy, so sue me :-) On the other hand Jawbone purchased Body Media recently so I expect great things out of them in the future. Their API is in beta right now, so you can’t really access your data yet… but I expect this to be fixed soon.

Withings Pulse

This device is brand new, and has gotten rave reviews. I’m not exactly sure why… it’s basically a fitbit with a few extra bells and whistles. It tracks your activity well, and has a spiffy touchscreen. You can manually check your heartrate (although I’ve found this to be somewhat twitchy) and tell it you’re going to sleep. You sync it by pushing the button on the device, which activates the bluetooth connection to your smartphone. Anyhow, it’s cool. I just don’t love it. And it doesn’t sync automatically which makes it less awesome than the fitbit. Also the website support and API interaction are pretty limited at this point, but I expect that they’ll get this fixed soon. The Withings App on smartphones does know about the pulse and tracks your activity pretty well.


I love this device. Yes, the charging cradle is a pain to use. The battery only lasts a few days, when I remember to turn off the bluetooth between syncs. But it tracks my resting heartrate every night, and the website insights and data are rockstar awesome. I love being able to look through my data how I want, and I’m looking forward to when they have an API to play with. Right now you have to get on a waitlist to get the product but I really, truly adore this device. Also, it’s a watch! Yes, it’s not exactly fashionable, but it tells the time and you don’t look like you’re wearing a medical device. The idea behind Basis is very Quantified Self-friendly – it watches your activity and then suggests habits you should try to attain – sometimes it’s going to sleep at a regular time, for some people it suggests getting up once an hour during work time. If you read out the habit suggestions in the right voice you realize that you’re strapping a Jewish mother on your wrist… not that that’s a bad thing!

The thing I love best about this device is something that’s not immediately obvious – although it isn’t great at tracking your heart rate while you’re exercising, it does a fantastic job of tracking your heart rate while you sleep. Anyone who has ever done heart rate training knows that your resting heart rate is one of the best measures of cardiac fitness – both as an indicator of general fitness (average) and a flag to let you know when you’ve overtrained, or you’re fighting sickness (or perhaps partied too hard the night before). Having this data tracked and recorded is incredibly useful, and this feature should be hailed much more widely.

Related Sites

Check out RunKeeper, who has a nice smartphone application for running/walking as well as a fantastic health graph API. If and when I create an awesome workout application, I’ll totally be using the Runkeeper API as my backend.


Bodymedia Fit Fitbit Jawbone Up Withings Pulse Basis
Price $150 + $7/month $69-$99 $69-$99 $99 $199
Heart Rate No No No No Yes
Yes No No No Yes
Sleep Auto Manual (button) Manual (button) Manual Auto
API Excellent Excellent Excellent Minimal None yet
Dashboard App + Website: Excellent App + Website: Excellent App + Website App only App: Minimal Website: Excellent
Charging Dongle/1 hour Dongle/1 hour Dongle/1 hour Dongle/ 1 hour Cradle (weird): 2 hours
Battery Life 3 days 7 days 7 days 7 days 3 days
Sync Tethered or Bluetooth (manual) Automatic when near phone, base – or manual via bluetooth Plug into smartphone (headphone jack) Manual via bluetooth Manual via bluetooth or when charging

Posted in Geek Stuff.

“Working” from Home

Telecommuting has been all the buzz this past week, with Yahoo eliminating work from home options for their fulltime telecommuters (but you can still wait for the cable guy!) and Best Buy following their lead soon after.  The articles covering these changes are quite interesting in their slant.

From Mashable, Best Buy’s announcement was spun this way:

Best Buy announced Monday that it has eliminated a flexible work program introduced in 2005 that gave corporate employees the freedom to work when and where they chose. Through the program, known as Results Only Work Environment, employees were assessed based solely on the quality of work they got done rather than whether or not they showed up to the office. Now, these employees will be expected to work a standard 40-hour workweek and come into the office “as much as possible.”

So let me get this straight.  The quality of work you do is less important to Best Buy than where you do it.  Seriously.

And in the New York Times article on Yahoo’s shift:

Parking lots and entire floors of cubicles were nearly empty because some employees were working as little as possible and leaving early.

Then there were the 200 or so people who had work-at-home arrangements. Although they collected Yahoo paychecks, some did little work for the company and a few had even begun their own start-ups on the side.

Does this strike anyone else as odd?  When you have employees who are not getting work done, you fire them.  Managers are supposed to be held responsible for the productivity of their groups.  If your managers can’t manage people except by counting the hours they’re in the office, then they should be fired as well.

I’ve been working for more than 20 years in an industry where I have to focus closely on my work sometimes, brainstorm sometimes, invent sometimes, and always be creative and alert.  Cube farms are soul sucking zombie cages where I just don’t produce nearly as much.  I am far more productive when working on my own schedule from where I want and I far exceed the output and effectiveness of most people I know.  I have a ransom note on top of my LinkedIn profile which somewhat arrogantly announces that I only want to work for people who want me to work for them.

I have fantastic skills, insight, and motivation.  I will do my best work and encourage and assist others to do the same.  I have experience in varied areas which allow me to complement those around me well.  Sitting in a seat in the office is not something I’m particularly good at, nor is it something I want to be measured on.

Best Buy and Yahoo are saying they need to rely on the same metrics used by retail establishments.  If you clock in and out on time and don’t annoy the customers, you’re doing well.  Your actual contributions to the company are not as important as your presence in the office.  How motivating is that? I can clock in, play scrabble, clock out and I should be fine.  Fortunately for my employer, that sounds horrendous to me.  I love contributing, fixing problems, helping people and getting things done.  Fortunately for me, that’s what they keep me around for.

A friend of mine pointed out how ironic it was that Marissa Meyer instituted this policy while spending the money and resources to have a nursery installed in her office at Yahoo.  Perhaps, he suggests, these struggling companies are simply trying to increase attrition to help fend off a future layoff.  If someone’s built their life structure around a particular work arrangement and has to move or find child care, they may just… leave.  And if they quit, you don’t have to fire them.  Win-win, I guess.

Posted in Uncategorized.

Happy Developers

I’ve always been all about removing barriers to help people get where they want to go.  It’s how I get gratification from my work, my play, my entire life.  This week at the amazing API Strategy and Practice Conference I’m surrounded by engaged, curious, challenging people who are committed to bringing the API world to the next level.  We’ve grown the API ecosystem organically up until now, all doing our best to meet our own goals, but being around people who have looked at this world from so many different directions makes it possible for us to see the patterns.

I spoke yesterday on a panel on API design – and I talked, as I frequently do, about happy developers. Jakub Nešetřil of made an insightful comment when he noted that we’re used to designing APIs for computers – they are, after all, programmatic interfaces. But in this world where developers are the interaction level, we have to make user interface choices. Developers are users of your API and if you don’t treat them as such they’ll be frustrated and less likely to be successful.

My talk, as always, was peppered with visual metaphors.  Developers are the wild animals out on the savannah, looking for a watering hole they like.  When they come to check out your API they’re giving you a valuable gift of something they’ll never get more of – their time.  Show them you value and respect that gift by communicating with them clearly about your product, your goals, and giving them ways to engage right away.  Play with them – provide tutorials and examples that give them a sense of progress (yes, level them up.  Game theory isn’t a bad thing.  Developers are, by nature, gamers.  Use that.)  Give them building blocks of working code, like legos so they can build new toys.  Play with your developers.  Seriously.

And support them.  Not just in the usual ways, but if you want to earn developer love, get them out there evangelizing your product, helping you support other users… teach them to be successful.  Give them tools, give them strategies, tell them how to solve the problems on their own so when they’re hacking at 3AM they can finish the toy they wanted to build.  Celebrate their successes with them because when you work with them, those successes are yours as well.

Your developers are users, yes, but they are also your peers.  They may have a different context or be coming to your system from a foreign programming world, but they want to interact with your product, and helping them to be successful is the best way to ensure the success of your own platform.

Posted in Geek Stuff.

Review: Intermediate Perl

As a programmer, I’m always looking for ways to take my coding practices to the next level. As most developers have, I’ve attended numerous conferences and sat through countless tutorials, only to find that while the topic is engaging during the talk, it doesn’t “stick” when I leave the room. The biggest exception to this was Randal Schwartz’ “Packages, References, Objects and Modules” – I attended this talk upwards of ten years ago, and it still stands out in my head as the biggest stepping stone to improve my maturity as a programmer.

This book, “Intermediate Perl” is the newest version of that curriculum, and it’s a fantastic way for someone who has become comfortable with perl but who finds themselves stuck copying other people’s code or doing things by rote because they don’t understand all the choices available to them. The book covers Perl’s reference system, which feels like black magic if you don’t understand it; it addresses ways to work with complex data and file handles, and advanced use of regular expressions. There’s also an excellent section on Perl’s OO functionality (such as it is) and testing.

As someone who has worked with perl for almost 20 years, who went through this class several years ago, I still found the book to be incredibly useful. I love Perl Best Practices, but that’s more a primer on doing things in a sensible and efficient way. This book takes the time to walk you through these concepts which are so central to Perl, from simple examples to more complex. Best of all, there are exercises at the back of each chapter which really require thought in applying the concepts just discussed. For me, this is the best way to really internalize new information – give me something to play with, some way to use it, and I’m much more likely to start using it in my own work.  Each exercise has a solution in the back of the book – and these solutions are thoughtfully explained so you understand why it works the way it does.

In short, “Intermediate Perl” is an excellent next step after “Learning Perl.”  I’d go so far as to say that it’s a good workbook for long-time Perl programmers to use – go through the exercises at the back of each chapter and see if there’s some way you can improve your coding, or something you’re doing inefficiently.  In the overwhelming sea of blue Perl books, this one provides a great set of exercises to take your development skills to the next level.



Posted in Uncategorized.

Asking Questions Well

One of the most frequent issues developers (and support people) have is that questions aren’t asked completely, and both people end up enmeshed in an iterative clarification cycle, unable to resolve the issue until both people understand what’s going on. There’s a decent FAQ on asking questions on StackOverflow, which does tell you to share details and context, but I’m going to go a little further and give you a template for asking questions which should help you get answers quickly.

What does a good problem statement look like?

I did X.
I expected Y to happen.
To my dismay, Z happened instead.

All three pieces are necessary, because frequently the problem is your expectation, not the system you’re trying to use. To clarify, let’s look at an example.

I jumped off a cliff.
I expected to sprout wings and fly across the ravine.
To my dismay, I plunged to my death instead.

Bummer. But the problem wasn’t really about gravity, it was that you expected something to magically happen to counteract it. Frequently users don’t understand how a system works, or what should be expected – this doesn’t make your question bad, by any stretch of the imagination. Part of supporting a product is making sure your users know what to expect, and the documentation may need updating. But the support person isn’t going to be able to help you if they don’t know why you think it’s broken. It may seem obvious to you what should happen, but trust me – more often than not including this piece will get you to your answer faster.

So, let’s go through this piece by piece…

I did X

When you’re explaining what you did, you need to include all the context you can.

  • What were you doing, exactly?  Can you reproduce the issue consistently?
  • Was it a web issue?  If so, which browser were you using?  Did you try other ones?
  • What exact URL or application were you using?  How can someone exactly recreate what you were doing?
  • If you’re coding, what language are you using?  Are you using a library?  Include an example of the specific code in question, with pointers to the library and examples you’re following (if applicable)
  • If you’re using a web service, what exactly did your request look like?  What were the headers and URL?  If you don’t know how to get those pieces, you need to know – read this article for some pointers.

I expected Y to happen

What did you hope to happen?  Lots of times with an error message or broken request, it’s obvious what you wanted to happen.  It’s still worth your while to include a quick statement.  But if you got a successful response that didn’t match what your expectation was, explain completely and clearly what you were expecting to get, and why.  Point to the documentation supporting your expectation (because it may well need an update or clarification).

To my dismay, Z happened instead

I try to always keep the “To my dismay…” part in there.  It amuses me, it amuses the person answering my question, and it acknowledges the truth of asking a question in this situation – I’m frustrated and sad that I’m stuck because something isn’t working how I want.

But just like with the “I tried X” part, you need to be very specific about what happened.

  • Did you get an error in a console or as a result? What did it say, exactly?
  • What did you try to get around the issue?
  • If you’re using a web service, what were the headers, status code, and/or body returned by the server
  • Note that if you are using a library that wraps a web service but asking for help from the API provider themselves, you likely need to dig down and see what the call itself looks like (as the API provider probably doesn’t support the third party libraries)

Following these guidelines will mean it takes a little longer for you to craft your question, but it will save you time, and most importantly, it will save people who might have the answer time. Posting incomplete questions wastes the time and energy of people who might otherwise help you (and they may not come back to see if you’ve added clarification). Showing them that you value their time and energy by completely explaining the issue will increase your chances of getting a complete answer – and if iteration cycles are needed they’ll be much more valuable.

Posted in Uncategorized.

LinkedIn Today SMS Notification System using LinkedIn, Twilio, Django and Heroku

At LinkedIn we enjoy a monthly Hackday, and I usually use the opportunity to figure out new ways to combine our platform with other systems – I love combining different APIs to create new functionality.

This month I decided to use my hackday time to create an SMS notification system for our LinkedIn today system. Since I’m not always on the website, I frequently miss the viral articles being discussed until they’ve passed into my history, and I’d like to be able to jump in on the conversation while it’s hot.  Using the LinkedIn Today API (currently only available internally), the system periodically checks each members feed ‘s see if there are any articles which are currently generating a huge amount of activity .  Once these “hot” articles are found, they are sent via SMS (using Twilio) to the user’s cell phone and recorded in the DB.  The member can reply to the SMS with “save” to add the article to their saved articles at LinkedIn, “share” the article with their LinkedIn network, change the notification level or cancel notifications entirely.
Welcome message Welcome message Welcome message
In order to build this system, I realized that I had to get several different systems to work together, and working through this process I realized that I had to solve several common problems.  This series of posts will be a tutorial on how to do the following:

These are building blocks that can be used for many similar projects, so I include them here to be an inspiration for future developers trying to get something working.  I enjoyed the resulting demo application enough that I ended up creating a web front-end to the system (so people could configure it) – you can get to the application at

Posted in Uncategorized.

Tagged with , , , , .

Handling SMS Responses from Twilio using Django on Heroku

The final piece in the application I’m working on requires that I process SMS replies from Twilio.  To do this, I need a public-facing server that can handle a POST request from another server so it can do the right thing on my end.  This tutorial will walk through how to do that with a Django server on Heroku.  You can grab the twilio_sms application from within my GitHub project for this application, or you can build it based on the walkthrough here.

URL Setup

Add a mapping for /reply/ to the in your top level Django project.

url(r'^reply', 'twilio_sms.views.sms_reply'),

You’ll need to tell Twilio where to send SMS replies on the dashboard.  For instance, my Heroku instance runs at so my SMS Response URL is set to

Creating twilio_sms

In order for that URL mapping to do the right thing it needs to point to the right application.  If you aren’t using the code samples from GitHub, you can create a new application with: startapp twilio_sms

Now we need to build the file to handle the responses.

The setup is similar to the linkedin application we made previously.

# Python
import oauth2 as oauth
import simplejson as json
import re
from xml.dom.minidom import getDOMImplementation,parse,parseString

# Django
from django.http import HttpResponse
from django.shortcuts import render_to_response
from django.http import HttpResponseRedirect
from django.conf import settings
from django.views.decorators.csrf import csrf_exempt
from django.contrib.auth.models import User

# Project
from linkedin.models import UserProfile,SentArticle

# from
consumer = oauth.Consumer(settings.LINKEDIN_TOKEN, settings.LINKEDIN_SECRET)
client = oauth.Client(consumer)

We’ll create a convenience method to build SMS responses in the format expected by the Twilio server (so that the user gets a reasonable SMS response)

def createSmsResponse(responsestring):
	impl = getDOMImplementation()
	responsedoc = impl.createDocument(None,"Response",None)
	top_element = responsedoc.documentElement
	sms_element = responsedoc.createElement("Sms")
	text_node = responsedoc.createTextNode(responsestring)
	html = responsedoc.toxml(encoding="utf-8")
	return html

Because the POST will be coming from an external server without a session cookie, we need to use the @csrf_exempt decorator to tell Django to allow these POSTs without authentication.  For security, you might check the incoming IP to make sure it’s coming from Twilio, or make sure that the other information matches what you expect.  For this demo, we’ll allow it to proceed assuming it’s the right thing.

Grab the parameters, get the user’s phone number, and determine which of our users matches that phone number, then grab their credentials and create the LinkedIn client to make requests.

def sms_reply(request):
    if request.method == 'POST':
        params = request.POST
        phone = re.sub('\+1','',params['From'])
        smsuser = User.objects.get(userprofile__phone_number=phone)
        responsetext = "This is my reply text"
        token = oauth.Token(smsuser.get_profile().oauth_token, smsuser.get_profile().oauth_secret)
        client = oauth.Client(consumer,token)

Figure out what the user wants us to do (save, search, cancel, help, level)

commandmatch = re.compile(r'(\w+)\b',re.I)
        matches = commandmatch.match(params['Body'])
        command =

“Cancel” tells the system the user doesn’t want notifications anymore.  For now, we’re going to keep their user and profile around, so that we don’t send them all the same articles again in the future. But won’t send them anything if the level is set to zero.

        # Cancel notifications by setting score to zero
        # Command is 'cancel'
        if command == 'cancel':
        	profile = smsuser.get_profile()
        	profile.min_score = 0
        	return HttpResponse(createSmsResponse("Today SMS Service Cancelled"))

“Level <number>” tells the system the user wants to change their notification level.  Higher means fewer messages, as this is used as the “score” check against articles based on relevance.  See the previous post on Twilio notifications to see how this is implemented.  Notice that in this and the following methods we’re doing generic error catching – there’s a few reasons why it might fail, but the important thing is to tell the user their action didn’t succeed and give them a hint as to why that might be.

        # Change level for notifications by setting score to requested level
        # Command is 'level \d'
        if command == 'level':
        	levelmatch = re.compile(r'level (\d)(.*)',re.I)
        	matches =['Body'])

	        	level = int(
	        	e = sys.exc_info()[1]
	        	print "ERROR: %s" % (str(e))
	        	return HttpResponse(createSmsResponse("Please use a valid level (1-9)."))

        	profile = smsuser.get_profile()
        	profile.min_score = level
        	return HttpResponse(createSmsResponse("Today SMS minimum score changed to %d" % int(level)))

“Save <article number>” saves an article to the user’s LinkedIn saved articles.  Remember that in the setup we grabbed the credentials for the user who sent the SMS based on their phone number, so this (and share) are done against the LinkedIn API on their behalf.  In this new (preview only) API JSON doesn’t seem to be working well, so I’m building and using XML.

        # Save an article
        # Command is 'save <articlenum>'
        if command == 'save':
        	savematch = re.compile(r'save (\d+)(.*)',re.I)
        	matches =['Body'])
	        	article =
        		sentarticle = SentArticle.objects.get(user=smsuser, id=article)
	        	e = sys.exc_info()[1]
	        	print "ERROR: %s" % (str(e))
	        	return HttpResponse(createSmsResponse("Please use a valid article number with save."))

        	responsetext = "Saved article: %s" % (sentarticle.article_title)
        	saveurl = ""

        	# Oddly JSON doesn't seem to work with the article save API
        	# Using XML instead
        	impl = getDOMImplementation()
        	xmlsavedoc = impl.createDocument(None,"article",None)
        	top_element = xmlsavedoc.documentElement
        	article_content_element = xmlsavedoc.createElement("article-content")
        	id_element = xmlsavedoc.createElement("id")
        	text_node = xmlsavedoc.createTextNode(sentarticle.article_number)
        	body = xmlsavedoc.toxml(encoding="utf-8")

        	resp, content = client.request(saveurl, "POST",body=body,headers={"Content-Type":"text/xml"})
        	if (resp.status == 200):
        		return HttpResponse(createSmsResponse(responsetext))
        		return HttpResponse(createSmsResponse("Unable to save post: %s" % content))

“Share <article number> <comment>” shares an article to the user’s network with a comment.  The comment shouldn’t really be optional, but typing on T-9 keyboards is a pain, so I wanted to give a default share message.  I’m not sure I love it as an answer though…

        # Share an article
        # Command is 'share <articlenum> <comment>'
        # If no comment is included, a generic one is sent
        if command == 'share':
        	sharematch = re.compile(r'Share (\d+) (.*)')
        	matches =['Body'])
        		article =
	        	sentarticle = SentArticle.objects.get(user=smsuser, id=article)
        		comment =
	        	if sentarticle and not comment:
	        		comment = "Sharing an article from the LinkedIn SMS System"
	        		e = sys.exc_info()[1]
	        		print "ERROR: %s" % (str(e))
	        		return HttpResponse(createSmsResponse("Please use a valid article number with share and include a comment."))

        	responsetext = "Shared article: %s" % (sentarticle.article_title)
        	shareurl = ""
        	body = {"comment":comment,

        	resp, content = client.request(shareurl, "POST",body=json.dumps(body),headers={"Content-Type":"application/json"})
        	if (resp.status == 201):
        		return HttpResponse(createSmsResponse(responsetext))
        		return HttpResponse(createSmsResponse("Unable to share post: %s" % content))

If we’ve fallen through to here, the user may have asked for ‘help’ – but whatever they did we didn’t understand it so we should give them the help text in any case.

       # If command is help, or anything we didn't recognize, send help back
        helpstring = "Commands: 'cancel' to cancel Today SMS; 'level #number#' to change minimum score;"
        helpstring += "'save #article#' to save; 'share #article# #comment#' to share"
        return HttpResponse(createSmsResponse(help string))

… and, if the request wasn’t a POST, send a 405 response back to the system (it won’t be Twilio, it might have been someone else).  This URL is only for processing these SMS messages.

    # If it's not a post, return an error
    return HttpResponseNotAllowed('POST')

Posted in API, Geek Stuff, LinkedIn.

Tagged with , , , , .

Sending SMS Notifications using Twilio, Django and the Heroku Scheduler

For my Twilio SMS Demo App this month, I wanted to send notifications to LinkedIn members based on hot items for them in LinkedIn Today.  This is a tutorial on creating a python script to access user information in the Django database, pull information from a Web Service (in this case the LinkedIn Today API) and send an SMS to the user.  A couple of caveats are in order here to make sure that people don’t accidentally run afoul of the LinkedIn API Terms of Use when implementing something of this sort.

  • The LinkedIn Today API is only available internally at this point, and should only be used as an example of what’s possible
  • I have stored the user’s phone number in the database – when storing LinkedIn information you always need to notify the user that you’re doing that, explicitly.
All that having been said, you need the following to get this working:
  • A working Django system on Heroku.  If you set up my example LinkedIn Django application from this post, you’ll be able to extend that here.
  • A Twilio account – ideally with a purchased number, but you could get this system to work with the sandbox system.  You get $30 in Twilio credit to start out with, which is plenty to set up a demo of this sort.
  • A Google API key (this is optional, but it’ll get you more shortened links)

Install the Twilio Library

Assuming that you’re using the LinkedIn Django application from my previous post and are in the root level of that structure, you need to add the Twilio library to your installation:
% pip install twilio
% pip freeze > requirements.txt

Adding the Phone Number

Remember, if you’re going to store any information at all about a LinkedIn member, you need to explicitly tell them so.  That having been said, you can add the following to your LinkedIn application to get the phone number and store it.  I’m not going to provide the completed code to do this since you need to implement some user notification if you’re using this in a production system.


class UserProfile(models.Model):
    user = models.ForeignKey(User)
    oauth_token = models.CharField(max_length=200)
    oauth_secret = models.CharField(max_length=200)
    phone_number = models.CharField(max_length=10)


Modify the LinkedIn profile URL:

url = ",first-name,last-name,industry,phone-numbers)"

Add this logic before saving the user profile:

        if "phoneNumbers" in profile:
        	for phone_number in profile['phoneNumbers']['values']:
        		userprofile.phone_number = re.sub(r"\D","",phone_number['phoneNumber'])
        		if phone_number['phoneType'] == 'mobile':

If you’ve already created your DB you’ll need to wipe it out and recreate it with these new models in order for this to work.  You can most easily do this by just starting a new project for this version – while you can drop the database locally and recreate it with syncdb, for heroku you’ll need to create a new cedar project so a new database will be created.

Creating the Job

Now we’ll walk through the code.  This file wants to live at your LinkedIn project level (so in the example, in the hellodjango directory).  It’s also available in the GitHub repository for django-linkedin-simple.

First, do some basic setup:

from import setup_environ
import settings
from import TwilioRestClient
import oauth2 as oauth
import time
import simplejson
import datetime
import httplib2
import psycopg2

token = "TWILIO_TOKEN"
twilioclient = TwilioRestClient(account, token)

consumer = oauth.Consumer(
url = ",id,topic-stories:(topic-articles:(relevance-data,article-content:(id,title,resolved-url))))"
Grab the user profiles (this includes the user tokens and secrets)
users = User.objects.all()

Iterate over the users, grabbing their token, secret, django userid and phone number, then grab the LinkedIn Today feed for that user.

for djangouser in users:
	if djangouser.username == "admin":
	profile = UserProfile.objects.get(user=djangouser)
	token = oauth.Token(
	phone = profile.phone_number
	userid = profile.user_id
	user_articles = SentArticle.objects.filter(user=djangouser)

	# Now make the LinkedIn today call and get the articles in question
	client = oauth.Client(consumer, token)

	resp, content = client.request(url, headers={"x-li-format":'json'})
	results = simplejson.loads(content)

We only want to send alerts for articles which have a high “score”, indicating that they are currently hot topics for this user.  Look through the articles to find those “hot topics”:

	for topic in results['values']:
           for story in topic['topicStories']['values']:
                for article in story['topicArticles']['values']:
                        score = article['relevanceData']['score']
                        if score > 6:

Now, SMS messages are expensive for us (the application owner) and potentially for the user, so we only want to tell the user if they haven’t already heard about this article.

checkarticle = user_articles.filter(article_number__exact=article['articleContent']['id'])

If there aren’t any matching articles for this user, go ahead and send them a message, then log it in the database so we don’t tell them again later:

if len(checkarticle) == 0:
    # This is where we get the shortened URL from google because LinkedIn doesn't provide one
    http = httplib2.Http()
    body = {"longUrl": article['articleContent']['resolvedUrl']}
    resp,content = http.request("","POST",body=simplejson.dumps(body),headers={"Content-Type":"application/json"})
    googleresponse = simplejson.loads(content)
    sentarticle = SentArticle(article_number=article['articleContent']['id'],user=djangouser,
    bodytext = article['articleContent']['title'] + " " + googleresponse['id']
    bodytext += " ('save %s')" %
    message = twilioclient.sms.messages.create(to="+1" + phone, from_="+YOUR_TWILIO_NUMBER", body=bodytext)

Great, we’ve got an application that’ll send messages for the users in the Django system when something hot is there to talk about.

Pushing to Heroku

Now we need the job to work on Heroku, and get scheduled.  If you went through my previous post using the existing GitHub example, the file is already up at Heroku, otherwise you’ll need to push the file to your Heroku instance:

% git add hellodjango/
% git push heroku master

Whether it was already there or you just put it there, you should check to make sure that it runs correctly.  Make sure you’ve logged into the system using your LinkedIn ID and stored your phone number there.

% heroku run python hellodjango/

Did it work?  Great!

Schedule it in Heroku

The Heroku Scheduler is a free add-on, but remember that since it uses machine time it can start racking up the charges if you don’t keep an eye on it.  You can add it using the command line, or you can add it under “Resources” for your application on the Heroku site.  Once you’ve added it you can manage it using the scheduler dashboard.

Pick whatever frequency you like, and put “python hellodjango/” in the field, and you’re done.

Don’t forget to stop the scheduler when you’re no longer testing so you don’t get a surprise bill from Heroku later.

The next post will cover handling POST requests from Twilio in response to SMS replies from your users.

Posted in API, Geek Stuff, Heroku, LinkedIn, Python.

Tagged with , , , , .