Princess Polymath

The final piece in the application I’m working on requires that I process SMS replies from Twilio.  To do this, I need a public-facing server that can handle a POST request from another server so it can do the right thing on my end.  This tutorial will walk through how to do that with a Django server on Heroku.  You can grab the twilio_sms application from within my GitHub project for this application, or you can build it based on the walkthrough here.

URL Setup

Add a mapping for /reply/ to the urls.py in your top level Django project.

urls.py

url(r'^reply', 'twilio_sms.views.sms_reply'),

You’ll need to tell Twilio where to send SMS replies on the dashboard.  For instance, my Heroku instance runs at http://falling-summer-4605.herokuapp.com so my SMS Response URL is set to http://falling-summer-4605.herokuapp.com/reply.

Creating twilio_sms

In order for that URL mapping to do the right thing it needs to point to the right application.  If you aren’t using the code samples from GitHub, you can create a new application with:

django-admin.py startapp twilio_sms

Now we need to build the views.py file to handle the responses.

The setup is similar to the linkedin application we made previously.

# Python
import oauth2 as oauth
import simplejson as json
import re
from xml.dom.minidom import getDOMImplementation,parse,parseString

# Django
from django.http import HttpResponse
from django.shortcuts import render_to_response
from django.http import HttpResponseRedirect
from django.conf import settings
from django.views.decorators.csrf import csrf_exempt
from django.contrib.auth.models import User

# Project
from linkedin.models import UserProfile,SentArticle

# from settings.py
consumer = oauth.Consumer(settings.LINKEDIN_TOKEN, settings.LINKEDIN_SECRET)
client = oauth.Client(consumer)

We’ll create a convenience method to build SMS responses in the format expected by the Twilio server (so that the user gets a reasonable SMS response)

def createSmsResponse(responsestring):
	impl = getDOMImplementation()
	responsedoc = impl.createDocument(None,"Response",None)
	top_element = responsedoc.documentElement
	sms_element = responsedoc.createElement("Sms")
	top_element.appendChild(sms_element)
	text_node = responsedoc.createTextNode(responsestring)
	sms_element.appendChild(text_node)
	html = responsedoc.toxml(encoding="utf-8")
	return html

Because the POST will be coming from an external server without a session cookie, we need to use the @csrf_exempt decorator to tell Django to allow these POSTs without authentication.  For security, you might check the incoming IP to make sure it’s coming from Twilio, or make sure that the other information matches what you expect.  For this demo, we’ll allow it to proceed assuming it’s the right thing.

Grab the parameters, get the user’s phone number, and determine which of our users matches that phone number, then grab their credentials and create the LinkedIn client to make requests.

@csrf_exempt
def sms_reply(request):
    if request.method == 'POST':
        params = request.POST
        phone = re.sub('\+1','',params['From'])
        smsuser = User.objects.get(userprofile__phone_number=phone)
        responsetext = "This is my reply text"
        token = oauth.Token(smsuser.get_profile().oauth_token, smsuser.get_profile().oauth_secret)
        client = oauth.Client(consumer,token)

Figure out what the user wants us to do (save, search, cancel, help, level)

commandmatch = re.compile(r'(\w+)\b',re.I)
        matches = commandmatch.match(params['Body'])
        command = matches.group(0).lower()

“Cancel” tells the system the user doesn’t want notifications anymore.  For now, we’re going to keep their user and profile around, so that we don’t send them all the same articles again in the future. But sendArticles.py won’t send them anything if the level is set to zero.

        # Cancel notifications by setting score to zero
        # Command is 'cancel'
        if command == 'cancel':
        	profile = smsuser.get_profile()
        	profile.min_score = 0
        	profile.save()
        	return HttpResponse(createSmsResponse("Today SMS Service Cancelled"))

“Level <number>” tells the system the user wants to change their notification level.  Higher means fewer messages, as this is used as the “score” check against articles based on relevance.  See the previous post on Twilio notifications to see how this is implemented.  Notice that in this and the following methods we’re doing generic error catching – there’s a few reasons why it might fail, but the important thing is to tell the user their action didn’t succeed and give them a hint as to why that might be.

        # Change level for notifications by setting score to requested level
        # Command is 'level \d'
        if command == 'level':
        	levelmatch = re.compile(r'level (\d)(.*)',re.I)
        	matches = levelmatch.search(params['Body'])

        	try:
	        	level = int(matches.group(1))
	        except:
	        	e = sys.exc_info()[1]
	        	print "ERROR: %s" % (str(e))
	        	return HttpResponse(createSmsResponse("Please use a valid level (1-9)."))

        	profile = smsuser.get_profile()
        	profile.min_score = level
        	profile.save()
        	return HttpResponse(createSmsResponse("Today SMS minimum score changed to %d" % int(level)))

“Save <article number>” saves an article to the user’s LinkedIn saved articles.  Remember that in the setup we grabbed the credentials for the user who sent the SMS based on their phone number, so this (and share) are done against the LinkedIn API on their behalf.  In this new (preview only) API JSON doesn’t seem to be working well, so I’m building and using XML.

        # Save an article
        # Command is 'save <articlenum>'
        if command == 'save':
        	savematch = re.compile(r'save (\d+)(.*)',re.I)
        	matches = savematch.search(params['Body'])
        	try:
	        	article = matches.group(1)
        		sentarticle = SentArticle.objects.get(user=smsuser, id=article)
	        except:
	        	e = sys.exc_info()[1]
	        	print "ERROR: %s" % (str(e))
	        	return HttpResponse(createSmsResponse("Please use a valid article number with save."))

        	responsetext = "Saved article: %s" % (sentarticle.article_title)
        	saveurl = "http://api.linkedin.com/v1/people/~/articles"

        	# Oddly JSON doesn't seem to work with the article save API
        	# Using XML instead
        	impl = getDOMImplementation()
        	xmlsavedoc = impl.createDocument(None,"article",None)
        	top_element = xmlsavedoc.documentElement
        	article_content_element = xmlsavedoc.createElement("article-content")
        	top_element.appendChild(article_content_element)
        	id_element = xmlsavedoc.createElement("id")
        	article_content_element.appendChild(id_element)
        	text_node = xmlsavedoc.createTextNode(sentarticle.article_number)
        	id_element.appendChild(text_node)
        	body = xmlsavedoc.toxml(encoding="utf-8")

        	resp, content = client.request(saveurl, "POST",body=body,headers={"Content-Type":"text/xml"})
        	if (resp.status == 200):
        		return HttpResponse(createSmsResponse(responsetext))
        	else:
        		return HttpResponse(createSmsResponse("Unable to save post: %s" % content))

“Share <article number> <comment>” shares an article to the user’s network with a comment.  The comment shouldn’t really be optional, but typing on T-9 keyboards is a pain, so I wanted to give a default share message.  I’m not sure I love it as an answer though…

        # Share an article
        # Command is 'share <articlenum> <comment>'
        # If no comment is included, a generic one is sent
        if command == 'share':
        	sharematch = re.compile(r'Share (\d+) (.*)')
        	matches = sharematch.search(params['Body'])
        	try:
        		article = matches.group(1)
	        	sentarticle = SentArticle.objects.get(user=smsuser, id=article)
        		comment = matches.group(2)
	        except:
	        	if sentarticle and not comment:
	        		comment = "Sharing an article from the LinkedIn SMS System"
	        	else:
	        		e = sys.exc_info()[1]
	        		print "ERROR: %s" % (str(e))
	        		return HttpResponse(createSmsResponse("Please use a valid article number with share and include a comment."))

        	responsetext = "Shared article: %s" % (sentarticle.article_title)
        	shareurl = "http://api.linkedin.com/v1/people/~/shares"
        	body = {"comment":comment,
        		"content":{
        			"article-id":sentarticle.article_number
       	 		},
        	"visibility":{"code":"anyone"}
        	}

        	resp, content = client.request(shareurl, "POST",body=json.dumps(body),headers={"Content-Type":"application/json"})
        	if (resp.status == 201):
        		return HttpResponse(createSmsResponse(responsetext))
        	else:
        		return HttpResponse(createSmsResponse("Unable to share post: %s" % content))

If we’ve fallen through to here, the user may have asked for ‘help’ – but whatever they did we didn’t understand it so we should give them the help text in any case.

       # If command is help, or anything we didn't recognize, send help back
        helpstring = "Commands: 'cancel' to cancel Today SMS; 'level #number#' to change minimum score;"
        helpstring += "'save #article#' to save; 'share #article# #comment#' to share"
        return HttpResponse(createSmsResponse(help string))

… and, if the request wasn’t a POST, send a 405 response back to the system (it won’t be Twilio, it might have been someone else).  This URL is only for processing these SMS messages.

    # If it's not a post, return an error
    return HttpResponseNotAllowed('POST')

For my Twilio SMS Demo App this month, I wanted to send notifications to LinkedIn members based on hot items for them in LinkedIn Today.  This is a tutorial on creating a python script to access user information in the Django database, pull information from a Web Service (in this case the LinkedIn Today API) and send an SMS to the user.  A couple of caveats are in order here to make sure that people don’t accidentally run afoul of the LinkedIn API Terms of Use when implementing something of this sort.

• The LinkedIn Today API is only available internally at this point, and should only be used as an example of what’s possible
• I have stored the user’s phone number in the database – when storing LinkedIn information you always need to notify the user that you’re doing that, explicitly.

Install the Twilio Library

% pip install twilio
% pip freeze > requirements.txt

Adding the Phone Number

Remember, if you’re going to store any information at all about a LinkedIn member, you need to explicitly tell them so.  That having been said, you can add the following to your LinkedIn application to get the phone number and store it.  I’m not going to provide the completed code to do this since you need to implement some user notification if you’re using this in a production system.

linkedin/models.py

class UserProfile(models.Model):
    user = models.ForeignKey(User)
    oauth_token = models.CharField(max_length=200)
    oauth_secret = models.CharField(max_length=200)
    phone_number = models.CharField(max_length=10)

linkedin/views.py

Modify the LinkedIn profile URL:

url = "http://api.linkedin.com/v1/people/~:(id,first-name,last-name,industry,phone-numbers)"

Add this logic before saving the user profile:

        if "phoneNumbers" in profile:
        	for phone_number in profile['phoneNumbers']['values']:
        		userprofile.phone_number = re.sub(r"\D","",phone_number['phoneNumber'])
        		if phone_number['phoneType'] == 'mobile':
        			continue

If you’ve already created your DB you’ll need to wipe it out and recreate it with these new models in order for this to work.  You can most easily do this by just starting a new project for this version – while you can drop the database locally and recreate it with syncdb, for heroku you’ll need to create a new cedar project so a new database will be created.

Creating the Job

Now we’ll walk through the code.  This file wants to live at your LinkedIn project level (so in the example, in the hellodjango directory).  It’s also available in the GitHub repository for django-linkedin-simple.

First, do some basic setup:

from django.core.management import setup_environ
import settings
setup_environ(settings)

from twilio.rest import TwilioRestClient
import oauth2 as oauth
import time
import simplejson
import datetime
import httplib2
import psycopg2

account = "TWILIO_ACCT_NUMBER"
token = "TWILIO_TOKEN"
twilioclient = TwilioRestClient(account, token)

consumer = oauth.Consumer(
        key="LINKEDIN_CONSUMER_KEY",
        secret="LINKEDIN_CONSUMER_SECRET")
url = "http://api.linkedin.com/v1/people/~/topics:(description,id,topic-stories:(topic-articles:(relevance-data,article-content:(id,title,resolved-url))))"

Grab the user profiles (this includes the user tokens and secrets)

users = User.objects.all()

Iterate over the users, grabbing their token, secret, django userid and phone number, then grab the LinkedIn Today feed for that user.

for djangouser in users:
	if djangouser.username == "admin":
		continue
	profile = UserProfile.objects.get(user=djangouser)
	token = oauth.Token(
        	key=profile.oauth_token,
        	secret=profile.oauth_secret)
	phone = profile.phone_number
	userid = profile.user_id
	user_articles = SentArticle.objects.filter(user=djangouser)

	# Now make the LinkedIn today call and get the articles in question
	client = oauth.Client(consumer, token)

	resp, content = client.request(url, headers={"x-li-format":'json'})
	results = simplejson.loads(content)

We only want to send alerts for articles which have a high “score”, indicating that they are currently hot topics for this user.  Look through the articles to find those “hot topics”:

	for topic in results['values']:
           for story in topic['topicStories']['values']:
                for article in story['topicArticles']['values']:
                        score = article['relevanceData']['score']
                        if score > 6:

Now, SMS messages are expensive for us (the application owner) and potentially for the user, so we only want to tell the user if they haven’t already heard about this article.

checkarticle = user_articles.filter(article_number__exact=article['articleContent']['id'])

If there aren’t any matching articles for this user, go ahead and send them a message, then log it in the database so we don’t tell them again later:

if len(checkarticle) == 0:
    # This is where we get the shortened URL from google because LinkedIn doesn't provide one
    http = httplib2.Http()
    body = {"longUrl": article['articleContent']['resolvedUrl']}
    resp,content = http.request("https://www.googleapis.com/urlshortener/v1/url?key=YOUR_GOOGLE_API_KEY","POST",body=simplejson.dumps(body),headers={"Content-Type":"application/json"})
    googleresponse = simplejson.loads(content)
    sentarticle = SentArticle(article_number=article['articleContent']['id'],user=djangouser,timestamp=datetime.datetime.today())
    sentarticle.save()
    bodytext = article['articleContent']['title'] + " " + googleresponse['id']
    bodytext += " ('save %s')" % sentarticle.id
    message = twilioclient.sms.messages.create(to="+1" + phone, from_="+YOUR_TWILIO_NUMBER", body=bodytext)

Great, we’ve got an application that’ll send messages for the users in the Django system when something hot is there to talk about.

Pushing to Heroku

Now we need the job to work on Heroku, and get scheduled.  If you went through my previous post using the existing GitHub example, the file is already up at Heroku, otherwise you’ll need to push the file to your Heroku instance:

% git add hellodjango/sendArticles.py
% git push heroku master

Whether it was already there or you just put it there, you should check to make sure that it runs correctly.  Make sure you’ve logged into the system using your LinkedIn ID and stored your phone number there.

% heroku run python hellodjango/sendArticles.py

Did it work?  Great!

Schedule it in Heroku

The Heroku Scheduler is a free add-on, but remember that since it uses machine time it can start racking up the charges if you don’t keep an eye on it.  You can add it using the command line, or you can add it under “Resources” for your application on the Heroku site.  Once you’ve added it you can manage it using the scheduler dashboard.

Pick whatever frequency you like, and put “python hellodjango/sendArticles.py” in the field, and you’re done.

Don’t forget to stop the scheduler when you’re no longer testing so you don’t get a surprise bill from Heroku later.

The next post will cover handling POST requests from Twilio in response to SMS replies from your users.

To build the LinkedIn Today SMS Notification System, I needed to find a framework where I could do the following:

• Integrate LinkedIn OAuth with a more general user management system
• Post the system to a public site
• Handle POST requests from external clients

For this set of criteria, I chose to use Django and publish it using Heroku.  This post is similar to the Django Getting Started Guide for Heroku, but it’s designed for this specific use case – Using LinkedIn OAuth as the login system and integrating it with the Django user management system.  The code for this example is available here.

Here are the prerequisites for this tutorial:

• Python, virtualenv, pip, and the Heroku toolbelt as described in Getting Started with Python on Heroku/Cedar.
• An installed version of Postgres to test locally
• A Postgres user and a database to use for Django (in this case I’m using ‘linkedin’)
• A LinkedIn API Key.  If you haven’t worked with the LinkedIn API before, you’re encouraged to go through the Quick Start Guide.

Create Your Django Environment

% mkdir hellodjango && cd hellodjango
% virtualenv --no-site-packages vent
% source venv/bin/activate
% pip install Django psycopg2 oauth2 simplejson
% django-admin.py startproject hellodjango
% cd hellodjango
% python manage.py startapp linkedin

Now that you’ve done all these pieces, you have a working django system.

File Setup

In order to add the LinkedIn integration, you’ll need to edit the settings and URL files.  I’ve created a GitHub repository with the code needed for this example, and you can use that as your model – but I’ll walk through the changes here so you understand how they work.

hellodjango/settings.py

Edit the DATABASES section to configure it to use postgres, and your database:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'linkedin',
        'USER': 'postgres',
        'PASSWORD': 'xxx',
        'HOST': '',
        'PORT': '5432',
    }
}

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'linkedin'
    # Uncomment the next line to enable the admin:
    # 'django.contrib.admin',
    # Uncomment the next line to enable admin documentation:
    # 'django.contrib.admindocs',
)

AUTH_PROFILE_MODULE= 'linkedin.UserProfile'
LINKEDIN_TOKEN='xxx'
LINKEDIN_SECRET='xxx'
LOGIN_URL='/login/'

url(r'^login/?$', 'linkedin.views.oauth_login'),
url(r'^logout/?$', 'linkedin.views.oauth_logout'),
url(r'^login/authenticated/?$', 'linkedin.views.oauth_authenticated'),
url(r'^$','linkedin.views.home'),

from django.db import models
from django.contrib.auth.models import User
class UserProfile(models.Model):
    user = models.ForeignKey(User)
    oauth_token = models.CharField(max_length=200)
    oauth_secret = models.CharField(max_length=200)

# Python
import oauth2 as oauth
import cgi
import simplejson as json
import datetime
import re

# Django
from django.http import HttpResponse
from django.shortcuts import render_to_response
from django.http import HttpResponseRedirect
from django.conf import settings
from django.contrib.auth import authenticate, login, logout
from django.contrib.auth.models import User
from django.contrib.auth.decorators import login_required

# Project
from linkedin.models import UserProfile

# from settings.py
consumer = oauth.Consumer(settings.LINKEDIN_TOKEN, settings.LINKEDIN_SECRET)
client = oauth.Client(consumer)
request_token_url = 'https://api.linkedin.com/uas/oauth/requestToken'
access_token_url = 'https://api.linkedin.com/uas/oauth/accessToken'
authenticate_url = 'https://www.linkedin.com/uas/oauth/authenticate'

# /login
def oauth_login(request):

    # Step 0. Get the current hostname and port for the callback
    if request.META['SERVER_PORT'] == 443:
     current_server = "https://" + request.META['HTTP_HOST']
    else:
     current_server = "http://" + request.META['HTTP_HOST']
     oauth_callback = current_server + "/login/authenticated"

    # Step 1. Get a request token from Provider.
    resp, content = client.request("%s?oauth_callback=%s" % (request_token_url,oauth_callback), "GET")
    if resp['status'] != '200':
        raise Exception("Invalid response from Provider.")
    # Step 2. Store the request token in a session for later use.
    request.session['request_token'] = dict(cgi.parse_qsl(content))
    # Step 3. Redirect the user to the authentication URL.
    url = "%s?oauth_token=%s" % (authenticate_url,
        request.session['request_token']['oauth_token'])
    print url
    return HttpResponseRedirect(url)

# / (requires login)
@login_required
def home(request):

    html = "<html><body>"
    token = oauth.Token(request.user.get_profile().oauth_token,request.user.get_profile().oauth_secret)
    client = oauth.Client(consumer,token)
    headers = {'x-li-format':'json'}
    url = "http://api.linkedin.com/v1/people/~:(id,first-name,last-name,headline)"
    resp, content = client.request(url, "GET", headers=headers)
    profile = json.loads(content)
    html += profile['firstName'] + " " + profile['lastName'] + "<br/>" + profile['headline']
    return HttpResponse(html)

# /logout (requires login)
@login_required
def oauth_logout(request):

    # Log a user out using Django's logout function and redirect them
    # back to the homepage.
    logout(request)
    return HttpResponseRedirect('/')

#/login/authenticated/
def oauth_authenticated(request):
    # Step 1. Use the request token in the session to build a new client.
    token = oauth.Token(request.session['request_token']['oauth_token'],
        request.session['request_token']['oauth_token_secret'])
    if 'oauth_verifier' in request.GET:
        token.set_verifier(request.GET['oauth_verifier'])
    client = oauth.Client(consumer, token)
    # Step 2. Request the authorized access token from Provider.
    resp, content = client.request(access_token_url, "GET")
    if resp['status'] != '200':
        print content
        raise Exception("Invalid response from Provider.")
    access_token = dict(cgi.parse_qsl(content))
    headers = {'x-li-format':'json'}
    url = "http://api.linkedin.com/v1/people/~:(id,first-name,last-name,industry)"
    token = oauth.Token(access_token['oauth_token'],
        access_token['oauth_token_secret'])
    client = oauth.Client(consumer,token)
    resp, content = client.request(url, "GET", headers=headers)
    profile = json.loads(content)    

    # Step 3. Lookup the user or create them if they don't exist.
    firstname = profile['firstName']
    lastname = profile['lastName']
    identifier = profile['id']
    try:
        user = User.objects.get(username=identifier)
    except User.DoesNotExist:
        user = User.objects.create_user(identifier,
            '%s@linkedin.com' % identifier,
            access_token['oauth_token_secret'])                 user.first_name = firstname
        user.last_name = lastname
        user.save()
        # Save our permanent token and secret for later.
        userprofile = UserProfile()
        userprofile.user = user
        userprofile.oauth_token = access_token['oauth_token']
        userprofile.oauth_secret = access_token['oauth_token_secret']
        userprofile.save()
    # Authenticate the user and log them in using Django's pre-built
    # functions for these things.
    user = authenticate(username=identifier,
        password=access_token['oauth_token_secret'])

    login(request, user)
    return HttpResponseRedirect('/')

web: python hellodjango/manage.py runserver "0.0.0.0:$PORT"

venv
*.pyc

% python manage.py syncdb
% python manage.py runserver

% git init
% git add .
% git commit -m "My LinkedIn Django App"
% heroku create --stack cedar
% git push heroku master
% heroku run python hellodjango/manage.py syncdb 
% heroku open

Neil Mansilla over at Mashery has created an awesome new system called IODocs, which can be used to create a system for browsing through OAuth-secured APIs. Being a Developer Advocate at LinkedIn, I decided to try and get this up and running with LinkedIn’s API.  Since it was LinkedIn’s internal hackday on Friday, I decided to go the extra mile and try to build the configuration dynamically from our nascent discovery API system, and run it on one of Heroku’s node.js hosted systems.

Thanks in part to a heavy piece of furniture (which fell on me on Friday, relegating me to bed rest for the weekend) I got it working. You can visit the LinkedIn API Documentation site and use your LinkedIn API key and secret to browse through the many API resources LinkedIn has to offer.

Getting the IODocs system running on Heroku required several steps, but once you have them it’s pretty straightforward. So I share them with you here so you can get IODocs running on your own Heroku instance, in the hopes that all of the OAuth APIs out there can leverage this functionality and improve developers’ lives.

Consider this an inclusive set of instructions to the ones they have on the site.

1. Get “Local Workstation Setup” done from http://devcenter.heroku.com/articles/node-js
2. Do the initial IODocs setup

   % git clone http://github.com/mashery/iodocs.git
   % cd iodocs/
   % npm install
   % echo "web: node app.js" > Procfile
   % cp config.json.sample config.json
   % vi config.json (remove address line)

3. Add the following block under “var db;” to app.js:

   if (process.env.REDISTOGO_URL) {
      var rtg   = require("url").parse(process.env.REDISTOGO_URL);
      db = require("redis").createClient(rtg.port, rtg.hostname);
      db.auth(rtg.auth.split(":")[1]);
   } else {
      db = redis.createClient(config.redis.port, config.redis.host);
      db.auth(config.redis.password);
   }
   
   And then this in the Load API Configs section, after reading the config file:
   
   var app = module.exports = express.createServer();
   
   var hostname, port, password
   
   if (process.env.REDISTOGO_URL) {
       var rtg   = require("url").parse(process.env.REDISTOGO_URL);
       hostname = rtg.hostname;
       port = rtg.port;
       password = rtg.auth.split(":")[1];
   } else {
       hostname = config.redis.host;
       port = config.redis.port;
       password = config.redis.password;
   }
   
   
   

4. Add a LinkedIn block to public/data/apiconfig.json:

   "linkedin": {
      "name": "LinkedIn",
      "protocol": "http",
      "baseURL": "api.linkedin.com",
      "publicPath": "",
      "privatePath": "/v1",
      "auth": "oauth",
      "oauth": {
         "type": "three-legged",
         "requestURL": "https://api.linkedin.com/uas/oauth/requestToken",
         "signinURL": "https://api.linkedin.com/uas/oauth/authorize?oauth_token=",
         "accessURL": "https://api.linkedin.com/uas/oauth/accessToken",
         "version": "1.0",
         "crypt": "HMAC-SHA1"
      },
      "keyParam":""
    },

5. Copy linkedin.json into your public/data folder
6. Set up your git repository for heroku

   % git init
   % git add .
   % git add -f config.json Procfile public/data/*
   % git commit -m "init"

7. Set up heroku

   % heroku create --stack cedar;
   % heroku addons:add redistogo [note: this may require that you add a credit card to your heroku account.  there is no charge, though]
   % heroku addons:add logging 
   % git push heroku master
   % heroku ps:scale web=1

Once you’ve gotten all this set up, you’re ready to rock.  Heroku will give you the right system name and port, and you can use ‘heroku logs’ to check the logs for the node.js server.

You can set up your own APIs using the config file – the IODocs documentation is fantastic, with OCD-level detail on how to set up configurations for new APIs, and I’m certain you can make it work with whatever you’re trying to use.

One of the best things about working as an engineer at LinkedIn is the monthly hackdays. This is an amazing opportunity that feels a lot, to me, like playing the lottery – it’s a time I can set aside to let myself imagine the possible, to think about what people might want to do, play with, or learn.  Think of better ways to accomplish the things that frustrate me, or to share things I love with other people.  Individuals and teams of LinkedIn employees take the day (and sometimes the weekend afterwards) to create mockups, prototypes, and sometimes complete applications to demonstrate the ideas bouncing around in their head.  Our company benefits by getting tons of new ideas, and our engineers benefit because… well, because play is fun, and this is play.  And we get to present our ideas to the executive team, and earn Apple gift cards for our efforts.

Not wanting to keep all this fun to ourselves, on July 29 LinkedIn will be hosting our first ever Intern Hackday – we’re offering cool prizes and free food, and the opportunity to collaborate with a bunch of other inspired and inspirational interns for 24 hours.

But that’s not all!  The open source developers among you know that OSCON is coming up at the end of the month, and Jeremy Johnstone and I will be there all week talking to developers and helping them started with the LinkedIn platform.  We’ve just teamed up with Mashery, Urban Airship, and a host of other fantastic companies to help put together the API Hackday PDX on Saturday, July 30.  I’ll be there to give a presentation on our APIs and help people with LinkedIn platform questions, and LinkedIn is going to give 3 $100 Apple Gift Cards to the best applications using our platform – whether it be brand new applications, multi-platform mashups, or integrations into existing systems. Come show us what you’ve got, get some LinkedIn shwag, learn about our platform and win some Apple dough!

If you want to get a jump on the competition, head over to our Developer Portal and start playing with our APIs. We’ve got a Javascript API and a REST API, along with some plugins you can use for quick integrations.  We’ve got cool tutorials, documentation, and a very active community in our forums.

Here are some links to code walkthroughs you might find helpful:

• Our faceted search enables you to make semantic applications – Browsing your LinkedIn Connections by Industry
• Our Javascript API is easy to use: How to Create a LinkedIn Application in 3 Easy Steps
• Our Javascript Tutorials have lots of helpful code walkthroughs, such as the Javascript API Search Example

Posted in API, Geek Stuff, JSAPI and LinkedIn here.  Posted in Friends and Open Source in G+.

Like many of you, I have a Twitter account, a LinkedIn account, a Facebook account, and I have my tweets go to LinkedIn and Facebook so that I can lazily update everyone when I have something to say.  The resulting stream ends up being pretty spammy and unfocused, and so I post much less frequently than I might.  My “regular” friends feel like I’m super spammy, and my “geeky” friends complain they never hear from me.  This is exactly the kind of problem that G+ circles should be able to solve.

For this reason, and also because I love mashing stuff up, I can’t wait until G+ releases their APIs.  I want to build (or use) integrations with other systems so I don’t spend all my life bouncing from social network to social network – or pretending to engage in them by tossing messages over from other systems.

I’d love to have a LinkedIn circle, a Twitter circle, and a Facebook circle, so when I share something that I intend for my LinkedIn audience I can just add them and it’ll appear in my LinkedIn stream.   Now that I have G+ I don’t spend much time at all on Facebook (although I do on LinkedIn) and I feel a little guilty .  G+ could add this integration themselves, or third party developers (like me) could create applications that do the shuttling from place to place.  I’d love to have one place to keep my stream – and decide what other streams should get any given update.  This would make me much more likely to participate actively, because I could protect my less-geeky friends from seeing updates about semantic web visualizations.  And honestly, it’ll be a long time before everyone I know is on G+ (if ever) so I do want to make sure my Farmville-addicted friends see my updates too.

I’d also love to see incoming integration.  For instance, Runkeeper integration would totally rock.  It already takes input from my FitBit, so everything’s in RunKeeper, so I could send those updates to my Fitness circle of people-who-actually-care-about-my-training.  My Runkeeper updates go to Facebook, but again, it’s kind of spammy.  I’m generally more ok with dumping everything to Facebook because the stream is so noisy to begin with, but I’d really rather target those messages to folks who have an interest in hearing about it.

So, G+, can we have some APIs soon?

Sharing this on G+ with Friends, APIs

Tagging here as Geek Stuff, Google +, API

Last night was the Silicon Valley iOS Developer Meetup hosted here at LinkedIn. I talked through a sample client for the iPhone which iOS developers can use as a springboard for creating their own LinkedIn-powered apps.

Slides can be found here.

The LinkedIn API has all sorts of juicy goodness to it, but the authentication and structure take some learning. LinkedIn has created a JS API called Connect to help you get around those things, so that you can focus on the logic and presentation of your application without spending too much time considering the back end.

I made an internal presentation covering this material at LinkedIn on 2/18, which is available in PDF format. This tutorial has also been integrated into the general JSAPI tutorials on the developer portal. That version has a lot less narration and is just about getting the code written and working.

Everyone loves activity streams, so to illustrate the application development process I’m going to walk through the steps needed to create an interactive LinkedIn application. StreamIn’ has a profile badge for the logged in member, a stream of the status updates from members in their network, and the ability to like/unlike particular items.

A note on this tutorial – there is detailed documentation for the JSAPI on the LinkedIn Developer Portal – there are links in the text appropriate, but detailed options aren’t covered here to keep the tutorial reasonably sized. It’s a lot more fun to get started in a framework if you’ve built something from start to finish, and that’s what we’ll do here.

Getting Started

To get started on our exercise, we need a shell for the application.  Here we have a basic layout, with places for the badge and stream to be inserted.

<div id="header"><h1>StreamIN'</h1></div>
  <div id="wrapper">
    <div id="profile">
	    <div id="badge">
	    Profile info goes here!
	    </div>
    </div>
    <div id="stream">
        Stream stuff goes here!
    </div>
    <br clear="both">
    <div id="footer">
    <div id="nav"> [ << ] <a href="Step1.html">[ >> ]</a></div>
</div>

We’ll start with this and build the application in stages.

Step 0: Check out the Console

The Connect framework does all of the authentication work for you, and loads the things you need in order to get things working in your application.  There are various ways to present login options for the member, and you can explore them using the new JSAPI Console.  At the very least you’ll want to explore the Login options (Login Button, Login Button with Events, Login Button Label). The console is an incredibly useful tool!  Take a few moments now to look around and see what it has to offer.  You can even edit the Code section and click “Run” to see how the Results change.

Step 1: Get the Profile

In order to show the user’s profile, we’ll need to do a few things.

• Import the framework
• Add a login button
• Add an API call to get the profile data and display it

I know, that seems like a lot of sub-steps, but it’s really very easy.

Import the Framework

The first thing we need to do when using Connect is get the framework in our page.  In the <head> of the document, we’ll add a small script to grab the framework.  To do this you need to get a LinkedIn API key – if you don’t have one, go ahead and get one and then come back – it doesn’t take long and it’s much more fun to play along than just read about what I did.

Here’s your code. Use your own api_key. “authorize:true” tells the framework to authorize the member if they’ve visited your application before, rather than logging in each time.

<script type="text/javascript" src="http://platform.linkedin.com/in.js">
   api_key: API_KEY
   authorize: true
</script>

Add a Login Button

Let’s add the button at the bottom of the page.  Here’s the code.  No really, this is all there is!  The data-onAuth variable tells the framework to fire the loadData function when the member has been authorized.

<script type="IN/Login" data-onAuth="loadData"></script>

Load the User’s Profile

Here’s the meat of the step – now the framework is in place, so it’s possible to make calls on behalf of the member. When the member has been authorized the loadData() function will be called. This function calls the Profile method. We need a couple of extra fields from that call, so we use .fields to tell the call what to request.  And the .result sets a callback to perform as soon as the call returns.  You can put an anonymous function there (as I have here) or give it the name of another function to process your returned data.

The callback in this case just pulls the user’s information from the result and builds a profile badge, inserting it in the badge <div>.

function loadData() {
// we pass field selectors as a single parameter (array of strings)
IN.API.Profile("me")
   .fields(["id", "firstName", "lastName", "pictureUrl","headline"])
   .result(function(result) {
      profile = result.values[0];
      profHTML = "<p><a href=\"" + profile.publicProfileUrl + "\">";
      profHTML +=  "<img align=\"left\" src=\"" + profile.pictureUrl + "\"></a>";
      profHTML +=  "<a href=\"" + profile.publicProfileUrl + "\">";
      profHTML +=  "<h2>" + profile.firstName + " " + profile.lastName + "</a> </h2>";
      profHTML += "<span>" + profile.headline + "</span>";
$("#badge").html(profHTML);
});
}

This is how that renders – and you can look at the code for this step.

Step 2: Add Share Stream

The member is authorized and we can see their information. The next thing the application needs to do is grab all of the status updates in their network stream and insert the information into the application.

This can be done using the NetworkUpdates function. To restrict the type of update to “STAT” (Status) updates, .params is used. For simplicity we’re using another inline function, which iterates over the updates and builds individual stream items for each one, then injects the html into the stream <div>.

There also needs to be a call to getUpdateStream() in the loadData() function so it gets called after the profile is loaded.

function getUpdateStream() {  
	IN.API.NetworkUpdates()
	.params({type:"STAT"})
	.result(function(result) {
	    var streamHTML = "";
		for (var update in result.values) {
			var thisupdate = result.values[update]
		
			// Build each individual stream update item
			person = thisupdate.updateContent.person
			var thisHTML = "<div class=streamitem>";
			
			// Person's picture,  linked name, and status
			thisHTML += "<img align=\"left\" class=img_border height=\"50\" src=\"" + person.pictureUrl + "\"></a>"; 
			thisHTML += "<a href=\"" + person.publicProfileUrl + "\">";
			thisHTML += "<span class=updater>" + person.firstName + " " + person.lastName + "</span></a>";			
			thisHTML += "<p class=update>" + activateLinks(person.currentStatus) + "</p>"
			thisHTML += "</div>";
			streamHTML += thisHTML
		}
		$("#stream").html(streamHTML);
	});
}

And now we have this. Code is here.

Step 3: Adding Interaction

Reading from the API is all well and good, but it’s pretty hard to make a compelling application if the member can’t do anything with it. Status updates can always be “liked”, so let’s add some like/unlike action to StreamIn’.

This part is a little more complicated. getUpdateStream() needs to be extended to add Like/Unlike buttons, and those buttons have to perform the appropriate action when pressed.

The Connect calls for these buttons are a little trickier. Since there is no Like/Unlike convenience method in Connect, we’ll pass through a raw call to the backend API. For this call, the method needs to be “PUT”, and the body is simply “true” or “false”. There’s an “alert” here to tell the user something happened, and then the stream is reloaded and redisplayed.

Here’s getUpdateStream again, with the newly added code in bold. The first section simply adds a button to the stream item – if the user has not yet liked the item, a Like button is shown. Otherwise, there’s an Unlike button.
The other two functions are event handlers for clicks on these buttons.

function getUpdateStream() {  
	IN.API.NetworkUpdates()
	.params({type:"SHAR"})
	.result(function(result) {
	    var streamHTML = "";
		for (var update in result.values) {
			var thisupdate = result.values[update]
		
			// Build each individual stream update item
			person = thisupdate.updateContent.person
			var thisHTML = "<div class=streamitem>";
			
			// Person's picture,  linked name, and status
			thisHTML += "<div class=updateperson>" ;
			thisHTML += "<img class=img_border align=\"left\" height=\"50\" src=\"" + person.pictureUrl + "\"></a>"; 
			thisHTML += "<a href=\"" + person.publicProfileUrl + "\">";
			thisHTML += "<span class=updater>" + person.firstName + " " + person.lastName + "</a></span>";	
			thisHTML += "<p class=update>" + activateLinks(person.currentShare.comment) + "</p></div>";
						
			// Present a like button
			if (! thisupdate.isLiked) {
				thisHTML += "<div id=button><button class=\"likebutton ui-corner-all\" id=\"" + 
                                thisupdate.updateKey + "\"><img src=\"Thumbs_up.png\"> Like</button></div>"
			} else {
				thisHTML += "<div id=button><button class=\"unlikebutton ui-corner-all\" id=\"" +
                                thisupdate.updateKey + "\"><img src=\"Thumbs_down.png\"> Unlike</button></div>"
			}	
			thisHTML += "</div>";
			
			// Slap this onto the HTML we're building
			streamHTML += thisHTML;
		}
		$("#stream").html(streamHTML);
	});
	
	$( ".likebutton" ).live("click", function() {
		   likeURL = "/people/~/network/updates/key=" + $(this).attr("id") + "/is-liked"
		   IN.API.Raw(likeURL)
			.method("PUT")
			.body("true")
			.result(function(result) {
				alert ("Liked");
				getUpdateStream();
			})
	});	
	
	$( ".unlikebutton" ).live("click", function() {
		   likeURL = "/people/~/network/updates/key=" + $(this).attr("id") + "/is-liked"
		   IN.API.Raw(likeURL)
			.method("PUT")
			.body("false")
			.result(function(result) {
				alert ("Unliked");
				getUpdateStream();
			})
	});	
}

So that’s it. The code for Streamin’ code is here. Now that you’ve gotten an app up and running, check out the docs and see what you can build!

As many of you know, I’m now happily employed as a Developer Advocate at LinkedIn. Part of my job there is to create interesting examples using our APIs, to help other developers find their footing and create new and interesting applications.  This will be the first in a long line of posts demonstrating how to use our API to create cool apps.

Note that this, as with many other applications, will be much less interesting with small networks. Grab more connections and it’ll have more cool data to play with.

Faceted Search

My passion has always been for showing interesting intersections of data.  The LinkedIn website has a faceted search feature you can see when browsing people you may know. On the left hand side of that page, you’ll see companies and schools listed there.  Selecting one or more companies restricts the people to those folks who match one of those companies.  Selecting a school or schools does the same thing.  Selecting both creates an “and”.

Try these examples on the website to get a feel for how the facet searches work. Because these facets are tuned to your network, you’ll have different companies and schools, but you can still see how the intersections work.

• People who worked at Netflix in the past
• People who worked at Netflix in the past *and* attended Stanford University
• People who worked at Intel in the past and work at Yahoo! now
• People who worked at either Cisco Systems or Microsoft in the Past

This same functionality is available via the LinkedIn API. Information on faceted search can be found in the People Search API documentation.

FacetBrowse

So how does this translate into a real application? The application above shows one possible implementation. But how does it work?  I’ll walk through the pieces of this application.  If you want to see the source application (which uses the LinkedIn JSAPI for interaction with LinkedIn, and YUI for presentation) you can look at the page directly at http://www.princesspolymath.com/facetbrowsesmall.html

On the left hand side of the application you see a list of buttons – by default you’re looking at the companies, but go ahead and switch to schools. This list is built as soon as the user has authenticated with LinkedIn, with a raw query to the API:

function onLinkedInAuth() {
    IN.API.Raw("/people-search:" +
   "(people:(first-name,last-name,positions:" +
    "(company:(ticker,name)),educations,picture-url),facets:" +
    "(code,buckets:(name,code,count)))?" +
    "facets=current-company,school&sort=distance&count=25")
    .result(displayFacetResults)
}

This returns the top 10 schools in your network, the top 10 companies in your network, and 25 of your contacts. FacetBrowse builds the buttons for the left hand side of the widget and labels them so the app knows what’s been selected when someone clicks one of the buttons. It then trims out contacts who don’t match any of the top companies or schools, leaving you with something that looks like this:

Any time one of the buttons is clicked, the function refreshConnections is called, which does the same facet search – this time specifying a company and/or school.

function onLinkedInAuth() {
    IN.API.Raw("/people-search:" +
   "(people:(first-name,last-name,positions:" +
    "(company:(ticker,name)),educations,picture-url),facets:" +
    "(code,buckets:(name,code,count)))?" +
    "facets=current-company,school&sort=distance&count=25" + 
    "facet=past-company,{company-code}&facet=school,{school-code})
    .result(displayFacetResults)
}

Note that because of the way the facet search works, you can’t create intersections within the request (this company *and* that company). Since I want to create these intersections, when I’m refreshing connections, I ask for one of the companies and one of the companies and then eliminate matches which don’t match all of the company/school choices.

For instance, when the user selects two different companies, such as they have here:

The application asks for the people-search and declares “current-company=“. When the results come back, each person is inspected to make sure that they have both Microsoft and Netflix within their positions. Matching results are displayed in the grid.

If a school is then selected (so now we have Stanford, Microsoft and Netflix):

The query requests Stanford University as a school and Microsoft as a company, and then checks each returned person to make sure they also have Netflix in their profile.

This application is just a small example of what can be done with the LinkedIn faceted search. Tell me in the comments if you’ve found other uses for it, or have questions about how it works.