Creating Netflix Widgets from Freebase Queries

Freebase has just finished creating a process to import Netflix film data on a daily basis. Freebase now includes in their movie metadata the key for the Netflix API. For Netflix API developers, this makes it possible to leverage Freebase’s powerful query engine and create amazing application interfaces. But even for those people who have never used the Netflix API, this functionality allows developers to leverage Netflix’ widgets to create interfaces within their existing applications, without the hassle of OAuth. I’ll walk through the process of creating a Netflix widget based on a freebase query here.

Getting the Movie

Mel Gibson is a popular guy, who’s done a bunch of movies. But what has he done for us lately? The following freebase query asks what movies he’s done since January 1, 2009. I’ll go ahead and ask for the Netflix “Tiny URL” as that’s what we’ll need to create the Netflix widget.

[{
  "name" : "Mel Gibson",
  "film": [{
    "film":  [{"initial_release_date>":"2009-01-01", "name":null,
               "key":[{"namespace":"/authority/netflix/tiny", "value":null}]}],
    "id":    null
  }],
  "type":          "/film/actor"
}]​

Turns out, he’s been taking it easy recently. There’s just a couple of movies released on DVD since the beginning of last year.

{
  "code":          "/api/status/ok",
  "result": [{
    "film": [
      {
        "film": [{
          "key": [{
            "namespace": "/authority/netflix/tiny",
            "value":     "BVmIj"
          }],
          "name": "Edge of Darkness"
        }],
        "id": "/m/04lr63t"
      },
      {
        "film": [{
          "key": [{
            "namespace": "/authority/netflix/tiny",
            "value":     "BVssN"
          }],
          "name": "The Beaver"
        }],
        "id": "/m/09vnjl6"
      }
    ],
    "name": "Mel Gibson",
    "type": "/film/actor"
  }],
  "status":        "200 OK",
  "transaction_id": "cache;cache01.p01.sjc1:8101;2010-07-16T21:13:40Z;0034"
}

Creating the Widget

Now that we have the tiny URL, we can create a widget. The Netflix widget builder can be used to create a template for the widget you want to create. When you’re done configuring your widget, you’ll end up with some Javascript code that looks like this:

<script src=”http://jsapi.netflix.com/us/api/w/s/sp100.js” settings=”id=http://movi.es/BVdT5″></script>

Once you’ve got that, it’s a simple task to create new widgets. The Tiny URL you grabbed above is used to update the ID, so in the case of “Edge of Darkness” you’d put “BVmIj” in place of BVdT5 and end up with this:

<script src=”http://jsapi.netflix.com/us/api/w/s/sp100.js” settings=”id=http://movi.es/BVmIj”></script>

Which will render on your page like so:

Once you’ve gotten this all working to your satisfaction, you can even set up the widgets to include your developer key and earn money from Netflix via the developer Affilliate Program.

Tags: freebase, netflix, widget

Twitter badge with location

In my job as an elf for Santa, I frequently have to travel back to the North Pole to visit with my fellow elves, so it’s not always easy to tell where I might be.  Given that fact, and my desire to procrastinate a bit, I decided to make a handy badge for my blog that tells people where I am.  Most twitter clients allow you to throw some location information in your posts, so it’s relatively easy to make it relatively easy for someone to stalk you.

Now that you’ve got that put on your server, you just need to put some code in your blog template (wherever your other badge stuff is) to get your fabulous information to show up.  The code for the HTML looks yucky in my post, but you can view source and grab the stuff marked as “TWITTER”.

#twitter_div {
border-bottom-style: solid;
border-bottom-width: 1px;
margin-bottom: 10px;
font-family: Arial, Helvetica, sans-serif;
font-size: 0.9em;
padding-right: 5px;
padding-left: 5px;
text-decoration: none;
}
#twitter_location {
margin-left: 25px;
text-decoration: none;
margin-bottom: 10px;
}
#twitter_div ul li {
text-decoration: none;
border-bottom-style: solid;
border-bottom-width: 1px;
padding-bottom: 5px;
margin-bottom: 5px;
}
#twitter_div p {
text-align: right;
padding-right: 6px;
margin-bottom: 10px;
}

Tags: location, twitter

Getting something done

I’m sure I’m not the only one who gets periodically overwhelmed by the number of things I’m not doing right. I still shampoo my hair, my kids watch TV and play video games and despite my best efforts we still seem to end up with transfats and other evil plastic foods in my house. Every time I turn around, there’s something new to feel guilty about, and the packrat research monster inside of me insists that I need to dive deeply into every single topic to determine the truthfulness of the claim, which just increases my anxiety about my worth as a human being, while not actually giving me space in my head to contemplate anything of actual worth.
I have spent many an hour railing against the mean and nasty high bandwidth world, providing so much chaos and so few answers… but in a moment of clarity, I discovered that these gifts brought to me by the intertubes, clamoring for my attention, are not actually the main source of my anxiety. My anxiety comes from having too many things bouncing around in my brain like colored balls in a Busy Ball Popper. And I have found something that actually helps.
I read Getting Things Done a while back, and it kinda was ok, and seemed like a pretty good idea, but I wasn’t really sold. Then a few weeks ago I was pointed to OmniFocus, a GTD tracker for the Mac, and I discovered that if you wrap up a good set of ideas in a cute and sexy UI I will actually use it. And when I spend my time actually doing things instead of scrambling to remember which next thing I was supposed to be doing, things get done faster, and I am calmer. I sort of feel like I’m playing that game from ST:NG where they had the little brain game that gave you zaps of happiness whenever you make a goal. I check a box, it gets a line through it, and I feel a little swell of pride. It integrates pretty nicely with my iphone (courtesy of Toodledoo) and it allows you to see all your ‘errands’ or ‘phone things’ in a list separate from the project they’re associated with. Which is great when you’re trying to get all those lame phone calls out of the way at once.
I’m not sure if I’m really any closer to zen mastery, but I sure feel happier. Even though I’ve been battling the most rotten cold ever this week, I’ve gotten a lot done for home and work, and I don’t feel like the world is sitting on my shoulders, waiting to collapse. And now I can check off the item marked “Make blog post about OmniFocus.” Woo. That felt good. Maybe I should call and schedule that dentist appointment next.

Django is great^h^h^h^h^hfrustrating

Don’t get me wrong. I do like django. I’ve been working back and forth in Pylons and Django, trying to learn each of them well enough so that I can figure out which one will give the right answer when I know better what the powers that be want.
So, if you, like me, have a debian etch box upon which you want to install django, have it work with the tutorials in the book and on the site (thus needing python2.5), using mod_python so that you can work on a remote server, with postgres, do the following:

• Use the Django from subversion. It lives in http://code.djangoproject.com/svn/django/trunk (don’t forget to uninstall python-django if you’ve already installed that.
• It requires python2.5. The packaged version is fine. apt-get install python2.5
• Get mod_perl as a package, because it will make all the connections correctly (but it will be linked to python2.4), and then
• Install apache2-prefork-dev so that you have the right apxs2 to build mod_python against python2.5
• Download mod_python from http://ftp.wayne.edu/apache/httpd/modpython/mod_python-3.3.1.tgz
• Configure it (with –apxs=/usr/bin/apxs2)
• Install it
• Get psycopg from http://www.initd.org/pub/software/psycopg/PSYCOPG-2-0/psycopg2-2.0.5.1.tar.gz. Don’t get fancy and try the new one. It doesn’t work.
• python setup.py install that sucker
• And then restart everything and all should be lovely in the world

So that’s…

svn co http://code.djangoproject.com/svn/django/trunk django
ln -s `pwd`/django /usr/lib/python2.5/site-packages/
ln -s `pwd`/django/django/bin/django-admin.py /usr/local/bin
apt-get install python2.5 libapache2-mod-python apache2-prefork-dev
wget http://ftp.wayne.edu/apache/httpd/modpython/mod_python-3.3.1.tgz
tar xzf mod_python-3.3.1.tgz
cd mod_python-3.3.1
./configure --apxs=/usr/bin/apxs2
sudo make install
cd ..
wget http://www.initd.org/pub/software/psycopg/PSYCOPG-2-0/psycopg2-2.0.5.1.tar.gz
tar xzf psycopg2-2.0.5.1.tar.gz
cd psycopg2.2.0.5.1
sudo python setup.py install
sudo /etc/init.d/apache2 restart

I spent a good deal of today trying to find these answers. So, you’re welcome

Working like Sheep

I have a friend who hearkens from the heartland of America, and is thus more schooled than I in the cosmic truths to be found on a farm. We were discussing the relative dimness of various farm animals, and he mentioned that sheep were pretty much the stupidest animals around. I asked why, and he said that a lamb, when confronted with a meadow full of tall grass, will eat through the grass, leaving a 1-lamb-wide path behind them. When the lamb is full, however, it is faced with a horrible situation. Walls of grass surround it on the front and the sides. After looking left and right in a panic, the lamb will start to bleat piteously, hoping for someone to rescue it from its plight, eventually sitting down to wait until it’s hungry again so that it can extend the path further. I’m not sure how true the story is, but it makes for a compelling mental image.
I was having lunch with my friend Eugene yesterday, and we started talking about my last post on making space, and about the sheep analogy, and I realized that in fact the two are very related. I frequently find myself in a position where I am trying to solve a difficult problem. The more I push, the more the answer eludes me, but I have this underlying fear that if I break away and come back to the problem with fresh eyes, I’ll lose the context I’ve worked so hard to achieve. The reality is that I’m just like the lamb. The answer I need will only be clear when I back up. The context I’ve built up is *broken*, which is why I’m not finding the answer.
I spent a lot of yesterday working like a lamb, pounding my head against a problem which turned out to be fairly simple to solve after I returned from lunch. Chastised by the universal forces, I meekly turned off the computer at 5 so that my brain cells could recharge before I tackle the next problem this morning.

Creative Space

So I’ve now been at my new job for about 2 months, and there’s a major contrast between my last job and this job.
We’ll start with the negatives because there aren’t many. I’m a remote worker at a company that does most of its collaboration and communication in person, so I’ve had to work hard at staying in touch with the people at the office. It helps that I go down there at least once a month to touch base, and I’m getting to know the folks (and they me) so remote communication is more effective. Also, without anyone checking in with me every day it takes more discipline to buckle down on days when my motivator unit is broken. And I’m currently the only person on my ‘team’ which makes it a little hard to get traction. And I miss all my ex-coworkers.
Now, the positives. Unlike the company I came from, there is no drama here. Of course, it’s replaced by its own special type of politics, of a grown-up type I’m somewhat unaccustomed to. Fortunately I seem to be negotiating these new waters fairly well, and the end result of this environment is that my job is much, much less stressful. Being the only remote worker also means that I’m almost never interrupted for anything, and my velocity when I’m writing code or reading documentation is generally quite high. And working somewhere that brainstorming is a valued activity has given me the opportunity to discover that in fact I’m good at thinking up ideas, when I have some time to do so.
All in all, the positives far outweigh the negatives, and I feel like I’m learning, contributing, and working in a positive environment. It’s still hard to get out of the habit of feeling horribly guilty if I’m out of contact for a bit, but I’m slowly becoming more relaxed and just enjoying the work I’m doing. It seemed very odd to me, coming from a place of such urgency to a place that seems so much more laid back, but the folks at AMI have the space to think up cool stuff – without that space it’s so much harder to let the magic happen.
I know that small startups have less space to allow their employees to stop and think and explore, but I wonder if sometimes the need to rush creates more need to rush, and squelches the innovative ideas best suited to an agile, entreprenurial company. I was reminded of this several times when working at Socialtext, when I stopped rushing around for a few days and as a result found the solutions to the problems that had been pestering me for weeks.
Companies need to create space for their employees, especially when the employees aren’t good at doing so for themselves. People need to create space for themselves. All of us, all programmers, have found ourselves staring intently at the screen, the zone long past, trying to find the solution to a problem… and then flash on the solution as soon as we step away to feed ourselves and look at the sky. Make space for yourself, and try to help others around you find space for themselves as well.

Perl Makes Good Programmers

I bet you think that I’m kidding, right? Perl allows programmers to build extremely bad habits, doesn’t force discipline, encourages shortcuts that simply aren’t possible in other languages. But I’m actually serious.
I had a conversation with my new boss a few weeks ago that went something like this:
Boss: Do you know what data driven design is?
Me: No.
Boss: (google it if you don’t know
Me: Oh. That’s how I code.
Boss: Good
So it occurred to me to wonder why it was that I code that way, what with my lack of formal education and all. I went to the Pittsburgh Perl Workshop last weekend (my OpenID presentation went great, thanks for asking – I used sock puppets to describe the user/website/server interaction) and spent a great deal of time thinking about how perl people write code and why, and I realized that I code this way (with reusable, modular, configurable code) *because* I work in Perl.
The easiest way to get started doing something with Perl is to pull down some helper code from CPAN. All of that code (ok, not all of it, but most of it… well, ok, the parts I actually use) was written in a modular way, designed to be configured to work for multiple applications. The more time someone spends writing Perl code, the more likely they are to approach each new problem in a modular way (I need something to do A and B, and then C to tie them together) – this makes it easier to use CPAN to reduce your workload, but also means that when you write the code you’re likely to avoid hard coding anything in the program itself.
When I started working in Flex, I was a little frustrated that it was so difficult to find examples where developers had pulled the configuration information out into a separate file, but the truth is that’s not terribly surprising. First, there simply aren’t nearly as many examples of Flex code out there, so it’s harder to find any specific thing. Second, without the external community pushing developers to think beyond their current application, it’s easy to fall into the habit of taking the shortest path to “done.” Working with Perl (and some seriously critical programmers) for (ack!) thirteen years has given me an allergy to hard coding *anything* in my programs. Which makes it harder for me to spit out something quick and dirty, but makes the things I *do* make much more powerful.

Now I remember why I do this job…

Today I had one of those days you get as a programmer, when everything coalesces in a perfect way to shine a happy light on the universe. I had a list of bugs and a single feature I wanted to add to my Flex program. The single feature (dynamic filtering based on values in a set of arrays) took me a long time, and I wrestled with it mightily for several hours. I wanted desperately to make a filter which was extensible, which was easy to add new fields to, and passing variables around in Flex is not something I’ve become comfortable with.
So there I was, pounding my head against the thing, frustrated and feeling incompetent, until 4PM when a light shined down upon me from the sky, the scales fell from my eyes, whatever cliche you want to use… and the feature was complete. And when the dust cleared, I realized that 3 of the bugs on my ‘to-do’ list had gotten fixed in the process of creating this feature.
It’s an awesome day when you feel “done” with what you’re working on. I wasn’t even tempted to start on something else… I just basked in the glow of having gotten it done.
For anyone wandering here from flex land who wants to see the code, here it is:

private var filterObj:Object = {this:filterThis,that:filterThat};
// These arrays are populated elsewhere with a list of strings to filter on
private var textObj:Object = {this:"",that:""};
private function filterAll():void {
var filtered:Boolean = false;
var filterType:String;
for (filterType in filterObj) {
if (filterObj[filterType].length > 0)
filtered = true;
}
if (filtered) {
dataSet.filterFunction = filterGeneralSet;
dataSet.refresh();
} else {
dataSet.filterFunction = null;
dataSet.refresh();
}
private function filterGeneralSet(item:XML):Boolean {
var filterType:String;
var element:String;
var myArray:Array;
// This is an 'and' filter where everything has to match for the element to return true
for (filterType in filterObj) {
if (filterObj[filterType].length > 0) {
myArray = filterObj[filterType];
for each (element in filterObj[filterType]) {
var itemFilter:String = item.child(filterType)[0];
if (itemFilter != element) {
return false;
}
}
}
}
return true;
}

Conceptual Reference Books vs. Tutorials

I need to learn Flex so that I can throw together a demo of an application in the next couple of weeks. I know what I need to do, some other kind soul has done the programming and even published the source for the hardest element in the application, so I just need to ramp up on the language so I can take that code and run with it.
Being a research queen, I headed over to Amazon to look at the reviews for Flex books. I read all of them and decided on Programming Flex 2… forgetting, apparently, that I have the attention span of a gnat and so a conceptual programming book is doomed to fail to teach me something new.
I did make it through 100 pages before MEGO (My Eyes Glaze Over) set in and I stopped being able to understand any of the words on the page. I shook the cobwebs out of my head and went back to Amazon and discovered that I probably wanted this book instead, since it’s in a tutorial format instead of the bone-dry lecture format of the other one. I was going to head down to Borders to get it, but my daughter called to be picked up from school so I didn’t get the chance.
Having returned home, I hunted for reasonable online FlexBuilder tutorials and found that Adobe actually has some really good tutorials in their documentation. Turns out that there’s also a special deal with their training provider for 30 days free, so I signed up for that. In fact, these two training courses map almost exactly to the book I didn’t end up buying and I’m probably going to get more out of an online presentation.
So, having read 5 chapters/100 pages of a perfectly good reference book in 3 hours this morning, I had a vague understanding of the overall application and how it worked, but no idea at all how to do anything in the application. Doing 11 tutorials later in the day (about 2 hours) got me much closer to where I want to be – I finally said “uncle” when my brains started dribbling out of my ears. I plan to spend the next couple of days doing more tutorials and then I think I’ll have more than enough experience to do what I need to do. Heck, a goodly part of my job is going to be making prototypes – I might as well learn this one really well before trying out OpenLaszlo.
What did I learn? O’Reilly makes really good reference books, but I just can’t learn from them. I need someone to tell me how to do various things and I’ll pick up all of the intricacies from that. I’m good at identifying when my kids aren’t learning something with a particular method, but it’s harder to remember that I have the same limitations. On the other hand, the O’Reilly book will be a good reference, so it wasn’t a waste of money.

Pragmatic (don’t be dumb) Programming

Since I have no formal degree in programming, I have been extremely fortunate to work on teams of very talented, smart and diligent coders. Wherever I have worked, the bar has been high, the code has been challenging, and the expectation is that everyone on the team will do their best work every time. The terminology used among these various teams has varied, but the general sentiment is always “Don’t do something you’re going to regret later.” Don’t tie the data to the code, don’t make it hard to extend the program, don’t tie code together unless you have to, don’t make assumptions about what people will do with the application… in short, don’t be dumb. When we were working on the REST interface for Socialtext it was one of the best projects, because decoupling the data from the functions was useful and fun and it just *felt right.* The words used to describe these concepts were general terms like abstraction, pluggability, configurability.
Coming to Applied Minds, I find myself awash in new vocabulary words and frequently find myself scurrying to look up a term that’s been thrown my way. Fortunately, once I do that it generally turns out that yes, I do that, whatever it is. For instance, Data Driven Design falls under abstraction, configurability, and “Don’t be dumb.” But I didn’t know that.
I have to learn a couple of new languages next week in order to create a demo application for us to play with, so I headed off to Borders to pick up a book or two. While I was there, I also picked up the “Pragmatic Programmer” book which I heard referred to so frequently by my cohorts at Socialtext. Yesterday and today I read through it and discovered that it is all full of “Don’t be dumb.” Things I have observed and internalized, but couldn’t articulate quite yet.
The book is an excellent discussion of the way a good programmer behaves, the things they do and don’t do, and how to incorporate them into your workflow. Many of the sections are likely to add more overhead to your project up front – test harnesses, contracts, decoupling, abstraction of metadata. But the return on investment is huge. *Not* doing these things, paying attention only to the here and now and neglecting the future, is where almost everybody gets caught.
I have a strong tendency to want to get to the finish line as quickly as possible, and am frequently stung by not doing these things. Here, I’m starting a new project. It’s not a rush (well, the demo wants to be created in two weeks, but that will be a true “throwaway” prototype for the purposes of playing with the UI elements). So I’m pledging to myself that I’ll keep the discipline necessary to do things right, take the extra time to think abstractly and control the interfaces between the moving parts.
It’s a good book. If you are a programmer and you haven’t read it, you probably should.