Graph Databases are the New Black

By
Synedra
on June 19, 2009 9:57 AM | | Comments (0) | TrackBacks (1)
When I started working on computers, the main thing people used databases for was storing tabular data.  Sales reports, inventory records, lists of things.  Relational databases were cool because you could associate a transaction in your sales table with an item in your inventory table.  As the internet has grown, however, our database needs have outgrown the abilities of the standard relational database system.  Social networks are a great example of this - the type of query needed to ask for six degrees of Kevin Bacon separation in a relational database is complex, resource intensive, and extremely time consuming.  Many very smart people have come up with ways to work around this problem, but the real problem is that this type of question is not what relational databases are designed to handle.

Graph databases, on the other hand, are optimized for exactly this type of question.  The problem with graph databases is that it's a whole different data structure that we've got to wrap our minds around, and it's difficult to do that without some starting place.  Freebase is a publicly  accessible, open graph database, available for people to use as the back end for their applications.  They've even created a suite of tools to make it easier to build fun things on top of their extensive data. They will be giving a tutorial on Semantic Technologies at OSCON, and if you have any interest in learning about graph structure and what you can do with it, you should definitely check it out.

If you want a more data-agnostic view of graph technologies, or want to try installing a graph database on your own, you can learn about Neo4J, an open source graph database, on Wednesday afternoon.

Open Source iPhone Development at OSCON

By
Synedra
on June 10, 2009 4:58 PM | | Comments (0) | TrackBacks (0)
Developing applications for mobile devices is often a daunting thought for those of us more comfortable in the land of high-level languages.  However, in the wake of the new iPhone announcement, I thought I would check out the mobile development offerings at OSCON to see if there was anything I could sink my teeth into.  It turns out that it's become much easier to create content for the iPhone without having to turn to objective C.

The most intriguing tutorial I found was an introduction to the PhoneGap project, a platform which makes it easy for developers to create applications which work on the iPhone as well as several other mobile platforms.  A developer creates a web application using Javascript/HTML, and the framework provides a wrapper for this application which works as a native application on the mobile device.  The tutorial covers development under PhoneGap as well as the administrivia needed to get the resulting application into the App Store.  I was curious about how well this kind of framework could work for general iPhone applications, so I browsed through their applications page and found a variety of applications including highly rated games and locationally-aware utilities.  I'll be excited to hear more about it at OSCON so I can get started playing with it.

I do have a macintosh, which has up until now been a requirement for most iPhone development - but for those of you who aren't in the Cult of Jobs, another tutorial gives you the information you need to develop for the iPhone using open source tools on Linux or Windows.

Later in the week, Ian Dees will be talking about testing your iPhone apps using Ruby and Cucumber, and Patrick Collson will be telling you how to turn your iPhone into the Hitchhiker's Guide to the Galaxy.  OK, actually he'll just be talking about how they managed to cram Wikipedia into the iPhone, but it's pretty much the same thing, isn't it?

Of course, you still have to get a developer's license from Apple before you can create an installable application for the iPhone.  If you're curious about iPhone development, these sessions should help you decide if you want to jump in and give it a try.  

Taking my application by LoadStorm

By
Synedra
on May 19, 2009 4:33 PM | | Comments (0) | TrackBacks (0)
One of my work projects has gotten to the point where we really need to have some performance numbers under stress.  I played with The Grinder, poked at JUnit, and looked around at some of the other tools out there.  I was limited by the fact that I only have one system, and it's very hard to make a single system look like a hundred systems... and real load testing works best when you have multiple clients whacking on the system for you.

So, given my previous experience setting up parallel processing things on Amazon Web Services using hadoop, I thought perhaps I should just build a new system using AWS.  But that seemed like a lot of up front work considering that the actual questions we were trying to answer were "How many concurrent users can the system handle?" and "What happens when you go over that limit?"  Simple enough questions, and hard to justify tens of man hours putting together a new prototype just to answer them.

So, despite my engineery heart insisting that anything Not Invented Here couldn't possibly do the job, I poked around half-heartedly looking for an existing system to do what I needed.  And lo and behold, I found one!  LoadStorm is a web-based SAAS which does load/stress/performance testing for your system.  It's free for up to 100 users/minute (and up to 60 minutes), and you can give it multiple steps for each of your users to do (along with some random wait time to better emulate real people).

LoadStorm has a reasonably friendly front-end for setting up your servers and tests, but where it really shines is in the analysis - manager-friendly charts and graphs to make the testing results palatable to the people who really need to understand them.  Their team is extremely attentive and helpful, and I would highly recommend them to anyone curious about the outer bounds of their system's capabilities.

I was able to use LoadStorm to quantify the improvement we gained by making a structural change to the system, in an easy-to-understand format.  Playing with it further has given me a much stronger understanding of the strengths and weaknesses of our system.

Like I said, highly recommended.  Screenshots below.


users.jpg

errors.jpg

Ada Lovelace Day

By
Synedra
on March 24, 2009 9:05 AM | | Comments (0) | TrackBacks (0)
Today is Ada Lovelace Day, and along with thousands of other bloggers, I'm going to write a little bit today about a woman I have found inspiring in the technical world.  This isn't an easy task, picking one woman.  Like Liz, I am blessed to personally know a huge number of inspiring geeky women, but in the spirit of the day I'm going to pick just one, who I've never met personally, but who has made a significant contribution to my personal success as a programmer.

Kathy Sierra was the author of the original Head First book, and that series of books has done more for my technical progress than any other single factor.  I'm known by many people as a highly skilled programmer, but I didn't learn that stuff in school - I have a BA in Philosophy.  I'm not good at learning from dry crusty tomes - regular programming books tend to hold my attention through about the first 3 chapters and then I'm off to try the stuff on my own.  The Head First books, of which I have 9, engage me throughout and get the needed information in my brain, so that I can dive into a programming project with the sense that I have the tools I need.

That's not all she's done, though.   I saw her speak at OSCON a few years ago and was inspired to go find her blog and read it.  For the next couple of years, I read Creating Passionate Users religiously, hoping to glean some sense of how best to engage my users and make them feel like superheroes while using my tools. 

A couple of years ago, some internet trolls crossed the line in a serious way, threatening Kathy in a way that made her feel uncomfortable about continuing to participate in the blog community.  She is still out there sharing her ideas in other ways, but I am still saddened and infuriated that anyone could do such a thing to someone who has given so much to our community and to our level of discourse.

Kathy Sierra is my unsung hero.  Her ideas have helped me become a better programmer and a better interface designer.  Happy Ada Lovelace Day, everyone!

Installing OpenCV on OS/X with Python

By
Synedra
on February 27, 2009 6:42 PM | | Comments (0) | TrackBacks (0)
I need to get OpenCV working on my system for a couple of image processing tasks I have for work, and as with many complex software systems, it was somewhat difficult to get it working on my system, a macbook pro running Leopard (OS/X 10.5).

Their installation instructions were fairly good.  I first tried to use regular old make, but I had a few issues with libraries that regular old make couldn't find.  So I installed cmake to use for this purpose.  I highly recommend it.

With cmake, I was successful with the suggested instructions:
% mkdir build
% cd build
% cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -DBUILD_PYTHON_SUPPORT=ON ../

After that, 'make' and 'make install'.  

I ran into a few issues during this process.

  • No swig on my system.  The installation page lists it as an optional install, but it's needed for python, and python is what I want (they say this, but I thought it would do something other than just ignore my desire for python).  So install swig, and use the CMake GUI tool to make sure that your SWIG_DIR is set correctly.
  • Couldn't find my Python.h file.  Since I had installed my python using darwinports, my Python.h isn't really where it was looking.  I fixed this by editing the CMakeCache.txt to change PYTHON_INCLUDE_PATH:PATH=/sw/include/python2.5.  If your swig stuff is complaining about no Python.h (but you have one), find your Python.h and change this line in CMakeCache.txt.  There's probably a more elegant solution, but this worked.
  • For some reason CMake was pointing to the wrong SDK on my system (.  It wanted to use the 1.4u version, but I've got 1.5, so I changed /Developer/SDKs/MacOSX10.4u.sdk to /Developer/SDKs/MacOSX10.5.sdk using the CMake GUI.
  • Couldn't find my resulting opencv library.  I copied it to the /Library/Python path.
  • I wanted to make sure it installed all of the examples, so I selected them using hte CMake GUI utility.
Note that I did try the darwinports version, and lost a couple of hours trying to figure out how to get the python piece of that to work on my system.  I don't suggest that path.

I am pleased to report that now the python examples (at least) work on my system.  Including the houghlines.py which I needed to do the actual task ahead of me.  So, hurray.

I decided that to implement a houghcircles example I should really get the C code examples working, and thanks to the helpful instructions here I was able to do it easily. Follow those instructions and then you'll be able to compile and run all of the C example files (and make your own, and run them too!)

So I wrote a houghcircles example using some of the example code from the opencv book, and it works, kinda.  Pretty well I think, for just futzing around for an evening:


screenshot.jpg

Someone wrote to me about this blog post and said they were having some trouble with using the iSight for video input.  I discovered I was having the same problem.  The answer, gleaned from various places on the internets, was to unset DYLD_LIBRARY_PATH in my environment.  Confuses the heck out of openCV.  Which is kind of a problem because ROS requires it, at least to build.  So if you get weird library missing/wrong errors, unset that bad boy and see if it helps.

Python Netflix API

By
Synedra
on February 18, 2009 8:39 PM | | Comments (0) | TrackBacks (0)
I was having some fun poking around at the Netflix API, and realized that there wasn't a Netflix module for python, so I put one together.  First version is up at http://code.google.com/p/pyflix. Has installable code, an example file, and a py.test compatible test file.

Tummy update

By
Synedra
on February 10, 2009 10:37 AM | | Comments (0) | TrackBacks (0)
When last we left my intrepid tummy, we were headed to the gastro doc to talk about why it was so sad.  She decided I should have a colonoscopy to make sure my intestines weren't problematic, and a CAT scan to make sure my pancreas wasn't puffy.  So, after finishing my 3 weeks of the elimination diet, I spent a day drinking barium for the cat scan, and then later that week I spent a day eating clear liquids for the colonoscopy.

Verdict: My pancreas is fine.  My intestines did have a flat polyp which she removed (hurray!)  No crohn's, no other problems.

So, we're left with the likely problem of Small Intestine Bacterial Overgrowth.  I've done some research on this and I've found a program that seems to really be working. So, anyone in a similar situation can take heed and see if something like this will help you too:

  • Enteric-coated peppermint oil.  This stuff is awesome.  It kills bacteria in your small intestine and reduces gas and bloating.  3x/day.  I'm gonna take this for a month and then switch to building up my flora, because I'm guessing the peppermint oil isn't smart enough to distinguish between "good flora in the large intestine" and "misguided flora in the small intestine"
  • Much less crap.  I'm trying to keep myself to foods which are as close to whole as possible.  I know, duh, but really, this is making a huge difference.  I'm making bread in my breadmaker almost every morning, making dinners from actual ingredients, avoiding foods that tend to cause acidity in your system.  I noticed that eating more alkaline-causing foods made my body happier so I'm working on that.  Note that artificial sweeteners are the acidest foods of all.
  • UltraClear Ph.  This is medical food that helps keep your Ph in the right place.  And it seems to help my system be happy.
So much success I'm having on this plan that two days ago I WORE PANTS for the first time in 3 years.  And didn't have to take them off from uncomfortable bloating.

Yet Another Way for the BSA to Discriminate

By
Synedra
on January 15, 2009 8:30 AM | | Comments (0) | TrackBacks (0)
My friend has been a boy scout leader for years.  She doesn't have any children, but she and her husband are active scout leaders nonetheless, giving their support and guidance to kids because they enjoy it, and because they are good people.

My friend has lost a lot of weight and goes to boot camp and bikram yoga regularly.  She did the Danskin triathlon with me last year.  She could almost certainly pass whatever fitness test the BSA threw at her to prove she's able to survive in the severe conditions present at a Boy Scout Camping event.

The BSA has just updated their guidelines for their leaders.  According to the chart they have, unless she loses weight to meet an arbitrary standard, she will no longer be able to participate in remote or strenuous boy scout events, and the organization strongly suggests that she not be able to participate in *ANY* boy scout event. If their goal is to provide the boys with a good role model, a larger person who is active and healthy is a much better role model than a normal weight person who smokes, or drinks, or just plain doesn't exercise.  I would take her as a leader for my den in a heartbeat - she is a smart, moral, and fit person who embodies the values I'd like my son to strive for.

I hope that someone sues the Boy Scouts over this.  It is pure discrimination with no reasonable defense.  I will have to consider strongly whether my son should continue in an organization which believes that they can judge his moral worth based on his belt size - or the belt size of his leaders.  I realize that the Boy Scouts are a private organization, but this kind of ignorant elitism is unacceptable.  If you want to make sure that people are fit to camp, test them.  The results might surprise you.  The only reason for this particular decision has to be that someone in BSA headquarters doesn't want their organization associated with overweight people.  Pure, unadulterated, bigotry.

If you'd like to see the new health form yourself you can see it on their website. The precise wording is "Individuals desiring to participate in any high-adventure activity or events in which emergency evacuation would take longer than 30 minutes by ground transportation will not be permitted to do so if they exceed the weight limit as documented at the bottom of this page. Enforcing the height/weight limit is strongly encouraged for all other events, but it is not mandatory."

The Trials and Tribulations of my Tummy

By
Synedra
on January 2, 2009 4:38 PM | | Comments (0) | TrackBacks (0)
My GI tract has been doing some super weird things over the last couple of years.  When I eat pretty much anything, my stomach puffs up by 5 inches, and it gets totally hard.  My answer to this conundrum up until now has been to only wear skirts, and only skirts with elastic waistbands, but it occurred to me that perhaps I should mention this to my doctor.   So I did, which started me on a super fun game called "What the heck is wrong with this system, anyhow?"

The first thing I did was a CDSA test, which indicated that I have a lot of inflammation markers in my gut.  This can be caused by a wide range of things, including Celiac, other food sensitivities, an infection, or cancer.  Also, my happy bacteria were completely missing, which wouldn't be totally weird, except that I've been taking probiotics filled with them for a few months, so somehow they're getting gobbled up and not colonizing.    And I had some little benign amoebas taking up residence.

So then we did a celiac test.  Negative.  That's good, I guess, although at this point I'd rather have that than some of the other options.  I took Flagyl for a few days to kill the amoebas (in case maybe they weren't benign, and were causing the problems).  It almost killed me, so I begged off of that for now.

Next stop, Elimination Diet.  So this week I started my 3-week diet with no corn, wheat, soy, milk, eggs, red meat, caffeine, sugar, peanuts... pretty much nothing.  I'm glad I'm on vacation this week while I get started with this, and I'm happy to say that I'm almost through day 5 and am still alive. I just finished a pretty decent lunch - homemade turkey chili over roasted potatoes, with slices of avocado on top.  My life became a lot better when I figured out how to make a pretty decent representation of a rice crispy treat with brown rice syrup, almond butter, vanilla, and brown rice crispy cereal.  I got to take those to the movies, and they were a perfect snacky treat.  

In a couple of weeks I'll be visiting the gastroenterologist, to see what she thinks about this whole thing.  My tummy isn't getting all that much better, but I'm going to go through this for 3 weeks so we know if it helps or not.  As long as I have my rice crispy treats, I think it'll be ok.

Finding people on Twitter

By
Synedra
on November 30, 2008 9:05 AM | | Comments (0) | TrackBacks (0)
After I got my new sidebar widget working, I thought I'd go and see if any more of my friends were listed on Twitter, and discovered to my chagrin that they have elected to remove the "search for people" function on Twitter.  It's been replaced by an "Allow Twitter to Rifle Your Rolodex" function where it looks to see if anyone you know is on twitter - and then does the super annoying Facebook-like list of everyone who isn't enTwittered yet - "These losers haven't signed up yet.  Spam them for us!"  Yeah, no.  

Two third party folks have made cool applications to replace this functionality.  Neither is perfect (and they're both pretty overwhelmed), but they're better than nothing.  You can find people with the name of <whatever> at twitterdir, and you can find friends-of-friends at twubble.  I had a lot of fun futzing with the latter this morning.

Your other alternative is to google "site:twitter.com Kirsten Jones"

Note that if you play with twubble for too long, Twitter will throttle you, and you'll end up with no functionality for your cool sidebar widget thing for an hour while you contemplate the error of your ways. 
Archives

Search

Where's Kirsten?

Doing what?

Follow me on Twitter

Recent Entries

Tag Cloud

Synedra's tweets

www.flickr.com
This is a Flickr badge showing public photos and videos from synedra. Make your own badge here.

Find recent content on the main index or look in the archives to find all content.