The Wantrepreneur

Semantic Gifts: Valentine's Day Edition

2009-02-02T17:10:00.000-08:00

Just in time for Valentine's Day shopping, a new version of Semantic Gifts ( http://semanticgifts.com ) went live today. Representing much more than new romantically themed gift ideas, the update approaches the problem of gift recommendation using social media in some fundamentally different ways.

The first obvious change is the target audience: guys looking for gifts for their girlfriends or wives. A particularly challenging purchase for many men, romantic gestures can be much more stressful than a more open ended gift like those used by the original Semantic Gifts. Additionally being a more tightly "themed" holiday there is naturally a smaller scope of gifts which are pertinent, and we found they fit well into more of a categorization model.

Though the app is targeted at guys looking for gifts for their significant others, girls can participate as well, with a short questionnaire on the types of gifts they'd like to receive.

The results have moved to a categorical model - instead of suggesting a particular product, the engine now recommends one of ten types of gifts that most closely match the text streams entered. In a move toward further transparency the system also provides some feedback on the observations that led up to the recommendation, something that the users of the first edition indicated would be helpful.

From a technology perspective the engine is largely unchanged. The emphasis on the first edition leaned more on profiling users against archetypal sources using topic modelling techniques, while the direct feedback of the Valentine's Edition and some better extraction algorithms wound up with more mass on the statistical side this time around.

Check out the new version and let us know what you think - many of the changes (and not-changed things) were based on feedback from the first edition. And stay tuned for the next version coming up, we've got some exciting ideas in the works.

On Facebook, Friendfeed

2009-01-28T15:35:00.000-08:00

One of the more difficult decisions working on the latest Valentine's Day version of Semantic Gifts was the removal of Facebook and FriendFeed as source options for gift recommendations. In my Christmas edition I had the ability to use Facebook, FriendFeed, Twitter, or an open Http:// string. For the Valentine's edition I've decided to use only Twitter. Though my extreme minimalism was the antagonist for the removal of each, they all had their own list of pros and cons.

I get many things from Facebook integration. Name, sex, location, friend ratios, and occasionally interests, hobbies, movies, etc. But as the API stands I can't get the text of the recipient's comments, posts, or historical status updates. Technically speaking I have named entities to extract but I don't have natural language to process, and Semantic Gifts is very much an NLP driven app. The first response from everyone is "Aren't you cutting out a wide swath of your potential userbase?" and it's true. There are lots of people who use Facebook who aren't yet on Twitter. But without free writing samples I can't demonstrate what Semantic Gifts can do, and there's nothing I can say based on a list of interests that you couldn't figure out on your own. It's actually worse than that, because a given user will assume I can get everything that person has ever said on Facebook, which isn't possible, and will thus be doubly disappointed.

I'd add that in conjunction with other sources Facebook is awesome though, and is slotted to return for the next iteration of Semantic Gifts "Classic". The hard data used to seed the algorithm is instead coming from a Girls' page this time around. Each has advantages and disadvantages.

Twitter is the 500 lb gorilla. I loathe the idea of Semantic Gifts being categorized as a "Twitter app", but in general it gives us the data we need to make Semantic Gifts do what it does. And it is pretty awesome. Sooner or later everyone is going to catch on that combining microblog pithiness with modern concept mining you can extract some pretty good scores without waiting for ratings or search strings, and everyone will do recommendation this way. Twitter is good, and would be better if everyone used it the same way...

Which is what makes FriendFeed much harder to exclude from this round. On the one hand it seems much more heterogeneous - you have all kinds of different primary sources. But when you look closer at the text that comes back... the ubiquity of opinion expression that is our best recommendation fuel... you find topic introduction and discussion in a wonderfully contextualized way. I am coming around to the idea that for our type of recommendation to truly take over it will be running against lifestreams, not microblogs.

In the meantime the traffic numbers haven't justified the real estate for FriendFeed in the Valentine's version. I expect it to stay on for the classic edition and potentially become my primary source in the future (I am looking forward to that).

A little design commentary is necessary to put the cuts in context... the primary driver to remove anything at all is a strong (possibly relentless) drive for minimalism in the interface. The longer I do this the more strongly I believe each element has to justify its continued presence every major release.

Which is easy for me to preach about because Semantic Gifts is a casual app with a nascent technology in a not quite emerging market, but the exercise of touching each item on your site with a skeptical eye and forcing yourself to justify not commenting it out is an approach that I think should be applied to any application.

Getting Started in Collective Intelligence

2008-12-09T15:39:00.000-08:00

Have you ever wanted to write a program which learns from user feedback and recommends the perfect song, news article, stock pick, or significant other? Is your startup’s alternate revenue model listed as “Win Netflix Prize”? While a bit more complicated than your average web application, the skills needed to create an application using machine learning, natural language processing, or information retrieval algorithms can be learned using tools accessible to the leanest of startups.

Since going out on my own as a startup founder I’ve found that the field has some pros and cons for bootstrapping. Since the number of features available to improve your algorithm are effectively infinite, the issue of processor management becomes a factor day one. The other key challenge is to resist the lure of academia and to put product first. The possibility of advancing the state of the art is absolutely the most exciting thing about this (or any) science, but as a startup founder it’s important to focus on pragmatic improvements over technology for its own sake. If it helps, remember that the discipline of a solution focus can be a great lens for innovation. However you are able to strike this balance, plan to do some serious homework and a lot of trial and error.

Start Studying

If you’ve got a background in Computer Science with exposure to Linear Algebra and Statistics you’re in great shape to get going, otherwise you may need to brush up on those areas as you go. Without further ado these are my favorite sources for self study:

Programming Collective Intelligence – The classic programmer’s introduction to machine learning algorithms. If your trade is in software development, this is a soft place to start. All of the scary math equations are hidden at the back of the book which helps when you’re just getting familiar - though you’ll need to face them eventually. I suggest coding the examples and learning the equations behind them as you go along, or you’ll find yourself backtracking later. Bonus tip - the source code for all the examples is available here.

“The Carnegie Mellon Slides” – Andrew Moore’s slideshows provide a more solid foundation in applied Bayesian statistics. There are many others out there but his wry sense of humor makes these an entertaining place to get comfortable with the math side. He covers advanced topics as well but for those I personally like…

“The Stanford Lectures” – Comprehensive online video courses on Natural Language Processing and Machine Learning. If you take one thing away from this guide, check these out.

JMLR – The Journal of Machine Learning Research. Free papers representing research’s bleeding edge. The papers are great for thinking about creative startup ideas when you’re ready for them.

Utility Belt

As you’re studying, you’ll want to get acquainted with some handy tools of the trade.

Wordnet – Practical applications of synsets have their strengths and weaknesses, but overall Wordnet deserves its ubiquity.

Calais – Dead simple API from Reuters that returns keywords for a given piece of text. Great especially when you’re starting out.

Link Grammar – Necessary for breaking up sentences and finding parts of speech. I personally like tools based on the Penn Treebank, but I don’t know any that are free for commercial use. The Stanford Parser or Python’s extensive NLTK package are also useful for evaluation.

LDC – This is the source for “Big Data”. I would caution the thrifty bootstrapper to resist the seductive power of the Google Corpus and its 5-gram goodness unless you know for sure it’s what you need - it takes a lot of cores to query that much data effectively. The highest quality data for your app will always be the data you generate from your actual source materials, but there are many instances where a large amount of data really helps.

There are online source code versions of most of the common machine learning models available, so when you know what approach you want to evaluate a quick Google search should lead you to a version to play with. Always make sure to read the license fine print carefully if you plan to use it commercially, or just plan to write your own version.

Python is a natural choice for experimentation for the PCI examples and the NLTK. The Stanford guys use Java, and most of the packaged engines are written in C/C++. In the end expect to mix and match depending on the application, however, so don’t get too hung up on the platform. My current codebase is actually based in Ruby but has used all of the languages above for various tasks.

Like other topics in software engineering the key to a successful implementation is in a judicious use of the right algorithms for the job. Making those choices well requires experience and perspective, and the latter is something more difficult to get through self-study than at a university - you miss out on a professor’s point of view on the comparative advantages of different approaches. On the other hand there is a wealth of information available online to develop experience, and often practical implementations and academic theory diverge in emphasis – especially in the startup world.

Adam is the creator of Semantic Gifts, a gift recommender that mines social media. He’s been developing software for over a decade, and has an uncommon fascination with grammar rules.

Semantic Gifts

2008-11-25T10:30:00.000-08:00

The holidays are coming right up, and for many the season means a perennial challenge selecting the right gift for family, loved ones, and secret santas. Well fret no longer because there is a new way to find the perfect idea, Semantic Gifts (http://semanticgifts.com).

Here's how to use it:

Step 1: Select whatever information you have about their Facebook, Twitter, FriendFeed, or other public profile.
Step 2: Get gift ideas chosen for their unique interests.

Like any good oracle if the recommendations miss you can always try again by selecting "More please." The app will return gift ideas until it runs out of picks it feels enthusiastic about.

Gifts are chosen using an algorithm that finds matches using two approaches. First a natural language processing filter analyzes the things your friend has said and matches the concepts they've discussed to gifts associated with similar topics. Second, a topic modeler looks at everything they've said and creates a profile, making some guesses about what they might be like. We can then recommend gifts that we've identified for people with similar overall interests. It's this combination of "what are they like" as well as "what interests can we determine they have" that can give us a good idea about what gift they'd enjoy.

Semantic Gifts is the first consumer-facing app using the analytics suite created over the past few months under the Morning Set banner. The engine could be described as a concept mining/information retrieval program with a heavy focus on NLP using "dirty" sources like RSS and micro blog feeds, so gift recommendation is a perfect fit for the technology. The two big challenges that Semantic Gifts posed for the app were the tendency to overfit because of the often small input size, and the need to perform the analysis very quickly.

The first problem was solved by creating tiers of analysis intensity based on how much information our crawler is able to extract. Very small and very large inputs are worked over lightly, though for opposite reasons - it's easy to draw too many conclusions when you have little to work with, and you don't need to work very hard at all when they've said a lot. Fortunately the system is tuned in such a way that average blog and twitter feed sizes fall right in between.

The second problem was speed. Especially when you start to re-index statistical machine learning models at a corpus large enough to be useful, you can blow the budget of a bootstrapped startup just on EC2 processor spikes. When developing the analytics suite this problem was avoided by handling it asynchronously. Give me your data today, then come back tomorrow for your model. For Semantic Gifts that clearly wouldn't work, so we looked for components of the suite which would give us the most bang for the processor buck and also ways to "pre-bake" as much as possible. Though there is a substantial laundry list of things to add the compromise appears successful so far.

Anyway I hope Semantic Gifts is as fun to use as it was to make, and I look forward to getting some good feedback.

How Rackspace can keep Slicehost awesome

2008-10-22T14:08:00.000-07:00

Extracted from an email I sent my contact at Rackspace, here are the things I love about Slicehost that I hope Rackspace maintains:

1. Self serve sizing on the fly. I can move from one plan to another automatically online and the only change to my environment are the resources available and the bill at the end of the month.

2. Self serve, unlimited free server wipes and reinstallation. Was more of an issue early on, but knowing I have a reset button is great. The last straw with my previous host was when they tried to charge me for this and I knew Slicehost would do it for free.

3. Root access and Ubuntu Hardy. It's a wonderful thing.

4. Really great documentation. Baby steps instructions for locking down and setting up your environment that are the best I've ever used. "PickledOnion" is a particularly gifted technical writer.

Fun with Capitalization

2008-10-21T14:33:00.000-07:00

Though I'm still disinclined to talk about high level topics in NLP/ML/Semantic Hype I thought it'd be fun to bring up some little discussions about specific features. Today I'm going to talk about my approach to deciding which features should be considered when deciding how much more likely a word being capitalized is to that word being picked as a keyword in a document.

So the clean class of capitalization consists of three states: lower case, Capitalized, and ALLCAPS. I treat camelCase as lower which is working ok for my purposes so far, though I haven't parsed any Java blogs yet.

Moving outward there are two positional features that are immediately required, "Begins a Sentence" and "In the Title". Both of these pull the capitalization coefficient down, though both are positive indicators in aggregate.

For my purposes I don't currently take it further into literal positioning - whether a capital letter is in the 14th position in the document or sentence is statistically relevant, even more so because I deal with small documents, but its range is within my margin of error so it isn't the best way to spend my time right now.

Grammar is certainly a bigger deal, but so far I haven't seen it diverge much from its component probabilities. What I mean by that is that the odds that a word tagged NNP (proper noun) is a keyword plus the odds that a capitalized word is a keyword is roughly the same as the odds that an NNP Capital is a keyword. It isn't something I've crunched though, so I may adjust my opinion down the road.

Same goes for "grammar position" - whether the word was preceded by a verb or a pronoun has significant predictive relevance for keywording but my instinct is that capitalization is barely associated with that number - again the component probabilities have sway.

After that we get into the magical, non-linear universe of composite features that aren't in the scope of this blog. I mention it mainly because I believe dependence features for grammar and literal position become relevant at that cardinality.

Another capitalization topic that is harder to winnow out with seed data (because we present one permutation to the user) are words that appear with multiple capitalization profiles across the corpus and in a particular document. Three cases can give throw us a life preserver - Person's Name (the lower instance was either a typo or we need to disassociate their scores as in "Lily picked a lily" ), Begins a Sentence, and Appears in Title (the lower instance is probably the correct one). Easy peasey, but it's the cases where we don't have this helping us which are challenging to me.

A most likely cap profile in document weighted along with cap profile in corpus is the most intuitive approach ( 43 times we saw the word appear mid sentence capitalized and 23 times lower so when we present it we'll capitalize it) but I see two problems. First, we're keywording very short documents. We'll very rarely see the same word appearing but handled differently mid-sentence twice in the same document, so we lose that nice context. Second is that my gut says there are features at work that are definable better than simple probability...

Across the corpus we can use an author's upcase or downcase bias coefficient (not too hard to figure out). If they have a strong one to help us we're set, but this is easy to overfit unless you're using microblogs where MR. ALLCAPS is fairly common. Which we are. But what about in the case where our problem is in two instances from the same author? What does the instance which is bucking their capitalization bias tell us separately for an upcase or a downcase bias?

When trying to figure out whether to weight more heavily the downcased or the upcased instance, what we're really asking is whether the downcased version was "uncharacteristic carelessness" or the upcased version was "uncharacteristic overemphasis". Remember they had a version of each. Two good ways to help determine that are the capitalization of the words around the word - whether they're anomalous the same way (up or down uncharacteristically - but be careful as this is extremely easy to cause an echo chamber effect if you let each word affect each of the others) and slightly more cleverly how many hits we get from the slang table compared to the author's normal "slang quotient". I have a theory that emphasis and carelessness will both hit the slang table with a similar spike - though on different words. Similarly we can compare their misspelling rate on that sentence to their normal rate of misspellings, and we expect to a similar uptick when they are being uncharacteristically careless or prone to overemphasis.

If none of that helps me decide, I just capitalize the darn thing and move on.

I should qualify all that by pointing out we're just talking about how present the keyword as a final product. When we read a doc we store the cap profile of everything we get then downcase and stem everything for crunching and comparison. The challenge I'm discussing here is the best algorithm to "reconstitute" the stems when you have multiple cap profiles for the same word.

So that's where I currently am on answering the question "If a word is capitalized, how much more likely is it a keyword than if it was not?"

Tricks for language agnosticism

2008-10-09T12:49:00.000-07:00

When evaluating different approaches for a non-trivial problem like NLP I've found the libraries created by others to be invaluable for benchmarking different techniques. Unfortunately these libraries, kits, and code snippets are written in every language under the sun and are in various stages of broken, so some native language programming is generally required. For me the ability to try out different algorithms in any language is critical for being able to be the sole technical resource working on challenging problems. Here are some general tricks I've learned:

1. Develop an "interlingua"
In the spirit of proto-Esperanto and the machine translation concept, the idea of an interlingua in programming for me is a set of algorithm syntax broad enough to cover most programming tasks but general enough to be applied in most languages. Though language-unique techniques are often critical for tuning a production-ready app, straightforward code expressions are ideal for iterative programming cycles and ramping up newer programmers. I usually use simple loops and conditions and break apart multi-stage tasks onto their own lines. Basic classes and methods in a standard MVC structure work in most environments. One exception to this approach: ORM. When using a language which can handle my SQL for me, I'm happy to do whatever it takes to get that off my plate.

2. Use batch processes liberally
One of the hardest elements of creating a multi-lingual application is the "putty" layer - making code in one language talk to the other. Interoperability is eventually necessary for threaded or synchronous tasks, but if you're just testing an approach you can often skip this layer. Try to find a way for both codebases to talk to the same data layer, and then run one as a batch process. MySQL extensions are fairly ubiquitous these days.

3. Get good at scripting
This goes right along with number two... the better you are at scripting Perl, Python, or Ruby, the more you can massage the data going in and out of the unfamiliar languages' code and the less native programming you'll need to do for testing purposes.

4. Invest in the native runtime
When experimenting with a new language it is important to be able to do trial and error and iterate quickly. It is worth the investment to write good make/ant/rake files and check the source into subversion for testing. This will sound silly, but also make sure you write down the directory structures and execution commands for each environment you'll need to remember.

5. Have a Escape Plan
Obviously you don't want to end up with a seven-headed hydra of a production application, so you should have a sense of how you're going to detangle your technology stack. Generally this means rewriting the modules using the approach you end up with in one of your home languages. Personally I prefer a two language stack - one interpreted language which is fast to develop in and one compiled language which performs well.

There's no one language which offers the libraries that all languages do and for more intensive research-related tasks a multi-language approach can save a huge amount of time. If you follow the KISS principal and the "build one to throw away" approach, you'll be able to get hands-on experience with many more techniques in areas like computational linguistics, image mining, or AI than you could with a single stack.

Monthly Roundup

2008-08-27T14:10:00.000-07:00

In the spirit of Philippe from Achewood's mini newspapers, here comes a special boy!

I don't like Quizno's as well as other sandwich options like Specialty's, Grand Central, or Jimmy Johns because they provide too many condiment options. "But Adam", you say, "you're the condiment king." And it's true, I am the king of all condiments. But when given too many options I forget all of my logic regarding "flavor profiles" and end up with a sandwich laden with way too many sauces and pickled vegetables, rendering it overly viscous and challenging to eat. Whereas at Jimmy Johns (who has better bread anyway) I don't have free reign over my condiment selection, and so I end up with a much better balanced sandwich. Yes that was an allegory about interface design disguised as a story about sandwiches.

My Friend Feed and Twitter profiles are locked down now, because privacy issues freak me out. Before I hit "submit" on anything I think through my standard trio: "What would a future boss think", "What would a future investor think", "What would my mom think", but it still gives me the heebie jeebies that the first three pages of Google results for any nerd I know are social networking sites. You know it ought to make me feel better because my previous paranoia was about how easy it would be for anyone with a grudge and a knowledge of SEO to seriously mess up your life. Which is still a concern, but as someone who doesn't have any SEO gurus upset with them and is careful enough about what they say publicly to land a job in Human Resources, I wonder why no one else seems freaked out about this stuff. We live in a culture of umbrage and use a medium of absolute permanence. This should scare you.

My eee PC is really pretty awesome. It plays WoW surprisingly well (using an iPod for the hard drive), is the perfect recipe holder in the kitchen, and goes everywhere with me. It is named "Sweetness" and my big laptop is named "Dreadnought". We actually have six computers in the house and now with Google App Engine that makes three separate host environments. Have you ever heard of anything so nerdy?

In much the same way re-urbanization is the aspiration of the successful suburbanite and Whole Foods "bobo"s are the next paradigm from the "any food, any time" Chilean agro-revolution, I would like to posit three more ideas which may be counter-intuitive to our consumption culture but I believe resonate better with the human condition...

True luxury is throwing stuff away. Living with all my possessions strapped to my back for 5 1/2 months taught me this one but it has been a fantastic lesson. The greatest luxury is being able to get rid of things (or donate or craigslist) that you don't have a near-term use for. You can live in a smaller space with less to maintain and less to clean and less to worry about. And your space will look better.

Do you ever write the same blog post over and over but decide that is comes out not quite right each time and redact it? Ideas two and three fall into that category I'm afraid. The human condition is too feisty a topic for this Director of Human Resources wannabe. Take care of each other and maintain a healthy perspective regarding your bad self and you will be fine.

I hope you've enjoyed the potpourri... I'm giving up on the Fixing the Semantic Web thread, to be honest I have not felt like I've done a good job at making a compelling case for why it is going to be totally awesome for blogs to create and distribute their own colloquial nomenclature dictionaries which are tied into the big aggregator/recommenders using "similarity quilts". Take my word for it: it is totally awesome and if I don't get there first someone else will. Which is a more distinct possibility than usual - I haven't programmed Python before (Morning Set is Rails based). There are some aspects which appear to be easier to do using Python... certainly it is a more well-worn trail from the math perspective.

Beware Ten Queen Off Suit Startups

2008-08-06T15:48:00.000-07:00

There was an article on Hacker News today about a startup putting a brave face on news of a bigger competitor beating them to the punch. I don't know any of the details of the startup - they're in sealth mode, which is another rant altogether - but the tone reminds me of an analogy.

In poker, drawing a bad hand like 7-3 off suit is not a big deal. The normal play is to fold before it becomes expensive, often before the flop. 98% of startup ideas are 7-3 off suit ideas after a cursory analysis of feasibility and business case. You spend a few days cautiously optimistic while you do some research and throw together a prototype until you learn about the conference in Bern they've had every year since 1979 to talk about the field you thought you invented, or you find a link to Google YourIdea Beta, or otherwise get your comeuppance.

As poker hands get better, the danger of losing your shirt increases. We have a tendency to ignore the odds and chase bad money with good. What might have been a winning hand "heads up" quickly diminishes in probability when several players re-raise. But the temptation to stay in the game kicks in, and we find ourselves losing much more on good-not-great hands than on bad ones.

Startup ideas operate much the same way, and it seems like every day you see some of the same rationalization process that keeps people in poker hands long after they should have folded. Often it's much better to pack it in and move the project in a different direction than to continue chasing a non-existent market. I can speak from personal experience when I say that it SUCKS to acknowledge failure and move on (start over) with the next idea, but it's much better than the alternative.

On the other hand (meaning I'm about to contradict myself) there is the maxim that every good business idea can be rationally argued against. You can come up with a feasible reason why any unproven business plan could fail. A huge corporation could always change direction and squash your startup. The market could fall out. There are always risks and everyone successful has had to take them. Sometimes pocket aces lose - sometimes even to 7-3 off suit. Obviously that doesn't mean you should fold pocket aces.

The trick is being able to take off your rose colored glasses as well as your "fail" tinted ones. I believe that the ability to provide a reality check is a significant reason why teams of two or more founders are successful so much more often than solo entrepreneurs. Being a good poker player means being able to take the size of the pot into account - while detaching oneself from the percentage of the pot that came from their own pile of chips. Being a good entrepreneur means being able to analyze the opportunity for your startup to succeed regardless of how much time and money you've already invested or your desire to hold onto that euphoric feeling of having "a good idea". I am not sure it's even possible to be completely objective ourselves, which is what makes cofounders and trusted advisers so important.

Rake + Cron = Goodness

2008-06-19T13:28:00.000-07:00

Our new site does some crawler tasks that run periodically throughout the day. While in development I just kicked these off using an http request from my browser, but moving to something resembling alpha we needed to get them on a schedule. Since we're running on a Joyent OpenSolaris Accelerator we have access to old-school cron goodness for scheduling, but we needed access to our Rails application environment - simply creating a cron task to execute a .rb file wouldn't cut it.

Fortunately rake is there to save the day.

Step 1 - Create your rake file
Touch a new file called yourname.rake and put it in /yourapp/lib/tasks. I called mine 'update.rake'. Add the following:

namespace :update do
task(:say_hello => :environment) do
puts 'Hello, World!'
end
end

Putting the file in lib/tasks and using :environment makes the script seemingly magically aware of your rails environment, so you can use ActiveRecord and everything else in your application.

Step 2 - Execute and Test
Next navigate to your app's home directory and run using your environment:

rake RAILS_ENV=development update:say_hello

you should see...

Hello, World!

Step 3 - Schedule
Next set up your cron job as you would normally. With Joyent the "Schedule Cron Jobs" section of Webmin makes it a snap, but there are as many ways to use cron as decades since it was created. Set up the following command using the path to rake on your server (use "which rake" if you aren't sure) and your rails app.

cd /your/rails/app && /opt/local/bin/rake RAILS_ENV=development update:say_hello

Now you should be able to set up application-aware rails code which executes whenever you please.

Extra Credit
You can make your rake script more object-oriented by putting real Ruby methods in your rake script. Check it out!

namespace :update do

task(:say_hello => :environment) do

delegate_hello

end

def delegate_hello
puts 'Hello World!'
end

end

Startup Hats and Combinations

2008-06-14T17:51:00.000-07:00

In my last entry I alluded to the discrete hats that somebody needs to be wearing in any sort of web startup. Others have expressed a long list of roles, and just the other day on the Seattle Tech Startups email list someone inquired about how they could get into a Project Manager role on a startup, so it's clear there is some question over what is needed. So, here are the hats according to me, and the configurations I've seen work to cover the bases well.

Hats

Developer - Someone has to build the thing. You are generally much better off with MacGuyver than with "Wall Full of Certifications Guy", and absolutely avoid a programming theologian/idealist. They must be very fast, good at writing straightforward, readable code, use but not go overboard with object orientation, and generally be more proud of the site they create than the structure of the code they wrote.

People Skills - Evangelist, salesperson, press releaser, blogger, pitcher, glad-hander, copy-writer, PR strategist. The face and the voice of your company.

Computer Guy - Linux/Solaris Admin, caching strategist, cluster deployer, migration script writer, database maintainer, scaling planner. Someone who giggles when you say "sudo make me a sandwich".

Right Brain - Someone who knows their way around Photoshop. A user experience design pro - pixel pushing and clickstream, ideally with freehand ability for the logo. Color, font, and CSS person.

Product Specialist - Social networking sites can skip over this one, but many niche startups will have a unique skillset required for what they do. For us it's a Mathematician, but it's as specialized as your startup.

Combinations

Most startups won't start with five people one wearing each hat, nor is it maintainable for one person to wear all five - it's overwhelming. Natural combinations are between:

Right Brain - People Skills and Developer- Computer Guy

I've made the case previously that from an hours-of-work perspective that one founder wearing the developer hat isn't enough and I still think so... to me an ideal pair is:

1. Developer-Computer Guy-Product Specialist and 2. Developer-Right Brain-People Skills

Outsourcing

You periodically see MBAs (People Skills) who have "an idea" and try to get funding so they can outsource everything else. More power to them. In my opinion Developer and People Skills are integral and can't ever be totally outsourced.

Right Brain is the easiest to farm out... in my experience it's a gift your founders either have or they don't - you can study enough to fake it and get by, but inspired user experiences come from naturally talented professional designers. It's also relatively narrow-term. Once you have your UI and "experience arcs" defined, this hat is largely pixel pushing until your feature set expands.
Computer Guy is tricky. On the one hand getting set up and developing a plan to scale is very short term, but you want more than a contractor's phone number when your server says it's in "maintenance" mode at 3 am. Most good developers at least know their way around a unix box, which is enough to get started. I can't imagine a core founder who only brings the Computer Guy skill to the table for a web app, but I suppose it depends.

People Skills - A true passion for your product is the best PR skill there is. The debate over hiring a PR firm will be raging years from now, and I've never done it, so I can't say. But I do think it's important for one founder to be able to make friends and influence people, and I've yet to meet someone who could do that who wouldn't make a fine People Skills guy.

Outsourcing development is another eternal question... if you can use it to turn dollars into an earlier release date it's a lovely idea, just make sure that you actually can. If you don't have someone on staff who can review the code you're getting you will probably come to wish you did.

All that said, I don't know any perfect teams. Motivated, passionate people can move mountains, and often do a much better job at tasks they have no experience with than professional contractors would.

Observations on Early Startup Life

2008-06-12T14:38:00.000-07:00

Lately I've been reflecting on this fantastic job of mine, and if you'll pardon the solipsism I thought I would share some observations on what it is and isn't like.

I should preface with a little bit more than usual about what my job is... I am a full time startup co-founder building a not-quite-beta startup. I am lucky enough to be able to at least fake at wearing all the primary hats, so I do everything from programming to pixels to PR. I've been doing it and some small business contract work on the side full time for about nine months.

Founder Types

They say there are two kinds of founders, "builders" and "entrepreneurs". I have found this to be true. Builders think about problems and their instinct is to wonder what solutions could solve those problems in an elegant way... entrepreneurs think about problems and instinctively wonder what untapped markets could be opened by solving them. The best litmus test I have found is to ask "If you were wealthy (whatever number that means to you), would you still be doing this?" It isn't perfect; there are plenty of entrepreneurs for whom finding "angles" is an addiction, but builders would definitely be creating products if they didn't have to.

Bad Reasons

If you decide to create a startup because you want to "be your own boss", you're making a mistake. You'll always have people to answer to in some form or another, whether they're clients, investors, or customers. For general headache-factor, a decent boss beats any of them. Give me Voltaire's "enlightened despot" or even Edward James Olmos any day. You can't avoid dealing with people and their quirks, and in the end you probably wouldn't if you could.

If you create a startup because you want to "make your own hours" you're making an even bigger mistake. There are plenty of opportunities to work in your pajamas in 2008 and still be on someone's payroll. To do the startup thing right you are going to be working all the time. The biggest difference between a founder and a telecommuter is that the telecommuter gets to clock out at the end of the day, whereas the founder has "dinner break" before the late shift starts. The lines between "work" and "non-work" tend to blur. Additionally, I am most productive when I don't sleep in and prefer to work at a desk a lot of the time, so I'm in an office co-op called Giraffe Labs. Logistically speaking, the startup is an awful lot like a flexible corporate gig.

So what makes it such a great job? The fact that the things you create are your babies. I feel intense pride for the software I've had a major hand in the creation of. The hours are ridiculous, but the definition of work has changed completely. I'll still be "The Wantrepreneur" until I have a liquidity event, but whatever happens the experience has been a huge success... which I realize makes me a "builder".

Twitter is the new MySpace

2008-05-30T08:44:00.000-07:00

I remember when MySpace was the hot new thing. Everyone I knew had an account; it felt much cleaner than Friendster, and you could look up bands and find fuzzy recordings that you could then take back and embed on your profile. The track would play in the background while your friends checked out your CSS templating skills. It all seems very twee and sepia toned now, but I have a handful of offline friends with whom my only active connection is with our MySpace friendship so I leave the account open to atrophy.

Twitter is rapidly becoming the same for me. Anthony Stevens wrote this morning about how the decision was like super-delegates choosing sides. Without getting too political I think the metaphor is apt only if you consider the demographics which tend to come down on either side of that particular decision. Chris Pirillo said it best that Twitter is "asynchronous IRC" - a raw feed. The loudest voices dominate my twitter window, and the noise is deafening. Finding a "new voice" is like a needle in a haystack.

FriendFeed is like Twitter with actual conversations. Someone marks an article from basically anywhere, and it forms a locus for comments on that particular topic. No more sifting through @s and "in reply to's" that don't even point to the same post anymore.

The second big advantage of FriendFeed is in the ability to widen or shrink the pipe you transmit and receive. By opting into services you control what you communicate, and with the hide function and rooms you have a dial to control the gain on the noise stream.

Third, FriendFeed is an amazing place for finding new voices. When a friend-of-a-friend comments on an article you can get a feel for their perspective as an introduction, rather than finding out where they had lunch. If you have something interesting to say it is easy to make your voice heard in context, something virtually impossible with Twitter.

Armchair pundit: the next great movement will be Seesmic + FriendFeed. Video is spectacularly expressive, and in some ways the barrier is actually lower than with written commentary. The main downside is the inability to skim... and on video especially people tend to ramble. I predict new cultural memes for mini-video brevity to fight the problem, followed eventually by software organization... FriendFeed for Seesmic.

In the meantime I'll keep my Twitter account because a lot of friends and colleagues use it, but there is no question in my mind which technology has the edge.

Can UGC really go bad?

2008-05-27T09:21:00.000-07:00

There was a ReadWriteWeb post this morning from Sarah Perez about the potential for negative feedback in a blog 2.0 context. After watching a couple of startups self-destruct after getting critical write ups it's clear that even for web companies the changing structure of corporate communication has not universally sunk in.

The lines get blurry, but there are two broad categories of feedback which jump out: call them "trolls" and "critics". A good place to find some old fashioned trolling is in the comments section of most articles on TechCrunch; an especially feisty example during the Wired spat here. Does a reasonable reader believe Michael Arrington is a #(&$ #($&##@ ? Of course not... I like to think of crackpot trolls as a status symbol like paparazzi. Name calling and ad-hominem attacks (DH0/DH1 on the Paul Graham scale) can be safely ignored in almost all circumstances, and your readers will do the same. They hurt your brand in the same way someone littering a fast food bag hurts that company's brand. Given the opportunity you'd rather craft the message differently than seeing your logo in a ditch by the side of the road, but most customers will be minimally affected and any response by you would absolutely make it worse.

Negative write ups are the second best thing that can happen for your startup from a PR point of view - after positive ones. In this day and age there is such a thing as bad press but I'd argue only in the contexts of HR or accounting. An early adopter interested enough in your product to discuss why it doesn't fit their needs is bad only for your ego. First, it gets attention to your product, and second it's great constructive feedback. There is an embarrassment of riches in terms of product feedback available today, and I think we're just seeing the first glimmer of companies tapping into that in an organized way. Companies that are already doing so have the added benefits of being seen as progressive and responsive.

Ira Glass on making things that suck

2008-05-24T13:51:00.000-07:00

This was posted on Hacker News this morning and I thought it was worth reposting. Failing sucks, but you can't do this and be afraid to. There's no magic formula that some people are born knowing, just a few lottery winners and a few others who work harder than everyone else.

Instant Rails = Epic Win

2008-05-20T13:39:00.000-07:00

This week Lappy 9500 had a little zombie problem in its hard drive (for the second time in nine months which is how old it is), so I had to switch to the creaky old silver Dell while I wait for a new hard drive to get here. Reasons why I should have bought a Mac aside, my eyes were opened to the wonder which is Instant Rails.

The migration of my desktop development envrionment consisted of:

Step 1: Copy C:\iRails folder from freshly bitten undead Lappy 9500

Step 2: Paste iRails folder onto C:\ drive of Dell Slowpoke.

Step 3: Recognize.

Seriously... Database: check. Web server: ready. Environment variables: check. IDE, are you kidding: check. Codebase: check. It was glorious.

Interestingly getting set up to use Heroku on a new Windows machine is orders of magnitude more complicated. I understand cloud hosting vs. local environment, but if the story is how super easy Heroku is to use to get up and running with RoR, there is no comparison, it fails in comparison to Instant Rails. My other gripes about Heroku are the lack of a console window and the impossibly small pipe. Building an app on free-beta Heroku can feel like riding a Ninja 250 - the rev limiter is pegged right from the start, and it starts throwing errors above 55 mph.

For the record, to get Heroku back up on Silver Lappy would require installation of Firefox, 7-Zip, GitBash, my codebase and data scripts. I know, it's apples and... green apples, but for keeping me working seamlessly Instant Rails gets my gold star this week.

While I'm digressing on scrappy bootstrapping tools, I also installed ColorPic and the Gimp. ColorPic is a lightweight way to get the hex for any color you find, then fine tune it on your site using the raw hex values. Gimp is a sad pale shadow of Photoshop, but it's free. It isn't good for details or freehanding, but if you just need a curvy shape, a color change, a scale or a crop it gets the job done. Microsoft Expression Design is targetted at doing just this sort of thing and is comparatively really good at it, but if you're going to spend the money my opinion is that you might as well get Photoshop.

I want to hold your hand

2008-05-15T14:45:00.000-07:00

I am coming around, guys. An average tech-savvy-ish American: uses iGoogle widgets so they get news from various sources every day. But doesn't what RSS is, much less why they'd want to export their opml file. Twitter sounds like other peoples' business, and FriendFeed sounds like the sort of thing the government is up to.

So... they have a sense of getting headlines from multiple sources, maybe even more than what Google News can provide. They might even do a little light configuration if there was adequate hand-holding involved. Google are the *masters* of hand holding.

Kevin Fox once criticized a post of mine in FriendFeed; It was a smidgen out of context at the time, but it is dead on from a market positioning perspective:

"It's ever expanding circles. First design for Louis Gray, then for the other 50,000 early adopters, then for their 500,000 friends, then their moms. If you design for their moms first you'll never get the early adopters and moms won't use a service without momentum because they're not good at seeking out the new. Mind you, this is for actual startups. Products with guaranteed initial exposure (like anything Google launches) has different rules. - Kevin Fox"

The actual sentence in question was "When designing UX think about what your mom would use, not what Louis Gray would use." Which is lucky, because I have struggled mightly with the realization that the project that we're going with is in some ways just more duct tape in a web 2.5 environment that seems to be ALL DUCT TAPE. It is some consolation that our idea is dedicated to 'noise' elimination, but in a fit of pique I might very well have said "Forget the echo chamber, let's design it for everyman!"

But what I (fortunately) was going for in that statement was that I want the UX to have that "hand holding" feel Google does. I want you to be using the site well before you even realize you are doing so. Because frankly, I may be an earlier-ish adopter, but I'm everyman too. I smiled in real life when I found out how easy it was to set up Google AdSense. I still get frustrated and quit a new site at the first counterintuitive speed bump or forced account creation. Almost no site "takes" on the first try with me. I would argue that we never have a 'hair on fire' use case - when our urgent, unfilled need makes us willing to struggle through inferior interfaces and functionality - when it comes to news or networking.

So for product positioning by all means target Louis Gray and the other first-adopters, but when you're designing an interface make sure to hold your users' hands.

photo credit vc wear

Where do you comment?

2008-05-13T09:40:00.000-07:00

Twitter, FriendFeed, and RSS sharing are swiftly ensconsing themselves as the primary modes of communicating interesting information among friends on the web. Though each has a unique role, there is a significant amount (>50%?) of redundancy between them when a user identifies an interesting piece of information. Currently we have great tools for what I believe are emerging as a distinct set of use cases, but we have some redundancy around sharing and commenting that hasn't sorted itself out yet.

The RSS reader is where users find the information they think would be useful to their friends. First, they comment on the initial blog post itself, which gets broadcast using a comment network like disqus or intensedebate. Then they elect to share it in Reader, adding a note. It shows up in your google reader friends' RSS readers as well. Friendfeed also picks this up and broadcasts it to your network there. Finally, you go on twitter and post a tinyurl to link to the article, which also gets picked up on friendfeed.

It can actually get much worse than that if you have mobile listeners, facebook, chatterous, etc.; I use the others because they constitute the main use cases: primary source, a reader of raw material, a streamer, and a broadcaster. The others (while plenty useful) simply rebroadcast broadcasts or stream less comprehensively or read in a more prepackaged or esoteric way. To put it another way, you can insert whatever brand name you prefer for commenting, reading, broadcasting, and streaming/recording.

So, here is my open question: What does it mean when you comment on an article on the article itself vs. in your RSS reader vs. Twittering about it vs. commenting on a share in FriendFeed? Does it depend on your networks for each - i.e. you think one story would appeal to a wider audience than another? Is it random? Would it change if (when?) everyone on Facebook used these tools every day?

For me, my RSS friends are the smallest group, my twitter friends a bit larger, and my friendfeed network are (ironically) usually people I don't know personally but follow because they find interesting items. I don't generally comment on friendfeed items unless it is in reply to a friendfeed comment someone else has made, mostly because I feel like it interacts with a very small audience. I only broadcast on Twitter if I think something is pithy, funny, if it is or pertinent to an individual I know (also ironic). I am trying to do better about commenting on blogs because I like it when people comment on mine, but I have a sense of writing it in stone, which I am loathe to do because I am never sure-sure. It isn't rational. Commenting on a share on an RSS? I haven't figured that out yet, so I generally don't do it.

Suggestion, Recommendation, and Filtering

2008-05-11T22:13:00.000-07:00

As I mentioned in a previous post, Bayesian algorithms have different sets of criteria depending on the expectations of their users. I find three levels that applications fall into.

Purely implicit suggestion algorithms by definition have the lowest level of interaction - they measure what other users have been interested in in the past, compare it to the things you do, and using some reasonably complicated math make suggestions based on that information. Targeted advertising campaigns make sense for this approach - most people aren't willing to make an effort to make it easy to advertise to them, and expectations are zero that you'll be advertised to, and you don't need any trust at all to make them effective.

Amazon is an example of a purely implicit approach taken as far as it can go... and maybe a little too far. An advertising algorithm that directly interacts with users by making explicit recommendations - it tells me "Hey Adam, we think you'd like this..." as opposed to a passive advertising engine like adsense. Unfortunately since the algorithm is implicit I don't have a way to tell the algorithm that the South American Flute music CD was a gift 5 years ago, and I have no interest in the emails they send me suggesting other world music.

Without any more information than the things I buy or view however, the ability for this algorithm to provide useful suggestions faces an asymptote as the math behind it gets better and better - my interest profile snapshot at this moment in time is much more complex than can be gleaned from my historical shopping data.

Recommendation is the next tier up from suggestion. The difference is that suggestion supplies data without a question being asked, while recommendation is in response - potentially abstract - to a question which the user has posed. Pandora, Digg, Delicious - we have become accustomed to interacting with applications that take our input and use it to make direct recommendations. They still use Bayesian algorithms of course, but they are fuelled with direct feedback, which can eventually provide enough information to make pretty good recommendations about things you might be interest in.

True filtering is a lot more serious. Filtering is as far from recommendation as recommendation is from suggestion, because it requires trust. You provide the raw data you want filtered, and then trust the system to give you the subset you are actually interested in and not cut out the things you consider critical. Spam filters are a great example of this. When my radio station plays something I don't like, it is much less serious than when the filter deletes that email from grandma. Alex Iskold wrote an interesting article that was applicable for netflix, but it's not a good fit for filtering as we're talking about it - when filtering messes up the analogy isn't like finding a restaurant that wasn't recommended but you liked anyway; it's more like finding out all your friends had a party that they didn't invite you to because they didn't think you'd be interested in.

I don't want to talk too much about the stuff we're working on, but I can say this: if you're primarily focused on the wisdom of crowds, ur doing it wrong. Your holy grail is the Greatest Newspaper In The World, which... misses the point. There are a lot of editors at every major paper in the world who get up every morning and work their butts off to do just that... and then Google News and Reuters skim off the cream. If you want to cobble together the world's best most popular newspaper, you're playing for scraps. No, the goal of the filtering web 3.0 evolution is going to be to finish the job RSS started, to create your ideal newspaper. Stay tuned.

Noise vs. Missing Out

2008-05-08T23:54:00.000-07:00

There has been an interesting conversation going on in the last couple of days about the "noise" Friendfeed's adoption in particular but social media as a whole foists upon the tuned in on a daily basis. David Risley's post got it started, and it ironically stirred up a flurry of noise on Friendfeed almost immediately. Robert Scoble's response was that with any new technology the adopters and their noise outpace the filtering functionality to keep it contained.

As someone who is interested - ok, obsessed - with this topic, I've been mulling it over. I think we all have a slightly different use case for these tools, and with each use case comes a different granularity that is our optimal noise filtering level - but I think we'll find that the need is the same. I'd posit two concepts, a "Perfect Filter" (an impossibility), and a "Crude Filter", and wonder how their application would affect the different taste levels on the Noise to valuable information ratio scale.

Scoble in swim trunks is a vivid image on one end of the spectrum - I persoinally like using Twitter's "Everyone" tab when I want to get my mind blown by the fire hose, but it's a novelty for me. The A-listers don't want to miss a thing, and are willing to invest the time to weed through a lot of chaff to find it - in lieu of a better system. Their experience with the Perfect Filter might have the fewest number of posts removed... slice a few trollish comments and implement way to roll-up duplicates, a nice graph representing trackbacks and the discussion nodes around the topics of the day, but by and large a similar overall set of data. This subtle approach multiplied by their amount of time using the tools might wind up benefiting the most. The Crude Filter is simply not useful for this group - it cuts out too many items they want to read.

The regular early adopters choose not to spend hours on end sifting through posts and reposts and comments and cetera about every topic that flows through friendfeed, twitter, and the blogosphere on a given day. Instead of being an annoyance the natural state is not acceptable, and so the Crude Filter becomes valuable. Give me all posts that others (presumably the famous guys) have already marked as interesting. It'd be nice to see if anything is associated with these keywords and make sure to cut the items that use this other one. Items will be missed, but in the end "If the news is important enough it will find me". The only difference between the crude and the perfect for this user are those serendipitous stories that somehow the A-list hasn't caught onto yet or more likely that appeal to an esoteric interest of yours. A filter (or inversely a recommendation engine) using math that corresponds to this temporal plane, i.e. "is doable" can get the average early adopting cool kid a whisker away from perfect. They don't mind waiting until Arrington gets back from lunch and lights the story up to find out about the latest acquisition. They can set up keywords for their interest in lolrus bukkits and catch... most articles. And do it in 10 minute breaks 2-4 times a day.

Which is what leads me to make crazy statements like "design for your mom, not Louis Gray". Since you guys don't know my mom, a better analogy would be "design for the people who have customized iGoogle or Yahoo's homepage, but spend less than an hour a day reading the news online". There are the tools available and the unmet need to really improve this very large segment's experience. The A-list is looking for duplicate filtering and article grouping, but my gut instinct is that to actually filter out unique content would require a level of sensitivity to earn their trust that the technology is not yet extant to accomplish. Or if it does, news of it has fallen through my filter...

Minimalism in website design

2008-05-07T23:24:00.000-07:00

There is a saying among long distance backpackers, "If you don't have it, you don't need it." The idea is that you can cut far more from your kit than you thought possible, and your being around to miss something implies anthropically that it wasn't as essential as you thought. Health and mileage are what matters, and the stuff on your back becomes a separate equation, a tradeoff between discomfort while carrying for comfort in camp. You quickly shed the non-essentials, then the essentials, then the dire until your pack, now well under 15 pounds in warm weather, ceases to be noticed at all. At this point you find yourself weighing, debating, and finally allowing in the smallest luxuries... a rain jacket, a small paperback, an ipod. Just enough to soften the roughest edges of the experience without interfering with what matters - health and mileage.

My favorite site designs take the same approach. A few months ago the Economist did a story about Twitter's Evan Williams I related to.

"... Mr Williams... tries mental tricks. One is to ask “what can we take away to create something new?” A decade ago, you could have started with Yahoo! and taken away all the clutter around the search box to get Google. When he took Blogger and took away everything except one 140-character line, he had Twitter. Radical constraints, he believes, can lead to breakthroughs in simplicity and entirely new things. "

Twitter my be a little baroque for my taste, but the philosophy is there. Some other newer sites that embody this spirit are Muxtape and Twistori.

Hillel from Jackson Fish Market talks about "artisanal software" with a gourmet foods metaphor, and it's an excellent analogy for startups and the way our sites work best when small in scope and carefully crafted. We need to be careful not to fall into the other trap of artisanal foods - making our hobby our business - but solving small problems extremely well is what we do best.

So to make a beautiful website: start with a scope that covers only the essential solution of a problem that needs solving for your primary use case "story". Cut all features that aren't absolutely necessary. 90% of the "good ideas" is about right. You'll save yourself effort if you can identify them ahead of time... I am notoriously bad about coding things before I realize they are non-essential. Then come up with 15-20 different visual designs that add as little as possible to the greatest effect. Think of every .png as a rock in your backpack.

This philosophy can be applied to almost everything you do or create, and it has never let me down. The less clutter you bring into your life and your work, the more effective you can be at the things you do and the more time you can spend doing what you love.

5 techniques for optimal productivity

2008-05-05T12:24:00.000-07:00

I am just coming up for air after a very intense 20 days, where I took an idea from "hey, what if..." to a stable application that does some non-trivial processing. I have found five techniques that really help me stay sane and happy while working very hard. Your mileage may vary.

1. Set an alarm. I wake up at 8:00 every morning, even on weekends, and even though I can wake up whenever I want. The idea is to help you get to sleep consistently at the same time. I find that my natural cycle moves toward 26 hour days if I don't train it otherwise.

2. Total sobriety. I do my best work late at night, which would be compromised after a few beers. I sleep better, feel better, and work better when I am completely sober. Besides, I find that when you are in love with your job you don't have any use for a numbing agent.

3. Work out. I do a quick weight circuit every other day, and get my cardio from cycling and walking everywhere I go on the hills of Seattle. Free weights and a swiss ball are all I need to stay in shape.

4. Eat all day. I am never interested in the first splash of cereal I make myself eat first thing in the morning, but I am always glad I do. I munch all day long on fruits, veggies, and other healthy snacks, and try to eat small-ish meals. I avoid "bombs" for lunch because I'll be sleepy in the afternoon.

5. Meditate. I make time to meditate most days lately, and find it very helpful for keeping me relaxed and clear-headed.

The Echo Chamber

2008-04-29T09:05:00.000-07:00

Most people reading this blog are hopefully aware of the little bubble that encircles the internet startup community's culture, mindshare, and perspective. It is one of those topics that makes for good link bait, but rarely do stories address the potential for observer error or ways to address it.

From a cultural standpoint, most people don't know what a "meme" is, much less what it means to be rick rolled or Lacyed. Which is common to every industry, association, or school, I'm sure... but there is something about the twitter echo chamber that makes them seem ubiquitous in such a way that it is particularly easy to lose perspective. Twitter is a good (if cliched) example. The vast majority of people have never heard of it. Really bright but non-bubble people don't understand when you explain it. But probably two-thirds of the new startups I've heard of in 2008 leverage twitter in some way. Think about it - their mass adoption (which is surely a success criteria, right?) requires graduation from a leap that most smart but non-internet culture people do not find intuitive.

Mindshare is the tricky part. You'd think with all the navel gazing that "outsider art" would be the norm, that the apps which find mass market appeal would be from the people who don't have the problems FriendFeed is trying to solve, i.e. 99% of the population. It just isn't the case though. Without knowing what's out there, what problems are being worked on, what's worked and what's failed, it is very difficult to pick an idea worth pursuing. Beyond competitive analysis, it's an educational foundation that is key to giving startups the best chance at success.

The power of the tools is another example of the absolute necessity of integrating with the bubble. To build a 3-tier .NET component for a web application used to be an assignment we'd set out an evening for. In Ruby it takes, oh, 20 minutes? Libraries, frameworks... it is definitely better building websites from the inside.

So as an entrepreneur, what do you take away? If you read my other postings about giving yourself the best chance of success, I think you have to completely disregard this entry, right? If your goal is mid seven figure early retirement, you should plan your strategy around a market worth mid seven figures. The echo chamber should have that. Mass market appeal is not. Additionally the buzz machine that causes you to lose perspective is an awe inspiring PR machine... at least inside the bubble. I hadn't heard of brightkite before yesterday morning... by 10am I'd seen at least a dozen disparate references.

Going after a niche is an obvious solution... maybe not a niche product like farecast but a niche market like buzz feed. Of course that requires a whole other bubble in which you'll need to become an expert in, but that comes with the territory. Either of these niches (product or market) seem to be talked about much less, but sold much more.

A really good way to deal with this paradox of needing to drink the kool aid to recognize the opportunities but losing touch with what everyday users care about when you do is to have advisors you trust who are on the fringes or outside the twitterverse entirely. When designing UX think about what your mom would use, not what Louis Gray would use. Be mindful of the potential for a PR strategy that relies heavily on "going viral" to steer your product toward the rocks of esoteria, even as you fully appreciate the spectacular power of an idea getting picked up. In essence I believe success is in walking a fine line by taking what you need from the echo chamber but not losing perspective in it.

The megalomania trap

2008-04-23T08:58:00.000-07:00

Have you noticed the way media stories - even 250 cool kid blog stories - seem stuck in the description of the startup landscape as either "the next (insert billion dollar company name)" or "so much fun, working from home weaving colorful baskets!" lifestyle companies. These two extremes paint a lot of companies' visions and set themselves up to fail.

There's something in our psychology that has trouble differentiating risk, especially the number of zeros behind the odds of success. A 1/10 success still requires you to "do something crazy" in the eyes of the careerists, so heck, you might as well shoot for "Japanese Palace in Palo Alto" money and go for the 1/10,000 shot.

And so these companies do the things companies do when they're trying to get huge and they fail, when if they'd just kept their costs down and moved into more of a niche space they could have been very successful. If they're lucky - when you swing for the fences you often don't get the disappointment of hitting singles. Fortunately it isn't that difficult to have six decent ideas and take them far enough to see how the market reacts, giving yourself better than even odds of a nice retirement. But I've ranted before about the awful space some entrepreneurs get in where they never have any ideas because they're waiting for "the big one", not realizing that the people who've succeeded with a "big one" didn't set out to conquer the world, they just went out to their garage and started tinkering on a project for themselves. That is if you even accept the idea that the big successes aren't flukes to begin with and that they have some lesson for the rest of us. What's the old saying... to have another Microsoft you'd first need another IBM to bend over?

My advice for other realistic-success minded entrepreneurs - or those pursuing "lifestyle" companies or global domination behemoths for that matter is to find cofounders who fall into the same camp. My advice for the megalomaniacs is to build into your plans for world domination a small success at first. If you are really bent on having your own 767 then surely guiding one small startup to a "liquidity event" won't present too much of a challenge.

The Cloud

2008-04-22T21:41:00.000-07:00

Like most nerdy-types I've been interested in cloud computing for awhile now. As long as I've been doing independent consulting I've understood all too well the difference between a "managed server" and a "hosted server": the former implying that some infrastructure guy was going to be handling the patching and the installing of the other scripty things that make the computer not crash, the latter meaning they would provide a server, a rack space, and the phillips-head screws with which to mount it. Clients like the price tag of the latter and don't differentiate types of computer guy, and why should they... part of delivering really really great service means that they can call us and we ought to figure it out and not make excuses like "sounds like a server issue". In the blood and guts of long term support your best hope is that when you say "sounds like a server issue" it doesn't turn out to be your program after all, and you do the best you can to troubleshoot it.

It's these campaigns in the blood and guts of long term support that make The Cloud seem so shiny and light. Not only could you get someone to manage the stuff you are tragically unqualified to do... suddenly that whole mess gets abstracted into the commodity our programmer instincts think it ought to have been all along. The electricity analogy is apt... no more tinkering around with the 5 hp engine on the generator. If a "managed server" where the host does the updates and more importantly has a sense of implicit responsibility for keeping things running is like hiring someone else to manage your generator out in the back yard, the idea of The Cloud is like being able to pull all your electricity from the grid.

Until I watched Jeff Bezos' Startup School speech this was roughly my perception of what cloud computing could offer me - getting Windows Update as far from me as possible. But the disasterous consequences of overnight success on your server situation is the true killer use case for The Cloud. 50 instances today, 3500 tomorrow. You can't plan ahead for that, you'd be a fool to build out that kind of infrastructure. You can't react to it either, by the time you caught up you'd have suffered significant outtages. But the warehouses full of computers at AWS, Google, or Mosso could easily handle your 3450 instance blip.

I've been playing with Heroku lately, a neat tool where you upload your Ruby code, then edit it right in the browser. With Ruby being interpreted it truly achieves Anthony Stevens' target of One-Click Deployment.