The 7 Testing Phases

You probably already know about unit testing, functional testing, acceptance testing, load testing, performance testing, and all sorts of test phases you wish your company did a better job on (or did at all!).  Here is my rule of thumb, though, for what happens in a real product testing cycle.

1. Unit testing needs to be cleaned up.
2. End-to-end functional tests reveal areas that were not well thought out or are showstoppers preventing the testing. Rework and repeat.
3. Time is still left in the testing phase, so all sorts of “minor” fixes are crammed in.
4. Realize minor fixes are now causing other bugs. Stop trying to fix anything new.  (This is the actual “feature complete” moment, not the one before testing starts.)
5. End-to-end testing finally works, but things are too slow. Last minute performance fixes are done.
6. Feel like things are close, then a terrible, hard to fix bug is found.  Entire team tries to come up with easy workaround. Someone has to stay late to fix it for the next day. Issues are logged for a real fix in the next release.
7. More testing is done, more bugs are found – and it turns out these bugs were in the previous release but never found.  This means this testing cycle is done!

I will, for the sake of my own sanity, refrain from putting Step 8 (“The Patch Release”) from being an actual part of the process.  I’m sure we won’t need it this time!

Cleaning the Kitchen

One of the problems all startups have that no one will tell you about is that the kitchen is always dirty – especially the coffee maker – and despite any number of email pleadings, no one will fess up and the problem never resolves itself.  The only thing that prevents the mold from taking over the entire room are the employees who get so grossed out they suck it up and use the ancient scrubbing device stuck to the counter just behind the sink to clean things up.  After rewarding themselves to a fresh pot of coffee and a snack on a clean plate (perhaps even eaten with a clean, metal fork!) the next day the kitchen has reverted right back to its normal form.

This is somehow symbolic of how all aspects of startups work – everyone has some tolerance to the messiness inherent to how things work.  If something gets too far out of whack, someone has to step up and clean things, even if they didn’t create the mess.  Hopefully you can figure out who is causing the messes and give them a real talking to, so things can improve for a while.  But mostly you just keep things functioning one way or another in hopes that someday you will be able to afford that fancy espresso machine and a cleaning staff to make it shiny every night.

Or until you get bought by a big company that stops providing free coffee and moves you to a cube further from the coffee machine than from the nearest coffee shop.

At a previous job, the guy who used to come by to fill the free beverage refrigerator told me that one of the first signs the company was going downhill was when his contract got canceled.  We stopped getting the good juices a few months later, and I wasn’t sure if that was good, because we still had some free options, or bad, because the cost cutting had begun.

Recruiting, or Why Don’t We Have Better Free Food

Writing these entries is becoming more and more difficult because I am resisting some of my natural inclination to be bitter and sarcastic.

Recruiting for a startup can be very challenging, especially if you are doing something boring (i.e. something businesses might pay money for) instead of something sexy (i.e. anything given away for free.) Have a solid business model and reputable clients? You might be better off not bringing that up, because most people interviewing seem to be more interested in the free perks and the sexiness of the industry. Doing something in online music? Good. Social networking? Even better. Does it matter if you are offering ISOs or NQSOs? No. Sure, people like to work for companies their friends have heard of (or gets them laid) is a good thing, and being able to combine your outside interests with your work can be fantastic (until it ruins your outside interest).  In my experience, though, working with good people you like trumps what you are working on, and free food does not make up for capricious management.  If you can get the free food and work on sexy things with good people, jump at the chance.  But if you have to prioritize, be careful.

So how do startups recruit effectively?  It can be difficult for people who have only worked in large companies, or are coming straight out of school, to understand the benefits of a startup environment.  And it isn’t the right fit for every personality.  Try to find people who get excited when you start talking about having freedom to try new ideas out quickly.  People who are excited to have fewer barriers to getting things done.  People who like to learn and do things beyond the scope of what their official role usually entails.

My father in law would also recommend avoiding people who play golf – he thinks people who play a lot of golf have too much free time.

Maybe this is less of an issue if you are a star entrepreneur, but for most startups you need to make sure you can sell yourself to the people who actually do meet all of your criteria.

The Worst Thing About a Startup

The worst thing about a startup is what we’re going through now: having to get rid of someone. It is bad enough when it is clear the person is not a good fit – sure, it will be better for both you and them in the long run, but the timing is never good and it never feels good to go through the process. It is even worse when the person is a reasonable fit but you have to let them go because of cash flow or other indirect reasons. How do you tell them this? “It’s not you, it’s me” seems a little trite for such a life-altering thing. I’m sure in this case the people involved will be able to find something else quickly – maybe something better. I’ve heard the stories from the old days where someone was laid off from SGI and soon found a job at a little startup called eBay (pre-IPO!). I think the people who retained their jobs at SGI were the ones who felt bad in that situation once the friend made their millions. Is that what makes this worse? The feeling that this wouldn’t be happening if the company were successful? If we had made better decisions? If we hadn’t wasted that money earlier on overpriced, ineffective sales people? If we had tried to get that next round of funding earlier? I’ve often heard executives use that as justification for their high salaries – they need to make these high-stress decisions that affect the lives of everyone in the company, and they need to carry that responsibility.

Somehow, though, making that decision cannot be nearly as bad as actually being the person who loses their job.

Anyways, I will hope this is not the first of an ongoing series as I keep finding new “worst things.”

How Microsoft Exchange Ruined an IT Department

I have seen this happen:

The company executives want to have outlook so they can use it to schedule meetings.  This means they need an Exchange server. This means they need a windows server! Of course, since you have Exchange, all email gets moved over to exchange, even for the devs that still happy using Pine or some other obscure email-reader that Exchange doesn’t support well and can’t take advantage of the calendaring (which may just be smart ploy to avoid meetings).

Well, that should not be the end of the IT department right?  But the problem sets in quickly – you need a real Windows admin to handle Exchange randomly crapping out, especially as the execs keep sending huge powerpoint presentations back and forth and demanding their mailbox sizes get increased so they don’t need to learn what “autoarchive” is.  And this may be the company’s first real IT staff, since the devs used to handle all the unix admin stuff part-time because there is always some dev that enjoys that kind of thing (or at least used to enjoy it, until they had to do it part-time for the company.)  Also, now that you have the windows box, you decide to try to use it for other things since it isn’t at full capacity yet – internal website, shared folders, whatever it is.

And now you are screwed.  Because as the devs give up control over IT stuff, it is taken up by a Windows admin.  Smaller internal projects are solved with windows-only solutions.  And all of a sudden, everyone needs to have a Windows desktop. And use Outlook. And has tiny mailboxes.  And hates the IT department.

So how do you avoid this fate, other than not having execs, which is tempting but difficult to get funded? If you really have to go down the Exchange route, buy it as a hosted service, not as something you will manage internally*.  This is so much easier, and it won’t be able to creep into other parts of your IT structure.  If you are really daring, try to avoid the Exchange thing completely by using a different calendaring server – although, honestly, even these days it is difficult to find something that works as well as Exchange does.

Not that I would ever say that in any official recommendation I would make!

*This is actually the entire solution to how you should manage your IT department as a startup: don’t do it. Don’t rely on your devs to do it. Outsource it with hosted solutions – email, backups, wiki, website hosting, datacenter, whatever.

Team Building through Sports Analogies

I worked at a place where you could have made a great drinking game during our CEO speeches at all-hands meetings just by drinking during each sports analogy.  2 drinks if the analogy was left incomplete!  Only 1 drink if it was totally incomprehensible.

It was like some company-wide team-building activity.  Let’s get some cute mottoes (mission statements?) and a big banner and have a scavenger hunt and then we’ll be a team!  And we’ll win!

If this is what motivates your team, and your team only interacts outside of work during the once-a-year forced (required! mandatory!) team outings, it is not a good signs.  Good teams enjoy each other as both people and as co-workers.  It comes through a respect that each team-member is contributing something useful, management is appreciating the good things being done (with credit being given to the people actually putting in the effort), and any stupidity is noticed and there is a plan to make it go away.  That is the situation where people will go above and beyond to make things successful.

That is why startups are so addictive – this kind of environment is so much more likely than in a big company, where you are likely to be lost in a crowd of mediocrity, abused by a manager trying to make their way up the ladder, working on a project that will never be released to the public, unable to explain what you are doing to anyone not in the industry.