If We Had Done The Right Thing To Begin With

Error-avoiding is overrated. Too often people want to wait surprisingly long periods of time before making a move, in fear that their move might be the wrong thing to do. The amount of fear and pre-thought we put into projects can be staggering.

I'm not here to say that design and architecture and platform design aren't important. Clearly there is value to having a workable plan and a goal in sight.  However, the idea that you must be right on the first day of development flies in the face of all our experience with application development. You only need to be "right enough" at first, and able to correct as time goes on.

We can become invested in concepts, ideas, development stacks, and architectures to the point that we forget that all up-front design decisions are just guesses. It's akin to a man walking up to the blackjack table with the intention to draw twice on the first hand and once on the second, even though those hands have not been dealt. The problem is that many smart developers will stick to the plan even if it fails to make development fluid and predictable, but if the gambler draws an 18 he wil decide not to draw twice after all. Do we realize that design is a bet? Do we continue to evaluate it, or do we take it as law on the day it is handed down from The Architect?

There are a small number of problems with being "right to begin with":

1. You'd have to know what "the right thing" is. A lot of systems have emergent designs that surprise their developers. Larry Wall is famous for saying that he had no idea Perl would look like it does today. An application may be written well in many ways, how can you know the One Right way in advance of actually building it? 
2. You'd have to know that "the right thing" is right. In a system with many choices, any number of right-seeming ideas may be wrong. The iPhone showed us that a radical design may be more right and more influential than many conventional ideas of what is right. Countless other unnamed products have shown us the reverse. Being convinced of the rightness of their conventional design, one large mobile phone company ended up missing out on the most lucrative phone design of our lifetimes.
3. "The right thing" would have to already exist. It might be that the innovation your product needs will be invented/released/discovered six months into your product development effort. A Lean Startup approach involves constant "persevere or pivot" decisions, which can be applied to architecture as well as feature set.
4. The cost of discovering "the right thing" up-front needs to be less than the cost of discovering it through experimentation and development. If you took three months to design a system, and you could have discovered the same solution by building three three-weeks prototypes, you were not being responsible but wasteful. Always remember that design is a gamble. 

The alternative is to learn and adjust as you go. It's no surprise that this is how software projects have succeeded (when they've succeeded) over the years.

From Agile Otter Blog

Architectural decisions are hard to make once-for-all in a system where functionality and platform and development practices change fluidly. A good choice of language, platform, and early features will give us a good early start, but these decisions need to change (with requisite changes in code) if they no longer serve the needs of the business, which includes the need for fluid delivery of functionality to customers.

Maybe finding the right thing is a combination of up-front thinking and ongoing discovery; maybe it always has been.

What should the well-dressed virtualenv wear?

I'm back to hacking some python (fun stuff!) and have been noting some packages that every well-dressed virtualenv should have. I'm interested in knowing what you use, and for what.

Essential packages I can hardly live without:

• bpython - because it's my favorite shell
• sniffer - because CT is the bomb, and sniffer works on mac (autonose not so much)
• mox - though I don't totally love it, I'm somewhat used to it now. May switch to something more context-manager-like.

Task-specific stuff:

• BeautifulSoup - if I'm parsing stuff. I really like this library
• mechanize - if I'm writing tests on a web framework. Other than js weakness, it's quite nice
• feedparser - for rss stuff
• PIL - because image manipulation is no fun without it

What do you feel a well-dressed virtualenv should wear this season?

Mercurial team workflow

Dear Mercurial, I don't love you (yet).

This looks fine if you only spend a few minutes in the 2-3-2-3 loop. On the other hand, if you're in a very busy team and there are refactorings going on (and when shouldn't there be?) then you will want to have a loop from 3 to 1 pretty darned often (at LEAST a handful of times a day).

At step 1, "For a while" should mean "if a couple of commits have been made by coworkers."

The problem I have with mercurial is that the process of pull, merge, commit is too cumbersome. I have coworkers who swear by rebase, and others who have been bitten enough that they're scared to death to use rebase, and instead pile up the "merge" commits.
There needs to be some kind of change made, if only to fix rebase, because this work flow is broken for active, productive, agile teams.

The First Puzzle Challenge

Today we held the first ever Puzzle Challenge at my client's site.

The goal of the challenge was for each team to pick a work style that aided them in getting the greatest number of puzzles completed in a very short time (10 minutes per sprint). The set of puzzles to solved was a mix of crosswords, mazes, word-search, sudoku, word jumbles, and number blocks. The teams were told that there was no partial credit at the end of the ten minute sprint. The teams were given a menu of practices to choose from:

Team members could one mode of adaptation, leadership, teamwork, noise level, and task switching. A style of 11111 would mean a 1 in each of these categories, a style strongly resembling an "ideal" organization in the buttoned-down 80s. A style of 33333 is basically a productive chaos, which might have been more widely recommended in the free-spirited 60s.

Teams made an initial selection, then were allowed some adaptation.

In initial selection of work style, teams tended toward twos or threes. As the challenge progressed and they realized how hard the "work" was and how short the time was, they tended to migrate toward 33333 in their style.

Is it surprising that when the chips are down, they opted for less command-and-control to help them increase speed?

.

Teams with the highest scores did a few surprising things. One is that they "cheated" by rejecting all work they could not accomplish easily within the 10-minute span. Although they were initially told that they should do the work in the priority order (as it was given to them), they quickly realized that the system was flawed and that taking on tasks larger than 10-minutes was a waste of their time and talent. Rejecting over-sized work gave them a distinctive edge. This is a lesson most agile teams should learn.

Another advantage went to teams where people chose the work based on their skills and interests instead of having work handed to them by a strong leader or by following strict priority. Doing work that interested them, in areas where they had already developed skills, moved them forward. Even so, they found a huge disadvantage in working alone. Most puzzles had at least two people working on them by the end of the second sprint, and this trend continued through the end of the challenge. Even when skills differed, there was no other advantage as great as having a partner.

Teams were further accelerated if they found the "Work-In-Progress sweet spot."  If they had too much in progress, very little got finished. If they had too little in progress, stuff got done but scores were lower.

Some teams stopped taking on new work at the 2 minute (remaining) mark. There was little chance in starting jobs that could not be finished by the end of the sprint. Instead they spent the time talking about how they could perform better on the next sprint. Again, a lesson worth learning.

One challenge attendee said that he learned that chaos is hard to manage or predict, but it really helped get things done. I applaud his insight.

I was a little disappointed that some rebel didn't totally crush it by using a style I would never have chosen. That would have been cool, if puzzling. I suppose I'll have to settle for having my agile mindset validated once again.