XP Redux

This is about software development, specifically the project management thereof, executed with a pattern of small iterations. This is a brief re-hash of Extreme Programming, based on my limited experience with such practices to-date.

A colleague asked how I'd do it, so here's the general idea.

Outline

• A project is defined as something that has many small iterations of start-to-finish, instead of one big start and one big finish.
• Depending on the size of the project, iterations can be hours long (hackathon), or they can be months long (multi-MM $ projects).
• For the sake of parsimony, we'll assume that it's preferable for iterations to be regular (i.e. they are all of the same length). You may experiment with irregular iterations at your leisure.

Specifications

In software, the conventional divide is between the two following levels of abstraction.

• Functional Specification
  
  This is the higher level of abstraction. The project targets are described conceptually. More detail is better, and ideally there should be no loose ends. These could be implemented as follows.
  1. brainstorming: random thoughts
  2. analysis: remove contradictions
  3. analysis: map out major dependencies; necessary and/or sufficient conditions for success
  4. analysis: map out every single dependency
• User Interface (UI) Specification
  
  This is the lower level of abstraction. The entire surface of the software-to-be-developed is specified in space and time - if any part of the software is perceptible to the user, this is the time to decide on how exactly that will happen. This could be implemented as follows.
  1. doodling: rough sketches
  2. analysis: remove contradictions
  3. analysis: map out dependencies; necessary and/or sufficient conditions for success
  4. final art: major states of affairs
  5. final art: every possible state of affairs

The steps in blue are ideally completed at this stage, but are often procrastinated to step (Iterating 2.) in the schema below. Following the feasibility study, the depth of which is an art to be mastered, there should be enough information with which to divide the project's wish list into iterations.

Before Iterating

Prior to iterating, moderately detailed quantitative estimate should be conducted. This is often referred to as a feasibility study. It involves one or more quick and dirty surveys of the project's specifications.

Iterating

Each iteration could look like this, for example:

1. A review of the entire project's specifications at both levels of abstraction, progress (if any) in previous iterations, and a revisiting of whether each outstanding item should go into the current iteration, a future iteration in this project, or into a "future project"  (when a project has been broken into discrete groups of iterations, each group may be referred to as a "phase").
   
2. Final specification of work to be developed in this iteration (from the final blue steps of the feasibility study, otherwise you now have to execute the blue steps; if you ignore the blue steps completely, you risk feature creep and meandering timelines; in web development, for example, ideally you've already got CSS and HTML).
3. Development.
   
   • Automated testing - only if things pass, they move to the next step. [CI]
   • User-acceptance testing (UAT) - only if things pass, they move to the next step. [CI]
   • Product-owner signs off only on specifications that are considered complete.

Continuous Integration

If you practice continuous integration (CI), that happens throughout stage (Iterating 3.). If you don't, then you must allot time for it elsewhere, and therein lies more uncertainty, and therein lies the reason that CI is a preferred practice.

Continuous Deployment

If you practice continuous deployment (CD), that happens in each iteration after stage (Iterating 3.). If you don't, then you must allot time for it elsewhere, and therein lies further risk, and therein lies the reason that CD is a preferred practice. This pattern is possible only when the use-cases of the software are tolerant of failure, and when the cost of deployment is trivial - you don't want to try this if you're planning the first mission to Mars, for example.