At the turn of the century, while many in the industry lost sleep over the millennium bug, a wave of new approaches to team-based software development appeared over the horizon. The first to break on the beach of mainstream awareness was Kent Beck's very cool sounding eXtreme Programming or XP for short (no relation to the similarly named version of Microsoft Windows).
In a Nutshell
As described in the first edition of Kent Beck's book, Extreme Programming Explained, published in 1999, eXtreme Programming is originally based on twelve best practices, four core values, and an options-based economic model.
The premise behind eXtreme Programming is that the cost of making changes to software does not need to grow exponentially more expensive throughout a project or product development life-cycle. That the cost of change does increase exponentially over-time is attributed to unnecessary complexity added early in the project product life-cycle; complexity that provides the flexibility to address all the changes that might be asked for in the future.
The assumption in eXtreme Programming is that few of the potential changes catered for early in the life of the project are ever requested. In contrast changes that were not predicted are requested and the unneeded complexity designed to cater for changes not asked for impedes the ability to cater for the unforeseen changes that are asked for. The argument is that time invested adding flexibility early in the project frequently does not realize its expected return on that investment. Therefore, it is better postpone that investment, do the simplest thing that works now, and defer the decisions to add complexity/flexibility until it is proven that they are actually needed.
For this reason, the core values and best practices of eXtreme Programming are designed to keep the cost of change relatively constant throughout a project. If this is achieved then instead of trying to minimize the amount of change later in a project, late changes are accommodated and even welcomed. If the cost of change is reasonable, then a software development team can 'embrace change'.
The four original values of eXtreme Programming are:
- Communication - the
eXtreme Programming practices are designed to
keep communication flowing within the development team and with
- Simplicity - eXtreme
Programming bets that doing the simplest
thing that works today and paying tomorrow if a change is needed, is
cheaper than doing a more complicated thing today that adds flexibility
that may never be needed.
- Feedback - concrete
feedback based on production-quality, working
code enables better communication.
- Courage - the courage to make changes both supports and is supported by the other three values.
The second edition of Extreme Programming Explained adds a fifth value, respect; respect for other members and roles within the project team and respect for all interested parties.
The twelve original best practices of eXtreme programming are:
- The Planning Game - The business and development work
together to produce a list of estimated, prioritized user stories for
development to work on. The business is responsible for scope,
priority, and composition and date of releases. Development is
responsible for estimates, communicating consequences of technical
choices, process, and scheduling of work within releases. The Planning
Game is repeated after each iteration and release. A very short book, Planning
Extreme Programming, by Martin Fowler and Kent Beck
in 2000 explains this crucial aspect of eXtreme Programming in a little more practical detail.
- Small Releases - The aim of the Planning Game is to
produce an initial overall plan comprising of small releases each
delivering the currently outstanding requirements of most value to the
- Metaphor - the metaphor
is eXtreme Programming's replacement for an initial, high-level
architecture, and probably the vaguest, most misunderstood and
under-used practice. It is an overall guiding statement for the
project. In Domain-Driven Design and Feature-Driven
Development, a domain model performs the equivalent role.
- Simple Design - select
the simplest design that works. Don't design for tomorrow if tomorrow's
needs are not fully known.
- Testing - provide
comprehensive automated unit and functional tests for every feature.
The idea is to increase developer and stakeholder confidence in the
software making refactoring easier and enabling iterative development.
- Refactoring - refactoring is changing the internal
design and implementation of features without changing
the externally visible behaviour of that code. eXtreme Progrmaming
advocates frequent refactoring to simplify design and code to make it
easier to add new features.
- Pair Programming -
all production code in eXtreme Programming is written by two people
working together at one workstation. This is intended to provide a form
of continuous design and code peer review catching many errors in
design and programming logic as they are made.
- Collective Ownership -
any programming pair may edit any of the code base of the project at
any time they need to. This practice is designed to avoid delays caused
by requesting, explaining and waiting for someone to make a needed
modification to 'their' code.
- Continuous Integration -
every few hours, the whole system is built and all the automated tests
run against it to prove that recent changes have not broken anything or
clashed with anyone else's changes. This practice has matured
considerably since eXtreme Programming first arrived on the scene.
Proponents of the practice have produced a number of popular tools and
books devoted to the topic, including Paul Duvall's, Continuous
Integration and more recently Jez Humble has
written about extending the practice to encompass continuous
- 40-hour week - to sustain productivity on an
iterative project requires team members work at a sustainable pace. eXtreme Programming mandates a 40 hour maximum
working week. If a team really must work longer one week to achieve a project goal, then the subsequent week should
include slack time to enable the team to recover to full potential.
- On-site customer -
communication is best done face-to-face and immediately the need arises. Someone permanently sitting with the
development team able to answer questions about requirement details eliminates a large amount of communication
overhead and delays caused by communicating formally to
someone remotely located from the project team.
- Coding Standards - collective ownership and pair programming makes it
essential that code is written and formatted consistently to an agreed set of conventions.
In the second edition of , eXtreme Programming Explained, Kent Beck did some serious refactoring of the best practices. eXtreme Programming now comprises 13 primary practices with 11 corollary practices.
- Sit Together - everybody involve in the project should
be located in the same open space office area.
- Whole Team - otherwise known as cross-functional
teams; the project team needs to include people with all the skills necessary to complete the project.
- Informative Workspace - large, frequently-updated wall
posters, lists, pin-boards and charts are used to communicate status and progress among other items
- Energized Work - replaces the 40 hour working week
practice with something more comprehensive
- Pair Programming - unchanged
- Stories (user stories) -
the use of user stories as a tool for requirements elicitation,
elaboration, planning and progress reporting remains unchanged
- Weekly Cycle - work in
- Quarterly Cycle - the
equivalent of other agile advocates concept of release planning. A
regular, planned opportunity to step back and perfrom high-level
planning, look at improving the process, remove bottlenecks, etc.
- Ten Minute Build - mainitinaing the ability to build and
test the system under development within ten minutes so that develoeprs
can do this frequently.
- Continuous Integration -
- Test First Programming - write tests before writing production
- Incremental Design - same as simple design but with an
emphasis on doing a little design every day
- Real Customer Involvement
- Incremental Deployment
- Team Continuity
- Shrinking Teams
- Root Cause Analysis
- Shared Code
- Code And Tests
- Single Code Base
- Daily Deployment
- Negotiated Scope Contract
- Pay Per Use
Most of the strengths of eXtreme Programming are almost taken for granted within modern software development approaches now. This was not the case when eXtreme Programming, Scrum and Feature-Driven Development (FDD) where first formulated and developed in the late 1990's.
All of the self-styled agile processes including eXtreme Programming divide work into short development iterations or timeboxes of between a few days and four weeks. This has a number of benefits over processes comprising a sequence of lengthy phases:
- Rapid feedback from short iterations can radically reduce the distance between the introduction of errors and their detection.
- Short iterations enable the team to experience the feeling of finishing something on a frequent basis, improving job satisfaction and increasing team morale.
- Short iterations enable a team to get into a habit-forming rhythm.
- Short iterations provide an opportunity to improve the way the team works together.
Small, Cross-functional Teams
Similarly, all agile approaches emphasize the need for cross functional
teams. Like many of these practices, they are not new ideas.
Cross functional teams were proposed by Marlin Hills as surgical-style
teams lead by a Chief Programmer and promoted by
Fred Brooks in his book, The
, first published in 1975, as a means of maintaining the conceptual integrity of a design. eXtreme Programming must be
given some credit for pushing this idea once again.
Automated TestingeXtreme Programming's emphasis on automated testing coupled with the availability of the xUnit family of testing frameworks has without question transformed the unit testing landscape. xUnit-style automated unit testing is now almost taken for granted as a best practice by most development teams. The question is no longer whether automated unit tests should be written but on keeping the tests running quickly and evaluating whether or not tests are earning their keep by providing more value in terms of defect detection than the cost of writing and maintaining them.
Improvements in computing performance have enabled teams to quickly and cheaply build and regression test their entire system whenever they want to. Unfortunately the name for this practice is a poor one. The integration is not without interruption as the name implies; it is not continuous. Continual Integration would be a more accurate name.
The most controversial practice within eXtreme Programming is pair programming. I have worked on two projects that professed to be using eXtreme Programming. In both cases, the teams had dropped the use of pair programming by the time I joined the project; one team had dispensed with pair programming completely and the other used it only for user stories where they had not done something similar before.
Testing checks quality attributes that are visible to an end-user such as correctness, robustness, and performance. Testing does not check quality attributes that are visible only to developers such as code simplicity, loose coupling, high cohesion, appropriate encapsualtion, compliance with best practice coding conventions, good use of design patterns, etc. Dropping pair programming eliminates a key quality assurance mechanism for these internal quality factors.
Static analysis tools can check and catch an increasing number of these internal quality problems when run within developers' IDEs and as part of the continuous integration build process. However, they cannot catch the more important, more subjective items or identify logic problems. Some form of design and code review or inspection is needed to catch these. Pair programming, while almost certainly not as effective as well run traditional design and code inspections, is eXtreme Programmings peer review mechanism. Without it, the ability to catch important internal design and coding quality issues early is largely lost. And as Bertrand Meyer states in Object-Oriented Software Construction, "In the end, only external factors matter. ... But the key to achieving these external factors is the internal ones."
Where pair programming is practiced properly, it is not a like-for-like replacement for more structured, well run, design and code reviews. Design and code reviews give a chance to step back and pause before the review, they add additional eyes and brains for a short, sharp focused period of time.
Ironically, frequently having two pairs of eyes looking at something that only requires one, could be argued is a form of waste that Lean Software Development requires be removed to increase productivity.
The metaphor practice is vague; probably deliberately so. The domain model-oriented approaches such as Domain-Driven Design and Feature-Driven Development are stronger in this area especially when it comes to enterprise software systems for complex business areas such as global commercial financing and insurance.
eXteme Programming is designed for small co-located teams. Scaling to larger teams and projects or projects involving of multiple development teams puts practices such as collective ownership under unsustainable pressure.Feature-Driven Development's feature teams provides an alternative way to structure larger efforts.
Other weaknesses of eXtreme programming are not so much in what it promotes as best practice but in what it enables developers to easily get away with in terms of poor practice. For example, it is very easy for developers to substitute 'simplest design that works' for 'first design that springs to mind'. In addition, it is too easy for developers to avoid delivering necessary supporting documentation using the excuse that the source code including test code is all the documentation needed. This is simply not true.
In revolutions aimed at countering extremely dysfunctional behaviour, the tendency is to take the counter proposal to the opposite extreme. This is true of eXtreme Programming and makes its name somewhat ironic. For many teams, eXtreme programming is simply too extreme to be a practical solution over the medium to long term. The doubling of the number of practices to over twenty has not helped either because it makes the whole thing much more unwieldy to learn and remember.
Today the popularity of eXtreme Programming is in decline, being
largely superseded by Ken Schwaber
and Jeff Sutherland's Scrum approach
in terms of mainstream popularity. However, it still heavily influences
many of the design and coding practices that Scrum teams use.
Andy Carmichael, Better Software Faster, 2002
Scrum splits a project into a number of fixed length time boxes or iterations called
sprints. At the beginning of each sprint the project team meets to select a subset of items from a priortised to-do
list known as the product backlog. The subset of items chosen from the product backlog are the items that provide the
most business value for the customer and which the team believe they can complete by the end of the sprint. This
selection process is known as Sprint Planning.
FDD combines key advantages of other agile approaches with model-centric techniques and
fluid team organisation. This means that FDD easily scales to much larger teams and projects than generally recommended
for other agile approaches. FDD is also characterised by an emphasis on quality throughout the process, and timely,
accurate, meaningful status reporting and progress tracking for all levels of leadership.
Domain-Driven Design comes as a bit of a surprise to those who have been taught
that agile software development and especially eXtreme Programming do away with the need for analysis and design
techniques like object modeling and notations like the Unified Modeling
Language (UML). It is actually very obvious that any software team working in a .Net language, Java, Objective-C,
C++ or any of the object-oriented dynamic languages such as Python or Ruby has an object model at the heart of their