In The Seven Habits of Highly Effective People, Stephen R. Covey names "Begin with the End in Mind" as the second of the seven habits. This habit applies not just to individuals, but to software development teams as well. In Acceptance Test Driven Development (ATDD), the Product Owner begins requirements discussions with expectations and examples, and the whole team collaborates to distill these into acceptance tests that define the essence of “Done." Modern testing frameworks enable the team to express the tests in natural language while connecting them to the software so that the tests are automated while the software is being developed. The end result is that the acceptance tests become executable requirements. These slides explain the ATDD cycle and how it fits with other Agile development and testing practices including TDD, Continuous Integration, and Exploratory Testing.

1. Beginning with the End
in Mind: Driving
Development with
Acceptance Tests
Elisabeth Hendrickson
Quality Tree Software, Inc.
www.qualitytree.com
esh@qualitytree.com
Last updated November 10, 2009
This work is licensed under the Creative Commons Attribution 3.0
United States License. View a copy of this license.

2. What is ATDD?

3. Acceptance-Test Driven Development (ATDD) Cycle
0&1(/.2
3,.4#15
-21&6
-21&6
-21&6
-21&6
-21&6
Copyright © 2010 Quality Tree Software, Inc.
0&1(/.2
!$-./-- '" (
*
+ #" ,
) &" "
*
!"#$%"& !"%"#17
(Model developed with Pekka Klärck, Bas Vodde, and Craig Larman.)

4. ATDD: Discuss
!"#$%&'
()&*+#, !"#$%&
Copyright © 2010 Quality Tree Software, Inc.
'(&%$)*#(&+(,
-'#".
-'#".
-'#".
-'#". Concrete
Examples with
-'#". Expectations
(Slide developed in collaboration with Pekka Klärck.)

5. ATDD: Develop
'*&+%","-."/,/
Copyright © 2010 Quality Tree Software, Inc.
'()*+"&
%#$&
!"# ,-(*
.&/*/
!"#$
&
' (" )
$ %" "
%#$&
&
(Slide developed in collaboration with Pekka Klärck.)

6. ATDD: Deliver
Copyright © 2010 Quality Tree Software, Inc.
Product
Feature Feature
Feature Feature
Feature
Feature
FeatureFeature
Feature
(Slide developed in collaboration with Pekka Klärck.)

7. Introducing an Example

8. Start with the Story
As an administrator, I want users
creating accounts to be required
Copyright © 2010 Quality Tree Software, Inc.
to choose secure passwords so
that their accounts cannot be
hacked by someone using a
password guessing program.

9. Discuss
And if a user provides Who’s in the
an insecure password, room?
display an error
Product
message.
Owner,
Testers,
Copyright © 2010 Quality Tree Software, Inc.
Developers,
and anyone
else who will
touch the
What does story.
“secure” mean
to you?
At least 6 characters with
at least one letter, one
symbol, and one number.

10. Capture Concrete Expectations and Examples
Password Valid? Can be
expressed as
“p@ssw0rd” Yes “Given - When -
“p@s5” No Then”
“passw0rd” No
Copyright © 2010 Quality Tree Software, Inc.
Given a user is creating an account
“p@ssword” No
When they specify an insecure
“@#$%1234” No password
Or can be Then they see a message,
expressed in tables “Passwords must be at least 6
characters long with at least one
Or in other formats letter, one number, and one
depending on the symbol.”
Framework

11. Why ATDD?
Reason #1:
Drive Out Ambiguity
and Clarify
Expectations

12. This is not an Argument about a Bug
“Bug Triage Meeting”
The Tuesday before release.
It’s a bug.
Copyright © 2010 Quality Tree Software, Inc.
No it’s not.
Whether or not
it’s a bug, if we
Is too. make a change
we’ll blow the
IS NOT. schedule.
IS TOO!
NOT NOT NOT!

13. Acceptance Tests Define Scope
Copyright © 2010 Quality Tree Software, Inc.

14. A Short Digression on
ATDD-Friendly Tools

15. Examples of ATDD-Friendly Frameworks
• Cucumber: a Ruby-based BDD tool that
supports “Given-When-Then”
• Fitnesse: a table-driven framework that
uses a wiki for displaying and editing
Copyright © 2010 Quality Tree Software, Inc.
tests
• Robot Framework: keyword-driven
framework that supports text or tables
• Concordion: Java-based framework for
expressing expectations in prose

16. Frameworks, Interfaces, and Drivers
Created in Code written during
collaboration. development in a GUI Driver
Format defined programming language (e.g. SeleniumRC)
by the determined by the
framework framework.
Copyright © 2010 Quality Tree Software, Inc.
GUI
Public API
Test
Natural “Guts”
“Fixture”
Language
Expectations Other
interfaces
Implementation

17. Characteristics of ATDD-Friendly Frameworks
• Support expressing expectations in a language
and format that fits the context
• Support collaboration among the whole team
including developers, testers, & the product owner
Copyright © 2010 Quality Tree Software, Inc.
• Connect expectations to the system under test
with a minimum of test code (“fixtures,” “libraries,”
“steps”) to leverage expectations as executable
requirements
• Play nicely with source control systems and
continuous integration
• Pluggable to support a variety of interfaces

18. Contrasting View: Traditional Test Automation
GUI Automated Test
Scripts:
Public API Combination of
“Guts” business-facing
Copyright © 2010 Quality Tree Software, Inc.
expectations and
Other implementation
interfaces details.
Implementation
Written or recorded
after the fact.
Expectations are
translated, not
leveraged.

19. Back to the Example…

20. Take the Acceptance Tests…
Password Valid?
“p@ssw0rd” Yes
“p@s5” No
“passw0rd” No
Copyright © 2010 Quality Tree Software, Inc.
Given a user is creating an account
“p@ssword” No
When they specify an insecure
“@#$%1234” No password
(Note that these Then they see a message,
expectations are “Passwords must be at least 6
implementation-agnostic characters long with at least one
and express just the letter, one number, and one
essence of the symbol.”
expectation.)

21. …and Write the Code
Password Valid? Test GUI
“p@ssw0rd” Yes “Fixture”
“p@s5” No Public API
Copyright © 2010 Quality Tree Software, Inc.
“passw0rd” No
“Guts”
“p@ssword” No
“@#$%1234” No Other
interfaces
Implementation

22. Why ATDD?
Reason #2:
Make progress visible.

23. Are We There Yet?
Copyright © 2010 Quality Tree Software, Inc.

24. Why ATDD?
Reason #3:
Leverage, Efficiency,
and Executable
Specifications

25. Traditional Approaches
Requirements
Management
System
Copyright © 2010 Quality Tree Software, Inc.
Traceability
Matrix
Test
Technical Management
Specifications ? System

26. Efficiency, Reusability, Maintainability
Implementation
The tests New interface?
change? Make a
define the Just add a test
localized update to
requirements. fixture
the test fixture.
Copyright © 2010 Quality Tree Software, Inc.
Natural Test
“Fixture” …
Language
Expectations
New Interface
Implementation
(No reconciling (Preserve valid (Leverage relevant
multiple, duplicate expectations.) expectations.)
artifacts.)

27. How does ATDD fit with
the rest of the process?

28. ATDD: Part of an Agile Testing Strategy
Support Critique
Business-
Copyright © 2010 Quality Tree Software, Inc.
facing Acceptance Tests Exploratory Testing
Code- Reviews, inspections,
facing Unit Tests pairing, code
quality metrics
(A variation on Brian Marick’s Agile Testing Quadrants as
published in his essay “Agile Testing Directions”)

29. Tests are Versioned with the Code
Source Control
Natural
Language
Expectations
Copyright © 2010 Quality Tree Software, Inc.
Test
“Fixture”
Code
Unit Tests
Production
Code

30. Tests Execute as Part of the Automated Build
Continuous Integration
Copyright © 2010 Quality Tree Software, Inc.
Images courtesy Mike Clark, www.pragmaticautomation.com.
Used with permission.

31. Why ATDD?
Reason #4:
No more bugs. (No, I’m
not kidding. But yes,
there is a catch.)

32. Zero Tolerance for Bugs
Copyright © 2010 Quality Tree Software, Inc.
!"#$%"%&'%( !"#$%"%&'%(
)'*+, )'*+,

33. But Not Everything is a Bug
In this context,
a BUG
is behavior that
violates the letter
Copyright © 2010 Quality Tree Software, Inc.
or spirit of the
Product Owner’s
expectations for the
implemented story.
If the behavior does not violate expectations
related to the implemented stories, it’s an item
for the backlog.

34. Given all that…
Why not ATDD?

35. Resources
Adzic, Gojko (2009). Bridging the Communication Gap:
Specification by Example and Agile Acceptance Testing.
Neuri Limited.
Crispin, Lisa (2005). “Using Customer Tests to Drive
Development.” Methods & Tools. Summer 2005 Issue.
Copyright © 2010 Quality Tree Software, Inc.
Available online at http://www.methodsandtools.com/
archive/archive.php?id=23
Crispin, L., & Gregory, J. (2009). Agile Testing: A Practical
Guide for Testers and Agile Teams (Addison-Wesley
Signature Series (Cohn)). New York: Addison-wesley
Professional.
Marick, Brian (2003). “Agile Testing Directions.” (An
explanation of business-facing v. code- facing tests.)
Available online at http://www.exampler.com/old-blog/
2003/08/22/#agile- testing-project-2