Paul Hammant's Blog: BDD Acceptance Criteria Pay For Themselves Multiple Times
For two previous blog entries on Story writing in advance of development, here and here, Jeffrey Davidson was my collaborator and editor before publication. Jeffrey also wrote a companion piece to my smaller stories argument here. He used to talk about these ideas at conferences with presentations like this one. Note, I have written before on BDD acceptance criteria for the well-known “Todo MVC” technology demonstrator app. And Jeffrey’s mirror of this article is here
In this article Jeffrey and I are extending our conversation about Stories. We both firmly believe in using Behaviour Driven Development (BDD) as the model for acceptance criteria and we’re taking the previous “candidate stories” examples as a starting point. This sort of stuff isn’t our primary area of work anymore, but it is fun and crucial to smooth flow of business ideas into production. And from that, we believe, gives a high return on investment, regardless of whether you intend to use the same BDD acceptance criteria for test automation.
Rationale for our belief
Bottlenecks can appear in any software development production line at any time. And as you solve them, a new bottleneck will appear somewhere else in the production line.
One common bottleneck is with stories written weeks or months before developers start implementing them. Such stories can be viewed as vague and underdeveloped (under-specified?) by the time developers and test engineers take a look at them in order to estimate them. Alternatively, the elaboration of the story is missing key concepts needed to meet the business needs.
Either of those issues lead to defects during development and waste later. Even when stories are well-phrased, misunderstandings may result in longer development time and unintended defects.
Classically, the defects we focus on are those in production code that reach production. Defects can also happen in test code too. Sure, test code doesn’t reach production as such, but work being marked as “QA approved” when that wasn’t correct, is a real second-order defect. This happens when people misunderstand what they are testing. The same thing can happen for manual testing as well as test automation, of course.
Behavior Driven Development (BDD) acceptance criteria
Dan North’s BDD encourages writing ‘scenarios’ using the words “Given, When, and Then” but otherwise uses plain English. Scenarios are perfect acceptance criteria. Writing them in the story definition in the story tracker (Jira, Rally, etc.) is expedient.
For the initiate, “Given” focuses on the system’s existing condition; the state before the user roles performs a specific action. “When” describes the action the user takes. “Then” describes the results the users sees after the system responds. In all cases, this should be from the user’s perspective and written in business language. It should not be overly technical, include a host of design elements, or written from the system’s perspective.
Note: For the rest of this article we are adopting the term “gherkins” instead of BDD acceptance criteria. This term was popularized by fans of Cucumber, the most popular BDD framework.
Worked Examples
The “candidate stories” article had the two stories from the Insurance domain. In that article they were referred by number. Both #11 and #12 and are worth expanding to have acceptance criteria in a gherkin style for this article. Note below that we’re using yellow highlighting to hint at variance for lines we have used more than once.
Story #11, “pro-rata refunds on cancellations”:
Narrative (as before)
In order to meet regulations concerning termination of policies,
As an underwriter, I want to be able to cancel policies, effective
mid term, and offer a pro-rata refund of premium.
Acceptance Criteria (new)
Given the insured has an in-force policy
When they cancel the policy mid-term
Then the policy status is changed to canceled
And a pro-rata return of premium is returned to them
Given the insured has an in-force policy with half the annual premium unpaid
When they cancel the policy ¾ of the way through the term
Then the policy status is changed to canceled
And the debt owed is reduced by the amount of the pro-rata return of premium
And no premium is returned to them
Given the insured has an in-force policy with half the annual premium unpaid
When they cancel the policy ¼ of the way through the term
Then the policy status is changed to canceled
And a pro-rata return of premium less the debt owed is returned to them
Given an Insured with an expired policy
When they cancel the policy mid-term
Then they are informed the policy cannot be canceled after the natural expiry date of the policy
Story 12, “pro-rata claw-back on reinstatement”:
Agile’s INVEST mnemonic suggests we should snip stories up where we can. A reinstatement story is clearly something that comes after a cancellation story, and there is an order to that. Otherwise, there is a perfect INVEST separation to the two. You could, as a budding insurance company, have gone live without either and be busy adding them as people signed as ‘new business’. Strategies to cancel policies might include “do it on paper” if the cost for that isn’t going to hockey-stick too soon. Anyway, reinstatements come after cancellations:
Narrative (as before)
In order to correct erroneously cancelled policies,
As an underwriter, I want to be able to reinstate policies effective
from their cancellation date provided it is not in the past, and
claw-back refunded premiums on reinstatement
Acceptance Criteria (new)
Given the insured has a canceled policy with an expiry date in the future
When they reinstate the policy from the date of cancellation
Then the policy status is changed to active
And they are charged the previously returned premium
Given the insured has a canceled policy with an expiry date in the past
When they reinstate the policy from the date of cancellation
Then they should be informed reinstatements must be completed before the natural expiry date of the policy
A common BDD acceptance criteria pitfall
Referring to “Login” in the Given lines is a mistake many teams new to this make. Much of the BDD community now agrees: please try as hard as you can to not begin gherkins with “Given I am logged in.” Unless your scenario is about logging into the system, you can safely assume this has already happened.
When test engineers are automating the gherkin acceptance criteria, then tests should attempt to explore the functionality of the UI in a ‘microcosm’ stack that doesn’t require an actual login to test the focus area of the story. Your Given lines should assume the UI testing technology has a way of getting straight to the rectangle of the UI that is the subject of the test. I.e. bypassing a bunch of interstitial pages, including login.
The need for a glossary and style guide
A glossary and style guide owned by the Product Owner (PO), or Business Analysts (BAs) are “must haves.” The glossary is all the terms the business and users would use to describe the system. A style guide defines look-and-feel elements and key user interface functionality. Using a style guide allows gherkin statements to be simpler and avoids unnecessary wordiness.
A primary goal of gherkin statements is understandability; anyone with the business domain knowledge should be able to understand the steps and goals. Adding a glossary and style guide increase the team’s understanding of both the domain and the system. These documents also give guidance and will help the author write better acceptance criteria, improving them before review by the larger team. If you have a glossary and style guide it is important verify that new stories adhere to them before they are up estimation, coding, and test automation.
In our insurance examples here, multiple terms would have an entry in the glossary: claw-back, pro-rata, cancellation, reinstatement, premium.
Two styles of gherkin acceptance criteria
With projects ranging across many years and multiple industries, we have found the above gherkin style to be a very good way for introducing BDD to teams. While some story writers find this to be a bit repetitive, this is an easy technique for getting started and clearly communicating system needs (up to its limits).
Let us go further. There is no better or faster way to get a new business analyst up to speed, or a mediocre analyst up to ‘good’, than BDD.
The following style of gherkin has some significant advantages over the above examples. It provides the information needed for test automation – it has variables and reduces some of the repetition found above. This approach uses data to describe the variations that would otherwise be repetitive.
Here’s an example from Cucumber’s Scenario Outlines page:
Given there are <start> cucumbers
When I eat <eat> cucumbers
Then I should have <left> cucumbers
Examples:
| start | eat | left |
| 12 | 5 | 7 |
| 20 | 5 | 15 |
Our experience has shown it is easier for new teams to start with the simpler, language-only, non-data BDD. While it is possible to jump straight to the latter method, we find this stepping stone leads to better understanding of how to use this technique and increases adoption.
Starting with the former gives the entire team the basis for building their business domain understanding. It also gives testers and test engineers additional time to develop the data required for scenarios; data needed for functional testing.
We recommend teams adept with BDD shift to the second form earlier in the story elaboration cycle. That is, instead of developing the data during or after development is complete, the data scenarios can supplement or replace the text only versions prior to development.
Having scenario data prior to development is an outstanding way of ensuring developers have a clear idea of the system expectations. We have witnessed this greater understanding leading to faster development, better code, and fewer defects.