After 20 years of working with test automation, I like to think I've become quite good at it. Recently, I was advocating for a change in approach at a company I had just joined. I knew it was crucial to clearly explain my method for writing maintainable and intent-revealing test automation.
Many people know they should use a pattern to avoid repeating themselves in their tests (like using page objects). But what are we really trying to achieve with a page object? Is it just about following the DRY principle?
In Gojko Adzic's talk at NDC London in 2016, one of the five key patterns for successful test automation he referenced was this:
💡
Divide expectations, workflows, interactions
Where we might consider workflows activities such as (as in his examples):
Authorise()
AddItems()
Checkout()
And interactions examples such as clicking a link or setting value in a field.
$('a[role=...').click()
$('input[name=....]').setValue('...')
We can summarize this as taking a what vs how approach to writing tests. Our tests should mainly refer to what we want to happen (e.g., checkout), while the technical steps required to complete it (the how) are abstracted away.
Benefits
Tests that focus on "how" tend to be fragile because they are closely tied to the specific details of the application. Any change in the UI or workflow can require extensive updates to the tests themselves. By focusing on the "what," your tests become more resilient to changes. They are based on high-level behaviors and outcomes, which are less likely to be affected by minor UI changes or workflow adjustments.
Not all your workflow need be UI driven
Whenever we specify workflows at the "what" level, it becomes easier to identify steps that can be achieved through other mechanisms. For example, our Authorize step could be accomplished using a combination of API calls and cookie settings, bypassing the need to perform the same steps via multiple pages.
Improved Test Development
When tests are designed around the "what," the complexity of test scripts is reduced. This minimizes the risk of falling into the copy/paste trap, where people might copy a series of "how" actions to reach a certain stage in the application without truly understanding the business behavior.
By removing specific interactions from the tests, test developers are encouraged to think about the business flows instead of just making it work through copying and pasting.
The right level of "what"
A "what" focused approach allows for greater flexibility in adapting to changes and scaling the test suite. However, it still requires care, attention, and a focus on true business behaviors, while removing incidental details.
Consider the example below. It might seem like a suitable level of abstraction since the interactions are not in the tests (no "how" level detail), but it doesn't go far enough.
DetailsPage.selectSize(‘XL’);
DetailsPage.selectQuantity(1);
DetailsPage.selectColor(“white”);
DetailsPage.selectReoccuringPurchase(false);
DetailsPage.addToCart();
In his wonderful blog post Blake Norrish discusses the literary idea of Checkov's Gun:
💡
“Chekhov’s Gun” is a fancy literary term coined by Russian playwright and author Anton Chekhov, who stated that everything mentioned in a story must, in some way, come into play later in the story, and everything that does not should be removed.
Taking that idea we might come up with something like the following, where we've removed incidental detail and focused on the core workflow.
DetailsPage.addValidItem();
And of course we may not stop there, as the same principal can apply across multiple pages and not just one.
Imagine we've done the above and we have removed incidental detail so our interactions look might look like this:
DetailsPage.addValidItem()
ShippingPage.enterShippingAddress()
BillingPage.enterBillingAddress()
PaymentPage.enterPaymentInformation()
PaymentPage.reviewOrder()
PaymentPage.placeOrder()
Note in this example, again we don't have any details of "how" in the tests (the technical interactions), but we haven't gone far enough, as we would be potentially repeating these workflows in many tests.
This could be summarised as the following where we've removed some "what" details about replaced it with a higher level facade, which communicates what we need to do.
DetailsPage.addValidItem()
Payment.placeOrder()
Of course, a lot of these code example are a gross simplification, but should give you the idea of how you can think about achieving the right level of "what".
Conclusion
By focusing on the desired outcomes and behaviors of our applications, through this focus on "what vs how" we've been able to create more resilient, maintainable and business focused tests to support the company in it's journey to deliver better software quality and ultimately to deliver greater value to our customers.
