Business-Driven E2E Tests
| Table of contents |
|---|
- 2026-01-18 Text is published.
- 2023-10-14 Idea.
Introduction
End-to-End tests (E2E) simulate real user behavior by interacting with the application through its user interface. The idea is to test from the end user's point of view, for example if the application is a website, by scrolling pages, clicking links and mimicking the user behaviour as authentically as possible, to ensure the whole stack works.
Unlike unit or functional tests, E2E tests validate the system as a whole.
This article focuses specifically on how to design E2E tests so that they maximize business coverage while minimizing execution cost.
Problem
The idea of E2E testing is brilliant in theory but in practice there are some challenges: to create a testing environment where everything works as it does in production and to create as effective tests as possible because using the user interface, clicking links, waiting for the page to load (etc.) is a very slow process, so if there are lots of E2E tests and they are run every time new code is deployed, it can quickly become a deployment bottleneck.
In this article we focus on the latter problem: how to create effective E2E tests, from the business point of view. The number of permutations is close to infinite if the application is somewhat complex so testing all the different combinations is not an option.
Solution 1: Critical paths
One approach is focusing on critical user paths which means testing the most obvious usage scenarios, for example in an e-commerce store, adding a product to the cart and going through the checkout is obviously a path that should always work.
Critical paths approach is a good first step but if a higher coverage is desired, a fishing net approach is more effective.
Solution 2: Fishing net
A fishing net approach refers to the idea where the application, for example an e-commerce store, is divided into logical components from the business point of view, such as different kinds of customers, products and actions that must work at minimum. In other words the fishing net approach models the application as a combination of business-relevant dimensions – customers, products and actions – and tests effective combinations instead of exhaustive permutations.
An analogy of the fishing net approach to the real world fishing net is to catch the biggest fish first, at least the most valuable ones, and then make the net denser to catch also the medium size fish, etc.
Modern E2E tools such as Playwright (reference) make this approach practical by allowing tests to simulate different devices, browsers and users so it's possible to create customers such as a basic customer who uses the latest Chrome browser on mobile viewport, accepts cookies and uses default choices all-the-way. The second customer can be opposite to the first one, not accepting cookies, logs in only when absolutely necessary and using a less common browser. The third customer can be an international business customer using VAT-0 and English language setting, and so on. It's a good idea at first to list all relevant user settings there are and then combine them intelligently so that the most important settings are tested at first.
Different kind of products can be created in the same way. First list all the different properties of the possible products, then prioritize them and combine.
The customers and products that are created as presented above, can become imaginary customers and products that don't exist in real world, being artificial so to speak, but that's not a problem. The idea is not to mimic real users or real products but to systematically validate business-relevant properties, as effectively as possible. The same goes with actions: for example a normal user wouldn't seek the same product in many different places in a row but an effective E2E test can do that, to test all the different actions, at least tier-1 actions, that relate to product search, in one test case.
Below is a simplified example of how customers, products and actions can be defined:
## Customers CUSTOMER-1 * Chrome * Mobile viewport * Accepts cookies * Logged in * Default settings throughout CUSTOMER-2 * Safari * Desktop viewport * Rejects cookies * Logs in only when necessary * [SETTINGS-1] CUSTOMER-3 * Firefox * Tablet viewport * [SETTINGS-2] ## Products PRODUCT-1 * Physical * Available * Multi-category PRODUCT-2 * Electronic * Not available * Campaign PRODUCT-3 * Clearance * Heavy * Discount ## Actions TIER-1-ACTIONS * Go to the front page * Go to category page * Go to product page [PRODUCT-1] * Add a product to the cart from the main call-to-action button [PRODUCT-1] * Go to brand page * Add a product to the cart from the category page [PRODUCT-2] * Search ... etc. TIER-2-ACTIONS * Add a product to the cart using secondary ways. * Other secondary actions. ## Test cases TEST-1 = CUSTOMER-1 executes TIER-1-ACTIONS TEST-2 = CUSTOMER-2 executes TIER-1-ACTIONS TEST-3 = CUSTOMER-1 executes TIER-2-ACTIONS TEST-4 = CUSTOMER-2 executes TIER-2-ACTIONS TEST-5 = CUSTOMER-3 executes TIER-1-ACTIONS TEST-6 = CUSTOMER-3 executes TIER-2-ACTIONS
The key advantage of the fishing net approach is that when the plan is made, the developer can write the tests without needing to think what should be tested and what should not, and what is already tested in some other test case.
When the customers, products and test cases are created and documented, it's important to keep the list in sync with the program code where the actual tests cases are. If some test can't be created for a technical reason the code should still have a placeholder for the test telling the reason why it's missing.
Discussion
The fishing net approach offers several advantages. First, it's effective if the application is complex and testing coverage is wanted to get higher. Second, it's a straightforward approach for the developer because prioritization has already been done by someone else, a business representative. Also because the tests are defined from the business perspective, they often expose architectural or systemic weaknesses that would remain invisible in component-level testing.
End-to-End testing can be done manually as well, for example by giving every new employee a list of the actions to test – and an opportunity to learn the system at the same time – or emailing a list every day to a random person to test this and that.
In very demanding environments where the system must endure as high resilience as possible, chaos engineering and red teams can be used (reference) (reference).