Following the popularity of my last post regarding a Test Automation framework for Appium (Java) centered testing, I have had many requests to create one that is more centred around Selenium & C#.  

 As with my previous blog, the purpose of this post is to provide an evolving best practice guidelines for structuring & writing automation frameworks, using Behaviour Driven Development (through cucumber) and good code structure (through Page Object Model pattern):

HIGH-LEVEL STRUCTURE:
Below is a proposed high-level structure for test automation projects. Each blue box represents a collection of classes or feature files:

1. Feature files are made up of User Stories & Acceptance Criteria used to drive the tests. They are written in gherkin syntax (Given, When, Then).
2. Step definitions allow the gherkin syntax to drive the lower level test automation code.
3. Object Buckets are used to store/reuse data across test steps within the same scenario (e.g. instances of page objects created), using Dependency Injection included with SpecFlow.
4. Test Data Objects are logical groupings of test data (e.g. a test user class with an email address and password property).
5. Step definitions communicate with Page Objects, which are representations of the pages/menus within the Website being tested.
6. Driver Facade class provides a wrapper for Selenium WebDriver allowing additional logging and any other interaction specific code required.

​
Feature files:
 
These represent the User Stories/Acceptance Criteria (similar to business requirements) and are written in ‘business speak’ using Gherkin syntax. 

Best Practice:

• A separate feature file should be created for each feature/functional area within the app.
• ‘Given’ sentences can be written using past tense, ‘Whens’ using present tense and ‘Thens’ using future tense (though some prefer present tense for all (e.g. Given/When reuse becomes easier)
• The ‘When’ section of a scenario will commonly be only one row long (making the purpose of test easily identifiable)
• Given/When sections will rarely appear after a Then statement.
• Scenario titles should not be too long. If more detail is needed add description row(s) underneath the title.
• Use ‘Given I have navigated to XXXX page’ to get test pre-condition to start on a certain page. Both these statements can be used in the scenario Background.

​Step definitions:
 
Step definitions are the interface between the ‘business speak’ gherkin and the technically coded Page Objects.  They are made up of methods which are called when the higher level Cucumber scenarios are run.

Best Practice:
 

• Within the constructor, create/reuse any Object Bucket instances (e.g. Pages class) that are needed within that specific Step Definition file.  This can be done using SpecFlow’s dependency injection system.
• Use instances of page objects and test data objects within the step definition file to drive your tests. 
• Each feature file could have its own step definition file
• Step definition files should contain your asserts.

 
Code Reuse:
 

• Write your code for a step definition.  If that code is required by other step definitions within the same file, refactor that code into a private method (see example below). If different step definition files also require the same code, refactor it into a ‘CommonSteps’ step definition file.

​Common Step Definitions file:
 
Create a Common Step Definitions file which can be used for step definitions used across different Feature files. 
This file can also be used to construct the DriverFacade class and create the initial page object (e.g. for the HomePage).  It can also quit the DriverFacade class at the end of tests. This means DriverFacade only needs to be exposed on this one Step Definition file, remaining hidden & passed between the Page Objects on all other Step Definition files (reducing the likelihood of its misuse).
Here is an example implementation (where we have refactored some of the previous steps into this commons steps file):

​Object Buckets:
 
These are used to persist data across Step Definitions as a Cucumber scenario runs.

Best Practice:

• Buckets will simply be ‘plain old’ classes with properties, allowing data to be stored only for the duration of a single test scenario. For example, we recommend storing page object instances between scenarios (so that the DriverFacade can be hidden from the Step definition layer).
• Instances of these buckets will be destroyed automatically by Cucumber at the end of each test scenario.

 
 
Test Data Objects:
 
These objects are used to store logical groupings of test data information, for example a Test User with an email and password property.  They allow your test suite to pass around representations of real world objects as single entities, rather than as lots of individual variables (which can get confusing & cumbersome to maintain).

​Best Practice:

• Use these ‘plain old’ classes with properties to represent different Test Data that will be used by your tests. For example, a TestUser class might have a default username & password property set in its constructor. You could then pass an instance of testUser into a relevant method: loginPage.loginWithValidUser(testUser);
• Updates to default values should be made in the Step definition files so the purpose of the test is obvious, e.g:

TestUser user1 = new TestUser();
user1.password = “invalid_password”;
loginPage.loginWithValidUser(user1);

• Alternatively, you can pass enum values (e.g. TestUserType.Invalid) into the Test Data constructor to achieve the same outcome.
• Use getters & setters for test data objects (so that controls can be added if needed at a later stage).

 
 
Page Objects:
 
A separate class is created for each page or section (e.g. navigation menu) of the Website. The methods on these classes will drive the Web pages (e.g. Actions like: loginWithAValidUser()) or return information about the state of the website (e.g. Queries like: getUsernameDisplayed()).

Best Practice:

• Pass the single instance of DriverFacade into all page objects’ constructor method (see screenshot above).
• Methods can be functional (e.g. loginWithValidUser) or more specific (e.g. enterEmailAddress). Often functional methods will call specific methods (meaning you maximise the testing potential of the page object and avoid code duplication).
• Any navigation methods (e.g. GotoLoginPage) should return an instance of the subsequent page object (e.g. LoginPage).  This will allow test writers to navigate through the test suites they are running more easily (since resulting pathway flows will be apparent from the code design). It can also help identify failing tests when, for example, a page object no longer links to a certain page.  Additionally it will mean the FacadeDriver is not exposed to the Step Definition pages (making its misuse less likely).
• Common navigation menus should form separate page object classes. Any page object (e.g. HomePage) that has the navigation menu should then have the navigation menu page object as a property of itself (using composition rather than inheritance).
• Page Objects should not make assertions, except optionally in the constructor to check the page is displayed (which can help make sure the correct page is correctly loaded before proceeding).
• If a single flow in the application can lead to different pages being presented, then create a separate method for each journey. E.g: loginWithValidUser (returns welcome page) vs loginWithInvalidUser (returns error page).
• When using NavigationControllers, a single page’s back button might return you to different pages. When this happens you can create multiple methods e.g.  'Back to homepage', 'Back to Account Page' that call the same private method which actually selects the ‘back’ button.
• For websites, if name and id are not available as properties of objects, use CSS if possible (faster & more consistent than xpath). For Appium xpath is used instead of CSS.
• Page Objects should not use getters & setters, since the page objects are encapsulating the UI and so will get and set any values using the UI (not internal class properties).

 
DriverFacade:
 
This is simply a wrapper class between our test automation code and the third party test automation framework (e.g. Selenium WebDriver).  

Best Practice:

• The page objects should interact with FacadeDriver rather than directly with WebDriver.  This will allow far more control over test runs (e.g. logging), as well as providing the ability to switch in and out of different automation tools if required.
• The DriverFacade should be the only class in the test automation framework that interacts directly with Selenium / Appium.
• Set the implicit wait to be a low value (e.g. 5 or 10 seconds), and then wait for elements with a longer time out when required. This will help ensure tests are faster to run.

 
Folder Structure:
 
Below is an example of a folder structure you could use for implementing the aforementioned process:

Future additions to this process:
 

• Adding selenium’s Page Factory.
• Process for writing Gherkin.  Perhaps agree scenario titles & relative priorities as a group across Dev, Test & Business (to ensure nothing is missed), but then write individual steps as a tester (to ensure correct wording & ordering of steps is achieved).

Having done both Development & Test Automation for over 10 years now, I realised the focus of this blog has been primarily on development.  Therefore, I thought it would be interesting to cover some Test Automation.  

Gherkin (the human readable coding language used by Cucumber) is becoming increasingly popular as more companies embrace Behaviour Driven Development (BDD).  The Page Object Model is an equally important tool in the test automator's arsenal and so I was keen to show how these two structures can work together to provide a well structured, modular & easily updatable test automation framework.  Please do leave any comments below, since I plan to update this framework approach as suggestions come in.

In summary, the purpose of this post is to provide an evolving best practice guidelines for structuring & writing automation frameworks, using Behaviour Driven Development (through cucumber) and good code structure (through Page Object Model pattern).

HIGH-LEVEL STRUCTURE:

1. Feature files are requirements used to drive the tests. They are written in gherkin syntax (given, when, then).
2. Step definitions allow the gherkin syntax to drive the lower level test automation code.
3. Object Buckets are used to store/reuse data across test steps within the same scenario (e.g. instances of page objects created).
4. Test Data Objects are logical groupings of test data (e.g. a test user class with an email address and password property).
5. Step definitions communicate with Page Objects, which are representations of the pages/menus within the Application Under Test (AUT).
6. App Driver class provides a wrapper for Selenium WebDriver allowing additional logging and any other interaction specific code required.

FEATURE FILES:
These represent the business requirements and are written in ‘business speak’ using Gherkin syntax

​Best Practice:

• A separate feature file should be created for each feature/functional area within the app.
• ‘Given’ sentences should be written using past tense, ‘Whens’ using present tense and ‘Thens’ using future tense
• The ‘When’ section of a scenario will commonly be only one row long (making the purpose of test easily identifiable)
• Scenario titles should not be too long. If more detail is needed add description row(s) underneath the title.
• Use a ‘Given I was on the launch page’ to setup new instance of AppDriver and to place user on first screen of app. Use ‘Given I have navigated to XXXX page’ to get test pre-condition to start on a certain page. Both these statements can be used in the scenario Background.

​
STEP DEFINITIONS:
 
Step definitions are the interface between the ‘business speak’ gherkin and the technically coded Page Objects.  They are made up of methods which are called when the higher level Cucumber scenarios are run.

Best Practice:
 

• Within the constructor, create/reuse any Object Bucket instances that are needed within that specific Step Definition file.  This can be done using Cucumber’s dependency injection.
• Use instances of page objects and test data objects within the step definition file to drive your tests. 
• The step definition files should contain your asserts.
• You may choose to have 1 step definition class that is responsible for the creation/closure of the app driver class (if using web automation). This is the only step definition class that will contain a private instance of the web driver, since page objects will pass that instance between themselves.

 
Code Reuse:
 

• Write your code for a step definition.  If that code is required by other step definitions within the same file, refactor that code into a private method. If different step definition files also require the same code, refactor it into a ‘CommonSteps’ step definition file. Here is an example:

​OBJECT BUCKETS:
 
These are used to persist data across Gherkin test steps as a Cucumber scenario runs. The data will be wiped at the end of each scenario.  Rather than creating new instances of pages manually at the start of each step definition method, this way means you don't have to expose the Driver instance on every Step definition page (meaning lazy use of e.g. driver.FindElementBy().Click() at the step definition level is not possible)

Best Practice:

• Buckets will simply be ‘plain old’ classes with properties, allowing data to be stored only for the duration of a single test scenario. For example, a UserBucket might be used to gather user data during the ‘given’ and ‘when’ sections of a test run and then test that data as part of the ‘then’ section asserts (though usually these asserts will be carried out using data direct from the UI).
• Instances of these buckets will be destroyed automatically by Cucumber at the end of each test scenario.
• Most projects will have a PageBucket class to hold new page object instances as they are created during a test.

 
TEST DATA OBJECTS:
 
These objects are used to store logical groupings of test data information, for example a Test User with an email and password property.  They allow your test suite to pass around representations of real world objects as single entities, rather than as lots of individual variables (which can get confusing & cumbersome to maintain).

Best Practice:

• Use these ‘plain old’ classes with properties to represent different Test Data that will be used by your tests. For example, a TestUser class might have a default username & password property set in its constructor. You could then pass an instance of testUser into a relevant method: loginPage.loginWithValidUser(testUser);
• Updates to default values should be made in the Step definition files so the purpose of the test is obvious, e.g:

TestUser user1 = new TestUser();
user1.password = “invalid_password”;
loginPage.loginWithValidUser(user1);

• User getters & setters for test data objects (so that controls can be added if needed at a later stage).

 
 

 
PAGE OBJECTS:
 
A separate class is created for each page or navigation menu of the Application Under Test (AUT). The methods on these classes will drive the AUT pages (e.g. Actions like: loginWithAValidUser) or return information about the state of the app (e.g.Queries like: getUsernameDisplayed).

Best Practice:

• Pass the single instance of AppDriver into all page objects’ constructor method (see screenshot above)
• Keep methods functional (e.g. loginWithValidUser) rather than too specific (e.g. enterEmailAddress). This will make future refactoring far quicker as the app evolves over time.
• Any navigation methods (e.g. GotoLoginPage) should return an instance of the subsequent page object (e.g. LoginPage).  This will allow test writers to navigate through the test suites they are running more easily (since resulting pathway flows will be apparent from the code design). It can also help identify failing tests when, for example, a page object no longer links to a certain page.
• Common navigation menus should form separate page object classes. Any page object (e.g. HomePage) that has the navigation menu should then have the navigation menu page object as a property of itself (using composition rather than inheritance).
• Page Objects should not make assertions, except in the constructor to check the page is displayed. This can help make sure the correct page is correctly loaded before proceeding.
• If a single flow in the application can lead to different pages being presented, then create a separate method for each journey. E.g: loginWithValidUser (returns welcome page) vs loginWithInvalidUser (returns error page).
• When using NavigationControllers, a single page’s back button might return you to different pages. When this happens create multiple methods e.g.  'Back to homepage', 'Back to Account Page'.
• For websites, if name and id are not available as properties of objects, use CSS if possible (faster & more consistent than xpath). For iOS apps xpath is fine instead of CSS.
• Page Objects should not use getters & setters, since the page objects are encapsulating the UI and so will get and set any values using the UI (not internal class properties).

​APPDRIVER:
 
This is simply a wrapper class between our test automation code and the third party test automation framework (e.g. Selenium WebDriver).  

Best Practice:

• The page objects should interact with AppDriver rather than directly with WebDriver.  This will allow far more control over test runs (e.g. logging), as well as providing the ability to switch in and out of different automation tools if required.
• Environment variables (e.g. iOS version) should ideally not be hard coded within the constructor (contrary to the screenshot above), but instead set up as part of test run configuration.
• The AppDriver should be the only class in the test automation framework that interacts with directly with Calabash / Appium / Selenium.
• Set the implicit wait to be a low value (e.g. 5 or 10 seconds), and then wait for elements with a longer time out when required. This will help ensure tests are faster to run.

FOLDER STRUCTURE:
 
Below is an example of a folder structure you could use for implementing the aforementioned process:

DataObjects:
This folder can contain Test Data Objects and any other object representations you may need for your test suite.
 
Facades:
A façade class is one that provides a simplified gateway to more complex classes.  AppDriver is one example of a façade but others can be stored in this folder.
 
Utilities:
The Utilities folder will contain classes that use static methods to provide time saving actions, such as logging.
 
The remaining folder titles, in the screenshot above, have been covered earlier in this document.



ADDITIONS YOU COULD MAKE:
 

• Adding Selenium’s Page Factory.
• Agree a process for writing Gherkin.  Perhaps agree scenario titles & relative priorities as a group across Dev, Test & Business (to ensure nothing is missed), but then write individual steps as a tester (to ensure correct wording & ordering of steps is achieved).

If you are new to coding you might be wondering about best techniques when designing classes.  Here are some thoughts on the matter, deemed from a highly talented Developer I work with.  The example uses a simple calculator class that takes 2 numbers & performs a calculation on them (e.g. add, subract


RULE 1: Pass values in via a constructor if they are mandatory AND will not generally change
Calculator calc = new Calculator(number1, number2, transactionType);
double result = calc.GetResult();

The trouble with the code above is you would want to pass in different numbers and calculation types.  So in effect your class goes from being a flexible ‘calculator’ class to an inflexible ‘singleCalculation’ class.

RULE 2: Don’t allow users to misuse your class or interface
Calculator calc = new Calculator();
calc.number1 = 5;
calc.number2 = 7;
calc.transactionType = TransactionType.Multiply
double result = calc.GetResult();                  

The trouble with the code above is that if the user of your code forgets to set calc.transactionType (which could be easily done since it doesn’t get passed into the constructor or the GetResult() method) then the class will error

RULE 3: Save memory

Calculator calc = new Calculator();
double result1 = calc.GetResult(5, 7, TransactionType.Multiply);
double result2 = calc.GetResult(result1, 7, TransactionType.Multiply);

Passing the parameters into GetResults mean we pass both Rules 1 and 2 above and also save on memory since we only have to construct one instance of calculator class.