Best Practices for Regression Tests

Updated 4 months ago by Copado Solutions

Maintenance of Regression Testing can be a daunting exercise, but if we pay attention to how our tests are organized, selected and executed, our tester’s life will be easier and our software product more reliable.

Creating Short Tests

When you run tests frequently and throughout the software development cycle, consider having small tests to save time and resources. Many tests are longer than we think, so you could break those tests into smaller fractions such as UI testing, function testing, security testing, UX testing and so on.

Identify your critical features and create short and easy tests to rapidly be aware of any issue there. Complicated tests tend to fail for different reasons, making regression tests maintenance complex and hard. The most recommendable tests to have packaged within your regression tests collection are the UI tests, which allow you to have the certainty that the basic functionalities are running as expected (CRUD tests are a good option), buttons and links are present, etc.

You should also avoid running your tests on many browsers, as doing so will decrease the speed of your tests radically.

Defining Your Naming Conventions

A proper naming convention for your regression tests will let you have them clearly identified and organized. Here we suggest an example of a simple naming convention for your Test Cases, Test Suites and Test Groups, but you can adapt it to your own needs.

Test Cases
  • If the Test Case you are defining is part of the Regression Tests pack, add an "RT_" prefix to its name in order to better identify the test case and make searches easier. You can use the same criteria for other types of tests (AC for acceptance tests, UT for unit tests, etc.).
  • All records created by test cases should start with an "RT_{!RUN_ID} <Your record>" prefix, for example for your accounts, opportunities, leads, user stories, projects, etc.

RT_{!RUN_ID} My account

RT_{!RUN_ID} My lead

RT_{!RUN_ID} My User Story

RT_{!RUN_ID} My Project

It is advisable to add some unique text at the end of each item name to differentiate it from any other items that might have been created for you or another user in a different Test Case but are part of the same execution (test suite, test group…).

Test Suites
  • If a Test Suite is part of the Regression Tests, add an "RT_" prefix to its name to better identify it and make searches easier.
  • Add a status at the end of the name to indicate the status of the tests included there. See some samples below:
    • Work In Progress (WIP): Currently working on this suite and the tests included in it.
    • Previous Release or Next Release  (PR / NR): To make sure this test is run in a particular version. That way you will be able to deprecate it or add it to the regression tests when your new release is ready.

These texts will be replaced by a Status field in future CST versions.

Test Runs
  • When you use the Manage Test Runs manually checkbox functionality within a test group, you should be able to identify which test runs are part of a Regression Tests Group, so all the test runs included in that group may have an “RT_” prefix added to their names.
Test Groups
  • If the test group is part of the Regression Tests, add an “RT_” prefix to its name to better identify and search for regression test groups.
You may want to create test groups that contain tests sharing the same criteria (context, common functionality, application, etc).

Removing Your Regression Tests Records

Once the testing process is completed, all records created during this process must be cleaned. A cleaning record task of testing data is a highly beneficial practice.

  • Create Apex instructions before execution to delete "RT_*"

Examples:

delete [SELECT Id FROM copado__Project__c WHERE Name LIKE 'RT_%'];

delete [SELECT Id FROM copado__User_Story__c WHERE copado__User_Story_Title__c LIKE 'RT_%'];

Be very careful when using these commands because if this type of record is used within another test (since several tests are being executed at the same time) you might be removing records from another test case. Add more definition in the removing pattern, e.g. ‘RT_userstory_for_promotions%’.
  • Create Apex instructions after execution to delete "RT_{!RUN_ID}*"

Examples:

delete [SELECT Id FROM copado__Project__c WHERE Name LIKE 'RT_{!RUN_ID}%'];

delete [SELECT Id FROM copado__User_Story__c WHERE copado__User_Story_Title__c LIKE 'RT_{!RUN_ID}%'];

Be very careful when using these commands to avoid deleting other users' records.

Automating Your Regression Tests

Where possible, regression tests must be automated. Running the same tests over and over again, with the same variables and conditions would not yield any new defects and will become a repetitive work causing loss of interest.

Another advantage of automating regression tests is that more test cases can be added to the regression packs without much impact on time. An automated regression pack can be run overnight or in parallel with the manual tests and would free up the resource.

Copado Selenium Testing lets you create scheduled jobs and forget about your regression tests executions. You can also create a workflow rule which will detect any failing test run and will send you an email.

Some manual tasks executed post deployment can be automated via Selenium. You can record these as test cases and add them into a test run. Each test run can be executed via a webhook. This webhook can be added to the Promotion deployment using the URL Callout deployment step. Simply select the Webhook URL lookup button to find the test run that should be executed.

Running Your Tests in Different Orgs

During the development stage, designers and developers are constantly making changes, mostly in a collaborative manner, so testing the application within the development environment is not advisable. Since CST allows you to create different Selenium settings and run your Selenium tests in different environments, you can record your tests in one org and run them, for example, in two other orgs, one with your latest stable version and another one with your in-progress version (master branch).

Copado Selenium Testing's test groups were created with this particular objective in mind: to be able to run the same test suites in different orgs.


How did we do?