Best Practices for Regression Tests

Updated 8 months ago by Estela García

Maintenance of Regression Testing can be a daunting exercise, but if we pay attention at how our tests are organized, selected and executed, our tester’s life will be easier and our software product more reliable.

Create Short Tests

When you run tests frequently and throughout the software development cycle, consider having small tests to save time and resources. Many tests are longer than we think, so break such tests into smaller fractions such as UI testing, function testing, security testing, UX testing and so on.

Identify your critical features and create short and easy tests to rapidly be aware of any issue there. Complicated tests tend to fail for different reasons and make regression tests maintenance complex and hard. The most recommendable tests to have packaged within your regression tests collection are the UI tests, allowing you to have the certainty that the basic functionalities are running as expected (CRUD tests are a good option), buttons and links present, etc.

Also avoid running your tests on many browsers, it will decrease your tests speed radically.

Define Your Naming Conventions

A proper naming convention for your regression tests will let you have them clearly identified and organized. Here we suggest an example of a simple naming convention for your Test Cases, Test Suites and Test Groups, but you can adapt it to your own needs.

Test Cases

  • If the Test Case you are defining is part of the Regression Tests pack, add “RT_” prefix to its name to better identify them and make searches easier. You can use same criteria for other type of tests (AC for acceptance tests, UT for unit tests, etc.).
  • All records created by the test cases should start with "RT_{!RUN_ID} <Your record>" prefix, for example for your accounts, opportunities, leads, user stories, projects, etc…

RT_{!RUN_ID} My account

RT_{!RUN_ID} My lead

RT_{!RUN_ID} My User Story

RT_{!RUN_ID} My Project

Be aware that it is recommendable to add some unique text at the end of each item name to make it different from other items that might be created for you or another user in another Test Case but part of the same execution (test suite, test group…).

Test Suites

  • If a Test Suite is part of the Regression Tests, add “RT_” prefix to its name to better identify it and make searches easier.
  • Add a Status at the end of the name to indicate the status of the tests included there. Some samples:
    • Work In Progress (WIP): currently working on this Suite and the tests included in it.
    • Previous Release or NextRelease  (PR / NR): to make sure this test is run against a particular version. That way you will be able to deprecate it or add it to the regression tests when your new release is ready.
    • etc.

These texts will be replaced by a Status field in future CST versions.

Test Runs

  • When we use “Manage Test Runs manually” checkbox functionality within a Test Group, we should be able to identify which Test Runs are part of a Regression Tests Group, so all the Test Runs included in that Group may have “RT_” prefix added to its name.

Test Groups

  • If the Test Group is part of the Regression Tests, add “RT_” prefix to its name to better identify and search for regression test groups.

You may want to create Test Groups containing tests sharing a same criteria (context, common functionality, application, etc).

Remove your Regression Tests Records

Once the testing process is completed, all records created during it must be cleaned. A cleaning record task of testing data is a highly beneficial practice.

  • Create Apex instructions before execution to delete "RT_*"

Examples:

delete [SELECT Id FROM copado__Project__c WHERE Name LIKE 'RT_%'];

delete [SELECT Id FROM copado__User_Story__c WHERE copado__User_Story_Title__c LIKE 'RT_%'];

Be very careful when using these commands because if this type of record is used within another test (since several tests are being executed at the same time) you might be removing records from another test case. Add more definition in the removing pattern, E.G. ‘RT_userstory_for_promotions%’.

  • Create Apex instructions after execution to delete "RT_{!RUN_ID}*"

Examples:

delete [SELECT Id FROM copado__Project__c WHERE Name LIKE 'RT_{!RUN_ID}%'];

delete [SELECT Id FROM copado__User_Story__c WHERE copado__User_Story_Title__c LIKE 'RT_{!RUN_ID}%'];

Be very careful when using these commands to avoid deleting other user’s records.

Automate Your Regression Tests

Where possible, regression tests must be automated. Running the same tests over and over again, with the same variables and conditions would not yield any new defects and will become a repetitive work causing loss of interest.

Another advantage of automating the regression tests is that more test cases can be added to the regression packs without much impact on the time. An automated regression pack can be run overnight or in parallel with the manual tests and would free up the resource.

Copado Selenium Testing lets you create Scheduled Jobs and forget about your Regression Tests executions. You can also create a workflow rule which will detect any failing test run and will send you an email.

Run Your Tests on Different Orgs

During the development stage, designers and developers make constant changes mostly in a collaborative manner so, testing the application within the development environment is not advisable. Since CST allows you to create different Selenium Settings and run your Selenium Tests on different environments, you can record your tests on one Org and run them, for example, on other 2 Orgs, one with your latest stable version and another with your in-progress version (master branch).

Copado Selenium Testing Test Groups were created with this particular objective in mind: being able to run same Test Suites on different Orgs.


How did we do?