TestRail Guide - Overdose Tech Club

Introduction

TestRail is a web-based tool that allows managing the test cases. The primary purpose of TestRail is to provide the team leads and developers the opportunity to organize, manage, and track the testing work. The easy-to-use interface and web-based nature of TestRail imply that anyone can enter test cases, organize test suites, execute test runs, and track their results. Therefore, it is considered one of the most effective software for test management. This guide will explain the basic principles of how to create and run tests with TestRail.

Which account should I use for work?

To start your work with TestRail, you should log in with your credentials and use the Admin account. Companies usually use one account for all projects to gather all important information and make it available for all QAs.

1. Open the next corporate link to login into the TestRail

2. Click the ‘Email’ field

3. Enter email and password or choose the account from the password manager

4. Log In

Projects and how to start your project

Projects are the main organizational unit in TestRail. It is generally recommended to create a TestRail project for every real software project you want to manage within TestRail. All other data such as test runs, test results, milestones, etc. are directly associated with a specific project.

To start a new project, follow the next steps:

1. Go to the Dashboard page

2. Click the ‘+Add Project’ button at the top of the right column

3. Add Project name

4. Add links to the Project in the ‘Announcement’ input (for ex. Blueprint link, designs, production, and staging links.

5. Select ‘Show the announcement on the overview page’ checkbox

6. Choose the first type of project, that recommended

7. Click the ‘Add Project’ button

To see more information about the project types, go to this link and watch the video.

To see the project overview page click the next icon in the list

TestRail Dashboard

After logging in to TestRail, the first page you usually see is the dashboard. The dashboard provides you with an overview of available projects, recent project activities, and your todos.

When you navigate to a project (by clicking on a project title), you switch to the project view and land on the project’s overview page, showing project details such as test cases, active test runs, project activity, and so on. Whenever you need to switch to another project, just return to the dashboard by clicking the link in the upper left corner.

OVERVIEW’ TAB

After clicking on the project title, you open the ‘Overview’ tab of your project. You can find there the next useful information:

1. Your test activities in the Test Runs (the amount of the passed/failed/blocked/retest cases). You can save the chart information for a report as a .csv file or an image.

2. Milestones and Test Runs

3. Project Activity (that includes history and changes)

‘MILESTONES’ TAB

In case you have separated milestones on your project, you can also add them to the TestRail. This step is not required, but it could be done for your convenience. These Milestones can be assigned to the Test Runs then.

For example, there were 6 milestones on a project and they were added to the TestRail:

To add a new Milestone follow the next steps:

1. Click the ‘Milestones’ tab

2. Click the ‘Add Milestone button

3. Add ‘Name’, like just ‘Milestone1’, ‘Version 1.0’, ‘Internal Beta’ or another convenient name according to the project in the Wrike

4. You can add helpful description information or start and end dates

5. Click the ‘Add Milestone button

Once you have added your milestones to TestRail, you can assign test runs to specific milestones.

Assigning test runs to milestones has the advantage that you can easily track the milestones’ test results and progress separately.

You can use this tab or not, it’s up to you. Go to the ‘Test cases’ tab then, this one is required.

‘TEST CASES’ TAB

A test case in TestRail consists of a description of the test’s preconditions, a list of test steps and the expected result.

Depending on the functionality of the project, companies write test cases or checklists. The most logical thing would be to organize the checklists into groups that will correspond to the tasks of the project in Wrike(or other project management tool) or Test Plan.

You should use checklists if you describe a standard functionality that all team members are familiar with (for example, login, search, header/footer, help cms pages, etc.)

And you should specify the test cases if you are trying to describe a non-default Magento feature, for example, a new extension that is rarely used.

In TestRail, test cases are organized into sections and sub-sections. Sections and sub-sections are a collection of test cases or checklists and are often created for specific project modules or areas of your product. How you organize your sections depends on the size of your project. If you have a lot of test cases for a project module, it is recommended to further break down your sections and create sub-sections for specific features or functionality.

Creation of the project tree and checklist

To create Project tree, follow the next steps:

1. Go to the ‘Test cases’ tab

2. Click ‘Add section’ for creating tree subcategories according to the Test Plan

3. Add Name (Ticket name according to the Test Plan category) and Description (Links to the ticket and designs) in the opened window.

Click ‘Add Case’ under section name for adding checklist item.

5. Add a Title for your checklist with the [Checklist] prefix

So after these steps, you will have the next project displayed.

Create all folders according to the test plan and add checklists to each folder.

To see the project tree fully, click ‘Compact view’ in the right sidebar with the project tree.

Test cases creation

In case you need to specify preconditions, steps to reproduce, expected result or other important information, you should create a Test Case. To do this go to the ‘Test cases’ tab and choose the section for your test case. Follow the next steps then:

1. Click ‘Add Case’ to the right of sort and filter functionality for creating a detailed test case.

2. Add Title for you test case with [Test Case] prefix

3. Save it

4. Edit the test case you just added

On the test case editing page, you can change the title, type of testing, priority, type of automation, as well as prescribe steps to reproduce, preconditions, expected result.

There are two ways to add steps to reproduce, choose the template that suits you best:

– Test Case (Steps) – you can add the expected result to each step

– Test Case (Text) – you can add just list of the steps as text separately from the preconditions and expected results

TestRail’s toolbar

One page element you can find on most pages in TestRail’s user interface is the toolbar. The toolbar provides useful functionality and options for the current page. For example, importing and exporting test cases is done through the toolbar on a test case page. Other useful things you can do from the toolbar include printing reports or jumping to related pages.

Moving and copying test cases

Moving and copying test cases (or entire sections) are often useful to duplicate or rearrange your test suites and test cases.

In case when you have one main project and 1+ more clones of this project with the same pages and functionalities but different styles, you can use this feature which will make your work much easier. So you can just copy checklists and even sections and then edit checklists if there will be a need to do this.

To do this, follow the next steps:

Open the new project without test cases and checklists
Click ‘Test cases’ tab
Click ‘Copy or move cases’ button on the toolbar
Choose Source project from the list (the project that already has checklist)
Choose Section, Subsection or even individual cases that you want to copy
Choose ‘Move/Copy cases only’ in the dropdown of the bottom of the window (or another variant according to your request)
Choose ‘Appends To’ place from the list (the folder of the new project where you want to save the cases you selected before in the modal window)
Click ‘Copy’ button

Exporting test cases and sections to CSV or Excel file according to PM’s request before the UAT testing

If you have received a request from the PM to prepare a checklist file for the client before UAT testing, you can use the export function.

To do this, follow the next steps:

1. Click the “Export cases” button on the toolbar

2. Choose the type of the file (for example CSV)

3. Choose ‘Export all sections’

4. Select the next columns:

ID
Title
Created by
Priority
Section
Section Description
Suit
Suit ID

5. Click the ‘Export’ button

Also, you can save this file in the .pdf format so all your detailed test cases with steps will be saved.

To do this, click the print icon and choose ‘Details’ in the dropdown.

‘TEST RUNS & RESULTS’ TAB

To run a test and enter test results for the cases you added, you start a test run for a particular test suite. While a test suite is just like a plan that specifies how an application is tested, a test run is an actual test you are conducting.

After you finish writing test cases and adding milestones, you can go to configure Test Run. To do new Test Run follow the next steps:

1. Go to the ‘Test Runs & Results’ tab

2. Click ‘Add Test Run’ button

3. Add Test Run name

4. Choose Milestone if you added it before in the ‘Milestone’ tab

5. Select ‘Select specific test cases’ variant

6. Click ‘Change selection’

7. Select test suit for your Test Run

8. Click OK button

So you will see that the qty of the test cases was included in this test run.

Once all Test Runs have been added, they will be listed on a tab with a progress bar.

It is very convenient to create not just one common Test Run with all blocks, but several that will correspond to the tasks of the Test Plan and tasks in Wrike (or other project management tool). So you can quickly evaluate the progress on the project, see which blocks have problems and what tasks haven’t been completed yet.

You can delete the Test Run from this tab and open the specific test run for editing or analyzing.

A run consists of individual tests for each case that you add. By default, each test has one of five different statuses that are signaled in TestRail by specific colors. You can add a test result and change the status of a test either by clicking the Add Result button on the test page, or by changing it directly on the run page.

The following test statuses are available by default:

Untested: By default, new tests have the status Untested. Once a test result has been added to a test, it can never receive the Untested status again.
Passed: A test is marked as Passed when a tester verified the test steps and the expected results.
Failed: A tester marks a test as Failed if one of the specified test steps resulted in an error or if the expected result differs from the actual test result.
Retest: If a developer or tester determines that a test should be tested again, she marks the test as Retest. E.g., if a test originally failed and the developer fixed the problem, the developer can mark it to be retested.
Blocked: The Blocked status is used to signal that a test cannot be executed currently because of some external dependency (such as a bug that needs to be fixed before being able to complete the test). It is often used in combination with the Retest status.

After choosing one of the statuses, you should add a comment with a screenshot or video that describes and confirms the result.

Once a test run has been completed, you can close it from the run’s edit page. Tests of a closed run cannot be edited or changed, making it ideal for archiving test runs. Additionally, if you change any of the associated cases’ attributes (such as the expected result), the change won’t be applied to the closed and archived run.

But also, you can leave all test runs completed with 100% results like on the screenshot.

Adding multiple test results

You can easily add test results for multiple tests by using the mass-action buttons on the test run page. Simply select multiple test results and use the Add Test Result button at the bottom of the page: