What’s New in TestRail 7.6

On-Demand Webinar

 

Check out this recording of the live webinar “What’s New on TestRail 7.6”, where Product Marketing Manager Heather Vercillo presents the new features introduced in TestRail 7.6 with demonstrations by Adam Reue (Product Manager) and Simon Knight (Lead Product Manager).

Watch the full replay above, or read the highlights below.

The features introduced in this release are available for cloud customers on the Early Access release track only. Click here to switch to Early Access and receive the features introduced in TestRail 7.6. 

New Feature: Test Parameterization

Test Parameterization is the concept of running the same test case with multiple inputs so that you can verify multiple scenarios without having to duplicate the test case. It is a style of data-driven testing that allows you to test the same scenario multiple times while using different inputs for each test.

With TestRail Enterprise, you can now parameterize your tests by defining variables and datasets, inserting variables into test cases, and generating test runs based on existing datasets. A single test case can increase coverage because you can run multiple tests based on all the variables in a dataset.

Why Test Parameterization?
✅ Faster test case design
✅ Reduces chance of human error
✅ Leverage dynamic data to ensure accuracy
✅ Reduce time spent on maintenance
✅ Test case organization is easier

Test Parameterization is now available for Early Access Enterprise Cloud customers on TestRail 7.6 or later. 

Improved Jira Integration

Users on all TestRail plans now can update linked Jira issues without needing to switch between tools. When hovering over a linked Jira issue ID inside TestRail, you will now see an Edit button for making updates to the Jira entity. We’ve also added a References field to Test Plans, which will give testers the ability to link an entire Test Plan in TestRail to external requirements, user stories, or other entities. This field will be available for any integrated tool, including Jira.

The TestRail Integration for Jira add-on received a few updates for this release as well. To accompany support for linked Test Plans in TestRail, the add-on now has a Plans tab available for viewing Test Plan data. We also added a Milestones tab to the add-on, to support the available References field in TestRail’s Milestones. Finally, the TestRail Integration for Jira add-on will load inside Jira issues in a collapsed view, giving Jira users more screen space if they aren’t interested in linked TestRail data.

What does that mean?
✅ Stay in your TestRail workflow
✅ Increased visibility
✅ Increased traceability
✅ Increased performance in Jira

 

Q&A Follow-up from the ‘What’s New in TR 7.6” webinar

 

Test Data Parameterization

How do I get to the Test Data feature in TestRail?
From the Test Cases window, just click the “View Test Data” button.

On what levels can we add variables? Test case, Test run, Test Plan, Global?
Variable placeholders are written into Test Cases. Then, when creating a Test Run or a Test Plan, you select a dataset and the placeholders on the Test Cases will be replaced by the variables from that Dataset.

Can TestRail handle multiple test datasets (with different test suites / test cases)?
Right now TestRail only supports one table for the datasets feature per project. You can add test data from what would be multiple datasets on that same table in various columns.

Can a dataset be shared between separate test cases?
Yes, you should be able to use the variables from the dataset in all test cases. Datasets aren’t specific to individual test cases. You can have multiple test cases use the same variables.

What is the difference between Test Data and Configurations (“Configs”)? If configs are entered, will test data overwrite them?
Test data is used to dynamically insert data values from your dataset into tests when they are added as part of test runs or plans. Configurations are used when you need to run the exact same tests against different operating systems, browsers, etc.

You can use configurations and parameterized tests with datasets as part of the same test plan. When you use configurations to auto-generate test runs in a test plan, that doesn’t actually affect the test cases you select for those test runs themselves, so test data won’t overwrite the configurations.

In general, we recommend using configurations to indicate more detail about the system or browser where you are testing, whereas test data can be used to specify various data inputs or values you want to test on that system.

If configs were added to the run, will datasets be added to the same run separately? Is there a way to iterate over all the data sets similar to configurations?
Each Test Run can have one associated dataset, which means that you can add a dataset to a run as part of a test plan editing the test plan where the run was added. You can then add Configurations on top of it, which will multiply the Test Run according to the selected configurations.

The fastest way to create test runs for each dataset is to navigate to the View Test Data page from your Test Case repository, then click Create Test Runs. Select the datasets you’d like to generate test runs for, then click Create 1 Test Plan containing these X new Test Runs (where X will correspond to the number of datasets you want to generate test runs for).

Is it possible to use multiple datasets with the same test cases?
In different Test Runs, yes. You would simply add the same test cases to different test runs (because there is always only one dataset per test run).

How are datasets versioned? A test can pass or not based on the data set and if changed after the fact can impact execution history.
Datasets themselves are not currently versioned. A user could always create a new dataset rather than to update an existing dataset to prevent changes after execution.

Additionally, when you close a test run, that freezes the values from the datasets that were used to run the tests in that run and prevent the values from changing if the original dataset changes at some point in the future.

How do the results from parameterized test case runs flow into reports? How can I view reports for test runs where I am using parameterized tests?
Reporting should reflect run results normally. Since each test run corresponds to one dataset, you can compare the results for various datasets by simply comparing the results from each of those various test runs where you had tested different data sets. There is currently nothing in TestRail Reports that displays which dataset was used in a specific test run by default, although this is definitely something we will look into for the future. For now, as a workaround, you can manually add the dataset name to the name of the test run.

Are there API endpoints to retrieve the datasets?
It’s all there already! Please see the following documentation articles:
https://www.gurock.com/testrail/docs/api/reference/datasets/
https://www.gurock.com/testrail/docs/api/reference/variables/

TestRail Jira Integration

Can I edit or add comments to a Jira issue from TestRail?
You can’t edit or add comments to a Jira issue from TestRail specifically. But with TR 7.6, assuming you have set up the defect / reference Jira plugin for your TestRail project, you will be able to edit any fields on your Jira issue that have been enabled in your integration settings. You can also automatically add information to the description field of a Jira issue when you push a defect from TestRail, including data from your test case, test run, and comments you added during testing.

Can we now link the Test Plan in TestRail to a Story in Jira?
Yes! Simply copy/paste the Jira Issue ID for the story (or any other type of Jira issue) in the references field on Test Plans (or Milestones) to link them.

How can I edit a linked Jira issue from TestRail?
To edit a Jira issue from TestRail, you must have set up the defect or reference plugin for Jira in your TestRail project. Then simply hover over the Jira issue ID that you’ve linked in a Defects or References field in TestRail to view the Jira fields for that issue, and click the “Edit” button on the bottom-left corner of the issue view.

For the Jira integration features in TestRail 7.6, it doesn’t need configuration, right? (i.e. it’s visible as default in v7.6 release)
That is correct, the ability to link Jira Issues to Milestones and Test Plans and edit linked Jira issues from the TestRail application as soon as you have updated to TestRail 7.6. You will also be able to view the linked Milestones and Test Plans from the TestRail Jira app as soon as you have updated to TestRail 7.6.

TestRail Release Tracks

When will this release be available for TR Server?
This release should be available for TestRail Server when TR 7.6 is released to the deferred release track, which we currently anticipate to take place in December 2022.

How do we know which track we are in currently?
TestRail users and administrators can easily view their release track and version number on their TestRail instance login page or by navigating to Help & Feedback>About TestRail from within their TestRail instance. Administrators may also check this in the Info section of the Administration area of their TestRail instance. For sake of example (numbers used are arbitrary), the following format will be used to identify the release track of your instance:

Early Access Release – 7.6.0 Early Access (1020)
Default Release – 7.5.2 Default (1002)
Deferred Release – 7.5.8 Deferred (2043)