What's New in TestRail 7.6 - TestRail

What’s New in TestRail 7.6

On-Demand Webinar


Q&A Follow-up from the ‘What’s New in TR 7.6” webinar on Wednesday, June 1, 2022

Test Data Parameterization

How do I get to the Test Data feature in TestRail?
From the Test Cases window, just click the “View Test Data” button.

On what levels can we add variables? Test case, Test run, Test Plan, Global?
Variable placeholders are written into Test Cases. Then, when creating a Test Run or a Test Plan, you select a dataset and the placeholders on the Test Cases will be replaced by the variables from that Dataset.

Can TestRail handle multiple test datasets (with different test suites / test cases)?
Right now TestRail only supports one table for the datasets feature per project. You can add test data from what would be multiple datasets on that same table in various columns.

Can a dataset be shared between separate test cases?
Yes, you should be able to use the variables from the dataset in all test cases. Datasets aren’t specific to individual test cases. You can have multiple test cases use the same variables.

What is the difference between Test Data and Configurations (“Configs”)? If configs are entered, will test data overwrite them?
Test data is used to dynamically insert data values from your dataset into tests when they are added as part of test runs or plans. Configurations are used when you need to run the exact same tests against different operating systems, browsers, etc.

You can use configurations and parameterized tests with datasets as part of the same test plan. When you use configurations to auto-generate test runs in a test plan, that doesn’t actually affect the test cases you select for those test runs themselves, so test data won’t overwrite the configurations.

In general, we recommend using configurations to indicate more detail about the system or browser where you are testing, whereas test data can be used to specify various data inputs or values you want to test on that system.

If configs were added to the run, will datasets be added to the same run separately? Is there a way to iterate over all the data sets similar to configurations?
Each Test Run can have one associated dataset, which means that you can add a dataset to a run as part of a test plan editing the test plan where the run was added. You can then add Configurations on top of it, which will multiply the Test Run according to the selected configurations.

The fastest way to create test runs for each dataset is to navigate to the View Test Data page from your Test Case repository, then click Create Test Runs. Select the datasets you’d like to generate test runs for, then click Create 1 Test Plan containing these X new Test Runs (where X will correspond to the number of datasets you want to generate test runs for).

Is it possible to use multiple datasets with the same test cases?
In different Test Runs, yes. You would simply add the same test cases to different test runs (because there is always only one dataset per test run).

How are datasets versioned? A test can pass or not based on the data set and if changed after the fact can impact execution history.
Datasets themselves are not currently versioned. A user could always create a new dataset rather than to update an existing dataset to prevent changes after execution.

Additionally, when you close a test run, that freezes the values from the datasets that were used to run the tests in that run and prevent the values from changing if the original dataset changes at some point in the future.

How do the results from parameterized test case runs flow into reports? How can I view reports for test runs where I am using parameterized tests?
Reporting should reflect run results normally. Since each test run corresponds to one dataset, you can compare the results for various datasets by simply comparing the results from each of those various test runs where you had tested different data sets. There is currently nothing in TestRail Reports that displays which dataset was used in a specific test run by default, although this is definitely something we will look into for the future. For now, as a workaround, you can manually add the dataset name to the name of the test run.

Are there API endpoints to retrieve the datasets?
It’s all there already! Please see the following documentation articles:

TestRail Jira Integration

Can I edit or add comments to a Jira issue from TestRail?
You can’t edit or add comments to a Jira issue from TestRail specifically. But with TR 7.6, assuming you have set up the defect / reference Jira plugin for your TestRail project, you will be able to edit any fields on your Jira issue that have been enabled in your integration settings. You can also automatically add information to the description field of a Jira issue when you push a defect from TestRail, including data from your test case, test run, and comments you added during testing.

Can we now link the Test Plan in TestRail to a Story in Jira?
Yes! Simply copy/paste the Jira Issue ID for the story (or any other type of Jira issue) in the references field on Test Plans (or Milestones) to link them.

How can I edit a linked Jira issue from TestRail?
To edit a Jira issue from TestRail, you must have set up the defect or reference plugin for Jira in your TestRail project. Then simply hover over the Jira issue ID that you’ve linked in a Defects or References field in TestRail to view the Jira fields for that issue, and click the “Edit” button on the bottom-left corner of the issue view.

For the Jira integration features in TestRail 7.6, it doesn’t need configuration, right? (i.e. it’s visible as default in v7.6 release)
That is correct, the ability to link Jira Issues to Milestones and Test Plans and edit linked Jira issues from the TestRail application as soon as you have updated to TestRail 7.6. You will also be able to view the linked Milestones and Test Plans from the TestRail Jira app as soon as you have updated to TestRail 7.6.

TestRail Release Tracks

When will this release be available for TR Server?
This release should be available for TestRail Server when TR 7.6 is released to the deferred release track, which we currently anticipate to take place in December 2022.

How do we know which track we are in currently?
TestRail users and administrators can easily view their release track and version number on their TestRail instance login page or by navigating to Help & Feedback>About TestRail from within their TestRail instance. Administrators may also check this in the Info section of the Administration area of their TestRail instance. For sake of example (numbers used are arbitrary), the following format will be used to identify the release track of your instance:

Early Access Release – 7.6.0 Early Access (1020)
Default Release – 7.5.2 Default (1002)
Deferred Release – 7.5.8 Deferred (2043)