Tools: GitHub - TestRail

Integrate with GitHub

TestRail integrates with GitHub Issues so your team can link, view, add, and push issues between TestRail and GitHub as bugs/defects, requirements/references, or both. 

The TestRail integration with GitHub Issues supports all four methods of integrating TestRail with the issue, defect, and requirements trackers:

  • Using defect URLs to link test results to GitHub Issues
  • Using the defect plugin for GitHub to push and hover over GitHub Issues
  • Using reference URLs to link test cases to GitHub Issues
  • Using the reference plugin for GitHub to look up GitHub issues in the References field of TestRail entities such as test cases, test runs, and milestones

You don’t have to use all listed integration options, and you can mix and match the integration with other tools. Configuring the integration usually only takes a few minutes.

You can find more information about TestRail’s bug, issue, and defect integrations here, or read more about TestRail’s integrations with requirements and references management systems here.

Configure the integration with GitHub

Defects

Defect URLs

The defect URLs are used to let TestRail know how to convert a Defect ID into a URL for your GitHub instance. There are two types of URLs: View URLs and Add URLs.

The Defect View URL is used to build a link to GitHub based on the Defect ID, so you can simply click on an ID to open the issue directly in GitHub.

When you set up the Defect View URL integration with GitHub Issues, you can paste the ID of a GitHub issue in the Defects field of a test result and TestRail will automatically convert that ID into a clickable link to view the issue in GitHub. This allows your team to link tests with GitHub issues and create defect and traceability reports in TestRail. 

The Defect Add URL is used to create a link to the issue creation screen inside GitHub. Once the URL is configured, a new Add link appears next to the Defects field in the Add Test Result dialog and the ‘Defects’ menu in TestRail’s Defect From Anywhere feature. This link allows you to jump to GitHub’s New Issue form to report a new bug.

Add Defect link in ‘Add Result’ dialog

Add Defect link in Push Defect From Anywhere

If you see the ‘Add’ link, you can copy the ID of your new GitHub issue and enter it into any Defects or References field inside TestRail.

To configure GitHub’s Defect URLs in TestRail, select Administration > Integration. You can alternatively enter separate bug tracker URLs for each project under Administration > Projects. Use the following example URLs to configure the addresses:

GitHub installation
Defect View Url: https://github.com/<owner>/<repository>/issues/%id%
Defect Add Url:  https://github.com/<owner>/<repository>/issues/new

Note: Please make sure to replace <owner> and <repository> with the actual keys of the owner and repository in GitHub you want to integrate with.

Example:

Here, the owner is varunagithub and the repository is New-Repository

Defect View URL: https://github.com/varunagithub/New-Repository/issues/%id%/

Defect Add URL: https://github.com/varunagithub/New-Repository/issues/new

Here’s how the Defect URLs look in TestRail:

 Additionally, if you have the defect plugin configured, you’ll also be able to hover over the defect ID to view the issue details directly from within TestRail.

Defect Plugin

Defect plugins can be used to implement a deeper bug tracker integration and TestRail comes with a ready-to-use GitHub defect plugin.

The TestRail defect plugin for GitHub allows you to push new issues to GitHub without leaving TestRail and automatically link them as defects. You can also hover over linked defects to view GitHub Issues fields like Title, Assignees, Labels, Projects, Milestone, and Linked pull requests.

To configure the defect plugin, select Administration > Integration and select GitHub in the Defect Plugin drop-down list. The next step is to configure all the necessary connection settings in the [connection] category. You can also configure the integration for a specific project by editing a project’s settings.

The Defect Plugin utilizes GitHub’s API, so valid credentials are needed to connect to the API. You can enter a single set of credentials here for all GitHub users. or you can insert User Variables to allow each TestRail user to configure their own credentials. These credentials may be a combination of an email address or username with a password or API key.

We strongly recommend configuring User Variables. You can learn more about configuring these in our documentation here.

If your team is using two-factor authentication or single sign-on with GitHub, your users will need an access token instead of your Github password in order to successfully use integration features. An access token for GitHub can be created here: https://github.com/settings/tokens

How to configure the GitHub Issues plugin

Step 1: Select Administration > Integration 

Step 2: Select GitHub in the Defect Plugin drop-down list.

Step 3: Configure all the necessary connection settings in the following categories

[connection]

user=varunagithub  (Whatever the username you are signed in as within GitHub)

password=ghp_ZE7tHmVtoken 

Note: To generate a GitHub API token in your GitHub account, click on Settings, then Developer settings. Then click on Personal Access Tokens in the left menu. Click Generate new Token.

Please refer to the below screenshot.

[repository]

owner=varunagithub 

name=New-Repository

[push.fields] and [hover.fields]

These fields are preconfigured. But to do any customizations, refer to the Defect Plugin Customizations section.

Once the connection settings are saved and the defect plugin configured, you should be able to  ‘Push’ and ‘hover’ over the Github issue ID within TestRail now.  

The option to push a new defect will appear in the Defect from Anywhere menu, as well as in the Add Result dialog within a test run. This dialog allows you to create a new issue in GitHub without leaving TestRail.

Push Defect in Defect from Anywhere menu

Push Defect link in ‘Add Result’ dialog

TestRail shows the defect description when hovered over the defect/issue ID 

Defect Plugin Customizations

The GitHub defect plugin was built to work with a standard GitHub configuration. TestRail allows you to customize the fields which appear in the Push dialog as well as when hovering over Defect IDs. Preconfigured push fields and hover fields can be set to ‘on’ or ‘off’ by typing the text (on/off) in their respective [push.fields] and [hover.fields] sections, or changing the order in which they appear by the fields in each section.

Note: Some fields (such as the title field) are always enabled and may not appear in these sections due to always beings required by either TestRail or GitHub.

The GitHub Defect Plugin also supports modifying field properties as well as custom fields.

In addition to enabling, disabling, or re-organizing fields as described above, it’s also possible to change additional options per field. To do this, you would define a section for the field, such as [push.field.description], and specify additional options within this section.

[push.field.description]
label=Detailed Description
required=true

Please see below for full list and explanation of the available field options:

Option Description
label Specifies the display name of the field in the Push Defect dialogue
size Specifies whether the field is displayed as a compact field (compact) or uses the full width of the dialogue (full)
remember Specifies if previously selected values for this field should be remembered and restored the next time the Push Defect dialogue is opened
required Specifies if this field is mandatory or optional and whether users are required to enter a value for this field in the Push Defect dialogue

 

References

Reference URLs

The reference URLs are implemented in a very similar manner to Defect URLs described above. The primary difference is Reference URLs are used for the ‘References’ field on various TestRail entities (instead of the Defects field).

The Reference View URL is configured in the same manner as described above and allows TestRail to create a direct link to a GitHub based on an ID entered into a References field.

The Reference  Add URL is also configured in the same way as the Defect Add URL. Once configured, an ‘Add’ link will appear next to each References field in your instance or project. If you use the ‘Add’ link, you can copy the ID of your new GitHub issue and enter it into any Defects or References field inside TestRail.

Add References link in Testcase

To configure GitHub’s Reference URLs in TestRail, select Administration > Integration. You can alternatively enter separate bug tracker URLs for each project under Administration > Projects. Use the following example URLs to configure the addresses:

GitHub installation
Reference View Url: https://github.com/<owner>/<repository>/issues/%id%
Reference Add Url:  https://github.com/<owner>/<repository>/issues/new

Note: Please make sure to replace <owner> and <repository> with the actual keys of the owner and repository in GitHub you want to integrate with.

Example:

Here, the owner is varunagithub and the repository is New-Repository

Reference View URL: https://github.com/varunagithub/New-Repository/issues/%id%/

Reference Add URL: https://github.com/varunagithub/New-Repository/issues/new

Here’s how the Reference URLs look in TestRail:

If you have the Reference plugin configured, you’ll also be able to hover over a Reference ID to view the issue details directly from within TestRail. Keep reading to learn how to use and configure the Defect and Reference Plugins.

Reference Plugin

Reference plugins can be used to retrieve additional information from IDs entered into the References field of TestRail entities, such as test cases, test runs, and milestones. The Reference Plugin is configured on the References tab of the integration settings, in the same manner, the Defect Plugin is configured.

 To configure the reference plugin, select Administration > Integration and select GitHub in the Reference Plugin drop-down list. The next step is configuring all the necessary connection settings in the [connection] category. You can also configure the integration for a specific project by editing a project’s settings.

Note: If you have the Defect Plugin already configured for GitHub, you can simply copy your [connection] and [hover.fields] sections from the Defect Plugin into the Reference plugin configuration. Alternatively, you can choose different fields to be displayed for References and Defects.

Note: The reference plugin supports hovering over Reference IDs but does not support a Push dialog for the References field. 

It is also possible to customize the fields which appear via link hovering and add custom fields, as described in the Defect Plugin Customizations section.

Once the connection settings are saved, you should be able to ‘hover’ over the Github issue ID within TestRail now. 

For more information about Reference plugins, read here.

Additional Integration Customizations

TestRail also supports custom defect plugins. If your team uses TestRail Server, it is possible to further customize the defect plugin (or build your own) to support additional field types or other data. Please see out documentation here for additional information.