Pytest Integration with CI Insights

This guide explains how to integrate Pytest with CI Insights using the pytest-mergify plugin. Once installed, test results are automatically uploaded to CI Insights without any extra workflow changes.

You need to install the pytest-mergify plugin to automatically upload your test results to CI Insights. This can be done in different ways depending on your Python dependency setup. Below are a few examples:

pip install pytest-mergify

Or add pytest-mergify to your requirements.txt.

setup(
    name="your-package",
    ...
    install_requires=[
        ...
    ],
    extras_require={
        "dev": ["pytest-mergify"]
    },
    ...
)

Make sure those dependencies are installed when running your tests.

[options.extras_require]
dev =
    pytest-mergify

Make sure those dependencies are installed when running your tests.

poetry add --group dev pytest-mergify

Your workflow should run your tests as usual while exporting the secret MERGIFY_TOKEN as an environment variable.

Add the following to the GitHub Actions step running your tests:

env:
  MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}

For example:

- name: Run Tests 🧪
  env:
    MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}
  run: pytest

Set MERGIFY_TOKEN as an environment variable in your pipeline step:

steps:
  - label: "Run Tests 🧪"
    command: pytest
    env:
      MERGIFY_TOKEN: "${MERGIFY_TOKEN}"

The plugin collects your test results and sends them to CI Insights.

Check the CI Insights dashboard afterward to view execution metrics, detect flaky tests, and review test trends.

If you’re using Tox to manage test environments, you can still use pytest-mergify by passing the MERGIFY_TOKEN and the rest of the CI environment variable into the test environment.

In your CI workflow:

# GitHub Actions
- name: Run Tox Tests
  env:
    MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}
  run: tox

# Buildkite
steps:
  - label: "Run Tox Tests"
    command: tox
    env:
      MERGIFY_TOKEN: "${MERGIFY_TOKEN}"

In your tox.ini, make sure the plugin is included in your testenv dependencies:

[testenv]
# You need to pass the MERGIFY_*, CI, GITHUB_*, etc variables
passenv = *
deps =
    pytest
    pytest-mergify
commands = pytest

If you’re using multiple environments (e.g. py38, py39, etc.), the plugin will work for all of them as long as the token is set correctly.

If you’re running multiple Tox environments (e.g., py38, py39, etc.), we recommend setting the MERGIFY_TEST_JOB_NAME environment variable to identify each environment’s report in CI Insights:

In your CI workflow:

# GitHub Actions
- name: Run Tox Tests
  env:
    MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}
    MERGIFY_TEST_JOB_NAME: tox-${{ matrix.python-version }}
  run: tox

# Buildkite
steps:
  - label: "Run Tox Tests ({{matrix}})"
    command: tox
    matrix:
      - "3.10"
      - "3.11"
      - "3.12"
    env:
      MERGIFY_TOKEN: "${MERGIFY_TOKEN}"
      MERGIFY_TEST_JOB_NAME: "tox-{{matrix}}"