Testream - Automated Test Management and Reporting for Jira

Testream vs Xray: Automated Test Reporting Without Test-Case Setup

Xray is powerful for teams that need manual test planning in Jira. Testream is built for automation-first teams that want test evidence in Jira without managing test cases as Jira issues.

Get Started Free Read Docs

Xray stores test cases, test plans, and test executions as Jira issues. This gives teams full JQL search over test data but adds thousands of extra issues to the Jira instance over time. Teams with large automated suites often find that maintaining a parallel test-case layer in Jira creates overhead without proportional value.

Testream takes a different approach: it does not store test cases as Jira issues. Automated test results are published from CI/CD pipelines through native reporters, and the evidence—pass/fail status, failures, artifacts, trends—appears in Jira without requiring any manual test-case creation or maintenance.

The choice depends on workflow: if your team needs structured manual test planning with JQL-powered test queries, Xray fits. If your team runs automated tests and wants Jira to reflect real CI/CD outcomes without duplicate test-case upkeep, Testream is the direct path.

Testream vs Xray at a glance

Comparison area

Xray

Testream

Test data model

Stores test cases, executions, and plans as Jira issues for deep JQL access.

Publishes automated run evidence into Jira without creating a parallel test-case issue layer.

Best fit

Teams that need formal manual test planning, governance, and issue-based traceability.

Automation-first teams that want real CI/CD outcomes visible in Jira with minimal setup.

Operational overhead

Requires ongoing maintenance of test entities and the mapping between automation and Jira test records.

Keeps the automated suite as the source of truth and removes duplicate test-case upkeep.

Team visibility

Visibility depends on Xray workflows and licensing for everyone who needs access.

Project-based pricing makes Jira-visible evidence available to the wider delivery team without per-seat tax.

Pricing model

Pricing scales per Jira user or team members who need access to Xray workflows and reports.

Pricing scales by Jira project workspace so teams can share evidence broadly without paying per team member.

Where Xray adds overhead for automated teams

Every test case, execution, and plan adds issues to your Jira instance, causing bloat at scale.
Automated test results require a separate sync step to link CI/CD output to Xray test entities.
Teams maintain two views of quality: the automated suite and the Jira test-case layer, which drift over time.
Per-user pricing means every Jira user who needs test visibility incurs a license cost, even if they only review results.

Key implementation facts

Testream publishes automated test results directly from CI/CD into Jira without creating test-case issues.
Xray stores all test entities as Jira issues, which can cause instance bloat at scale.
Testream's per-project pricing makes test evidence accessible to the whole team without per-user costs.

How Testream's approach differs

Step 1

Publish directly from test runners

Add a Testream reporter to Playwright, Jest, Cypress, or any supported framework. Every CI/CD run publishes results automatically with no Jira issue creation needed.

Step 2

Results land in Jira as run evidence

Pass/fail status, branch, commit, artifacts, and trend data appear in Jira without creating test-case issues. The evidence is visible in issue panels, release views, and dashboards.

Step 3

No duplicate test-case maintenance

Because Testream does not store tests as Jira issues, there is no sync step between CI/CD and Jira. The automated suite is the source of truth, not a parallel test-case layer.

Step 4

Project-based pricing without per-seat tax

Testream charges per Jira project, not per user. Every team member can view test evidence without additional license cost.

Why automation-first teams choose Testream over Xray

Testream was built for teams that already run automated tests and want Jira to reflect real CI/CD outcomes. The core insight is that maintaining a manual test-case layer alongside automated suites creates duplication and drift.

By skipping the test-case issue model, Testream removes the most common source of Jira bloat in test management while giving engineering, QA, and product teams the same evidence they need for release decisions.

Publish test results from CI/CD without Jira issue creation
Supported frameworks: Playwright, Jest, Cypress, JUnit, .NET, Vitest, WebdriverIO, Mocha, Pytest
Failure inspection with stack traces, screenshots, and artifacts
Trend dashboards for pass rate, flaky behavior, and suite growth
Release-linked quality views for go/no-go decisions
Project-based pricing with no per-seat cost

Frequently asked questions

Does Testream create Jira issues for test cases like Xray does?

No. Testream does not store test cases as Jira issues. Test results are published directly from CI/CD and appear as evidence in Jira without creating or maintaining a separate test-case layer.

Can I migrate from Xray to Testream?

Yes. You keep your existing test suites and CI/CD pipeline. Install a Testream reporter, configure your API key, and start publishing results. There is no data migration because Testream reads from your automated tests, not from Xray's issue store.

Does Testream support BDD and Cucumber like Xray does?

Yes. Testream supports BDD Gherkin specs through its Rovo AI integration, allowing teams to create and assess scenarios against real automated test evidence in Jira.

Do I lose JQL search over test data?

Test data appears in Jira as run evidence on issues, releases, and dashboards rather than as JQL-queryable issue types. Most teams find this trade-off beneficial because it eliminates issue bloat while keeping test evidence accessible where delivery decisions happen.

Explore more ways to connect testing workflows with Jira delivery.

Start Free

Popular search