Failure Triage

When tests fail, you need to understand why. Failure triage helps you categorize failures, spot patterns, and prioritize fixes.

Failure Tags

Failure tags are labels you create to categorize why tests fail. Examples:

• Network Error — API timeouts, connection failures
• Assertion Failed — Expected behavior didn't match
• Environment Issue — Test environment was unstable
• Flaky Selector — Element detection was unreliable
• App Bug — Genuine bug in the application

Setting Up Tags

1. Go to Settings > Failure Tags
2. Click Add Tag
3. Enter a name and pick a color
4. Save

Tags are team-scoped — everyone on your team shares the same set.

Annotating Failures

When a test fails, you can tag it from the test run detail page:

1. Open a failed test run
2. Click the tag selector in the header
3. Select one or more tags from the dropdown
4. Changes save automatically

You can also add a free-text note to provide additional context about the failure.

Filtering by Tags

On the Runs listing page (test runs view), use the Failure Tags filter to show only runs with specific tags. This helps you answer questions like:

• "How many failures this week were network errors?"
• "Are environment issues increasing?"
• "Which failures are still untagged?"

The filter includes an "Untagged failures" option to find failures that haven't been categorized yet.

AI Diagnosis

QAbyAI automatically analyzes failed test runs and provides an AI-generated diagnosis on the test run detail page:

• Category — Type of failure (e.g., "Network Error", "Element Not Found")
• Root Cause — Plain-language explanation of what went wrong
• Evidence Summary — Supporting information from the execution
• Technical Details — Error messages, stack traces, or relevant logs

AI diagnosis appears automatically for failed steps and helps you quickly understand failures without manually reviewing screenshots and logs.

Triage Workflow

A practical workflow for handling failures:

1. Failure happens — A test run fails
2. View the run — Open the test run detail page
3. Read AI diagnosis — Understand the root cause from the automatic analysis
4. Review media — Check screenshots, video, or trace for additional context
5. Tag it — Apply appropriate failure tags
6. Track patterns — Use tag filters on the runs page to spot recurring issues
7. File a bug — If it's an application bug, create a bug report

Over time, your tagged failure data reveals patterns: which failure categories are most common, whether they're increasing or decreasing, and where to focus your effort.