Organizing Tests

Test Suites

Group test cases into suites for organized execution and scheduling

A test suite is a collection of test cases that run together. Use suites to group related tests — for example, all checkout tests, all authentication tests, or a full regression suite.

Creating a Test Suite

  1. Navigate to Suites in the sidebar
  2. Click New Suite
  3. Name your suite (e.g., "Checkout Regression Suite")
  4. Add test cases to the suite

You can add or remove test cases from a suite at any time. A single test case can belong to multiple suites.

Flow Builder

The flow builder gives you a visual editor to control the execution order of tests within a suite.

  • Arrange test cases in the order you want them to execute
  • Visualize the execution flow as a graph
  • Reorder tests by dragging them in the flow view

Scheduling

Set up recurring test runs using cron-based scheduling:

  1. Open a test suite
  2. Go to suite settings
  3. Configure a schedule (e.g., every day at 9am, every hour, every Monday)
  4. Select the environment to run against
  5. Choose the browser type

Scheduled runs execute automatically and results appear in the Runs page. You can set up email notifications to be alerted when scheduled runs complete.

Suite Settings

Each suite has configurable settings:

  • Environment — Which environment to run tests against (staging, production, custom)
  • Browser — Chromium, Firefox, or WebKit
  • Notifications — Email notifications on completion
  • Retry policy — Whether to automatically retry failed tests

Running a Suite

You can trigger a suite run in several ways:

  • Manual — Click "Run Suite" from the suite page
  • Scheduled — Runs automatically based on your cron schedule
  • CI/CD — Trigger via API from your deployment pipeline (see CI/CD Integration)

When a suite runs, it creates a suite run containing individual test runs for each test case. View results in Test Runs & Suite Runs.

Test Suites | QAbyAI Documentation | QAbyAI Docs