QA Lead guide

This page walks a QA Lead through the work Orbit expects of them: watch quality, keep failures and flaky tests under control, run releases to a decision, and share the results.

Who this is for

QA Leads of a project. Everything here also works for Project Admins and System Admins, who have every QA Lead right.
QA Engineers can read this page too: they see the same pages, and the parts that need the QA Lead role are marked.

What only a QA Lead (or higher) may do: quarantine tests, create Jira issues, change run profiles, manage releases, cycles, gates, decisions and risks, change the quality settings, and save reports, schedules and share links. The full table is in People and roles and in Roles and permissions reference.

Before you start

You need the QA Lead role on the project. Ask a Project Admin if you only have QA Engineer or Viewer.
Choose the project in the project switcher at the top of the page. Every page below shows that project.
Read Quality figures explained once. It says what each figure counts and why two pages can show different verdicts. This guide links to it instead of repeating it.
The menu has four groups: Run (Executions, Tests), Quality (Quality, Failures, Flaky), Release (Releases, Requirements, Risks and gate) and Insights (Reports, Customers). Project settings is last.

Your day in Orbit

A typical day, in order:

Open Home. The Needs you list holds what waits for you, for example failure groups past their SLA, proposed risks to review, the flaky queue, quarantines that end soon and cycles that are behind. Failure radar shows the open failure groups, Quality pulse the health of the project, and Test debt the debt and how it moved this week.
Work the Failures page (see Triage failures).
Work the Flaky queue (see Deal with flaky tests).
Review proposed risks on Risks and gate (see Review risks).
Check the open cycles of your releases (see Run a release).

The bell in the header opens Notifications. Choose what reaches you on its Settings tab (see Notifications).

Watch quality

Open Quality in the menu. The page has four tabs, and a Workload and an Environment filter at the top (All workloads, All environments by default).

Tab	What it answers
Health	One verdict for the last 7 project days: Healthy, Watch, Critical or No data. It is the worst of five checks: pass rate, new and persistent failures, flaky share, duration change and P1 tests.
Coverage	Which catalog tests passed on their latest result, grouped by priority, suite or tag (Group by).
Performance	Which tests are getting slower, and, By site, how fast each environment answers from each office.
Debt	Test debt in twelve categories, with a 90-day trend.

Each figure has an info icon: choose it for “How is this computed?”. The thresholds and the worked example are in Quality figures explained.

Debt counts things to clean up, not a score. The categories are Quarantine overdue, Flaky, Persistent failure, Not run, Always skipped, Getting slower, No @id, No explicit owner, No requirement, Orphaned quarantine or triage, Old known issue and Requirement without a test. Pick a category to see the tests, entries or requirements behind its count. When a category is about the test code (No @id, No explicit owner, No requirement), pass the list to the workload authors: see Workload author guide.

Triage failures

Open Failures in the menu (page Failures and triage). Orbit puts every failure with the same error into one failure group. A group has an owner, a status and an SLA.

The page has three views: Groups (the list), Triage board (one test at a time, with the keyboard) and Investigations.

Statuses

Status	Meaning
New	The group was just created, or it came back after being resolved.
Triaged	Someone looked at it and it has an owner.
Resolved	Fixed. A new failure with this error reopens the group as New.
Known issue	Accepted for now. New failures are counted but do not reopen it.

Two marks are shown on top of the status: Recovered (no test fails with this error any more) and Investigating (an open investigation holds the error). A group that stays recovered for 14 days is resolved by Orbit on its own.

Steps

On Groups, filter by workload, priority or status. Past SLA shows the groups that missed their SLA.
Open a group. The page shows Owner and status, Affected tests (each test and environment that failed with this error in the last 30 days), Failures per day, Jira, Investigations and the Timeline.
Choose Take ownership, or pick an Owner (the owner must be a member of the project and is notified).
Choose Mark as triaged once you know the cause. Add a note to say what you found.
When it is fixed, choose Resolve…, pick Resolved or Known issue under Outcome, and say How it was resolved. Choose Resolve.

To work many groups at once, select them on Groups and use Assign or Resolve. Resolve recovered (N) resolves every group that no longer fails in one step.

On the Triage board, the left side lists the failing tests of each open group and the right side shows the focused test’s latest failure. Each action has a button and a key: j/k next or previous test, J/K next or previous group, b Mark as Bug, o Take ownership, r Resolve the group, q Quarantine…, i Investigate…, ? the shortcut sheet. Test labels (Bug, Environment issue, Known flaky) are for QA Engineers and higher; quarantine needs the QA Lead role.

The SLA

The SLA has two stages, counted in business days (Monday to Friday in the project time zone, no holiday calendar) from when the group opens or reopens. A group takes the highest priority of its tests.

Highest priority	Owner within	Resolved within
P1	1 day	3 days
P2	2 days	7 days
Other	3 days	14 days

These are the defaults. Change them in Quality settings (see Quality settings). The SLA is a reminder: it never changes a gate, readiness or a release decision. When a group passes its SLA, its owner and the QA Leads are notified.

Jira issues

A QA Lead can create a Jira issue from a failing result: open the execution, then Create Jira issue… on the failed test. Orbit keeps one issue per error, and the group page shows its key and Jira status. A Project Admin sets up Jira first (see Project Admin guide).

Investigations

An investigation is a case the team chases across executions, tests and failure signatures. Use Investigate… on a failure, a test or an execution to start one or add to an open one. Investigations do not change release readiness; when they should, open the case and choose Record a risk from this case….

Deal with flaky tests

Open Flaky in the menu. It has three views: Queue, Trends and Quarantined.

The Queue lists the tests that were flaky or failing in the last 7 days, P1 tests first. For each test choose one:

Action	Who	What happens
Investigate	QA Engineer	Opens an investigation. The test stays in the queue.
Known flaky	QA Engineer	Takes the test out of the queue. It still counts in the flaky share and in risk detection.
Quarantine	QA Lead	The test keeps running in shadow mode, but no longer counts.
Open test	Anyone	Opens the test’s page with its history.

Quarantine a test

In the Queue, select one or more tests and choose Quarantine.
Enter a Reason and choose Until (the end date).
Choose Quarantine. You become the owner of each entry. Stations apply the change at their next configuration refresh, within 5 minutes.

From the queue, a quarantine is always a Shadow run: the test still runs, but its result does not change the execution outcome. From a test’s own page (Tests, then the test, Quarantine this test) you can also choose Skip: Stations leave the test out of runs.

A shadow entry ends on its own once the test passes enough runs in a row (5 by default, the Quarantine release setting). A skipped test never runs, so its quarantine cannot end automatically: it ends on its date or by hand, and its row says “Runs are skipped: switch to shadow to earn release”. Quarantined lists the active entries with their release progress. To end an entry by hand, open the test’s page and choose End quarantine (or select tests on Tests and choose Release). Change on the test’s page extends it.

How quarantine changes the figures is in Quality figures explained.

Quality settings

Open Failures, then the ⋯ menu (More actions for failures), then Quality settings. QA Leads and Project Admins change them; everyone else sees them read only.

Section	What you set
Failure group SLA	Owner within and Resolved within per priority, in business days.
Quarantine release	Release after N passing runs (3 to 20).
Test debt	When a test counts as debt: days without a run, how long a quarantine may last, how old a known issue may get.
Notifications	When a run’s new errors, a pass rate drop or a flaky rise are worth a notification for this project, and how sensitive the alerts are (Sensitivity: Relaxed, Standard, Strict or Custom).

Use the defaults puts every value back. Choose Save settings.

Set the quality gate

A gate is a list of rules a project or a release must meet. Open Risks and gate, view Gate rules.

Choose Change preset (or Edit rules for a gate that exists). The presets are Standard (recommended), Strict, Lenient and Hotfix. Each shows “Would you pass today?” against the last 7 days.
Choose Use this, or Start from blank.
Adjust the rules. Each rule has a metric, a threshold and a level:
- Blocking: a failing rule fails the gate.
- Warning: a failing rule keeps the gate passing but marks a release “ready with risks”.
- Without data, a blocking rule gives “not enough data”, never a pass.
Choose Save as new version. Earlier versions stay under Earlier versions.

Preview now computes the last 7 days and stores nothing. Evaluate now stores a verdict under Latest verdicts.

A release uses the project rules unless it sets its own: on the release’s Readiness tab, section Gate rules, choose Set rules for this release (or Use the project rules to go back).

Review risks

Open Risks and gate, view Risks. Orbit checks every environment each hour and proposes risks it finds in recent results. A proposed risk counts only once you accept it.

Open the Proposed risks.
For each one, choose Accept (it becomes an open risk) or Dismiss (the same finding is not proposed again for 7 days, unless its level rises).
To add one by hand, choose Record a risk: a title, a Level, an Environment (or Whole project) and a Release (or Every release). Choose Record risk.
When the cause is gone, set the risk’s status to Mitigated or Closed. An accepted risk never closes on its own: it shows Condition no longer seen.

An open or acknowledged risk of medium level or higher keeps a release at “ready with risks”; a high or critical one blocks it. Detect now runs the check at once.

Run a release

A release is tested in regression cycles on its environments. It moves through the stages Planned, In testing, Quality validation, Ready for release, Released and Closed.

Create the release

Open Releases and choose New release.
Fill in Version, Name, Starts and Ends.
Tick the Environments the release is tested on, and mark the Sign-off ones. Sign-off environments decide whether the release is ready; if none is marked, all of them count.
Choose Create release.

Set the scope

On the release, tab Scope, choose Edit scope. Narrow the tests by Workloads, Suites, Priorities, Features, Modules, Processes and Requirements, include or exclude Single tests, then Save scope. Picking nothing in a filter keeps every value. New tests that match join on their own.

When you change the scope after cycles exist, open and planned cycles do not follow on their own: the release page offers Apply to cycles.

Plan and close cycles

On the Cycles tab, choose Plan a cycle. Pick the environments, Starts, Ends, an optional Name and, if you want, Suites. Choose Plan cycle.
Executions join a cycle by the time they start. Testers run as usual from the Station.
Open a cycle to follow its completion, pass rate and failing tests. A run that belongs elsewhere can be moved with Move to cycle… or Take out of cycles.
When the testing is done, choose Close cycle now. Closing a cycle takes a snapshot and evaluates the gate on its environment. A cycle also closes when its end date passes. Reopen undoes it.

The checklist

Every new release starts with the project’s checklist. Required items hold the release in Quality validation until they are done.

To change the template for new releases: Releases, the ⋯ menu (More actions for releases), Checklist template. Saving never changes existing releases.
To change one release: tab Readiness, section Checklist, Add an item to this release. Tick items as they are done.

Approvals

On the Readiness tab, section Approvals, choose Name approvers (or Edit approvers).
Choose Add an approver and pick a project member. Tick Required for people the release must wait for; optional approvers are told but not waited for. At most 20.
Choose Save approvers.

Each approver signs the latest proceed decision with Approve or Reject (a comment is needed to reject). A reject holds the release until a QA Lead records a new decision. An approver who left the project shows No longer a member; remove them with Remove.

Record a decision

On the Readiness tab, read the verdict at the top (Blocked, In progress, Ready or Ready with risks) and the list of what blocks the release. Quality figures explained lists every reason.
Choose Assess now to evaluate the gate on each sign-off environment and count the open risks. It also runs when the last cycle closes.
Under Record a decision, choose Proceed, Proceed with risk or Hold. A reason is required for the last two.
Choose Record proceed (or Record proceed with risk, Record hold).

Decisions cannot be changed or deleted. A later decision replaces an earlier one, and approvers sign the latest one.

When the release goes out, choose Mark as released. If something still blocks it, Orbit offers Release anyway and asks why; the note, who released it and what blocked it are kept on the Record tab and in the audit log. Undo release takes it back.

Summarize with AI on the Readiness tab writes a short summary of the release, when AI is turned on for the project.

The test matrix

Tab Matrix shows the latest result of each test in the scope on each environment. Choose Current cycle of each environment or Every cycle of the release, filter by environment, workload and suite, and Download CSV for a spreadsheet. Runs with local overrides do not count.

Save the evidence pack

An evidence pack freezes what was signed off and stamps it with a SHA-256 hash. Orbit saves one by itself when the approvals of a proceed decision are complete.

Open the release, tab Record, section Evidence packs.
Choose Save evidence pack to save one now (QA Lead).
Open a pack, then Download ZIP or Print or save as PDF.

Packs are kept for the life of the release, archive included. HTML reports, traces and videos are linked, not copied: they expire with their retention, and the pack lists each link with its expiry date. If Orbit could not save the pack after the approvals, the release says so with Save now.

Requirements and test debt

Open Requirements. A test links to a requirement with a @req:KEY tag in its code; a requirement is verified when all of its tests passed.

New requirement adds one: a Key (fixed once created), a Title, and optionally a Description, Priority, Feature and Link.
Import CSV adds or updates many at once: see Import requirements from a CSV file.
Open a requirement for Coverage by environment, its linked tests and the releases that name it. Archive leaves it out of coverage; a requirement that a release names cannot be deleted.
To narrow a release to some requirements, pick them in the release’s Scope tab.

The Requirement without a test and No requirement debt categories (on Quality, tab Debt) show the gaps in both directions.

Reports and schedules

Open Reports. It has three tabs: Generate, Saved and Schedules.

On Generate, pick the Report: Summary, Quality, Release (then Choose a release) or Personal (your own work; never saved or shared).
Pick the Period and the Environments, then choose Generate report.
Export it with Export report: Print or PDF, Markdown, CSV or JSON.
To keep it, choose Save to the project (QA Lead). Saved reports are kept 180 days and listed on Saved.

To share a saved report with someone outside Orbit, open it and choose Share, then Share outside Orbit. Pick Valid for (1, 7 or 30 days) and choose Create link. Anyone with the link can read this report, without signing in, until it expires. Copy the link at once: once you leave the page it cannot be shown again. Revoke stops a link.

Schedule a report

On Schedules, choose New schedule.
Pick the Report, the Period, the Environments, Repeat (Every week or Every day), the day and the time (At).
Optionally Post to Teams (a channel a Project Admin set up) and Email to members. Members get the title, key numbers and a link, and sign in to read the report. Email works once a System Admin sets it up.
Give the schedule a Name and choose Create schedule. Scheduled reports appear under Saved.

Run now, Pause, Resume, Edit and Delete are on each schedule.

Notifications for a QA Lead

Open the bell, then Settings. For each kind tick In Orbit and By email, and for some choose All or Only mine. The project’s Quality settings decide when failures and alerts are worth a notification at all.

As a QA Lead you are told when a failure group passes its SLA, when a resolved group without an owner fails again, and about the quality alerts of your projects (at most once per environment and day).

What can go wrong

You see	Why	What to do
A button is disabled with “Needs the qa-lead role”	Your role on this project is lower.	Ask a Project Admin for the QA Lead role.
A test is not in the flaky Queue	It is quarantined, marked Known flaky after its newest flaky run, or stable for 7 days.	Look on Quarantined, or on the test’s page.
The release stays Blocked after the fix	Cycles are not closed, a risk is still open, or no decision was recorded since the last cycle closed.	Read the list under the verdict. Quality figures explained has each case.
Approvers cannot sign	The latest decision is Hold, or there is no decision.	Record a Proceed or Proceed with risk decision.
“The release scope changed. N cycles do not follow it yet.”	The scope was edited after the cycles were made.	Choose Apply to cycles.
A quarantined test does not leave by itself	It is in Skip mode, or automatic release is off.	Switch it to Shadow run with Change, or choose End quarantine.
A failure group came back as New	The error was seen again after it was resolved.	Look at the new failure; resolve again or mark Known issue.
The shared link is lost	Links are shown once.	Revoke it and create a new one.
An error code appears	See its meaning and what to do.	Error codes reference.

Values to record

Value	Where you find it	Secret?
Share link of a report	Shown once after Create link	Treat as a secret until it expires: anyone with it can read the report. Send it to the reader only.
Evidence pack hash (SHA-256)	Release, Record tab, the pack	Not secret. Keep it with the release documents so the pack can be checked later.
The chosen gate preset and SLA values	Risks and gate, Gate rules; Quality settings	Not secret. Note changes in your team’s test plan.