Writing TestRail Cases

How to phrase TestRail steps and preconditions so they convert into deterministic actions in the Test Agent.

The Smart Converter turns each step’s text into an action using a regex grammar. When your wording matches a pattern, the step runs deterministically (fast, stable, type: nav). When it doesn’t, the step falls back to an LLM interact (slow, non-deterministic, may fail to find the element). This page tells you how to write so steps land on the deterministic path.

At a glance

  • Quote the on-screen label exactly: "Save", "Find device", "System Settings".

  • Name the element type (recommended): tab, widget, sidebar, drop-down, … — makes the step deterministic; optional for plain buttons (the LLM clicks those by label).

  • Use the canonical verb for that element (tables below). Verbs are matched strictly — Open the "Statistics" tab is not recognised (a tab needs Go to / Navigate to / Click).

  • Be concrete — name one explicit action, never Perform all required actions to … or a position ("top-right button"). Environment setup belongs in Preconditions.

  • A non-matching step still runs — just slower and less reliably (the LLM drives the browser). It won’t "break", but avoid it for critical navigation.
All example phrasings are in English — the converter grammar matches English text. Use straight quotes "…​". Curly quotes (“ ” ‘ ’), en/em dashes, and Cyrillic look-alike letters are auto-normalised, but prefer plain ASCII.

The Four Golden Rules

  1. Quote the UI label — exactly as it appears on screen.

  2. Name the element type — recommended — button / tab / widget / sidebar / drop-down / option / section. It is what makes a step deterministic, but it is not mandatory: a step without it still runs via the slower LLM path. It matters most for widgets, tabs and the sidebar; plain buttons the LLM usually clicks fine by their label.

  3. Use the canonical verb for that type (see the tables). Wrong verb → falls through to interact.

  4. Be concrete — avoid blanket or spatial wording. A step must name one explicit UI action, not a goal or a position. Phrasings like Perform all required actions to make the "Save" button active or Perform all required actions to create any diagnostics group update, and positional descriptions like "the button in the top-right" or "the third row", give the converter nothing to match — they fall through to a slow LLM interact (or get guessed wrong). Write the concrete click(s) instead. If it is environment setup (e.g. "create a diagnostics group update"), move it to Preconditions with a recognised trigger phrase.

Goal ✅ Write this ❌ Don’t write

Login

Log in to the Support portal
Log into the Angular Management Portal

Sign in, Open the portal, Log in to the dashboard

Open a device

Open the mentioned device
Open any TR-069 device
Open the device by clicking on it

Select a new device, Find the gateway

Tab

Go to the "Find device" tab
Navigate to the "Statistics" tab
Click "Configuration" tab

Open the Statistics tab (verb),
Go to Statistics tab (no quotes),
Go to the "Statistics" page (no word tab)

Left menu / sidebar

Select "Settings" from the left menu
Select the "System Settings" item from the left menu
Click "Device Diagnostics" in the sidebar

Select Settings (no quotes, no anchor),
Open device settings (an exclusion — see Traps)

Widget / cube

Navigate to the "Network Score" widget
Navigate to the "Diagnostics" cube

Open the … widget, Go to the … widget (needs verb Navigate to)

Section

Navigate to the "WAN" section

Scroll to the WAN section

Left menu — supported forms

Both from the left menu and in the sidebar resolve to a sidebar nav, but they are two different patterns with different verbs and quoting rules — don’t assume they are interchangeable.

Anchor from the left menu (also from the left-hand menu / from the left-side menu):

  • Allowed verbs: Select / Click / Choose / Open / Expand.

  • The label must be in quotes.

  • You may optionally insert the and a descriptor noun (item / tab / option / section / entry) between the label and from.

  • Examples:

    • Select "Settings" from the left menu

    • Select the "Settings" tab from the left menu

    • Select the "System Settings" item from the left menu

    • Expand the "System Settings" item from the left menu (verb Expand is OK too)

    • Open "Provision manager" from the left menu

Anchor in the sidebar:

  • Allowed verbs: only Click or Navigate — Select / Choose / Open / Expand do not match here (they fall through to interact), even though they work for from the left menu.

  • Quotes are optional here, but still write them for consistency.

  • Example: Click "Device Diagnostics" in the sidebar.

When in doubt, prefer the from the left menu form — it accepts more verbs and is the better-tested path.

Buttons

✅ Write this ❌ Don’t write

Click the "Save" button
Click on "Run Diagnostics" button
Use the "Refresh" icon

Press the "Save" button (verb Press doesn’t match a bare button)

Use "Add port" to open the port mapping form
Click "Generate" to create the report

—

  • Canonical button verb: Click (or Use). The literal word button or icon is required.

  • The form Use "<button>" to <verb> (verbs: open, create, add, view, run, start, save, apply, generate, delete, edit, send) treats the quoted token as the button and the tail as the destination — keeps the tail from being read as a menu item.

  • Checkbox + button in one sentence: Select the device by its check-box and click the "Uninstall" button → becomes two actions (tick the row, then the button).

Goal ✅ Write this

Open a dropdown

Expand the "Period" drop-down

Open + pick

Expand the "Configuration" drop-down and select the item corresponding to "5GHz"

Pick from an open menu

Select "Upload" from the drop-down menu

Pick a value

Select the "Enabled" value · Select "Bridge" option

Named field-dropdown

Select "Last hour" in the Period field
Choose "Dynamic" from the Tariff plan

  • The word drop-down / dropdown / drop down is required for Expand.

  • Strict field names (suffix optional): Period, Tariff plan, Tariff, Device group, Diagnostic type (also Diagnostics type), Time range, Channel bandwidth, Operating standard.

  • Loose field names (group, band, range, frequency) require an explicit suffix field / drop-down / list / selector: Select "5GHz" from the band field ✅, but Select "5GHz" from the band ❌ (falls through to interact).

⚠️ Traps (these fall through to the slow LLM interact)

  1. No quotes around the label in a tab / section / left-menu / dropdown / select step. (Exception: the in the sidebar form treats quotes as optional — but quote anyway for consistency.)

  2. Wrong verb for the anchor: Open for a tab/widget; Press for a button; Go to for a widget; Select / Choose / Open / Expand with the in the sidebar anchor (use from the left menu for those verbs, or Click with the sidebar anchor).

  3. Missing the type word (tab / button / widget / section / sidebar / drop-down).

  4. Open device <subview> — trace, setting(s), tree, info, activity, history, diagnostic(s), map, menu, sidebar, group, profile, model, type, list, location, configuration are deliberately not treated as "open the device" (they are sub-views of an already-open device). Phrase these as a tab or left-menu nav instead.

  5. … by clicking on it without the word device (e.g. Open the diagnostic by clicking on it) — stays interact.

Preconditions — use the exact trigger phrases

The converter turns precondition text into pre-flight actions that actually configure the environment before the run (see Run-Time Pre-Flight). Phrase them so the trigger matches:

What to set up Trigger wording in the precondition

Create monitoring

Monitoring is created / Create monitoring

Diagnostics performed

… diagnostics result / diagnostic(s) completed / Perform all required actions to get the diagnostics

Group update (diagnostic)

group update … diagnostic

Interference / auto-channel

Set auto channel, neighboring Wi-Fi, channel quality

Deployment file

mention deployment file near file management

UI setting

ShowPasswordAsPlainText = true · MonitoringSelectNodeTr… = true

ACS overwrite-provision

overwriteProvisionParameters set to 0, 1 or 2

ACS credential length

credentialMinLength = <n> · credentialMaxLength = <n>

Raw ACS task

an ACS_TASK precondition (spv / gpv_direct / add_object / delete_object / reboot / factory_reset)

  • Start each new precondition with Make sure that …, Make sure …, or a number 1. … (the number needs a literal dot: 1., not 1)).

  • Recognised diagnostic names: IP Ping, Download, Upload, NSLookup, Traceroute (the alias trace also resolves to Traceroute), Neighboring Wi-Fi.

  • ShowPasswordAsPlainText accepts only = as the separator (ShowPasswordAsPlainText = true), not set to/is. MonitoringSelectNode… and overwriteProvisionParameters are more lenient (= / is / set to / to all work). When unsure, use =.

  • For the <Parameter> = <true|false> UI-setting triggers, the menu path is optional — the converter matches the Parameter = value token anywhere in the precondition and ignores trailing text, so ShowPasswordAsPlainText = true and ShowPasswordAsPlainText = true in Settings > System settings > Interface are equivalent. Just keep the parameter and its value together (path after the value, never between them).

  • Pure-calculation lines (Make sure…, Ensure…, Verify that…, Param = value;, The sum of…, RSSI … equal …) are recognised as setup, not UI steps — that is expected.

Control / repeat constructs (understood verbatim)

  • Repeat steps 3-10 / Repeat 3-10

  • Repeat 3-10 steps for "5" GHz and "6" GHz bands (expands variants)

  • See the expected result 1

  • Wait until … (attaches to the previous click as a wait-for-content-change)

The Expected field

  • Fill in Expected on every verification step. The agent compares the live page against your Expected to decide whether the step passed — it is the step’s success condition.

  • A verification step with an empty Expected passes silently without checking anything. This is the real danger, and it is worse than a failure: the agent has no success condition, so it does not re-check the page or time out — it just reports the step green even when the page is broken. A missing Expected hides real bugs behind a passing run. (It does not "break" the step; it removes the verification entirely.)

  • Describe a concrete, observable result — which page opened, which buttons/fields are present, which value is shown. Avoid vague wording like "works correctly".

Case skeleton (copy and fill)

Title:  Management Portal > <Area> > <Sub-area> > <What is verified>
Preconditions:
  Make sure that <condition 1, exact trigger phrase>.
  Make sure that <condition 2>.
Steps:
  1. Log into the Angular Management Portal
  2. Select "<Item>" from the left menu
  3. Select the "<Sub-item>" item from the left menu
  4. Go to the "<Tab>" tab            | Expected: <page/elements open without errors>
  5. Click the "<Button>" button      | Expected: <what appeared/changed>
  6. Expand the "<Field>" drop-down and select the item corresponding to "<Value>"
  7. <Verification>                    | Expected: <concrete observable result>

Checklist before saving the case in TestRail

  • Every UI label is in straight quotes and matches the screen.

  • Recommended: navigation steps name the element type (tab/widget/sidebar/drop-down) for the deterministic path — optional for plain buttons.

  • The verb is canonical for the element type (see the tables).

  • Left menu uses … from the left menu or … in the sidebar.

  • Every verification step has a concrete Expected.

  • Preconditions start with Make sure that … and use the exact trigger phrases.

  • No Open device settings/trace/tree for navigation (use a tab or the left menu).

After editing in TestRail

After you change a case in TestRail, reimport it into the Test Agent (Reimport After Converter Changes) — the old YAML in the persistent volume is not overwritten automatically. Tick Force refresh (or pass force_refresh: true) to bypass the Stage-2 idempotency cache.