Adapting Legacy Cases

A stored error row with the text publish outcome unknown stays as it is. A job does not adapt it again, because a new adaptation could write the case twice. Re-gate or Re-run with engine clears it.

The job form sits under the header (write roles only):

Press Start. A job takes at most 1000 cases, and a case listed twice runs once. The job is refused when another adaptation job holds the slot (see One Job at a Time), when an engine is unknown, or when an ollama: engine is picked while no Ollama URL is set.

A change of a rule or of the vocabulary creates a new adapter version. Rows of an older version are stale. A refresh brings them to the current version with the rules only.

The refresh never calls a model:

The owner decides on the model spend for Waiting for model cases. Then select them and press Re-adapt with model. The route of the refresh is POST /api/adapt/jobs/refresh.

A model call gets 240 seconds and is not retried by the job. A call that fails, an answer that is not the expected JSON, and a model refusal each make the case an error row without a repair pass.

An Angular form shows some tabs and enables some buttons only after another action: the Automation events and Automation parameters tabs appear once the chosen model has a template, Add tasks stays disabled until its row has a condition, the Device profile rule tabs need a name and a model, and Send on Custom RPC or on a file upload asks a confirmation pop-up. A legacy case never says so — it was written for a portal that needed fewer actions — so the adapted case looked complete in TestRail and failed on the stand.

scripts/uiflow/registry.yaml lists those gates and the facts that open them. A fact carries the one sentence to perform it, for example Fill in the "Name" field with any value. It also carries the wordings already stating the fact and the converter action for the sentence. For seven of the eight facts, an inserted step runs and does not fall to the agent. The eighth fact, confirm_answered, becomes a noop on purpose. The runner already answers the confirmation pop-up as part of the click. The sentence stays for the reader of the report.

The devices_picked fact becomes a pick_devices step with the placeholder $serial. The runner replaces the placeholder with the first device from this list:

If the case has a serial, the converter already writes the serial into the step. The model on the form comes from the same device.

The stand’s own device uses the TR-069 protocol. A case can name another protocol, such as the User Services Platform (USP) or Message Queuing Telemetry Transport (MQTT). If such a case has no device of its own, the step fails. The step does not take the stand’s TR-069 device instead.

The step also fails if the dialog does not list the device. For example, a domain user sees only the devices of the domain. If no preflight moves a device into that domain, the dialog has no device for the step. The step never picks the first device in the dialog.

A gate names its screens, its mode (add, edit or any), the control, the facts it requires before that control, the ones that follow it (then — the two confirmations) and the Angular line that proves it. The file is data, not code: a malformed entry is a load error, and the worker then mounts no /api/adapt route at all rather than adapt cases with half a registry.

The rules walk each adapted scenario with state — where the tester stands, whether a form is open, which facts the case has already performed — and insert the missing sentence immediately before the step that reaches the gated control. An inserted step carries no source line and a UF-<fact> change record, which is what exempts it from the invented step check of the gate; the published numbering moves and every step N reference moves with it (R8). A step that hides the work behind an umbrella (Perform all required actions to …) keeps its own line and gets the spelled-out sentences in front of it — the umbrella often names where it goes next, which a replacement would drop. A step that opens a tab and uses that tab’s button in one breath is split in two (UF-split), with the facts between the halves. Over the 3738 source cases this adds 744 sentences to 447 cases and splits 45 steps. A case that already states a fact gets nothing for it, and a second pass over an adapted case adds nothing more.

What the walk cannot settle by itself is a BLOCKER, so the case waits in needs_attention:

Adding a gate. Find the condition in the oneiot_angular checkout. Add the fact — its sentence, the recognisers that spot a case which already does it, and the converter: action the sentence maps to — or reuse an existing one; then add the gate with its screens (keys of generated/portal_screens.json), control, requires / then and angular.evidence as path:line. Then run python scripts/sync_ui_gates.py: it rewrites generated/ui_gates.json, the census of every form-state condition in the checkout, and the new entry’s fingerprint goes into the gate. A condition held in TypeScript is not censused and keeps fingerprint: null — nothing checks such a gate automatically, so it is verified by hand on the portal. python scripts/sync_ui_gates.py --check exits 1 when the vendored census is stale and is due on every frontend bump: a gate whose fingerprint the census no longer carries loads unverified, and the cases that reach it are reported instead of adapted.

The registry, its loader and the census belong to the adapter version, exactly as the curated vocabulary does. Editing any of them makes every stored row stale and the next job over them adapts them again — so batch the registry work and run once. Only a case whose adapted text changes becomes Ready to write, and an administrator writes it.

A case is ready only when no check below finds a problem. The gate never runs the case on a stand.

Every finding lowers the Score — 100, minus 40 per BLOCKER, 10 per MAJOR and 3 per MINOR — and a problem found by a check above is a BLOCKER. The gate also reports findings that do not block, for example a number that the adapted title, preconditions or steps carry and the source does not (in an expected result it blocks), a share of catch-all interact steps in the dry conversion above Fallback warn, a button caption the vocabulary does not know, or a scenario that lost its expected result.

When every BLOCKER comes from the coverage map of the dry conversion — a sentence nothing performs (coverage-uncovered-…), a sentence that needs infrastructure the stand lacks (coverage-blocked-…, infra-blocked-<category>), a map that is not audited (coverage-not-audited) or a step the conversion lost (step-lost-…) — the adapted text itself is clean: the case is automation_blocked, no repair runs, and an administrator can write it. Any other BLOCKER — wording, navigation, values, source coverage, an unmappable element, a conversion that crashed, and a coverage map the gate could not build at all (coverage-unavailable, a failure of the gate itself) — keeps the case in needs_attention.

Some infrastructure is deferred rather than blocking. The converter stamps one category, usp_device, for a USP, MQTT or LwM2M device case, and it is the evidence that names the protocol (USP device, MQTT device, LWM2M device). The IoT emulator every worker carries (mcp-iot-emul) serves USP and MQTT, so for those the only thing missing is the converter’s route to it — and not LwM2M, which the stand does not run at all, so an LwM2M case keeps every BLOCKER and waits in automation_blocked exactly as before. A marker whose evidence names no protocol at all is treated the same way LwM2M is — as hard.

For a USP or MQTT device the infra finding becomes a MAJOR with the recommendation Automation coverage deferred: needs the IoT emulator route in the converter (mcp-iot-emul is on the stand), and so does every coverage finding that the same device is what blocks — the coverage-blocked-… of each sentence it stops and the coverage-uncovered-… that follow. The ids and the refs are unchanged, so the row panel still points at the same sentences; only the severity moves, and a case blocked by nothing else is therefore ready. The decision is per sentence wherever the coverage map records which reason blocked it, and per case otherwise: one reason the stand really lacks (file_server, license, oracle_db, an LwM2M device, …) keeps that sentence’s BLOCKER and leaves the case in automation_blocked. A wording, navigation, value or source-coverage BLOCKER is never deferred, so a case carrying one is still needs_attention.

When the dry conversion cannot reach a service it needs, there is no verdict: the row is stored as error with that reason, no repair runs, and the next job tries the case again.

A legacy control the gate reports as unknown wording is cleared by curating it in scripts/adapter/vocab/legacy_management.yaml, in one of two shapes. When the Angular portal has the control under another name, a labels: entry gives the name — always scoped to the element word the cases actually write (button, link, section, drop-down, checkbox, field, tab, left menu), because the value check trusts a curated rename only when its entry names an element kind: a rename of kind any on a step, a precondition or an expected result merely turns the wording finding into a Values one. A rename never contradicts the element word the source wrote either: where the only kind that could fire would publish a picker as a link or a screen as a page, the label is left uncurated and its cases wait for a human, because nothing in the table can rewrite that word. When the control is genuinely gone, a removed: entry with a one-sentence note naming the Angular source that proves it sends the case to a human with the evidence instead of a bare "no Angular name for it" — but the note’s phrase is matched over the whole source text, prose included, so a phrase also found in a case with a clean verdict today would block that case too and belongs in a report to QA rather than in the file. Either entry changes the vocabulary digest, hence the adapter version, so every stored row is adapted again by the next job: batch the vocabulary work and run once.

The last job line of the Status panel counts each case a refresh left for the model as needs model. The progress row of the job shows the same count as a needs model pill.

Under the form the tab shows the job’s status pill, <done> / <total> · now C<id>, a pill per row status, tokens <in> / <out> · billed <n> · cache hits <n>, a progress bar and the job’s error lines. The Cases list refreshes as cases finish, and the case in progress carries a blue running pill. When the job ends, a toast gives its status and counts.

Stop is shown to write roles while the job runs. It answers The job is not running on this worker when the job belongs to another worker.

To carry on after a pause or a stop, start a job over the same selection. Unchanged cases that already have a row are reused at no token cost; error rows and the cases the earlier job never reached are adapted. A case TestRail could not return leaves no row at all: the job counts it as an error and names it in its error lines.

Only one adaptation job runs at a time across all workers, and Re-run with engine counts as a job. The slot is taken in the database the workers share, so a second Start, on any worker, is refused with a message that names the holder: an adaptation job is already running: job <id> on worker <host>:<pid>.

A job’s live progress is known only to the worker that runs it. When a job holds the slot and the worker serving your session is not running it, the tab shows the slot instead: an amber job slot held pill and Held by <worker> (job <id>) since <time>, last heartbeat <time>. A running job refreshes its heartbeat every minute. The tab reads the slot when it opens, on Reload and after its own actions — not continuously.

A worker that crashed or was redeployed in the middle of a job leaves the slot taken; the slot frees itself 30 minutes after the last heartbeat. To free it sooner, a write role presses Clear stuck claim and confirms.

The header line names the source project, its case count and the time of the last read. The Reload button of the header reads the tab again. The Status panel sits under the header.

The worker reads the source suite once an hour. Before the first read, Not adapted shows —. The header then says case list not read yet.

An orphan is an adaptation row without a case in the source suite. The summary names the orphans when there are any.

The Sections table lists every source section in the TestRail tree order. Each line shows the total, one column per to-do status and the Done count. The to-do columns include In progress. The All sections line gives the totals. Click a section to filter Cases. Click it again to show every section.

Cases lists every case of the source suite, one row per case. The worker filters and sorts the list. Every filter covers the whole suite.

Filters:

Case, Section, Status, Verdict, Score and Updated sort on a click. A second click reverses the order. A page holds 50, 100 or 200 rows. The browser keeps the filters, the sort and the page.

Click a row to open the case detail. A Not adapted case has no detail. Its id links to TestRail. The links use the TestRail URL set on the worker of your session.

Export CSV downloads every row of the current filter. The columns are id, section, title, status, reason, verdict, change, project 15 state, hold reason, updated and stored status.

A checkbox selects a row. The header checkbox selects the page. Select all N cases matching the filter selects every page. A selection survives a page change and a filter change.

The bulk bar shows only the actions allowed for the selected statuses. Each button counts the cases it takes. Every action reads the selection again first.

Before a job starts, a dialog states the case count, the engines and the upper bound of model calls. The upper bound is the case count multiplied by one more than Max repairs. The dialog also counts the cases with a hand edit. A new adaptation replaces that edit.

Adapt / Re-adapt, Re-adapt with model and Write to TestRail wait while an adaptation job runs. Write to TestRail also waits while in-place writes are off. The button tooltip says why. Hold… and Release hold work during a job.

After a bulk write, the bar lists each case with its outcome. When a request fails, the toast names the number of cases sent before it.

The detail header shows the status pill and its reason line. Small text gives the verdict, the change, the engines and the tokens.

The detail shows the source case as TestRail has it now (Source (TestRail)), the adaptation as JSON (Adapted (editable JSON), read-only for viewers), Findings (N) with severity, id, message and recommendation, a line Dry conversion: <n> steps, <m> fallback (<p>%), step types: …, and the number of rule changes. Closing it with an unsaved edit asks Discard the edited adaptation?.

The JSON has this shape:

Re-gate (write roles) stores the edit and runs the gate again without a model: the row becomes ready, automation_blocked, needs_attention or error, and a toast shows the status and the score. The edit must be valid JSON in the shape above, checked the way a model’s answer is: a source_scenario outside the source’s scenarios, source_lines that are not a list of line numbers, a step without text, or preconditions / unmappable / notes that are not lists is refused with the reason, and nothing is stored.

Re-gate marks the case as edited by hand. A job that adapts the case again clears the mark. A new adaptation replaces your edit. Re-run with engine asks you to confirm before it replaces an edited case.

Re-run with engine (write roles) adapts the case again as a one-case job, with the engine picked beside the button as its first pass; its repairs use the Repair engine and Max repairs the worker has from Settings. The stored row changes only when the re-run finishes, and it is refused while an adaptation job holds the slot. A case the rules adapt completely never reaches a model, so its re-run applies the rules and the gate again, whatever engine is picked.

Hold and Release hold sit in the case detail for an administrator. Hold needs a reason and applies to cases still to do. Release hold shows for any held case. The writer refuses a held case.

A Written case stays Written after a later re-adaptation. When the new text is clean, the detail says a newer adaptation waits. In the Project 15 (in place) panel of the detail, press Preview in project 15, then Write to TestRail. When the re-adaptation fails, the detail shows the reason line and a note. A plain job would adapt the text now in TestRail. That text is the written text. The detail hides Re-run with engine for a Written case, and the server refuses it. An administrator presses Re-adapt from the original instead. The job adapts the saved original text. The button waits while a job runs. A restored case can still use Re-run with engine.

An administrator can name at most 200 cases in one Re-adapt from the original request. The bulk bar has no such action.