Converter Tab

The converter converts a TestRail case into a case file. The test runner executes the case file. The Converter tab shows the work of the converter on this stand.

A conversion has several stages. In Stage 2, a large language model (LLM) adds details to the steps. Stage 2 spends LLM quota.

The converter stores each conversion in the conversion cache. If the case and the import options stay the same, a later import reuses the cached conversion. The reuse skips Stage 2.

What the Converter Tab Shows

To open the tab, click Converter in the sidebar.

The tab has three views:

  • Overview: the state of the stand and the import buttons.

  • Journal: the import jobs and the result of each job for each case.

  • Compare: a TestRail run or suite compared with the stand, case by case.

Each import is a job. A sync or a re-import is also a job. The journal lists the jobs of every worker. A worker is one server of the test runner.

When you open the tab, the page reads the state of the stand. If an import runs on any worker, the Converter item in the sidebar shows live. While the import runs, the tab updates itself every few seconds. On other tabs, the live mark updates once a minute. To see a new import, open the tab again or click Refresh.

Check the Stand

The Overview shows these counters:

Counter Meaning

Case files on the stand

The case files of TestRail cases on the stand. Each scenario has its own file: C5 and C5-2 count as two. Copies and cases without a TestRail case id do not count.

Stale conversions

The case files from an older converter version. Re-import these files to apply the current converter. See Re-import Stale Cases.

Of them never stamped

Stale case files without a converter version. An old converter did not write the version into the file. These files count in Stale conversions too.

Blocked by the converter

The case files with a blocked mark. The runner does not start a blocked case on this stand. The reason is missing infrastructure or a known product bug. To run such a case anyway, click Run anyway (force) in the report of the blocked run. This count differs from Cannot run in the Readiness tab. Readiness builds that label from gap codes and run results.

Converter version

The version of the current converter. A case file with an older version counts as stale.

Default LLM mode on this stand

The LLM mode of a job without its own choice, for example an automatic sync. The value is one of the values in LLM Modes.

Last TestRail sync

The time of the last sync. A sync without changed cases also counts. An import of a suite through Import from TestRail… also updates this time. no sync target yet means the sync has no suite. See Sync Without a Target.

TestRail cases the sync knows

The number of TestRail cases with an import record. The sync uses these records to find changed cases. A case counts once, whatever the number of its files.

Auto-sync on this worker

The interval of the automatic sync, or off. Each worker has its own interval. One worker answers the page. The counter shows the interval of this worker. An administrator sets the interval in Settings, in Auto-sync Interval (minutes, 0 = off).

Under the counters, the Overview shows one line for each running import. The line shows:

  • the job id

  • the source of the job

  • the account and the worker

  • the number of cases done and the total

  • the last case

Import Cases

The import buttons show only for an account with the admin or editor role. A viewer account sees the counters and the journal only.

A worker runs one import at a time. If another import runs on the worker, the new import does not start. The error message names the running job.

Import from TestRail

  1. On the Overview, click Import from TestRail….

    The Import from TestRail dialog opens.

  2. Import the cases with the dialog. For the dialog steps, see TestRail Modal.

  3. Close the dialog.

    The Overview updates. A running job shows as a line.

The sync follows the suite of the last Import from TestRail…. See Sync Without a Target.

Import a TestRail XML Export

TestRail exports cases as an Extensible Markup Language (XML) file.

The XML file only names the cases. The import reads the content of each case from TestRail. If TestRail no longer has a case from the file, the job lists the case as an error.

  1. On the Overview, click Import TestRail XML….

    The browser opens a file dialog.

  2. Choose the XML export of TestRail.

    The Import test cases dialog opens.

    Overwrite existing cases replaces the case files. You lose your manual edits in the Editor.

  3. Choose Skip existing cases (default) or Overwrite existing cases.

  4. In LLM mode, choose the LLM mode. See LLM Modes.

    The default is No LLM, cached only. In this mode, the import skips a case without a cached conversion. A note in the dialog says so. The mode applies only to a TestRail export.

  5. Click Import.

    The job starts in the background. A message shows the number of cases. The Overview shows a line for the job. At the end of the job, a second message shows the result.

The XML import needs a working TestRail connection. If the suite name in the file differs from the suite name in TestRail, the import does not start. The error message asks for an export from this TestRail. The XML import does not change the suite of the sync. The same import is also in Test cases > Library > Import XML….

Sync with TestRail

The sync reads the cases of its suite from TestRail. To give the sync a suite, see Sync Without a Target. The sync re-imports only the cases changed in TestRail since their import. A case without an import record is new. The sync only counts the new cases.

  1. On the Overview, click Sync with TestRail now.

    The page counts the changed cases in TestRail. If nothing changed, a message says so. No job starts.

  2. If cases changed, read the confirmation dialog.

    The dialog shows these values:

    • the number of changed cases

    • the LLM mode list. The default is With LLM (full).

    • the model

    • the number of new cases, if TestRail has new cases

  3. In LLM mode, choose the LLM mode.

    In No LLM, cached only mode, the sync skips every changed case. A changed case has no cached conversion.

    The sync overwrites the case files of the changed cases. You lose your manual edits in the Editor on these cases.

  4. Click Sync.

    The sync starts a job for the changed cases. The Overview shows a line for the job. If more cases changed after the count, no job starts. The dialog opens again with the new number.

  5. If the dialog opens again, confirm the new number with Sync.

    The dialog keeps your choice in LLM mode. The sync starts a job for the changed cases.

To close the dialog without a sync, click Cancel. To import the new cases, use Import from TestRail….

Automatic Sync

The automatic sync runs on a schedule. The Auto-sync on this worker counter shows the interval.

Nobody confirms an automatic sync. For this reason, an automatic sync converts at most 10 changed cases. If more cases changed, the automatic sync converts nothing. The Journal then shows an auto-sync job with the status error. The error gives the number of changed cases.

To convert these cases, use Sync with TestRail now. The dialog asks you to confirm the number. Until then, the automatic sync checks TestRail again at each interval. The Journal does not repeat the same error job. A new number of changed cases adds a new error job. After a sync through Sync with TestRail now, the next error also adds a new job.

Re-import Stale Cases

Re-import the stale cases after a converter update. While Stale conversions shows 0, Reimport stale… stays disabled.

The Reimport Selected Tests dialog opens with No LLM, cached only in LLM. Force refresh (bypass the conversion cache) is selected.

  1. On the Overview, click Reimport stale….

    The page reads the case list again. Then the Reimport Selected Tests dialog opens. The dialog lists each stale case once, by its TestRail case id.

    A note warns about the cached-only mode. After a converter update, stale cases usually have no cached conversion. In No LLM, cached only mode, the re-import skips them.

  2. In LLM, choose the LLM mode. See LLM Modes.

    The line under LLM shows these values:

    • the number of cases

    • the LLM mode

    • the model

    The model comes from Model in the Import from TestRail dialog.

  3. Keep or clear Force refresh (bypass the conversion cache). See Force Refresh.

    In No LLM, cached only mode, the option is disabled.

    A re-import overwrites the case files with a fresh conversion. You lose your manual edits in the Editor.

  4. Click Reimport.

    The dialog shows the progress. At the end, the dialog shows:

    • the number of re-imported cases

    • one line for each error

    • one line for each change

    • in No LLM, cached only mode, the number of skipped cases

The re-import runs on the server. If you close the dialog or the browser, the re-import continues. If you close only the dialog, a message shows the result at the end. The Journal lists the job under the source reimport.

If the page cannot read the state of a job, the dialog shows Status unknown. The job can still run on the server. Check the job in the Journal.

One re-import job takes at most 1000 cases. The dialog splits a longer list into chunks of 1000 cases. Each chunk is a job of its own. The chunks run one after another. The progress line names the chunk, for example Chunk 2 of 3.

A chunk fails in these cases:

  • The job of the chunk does not start.

  • The job of the chunk stops with an error.

  • The page cannot read the state of the job.

After a failed chunk, the next chunks do not start. The dialog names the failed chunk and the number of cases not started.

The dialog skips stale files without a TestRail case id. If no stale case has a TestRail case id, the dialog shows No stale case with a TestRail id.

LLM Modes

Option in LLM Value Result

With LLM (full)

full

The full conversion. Stage 2 spends LLM quota for each case.

No LLM

off

A conversion without Stage 2. The job overwrites the file of each case. The case loses its Stage 2 details.

No LLM, cached only

cache_only

Cached conversions only. The job skips a case without a cached conversion. The file of that case stays unchanged. Force refresh (bypass the conversion cache) has no effect in this mode.

Each dialog has its own default. The sync starts with With LLM (full). The other dialogs start with No LLM, cached only:

  • Import from TestRail…

  • Import TestRail XML…

  • Reimport Test and Reimport Selected Tests

  • the import confirmation in Compare

In the Import from TestRail dialog, Import single test case uses the LLM mode in LLM. In No LLM, cached only mode, an import skips a case without a cached conversion. A case never converted before has no cached conversion. To convert such a case, choose No LLM or With LLM (full).

Force Refresh

The option Force refresh (bypass the conversion cache) decides how the re-import uses the conversion cache.

Option Result

On

Except in No LLM, cached only mode, the converter ignores the conversion cache. In With LLM (full) mode, Stage 2 runs again and spends LLM quota.

Off

If the cache has a conversion of the case, the converter replays the cached conversion. The replay updates only these parts of the case file:

  • the blocked mark

  • the project and the run labels

  • the preconditions from the TestRail text

  • the map from TestRail sentences to steps

Other converter changes do not apply to the case file.

A new converter version makes the old cache entries unusable. Most stale cases then have no cached conversion. For these cases, the option has no effect:

  • In With LLM (full) mode, Stage 2 runs and spends LLM quota.

  • In No LLM mode, the converter converts the case without Stage 2.

  • In No LLM, cached only mode, the job skips the case.

Compare with TestRail

The Compare view compares a TestRail run with the stand. The view shows the state of each case of the run. You can also compare a TestRail suite.

  1. On the Converter tab, click Compare.

  2. In the source list, choose TestRail run or TestRail suite.

  3. Enter the run id or the suite id, for example 319.

  4. Click Compare next to the id.

    The page reads the cases from TestRail. The line above the counters shows:

    • the run or the suite, with its name

    • the project and the suite

    • the time of the TestRail read

If TestRail does not answer, the page shows Could not compare with TestRail and the error.

The page remembers the last compared run or suite in this browser. On your next visit, the view compares this source at once.

The server keeps the TestRail answer for 5 minutes. To read TestRail again, click Refresh from TestRail. The page reads the state of the stand again at these moments:

  • You open the Compare view or the Converter tab.

  • You click Refresh at the top of the tab.

  • An import from this view ends.

  • You close the Reimport Selected Tests dialog. If the re-import still runs, the page also reads the stand at the end of the re-import.

Case States

At conversion, the converter stores a hash of the TestRail text in the case file. The page compares this text hash with the current TestRail text.

Each case has one state. The page checks the states in the order of this table. The first match gives the state of the case.

State Meaning

Not imported

The stand has no case file of the case. A copy of the case does not count. Detail shows the result of the last import, or no import recorded. If the source case of an adapted copy is on the stand, the copy also has this state. See No Duplicate Imports.

Changed in TestRail

The TestRail text differs from the converted text. For a case file without a text hash, TestRail updated the case after its import.

Cannot tell

The case file has no text hash. The case also has no import record. Re-import the case once. Then the page can compare the case.

Old converter

The TestRail text has no change. An older converter version converted the case.

Current

The case file matches the TestRail text and the current converter.

The counters above the table give the number of cases in each state. All cases counts every case of the run or the suite. Blocked counts the cases with the blocked tag. Duplicate counts the cases with the copy of C<id> tag or the copies tag.

To show only the cases of one counter, click the counter. To show every case again, click All cases. To narrow the list, type an id or a title in Filter by id or title.

The table shows at most 500 cases. If more cases match, a line under the table says so.

Column Meaning

Case

The TestRail case id. To see the history of the case, click the id.

Title

The title of the case.

State

The state of the case.

Detail

The reason for the state. The cell is empty for a Current case.

Tags

More facts about the case. See the tag list below.

The first column selects the case. The last column has the link Open. To open the case file in the Editor, click Open.

The Tags column shows these tags:

  • blocked: the case file has a blocked mark. The tag names the reasons in parentheses.

  • copy of C<id>: the case is an adapted copy. Its source case is on the stand.

  • copies: the stand has a copy of the case. The tag names the files of the copies. Duplicate in the More menu of the Editor makes such a copy.

  • old converter: an older converter version converted the case. This tag shows only for a case in another state, for example Changed in TestRail.

The page does not delete a copy. To remove a copy from the stand, open the copy in the Editor and click Delete in the More menu.

Import Missing Cases

The Import selected (N)… and Re-import selected (N)… buttons show only for an account with the admin or editor role. Each button shows the number of its selected cases in place of N.

A worker runs one import at a time. See Import Cases.

  1. Select the Not imported cases.

    To select each shown case for an import or a re-import, click Select shown. To clear the selection, click Clear selection.

    You cannot select a Current case. You cannot select an adapted copy of a case on the stand. The stand already has its source case.

  2. Click Import selected (N)….

    The confirmation dialog opens. The dialog shows the number of cases, their ids and the model. The dialog shows at most 20 ids. The ids include the selected cases outside the filter.

    If you select an adapted copy and its source case, the dialog names the copy. The import skips the copy. See No Duplicate Imports.

  3. In LLM mode, choose the LLM mode. See LLM Modes.

    The default is No LLM, cached only. In this mode, the import skips a case without a cached conversion. A case never converted before has none. To convert such a case, choose No LLM or With LLM (full). In With LLM (full) mode, Stage 2 can run for each case.

  4. Click Import.

    The import starts as a job. The Overview shows a line for the job. At the end of the job, the comparison updates.

The import keeps existing case files. The import does not change the suite of the sync or the Last TestRail sync time.

Re-import Changed Cases

You can re-import the cases in these states:

  • Changed in TestRail

  • Cannot tell

  • Old converter

A re-import overwrites the case files. You lose your manual edits in the Editor.

  1. Select the cases.

  2. Click Re-import selected (N)….

    The Reimport Selected Tests dialog opens. The dialog lists each case with its case file. LLM shows No LLM, cached only. Force refresh (bypass the conversion cache) is not selected. In With LLM (full) mode, the re-import then replays a cached conversion if the cache has one. See Force Refresh.

  3. Read the note in the dialog.

    In No LLM, cached only mode, the re-import skips a changed case. It also skips a case of an older converter: a case in the Old converter state or with the old converter tag. These cases have no cached conversion. The note gives their number. The re-import can skip more cases. For example, a Cannot tell case can have no cached conversion.

    The note also gives the maximum number of Stage 2 runs in With LLM (full) mode.

  4. To convert these cases, choose No LLM or With LLM (full) in LLM.

  5. Click Reimport.

    The re-import starts. When you close the dialog, the comparison updates. If the re-import still runs at that moment, the comparison updates again at the end of the re-import.

For the rest of the dialog, see Re-import Stale Cases.

Cases Outside the Run

For a run, the view can list more cases under the table. Each list shows only if it has cases.

  • On the stand, not in the run: the case file names the run. TestRail no longer lists the case in the run.

  • Pinned to the run on the stand: a person added the run label on the stand. TestRail does not list the case in the run.

  • In the run, not in its suite: TestRail lists the case in the run. The suite of the run no longer holds the case.

In the first two lists, click an id to open the case file in the Editor. The page does not delete these cases. To delete a case, open the case in the Editor.

No Duplicate Imports

An adapted copy is a TestRail case with a value in the Adapted from field. The field names the source case of the copy. See Adapting Legacy Cases.

If the source case is on the stand, the import skips the adapted copy. The job lists the copy as skipped. The reason is duplicate of C<id>. The rule applies to these imports:

  • Import from TestRail… and an import from the Compare view

  • the XML import

  • the sync

  • a re-import of a case without case files

If one job gets a case and its adapted copy, the job imports only the source case.

Except for the re-import, these imports also remove a skipped copy from TestRail cases the sync knows. The sync then counts the copy as a new case in TestRail. The sync does not import new cases. See Sync with TestRail.

If the copy has case files on the stand, the rule does not apply. The import converts the copy as usual.

Read the Journal

  1. On the Converter tab, click Journal.

    The table lists the newest 50 jobs of the last three months. The table includes the jobs of every worker.

To show one kind of job, choose a source in the source list. All sources shows every job.

Source Job

testrail

An import through Import from TestRail…, or an import from the Compare view

xml

An import of a TestRail XML export

sync

A sync through Sync with TestRail now

auto-sync

An automatic sync. An automatic sync with more than 10 changed cases is a job with the status error. See Automatic Sync.

reimport

A re-import from a user, for example through Reimport stale…

auto-reimport

A re-import of stale cases at the start of a worker. This job uses cached conversions only. This job is off by default. An administrator enables it for the whole deployment. Settings has no option for this job.

Job Table

Column Meaning

Started

The start time of the job.

Source

The kind of job. See the source table above.

By

The account of the job.

Status

The state of the job. See the status table below.

Cases

The number of cases in the job.

Created

The number of written case files. A new file and a replaced file both count. Each scenario has its own file. The number can therefore exceed Cases. A re-import counts each re-imported case once.

Skipped

The cases without a change to their case file.

Errors

The cases with an error.

LLM mode

The LLM mode of the job. An automatic sync uses the Default LLM mode on this stand.
Status Meaning

done

The job ended. Check Errors for failed cases.

running

The job runs now. The cell shows the cases done and the total, for example running 5/12.

error

The job stopped with an error. The job detail shows the error.

lost

The job sent no progress for 15 minutes. For example, its worker restarted during the job. The journal has no case results for a lost job.

The journal keeps the jobs of the last 12 months.

Job Detail

  1. Click a row of the job table.

    The job detail opens under the table.

The first line of the job detail shows:

  • the account and the worker of the job

  • the start time and the finish time

  • the LLM mode and the model

  • force refresh or cache allowed

  • the billed LLM tokens, if the job used the LLM

  • the error, if the job stopped with one

While the job runs, the job detail shows Results appear when the job ends. The job writes its case results at its end.

The job detail has one row for each case:

Column Meaning

Case

The id of the case or scenario. Click the id to see the history of the case.

Status

The result of the job for the case:

  • created: the job wrote the case file. The file is new or replaces the old file.

  • reimported: a re-import converted the case again. The re-import replaced the case file.

  • exists: the case file exists. The job kept the file.

  • skipped: the job did not convert the case. Detail gives the reason.

  • error: the conversion failed. Detail gives the error.

Detail

The reason for a skip or an error. For a re-import, the changes. Otherwise, the names of the case files.

Cache

The use of the conversion cache:

  • hit: the job reused a cached conversion.

  • miss: the cache had no conversion for the case.

  • bypassed: Force refresh was on.

Stage 2

ran, or skipped with the reason. Semicolons separate several entries.

Tree

The source of the device tree file, and the file name:

  • attached: the tree file from the TestRail case.

  • template: a built-in template tree. The case has no valid attached tree.

  • switched: the converter moved the case to another tree.

  • cached: the tree of the cached conversion.

  • attached <file> rejected: the attached tree file was not valid. The converter used another tree.

Seconds

The conversion time of the case, in seconds.

To close the job detail, click Close.

Sync Without a Target

The sync follows the suite of the last Import from TestRail…. Without such an import, the sync has no suite. Then Sync with TestRail now stays disabled. Last TestRail sync shows no sync target yet. The automatic sync does nothing.

To give the sync a suite:

  1. On the Overview, click Import from TestRail….

  2. Choose the Project.

  3. Choose the Suite for the sync.

  4. Click Preview.

  5. Select at least one case.

    The Import N cases button shows the number of selected cases in place of N.

  6. Click Import N cases.

    The import starts. At the end, the dialog shows the result.

  7. Click Done.

    Sync with TestRail now becomes available.

An import through Import TestRail run also gives the sync a suite. Import single test case does not change the suite. An XML import does not change the suite.

While a sync runs, Sync with TestRail now also stays disabled. Only one sync runs at a time on all workers. The line of the running sync on the Overview shows its progress.

See the History of One Case

The history of a case lists its imports and its edits in the Editor. The newest entry comes first. The history covers every scenario of the case and 12 months of the journal.

To open the history from the journal:

  1. On the Converter tab, click Journal.

  2. Type the TestRail case id in the case id box, for example C12155.

    The box takes a case without a scenario number: C5, not C5-2.

  3. Click Case history.

    The history of the case opens under the table.

To open the history from the Editor:

  1. In the Editor, open the case.

  2. Click More.

  3. Click Converter history.

    The Converter tab opens in the Journal view with the history of the case.

Converter history stays disabled for a case without a TestRail case id. More shows only for an account with the admin or editor role.

The history has these columns:

Column Meaning

Time

The time of the entry.

What

For an import, the source and the case status, for example reimport: reimported. For an edit, the kind of edit, for example edited.

By

The account of the entry.

Detail

The reason for a skip or an error. Otherwise, the changes or the names of the case files.