Installation & Deployment
- 1. Overview
- 2. Prerequisites
- 3. Third-Party Components
- 4. Network Requirements
- 5. Registry Authentication
- 6. Preparation
- 7. Deployment
- 8. Verification
- 9. Port Reference
- 10. Stack Management
- 11. Production Checklist
- 12. Troubleshooting
- 12.1. Container Fails to Start
- 12.2. Live Browser Is Blank
- 12.3. Chromium Crashes or Steps Time Out at Random
- 12.4. Every Proxied Call Returns "Login Required"
- 12.5. Runs Hang on "[backend-sync] Timeout"
- 12.6. TestRail Import Does Nothing
- 12.7. Imports and Interact Steps Fail Immediately
- 12.8. Port Already in Use
- 12.9. Getting Support
- 13. Related Documentation
1. Overview
The AI Test Agent is not a single container. It is a small set of services, and which of them you deploy depends on the topology:
| Service | Role | Required |
|---|---|---|
|
The agent itself: TestRail import, the Smart Converter, YAML case execution, the web UI and the REST API. It also runs Playwright MCP and a Chromium under Xvfb inside its own container, exposed over noVNC. |
Yes |
|
MCP sidecar that applies CPE parameters to the TR-069 emulator and triggers Inform. One instance per environment under test. |
Yes |
|
MCP sidecar that drives the IoT device emulator (USP/TR-369, LwM2M, MQTT).
Environment agnostic — the runner passes the target environment’s
|
Yes |
|
MCP sidecar that generates TR-181 data-model trees for imported cases. Environment agnostic — one shared instance is enough. |
Yes |
|
Single public entrypoint in a multi-stand deployment: authentication, roles, env-per-run routing, worker scheduling and the proxied noVNC view. |
Multi-stand only |
Two topologies are supported, and they are the two compose files in this repository:
Single stack (test_runner/docker-compose.yml) — one runner with its two sidecars, published directly on the host.
This is the evaluation and single-environment install: everyone shares one runner, one emulator and one Chromium, and retargeting means editing Settings before a run.
Worker pool (docker-compose.gateway.yml + docker-compose.stand.template.yml) — a gateway in front of N env-agnostic workers, each bundling its own emulator MCP sidecars.
This is what the production endpoint runs. The gateway pushes the chosen environment’s settings onto a free worker just before dispatch, so two environments can run at once.
The routing model, the scheduling rules and the auto-provisioning behaviour are documented in Environments & the Gateway; this page covers only what it takes to get those containers running.
Browser -> gateway (:8083) -> worker (test-runner) -> mcp-tr-emul -> TR-069 emulator / mcp-iot-emul -> IoT emulator
| |
+--> Chromium/Playwright --> portals +--> ACS
Docker Compose is the only supported deployment method — there is no bare-metal distribution. Portainer is used on the production host to manage those same compose stacks, and the gateway can create worker stacks through its API.
The environment under test — FTACS, the TR-069 emulator, the management and support portals, ClickHouse and the platform database — is assumed to be already deployed and reachable. This guide only points the agent at it.
2. Prerequisites
2.1. Host Requirements
| Component | Minimum | Recommended | Notes |
|---|---|---|---|
Docker Engine |
20.10 |
Latest stable |
Docker Compose is the only supported deployment method. |
Docker Compose |
2.0 |
Latest stable |
This guide uses the |
CPU |
verify with the infrastructure owner |
— |
No figure is published for this service. Size it per worker, not per host: each runner container drives its own Chromium, so a pool of N workers means N concurrent browsers. |
RAM |
verify with the infrastructure owner |
— |
Same caveat. The hard requirement is |
Shared memory |
2 GB per runner |
— |
|
Browser |
Latest stable Chrome, Edge or Firefox |
— |
For the agent’s own web UI. Not a requirement on the container host — the Chromium the tests drive ships inside the runner image. |
2.2. Required External Dependencies
These must be running and reachable before the containers start. None of them are deployed by this guide.
| Component | Why It Is Needed | Port | Required |
|---|---|---|---|
Logged-in Claude Code CLI on the host |
The runner shells out to the Claude CLI for converter enrichment and
|
— |
Yes |
TR-069 emulator of the target environment |
|
per environment |
Yes |
IoT emulator of the target environment |
|
per environment |
Yes |
FTACS |
The ACS the emulated devices register against. Use the |
80/8080 (HTTP) |
Yes |
Management / support portals |
What the tests actually drive in the browser, plus the portal backend the
preflight actions call ( |
per environment |
Yes |
ClickHouse |
Read back when a case verifies dashboard KPI values, and seeded before interference runs. HTTP port — the same one the platform uses. |
8123 (HTTP) |
No (feature-level) |
Platform database (MySQL or Oracle) |
State reads during verification. Set |
3306 / 1521 (TCP) |
No (feature-level) |
Knowledge MCP / Qdrant |
Backend RAG context for the converter. |
7030, 8095 (TCP) |
No (feature-level) |
TestRail |
Case import and sync. Left unconfigured, cases can still be imported from XML. |
443 (HTTPS) |
No (feature-level) |
Portainer |
Optional: manages the stacks, and lets the gateway provision workers and the runner UI restart sibling containers. Inert when its credentials are unset. |
9443 (HTTPS) |
No (optional) |
2.3. Supported Operating Systems
| Deployment | Operating system |
|---|---|
Docker Compose |
Linux (recommended), macOS, or Windows with WSL2 |
2.4. Registry Access
Network access to the hub.friendly-tech.com Docker registry, which publishes all four images:
| Service | Image |
|---|---|
|
|
|
|
|
|
|
|
Access to the deployment files as well: the compose file for your topology, and the environment values described in Environment Configuration.
3. Third-Party Components
Every third-party product this service depends on, with the version it is built or verified against.
Values are read off this repository — test_runner/Dockerfile, gateway_service/Dockerfile, the compose files and the requirements files — except the platform minimums at the bottom, which belong to the environment under test rather than to anything this repository pins.
The table below is this service’s own view. For the platform-wide picture — which third-party product version each published release and hot-fix was built against, across every FT service — see FT Version Control.
| Component | Technology / Version | Purpose |
|---|---|---|
Container runtime |
Docker Engine, Docker Compose — see Host Requirements for version minimums |
The only supported deployment method |
Runner base image |
|
Runtime image of the test-runner ( |
Gateway base image |
|
Runtime image of the gateway ( |
Claude Code CLI |
|
Converter enrichment and interact-step reasoning. Pinned deliberately — an unpinned global install sticks in the build cache and changes CLI behaviour between rebuilds |
Claude models |
|
Resolved from the |
Playwright MCP |
|
Deterministic browser control. The version is set once in
|
Browser |
Chromium + |
What the tests actually drive, headfully under Xvfb, exposed over noVNC |
TR-069 emulator MCP |
|
Applies CPE parameters, triggers Inform. One per environment |
BBF tree MCP |
|
TR-181 tree generation. One shared instance |
Web framework |
FastAPI >= 0.109.0 on Uvicorn >= 0.27.0 |
Runner API and gateway |
HTTP / WebSocket clients |
httpx >= 0.27.0, requests >= 2.31.0, websockets >= 12 |
MCP calls, upstream proxying, noVNC relay |
Vector store client |
qdrant-client >= 1.12.0 |
Backend RAG context for the converter |
Database drivers |
PyMySQL >= 1.1, oracledb >= 2.0 |
Platform state reads. Pure-python — |
SSH client |
paramiko >= 3.4 |
Post-test log error checks on the stand’s log server |
Portainer |
any release exposing API v2 |
Optional stack management and worker provisioning |
ClickHouse |
26.2 (platform minimum), HTTP on 8123 |
Dashboard KPI data read back during verification. Not deployed here |
MySQL or Oracle |
MySQL 8.4 / Oracle 21c (platform minimums) |
Platform state reads. Not deployed here |
4. Network Requirements
Connections the agent opens outward. Each row must be open through firewalls when the peer is on another host.
| Destination | Port | Protocol | Purpose |
|---|---|---|---|
FTACS |
80 / 8080 |
HTTP |
Device registration and ACS parameter operations ( |
TR-069 emulator |
per environment |
HTTP |
Driven through |
Management / support portals |
per environment |
HTTP/HTTPS |
What the browser drives, plus preflight REST calls ( |
ClickHouse |
8123 |
HTTP |
KPI read-back and interference dashboard seeding ( |
Platform database |
3306 (Oracle: 1521) |
TCP |
State reads ( |
Knowledge MCP |
7030 |
HTTP / SSE |
Prompt context ( |
Qdrant MCP |
8095 |
HTTP |
Vector search for converter enrichment ( |
TestRail |
443 |
HTTPS |
Case import and sync ( |
Anthropic API |
443 |
HTTPS |
The Claude CLI’s own upstream |
Portainer |
9443 |
HTTPS |
Stack management and worker provisioning ( |
Log server |
22 |
SSH |
Post-test log error checks ( |
Inbound traffic — browsers and API clients — arrives on the published port documented in Port Reference.
4.1. Docker Networking
Both topologies put the containers on a bridge network the services address by container name: a worker reaches its sidecar at http://mcp-tr-emul:8082, the gateway reaches a worker at http://test-runner-<name>:8083.
That traffic never leaves the host and needs no firewall rule.
In the worker-pool topology the network is test-runner-net and it is external — shared by the gateway stack and every worker stack, so it has to exist before any of them start:
docker network create test-runner-net
Peers outside Docker on the same host are reached through host.docker.internal, which the single-stack compose file maps explicitly:
extra_hosts:
- "host.docker.internal:host-gateway"
5. Registry Authentication
Authenticate once per host before the first pull.
docker login hub.friendly-tech.com
Enter the read-only pull credentials when prompted:
| Field | Value |
|---|---|
Username |
|
Password |
|
The readonly account provides pull-only access to the published images. It cannot push.
|
Non-interactively:
echo "fokxuw-fymte1-taSxyc" | docker login hub.friendly-tech.com -u readonly --password-stdin
5.1. Offline Servers
When the host cannot reach hub.friendly-tech.com, pull on a connected machine, export, transfer, and load.
Note that the runner image bundles Chromium and the Claude CLI, so it is large — budget accordingly.
An explicit --platform matching the offline host’s architecture is required; without it docker pull picks the connected machine’s architecture.
PLATFORM=linux/amd64
docker pull --platform "$PLATFORM" hub.friendly-tech.com/ai/test-runner-agent:latest
docker pull --platform "$PLATFORM" hub.friendly-tech.com/ai/test-runner-gateway:latest
docker pull --platform "$PLATFORM" hub.friendly-tech.com/mcp/tr069-emulator:1.0.7
docker pull --platform "$PLATFORM" hub.friendly-tech.com/mcp/bbf-tree-agent:1.4.0
docker save \
hub.friendly-tech.com/ai/test-runner-agent:latest \
hub.friendly-tech.com/ai/test-runner-gateway:latest \
hub.friendly-tech.com/mcp/tr069-emulator:1.0.7 \
hub.friendly-tech.com/mcp/bbf-tree-agent:1.4.0 \
| gzip > ft-ai-test-agent-images.tar.gz
|
On Windows, always write the archive with |
On the offline host:
gzip -dc ft-ai-test-agent-images.tar.gz | docker load
docker images | grep friendly-tech
The runner still needs outbound HTTPS to the Anthropic API at run time — the image being local does not make the agent offline-capable.
6. Preparation
6.1. Directory Structure
Create the shared bind mount the MCP services exchange generated tree files through, and — for the worker pool — the external network and volumes:
# Single stack
mkdir -p /opt/mcp-shared
# Worker pool (one-time, shared by every worker stack)
docker network create test-runner-net
docker volume create test-cases-pool
docker volume create run-reports-pool
docker volume create qa-notes-pool
mkdir -p /opt/mcp-shared-pool
The runner’s own state lives in named volumes, not on the host:
| Volume | Content | Backup |
|---|---|---|
|
Imported and hand-edited YAML cases. New cases shipped in the image are copied in on start without overwriting existing files. |
Yes |
|
Run reports, plus |
Yes |
|
Retired QA validation notes. Nothing reads this volume. Keep the data. |
Yes |
|
Gateway database: accounts, stands, profiles, sessions. |
Yes |
|
Generated TR-181 tree XML, shared between the runner and the MCP sidecars. Both sides must mount the same path. |
No |
6.2. Deployment Artifacts
The environment templates live in this repository, so a change to a service and a change to its configuration can travel in the same commit:
| Template | Corresponds to |
|---|---|
|
The root layer — everything the compose files reference as |
|
Everything one |
|
The gateway: authentication, routing and worker provisioning. |
|
The TR-069 emulator MCP sidecar — two fallback variables; the runner passes the env’s emulator and ACS URLs on every call. |
|
The shared tree-generation sidecar. |
They are templates: placeholders only, no host values, and no filled-in copy belongs in git.
6.3. Environment Configuration
Three layers decide what a runner actually uses, in increasing precedence:
-
Hardcoded defaults in
scripts/test_settings.py. -
Environment variables — the compose
environment:block, an.envfile, or the Portainer stack Env array. -
settings.jsonon therun-reportsvolume, written by the Settings tab.
So a value edited in the UI outranks the one set at deploy time and survives a redeploy. In the worker pool a fourth step sits on top: the gateway PUTs the chosen environment profile onto the worker immediately before a run, which replaces the target URLs and credentials for that run. Values in the compose file are therefore startup fallbacks, not the source of truth — see Environment Profiles.
|
A variable referenced as The failure is silent — the value falls back to its |
6.3.1. Secrets
No credential is a literal in the compose files.
TESTRAIL_API_KEY, TESTRAIL_EMAIL and PORTAINER_PASSWORD are referenced with no fallback — unset, the feature that needs them is simply unavailable and everything else keeps working:
| Unset variable | What stops working |
|---|---|
|
TestRail import and background sync. XML import and existing cases are unaffected. |
|
The "restart sibling container" action in the runner UI, and gateway worker auto-provisioning. Workers registered by hand keep working. |
|
Publishing procedures to the knowledge base. |
|
A literal |
7. Deployment
7.1. Single Stack
One runner with its two sidecars, published on the host.
test_runner/docker-compose.yml in this repository is that stack; deploy it with Compose or paste it into Portainer → Stacks → Add Stack.
docker login hub.friendly-tech.com
docker compose -f test_runner/docker-compose.yml up -d
docker compose -f test_runner/docker-compose.yml ps
Set the values from docker/.env.example in the .env next to the compose file, or in the stack’s Env array.
At minimum: TR069_ACS_URL, TR069_EMULATOR_URL, FT_PORTAL_*, and — if TestRail is used — TESTRAIL_EMAIL / TESTRAIL_API_KEY.
IOT_EMULATOR_URL is optional — it only seeds the mcp-iot-emul sidecar’s startup default, since every call carries the active profile’s iot_emulator_url instead.
7.2. Worker Pool
Deploy in this order; each step assumes the previous one succeeded.
-
Shared prerequisites — the external network, the pool volumes and the shared bind mount (Directory Structure).
-
Gateway (
docker-compose.gateway.yml).GATEWAY_SECREThas no default and the stack refuses to start without it:docker compose -f docker-compose.gateway.yml up -d -
One or more workers (
test_runner/docker-compose.stand.template.yml). Copy per worker, replace every<WORKER>placeholder with a unique name, and deploy each as its own stack. Workers publish no host ports and runTRUST_GATEWAY=1, so they have no login of their own.There is no per-environment emulator wrapper: every worker stack bundles its own
mcp-tr-emulandmcp-iot-emulsidecars, and the runner passes the profile’semulator_url/iot_emulator_urlon each call. A profile whose emulator and portal URLs belong to different stands is a split brain: the portal drives one stand while parameters are applied on another, and runs hang on[backend-sync] Timeout.GATEWAY_SECRETmust be identical on the gateway and on every worker — it signs the forwarded-identity headers. A mismatch makes workers reject every proxied request, which surfaces as "login required" rather than as an error about the secret.A worker in
TRUST_GATEWAY=1mode must never be reachable by users directly. -
Register each worker in the gateway UI → Workers → Add, with
base_url=http://test-runner-<WORKER>:8083.
With Portainer credentials configured, Add Worker provisions the worker stack for you (saving an environment profile provisions nothing: the sidecars are bundled). Provisioning is best-effort and inert when those credentials are unset — see Portainer Auto-Provisioning.
7.3. Startup Dependencies
Inside a stack, depends_on orders the runner after mcp-tr-emul and mcp-iot-emul.
Nothing gates on the environment under test: the emulator, ACS and portals live outside these stacks, so Compose cannot wait for them.
They only need to be up when a run starts, not when the container does — a runner with an unreachable environment starts fine and fails at the first preflight.
The one dependency that must exist before start is the logged-in Claude CLI directory being bind-mounted (Required External Dependencies).
8. Verification
8.1. Startup Log
docker compose -f test_runner/docker-compose.yml logs -f test-runner
start.sh brings the container up in a fixed order, and each step announces itself.
Expect, in this order: Xvfb on display :99, x11vnc on 5900, the noVNC proxy on 6080, the test-case sync from image to volume, the Playwright browser checks, Starting Playwright MCP server on port 3100, and finally Starting Test Runner API on port 8083.
A container that never reaches the last line is stuck in one of the browser installs, not in the API.
8.2. Endpoint Checks
Both the runner and the gateway serve /health, which is also what their container healthchecks call:
curl -s http://localhost:8083/health
docker compose -f test_runner/docker-compose.yml ps # expect: healthy
Then open the UI:
http://<host>:8083/
In a worker-pool deployment that URL is the gateway, so it asks you to log in first — see Accounts & Roles.
8.3. First Run
The deployment is only proven once a case actually executes:
-
Log in, open Test cases, pick the target environment in the first dropdown of the run bar.
-
Tick one short case and press Run Selected.
-
Watch the live MCP log, then click Live Browser — the noVNC view proves Xvfb, Chromium and Playwright MCP are all healthy inside the container.
A run that fails at the first preflight points at the environment (emulator, ACS, portal), not at the deployment.
9. Port Reference
Ports are written as published → container.
9.1. Published Ports
| Service | Port | Purpose | Exposure |
|---|---|---|---|
|
8083 → 8083 |
The only publicly published port in a worker-pool deployment: UI, REST API and proxied noVNC. |
Public |
|
8083 → 8083 |
UI and REST API. |
Public |
|
6080 → 6080 |
noVNC view of the Chromium inside the container. |
Public |
|
none |
Workers publish nothing — reachable only by the gateway over
|
Internal |
9.2. Internal Ports
Reachable on the Docker network or inside the container only.
| Service | Port | Purpose |
|---|---|---|
Playwright MCP |
3100 |
Started by |
x11vnc |
5900 |
Raw VNC behind the noVNC proxy |
|
8082 |
Emulator MCP, addressed by container name |
|
7020 |
Tree-generation MCP, SSE transport |
10. Stack Management
10.1. Logs
docker compose logs -f test-runner
docker compose logs --tail 100 test-runner-gateway
Run reports are not in the container log — they live on the run-reports volume and are served by the UI.
10.2. Start, Stop, Restart
docker compose ps
docker compose stop test-runner
docker compose start test-runner
docker compose restart test-runner
Restarting a worker mid-run loses that run: the Chromium and its Playwright session live in the container. Drain the worker first (disable it in the Workers tab) when the restart can wait.
10.3. Updating
docker compose pull
docker compose up -d
The named volumes survive — imported cases, reports, notes and settings.json are kept.
New cases bundled in a newer image are copied into the test-cases volume on start without overwriting existing files, so a locally edited case is never clobbered by an upgrade.
Upgrade the gateway and the workers together: they share GATEWAY_SECRET and the forwarded-identity contract.
11. Production Checklist
-
Change both bootstrap passwords —
GATEWAY_ADMIN_PASSWORDand, on a single stack,TEST_RUNNER_ADMIN_PASSWORD. Both default toadmin/admin. -
Set
GATEWAY_SECRETto a long random value (openssl rand -hex 32), identical on the gateway and every worker. -
Confirm no worker publishes a host port in a pooled deployment — a worker running
TRUST_GATEWAY=1has no authentication of its own. -
Keep credentials out of the compose files — TestRail, Portainer and portal credentials belong in the Env array or
.env, never as literals. -
Verify
shm_size: '2gb'on every runner container. -
Bind-mount a logged-in Claude CLI config and confirm the token refreshes (all workers share the directory).
-
Configure firewall rules — publish only the gateway’s 8083; restrict everything in Network Requirements to trusted hosts.
-
Confirm each environment profile’s
emulator_url,acs_url,iot_emulator_urlandft_portal_urlbelong to the same stand. -
Enable auto-restart —
restart: unless-stoppedin every stack. -
Configure backups for the
test-cases,run-reports,qa-notesandgateway-datavolumes. -
Decide TLS termination — the services speak plain HTTP; put a reverse proxy in front if the gateway is reachable beyond a trusted network.
12. Troubleshooting
For failures inside a run — a step that cannot find a control, a verdict that looks wrong — see the application troubleshooting page. The items below are specific to the container / deployment layer.
12.1. Container Fails to Start
docker compose logs test-runner
docker exec test-runner env | grep -E 'ACS|PORTAL|EMULATOR'
If the log stops before Starting Test Runner API on port 8083, the container is still in start.sh — usually a Playwright browser install without network access to the browser CDN.
12.2. Live Browser Is Blank
Symptom: the UI loads, runs start, but the Live Browser view shows nothing.
Fix:
-
Confirm the noVNC chain came up in the startup log: Xvfb, x11vnc, then
websockify. -
Stale X locks survive an unclean stop.
start.shremoves/tmp/.X99-lockon boot — a container that keeps failing here has a volume mounted over/tmp. -
In a pooled deployment the view is proxied and owner-gated: you see it only while you own the active run on that worker.
12.3. Chromium Crashes or Steps Time Out at Random
Symptom: runs fail unpredictably, often on pages with heavy tables.
Cause: shared memory. Chromium needs far more than Docker’s default 64 MB /dev/shm.
Fix: confirm shm_size: '2gb' on the runner service and recreate the container.
12.4. Every Proxied Call Returns "Login Required"
Symptom: the gateway authenticates you, but every action against a worker bounces back to the login screen.
Cause: GATEWAY_SECRET differs between the gateway and that worker, so the worker rejects the forwarded identity.
Fix:
docker exec test-runner-gateway printenv GATEWAY_SECRET
docker exec test-runner-<worker> printenv GATEWAY_SECRET
They must match exactly. Watch for a doubled $ that was meant as a literal (Secrets).
12.5. Runs Hang on "[backend-sync] Timeout"
Cause: the profile’s emulator and portal URLs point at different stands, so parameters are applied on one environment while the test verifies another.
Fix: check the environment profile in Environments: emulator_url, acs_url, iot_emulator_url and ft_portal_url must all belong to the same stand.
12.6. TestRail Import Does Nothing
Cause: TESTRAIL_API_KEY / TESTRAIL_EMAIL are unset. They deliberately have no fallback in the compose files (Secrets).
Fix: set them in the Env array or .env and recreate the container, or enter them in the Settings tab, which persists them on the run-reports volume.
12.7. Imports and Interact Steps Fail Immediately
Cause: the mounted Claude CLI config has no valid session.
Fix: log in with the CLI on the host, in the directory CLAUDE_CONFIG_DIR points at, and confirm the mount:
docker exec test-runner ls -la /root/.claude
12.8. Port Already in Use
sudo lsof -i :8083
Stop the process holding the port, or change the published port in the compose ports: mapping.
12.9. Getting Support
Collect before contacting Friendly Tech support:
-
The compose file for the affected stack, with credentials masked.
-
docker compose logsfor the runner and, if deployed, the gateway. -
docker versionoutput. -
The environment profile the failing run used.
-
The run report, from the UI.
13. Related Documentation
-
Environments & the Gateway — routing, scheduling, worker pool and Portainer provisioning.
-
Settings — the layer that outranks everything set at deploy time.
-
Accounts, Roles & Audit — logging in and the role model.
-
Architecture — Components — what each container does internally.
-
REST API — the surface published on port 8083.
-
Troubleshooting — failures inside a run rather than in the deployment.
-
Changelog — Current values — the external component versions this branch is verified against.