Installation & Deployment
This guide covers deploying FTACS — the TR-069/TR-369 Auto Configuration Server — as a single service.
Two deployment methods are supported: Docker Compose (recommended) and manual installation from the distribution package on bare metal or a VM.
The infrastructure FTACS depends on — a relational database, ClickHouse, Hazelcast, and FT Configs Service — is assumed to be already deployed and reachable; this guide only points FTACS at it. To deploy that infrastructure, see All in one server deployment or Separate server deployment.
For the environment variable reference, see Configuration Guide — Environment Variables.
1. Overview
FTACS is a single container (ftacs) — or a single JVM process in a manual deployment — exposing the TR-069, USP, MQTT, LWM2M, SOAP, and REST endpoints.
It is a plain Compose service on the ft-network bridge, deployed from /usr/local/ft-system/.
| Deployment method | What it installs | Section |
|---|---|---|
Docker Compose |
The |
|
Manual |
The distribution package on a JDK 25 host, driven by |
Everything else in the diagram below is a prerequisite, not part of this installation:
CPE devices -> FTACS -> MySQL / Oracle + ClickHouse + Hazelcast <- FT Configs Service
|
FTACS uses the Compose project name |
1.1. Choosing a Deployment Method
-
Docker Compose — recommended. Use this unless a policy forbids containers.
-
Manual — the distribution package runs directly on a JDK 25 host. Use it when Docker is unavailable, or when FTACS must run alongside existing bare-metal services.
|
Since version 6.5.2, FT Configs Service is a hard dependency — it owns the ACS runtime configuration that used to live in Deploying it is out of scope here: use FT Configs deployment, or Separate server deployment — Server C, which brings up FTACS and FT Configs Service together on one host. |
2. Prerequisites
2.1. Host Requirements
| Component | Minimum | Recommended | Notes |
|---|---|---|---|
CPU |
4 cores |
— |
Baseline for a single FTACS instance |
RAM |
8 GB |
— |
Must cover the JVM heap set through |
Disk |
20 GB |
— |
Application, configuration, and logs. The database and ClickHouse keep their data on their own hosts |
Network |
1 Gbps interface |
— |
Device traffic plus the outbound connections listed in Network Requirements |
Docker Engine |
20.10 |
— |
Docker deployment only; Docker Compose 2.0+ as well |
JDK |
25 |
— |
Manual deployment only — the application is built and run on Java 25 |
JVM sizing by deployment scale:
| Deployment size | Devices | JVM settings |
|---|---|---|
Small |
< 1,000 |
|
Medium |
1,000—5,000 |
|
Large |
5,000—20,000 |
|
Very large |
20,000+ |
|
2.2. Required External Dependencies
These services must be installed, running, and reachable from this host before FTACS starts. None of them are deployed by this guide.
| Component | Minimum Version | Why It Is Needed | Port | Required |
|---|---|---|---|---|
MySQL or Oracle |
MySQL 8.4 / Oracle 21c (the reference compose in this repo builds MySQL from |
Holds the |
3306 (TCP) for MySQL, 1521 (TCP) for Oracle |
Yes |
ClickHouse |
26.2 |
Stores the QoE and statistics data (schema |
8123 (HTTP) |
Yes |
Hazelcast |
5.5 (FTACS embeds the |
Distributed cache and the only channel through which FT Configs Service delivers the ACS runtime configuration. Without it FTACS cannot initialise its Hazelcast client and receives no configuration. |
5701, 5711 (TCP) |
Yes |
FT Configs Service |
verify with the FT Configs Service owner |
Owns the ACS runtime configuration since FTACS 6.5.2 and publishes it into the shared Hazelcast cluster. Without it FTACS starts with no ACS configuration, no web service users, and no parameter-name cache rules. |
No direct port — delivered through Hazelcast; its own health endpoint is 8087 (HTTP) |
Yes |
|
Deploying that infrastructure is out of scope here — see All in one server deployment for a single-host stack, or Separate server deployment for a split topology, and FT Configs deployment for FT Configs Service. The database schemas ( 26.2 is the supported ClickHouse version. The reference compose in this repo ( |
|
FTACS and FT Configs Service must be clients of the same Hazelcast cluster — that cluster is the only channel between them.
If |
2.3. Supported Operating Systems
| Deployment | Operating system |
|---|---|
Docker |
Linux with Docker Engine 20.10+ (recommended), macOS, or Windows with WSL2 |
Manual |
Linux (recommended), Windows Server — JDK 25 required |
2.4. Registry Access
Network access to the hub.friendly-tech.com Docker registry (or offline image archives — see Offline Servers).
Access to Friendly Tech SharePoint (FT_DISK) for the deployment files (compose.yml, .env, ftacs/.env) and the pre-configured ftacs-conf package.
3. Network Requirements
Outbound connections FTACS opens to its infrastructure, and inbound connections it accepts. Each row is a connection that must be open through firewalls when the peer is on another host; traffic to peers on the same Docker bridge needs no rule.
| Destination | Port | Protocol | Purpose |
|---|---|---|---|
Database |
3306 (Oracle: 1521) |
TCP |
Outbound: relational database connection |
ClickHouse |
8123 |
HTTP |
Outbound: ClickHouse HTTP API — FTACS uses the |
Hazelcast |
5701, 5711 |
TCP |
Outbound: Hazelcast client and member discovery |
FTACS |
8080, 8181, 8443 |
TCP |
Inbound from CPE devices: TR-069 (CWMP) sessions, SOAP, REST |
FTACS |
1883, 8883 |
TCP |
Inbound from CPE devices: MQTT and MQTT over TLS |
FTACS |
8025, 4430, 18881, 18882, 18891, 61617, 61618 |
TCP |
Inbound from CPE devices: USP (TR-369) — WebSocket, MQTT, Bulk Data, STOMP |
FTACS |
5783, 5784 |
UDP |
Inbound from CPE devices: USP CoAP and CoAP over DTLS |
FTACS |
5680—5685 |
UDP |
Inbound from CPE devices: LWM2M Bootstrap and LWM2M Server |
FTACS |
8080 |
TCP |
Inbound from monitoring: Prometheus scrapes |
|
For a quick connectivity check from any host:
|
3.1. Docker Networking
The compose file in Docker Compose puts FTACS on the ft-network bridge with
container_name: ftacs, so peers on that same bridge reach it as ftacs and need no firewall rule.
The infrastructure services are addressed the same way when they share the bridge — that is what the
comment on DB_HOST in the environment file means by "use the container name if it shares this
bridge".
Peers on another host must be given a routable address instead: DB_HOST, HZ_MEMBERS and
CLICKHOUSE_HOST take an IP or DNS name, and every port in the table above has to be open through
the firewall. Mixing the two is the usual cause of a container that starts and then cannot reach its
database: a container name only resolves inside the bridge that defines it.
If a peer runs on the same host but outside Docker, use host.docker.internal rather than
localhost — inside a container localhost is the container itself:
DB_HOST=host.docker.internal
HZ_MEMBERS=host.docker.internal:5701
CLICKHOUSE_HOST=host.docker.internal
host.docker.internal resolves to the host’s gateway address. On Docker Desktop (macOS, Windows) it
works out of the box; on Linux it exists only if the container maps it, which the compose file in
Docker Compose does:
extra_hosts:
- "host.docker.internal:host-gateway"
The alternative is the host’s real LAN IP address.
4. Registry Authentication
All FTACS images are pulled from hub.friendly-tech.com.
Authenticate once per host before the first docker compose up.
docker login hub.friendly-tech.com
Enter the read-only pull credentials when prompted:
| Field | Value |
|---|---|
Username |
|
Password |
|
|
The |
Alternatively, log in non-interactively:
echo "fokxuw-fymte1-taSxyc" | docker login hub.friendly-tech.com -u readonly --password-stdin
Verify authentication:
docker info | grep -A 5 Registry
4.1. Offline Servers
When the FTACS host cannot reach hub.friendly-tech.com, pull the image on a machine that does have registry access, export it to an archive, transfer it, and load it on the offline host.
The connected machine can run Linux, macOS, or Windows — commands are given for both shells below. It does not need to be the same platform as the offline host.
|
An explicit |
-
On the offline host, find out which architecture it runs — this is the value you will pass as
PLATFORMbelow. Ask Docker itself, since it reports what the daemon will actually accept:docker version --format '{{.Server.Arch}}'If Docker is not installed there yet, use the operating system instead —
uname -mon Linux, orecho $env:PROCESSOR_ARCHITECTUREin PowerShell on Windows. Map the result:docker versionreportsuname -m/ Windows reportsUse as PLATFORMamd64x86_64/AMD64linux/amd64arm64aarch64/ARM64linux/arm64On Windows with Docker Desktop,
{{.Server.Arch}}reports the architecture of the Linux VM that actually runs the containers — which is the value you want, not the Windows host’s own architecture. -
On a machine with registry access, log in. Run this on its own — it prompts for a password, so anything pasted after it on the same go would be swallowed as input:
docker login hub.friendly-tech.com -
Pull and export the image. Paste the whole block as-is; the only lines to change are
PLATFORMandTAGon top.Linux / macOS (bash):
PLATFORM=linux/amd64 TAG=latest docker pull --platform "$PLATFORM" "hub.friendly-tech.com/ftacs/ftacs:$TAG" docker save "hub.friendly-tech.com/ftacs/ftacs:$TAG" | gzip > "ftacs-$TAG.tar.gz"Windows (PowerShell):
$PLATFORM = "linux/amd64" $TAG = "latest" docker pull --platform $PLATFORM "hub.friendly-tech.com/ftacs/ftacs:$TAG" docker save -o "ftacs-$TAG.tar" "hub.friendly-tech.com/ftacs/ftacs:$TAG"On Windows, always write the archive with
docker save -o <file>. Piping or redirectingdocker savefrom PowerShell (docker save … > file.tar) corrupts the archive, because the PowerShell pipeline re-encodes the stream as text instead of passing raw bytes.docker loadthen fails withunexpected EOForinvalid tar header. To compress for transfer, use the bundledtar.exe(Windows 10 1803+ / Server 2019+):tar.exe -czf ftacs.tar.gz ftacs-$TAG.tar. -
Transfer the archive to the offline host, together with
compose.yml,.env,ftacs/.env, and the three files inftacs/config/. -
On the offline host, load the archive and start the service. Set
TAGto the same value you used above:Linux (bash):
TAG=latest gzip -dc "ftacs-$TAG.tar.gz" | docker load docker compose up -d ftacsWindows (PowerShell):
$TAG = "latest" docker load -i "ftacs-$TAG.tar" docker compose up -d ftacs
Confirm the image is present before starting, so a missing or mis-architected image fails here rather than mid-startup:
docker images hub.friendly-tech.com/ftacs/ftacs
Upgrades use the same flow: pull the new tag on the connected machine, transfer and load the archive, then docker compose up -d ftacs.
|
This covers the FTACS image only.
The database, ClickHouse, Hazelcast, and FT Configs Service are separate images on their own hosts — for the offline procedure covering a whole stack, see All in one server deployment — Offline Servers, which derives the image list from |
5. Preparation
Create the directories FTACS needs on the host and point it at the infrastructure listed in Prerequisites.
5.1. First: Check the Existing Database (7.1.2 and later)
Applies whenever 7.1.2 or later is brought up against a database that already contains devices — an upgrade, a re-install on an existing schema, or a migration of the database to a new host. A fresh, empty database needs none of this.
Release 7.1.2 carries a data-backfill migration: unlike a schema change, it rewrites existing rows,
so its cost scales with the size of the cpe table. Before anything else, find out how big that is:
SELECT COUNT(*) FROM cpe;
Devices in cpe |
What this means for the installation |
|---|---|
Up to ~1 million |
Nothing to prepare. The migration completes in seconds during startup. Continue with Directory Structure. |
More than ~1 million |
A manual step is required before starting the new version, and the cluster must be stopped for the upgrade rather than updated node by node. Read Upgrading a Large Installation now — it explains the backfill script, the verification queries and the failure modes of skipping it. |
|
On a large installation, starting 7.1.2 without that preparation is not merely slow: the whole |
5.2. Directory Structure
mkdir -p /usr/local/ft-system/ftacs/{config,logs}
cd /usr/local/ft-system
5.2.1. Directory Layout
FTACS is one slice of the platform layout described in Prepare Working Directory — the stack root holds compose.yml and the shared .env, and each service owns a directory below it:
/usr/local/ft-system/
├── compose.yml # from FT_DISK
├── .env # shared stack environment
└── ftacs/
├── .env # per-service environment
├── sensitive_params.xml # optional, mounted separately
├── config/ # -> /opt/FTACS/conf (read-only)
│ ├── ftacs.keystore
│ ├── hazelcast-client.xml
│ └── logback-spring.xml
└── logs/ # -> /opt/FTACS/logs
| Path | Content | Backup |
|---|---|---|
|
Stack definition; the FTACS service block is shown in Deployment |
Yes |
|
Shared stack environment — database connection, ClickHouse, Hazelcast, timezone |
Yes |
|
FTACS-only environment — published host ports, HikariCP pool sizes, JVM settings |
Yes |
|
The three files FTACS reads from disk, mounted read-only at |
Yes |
|
Optional; mounted individually into |
Yes |
|
Application file logs, mounted at |
No |
Download the following from FT_DISK:
| Component | FT_DISK | Target directory | Files |
|---|---|---|---|
FTACS environment |
|
Only one file: the |
|
FTACS configuration |
|
Only three files: |
The ftacs/config/ directory is mounted read-only at /opt/FTACS/conf inside the container and holds only the three files FTACS must read from disk before it can reach anything else:
| File | Purpose |
|---|---|
|
SSL keystore for the HTTPS ports |
|
Hazelcast cluster client configuration — also the transport FTACS receives its runtime configuration over |
|
Logging configuration |
|
Since version 6.5.2 the remaining runtime configuration lives in FT Configs Service, not on disk.
FT Configs Service publishes each configuration snapshot into the shared Hazelcast cluster, and FTACS consumes it from there; there is no direct URL from FTACS to the service. This is why both must be clients of the same Hazelcast cluster — see Prerequisites. |
See Configuration Guide for the reference on each configuration item, and FT Configs deployment for deploying the service that now owns them.
5.3. Environment Configuration
The stack uses two environment layers, and every service — FTACS included — reads both:
| File | Holds |
|---|---|
|
Shared by the whole stack: database connection ( |
|
FTACS only: published host ports, HikariCP pool sizes ( |
There is no per-vendor environment file. MySQL is the default; switching to Oracle means changing DB_PROFILE, SPRING_PROFILES_ACTIVE, DB_HOST, DB_PORT and DB_SERVICE inside the single root .env — see Switching to Oracle. compose.yml maps these generic DB_* names onto the vendor-specific names FTACS reads, through the x-db-env anchor shown in Deployment.
# /usr/local/ft-system/.env
TZ=Europe/Kyiv
# Database (existing)
DB_HOST=<db-host-ip> # <-- replace; use the container name if it shares this bridge
DB_PORT=3306
DB_USER=ftacs
DB_PASSWORD=<your-db-password> # <-- replace
DB_SERVICE=XEPDB1
DB_PROFILE=mysql
SPRING_PROFILES_ACTIVE=mysql
# Hazelcast (existing)
HZ_MEMBERS=<hazelcast-host-ip>:5701 # <-- replace
# ClickHouse (existing)
CLICKHOUSE_HOST=<clickhouse-host-ip> # <-- replace
CLICKHOUSE_PORT=9000
CLICKHOUSE_HTTP_PORT=8123
CLICKHOUSE_SCHEMA=ftacs_qoe_ui_data
CLICKHOUSE_USERNAME=ftacs
CLICKHOUSE_PASWORD=<your-ch-password> # <-- replace; single "S" is intentional, see the note below
# FTACS
FT_ACS_JAVA_RAM=-Xms1g -Xmx4g
|
The port changes together with the host, not only the hostname.
Across hosts you connect to the published port (the left side of |
|
The ClickHouse password variable is misspelled in the FTACS code: |
The variables FTACS itself reads are defined in this repository by FT_IoT/FTAPP/ftapp-build/docker/ftacs.env (Docker), FT_IoT/FTAPP/ftapp-build/bin/app.env (manual) and FT_IoT/FTAPP/ftapp-build/docker/docker-entrypoint.sh:
| Variable | Description | Default | Required |
|---|---|---|---|
|
Installation root; |
|
No |
|
Database vendor; selects the active Spring profile ( |
|
No |
|
Hostname or IP of the database server |
none |
Yes |
|
Password of the FTACS database user |
none |
Yes |
|
MySQL listener port |
|
Yes (MySQL) |
|
MySQL user owning the |
|
Yes (MySQL) |
|
Oracle listener port |
|
Yes (Oracle) |
|
Oracle service name |
|
Yes (Oracle) |
|
Oracle user owning the |
|
Yes (Oracle) |
|
Oracle user owning the |
|
Yes (Oracle) |
|
Hazelcast member list as |
none |
Yes |
|
Hostname or IP of the ClickHouse server |
none |
Yes |
|
ClickHouse port FTACS connects to. FTACS uses the HTTP client, so this must be the HTTP port |
|
Yes |
|
ClickHouse user |
|
Yes |
|
ClickHouse password. Misspelled in the code (single "S") — |
|
Yes |
|
ClickHouse database FTACS writes QoE data to. FTACS does not read |
|
Yes |
|
HTTP listener ports; comma or space separated |
|
No |
|
HTTPS listener port passed to the application as |
|
No |
|
Path to the Java keystore serving the TLS ports |
empty |
No |
|
Password of that keystore |
empty |
No |
|
Servlet path of the unauthenticated TR-069 endpoint |
|
No |
|
Servlet path of the HTTP Basic TR-069 endpoint |
|
No |
|
Servlet path of the HTTP Digest TR-069 endpoint |
|
No |
|
JVM options passed to the |
|
No |
The names used in the shared .env above (DB_PORT, DB_USER, DB_SERVICE, DB_PROFILE, CLICKHOUSE_HTTP_PORT, FT_ACS_JAVA_RAM) come from the portal-wide deployment bundle and do not appear in this repository’s env files; verify with FT DevOps which set applies to your build.
|
Create ftacs/.env with the per-service variables documented in Configuration Guide — Environment Variables; the full platform-wide reference is FTACS Environment Variables.
Click to expand a commented ftacs/.env reference
# =============================================================================
# HikariCP Connection Pools
# =============================================================================
# Shared pool defaults (apply to all datasources unless overridden below)
DB_MAX_POOL_SIZE=10
DB_MIN_IDLE=5
DB_CONNECTION_TIMEOUT_MS=30000
# Main FTACS datasource pool. Handles CPE management, parameters,
# provisioning operations. Needs a larger pool due to high concurrency.
FTACS_MAX_POOL=30
FTACS_MIN_POOL=5
# Quartz job scheduler datasource. Used for scheduled task persistence
# (UpdateGroup, QoE, Events). Lower load than the main datasource.
QUARTZ_MAX_POOL=10
QUARTZ_MIN_POOL=2
# ClickHouse analytics datasource. Used for QoE data and statistics.
CLICKHOUSE_MAX_POOL=10
CLICKHOUSE_MIN_POOL=2
# =============================================================================
# ACS Deployment Mode
# Controls which functionality is active. Used for distributed deployments
# where provisioning and management run on separate nodes.
# =============================================================================
# ALL = full functionality (provisioning + management) -- single-node default
# PROV = provisioning only (device bootstrap, session handling)
# MNGMNT = management only (device monitoring, configuration)
# In MNGMNT mode, FTACS connects to a separate Hazelcast "provision" cluster.
# FTACS_MODE=ALL
# =============================================================================
# Server IP Address
# Advertised to CPE devices for callbacks and connection requests.
# Auto-detected from the hostname if not set. Set explicitly in Docker/NAT
# environments where auto-detection returns an unreachable address.
# =============================================================================
# IPv4 address advertised to CPE devices (e.g. 192.168.1.100)
# FTACS_IP_ADDRESS=
# IPv6 address for dual-stack deployments. Brackets are added automatically.
# FTACS_IPV6_ADDRESS=
# =============================================================================
# Event Subscription Parameters
# Control when parameter change notifications are sent to subscriber URLs.
# =============================================================================
# 0 = notify on every parameter update (EACH_VALUE trigger)
# 1 = notify only when the value actually changes (VALUE_CHANGE trigger)
# FTACS_SUBSCRIPTION_PARAMETER_ONLY_VALUE_CHANGE=0
# 0 = include all parameter values (including null/empty)
# 1 = skip notifications for null or empty values
# FTACS_SUBSCRIPTION_PARAMETER_NOT_NULL_ONLY=0
# =============================================================================
# TR-069 Endpoint URL Mappings
# Each defines the servlet URL path for a different authentication mode.
# Multiple paths can be comma-separated.
# =============================================================================
# No authentication -- CPE connects without credentials
WEB_TR069_NO_AUTH=/ftacs/ACS
# HTTP Basic Authentication (RFC 7617) -- CPE sends base64(user:password)
WEB_TR069_BASIC=/ftacs-basic/ACS
# HTTP Digest Authentication (RFC 2617) -- challenge-response, no plaintext password
WEB_TR069_DIGEST=/ftacs-digest/ACS
# =============================================================================
# MQTT Protocol (optional)
# Activates the embedded MQTT broker for TR-369/USP MQTT-MTP transport.
# =============================================================================
MQTT_ENABLE=false
# Additional MQTT transport connector URI (ActiveMQ format).
# Format: mqtt+nio://<host>:<port> or mqtt+nio+ssl://<host>:<port>
# FT_MQTT_ADDITIONAL_CONNECTOR_URI=
# Active MQTT API versions. Incoming requests are validated against this list.
# MQTT_ACTIVE_API=api-v1,api-v2
# =============================================================================
# USP Protocol (optional)
# =============================================================================
# Path to the USP data model XML specification (BBF TR-181-2).
# Loaded from the filesystem first; falls back to a classpath resource.
# USP_SERVER_DM_PATH=tr-181-2-16-0-usp-full.xml
# =============================================================================
# X509 Certificates -- USP TLS/DTLS (optional)
# Used for secure USP MTP connections (WebSocket, STOMP, CoAP over DTLS).
# Format: JKS or PKCS12 keystore files.
# =============================================================================
# FT_USP_X509_KEYSTORE_PATH= # Server keystore with private key and certificate
# FT_USP_X509_KEYSTORE_PASSWORD= # Keystore file password
# FT_USP_X509_KEYSTORE_PK_PASSWORD= # Private key password (if it differs)
# FT_USP_X509_TRUSTSTORE_PATH= # Truststore with trusted CA certificates
# FT_USP_X509_TRUSTSTORE_PASSWORD= # Truststore file password
# =============================================================================
# X509 Certificates -- MQTT TLS (optional)
# Used for MQTTS on port 8883. Enables mutual TLS when a truststore is set.
# =============================================================================
# FT_MQTT_X509_KEYSTORE_PATH=
# FT_MQTT_X509_KEYSTORE_PASSWORD=
# FT_MQTT_X509_PK_PASSWORD=
# FT_MQTT_X509_TRUSTSTORE_PATH=
# FT_MQTT_X509_TRUSTSTORE_PASSWORD=
# =============================================================================
# General Async Executor (used by @Async methods without an explicit executor)
# =============================================================================
# Minimum threads always running. Idle core threads are reclaimed after
# ASYNC_KEEPALIVE seconds (allowCoreThreadTimeout=true).
ASYNC_CORE=10
# Maximum threads under load. When ASYNC_QUEUE=0, threads spawn immediately
# up to this limit before tasks go to an external 10,000-task overflow queue.
ASYNC_MAX=200
# Internal queue capacity before spawning new threads.
# 0 = no internal queue (SynchronousQueue); tasks spawn threads immediately.
# An external OverflowingExecutor with a 10,000-task queue wraps the pool as a
# safety net, blocking on overflow to provide backpressure.
ASYNC_QUEUE=0
# Seconds an idle thread stays alive before termination.
ASYNC_KEEPALIVE=60
# Thread pool size for @Scheduled methods (virtual threads enabled).
# Handles periodic tasks: statistics flush (every 60s), counter reset (daily).
# Quartz jobs use a separate JDBC-based scheduler, not this pool.
SCHED_POOL=300
# =============================================================================
# HikariCP Tuning (optional)
# Applies to ALL three datasources (ftacs, quartz, clickhouse).
# =============================================================================
# Max time (ms) a connection can sit idle before removal. Default: 60000
# HIKARI_IDLE_TIMEOUT_MS=60000
# Max time (ms) to wait for a connection from the pool. Default: 45000
# HIKARI_CONN_TIMEOUT_MS=45000
# Max age (ms) of a connection before forced rotation. Default: 180000
# Must be greater than HIKARI_IDLE_TIMEOUT_MS.
# HIKARI_MAX_LIFETIME_MS=180000
# Max time (ms) for the connection validation query (SELECT 1). Default: 3000
# HIKARI_VALIDATION_TIMEOUT_MS=3000
# Register HikariCP pool metrics as JMX MBeans (true/false).
# HIKARI_REGISTER_MBEANS=false
5.4. TLS Keystore (optional)
FTACS serves its HTTPS port (8443) and the other TLS ports from a single Java keystore, ftacs.keystore.
The bundle on FT_DISK ships a self-signed keystore — suitable for testing only.
This step applies to both deployment methods; only the target directory and the file holding the password differ.
To use a CA-issued or otherwise custom certificate:
-
Generate the keystore (or import an existing certificate into one):
keytool -genkey -alias ftacs -keyalg RSA -keysize 2048 -keystore ftacs.keystore -
Place
ftacs.keystorein the FTACS configuration directory:-
Docker:
ftacs/config/ftacs.keystore(mounted read-only at/opt/FTACS/conf). -
Manual:
conf/ftacs.keystorein the distribution package.
-
-
Set the keystore password.
SSL_KEYSTORE_PASSWORDmust match the password used when the keystore was created:-
Docker: in
ftacs/.env. -
Manual: in
bin/app.env, alongsideSSL_KEYSTORE=${CONF_DIR}/ftacs.keystore.
-
|
Both paths resolve to |
6. Deployment
Choose one method: Docker Compose (recommended) or the distribution package on a JDK 25 host.
6.1. Startup Dependencies
FTACS has no depends_on on its infrastructure — the database, ClickHouse, Hazelcast, and FT Configs Service live outside this stack, so Compose cannot gate on their health.
All of them must be up before FTACS launches: Liquibase applies the schema migrations on first start, the Hazelcast client connects during context initialisation, and FT Configs Service must already have published its configuration snapshots into Hazelcast.
Wait for each of them explicitly before starting the service:
# Wait for the database
until nc -z <db-host-ip> 3306; do sleep 2; done
# Wait for ClickHouse (HTTP port)
until curl -sf http://<clickhouse-host-ip>:8123/ping; do sleep 2; done
# Wait for Hazelcast
until nc -z <hazelcast-host-ip> 5701; do sleep 2; done
# Wait for FT Configs Service to be healthy (it warms up its Hazelcast snapshots on start)
until curl -sf http://<configs-service-host-ip>:8087/configs-service/actuator/health; do sleep 2; done
# Then start FTACS
docker compose up -d ftacs
FTACS itself restarts cleanly at any time once the infrastructure is up — restart: unless-stopped reconnects it after a host reboot.
6.2. Docker Compose
Click to expand compose.yml (FTACS)
x-db-env: &db-env
DB_VENDOR: ${DB_PROFILE}
DB_HOST: ${DB_HOST}
MYSQL_PORT: ${DB_PORT}
MYSQL_USER: ${DB_USER}
MYSQL_SCHEMA_FTACS: ftacs
MYSQL_SCHEMA_QUARTZ: ftacs_quartz
FTACS_PASSWORD: ${DB_PASSWORD}
QUARTZ_PASSWORD: ${DB_PASSWORD}
services:
ftacs:
image: hub.friendly-tech.com/ftacs/ftacs:latest
container_name: ftacs
# The database, ClickHouse and Hazelcast are external to this stack, so there
# is no depends_on. They must be healthy before ftacs starts -- DB_HOST,
# CLICKHOUSE_HOST and HZ_MEMBERS in .env point at them.
env_file:
- .env
- ./ftacs/.env
environment:
<<: *db-env
USE_NOHUP: 0
# ftacs uses the clickhouse-java HTTP client, so it must target the HTTP
# port (8123), not the native TCP port (9000) that CLICKHOUSE_PORT defaults
# to in the shared .env. Without this remap, ClickHouse connections fail.
CLICKHOUSE_PORT: "${CLICKHOUSE_HTTP_PORT:-8123}"
JAVA_OPTS: "${FT_ACS_JAVA_RAM:--Xms1g -Xmx4g} -Duser.timezone=${TZ:-Europe/Kyiv}"
ports:
# TR-069 / REST / SOAP
- "${FTACS_HTTP_PORT:-8080}:8080"
- "${FTACS_HTTPS_PORT:-8443}:8443"
- "${ACS_HTTP_ADDITIONAL:-8181}:8181"
# MQTT
- "${ACS_MQTT:-1883}:1883"
- "${ACS_MQTT_TLS:-8883}:8883"
# USP (TR-369)
- "${ACS_USP_WS:-8025}:8025"
- "${ACS_USP_WSS:-4430}:4430"
- "${ACS_USP_MQTT:-18881}:18881"
- "${ACS_USP_MQTT_TLS:-18882}:18882"
- "${ACS_USP_BULK:-18891}:18891"
- "${ACS_USP_STOMP:-61617}:61617"
- "${ACS_USP_STOMP_TLS:-61618}:61618"
- "${ACS_USP_CoAP:-5783}:5783/udp"
- "5784:5784/udp"
# LWM2M Bootstrap
- "${BS_PORT_NOSEC:-5680}:5680/udp"
- "${BS_PORT_PSK:-5681}:5681/udp"
- "${BS_PORT_X509:-5682}:5682/udp"
- "${BS_TCP_PORT_NOSEC:-5880}:5880"
# LWM2M Server
- "${LWM2M_PORT_NOSEC:-5683}:5683/udp"
- "${LWM2M_PORT_PSK:-5684}:5684/udp"
- "${LWM2M_PORT_X509:-5685}:5685/udp"
- "${LWM2M_TCP_PORT_NOSEC:-5883}:5883"
volumes:
- ./ftacs/config:/opt/FTACS/conf:ro
- ./ftacs/logs:/opt/FTACS/logs
healthcheck:
test: ["CMD", "pgrep", "-f", "com.friendly.acs.FtAcsApp"]
interval: 20s
timeout: 5s
retries: 10
start_period: 60s
restart: unless-stopped
stop_grace_period: 45s
extra_hosts:
- "host.docker.internal:host-gateway" # peers running on this host outside Docker
networks:
- ft-network
For Oracle, keep the same two env_file entries — there is no per-vendor environment file — set the vendor variables in the root .env as described in Switching to Oracle, and change the x-db-env anchor to Oracle-specific mappings:
x-db-env: &db-env
DB_VENDOR: ${DB_PROFILE}
DB_HOST: ${DB_HOST}
ORACLE_PORT: ${DB_PORT}
ORACLE_SERVICE: ${DB_SERVICE}
ORACLE_USER_FTACS: ${DB_USER}
ORACLE_USER_QUARTZ: ftacs_quartz
FTACS_PASSWORD: ${DB_PASSWORD}
QUARTZ_PASSWORD: ${DB_PASSWORD}
|
The anchors above map the shared |
6.2.1. Sensitive Parameters
To run FTACS with a sensitive_params.xml file, mount it from the host into the container as an additional volume:
volumes:
- ./ftacs/config:/opt/FTACS/conf:ro
- ./ftacs/logs:/opt/FTACS/logs
- ./ftacs/sensitive_params.xml:/opt/FTACS/app/WEB-INF/lib/sensitive_params.xml:ro
6.2.2. Start the Stack
|
Start FTACS only after the database, ClickHouse, Hazelcast, and FT Configs Service accept connections — see Startup Dependencies. |
cd /usr/local/ft-system
docker login hub.friendly-tech.com
docker compose up -d ftacs
docker compose ps
# FTACS takes 1-2 minutes to start
docker compose logs -f ftacs | grep -i "started"
curl -s -u <ws-user>:<ws-password> http://localhost:8080/acsstats
To run FTACS as a single container without Compose:
cd /usr/local/ft-system
docker run -d \
--name ftacs \
--restart unless-stopped \
--env-file .env --env-file ./ftacs/.env \
-p 8080:8080 \
-p 8443:8443 \
-v ./ftacs/config:/opt/FTACS/conf:ro \
-v ./ftacs/logs:/opt/FTACS/logs \
hub.friendly-tech.com/ftacs/ftacs:latest
|
|
6.3. Manual (dist package)
FTACS runs directly on a JDK 25 host from the distribution package. The database, ClickHouse, Hazelcast, and FT Configs Service must be reachable before starting the application — see Prerequisites.
6.3.1. Directory Structure
The extracted distribution package has this structure:
dist/ ├── app/ # Compiled application (exploded WAR) -- do NOT modify │ ├── META-INF/ # JAR/WAR manifest and persistence descriptors │ ├── org/ # Java bytecode packages (ACS core classes) │ └── WEB-INF/ # Spring Boot web configuration and libraries │ ├── bin/ # Executable scripts and environment definitions │ ├── app.env # Main environment file (DB, ports, JVM, etc.) │ ├── env_to_cmd.cmd # Helper: .env -> Windows CMD variables │ ├── start.cmd # Windows startup script │ ├── start.sh # Linux startup script │ ├── stop.cmd # Windows shutdown script │ └── stop.sh # Linux shutdown script │ ├── conf/ # Configuration files (editable by operators) │ ├── ftacs.keystore # SSL keystore for the HTTPS ports │ ├── hazelcast-client.xml # Hazelcast client -- also the FT Configs Service channel │ └── logback-spring.xml # Logging configuration │ # everything else moved to FT Configs Service in 6.5.2 │ ├── ext/ # External extensions / JARs loaded dynamically ├── flyway/ # ClickHouse migration scripts (managed by Flyway) ├── liquibase/ # Database migration scripts (managed by Liquibase) ├── logs/ # Runtime logs (created at startup) └── patches/ # Hot-fix patches
|
6.3.2. Install the JDK
- Linux
-
wget https://download.oracle.com/java/25/latest/jdk-25_linux-x64_bin.tar.gz tar -xzf jdk-25_linux-x64_bin.tar.gz sudo mv jdk-25 /opt/jdk-25 # Add to ~/.bashrc or /etc/profile export JAVA_HOME=/opt/jdk-25 export PATH=$JAVA_HOME/bin:$PATH java -version - Windows
-
-
Download the JDK 25 installer from the Oracle website.
-
Run the installer (e.g.
jdk-25_windows-x64_bin.exe). -
Set the
JAVA_HOMEenvironment variable toC:\Program Files\Java\jdk-25. -
Add
%JAVA_HOME%\bintoPATH. -
Verify: open Command Prompt and run
java -version.
-
Then extract the distribution package:
# Linux
tar -xzf ftacs-dist-<version>.tar.gz
cd dist/
# Windows
unzip ftacs-dist-<version>.zip
cd dist\
6.3.3. Environment Configuration
Edit bin/app.env in the extracted distribution package.
# === Base directories (all other paths derive from APP_HOME) ===
APP_HOME=/usr/local/ftacs/ # <-- replace with your dist path
CONF_DIR=${APP_HOME}/conf
LOG_DIR=${APP_HOME}/logs
EXT_DIR=${APP_HOME}/ext
PATCHES_DIR=${APP_HOME}/patches
DB_LIQUIBASE_DIR=${APP_HOME}/liquibase
# === Database connection ===
DB_HOST=localhost # <-- replace
DB_PASSWORD=<your-db-password> # <-- replace
# --- MySQL (default) ---
DB_VENDOR=mysql
MYSQL_PORT=3306
MYSQL_USER=ftacs
# --- Oracle (uncomment to switch) ---
#DB_VENDOR=oracle
#ORACLE_PORT=1521
#ORACLE_SERVICE=orclpdb1 # <-- replace with your PDB name
#ORACLE_USER_FTACS=ftacs
#ORACLE_USER_QUARTZ=ftacs_quartz
# === ClickHouse ===
CLICKHOUSE_HOST=localhost # <-- replace
CLICKHOUSE_PORT=8123
CLICKHOUSE_SCHEMA=ftacs_qoe_ui_data
CLICKHOUSE_USERNAME=ftacs
CLICKHOUSE_PASWORD=<your-ch-password> # <-- replace
# === Hazelcast ===
HZ_MEMBERS=127.0.0.1:5701 # <-- replace with cluster addresses
HZ_CLIENT_CONFIG=${CONF_DIR}/hazelcast-client.xml
# === TR-069 device URL mappings ===
WEB_TR069_NO_AUTH=/ftacs/ACS
WEB_TR069_BASIC=/ftacs-basic/ACS
WEB_TR069_DIGEST=/ftacs-digest/ACS
# === Web / SSL ===
HTTP_PORTS=8080,8181
HTTPS_PORTS=8443
SSL_KEYSTORE=${CONF_DIR}/ftacs.keystore
SSL_KEYSTORE_PASSWORD=<your-keystore-password> # <-- replace
# === JVM options ===
JAVA_OPTS='-Xms1g -Xmx4g'
# === Additional configuration ===
SPRING_CONFIG_EXTRA=file:${CONF_DIR}/
LOGBACK_CONFIG=${CONF_DIR}/logback-spring.xml
CXF_PATH=/ACSServer-ACS
QUARTZ_PASSWORD=${DB_PASSWORD}
|
Set
On Windows, use forward slashes ( |
7. Verification
7.1. Startup Log
Wait 30—60 seconds after start, then check the logs:
# Docker Compose
docker compose logs ftacs | tail -20
# Docker standalone
docker logs ftacs | tail -20
# Manual deployment
tail -f ${APP_HOME}/logs/server.log
A successful startup produces the following output (timestamps, version, and build number vary):
INFO [c.f.a.config.AcsAppConfig] (main)
==================================================
HTTP ports: 8080,8181
HTTPS ports: 8443
TR069 Device URLs:
NOAUTH: /ftacs/ACS
BASIC: /ftacs-basic/ACS
DIGEST: /ftacs-digest/ACS
WebServices:
/ACSServer-ACS/ACSWebService
/ACSServer-ACS/ACSWebServiceAdditional
/ACSServer-ACS/ACSWebServiceConfiguration
/ACSServer-ACS/ACSWebServiceEventSubscription
Swagger UI: /rest/swagger-ui
==================================================
INFO [c.f.i.c.s.FTIOTApplicationListener] (main) ==================== Friendly IOT Management Server V<version> Build <build>(<build-date>) started ====================
INFO [com.friendly.acs.FtAcsApp] (main) Started FtAcsApp in <seconds> seconds (process running for <seconds>)
The last two lines are the "ready" markers: the FTIOTApplicationListener banner reports the running version and build, and Started FtAcsApp confirms the process is up.
|
Liquibase migration messages go to |
|
The startup banner is produced by |
7.2. Endpoint Checks
# Application statistics -- HTTP Basic, use a web service user
curl -u <ws-user>:<ws-password> http://localhost:8080/acsstats
# TR-069 ACS endpoint -- POST only, so probe it with POST
curl -s -o /dev/null -w '%{http_code}\n' -X POST http://localhost:8080/ftacs/ACS
# SOAP WSDL -- HTTP Basic, use a web service user
curl -u <ws-user>:<ws-password> "http://localhost:8080/ACSServer-ACS/ACSWebService?wsdl"
# Swagger UI -- the entry point redirects, so follow the redirect
curl -sIL -o /dev/null -w '%{http_code}\n' http://localhost:8080/rest/swagger-ui
/acsstats and the SOAP WSDL must return HTTP 200.
/ftacs/ACS is POST-only (see HTTP Endpoints) — any status other than 404 means the endpoint is mapped and served; a curl -I HEAD probe returns 405 on a healthy deployment, which is not a fault.
/rest/swagger-ui redirects to the UI page, so it reaches 200 only when the redirect is followed (-L).
/acsstats and the SOAP web services sit behind the ftacsws HTTP Basic realm; their users are published by FT Configs Service, so a 401 means either wrong credentials or no published configuration — see FTACS Starts but Has No Configuration.
The web interface is reachable at http://localhost:8080 and https://localhost:8443.
8. Port Reference
Ports are written as published → container.
Across hosts you connect to the published port; inside the Docker bridge you connect to the container port.
All published ports are configurable through the .env variables shown in the compose file above.
8.1. HTTP / HTTPS
| Port | Protocol | Purpose | Exposure |
|---|---|---|---|
8080 → 8080 |
HTTP |
Main web interface: SOAP, REST, TR-069, statistics, bulk data ( |
Public (devices) |
8181 → 8181 |
HTTP |
Secondary HTTP port, the same services as 8080 ( |
Public (devices) |
8443 → 8443 |
HTTPS |
Secure web interface ( |
Public (devices) |
8.2. MQTT
| Port | Protocol | Purpose | Exposure |
|---|---|---|---|
1883 → 1883 |
MQTT |
MQTT broker, plain ( |
Public (devices) |
8883 → 8883 |
MQTT over TLS |
MQTT broker, TLS ( |
Public (devices) |
8.3. USP (TR-369)
| Port | Protocol | Purpose | Exposure |
|---|---|---|---|
8025 → 8025 |
TCP |
USP WebSocket, plain ( |
Public (devices) |
4430 → 4430 |
TCP |
USP WebSocket Secure, WSS ( |
Public (devices) |
18881 → 18881 |
TCP |
USP MQTT, plain ( |
Public (devices) |
18882 → 18882 |
TCP |
USP MQTT, TLS ( |
Public (devices) |
18891 → 18891 |
TCP |
USP Bulk Data receiver, plain ( |
Public (devices) |
61617 → 61617 |
TCP |
USP STOMP, plain ( |
Public (devices) |
61618 → 61618 |
TCP |
USP STOMP, TLS ( |
Public (devices) |
5783 → 5783 |
UDP |
USP CoAP, plain ( |
Public (devices) |
5784 → 5784 |
UDP |
USP CoAP, DTLS |
Public (devices) |
8.4. LWM2M
| Port | Protocol | Purpose | Exposure |
|---|---|---|---|
5680 → 5680 |
UDP |
Bootstrap server, no security ( |
Public (devices) |
5681 → 5681 |
UDP |
Bootstrap server, PSK ( |
Public (devices) |
5682 → 5682 |
UDP |
Bootstrap server, X.509 ( |
Public (devices) |
5880 → 5880 |
TCP |
Bootstrap server, no security ( |
Public (devices) |
5683 → 5683 |
UDP |
LWM2M server, no security ( |
Public (devices) |
5684 → 5684 |
UDP |
LWM2M server, PSK ( |
Public (devices) |
5685 → 5685 |
UDP |
LWM2M server, X.509 ( |
Public (devices) |
5883 → 5883 |
TCP |
LWM2M server, no security ( |
Public (devices) |
8.5. Outbound Connections
Ports FTACS dials on the existing infrastructure.
These are the published ports of those services — confirm them against whoever operates them, and set the matching variables in .env.
| Destination | Port | Protocol | Purpose |
|---|---|---|---|
Relational database |
3306 (Oracle: 1521) |
TCP |
ACS and Quartz schemas ( |
ClickHouse HTTP interface |
8123 |
HTTP |
QoE and statistics data ( |
Hazelcast cluster |
5701, 5711 |
TCP |
Distributed cache and the FT Configs Service configuration channel ( |
8.6. HTTP Endpoints
| Method | Path | Purpose | Auth |
|---|---|---|---|
GET |
|
TR-069 protocol statistics |
HTTP Basic (web service user) |
POST |
|
TR-069 endpoint |
None |
POST |
|
TR-069 endpoint |
HTTP Basic (device credentials) |
POST |
|
TR-069 endpoint |
HTTP Digest (device credentials) |
GET, POST |
|
Primary SOAP web service; append |
HTTP Basic (web service user) |
GET, POST |
|
Additional SOAP web service; append |
HTTP Basic (web service user) |
GET |
|
Swagger UI documentation interface |
None |
GET |
|
Prometheus metrics scrape endpoint |
None |
|
The TR-069 endpoint paths are configurable through |
Endpoint URL examples:
# No authentication http://<your-server>:8080/ftacs/ACS # Basic authentication http://<your-server>:8080/ftacs-basic/ACS # Digest authentication (HTTPS) https://<your-server>:8443/ftacs-digest/ACS # SOAP web service http://<your-server>:8080/ACSServer-ACS/ACSWebService?wsdl # Swagger UI http://<your-server>:8080/rest/swagger-ui
|
Plain-text ports (no encryption): 1883, 5680—5685, 5783, 5880, 5883, 8025, 8080, 8181, 18881, 18891, 61617. Encrypted ports: 4430 (WSS), 5784 (CoAP DTLS), 8443 (HTTPS), 8883 (MQTT TLS), 18882 (USP MQTT TLS), 61618 (STOMP TLS). In production, firewall the plain-text ports and route device traffic through the encrypted endpoints. |
9. Stack Management
9.1. Logs
| Command | Description |
|---|---|
|
Follow the container log of the |
|
Print the last 100 lines of the container log |
|
Follow the main application log in a manual deployment |
# Follow logs for one service
docker compose logs -f ftacs
# Last 100 lines
docker compose logs --tail 100 ftacs
# Manual deployment
tail -f ${APP_HOME}/logs/server.log
The application writes several log files to ftacs/logs/ (Docker) or logs/ (manual):
| Log file | Purpose |
|---|---|
|
Main application log |
|
Web service calls and responses |
|
Database migration logs |
|
SQL query execution logs |
|
Bulk operation processing |
|
System monitoring events |
|
Database session management |
|
External system integration traces |
|
Data cleanup operations |
|
Connection pool diagnostics |
9.2. Start, Stop, Restart
| Command | Description |
|---|---|
|
Show the status of every service in the stack |
|
Stop the |
|
Start the stopped |
|
Restart the |
|
Stop and remove all containers of the stack |
cd /usr/local/ft-system
docker compose ps # status of every service
docker compose stop ftacs # stop one service
docker compose start ftacs # start it again
docker compose restart ftacs # restart it
docker compose down # stop and remove all containers
|
|
9.3. Shell Access
| Command | Description |
|---|---|
|
Open an interactive shell inside the |
|
Run a single command inside the container without a shell session |
docker exec -it ftacs bash
docker exec ftacs ls -la /opt/FTACS/logs
9.4. Updating FTACS
See Docker Image Management Guide for the full image lifecycle.
cd /usr/local/ft-system
# 1. Back up the FTACS configuration and logs
tar -czf ftacs-backup-$(date +%Y%m%d).tar.gz ./ftacs
# 2. Update the image tag in compose.yml, then:
docker compose pull ftacs
docker compose up -d ftacs
Liquibase migrates the database schema automatically on the first start of the new version.
|
Back up the ACS database before pulling a new image — the Liquibase migrations are applied on first start and are not reversible from within FTACS. The database backup is taken on the database host, which is outside this stack. |
For manual deployments, the upgrade procedure is:
-
Back up the database, configuration files, and logs.
-
Check for a data-backfill migration — see Upgrading a Large Installation. On a large installation this step is mandatory and is performed before stopping the application.
-
Stop the application — run
./stop.shfrombin/. -
Extract the new version into a new directory.
-
Migrate the configuration — copy and adapt
app.envfrom the old version; review new options. -
Start the new version — Liquibase migrates the schema automatically.
-
Verify device connections, web services, and monitoring dashboards.
-
Roll back if needed — stop the new version, restore the database backup, start the old version.
9.5. Upgrading a Large Installation
Most Liquibase changeSets are schema changes and complete instantly regardless of installation size. A data-backfill changeSet is different: it rewrites existing rows, and its cost scales with the table. Release notes flag such releases explicitly; before upgrading one, check the size of the affected table:
SELECT COUNT(*) FROM cpe;
Up to roughly 1 million rows there is nothing to prepare — the backfill completes in seconds during startup. Above that, run the release’s pre-upgrade backfill script while the ACS is still on the old version. The script applies exactly the same predicate as the migration, in primary-key batches, so afterwards the changeSet finds nothing to do and only pays for a table scan.
Rows in cpe |
Migration without the pre-upgrade script | Migration after it |
|---|---|---|
1 million |
5.8 s |
0.3 s |
5 million |
30.4 s |
1.5 s |
25 million |
~2.5 min (extrapolated) |
~8 s |
|
On a large installation, skipping the pre-upgrade script does not merely make the upgrade slow — it takes the service down.
|
|
FTACS starts Liquibase in strict mode ( |
9.5.1. Release 7.1.2 (DEV-2749) — location_id Backfill
Release 7.1.2 is such a release: it backfills cpe.location_id and cpe_credentials.isp_id.
|
The backfill is irreversible — the previous values are preserved nowhere, in neither the manual script nor the migration. Before running either one:
|
Pre-upgrade backfill script
The backfill is delivered as a single script, dev2749-backfill.sh, which does the same work as
the migration but in batches of existing rows: each statement locks only its own range, and between
batches the table is free.
It is safe to re-run and safe to interrupt — a repeat run picks up whatever is left.
It is published on the FT shared drive, next to the release it belongs to, as ACS/dev2749-backfill.sh:
-
FTDISK — prerelease / ACS / ACS v.7.1.2 (Friendly Technologies staff; the script is in the
ACSsubfolder) -
No access to the shared drive? Request the script from Friendly Technologies support, quoting DEV-2749.
Copy it to a host that can reach the database and make it executable:
chmod +x dev2749-backfill.sh
Requirements: the mysql client for MySQL, or sqlplus for Oracle, on that host.
The password is read from the DB_PASSWORD environment variable, or prompted for — it is never
passed on the command line, where it would be visible in ps and in the shell history.
Step 1 — report only, changes nothing:
# MySQL
DB_PASSWORD=secret ./dev2749-backfill.sh --check --engine mysql \
--host db.example.com --port 3306 --user ftacs --database ftacs
# Oracle
DB_PASSWORD=secret ./dev2749-backfill.sh --check --engine oracle \
--host db.example.com --port 1521 --user ftacs --service FREEPDB1
It prints the row count of cpe and the category breakdown for both tables — the same categories
as Verification queries.
Category A plus category B is the number of rows the upgrade would rewrite.
Step 2 — run the backfill (same options, --run instead of --check), with the ACS still on
the old version:
DB_PASSWORD=secret ./dev2749-backfill.sh --run --engine mysql \
--host db.example.com --port 3306 --user ftacs --database ftacs
--run asks for an explicit yes before touching anything, then:
-
exports every row it is about to change to
<outdir>/cpe-before-backfill.tsv— keep this file, it is the only record of the previous values; -
backfills both tables in batches, printing progress;
-
prints the category breakdown again, where A and B must both be
0.
Everything it prints also goes to <outdir>/run.log.
By default <outdir> is ./dev2749-<timestamp>; use --outdir to place it elsewhere, and
--batch to change the batch size (default 50000, which keeps each statement at roughly a quarter
of a second).
|
The script is safe on a small installation too — it simply reports that the migration will complete
on its own and, in |
Verification queries
Run before the upgrade to size the work and record the previous values, and after it to confirm the outcome. Portable as written — works on both MySQL 8 and Oracle.
-- 1. cpe.location_id - one row per category
SELECT 'A null (heals to 0)' AS category, COUNT(*) AS rows_affected FROM cpe c
WHERE c.location_id IS NULL
UNION ALL
SELECT 'B poisoned (heals to 0)', COUNT(*) FROM cpe c
WHERE c.location_id IS NOT NULL AND c.location_id != 0
AND NOT EXISTS (SELECT 1 FROM isp i WHERE i.id = c.location_id)
AND EXISTS (SELECT 1 FROM product_class_group g WHERE g.id = c.location_id)
UNION ALL
SELECT 'C real domain (untouched)', COUNT(*) FROM cpe c
WHERE c.location_id IS NOT NULL AND c.location_id != 0
AND EXISTS (SELECT 1 FROM isp i WHERE i.id = c.location_id)
AND NOT EXISTS (SELECT 1 FROM product_class_group g WHERE g.id = c.location_id)
UNION ALL
SELECT 'D orphaned domain (untouched)', COUNT(*) FROM cpe c
WHERE c.location_id IS NOT NULL AND c.location_id != 0
AND NOT EXISTS (SELECT 1 FROM isp i WHERE i.id = c.location_id)
AND NOT EXISTS (SELECT 1 FROM product_class_group g WHERE g.id = c.location_id)
UNION ALL
SELECT 'E already super domain', COUNT(*) FROM cpe c
WHERE c.location_id = 0
UNION ALL
SELECT 'F AMBIGUOUS - needs review', COUNT(*) FROM cpe c
WHERE c.location_id IS NOT NULL AND c.location_id != 0
AND EXISTS (SELECT 1 FROM isp i WHERE i.id = c.location_id)
AND EXISTS (SELECT 1 FROM product_class_group g WHERE g.id = c.location_id);
-- 2. cpe_credentials.isp_id - same categories (isp_id is NOT NULL DEFAULT 0)
SELECT 'B poisoned (heals to 0)' AS category, COUNT(*) AS rows_affected FROM cpe_credentials cc
WHERE cc.isp_id != 0
AND NOT EXISTS (SELECT 1 FROM isp i WHERE i.id = cc.isp_id)
AND EXISTS (SELECT 1 FROM product_class_group g WHERE g.id = cc.isp_id)
UNION ALL
SELECT 'C real domain (untouched)', COUNT(*) FROM cpe_credentials cc
WHERE cc.isp_id != 0
AND EXISTS (SELECT 1 FROM isp i WHERE i.id = cc.isp_id)
AND NOT EXISTS (SELECT 1 FROM product_class_group g WHERE g.id = cc.isp_id)
UNION ALL
SELECT 'D orphaned domain (untouched)', COUNT(*) FROM cpe_credentials cc
WHERE cc.isp_id != 0
AND NOT EXISTS (SELECT 1 FROM isp i WHERE i.id = cc.isp_id)
AND NOT EXISTS (SELECT 1 FROM product_class_group g WHERE g.id = cc.isp_id)
UNION ALL
SELECT 'E already super domain', COUNT(*) FROM cpe_credentials cc
WHERE cc.isp_id = 0
UNION ALL
SELECT 'F AMBIGUOUS - needs review', COUNT(*) FROM cpe_credentials cc
WHERE cc.isp_id != 0
AND EXISTS (SELECT 1 FROM isp i WHERE i.id = cc.isp_id)
AND EXISTS (SELECT 1 FROM product_class_group g WHERE g.id = cc.isp_id);
-- 3. Context: how large are the two id spaces, and do they overlap?
SELECT 'isp rows' AS metric, COUNT(*) AS value FROM isp
UNION ALL
SELECT 'product_class_group rows', COUNT(*) FROM product_class_group
UNION ALL
SELECT 'overlapping ids', COUNT(*) FROM isp i
WHERE EXISTS (SELECT 1 FROM product_class_group g WHERE g.id = i.id);
-- 4. The rows that are about to change - EXPORT THIS BEFORE UPGRADING.
-- Their previous values are preserved nowhere else.
SELECT c.id, c.serial, c.location_id, c.product_class_id, c.protocol_id, c.updated
FROM cpe c
WHERE c.location_id IS NULL
OR (c.location_id != 0
AND NOT EXISTS (SELECT 1 FROM isp i WHERE i.id = c.location_id)
AND EXISTS (SELECT 1 FROM product_class_group g WHERE g.id = c.location_id))
ORDER BY c.id;
Expected after the upgrade: A = 0, B = 0, C and D unchanged, E = Ebefore + Abefore + Bbefore, F unchanged.
After the upgrade
-
Re-run the verification queries. Categories A and B must both be
0. -
Verify device connections and domain assignment in the UI.
-
If the ACS failed to start with
Could not acquire change log lock, the previous attempt was interrupted mid-migration. Clear the lock and start again:DELETE FROM DATABASECHANGELOGLOCK;
Known limitation
A poisoned value that happens to equal a real domain id cannot be told apart from a legitimate assignment: both id sequences are independent and both start low, so such collisions are common rather than exceptional. Those rows are reported as category F by the verification queries, and neither the migration nor the runtime fix touches them — they need manual review.
10. Production Checklist
-
Change all default passwords in
.env/app.env(DB_PASSWORD,CLICKHOUSE_PASWORD— single "S", see Environment Configuration — andSSL_KEYSTORE_PASSWORD). -
Use a valid SSL certificate — replace the self-signed
ftacs.keystorewith a CA-issued certificate. -
Configure firewall rules — allow only the required ports; restrict database access to ACS hosts only.
-
Review TR-069 authentication — disable the no-auth endpoint (
/ftacs/ACS) if it is not required. -
Secure configuration files — restrict permissions:
chmod 600 bin/app.env. -
Enable auto-restart —
restart: unless-stoppedincompose.yml, or--restart unless-stoppedwithdocker run. -
Verify health checks — the compose file defines them for every service; add
--health-cmdfor standalonedocker run. -
Set up monitoring — deploy Prometheus and Grafana against
/actuator/prometheusand the ClickHouse data source. -
Configure backups — regular database backups, a configuration directory backup, a ClickHouse backup strategy, and log rotation.
-
Deploy multiple FTACS instances for high availability — identical configuration, an external Hazelcast cluster with 2+ nodes, and a load balancer in front.
-
Tune JVM settings — match the JVM sizing table to your deployment size.
11. Troubleshooting
11.1. Container Fails to Start
Symptom: The ftacs container exits immediately after docker compose up.
Fix:
-
Read the error message from the logs:
docker compose logs ftacs docker events --filter container=ftacs -
Confirm the environment files are loaded:
docker exec ftacs env | grep DB_ -
Confirm
ftacs/config/exists and is populated — an empty mount at/opt/FTACS/confis a common cause.
11.2. Database Connection Failure
Symptom: Startup fails with a JDBC connection error, or Liquibase cannot acquire a lock.
Fix:
-
Test connectivity to the database. It is external to this stack and addressed by IP through
DB_HOST(see Startup Dependencies), so there is nomysqlcontainer to reach or to read logs from — probe the address from the host and read the JDBC error in the FTACS log:# use the DB_HOST / DB_PORT values from .env nc -zv <db-host> <db-port> docker logs ftacs 2>&1 | grep -i -E 'jdbc|liquibase|hikari' -
Verify
DB_HOST,DB_PORT,DB_USER, andDB_PASSWORDin.envmatch the database configuration. -
If the database was just created, wait for its health check to pass before starting FTACS.
11.3. ClickHouse Connections Fail
Symptom: QoE and statistics queries fail; the log reports a ClickHouse connection or protocol error.
Fix:
-
Confirm FTACS targets the HTTP port
8123, not the native TCP port9000— FTACS uses theclickhouse-javaHTTP client. In the compose file this is theCLICKHOUSE_PORT: "$\{CLICKHOUSE_HTTP_PORT:-8123}"remap. -
Verify ClickHouse answers, from inside the container so the check uses the same network path FTACS does:
docker exec ftacs curl -s http://${CLICKHOUSE_HOST}:8123/ping -
If ClickHouse answers but federated queries against the relational database fail, the JDBC Bridge on the ClickHouse host is the likely cause — report it to whoever operates that stack.
11.4. FTACS Starts but Has No Configuration
Symptom: FTACS reaches Started FtAcsApp, but web service calls are rejected as unauthorised, ACS configuration values fall back to defaults, and the parameter-name cache rules have no effect. No error appears in server.log.
Cause: since 6.5.2 this configuration is owned by FT Configs Service and delivered through Hazelcast. FTACS finds no published snapshot — either the service is not running, or the two are on different Hazelcast clusters.
Fix:
-
Verify FT Configs Service is healthy:
curl -s http://<configs-service-host-ip>:8087/configs-service/actuator/health -
Compare
HZ_MEMBERSin FTACS’s.envwith the value used by FT Configs Service — both must resolve to the same cluster. -
Compare the cluster name in
ftacs/config/hazelcast-client.xmlwith the one FT Configs Service uses (default:dev). A mismatch produces two separate clusters that never see each other. -
Confirm the configuration was actually saved in FT Configs UI — an empty service publishes empty snapshots, which looks identical from the FTACS side.
-
Do not work around this by putting
acs_configuration.xmland friends back intoftacs/config/— FTACS no longer reads them from disk.
11.5. Hazelcast Cluster Issues
Symptom: The log shows Unable to connect to any address in the config or repeated Hazelcast client reconnects.
Fix:
-
Verify
HZ_MEMBERSin.envlists addresses this host can actually reach. The Hazelcast cluster is a prerequisite deployed outside this guide (see Prerequisites), so use the member host IPs —localhostor a bridge alias only works when a member runs on this host and on the same Docker network. -
Ensure ports 5701 and 5711 are open from this host to every Hazelcast member.
-
Check that the cluster name in
ftacs/config/hazelcast-client.xml— the only Hazelcast file this installation owns — matches the name the cluster was started with (default:dev). The member-sidehazelcast.xmlis not part of this installation; ask whoever operates the cluster.
11.6. Port Already in Use
Symptom: docker compose up fails with bind: address already in use.
Fix:
-
Find the process holding the port:
sudo lsof -i :8080 netstat -tulpn | grep 8080 -
Either stop that process, or change the published port through the corresponding
.envvariable (for exampleFTACS_HTTP_PORT=9080) and rundocker compose up -dagain.
11.7. Permission Denied on Mounted Volumes
Symptom: The container cannot write to ftacs/logs or the database cannot write to its data directory.
Fix:
sudo chown -R $(whoami):$(whoami) /usr/local/ft-system
chmod -R 755 /usr/local/ft-system
11.8. Out of Memory
Symptom: java.lang.OutOfMemoryError in server.log, or the container is killed by the OOM killer.
Fix:
-
Raise the container memory limit (Docker Desktop: Settings → Resources → Memory).
-
Increase the heap through
FT_ACS_JAVA_RAMin.env(Docker) orJAVA_OPTSinapp.env(manual), matching the JVM sizing table. -
If the problem persists, capture a heap dump and analyse it for leaks.
11.9. Containers Not Reachable From the Host
Symptom: The container is healthy and answers on localhost inside the container, published ports look correct (0.0.0.0:PORT→PORT), but requests from the host hang and time out — no ECONNREFUSED, only timeouts.
Cause: the host firewall blocks traffic on Docker bridge interfaces (br-*) in the host INPUT/OUTPUT path. This is not a Compose or application issue.
Fix:
-
Allow host-to-container traffic on Docker bridges:
# IPv4 (all Docker bridges) iptables -I INPUT 1 -i br+ -j ACCEPT iptables -I OUTPUT 1 -o br+ -j ACCEPT # IPv6 requires separate rules ip6tables -I INPUT 1 -i br+ -j ACCEPT ip6tables -I OUTPUT 1 -o br+ -j ACCEPT # Persist (Debian/Ubuntu) apt-get install -y iptables-persistent netfilter-persistent save -
Confirm the diagnosis step by step:
# 1. Container works internally docker exec ftacs curl -s -o /dev/null -w '%{http_code}\n' http://localhost:8080 # 2. Container IPv4 address docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' ftacs # 3. Host -> container IP nc -vz <container_ipv4> 8080 # 4. Identify the Docker bridge ip -4 addr | grep '^.*br-' # 5. Inspect the firewall iptables -S INPUT iptables -S OUTPUT
|
IPv4 working does not imply IPv6 works — they are separate firewall stacks. Docker programs NAT and FORWARD rules but does not override host INPUT/OUTPUT policies. |
11.10. Quartz Cluster Recovery Messages
Symptom: ClusterManager: detected failed or restarted instances appears during startup.
Fix: No action needed. This is normal when the application restarts or a previous instance was not shut down cleanly; Quartz recovers jobs from failed instances automatically.
11.11. Application Fails to Start (Manual Deployment)
Symptom: start.sh returns but no process is listening; server.log is empty or shows an early failure.
Fix:
-
Verify Java:
java -versionmust report JDK 25. -
Check
JAVA_HOMEandAPP_HOMEinapp.env— on WindowsAPP_HOMEmust use forward slashes and a trailing slash (D:/ftacs/dist/). -
Verify the database is reachable:
telnet <DB_HOST> <DB_PORT>. -
Check the credentials in
app.env. -
Review
logs/liquibase.logfor SQL errors.
11.12. Slow Performance
Symptom: High response times, growing session backlog, or CPE session timeouts.
Fix:
-
Check relational database query performance and index health.
-
Verify Hazelcast cache hit ratios.
-
Review the HikariCP pool sizes in
ftacs/.env(FTACS_MAX_POOL,QUARTZ_MAX_POOL,CLICKHOUSE_MAX_POOL). -
Measure network latency to the database and to Hazelcast.
-
Increase the JVM heap per the JVM sizing table.
-
Inspect the Grafana dashboards fed by
/actuator/prometheus.
11.13. Getting Support
If the issue persists, collect the following and contact Friendly Tech support:
-
.envandftacs/.env(orapp.env) with passwords masked. -
logs/server.log— the last 1000 lines. -
logs/liquibase.logif the problem is database-related. -
java -versionoutput. -
Operating system version.
-
Database type and version.
-
docker versionoutput, if applicable.
12. Related Documentation
-
Configuration Guide — Environment Variables — every FTACS environment variable and configuration item.
-
Architecture Overview — Infrastructure & Dependencies — how FTACS relates to the database, ClickHouse, Hazelcast, and FT Configs Service.
-
All in one server deployment — the whole stack on a single host.
-
Separate server deployment — Server C — FTACS and FT Configs Service on one host of a split topology.
-
FT Configs deployment — the service that owns the ACS runtime configuration since 6.5.2.
-
FTACS Environment Variables — the platform-wide environment reference.
-
Docker Image Management Guide — pulling, tagging, and updating the images.