FT Device Network Service Separate Host Deployment

Overview

This deployment runs FT Device Network Service on a dedicated host. This is a Spring Boot microservice that provides WiFi device management and network configuration capabilities.

Architecture

ft-device-network-architecture

Prerequisites

For detailed information about Docker registry and image management, see Docker Image Management Guide.

Required External Components

The following components must be installed and accessible:

Component Purpose Default Port

Hazelcast

Distributed cache for device data

5701

Clickhouse

Analytics database for QoE data

9000

System Requirements

  • Docker Engine 20.10+

  • Docker Compose 2.0+

  • Minimum 1GB RAM

  • 5GB free disk space

Preparation

1. Create Data Directory

mkdir -p /usr/local/ft-services/ft-data/ft-device-network/logs

2. Prepare Environment File

cp .env.template .env

Or create .env file with the following content:

# Service specific vars
FT_DEVICE_NETWORK_HTTP_PORT=8383
FT_DEVICE_NETWORK_JAVA_RAM=-Xms256m -Xmx1g
FT_DEVICE_NETWORK_LOG_LEVEL=info

# Common vars
DATA_FOLDER=/usr/local/ft-services
HZ_MEMBERS=~HAZEL_HOST~:5701
TZ=Europe/Kiev

# Clickhouse
CLICKHOUSE_HOST=~CLICKHOUSE_HOST~
CLICKHOUSE_HTTP_PORT=8123
CLICKHOUSE_DATABASE=ftacs_qoe_ui_data
CLICKHOUSE_USERNAME=ftacs
CLICKHOUSE_PASSWORD=ftacs

3. Configure Host Variables

Edit .env and replace placeholder values:

# Required replacements:
~HAZEL_HOST~      → Hazelcast server (e.g., 192.168.1.50)
~CLICKHOUSE_HOST~ → Clickhouse server (e.g., 192.168.1.70)

Example configuration:

HZ_MEMBERS=192.168.1.50:5701,192.168.1.51:5701,192.168.1.52:5701
CLICKHOUSE_HOST=192.168.1.70

Deployment

Deploy Service

docker compose up -d

Verify Deployment

Check service status:

docker compose ps

Expected output:

NAME                        STATUS              PORTS
ft-device-network-service   healthy             0.0.0.0:8383->8080/tcp

Container Startup Process

The service starts with the following initialization sequence:

  1. JVM Initialization

    • Loads JVM settings from JAVA_OPTS environment variable

    • Memory allocation: 256MB initial, 1GB maximum (configurable)

    • G1 garbage collector enabled for optimized performance

    • Heap dump on OutOfMemoryError for troubleshooting

  2. Application Startup

    • Initializes Spring Boot application

    • Connects to Hazelcast cluster for distributed caching

    • Connects to Clickhouse for analytics data storage

    • Registers REST API endpoints

  3. Health Check

    • Actuator health endpoint becomes available at /api/actuator/health

    • Service is marked as healthy after successful startup

    • Health check runs every 30 seconds

Environment Variables

Service Configuration

Variable Description Default Required

FT_DEVICE_NETWORK_HTTP_PORT

Service HTTP port (external)

8383

Yes

FT_DEVICE_NETWORK_JAVA_RAM

JVM memory settings (min and max heap)

-Xms256m -Xmx1g

No

FT_DEVICE_NETWORK_LOG_LEVEL

Application log level (trace, debug, info, warn, error)

info

No

Common Configuration

Variable Description Default Required

DATA_FOLDER

Base directory for persistent data

/usr/local/ft-services

Yes

TZ

Timezone for the service

Europe/Kiev

Yes

External Dependencies

Variable Description Default Required

HZ_MEMBERS

Hazelcast cluster members (comma-separated HOST:PORT)

HAZEL_HOST:5701

Yes

CLICKHOUSE_HOST

Clickhouse server address

CLICKHOUSE_HOST

Yes

CLICKHOUSE_PORT

Clickhouse port

9000

Yes

CLICKHOUSE_DATABASE

Clickhouse database name

ftacs_qoe_ui_data

Yes

CLICKHOUSE_USERNAME

Clickhouse username

ftacs

Yes

CLICKHOUSE_PASSWORD

Clickhouse password

ftacs

Yes

JVM Configuration Details

The JAVA_OPTS environment variable includes:

Option Description Purpose

-Xms256m

Initial heap size

Starting memory allocation

-Xmx1g

Maximum heap size

Memory limit (configurable via FT_DEVICE_NETWORK_JAVA_RAM)

-Duser.timezone=${TZ}

JVM timezone

Ensures correct timestamp handling

-XX:+UseG1GC

G1 garbage collector

Modern GC for better performance

-XX:+UseStringDeduplication

String deduplication

Reduces memory usage for duplicate strings

-XX:+HeapDumpOnOutOfMemoryError

Heap dump on OOM

Creates dump file for troubleshooting

-XX:HeapDumpPath=/app/logs/heapdump.hprof

Heap dump location

Stored in logs directory

Service Access

After successful deployment:

Endpoint URL Description

Health Check

http://<HOST>:8383/api/actuator/health

Service health status

Actuator Info

http://<HOST>:8383/api/actuator/info

Service information

API Documentation

http://<HOST>:8383/api/swagger-ui.html

Swagger UI (if enabled)

Exposed Ports

Port Service Protocol Purpose Configurable Via

8383

FT Device Network

HTTP

REST API

FT_DEVICE_NETWORK_HTTP_PORT

Verify port is listening:

netstat -tlnp | grep 8383
# or
ss -tlnp | grep 8383

Test service health:

curl http://localhost:8383/api/actuator/health

Expected response:

{
  "status": "UP"
}

Data Persistence

Persistent data location (assuming DATA_FOLDER=/usr/local/ft-services):

Path Content Backup Required

${DATA_FOLDER}/ft-data/ft-device-network/logs/

Application logs and heap dumps

No (unless debugging needed)

Folder structure:

/usr/local/ft-services/
└── ft-data/
    └── ft-device-network/
        └── logs/
            ├── application.log
            ├── spring.log
            └── heapdump.hprof (if OOM occurred)

Troubleshooting

Check Service Logs

# Via Docker Compose
docker compose logs -f

# Or direct file access
tail -f /usr/local/ft-services/ft-data/ft-device-network/logs/application.log

Common Issues

Service Not Starting

Check logs for errors:

docker compose logs ft-device-network-service

Common causes:

  1. Port already in use:

    • Change FT_DEVICE_NETWORK_HTTP_PORT in .env

    • Or find and stop the conflicting process

  2. Insufficient memory:

    • Reduce FT_DEVICE_NETWORK_JAVA_RAM values

    • Or allocate more RAM to Docker

Cannot Connect to Hazelcast

Symptom: Logs show Hazelcast connection timeouts or errors.

Check connectivity:

# Test Hazelcast reachability
telnet <HAZELCAST_HOST> 5701

# Check HZ_MEMBERS configuration
docker compose exec ft-device-network-service env | grep HZ_MEMBERS

Solution: - Verify HZ_MEMBERS addresses are correct in .env - Ensure Hazelcast cluster is running and accessible - Check firewall rules allow port 5701

Cannot Connect to Clickhouse

Symptom: Logs show Clickhouse connection errors.

Check connectivity:

# Test Clickhouse reachability
telnet <CLICKHOUSE_HOST> 9000

# Check Clickhouse configuration
docker compose exec ft-device-network-service env | grep CLICKHOUSE

Solution: - Verify CLICKHOUSE_HOST is correct in .env - Ensure Clickhouse is running and accessible - Verify credentials (CLICKHOUSE_USERNAME, CLICKHOUSE_PASSWORD) - Check database exists: CLICKHOUSE_DATABASE=ftacs_qoe_ui_data

Health Check Failing

Check health endpoint directly:

curl -v http://localhost:8383/api/actuator/health

If endpoint returns 503 or error: - Wait for application to fully initialize (can take 30-40 seconds) - Check logs for startup errors - Verify external dependencies (Hazelcast, Clickhouse) are accessible

OutOfMemoryError

Symptom: Service crashes with OOM error.

Check heap dump:

ls -lh /usr/local/ft-services/ft-data/ft-device-network/logs/heapdump.hprof

Solution: 1. Increase memory allocation:

FT_DEVICE_NETWORK_JAVA_RAM=-Xms512m -Xmx2g

  1. Restart service:

docker compose restart

  1. If issue persists, analyze heap dump or contact support

Verify External Dependencies

Test connectivity to all external services:

# Hazelcast
telnet <HAZELCAST_HOST> 5701

# Clickhouse
telnet <CLICKHOUSE_HOST> 9000

Restart Service

# Restart
docker compose restart

# Full restart (recreate container)
docker compose down
docker compose up -d

Debugging

Enable Debug Logging

Edit .env:

FT_DEVICE_NETWORK_LOG_LEVEL=debug

Restart service:

docker compose restart

Increase JVM Memory

For high-load environments, increase memory allocation:

FT_DEVICE_NETWORK_JAVA_RAM=-Xms512m -Xmx2g

Monitor Resource Usage

# Real-time container stats
docker stats ft-device-network-service

# Check heap usage via JMX (if exposed)
curl http://localhost:8383/api/actuator/metrics/jvm.memory.used

Analyze Heap Dump

If OOM occurred and heap dump was created:

# Copy heap dump for analysis
cp /usr/local/ft-services/ft-data/ft-device-network/logs/heapdump.hprof ./

# Analyze with tools like Eclipse MAT or VisualVM

View Detailed Logs

# Application logs
tail -f /usr/local/ft-services/ft-data/ft-device-network/logs/application.log

# Spring framework logs
tail -f /usr/local/ft-services/ft-data/ft-device-network/logs/spring.log

# Or via Docker
docker compose logs -f --tail=100

Maintenance

Update Service

# Pull latest image
docker compose pull

# Restart with new image
docker compose up -d

Force Repull Image

docker compose down
docker compose pull --ignore-pull-failures
docker compose up -d

Backup Logs

# Backup logs directory
tar -czf ft-device-network-logs-$(date +%Y%m%d).tar.gz \
  /usr/local/ft-services/ft-data/ft-device-network/logs/

# Optional: Compress old logs
cd /usr/local/ft-services/ft-data/ft-device-network/logs/
gzip application.log.* spring.log.*

Clean Old Logs

# Remove logs older than 30 days
find /usr/local/ft-services/ft-data/ft-device-network/logs/ \
  -name "*.log.*" -mtime +30 -delete

# Remove old heap dumps
find /usr/local/ft-services/ft-data/ft-device-network/logs/ \
  -name "heapdump*.hprof" -mtime +7 -delete

Stop Service

# Stop (keeps data)
docker compose down

# Stop and remove all data
docker compose down -v

Performance Tuning

Adjust Memory Based on Load

Monitor memory usage and adjust accordingly:

Load Level Recommended Setting Use Case

Low

-Xms256m -Xmx512m

Small deployments, <100 devices

Medium

-Xms512m -Xmx1g

Standard deployments, 100-1000 devices

High

-Xms1g -Xmx2g

Large deployments, 1000+ devices

Very High

-Xms2g -Xmx4g

Enterprise deployments, high concurrency

Monitor Performance Metrics

# JVM metrics
curl http://localhost:8383/api/actuator/metrics/jvm.memory.used
curl http://localhost:8383/api/actuator/metrics/jvm.gc.pause

# HTTP metrics
curl http://localhost:8383/api/actuator/metrics/http.server.requests

Related Documentation