Troubleshooting

1. Common Issues

1.1. Database Connection Failures

Symptom Application fails to start with connection refused errors

Cause

Database is unreachable or credentials are incorrect

Solution

  • Verify that MAIN_DB_URL, UI_DB_URL, and CLICKHOUSE_DB_URL are correct

  • Confirm database services are running and accepting connections

  • Check that MAIN_DB_USERNAME / MAIN_DB_PASSWORD and corresponding UI DB credentials are valid

1.2. Hazelcast Cluster Connection

Symptom Warnings about Hazelcast client unable to connect

Cause

Hazelcast cluster is not running or address is misconfigured

Solution

  • Verify HAZELCAST_CLUSTER_NAME and HAZELCAST_ADDRESS are correct

  • Ensure Hazelcast cluster members are running and accessible from the application host

  • Check firewall rules for Hazelcast ports (default: 5701)

1.3. ClickHouse Query Timeouts

Symptom Slow or timed-out responses on Dashboard, Device Groups, or Reports pages

Cause

Large data volume in ClickHouse or missing aggregated tables

Solution

  • Review the Keep data (days) setting in Settings > General to limit data retention

  • Verify that kpi_data_aggregated and kpi_data_latest tables are being populated

  • Check ClickHouse server performance and resource allocation

1.4. Devices Missing While the Total Keeps Counting Them

Symptom The Device list shows fewer rows than the filtered from N total figure printed next to it, and newly registered devices never appear

Cause

ft_cpe_info in ClickHouse is filled from the main database by the ClickHouse JDBC Bridge, and the sync takes MAX(created) from that table as its watermark. When the bridge and ClickHouse disagree on the time zone, rows arrive shifted by the local UTC offset — the watermark moves into the future and everything registered inside that window is skipped. The count next to the table comes from the main database, which still sees those devices.

Solution

  • Confirm the shift from ClickHouse — the two values must match:

    SELECT now(), (SELECT * FROM jdbc('ftacs','SELECT NOW()'));

  • Align the zone in all four places (JDBC_BRIDGE_TZ, JDBC_BRIDGE_JVM_OPTS, "timezone" / serverTimezone in the bridge datasource file, and ClickHouse <timezone>) — see JDBC Bridge Time Zone

  • ft_cpe_info is append-only, so rows already stored ahead of the clock keep the watermark in the future: run TRUNCATE TABLE ft_cpe_info once after the zones agree, and the sync refills it within a minute

1.5. Quartz Scheduler Issues

Symptom Scheduled tasks are not executing

Cause

Quartz DB connection issues or job store misconfiguration

Solution

  • Verify Quartz datasource configuration in application.yml

  • Check that the Quartz DB tables exist and are accessible

  • Review application logs for Quartz-related errors

2. Logging and Debugging

2.1. Log Configuration

The application uses SLF4J with Logback. Log levels can be adjusted via application.yml or environment variables:

logging:
  level:
    com.friendly.tr069.ftacs.qoe.monitoring.web: DEBUG
    org.springframework: INFO
    org.hibernate: WARN

2.2. Remote Debugging

When running with Docker, a debug port is exposed on 5005:

# Connect with a remote debugger on port 5005

For local development:

./gradlew bootRun --debug-jvm

2.3. Useful Log Patterns

  • Alarm processing — Search for AlarmProcessor or ThresholdProcessor in logs

  • Data sync tasks — Search for SyncTask or QoeCpeInfoSyncTask

  • JMS messages — Search for JmsConsumer or MessageSender

  • Cache operations — Search for CpeCache or KpiCache

← Back | Main Page