FTGetDeviceParameters Optimization via Direct ACS Data Response SDD

This document describes the design and implementation strategy for optimizing the FTGetDeviceParameters API, reducing response time and database load by shifting from polling-based architecture to a real-time, event-driven approach.

1. Overview

This document describes the design and implementation strategy for optimizing the FTGetDeviceParameters API, reducing response time and database load by shifting from polling-based architecture to a real-time, event-driven approach.

2. Problem Statement

Currently, the FTGetDeviceParameters API performs the following steps:

  • Northbound API calls ACS’s getParameterDataListFromCPE.

  • ACS creates and stores processing tasks, returns a transaction ID.

  • The API then polls the database once per second to check the transaction state.

  • When the transaction state is marked COMPLETED, it begins polling for parameter values.

  • Once parameters are available, the API fetches them from the database and returns them to the client.

2.1. Drawbacks:

  • Excessive load on the database due to frequent polling.

  • Increased latency for end-users, especially in cases where parameter persistence to DB is delayed (can take up to 3 minutes).

  • Poor resource utilization on both API and DB layers.

3. Objective

Eliminate polling and significantly reduce response latency by enabling ACS to proactively deliver parameter data directly to the API once received from the device.

4. Proposed Solution

Implement a real-time, event-driven delivery mechanism using in-memory messaging and direct completion of API requests.

5. ACS Implementation Details

5.1. Functional Changes:

  • Introduce a new method getParameterDataListDirectlyFromCPE in ACS WS.

  • Always create three task types for the requested parameter names:

    • GetParameterValues

    • GetParameterNames

    • GetParameterAttributes

  • Use CompletableFuture to track data delivery per transaction.

  • Store interim device data in a Hazelcast IMap (retrieveParameterTransactionData).

  • Use Hazelcast topic (retrieveTaskTransactionTopic) to broadcast task completions.

  • On receipt of GetParameterValuesResponse:

    • Immediately respond to the waiting API with parameter data.

  • If push fails or no data is received within 60 seconds:

    • Complete the request with a SOAP Fault.

  • Always set task priority = HIGHEST (1).

  • Always execute push command.

  • Default cpeStatusCheckTimeout = 10s if not provided.

5.2. Task Execution Order:

  • New Order: GetParameterValues → GetParameterNames → GetParameterAttributes

5.3. Multi-node Logic:

  • If the ACS node receiving the device session is different from the one handling the API call:

    • Use Hazelcast topic to send device parameter data across nodes.

6. API Integration Requirements

To ensure backward compatibility and prevent breaking changes for existing clients, you must preserve the legacy method interfaces.

6.1. Functional Changes:

  • Introduce a new method FTGetDeviceParametersWithTasks that fully replicates the existing polling-based logic.

  • FTGetDeviceParameters: switch to use the new ACS method getParameterDataListDirectlyFromCPE in case of source=0

  • getParameterDataListDirectlyFromCPE accepts:

    • cpeId (required)

    • parameterNameList (required)

    • user (optional)

    • cpeStatusCheckTimeout (optional)

  • Only one device per request is supported.

  • The API should not query or track task status in the database.

  • The API must only wait for the ACS to return either:

    • updated parameters from the device, or

    • an error response (timeout or push failure).

  • Upon receiving a response from ACS, the API returns it immediately to you.

6.2. Configuration:

New config flag to toggle between old polling logic and new direct response logic.

<FTGetDeviceParametersLogic Description="Defines which ACS method is used by 'FTGetDeviceParameter' method. The 'getParameterDataListDirectlyFromCPE' method is used when the parameter is set to 'Directly' or 'getParameterDataListFromCPE' when the parameter is set to 'Polling'. Default value: 'Directly'">Directly</FTGetDeviceParametersLogic>

7. Web Service API Changes

7.1. a. Old ACSWS Method

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
                  xmlns:ftac="http://ftacs.com/">
  <soapenv:Header/>
  <soapenv:Body>
    <ftac:getParameterDataFromCPE>
      <cpeList>
        <id>?</id>
      </cpeList>
      <paramList>
        <dataList>
          <string>?</string>
        </dataList>
        <parameterName>?</parameterName>
      </paramList>
      <priority>?</priority>
      <push>?</push>
      <user>?</user>
      <cpeStatusCheckTimeout>?</cpeStatusCheckTimeout>
    </ftac:getParameterDataFromCPE>
  </soapenv:Body>
</soapenv:Envelope>

Returns: TransactionIdResponse

7.2. b. New ACSWS Method

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
                  xmlns:ftac="http://ftacs.com/">
  <soapenv:Header/>
  <soapenv:Body>
    <ftac:getParameterDataListDirectlyFromCPE>
      <cpeId>?</cpeId>
      <parameterNameList>
        <string>?</string>
      </parameterNameList>
      <user>?</user>
      <cpeStatusCheckTimeout>?</cpeStatusCheckTimeout>
    </ftac:getParameterDataListDirectlyFromCPE>
  </soapenv:Body>
</soapenv:Envelope>

Returns: CpeParamListWS or SOAP Fault.

8. Sequence Diagrams

8.1. a. Existing System (DB Polling-Based)

  • Refer to diagram: schema 1

  • API polls DB for task status and updated parameters.

Sequence diagram showing existing DB polling-based FTGetDeviceParameters flow

8.2. b. Optimized System - Single Node

  • Refer to diagram: schema 2

  • ACS replies directly to API when data is ready.

Sequence diagram showing optimized single-node direct response flow

8.3. c. Optimized System - Multi-Node

  • Refer to diagram: schema 3

  • Uses Hazelcast pub/sub when device session and API request land on different ACS nodes.

Sequence diagram showing optimized multi-node flow with Hazelcast pub/sub

9. Backward Compatibility

  • The updated API remains backward-compatible.

  • A configuration flag allows switching between old polling mode and new async mode.

10. Testing Strategy

  • Functional Tests:

    • Validate data delivery correctness in both single-node and cross-node setups.

  • Load Testing:

    • Simulate up to 20 million daily requests.

    • Monitor latency and system resource utilization.

  • Fault Injection:

    • Simulate device offline, timeout, and partial response scenarios.

11. Benefits

  • Near real-time* response to API requests.

  • Elimination of database polling.

  • Reduced system load.

  • Better user experience via faster and deterministic responses.

*real-time definition — up to 1 second after receiving a response from the device.

12. Future Enhancements

  • Support for partial/streamed parameter delivery.

  • Integration with observability stack for end-to-end request tracing.

  • Extend Hazelcast TTL strategy to auto-clean stale transaction data.