Messaging

Both sides agree on this exact, closed list — no other string is ever placed in FccCommandAckDto.reason for a REJECTED result:

For an FccActivationEventDto of type ACTIVATION_FAILED, the equivalent closed vocabulary for reason is ACS_RESTART, ACS_UNAVAILABLE, INTERNAL.

On this service’s side the same vocabulary is FccAckReasonEnum, plus one constant the ACS side does not need: UNKNOWN. FccAckReasonEnum.fromWire normalizes the wire string and falls back to UNKNOWN for anything it does not recognize — a new reason added to a future ACS release is recorded and logged, never thrown on. FccAckService.reject stores the unwrapped reason (truncated to 255 chars) in fcc_activation.failure_reason, not the enum name, so the exact ACS text survives for a report; the enum is used for the closed-vocabulary check and the structured log line only. For a plain REJECTED the unwrapped reason is the raw wire string, so nothing about that case changed; for a DUPLICATE it is what the wire wrapper carried, so a reason of REJECTED:INVALID_CONFIG leaves INVALID_CONFIG in the column and the REJECTED: wrapper itself is never stored — see DUPLICATE Carries the Original Outcome. A null reason on the wire is stored as the literal UNKNOWN, and so is a DUPLICATE reading a bare REJECTED with no suffix. reject is guarded by the same source states as accept (PLANNED, DISPATCHING): a REJECTED for a run that has already started or already been closed is logged at INFO with the activation id, its state and the reason, and changes nothing. The concrete case is a lost ack — the ACS started the ACTIVATE, the row stayed SENT, the timeout poller republished the identical payload, and an ACS whose idempotency memory has expired answers REJECTED(STALE_SEQ) for a run that is live and streaming results.

Contract 1.3 added FccCommandAckDto.commandSeq — the seq of the command the ack is answering. An ack is therefore correlated against the fcc_acs_outbox row on both fields: the idempotencyKey finds the row, and the commandSeq says which send of that key this ack belongs to. The key alone cannot do that: it names the activation, not a particular send, so with the key alone every ack that arrives looks like the answer to the send currently outstanding — including one that has been sitting in the queue since a command this service has long since moved past.

DUPLICATE means "I have already processed this idempotencyKey`" — it says nothing, by itself, about whether that original processing succeeded. Since contract 1.3 the `reason field carries the original outcome and this service unwraps it and applies that outcome, instead of treating every DUPLICATE as an acceptance:

Matching is case-insensitive and both the whole reason and the part after the colon are trimmed, so the unwrap does not turn on the ACS’s spacing or casing.

The ACS saves fcc_sa_activation (+ devices), schedules FccSaActivationJob, and acks ACCEPTED immediately after saving and scheduling — not after any device has actually started. ACTIVATION_STARTED is emitted at the job’s first tick; ACTIVATION_FINISHED when every iteration has run and no device remains SENT; ACTIVATION_FAILED(reason) on an ACS restart with no surviving job (ACS_RESTART) or an unrecoverable error.

Window: the ACS computes N = totalDurationH * 60 / singleDurationMin iterations and startAt = max(now, windowStartAt). window_end_at on the ACS side is informational only; completion is decided by iterations_done == iterations_total and no device in flight. On this service’s side, FccAckService.accept recalculates windowEndAt from the actual start the moment an ACCEPTED ack — or a DUPLICATE that unwraps to an acceptance, DUPLICATE Carries the Original Outcome — lands (FccActivationMapper.windowEndAt(startedAt, totalDurationH)) — this part is implemented and code-read. The two records therefore diverge by the dispatch latency, and that is the contract, not a bug: the command already on the ACS still carries the original window, the ACS does not decide completion from it, and this service’s watchdog and reports use the re-anchored value so that a command which waited in the outbox still gets its full measurement window.

Since M2 the same re-anchoring also happens on ACTIVATION_STARTED, and it is applied once: the ack and the event travel on different queues, either may arrive first, and letting the second one move started_at would shift the window the watchdog and every report read (FccActivationTransitionService, SET_STARTED_AT only when the column is still null).

A watchdog that actively closes a run whose window has passed is implemented as of M2: FccActivationWatchdog / FccActivationClosingService claims RUNNING rows past window_end_at + ft.fcc.watchdog.finish-grace and closes them as FINISHED(WATCHDOG) — not as failures, because the window really did run to its end, nobody reported a fault, and the measurements taken inside it are valid; only the stop reason records that the closing decision was local. The campaign then gets the same ACTIVATION_FINISHED a reported completion would have produced, which is what plans the next daily reactivation — without it a single lost control message would end the campaign. That close is logged at WARN and not INFO, because it means a control event was lost between the ACS and here. See What the Watchdog Closes, and With Which Grace and The Five Pollers.

The ACS removes FccSaActivationJob, sets next_action = NONE on every device of the activation, cancels its own cpe_pending_task rows by transaction_id, and acks ACCEPTED immediately; it then emits ACTIVATION_STOPPED{inFlightCount} where inFlightCount is the number of devices still SENT at that moment. Results for those in-flight devices still arrive normally, each carrying afterStop = true. On this service’s side, the activation stays STOPPING until inFlightCount reaches 0 or ft.fcc.stop-grace (default 15 minutes) elapses, whichever comes first. Since contract 1.5 that same window also travels to the ACS: every ACTIVATE carries it as config.retry.stopDrainGraceSec, so the ACS’s own decision about how long to keep holding in-flight results after a STOP agrees with the wait this service applies locally — by contract, not by coincidence — see What Contract 1.5 Changed.

A STOP is always a new command — fresh idempotencyKey, command_seq = fcc_test.command_seq + 1 — written by the single FccStopCommandPublisher on all three paths that need one: the operator stopping or pausing a campaign, the operator deleting one whose run is still live, and FccStalledActivationService giving up on a run the ACS never acknowledged. The stop reason does not travel on the wire: the contract’s STOP carries no reason field, so USER_STOP / USER_PAUSE / WATCHDOG / ACS_UNAVAILABLE exist only in fcc_activation.stop_reason and on the log line.

A run still PLANNED is the one case where no STOP is sent at all — see the PLANNED divergence and commandPublished: false Is Not a Failure.

in_flight_count is written from the stop ack (ACTIVATION_STOPPED{inFlightCount}), only when the ACS actually reported one. A null count is left alone: writing 0 for it would make the drain check close the run immediately and discard every measurement the devices still in flight are producing. The state machine makes the same choice, treating an absent count exactly like a positive one — waiting out the stop grace costs a delay, closing early costs measurements.

From then on the figure is recomputed from fcc_device_state, never decremented, which is why FccActivationDao.decrementInFlight no longer exists:

A per-message decrement cannot be made idempotent on an at-least-once queue. The same batch redelivered after a failed consume, or two service instances handed overlapping batches, would each subtract again and drive the column to a number of devices that were never in flight. Counting the rows that are still SENT is exact by construction — it reads the state the upsert has just written, on the same connection and in the same transaction — and it costs one indexed count per stopping run rather than one update per device.

The MIN clamp is load-bearing: the column may only ever fall. Hazelcast does not order messages, so a redelivered or reordered batch can re-assert SENT for a device that has since finished; taking that recount at face value would resurrect devices the run has already drained and keep it open until the stop grace killed it.

The close is additionally guarded by stored_in_flight_count > 0, which is not the same condition as recount == 0. A run reaches STOPPING with the column still at its initial zero whenever the ACS did not report a count, and reading that zero as "drained" on the first device-state batch to arrive would undo exactly the decision described above.

When the recount does reach zero, IN_FLIGHT_DRAINED closes the run as STOPPED. When it does not, ft.fcc.stop-grace does — see What the Watchdog Closes, and With Which Grace.

PAUSE, as exposed by this service’s REST surface, is implemented as a STOP of the current activation with stop_reason = USER_PAUSE, plus the campaign moving to PAUSED instead of the NOT_ACTIVE a stop lands in. There is no PAUSE on the wire: what reaches the ACS is an ordinary STOP, so the running measurements end rather than being suspended. Resume is a plain POST /FCC/activate, which creates a new activation with a new idempotencyKey, a new seq and a fresh settings snapshot — it is not a resend of the paused one. See POST /FCC/pause.

The PAUSE and UPDATE_CONFIG command types exist in FccCommandTypeEnum for future use but the ACS answers REJECTED(UNSUPPORTED) to both in v1, and this service’s outbox never writes either type — see fcc_acs_outbox.

If no ack arrives within ft.fcc.command-ack-timeout (default 60s), FccAckTimeoutPoller (ticking every ft.fcc.outbox.ack-timeout-interval, default 30s) republishes the outbox row with the same idempotencyKey and the same commandSeq via FccOutboxPublisher.republishTimedOut, backing off along the same ladder as a fresh publish — see The Transactional Outbox. A republish is a resend of one command, not a new one, which is why neither identifier moves; the ACS recognises it by the key and answers DUPLICATE if it already has it. If an ACCEPTED (or a DUPLICATE whose reason unwraps to an acceptance, DUPLICATE Carries the Original Outcome) does arrive, but only after some delay, and that delay is still within ft.fcc.activation-max-delay (default 30 minutes), the run starts with a shifted window: FccAckService.accept moves the activation to RUNNING and recalculates windowEndAt from the actual start regardless of how long the ack took.

The commandSeq check of Matching an Ack: idempotencyKey and commandSeq is the second half of this: an ack whose seq is lower than the one recorded on the outbox row answers a send this service has already moved past, so a late ACCEPTED cannot restart a run that the checks below have already written off.

Independently of the outbox row’s own retry state, the same poller tick also runs FccStalledActivationService.failStalled, which claims every activation still DISPATCHING whose window_start_at is older than now - activation-max-delay — this check is driven by the activation’s age, not by the outbox row’s attempt count. Past max-delay with still no ack, the activation goes FAILED(ACS_UNAVAILABLE), the owning fcc_test moves to ERROR, and a brand-new STOP command (fresh idempotencyKey, command_seq = fcc_test.command_seq + 1) is written to the outbox for the same activationId — in case the ACS did in fact apply the original command after all, despite the lost ack.

failure_reason and in_flight_count are the two columns an event carries that are not part of any transition, and they are written only once the run transition is known to be allowed — never before.

The activation is a managed entity inside the transaction, so a field set on it is flushed at commit whether or not anything calls save. A late ACTIVATION_FAILED for a run that is already FINISHED is denied by the state machine, but writing the reason first would still have left its failure_reason on that finished run, next to a contradictory stop_reason. The fix is to write nothing until the transition is allowed.

Symmetrically, when the run transition is denied, the campaign is not touched either: applying only the campaign half of an event the run refused is how a stopped campaign gets resurrected by a late report.

Two cases where FccActivationEventService deliberately stops short of applyTest:

The PLAN_REACTIVATION effect brings FccReactivationService.planNext into play, which decides when the next daily run opens and creates it as a PLANNED fcc_activation with no command attached.

The cadence is the product rule: fcc_test.reactivation_days is a count of daily runs, so run n+1 opens twenty-four hours after run n opened — not twenty-four hours after it was reported finished, which would let the campaign drift by the length of every run. The anchor falls back from started_at to window_start_at to now: a run closed by the watchdog before the ACS ever reported a start has no started_at, and the cadence still has to be anchored on something the operator would recognise.

planNext re-checks finished.seq < test.reactivationDays — the same predicate FccTestTransitionContext.hasMoreReactivations() evaluated when it emitted the effect. The two disagreeing means the campaign was edited in between, and planning a run the campaign does not owe would make it measure longer than the operator configured, so it logs a WARN and plans nothing.

The planned row’s window is then claimed by the PLANNED poller when it opens, and FccActivationHandler.dispatch re-takes the settings snapshot on the way out — see A Planned Reactivation Re-Takes Its Snapshot at Dispatch.

Every remote report may simply never arrive — the ACS restarted, the queue was drained by another cluster, the message was unreadable — and a run nobody closes stays RUNNING or STOPPING forever: its campaign never reactivates, never completes, and its window sits open in every report.

Both bounds are computed in Java (now.minus(grace)) and passed in as parameters, so no statement depends on vendor date arithmetic or on the database clock agreeing with this service’s — and the grace stays a @ConfigurationProperties value rather than something baked into SQL.

FccDeviceStateDto.stateReason is an open string on the wire, qualifying FccDeviceStateEnum where the state alone does not say why. On this side it is the closed enum FccDeviceStateReasonEnum, stored in fcc_device_state.state_reason. It is not on any response since POST /FCC/devices/status was reduced to DEV-1381 #9’s six counters (POST /FCC/devices/status); it is kept because it is what the state column means and what the state-queue log lines carry, and it is what a per-device grid would render if one is added back.

This is a closed vocabulary on this side, but it is not the same closed vocabulary the ACS sends: the eight named reasons above are exactly FccDeviceStateReasonEnum’s constants, and an ACS `stateReason of USER_STOP maps to the enum constant of the same name rather than falling back to UNKNOWN — a reader who stops at ACTIVATION_GONE and assumes anything else lands in UNKNOWN would misread every STOPPED device the drain reports.

FccDeviceStateReasonEnum.fromWire trims and upper-cases the wire string and falls back to UNKNOWN for anything it does not know. This is the same split as FccAckReasonEnum, and for the same reason: the DTO package is the frozen contract shared byte-for-byte with the ACS, so a new constant added on one side must not break deserialization on the other. A device-state screen showing UNKNOWN for a reason this version has not learned yet is strictly better than an ACS upgrade that makes the whole state queue unreadable — and fcc_device_state.state_reason is an enum column here, so an unmapped value would otherwise poison it.

state itself is the opposite case: it is stored as the wire enum com.friendly.fcc.dto.FccDeviceStateEnum (INITIAL, OFFLINE, SKIPPED, WAN_CHECK, CROSS_TALK, SENT, FINISHED, ERROR, STOPPED) with no local copy at all, because the ACS owns that vocabulary end to end and a second dictionary would only add a mapping that can drift.

Contract 1.5 added STOPPED as the ninth constant: the device was halted by a STOP between steps — its WAN sample or its diagnostic was never sent. It is not a failure and not an error. It is its own state, exactly as CROSS_TALK is, and neither is ever counted as a failure: the drain recount ignores both (The STOPPING Drain: Recounted, Never Decremented), and the six counters of POST /FCC/devices/status fold STOPPED into notSent and CROSS_TALK into skipped, never into failed. The nine states themselves are what fcc_device_state holds; the six are only the shape of that one answer. See What Contract 1.5 Changed.

FccOutboxDao and FccActivationDao claim in two statements, not one, and only the first of the two is vendor-specific (selected by FccSqlDialect):

Why two statements: a capped view combined with FOR UPDATE is exactly what raises ORA-02014 ("cannot select FOR UPDATE from view with DISTINCT, GROUP BY, etc."), so an ordered Oracle claim and a locking Oracle claim cannot be the same statement. Splitting them sidesteps it. The predicate is repeated in step two because another instance can claim, process and commit a candidate between the two statements; a locking read sees the latest committed row, so the re-check is what keeps the two-step claim as safe as the single statement it replaces. A row another instance is holding is simply skipped by SKIP LOCKED, so the worst case is claiming fewer rows this tick.

FccSqlDialect.current() resolves the vendor from DatabaseMetaData.getDatabaseProductName() exactly once and caches it in a volatile field — but the detection itself is lazy, on first use, not eager at application startup: the constructor deliberately opens no connection, because the service is required to start even against an unreachable ftfcc database (it degrades and retries rather than failing the Spring context), and a constructor-time probe would break exactly that. In practice the first outbox dispatch tick (five seconds after startup by default) triggers the detection, so the effective difference from "at startup" is small, but the two are not the same guarantee.

A PING is buffered exactly like a DOWNLOAD — both are measurements of the same run and share one dedup key space and one buffer — and the batch is split only once its activations have been resolved:

Since M4 the two INSERT`s sit in two separate `try`s, each with its own half of the retryable rows — not the single `try M1-M3 used. They write different tables, from disjoint sets of measurements, under different natural keys: no row of one is the counterpart of a row of the other, so "half the flush is written" is not a corrupt state, it is the speed measurements stored and the ping measurements still owed, and the next tick owes exactly those.

The natural key is (activationId, serial, direction, iteration), joined into one string (FccResultIngestService.dedupKey) and written to the fccResultDedup Hazelcast IMap (FccMapNames.FCC_RESULT_DEDUP) with:

This has to be one atomic call, never a get followed by a conditional put. With more than one service instance consuming the same batch (or the ACS re-delivering it), two instances doing get first would both see "absent" in the window between the two calls and both would insert the row — the exact duplicate the mechanism exists to prevent. putIfAbsent is atomic on the member that owns the key’s partition, so only one of the two racing calls can return null. The TTL is ft.fcc.dedup.ttl (default 1h) — long enough to cover the ACS’s own retry window, short enough that the map does not grow without bound; see Configuration Reference for the operational note about also bounding it with a max-size in hazelcast.xml.

If the Hazelcast call itself fails (cluster unreachable, serialization error), isDuplicate answers "not a duplicate" and the row is stored anyway: a row a reader can still collapse on the natural key is recoverable, a silently dropped measurement is not.

The marker is written before the ClickHouse write, so every path that then drops the row releases it again (dedup.remove): the buffer overflow in buffer(), the exhausted re-offer in requeue(), and the @PreDestroy shutdown flush, which calls forgetDedup for every row it could not write before parking it (FccResultIngestService.flushOnShutdown). The invariant is a key may only outlive a row that was actually written — otherwise a recoverable drop would become a permanent loss, because the key would keep the slot for the full TTL and an ACS retransmission inside that window would be skipped as a duplicate; the dedup IMap lives on the Hazelcast cluster with that TTL, so it outlives this JVM and the invariant has to hold across a restart too. Each buffered row therefore carries its own dedup key.

Rows that pass de-duplication are pushed onto an in-memory LinkedBlockingQueue bounded by ft.fcc.result.queue-capacity (default 10000), still in wire form. The buffer is flushed — resolve the activation context, judge the rows (The Pass/Fail Formula — the Single Shared Definition), then FccResultSpeedDao.insert, one batched JDBC batchUpdate — as soon as it holds ft.fcc.result.batch-size (default 100) rows, and independently every ft.fcc.result.flush-interval (default 5s) via FccResultIngestService.flushTick, whichever happens first. A @PreDestroy flush drains it once more at shutdown: it runs after FccQueueListenerRunner has interrupted and joined the consumer threads, so nothing is still arriving, and spring.lifecycle.timeout-per-shutdown-phase (20s) leaves room for it.

The same buffer doubles as the retry queue: a failed flush — the ftfcc resolve or either ClickHouse batchUpdate — re-offers every retryable row of the failed batch back onto the queue (requeue) and computes the next retry time with the same FccOutboxBackoff ladder the outbox dispatcher uses, keyed on consecutive ClickHouse failures rather than per-row attempts. "Retryable" is the whole batch minus the late arrivals, which have already left this service for the dead letter and must not be parked a second time by the next attempt.

Only when the buffer itself is full does a row leave the buffer — logged as an ERROR with the row count, counted (fcc.result.parked), and setting FccDegradedState.setClickhouseStatus(UNAVAILABLE) — not SCHEMA_BEHIND, since a full buffer or a failed insert says nothing about why, and ClickHouseHealthIndicator refines the status to SCHEMA_BEHIND on its next probe if that turns out to be the cause (Health). Either status closes FccResultStorageGate and makes every gated endpoint start refusing new work until a batch is written successfully again: POST /FCC/activate; since M4, all four /FCC/reports/ endpoints; POST /FCC/admin/result-dead-letter/resend (FccAdminController), because a resend feeds the very ingest path a result store outage already broke; and the legacy-import start (FccLegacyImportService), because it would read millions of rows only to fail every insert against a store that cannot take them (POST /FCC/activate, Endpoints Implemented in M4). Since M2 such a row is not dropped: it is parked in fcc_result_dead_letter** — see The Dead Letter. The ERROR level is kept deliberately even though nothing is lost any more, because reaching the dead letter at all means the buffer has been full long enough to overflow, which needs a human whatever happens to the row afterwards.

The @PreDestroy shutdown flush parks whatever it could not write, for the same reason: a rolling deploy during a ClickHouse outage is the most likely way to meet that branch, and it is the one moment where there is no next tick to retry on. The ClickHouse backoff is reset before that final flush, since there is no next tick to wait for either.

Per-batch context (the activation’s fcc_test_id, the cross-talk thresholds and the speedSuccessPercent / latencySuccessMs a result is judged against, plus domain_id and hubb_id as a fallback, plus stoppedAt) is resolved once for the whole flush batch, not once per row: FccActivationContextResolver.resolve runs two queries (fcc_activation by id, then fcc_test by the resulting test ids) and returns an activation-id-keyed map. It is deliberately never cached across batches — a stale threshold would silently change a verdict, which a compliance report cannot tolerate — so every batch re-reads the activation snapshot it needs, which is exactly what makes an edited-while-running test still measured against the settings that were frozen at activation time (The fcc_activation Settings Snapshot).

Contract Q26. fcc_activation.stopped_at defines two boundaries, not one, and they mean different things:

For after_stop the wire flag wins when it is set — the ACS knows when it issued the diagnostic — and the local comparison is the backstop for an ACS that does not set it. Without that backstop the flag would be the only source, and an older ACS would have every one of its after-stop results silently indistinguishable from the rest.

Two details of the comparison matter:

fcc_result_dead_letter replaces both M1 TODO(M2) drop paths and the shutdown drop. Four paths write to it, and they carry one of the two reasons of the reason dictionary:

FccResultDeadLetterService.store runs in its own transaction (REQUIRES_NEW) and never throws: the ingest path that calls it is already failing and must not be able to roll back the record of that failure, nor be handed a second exception in the middle of a flush. If even that insert fails, the row is genuinely gone — and only then is fcc.result.lost incremented, separately from fcc.result.parked, because the two mean completely different things to an operator.

Rows are parked grouped by their arrival instant, because receivedAt belongs to the row and not to the call: a retry batch mixes measurements from several accept invocations, and parking them all under one instant would rewrite when they arrived.

The dedup marker (De-Duplication Is One Atomic putIfAbsent) is released on every path that parks a CLICKHOUSE_UNAVAILABLE row, for two reasons stacked on each other: the general invariant that a key may only outlive a row that was actually written, and the fact that a parked row is replayed through accept on resend and therefore has to be able to pass the dedup check again. A key left behind would make the resend a no-op and turn a recoverable outage into silent data loss. A LATE row is the one exception and deliberately does not release its key: it is durably recorded, it is never resent, and a retransmission of the same natural key would be judged late all over again.

Contract 1.3 added domainId and hubbId to FccResultDto itself. The ingest path therefore takes both from the result, and falls back to the activation snapshot only when the wire value is absent — a pre-1.3 ACS (no domainId/hubbId field at all), a pre-1.4 ACS (the field exists but the ACS had no domain to put in it, see [contract-1-4-domain-answered]), or an activation whose config carried no hubbId.

"Absent" is not the same test for the two fields, and the asymmetry is deliberate:

The snapshot resolve stays in place regardless. It is not made redundant by the two new fields: it is what supplies fcc_test_id, the cross-talk thresholds and speedSuccessPercent, and a batch cannot be judged at all without it (The Pass/Fail Formula — the Single Shared Definition). What the new fields remove is only the dependency of those two label columns on the snapshot: when the ACS names the domain and the hubb on the result, they are taken as sent and the snapshot value is not consulted. See fcc_result_speed for the columns themselves.

Three additive fields, none of them a shape change to an existing field — unlike 1.3’s Integer → Long widening, 1.5 needs no joint, queue-drained deploy.

Where duration_ms = eom - bom in milliseconds, computed as a double with no integer division anywhere in the chain, and tier_kbps is compared directly in Kbps — no division by 1024. Implemented, code-read, by FccSpeedStatusCalculator.evaluate (pure, no Spring, no I/O), called once per measurement from FccResultIngestService.

Implemented, code-read, by FccLatencyStatusCalculator.evaluate — pure, no Spring, no I/O — and called once per PING measurement from FccResultIngestService. latencySuccessMs falls back to FccRuntimeDefaults.LATENCY_SUCCESS_MS (100) when the activation snapshot carries no explicit value. In practice thresholdUlKbps is always null for a latency run, because the snapshot writes the upload columns null for LATENCY (The Snapshot Is Type-Driven) — the cross-talk clause is written to handle it anyway, so the two calculators stay literally the same predicate.

The resolution order is identical to the speed calculator’s, and that is a hard rule rather than a coincidence:

A compliance report reads the two tables as one set of iterations, so a measurement that would be CROSS_TALK as a download must not come out as FAILED merely because it was a ping. Reordering the branches here would make the same line condition mean two different things depending on which diagnostic happened to run on it.

packets_sent is reconstructed as successCount + failureCount and never read from the configured repetitions — see fcc_result_latency. An absent counter contributes 0 to that arithmetic, because the two packet columns are non-Nullable Int32, while the three round-trip values pass a null through unchanged because theirs are Nullable(Int32).

Contract 1.5 adds FccResultDto.pingRepetitions / .pingDataBlockSize next to this, and neither feeds packets_sent. They carry what the device actually did — the repetition count and data block size it ran the ping with — stored verbatim in the new fcc_result_latency.repetitions / .data_block_size columns, falling back to the activation snapshot (ping_repetitions / ping_data_block_size) when the wire value is null. packets_sent stays exactly what it has always been, the sum of successCount and failureCount: that sum is what the device reports having done, the new repetitions column is what it was asked for, and a run that ended early must not be able to claim otherwise by having the two collapse into one figure. See fcc_result_latency for the column-level detail.

The relevant TR-181 object is IPPing, and the three parameters the mapping reads back are:

TR-181 guarantees SuccessCount + FailureCount = NumberOfRepetitions; the configured value is what was asked for, the sum is what the device reports having done, and a run that ended early must not claim otherwise.

wan_usage_kbps = (bytes_2 - bytes_1) * 8.0 / gap_ms, where a wraparound of the 32-bit byte counter (bytes_2 < bytes_1) means the sample pair is discarded and resampled, and gap_ms comes from each side’s own monotonic clock (System.currentTimeMillis()), never from a database column. This sampling runs on the ACS side, outside this repository; ft-fcc-service only receives the resulting wan_usage_dl_kbps / wan_usage_ul_kbps figures on the wire and compares them against the cross-talk thresholds — for a latency run as well as for a speed one, which is why fcc_test_setting_latency carries a threshold_dl at all.

The closing half: a user stop, the STOP command, the ACS acknowledging it with an in-flight count, the device states draining that count back to zero, and the stop-grace timeout as the alternative branch. A pause is the same diagram with stop_reason = USER_PAUSE and the campaign landing in PAUSED.