Legacy FCC Import

ug_fcc_speed_result and ug_fcc_latency_result are not touched. Not updated, not flagged as imported, not truncated afterwards — the ftacs account this service holds is SELECT-only, the pool is opened read-only in code, and every statement in FtacsLegacyGroupDao and FtacsLegacyResultDao is a SELECT (app.ds.* Datasources).

That is a product decision and not only a permissions accident. Customers run their own SQL reports directly against those tables, and an ETL that marked or moved rows would silently change the answers those reports give — from an operation nobody outside this service asked for. The legacy tables stay exactly what they were; ClickHouse gains a second, queryable copy.

The consequence is the one every idempotency rule on this page exists for: the source can be read again, so the import must be safe to read again.

There is no poller and no schedule. The import is started by POST /FCC/admin/legacy-import/start and by nothing else, watched through /status, and asked to stop through /stop — see The Legacy Import Endpoints.

It is also re-run on purpose. The legacy flow keeps writing while this service runs, so one pass is never the last one: a completed pass means "every measurement whose start_date is below watermark_started_before is in ClickHouse", and the next start opens a new pass from that date to catch up. FccLegacyImportStateEnum.COMPLETED is therefore not a terminal state, and ft.fcc.legacy-import.enabled exists so an installation that has genuinely finished migrating can switch the start path off while /status keeps answering.

A legacy campaign cannot be correctly continued by this service, and continuing it is the only thing migrating its configuration would buy. It has no activations to continue from, frequently no domain, and a reactivation expressed as an arbitrary Quartz cron that this service’s daily-reactivation model cannot express. An imported campaign that offered an Activate button would publish an ACTIVATE command naming an update_group that no longer exists on an ACS that has since been upgraded.

So each legacy update group that has results becomes an anchor: a fcc_test row whose entire job is to be something the imported measurements and the reconstructed runs can hang from.

No URLs, no ping host, no connection counts, no upload file size, and no reactivation. Those lived in ug_set_param_value, a generic name/value table whose contents have to be interpreted as parameter paths per ACS version, and that is deliberately out of scope: every one of those columns is nullable precisely so that an archived campaign can decline to answer. A null configuration is written as nulls and never as defaults — a plausible duration filled in here would be a number nobody configured, sitting in a regulatory record, indistinguishable from one that was.

Two further consequences worth stating plainly:

A single legacy update_group can carry both an FCC speed task and an FCC latency task: ug_task.task_type = 103 and 104 are separate ug_task rows, with separate ug_fcc settings rows underneath them.

The ETL makes two anchors out of such a group, one per result kind — which is exactly why the unique index is on (legacy_ug_id, legacy_result_kind) and not on legacy_ug_id alone. A unique index on the group id by itself would make the second anchor impossible to insert and the entire latency half of that group unimportable.

Ordinary campaigns are unaffected: source = ACS_STREAM tests carry legacy_ug_id IS NULL, and both MySQL and Oracle allow unlimited rows with a NULL in a unique index, so the constraint is only ever asserted between imported anchors.

Two more legacy tables are deliberately out of the import entirely.

An anchor takes domain_id from update_group.location_id, which is NOT NULL DEFAULT 0, and stores it as it stands.

0 is the super domain, not "no domain". That is the platform’s convention and not a reading this service chose: ui-backend’s CommonUtils.isSuperDomain treats null, 0 and -1 alike, and the comment beside it gives the reason — a super-domain record has no row in isp at all, which is how Group Update, monitoring and automation have always stored theirs. A legacy group created by a super-domain operator therefore reads back as 0, and so does every run and measurement under its anchor. The three spellings are normalised to 0 on the way in, because a stored -1 would be invisible to a predicate written against 0.

The access-control consequence is unchanged, and it is the right one. 0 is in nobody’s X-FT-Domain-Ids list — ui-backend strips the sentinels before they reach the header — so such an anchor is visible only to a caller with X-FT-Domain-Scope: ALL, which is precisely the operator who owns it. A scope-LIST operator does not get a partial view; they get nothing, exactly as The Empty Intersection Selects Nothing describes for every other read. It does mean "the import ran and I still cannot see anything" has a second explanation besides a missing fleet snapshot: the campaign may be the super domain’s.

The legacy schema has no "activation" entity whatsoever. A legacy campaign ran for months, reactivating itself, and the only record that a particular run happened is the measurements it produced.

The import reconstructs one fcc_activation per day-window, because a day is what a legacy reactivation was. The window a measurement belongs to is decided by its start_date alone, and the time of day the window starts at is read from ug_reactivate.expression:

Each window becomes one fcc_activation with source = LEGACY, state = FINISHED (only closed windows are imported, so the run is over by definition), command_seq = 0, and:

The activation is created lazily, the first time a measurement of that window is seen, and never up front from the calendar. Which days a campaign actually produced measurements on is not knowable without reading the measurements: a campaign scheduled daily for a year may have run on forty days, and a row per calendar day would fill the report with runs that never happened.

window_start_at / window_end_at are facts about the window and are written at creation. started_at, finished_at and device_count are facts about the measurements and are applied monotonically once the campaign’s rows have been read, so a campaign imported in two halves converges on the true envelope instead of recording whichever half finished last. A pass that halts part way deliberately does not finalise its windows at all — it has seen a fragment, and the envelope of a window must not be narrowed to it.

iteration is the position of a measurement among the measurements of the same device, in the same direction, in the same window. Stated as SQL, it is exactly:

Two halves of that expression are load-bearing.

The truncation to the second. It is what makes rows written within one second rank deterministically instead of by insertion accident, and it is computed by the database (FtacsSqlDialect.startSecondExpression) so that the ordering the rows arrive in is the ordering the rank is defined over.

The id tiebreaker, which is mandatory. The legacy start_date carries a random sub-second fraction — between 0.100 s and 0.999 s — which the legacy code set deliberately to make rows unique. Being random, it carries no ordering information at all: without the id tiebreaker the order of rows inside one second is undefined, and a re-import would produce different iteration numbers for the same measurements. In a store where nothing can be deleted, that is the same measurement appearing twice under two identities.

The two legacy tables spell the tier differently.

The legacy ACS produced the speed string by dividing the configured Kbps tier by 1024 and truncating to three decimals with RoundingMode.DOWN. So the true value lies in [x, x + 1.024), and multiplying back by 1024 and taking the ceiling lands on it — while a round does not:

The multiplier is 1024 and not 1000, which looks wrong next to this product’s own Kbps/Mbps convention and is exactly right here: it undoes the division the legacy code actually performed, whatever that division ought to have been. The string goes straight into a BigDecimal and never through a double — a ceiling is the one operation that turns a floating-point representation error into a whole unit, in both directions and silently.

An unreadable tier does not drop the measurement. The row is imported with tier_kbps = 0 and tallied as tier_parse_failed. The measurement itself — the speed, the timestamps, the device — is a fact that happened, and losing it because the contract line it was judged against cannot be recovered would take real data out of a regulatory report to hide a metadata defect.

A speed string with no / separator is treated as unreadable in both directions rather than read as the download half: that is a shape this service has never seen written, and picking a half would be a guess with no evidence behind it. A latency tier has no separator by design — it is a plain Kbps integer and goes through a different path entirely, so this rule does not touch it. A half that is present but unreadable fails only its own direction: "12/" gives a download tier and an unreadable upload one, "/34" and "abc/12" the reverse.

A value above Integer.MAX_VALUE is clamped and counted with the unreadable ones — the ClickHouse column is an Int32, and two petabits per second is a corrupt string that happens to parse.

tier_parse_failed is counted once per imported row, not once per measurement. A speed measurement is two rows — one download, one upload — and each is tallied independently. Since a no-separator string is now unreadable in both directions, such a string is counted twice, once on each row, where before the fix it was counted once, on the upload row only (the download half read a real, if arbitrary, tier under the old behaviour). An operator comparing tierParseFailed across a re-run of the same source data should expect it to roughly double for campaigns whose legacy tier strings never carried a separator, and that is the counter behaving correctly rather than a regression in the source data.

fcc_result_speed.total_bytes is Nullable(Int64) for this (fcc_result_speed). A legacy value is refused, mapped to NULL and counted as bytes_overflow_count when either of the following holds:

Both checks run on both vendors, deliberately: the import must catch the overflow whichever way the storage engine expressed it, and an installation may well have been migrated between the two.

Storing the wrapped number instead would put a byte count into a regulatory report that is off by four gigabytes and looks perfectly plausible. A NULL plus a tally says the only true thing available: the input was destroyed, and here is how often.

Stated plainly because it changes how a model-filtered report over imported data should be read.

The legacy result rows carry a serial and nothing else. The only place a model can come from is the live ftacs.cpe row of that serial today, so a CPE that has been re-provisioned into another product class since the campaign ran is snapshotted under its current model. There is no way around it and no approximation that would be better: this is a genuine limitation of the source data, not of the resolver.

It is the one respect in which an imported fleet differs from a live one, where the snapshot is frozen at save time and therefore does describe the fleet as it was. The /start endpoint states it in its own description so an operator reading a model-filtered report knows what they are filtering by.

fcc_test_device.cpe_id is NOT NULL and stays so. There is no id to record for a serial the ACS no longer knows, and inventing one — a zero, a negative sentinel — would make the (test_id, cpe_id) unique index meaningless and put a device into the fleet that nothing can ever resolve.

So the device is left out of the snapshot and counted as devices_unresolved (logged at WARN), while:

Four more rules of the snapshot, each of which is there for a concrete failure:

The snapshot scan is restricted to the interval this pass covers — the same [watermark_started_before, cutoff) predicate its batches are read under — so an installation that has been importing for months does not re-resolve a fleet it recorded long ago; a device that only ever appears below the watermark was snapshotted by the pass that imported it. The rows a pass reads, the fleet it snapshots and the id high-water mark it reports are all written from one predicate in FtacsLegacyResultDao, because the watermark is only sound while the three describe exactly the same set. The resolve is not narrowed by domain: the import runs with scope ALL by contract, an anchor takes the domain of the legacy group rather than of the caller, and narrowing here would silently drop the devices of every other domain from the fleet of a campaign that measured them.

One batch is one unit of work in three steps, and the order is the design:

A non-null pending_result_id found at startup therefore means the previous run died inside exactly that window and the batch’s fate is unknown. The recovery is a single bounded read — SELECT DISTINCT legacy_result_id FROM fcc_result_* WHERE source = 'LEGACY_ETL' AND legacy_result_id BETWEEN ? AND ? — which runs at most once per resumed run and tells the importer precisely which ids to skip (counted as rows_skipped). Because the range is one batch, the import never has to reconcile itself against the whole table.

Four supporting details, each of which is there for a crash that actually reaches it:

The stop flag, the result-store health and the shutdown signal are all checked between batches, never inside one. Whatever ends a run therefore leaves a committed cursor the next one resumes from exactly.

IDLE and FAILED differ in exactly one thing: IDLE means interrupted, FAILED means broken. The cursor is committed in both cases, which is why a clean stop is IDLE even when it carries a last_error.

A pass covers one half-open interval of start_date and nothing else:

Rows at or after the cutoff are not read at all.

Because the iteration rank of a window still receiving rows would be computed over a partial set, and a later pass would renumber it. A renumbered rank means the same measurement appearing twice under two identities in a table where nothing can be deleted — the exact failure this whole section exists to prevent.

The cutoff is derived from the pass’s own started_at and never from now. A pass interrupted at campaign 50 and resumed an hour later must import exactly the windows it set out to: deriving the cutoff from now on resume would let windows that closed in the meantime be imported for the campaigns after the cursor and never for the ones before it — while the watermark written at the end declared all of them done.

The one place now is used instead is totalRowEstimate, which is measured before the claim decides whether the start is a fresh pass or a resumed one. A resumed pass’s denominator is therefore a few hours wider than the interval it will actually cover; it is explicitly a rough denominator and nothing decides that the import is finished from it.

The cursor and the watermark cannot be collapsed into one column: a position in an order that is not id order says nothing about which rows are complete, and a completeness mark says nothing about where to resume a ranking.

watermark_result_id is built from MAX(r.id) read per campaign, under the pass’s own predicate, and is folded in for every campaign of the pass — including the ones a resumed pass skips because the run it continues already imported them. The stored value only ever moves up: a later pass covers a later date interval whose highest id is not necessarily higher, and letting the mark fall would over-report rows_late_skipped.

A measurement can reach ftacs after the day-window it belongs to has already been imported: it matches start_date < watermark_started_before AND id > watermark_result_id, which is exactly what the two marks together are able to recognise. FtacsLegacyResultDao.countLateArrivals counts them per campaign, the runner tallies them as rows_late_skipped and logs a WARN naming the campaign, and /status reports the total.

The count is answered only when both marks are known. Before the first completed pass there is no "after its window was imported" for anything to be after, so the count is skipped entirely.

The read position is the tuple (serial, direction, start_date-to-the-second, id), and every arm of the resume predicate compares those columns with > and =. In SQL both x > NULL and x = NULL are UNKNOWN, so a cursor carrying a null in any component matches no row at all — the next batch comes back empty, and an empty batch is indistinguishable from the end of a campaign.

So a row with a null serial, a null start_date, or (for a speed row) a null type cannot become a cursor, and the pass stops there:

It deliberately does not skip the campaign and carry on. Doing that would move the cursor past the campaign, and the pass after it would resume above it, complete, and advance the watermark over rows nothing ever imported — the same permanent hole, reached by a different route.

serial and start_date are NOT NULL in the ACS DDL and a speed row’s type has only ever been written as a number, so such a row means the source is not what this import was specified against. Fix the source row and start again.

A single-threaded worker runs one kind at a time. Two importers would contend for the same ftacs read capacity and the same ClickHouse ingest — but the thing that must never happen, two passes over the same kind, is prevented by the claim and not by the executor.

The claim is one conditional UPDATE on fcc_legacy_import: a run may take the row when nobody holds it, or when the holder has written no heartbeat within ft.fcc.legacy-import.stale-after (default 5m), which is what a process killed mid-import leaves behind. Zero rows updated is the 409. Being a database statement, it holds across two service instances, which a thread pool never could.

Two properties of it are easy to miss:

The two kinds are otherwise independent by design, which is why fcc_legacy_import holds exactly two rows: an unreadable row or a stalled insert on one table is no reason to abandon a pass over the other, and either kind can be started, stopped or restarted alone.