Device Resolution

This is the page a developer reads to understand how a fleet comes into being.

A campaign’s device set starts as a selection — conditions over the ACS catalog, a list of serials pasted into a form, or an uploaded file — and ends as rows in fcc_test_device that are never refreshed again. Everything between those two points is in this page. The HTTP surface is on POST /FCC/devices/resolve and POST /FCC/devices/count; what the resulting rows hold is on fcc_test_device.

One Path, Four Callers

FccDeviceResolveService is the whole of it, and it is the only implementation.

Caller What it asks for

POST /FCC/devices/resolve

The fleet, one page of it, and every counter.

POST /FCC/devices/count

The same, without the page.

PUT /FCC/devices/import

The same as resolve, with the serials read from an uploaded XML inventory instead of a text list — see The XML Inventory FccSerialXmlParser Accepts.

PUT /FCC/item/details with configs[].sourceType

The same again, through resolveForSave, which adds exactly one rule: an unresolved serial fails the save.

That is not tidiness. The count is what the wizard shows while the operator is still choosing, and the save is what it stores — the two disagreeing would mean an operator storing a fleet other than the one they were shown, and both would be internally consistent about it. Every counter, every refusal, the selection limit and both serial reports are therefore computed by the same code.

PUT /FCC/item/details with an explicit devices list takes a different entry point — FccTestValidator.resolveDevices — but lands in the same DAO (FtacsDeviceResolveDao) with a cpe.id IN (…​) predicate instead of a condition. FtacsDeviceResolveDao is the only reader of ftacs.cpe in this service, and that is deliberate for the same reason: a second, narrower reader is how the fleet an operator was shown and the fleet that was stored come to be assembled by different sets of predicates.

The ACS Schema This Service Reads

Four tables, a read-only account, and no write of any kind. ftacs is not owned or migrated from here (Schema Ownership); the columns below are a contract this service depends on and checks at startup (ftacs Read-Only Schema Check).

fcc-ftacs-device-resolution

cpe

Column What this service depends on

id

The id the ACS addresses a device by in FccDeviceDto.cpeId. Stored as fcc_test_device.cpe_id.

serial

The id the ACS identifies a measurement by, and mandatory on the wire — see A Device Without a Serial Is Refused — Twice.

product_class_id

The model id. See product_class.

location_id

location_id IS the domain. What the UI, the identity headers, fcc_test.domain_id and every domainId field in this service call a domain is this column. This is stated in three places, not one: FtacsDeviceResolveDao carries the predicate for device resolution (c.location_id IN (:domainIds)), FtacsCatalogDao carries its own copy for the catalog read (SQL_DOMAIN_PREDICATE, WHERE c.location_id IN (:domainIds)), and FtacsSchemaCheckRunner spells cpe.location_id a third time in its required-column list. A rename of the column on the ACS side would have to be followed in all three, which is exactly the kind of fact this page exists to make explicit rather than leaving a reader to assume the class-level javadoc’s "single place" claim covers the whole service.

is_online

An int, not a boolean. Read with getInt + wasNull and compared as is_online = 1. MySQL hands back an Integer for an INT column while Oracle hands back a BigDecimal, so getObject plus a cast would work on one vendor only — that idiom is used for every numeric column in both ACS DAOs.

protocol_id

Nullable, and a null means TR-069 — not "unknown". The column was added long after the table, and a legacy CWMP device still carries a null. FccCpeProtocolEnum.fromProtocolId applies that coercion, and it is the same thing the ACS’s own fromOrdinal does with a null. This has a direct consequence for the SQL; see The protocol_id IS NULL Arm Is Not Optional.

firmware

A condition field, and part of FccResolvedDeviceItem. Added to the startup gate in M3.

The unique key is (serial, product_class_id), not serial.

One serial legitimately exists on two product classes — a re-badged unit, a model split, an import that created a second product class for the same hardware — and both rows are real CPEs the ACS can address and measure separately.

That single schema fact is why duplicateSerials exists at all, why a serial list can resolve to more devices than it holds lines, and why a duplicate does not fail a save. See notFoundSerials and duplicateSerials Are Two Different Defects.

product_class

product_class.id is what this product calls a model id, and the name is worth stating plainly because three different things key off it:

  • fcc_test_device.model_id — the frozen per-device model of a fleet;

  • fcc_model_capability.model_id — the connection ceilings (fcc_model_capability);

  • FccCatalogModelItem.modelId — what the wizard offers and what a condition.modelIds names.

Columns read: id, model, manuf_id, group_id.

Every join of the resolution read is LEFT, product_class included.

cpe.product_class_id is nullable and the ACS schema has no foreign keys, so a device with no product class — or with a dangling one — simply did not come back from an INNER join. That cost nothing on a model-based selection and everything on the other two:

  • a lookup by cpe id or by serial silently lost the device;

  • FccTestValidator.resolveDevices then reported it as an unknown or out-of-scope id;

  • and because the UI re-submits the whole stored fleet on every save, a frozen fleet containing one such device could no longer be saved at all — not even for a rename — while validateDeviceSetEditable forbids dropping it. That is the same trap validateAddedDeviceSerials was written to avoid for a cleared serial (A Device Without a Serial Is Refused — Twice).

The selection semantics are unchanged. A device with no product class cannot match pc.id IN (…​), pc.manuf_id IN (…​), pc.group_id IN (…​) or pc.model IN (…​) — a comparison against NULL is never true — so every model-based condition still excludes it, exactly as an INNER join did. What changed is that a lookup by id or serial now finds it, with a null modelId, model and manufacturer, which every consumer of these rows already tolerates: fcc_test_device.model_id is nullable, validateConnectionCeilings filters the nulls out before asking for capabilities, and countsByModel gives it a bucket of its own (see A Device With No Product Class Is Still a Device).

The CPE-level conditions now reach it too, and that is the one thing the change genuinely widens. c.firmware, c.serial, c.is_online, c.protocol_id and c.location_id are columns of the device: a selection made only of those never said anything about a product class, and excluding a class-less device from it was an artefact of the join rather than a rule anybody wrote. Such a device is a real CPE, in the caller’s own domain, that the ACS can be asked to measure.

The manufacturer and product_class_group joins were always LEFT: a missing row there costs only a display name.

The catalog’s join to cpe is INNER, and that one stays. FtacsCatalogDao.findModelsWithDevices joins product_class to cpe, so the catalog offers only models the caller actually has devices for: product_class holds every model the installation has ever seen, and listing all of them would fill step 1 of the wizard with hundreds of entries that can never yield a fleet. This is a different question from the one above — there the subject is a device and the product class is an attribute of it; here the subject is a model and a model with no devices is genuinely not on offer.

The one read that deliberately does not join cpe is findModelNames, which resolves the names of a capability admin page — a capability row is perfectly legitimate for a model the installation currently has no units of, because a capability is a property of the hardware and is entered before the first unit ships.

manufacturer

Columns read: id, name. LEFT-joined via product_class.manuf_id.

The display name falls back to product_class_group.manufacturer_name when manufacturer.name is missing, and the fallback is applied in the row mapper, not as a COALESCE in the statement: a coalesced expression would have to be repeated in the catalog’s GROUP BY, which is where the two vendors disagree most, and the fallback is a display decision anyway.

The fallback is repeated inside the predicate when the caller filters by manufacturer name:

(UPPER(m.name) IN (:manufacturerNames)
 OR (m.name IS NULL AND UPPER(g.manufacturer_name) IN (:manufacturerNames)))

Without the second arm, filtering by a name the catalog itself displayed would miss exactly the devices whose name came from the fallback.

M3 added this table to the startup schema gate. An installation upgrading from M2 must extend its read-only ftacs grant set with SELECT on manufacturer, or the service refuses to boot — see ftacs Read-Only Schema Check.

product_class_group

Columns read: id, manufacturer_name, protocol_id. (FtacsSchemaCheckRunner’s required-column list also names `product_class_group.product_class, but no statement in FtacsCatalogDao selects it — SQL_SELECT_MODELS reads only g.id, g.manufacturer_name and g.protocol_id off this table. That fourth entry looks like a surplus the M4 trim of the resolution read missed, and is worth removing from the startup gate rather than treated as a column this service depends on.)

g.protocol_id is the only place the catalog carries a protocol: product_class has no such column in the shipped ACS DDL, whatever the api-orm entity maps.

A null g.protocol_id means two different things, and g.id is what tells them apart.

The join is LEFT, so the column comes back null both for a legacy TR-098 group — a real fact about a real group, which FccCpeProtocolEnum.fromProtocolId correctly reads as TR069 — and for a model that has no group row at all, where it is simply the absence of one. Coercing the second case through the same method reported the model to the operator as TR-069 on no evidence whatsoever.

FtacsCatalogDao therefore selects g.id AS group_row_id although nothing displays it, carries the result as FtacsModelRow.hasGroupRow, and FccCatalogService emits FccCatalogModelItem.protocol = null when it is false. Null there is the honest answer, not a gap to be filled.

The catalog protocol is a hint, not the authority.

It is a property of the model’s group. The protocol that decides whether a device can actually be measured is cpe.protocol_id, per device: it is what POST /FCC/devices/resolve reports as protocol and supportedByAcs, what unsupportedProtocolCount is computed from, and what is frozen into fcc_test_device.protocol when the campaign is saved.

The two can legitimately disagree — a device of a TR-098 group re-registering with a TR-181 data model is an ordinary upgrade — and when they do, the per-device value is the one that decides whether a measurement happens. This field belongs in the catalog the operator browses and never in a decision about a run.

The Three Source Types and the Closed Condition Set

FccDeviceSourceTypeEnum decides which half of the request is read, and there is no default: guessing it from whichever field happens to be filled in is how a request that meant "these 12 serials" becomes a request for the whole installation.

sourceType What is read

CONDITION

The condition block, which is required — an absent one is a 400, never "select every device". The difference between "the operator has not chosen yet" and "the operator chose the whole installation" is twenty thousand devices, and only one of the two is ever meant by an absent field. serials is ignored and both serial reports come back empty, because a condition asks for no serial in particular.

INDIVIDUAL

serials, required and non-empty. The condition block, if sent, is stripped to onlyOnline and domainIds — the same narrowing-only rule the upload gets.

FILE

The serials parsed from the uploaded part. Same rules as INDIVIDUAL once parsed; the forcing rule is Why One URL Serves Both JSON and an Upload.

The Condition Set Is Closed and Typed

FccDeviceConditionRequest and FtacsDeviceFilter are records with named fields, not a map, not a list of (column, operator, value) triples, and above all not a predicate string.

Everything in them arrives over an HTTP body from ui-backend, and this service has no authentication of its own (No Authentication by Design). A condition the caller can spell freely is a query this service never intended to run. Adding a condition therefore means adding a field to the record and a predicate in FtacsDeviceResolveDao — there is no passthrough, and there is no way to add one by accident.

Every field is optional and means "not filtered" when null or empty. Present fields combine with AND; no two different fields are ever OR-ed.

Field Type SQL Notes

manufacturerIds

List<Integer>

pc.manuf_id IN (…​)

manufacturers

List<String>

UPPER(m.name) IN (…​) plus the group fallback

Trimmed, upper-cased and de-duplicated in the DAO, next to the UPPER(…​) the comparison is made against. The two halves of a case-insensitive match are one decision, and splitting them across two classes is how one of them silently stops being applied.

modelIds

List<Integer>

pc.id IN (…​)

The id form the catalog offers.

models

List<String>

pc.model IN (…​)

Exact, and not upper-cased — unlike manufacturers. See the note below.

productClassGroupIds

List<Integer>

pc.group_id IN (…​)

firmwares

List<String>

c.firmware IN (…​)

Exact.

protocols

List<String>

(c.protocol_id IN (…​) [OR c.protocol_id IS NULL])

FccCpeProtocolEnum names on the wire, mapped to raw protocol_id values in the service. The wire never carries the ordinal: it belongs to an enum declared in the ACS, and a constant inserted there would re-point a filter the UI had saved. See The protocol_id IS NULL Arm Is Not Optional.

serialContains

String

c.serial LIKE :p ESCAPE '!'

Wildcards in the value are escaped — it is a substring the operator pasted, not a pattern they write. See The LIKE Escape Character Is !, Not a Backslash.

onlyOnline

Boolean

c.is_online = 1

TRUE narrows; FALSE and absent are the same thing, because a fleet of provably offline devices is not something the wizard ever asks for.

domainIds

List<Integer>

c.location_id IN (…​), or in memory

Narrows within the caller’s scope only — The Empty Intersection Selects Nothing.

Two fields of FtacsDeviceFilter never come from a condition block: serials (from the two serial source types) and cpeIds (from the explicit-list save path). No caller ever sets both — see IN Lists Are Chunked at 1000, and Only One of Them Per Call.

A List of Nulls Is Refused, Not Quietly Emptied

FccDeviceFilterSanitizer.sanitized strips the nulls and blanks out of every caller-supplied list before any of them becomes a predicate, and then refuses a list that had entries and has none left.

Both halves matter, and they fail in opposite directions.

Stripping. A null inside an IN list is not ignored by SQL. It emits pc.id IN (123, NULL), and pc.id IN (NULL) is a perfectly legal term that is never true — so {"modelIds": [null]} narrowed the whole selection to zero devices with no error whatsoever, and the operator was shown an empty fleet for conditions they believed were right.

Refusing what is left. Silently dropping a condition that stripped to nothing would widen the selection: the caller asked to narrow by model and would get every model instead. Widening is the one direction this code must never fail in — the mistake is invisible, it is saved as a frozen fleet, and the campaign then measures subscribers nobody chose. The 400 says so:

condition.modelIds holds 3 entries and every one of them is empty; send the
values you mean to narrow by, or omit the field entirely, which is what
selects every value

condition.protocols gets the same treatment one level up, in FccDeviceResolveService.protocolIds: blank entries are dropped — a form that posts an empty row is the normal shape of an FT UI request — but a list left with nothing is refused rather than turned into "every protocol". It has to happen a step earlier there, because the names must be parsed before anything can tell whether a list has anything left in it.

The rule lives at the service boundary, in a component of its own, and no longer in the DAO.

It was written in FtacsDeviceResolveDao.sanitized first, called from count and find, which put it in the wrong layer twice over. A @Repository bound to the read-only ftacs datasource was deciding that a request is malformed and, implicitly, that the answer is an HTTP 400 out of this service’s error catalog — a status a repository has no business naming. It also split one rule across two layers: this half sat below the service while its sibling, "a protocol list that stripped to nothing must not widen the selection", was already in FccDeviceResolveService.protocolIds.

One component, because — at the time this rule was written — there were two filter builders and the rule had to hold for both. FccDeviceResolveService.filter builds the wizard’s selection; FccTestValidator.resolveDevices builds the explicit fleet of PUT /FCC/item/details. The two select devices for the same campaign moments apart, so a second implementation — or a builder that simply forgot — is exactly how one of them comes to accept a list the other refuses.

FccTestValidator.resolveDevices runs its filter through the sanitizer even though every id in it is already non-null by construction, and that is the point rather than a redundancy: FtacsDeviceResolveDao now trusts every filter it is handed equally, so the guarantee has to come from every builder equally. The next field added to a fleet request would otherwise arrive unsanitized while the wizard’s identical field is refused.

Since M4 there is a third filter builder, and it does NOT run the sanitizer — the one live exception to the rule above.

FccLegacyFleetSnapshotService.filterBySerials (used by topUp, the legacy-import fleet snapshot) constructs an FtacsDeviceFilter directly and hands it to ftacsDeviceResolveDao.find(…​) with no call to FccDeviceFilterSanitizer anywhere in the class — unlike FccDeviceResolveService.filter and FccTestValidator.resolveDevices, both of which run every filter through it.

It is harmless today, and the reason is specific rather than structural. The serials it builds the filter from come from FtacsLegacyResultDao.findSerials, which reads GROUP BY r.serial off a result table — a GROUP BY cannot itself produce a null grouping key for a NOT NULL legacy column — and the caller additionally runs every row through recencyBySerial, which drops anything that fails StringUtils.hasText before the list ever reaches filterBySerials. So the one failure mode the sanitizer exists for here — a null or blank entry silently narrowing an IN list to nothing — cannot occur on this particular input, by construction of its source rather than by a guard in this class.

That is exactly why it must not be read as a second, safe way to skip the sanitizer: the safety is a property of this caller’s upstream data, not of the DAO, which — as stated above — refuses nothing and validates nothing and will turn whatever list it is handed straight into a predicate. A future change to filterBySerials that lets an unfiltered or externally-supplied list reach it would reopen exactly the defect FccDeviceFilterSanitizer was written to close, with no compiler or test to catch it. It should run the sanitizer too, for the same reason FccTestValidator.resolveDevices runs it over an already-non-null list: the guarantee has to come from every builder equally, and this is currently the one builder where it does not.

domainIds is deliberately not sanitized.

It is not caller-supplied: it comes from FccDomainScopeResolver, which has already decided what this caller may see, and an empty list means scope ALL to every method of FtacsDeviceResolveDao (The Empty Intersection Selects Nothing). Refusing it here would break the super-domain read; emptying it would hand a caller the whole installation. The one place an empty domain scope is a selection of nothing is FccDomainScope.nothing(), and the service short-circuits on that before a filter is ever built.

models is matched exactly while manufacturers is matched case-insensitively, and the asymmetry is in the shipped code rather than a documentation slip.

It is defensible — a model string is offered to the caller by the catalog and comes back verbatim, whereas a manufacturer name is the field an operator pastes from a saved filter — but it is not stated anywhere in the code, and a caller who hand-types "f@st 5657" gets nothing on Oracle and a match on MySQL. Flagged here rather than papered over; if it is ever made symmetric, the upper-casing belongs in FtacsDeviceResolveDao.upperCase next to the predicate, for the reason that method’s own comment gives.

The SQL Rules That Are Load-Bearing

FtacsDeviceResolveDao is written under four constraints that are decisions, not omissions, and each of them has a failure mode attached.

Before any of them, one property of the class as a whole: it refuses nothing and validates nothing. The filter is expected to arrive already sanitized from whichever builder made it (A List of Nulls Is Refused, Not Quietly Emptied), so the DAO never decides that a selection is malformed and never names an HTTP status. That is what lets every method here trust its argument uniformly — and it is why a new caller must run FccDeviceFilterSanitizer rather than assume the DAO will catch a bad list, because it will not: it will turn it into a predicate. That expectation is not currently enforced on every path that reaches this DAO — FccLegacyFleetSnapshotService.filterBySerials is a live exception, see A List of Nulls Is Refused, Not Quietly Emptied, and is safe today only because of a property of its own upstream data rather than because anything here would catch it if that stopped being true.

No Vendor-Specific SQL, and No Paging in SQL

One statement text, exercised by every installation.

There is no LIMIT, no FETCH FIRST, no ROWNUM and no OFFSET anywhere in this DAO. The two vendors spell row limiting differently, so a paged statement would be two statement texts — of which only the one the developer’s machine runs is ever exercised, and the other is discovered at a customer site.

What bounds the read instead is the fleet limit:

count(filter)                      -- one statement, no rows read
  > ft.fcc.max-devices-per-test ?  -- 400, nothing further happens
find(filter)                       -- the WHOLE match, ORDER BY c.id
  -> page sliced in memory, counters derived from the same rows

The full set is therefore small enough to hold (at most 20 000 rows by construction), one read serves both the page the operator is looking at and the totals printed above it, and the page is a subList. Deriving the counters from the rows in hand rather than from further aggregate queries is also what keeps them consistent with one another: a device that goes online between two statements cannot make the sum of the per-model counts disagree with the total.

ORDER BY c.id is not decoration. A deterministic order is what makes slicing the page in memory reproducible between two calls — an unordered read may repeat a device on page two and drop another.

There is exactly one path where the limit cannot be counted first, and it is branched on explicitly rather than paid for silently.

When the caller is visible in more domains than one IN list accepts, the domain narrowing happens in memory (The Domain Predicate Goes In Memory Above 1000 Visible Domains) — so COUNT() has no honest answer of its own and FtacsDeviceResolveDao.count falls back to find(filter).size(), materialising the row set itself. Counting first *there would read the largest installations twice.

FccDeviceResolveService.read therefore asks ftacsDeviceResolveDao.narrowsDomainsInMemory(filter) first:

Path Order

Ordinary

count (one statement, no rows), gate, then find.

More than 1000 visible domains

find once, then the gate applied to the rows already in hand — same message, same status.

The read it comes after is still bounded by the caller’s visible domains, so this is not an unbounded read. narrowsDomainsInMemory is public on the DAO for precisely this reason: the threshold and the decision belong next to the SQL, and a second copy of "more than 1000 domains" in the service is how the two would drift apart.

The alternative — a COUNT(*) that ignores the caller’s scope — would be a cheaper number that is simply not true, and a count that silently ignores the caller’s scope is the kind of wrong that never shows up as an error.

IN Lists Are Chunked at 1000, and Only One of Them Per Call

Oracle refuses more than FccDbConstants.IN_CLAUSE_MAX_EXPRESSIONS (1000) expressions in a single IN list. MySQL has no such limit, but the same code has to run on both.

Three lists reaching this DAO can exceed it, and they are handled differently on purpose:

List Treatment

serials

Chunked into several statements. It is caller-sized — it can legitimately hold the whole fleet.

cpeIds

Chunked, same reason. It is the save path’s list of already-resolved devices.

domainIds

Not chunked at all. See The Domain Predicate Goes In Memory Above 1000 Visible Domains.

Only one of the two caller-sized lists is chunked per call, and that is the point of FtacsDeviceResolveDao.chunks: a request carries either a serial list or an id list, never both — the wizard selects by conditions or by serials, and the save path selects by the ids the wizard already resolved — and chunking both would produce a statement per pair rather than per chunk. Should a filter ever arrive with both, the serials are chunked and the ids ride along whole in every chunk: narrower than dropping a predicate, which is the one outcome that would silently widen a selection.

An unfiltered read is a single empty chunk, so that every method here is one loop and "no list given" is not a second code path. Both lists reach the DAO de-duplicated — FccSerialListParser for the serials, a LinkedHashSet for the ids — so a device matches exactly one chunk and a summed count cannot count it twice. When more than one chunk ran, the merged rows are re-sorted in Java, because ORDER BY only orders within one statement and the page is sliced from the merged list.

The condition lists are refused rather than chunked, at 1000 entries (FccDeviceResolveService.validateCondition): a condition list is sized by the catalog the caller picked from, so an oversized one is a malformed request and not a fleet — see the refusal table in POST /FCC/devices/resolve.

The Domain Predicate Goes In Memory Above 1000 Visible Domains

Visible domains Where the predicate is applied

None (scope ALL, no narrowing)

Nowhere. No domain predicate at all.

1..1000

In SQL: c.location_id IN (:domainIds).

More than 1000

In memory, on the rows that come back: rows.removeIf(row → !visible.contains(row.domainId())).

The domain list is not chunked because it combines with whichever caller-sized list is present — chunking it as well would multiply the two chunk counts. Applying it in memory stays cheap precisely because the row set is bounded by the fleet limit and not by the domain count, so a cross product of statements would buy nothing.

A row with a null location_id is dropped by the in-memory arm, matching what the SQL arm does with it.

The catalog read solves the same problem differently, and correctly: FtacsCatalogDao.findModelsWithDevices reads one chunk of domains at a time and sums the partial rows per modelId in memory. Summing is exactly right there and not an approximation — a cpe row carries one location_id, so the chunks partition the device set and no device can be counted by two of them.

The LIKE Escape Character Is !, Not a Backslash

Every other LIKE in this service uses a backslash. This one cannot.

ESCAPE '\'   -- MySQL: unterminated literal; MySQL needs '\\'
ESCAPE '\\'  -- Oracle: ORA-01425, escape character must be
             --         a character string of length 1

There is no spelling of a backslash escape that both vendors accept, and this DAO is built around having one statement text for both. A character with no meaning in either dialect’s literal syntax gives that, so the statement is:

c.serial LIKE :serialPattern ESCAPE '!'

and FtacsDeviceResolveDao.escapeLike escapes ! first, then % and , into !!, !% and !.

Escaping is the case that actually bites. Serials contain underscores, and an unescaped is a single-character wildcard on both vendors — so a search box containing AB_12 would quietly match every serial of the same length with AB and 12 in those positions, instead of the one device the operator pasted. The operator would see a plausible-looking list of the wrong devices.

The protocol_id IS NULL Arm Is Not Optional

When TR069 is among the requested protocols, the predicate gains a second arm:

(c.protocol_id IN (:protocolIds) OR c.protocol_id IS NULL)

cpe.protocol_id was added long after the table, and a legacy CWMP device still carries NULL there. FccCpeProtocolEnum.fromProtocolId(null) reads that null as TR069, so the enum, the display and fcc_test_device.protocol all agree that such a device is a TR-069 device — and a WHERE protocol_id IN (0) would then contradict every one of them.

Without the arm, asking for TR-069 silently drops exactly the oldest half of an installation: no error, no warning, a device count that looks plausible, and a compliance campaign that quietly excludes the subscribers most likely to be on legacy hardware.

The arm is added only when TR069 is requested (filter.protocolIds().contains(FccCpeProtocolEnum.TR069.protocolId())), which is correct: a caller asking for USP alone must not get the nulls.

The Serial File FccSerialListParser Accepts

The accepted shapes are deliberately wide. A file exported from Excel is one serial per line, a file exported from a ticketing system is comma separated, and a list pasted out of an email is neither. Refusing any of them would only mean the operator reformats by hand, on a list of thousands, and loses one serial in the process — which is exactly the defect the fleet is being assembled to avoid.

Rule Detail

Encoding

UTF-8, always — never the platform charset. The file is produced on the operator’s machine and read in a container, and a serial carrying a non-ASCII character would otherwise resolve on a developer’s laptop and not in production.

Byte order mark

A leading UTF-8 BOM is dropped, rather than becoming part of the first serial. Windows tooling prepends it, and the resulting "not found" on line 1 of every uploaded file is a support ticket nobody can diagnose from the text.

Line breaks

Any (\R): LF, CRLF, CR, and the Unicode line separators.

Separators within a line

Comma, semicolon, or any whitespace ([,;\s]+) — which covers tab- and space-separated exports.

Comments

A line whose first non-space character is # is skipped entirely.

Blank lines

Skipped.

De-duplication

Case-insensitive, first spelling wins. MySQL’s default collation matches serials case-insensitively while Oracle’s does not, and the same file must behave the same way on both.

Reported spelling

The caller’s own. notFoundSerials echoes what was in the file, so the operator can search for that exact string in the file they uploaded.

Size

More than ft.fcc.max-devices-per-test distinct serials is a 400 before any statement is built.

FccSerialListParser.normalize applies the de-duplication, the ordering and the limit to a JSON serials array as well, so sourceType=INDIVIDUAL and sourceType=FILE cannot disagree about what "the same serial twice" means.

The XML Inventory FccSerialXmlParser Accepts

The upload of PUT /FCC/devices/import, and a different document from the one above: XML, read for its <serial> elements.

It is deliberately lenient, which is the opposite of the rule the configuration document follows — see config import/export, where an element nobody knows fails the import. The difference is not an inconsistency:

  • The configuration document is a format this service defines. A half-applied configuration measures something other than what the file says, so an unknown element is refused.

  • A device inventory is a format this service merely reads. The file comes out of the Group Update screen, an inventory system, or an operator’s own tooling, wrapped in whatever element names that tooling uses — and refusing the wrapper would only mean somebody edits a list of twenty thousand devices by hand.

Rule Detail

What is read

Every <serial> element, at any depth and whatever encloses it. Everything else in the document is skipped. This is what ImportService.processSerialsInBatches does for PUT /GroupUpdate/devices/import, which this endpoint is the twin of.

Element name

Matched case-insensitively (<serial>, <Serial>, <SERIAL>), as Group Update matches it — a file that worked on that screen has to work on this one.

No <serial> at all

400. The one guard leniency needs: a file whose serials are spelled <serialNumber> would otherwise be answered with "0 devices", which reads as "none of your devices exist" rather than as "this is not the element I read". Group Update stores an empty list instead and lets the operator find out on the next screen.

Nested element inside <serial>

400 naming the element. A serial is the text of the element; a <serial><value>…</value></serial> is a file generated from the wrong template, and the message is what says which.

Blank <serial/>

Dropped, not refused — it is what a spreadsheet exports for an empty cell, and it could only ever be reported as a serial matching nothing. Surrounding whitespace is trimmed.

Split text

A serial interrupted by a comment or a CDATA boundary is reassembled. IS_COALESCING merges adjacent text but is not required to merge across either, and half a serial resolves to a device that does not exist.

DTDs and external entities

Switched off on the factory (SUPPORT_DTD, IS_SUPPORTING_EXTERNAL_ENTITIES), so a document carrying a DOCTYPE is refused rather than expanded. Group Update’s reader leaves both at their defaults. This service has no authentication of its own, which is what makes an XXE here reachable by anything that can open a socket to it.

File name

Must end in .xml, case-insensitively, and a part with no file name is refused. This is the error message, not the validation — an extension proves nothing about content. A .csv posted here fails either way; checked by name it fails with a refusal that names POST /FCC/devices/resolve, which does read their text list. The name survives the proxy: ft-iot-webservice forwards each part as MultipartFile.getResource(), which reports the original filename.

De-duplication, spelling, size

Not re-implemented. The document’s serials go through the same FccSerialListParser.normalize as the table above, so an XML inventory, a text list and a pasted list cannot disagree.

Encoding

UTF-8. The reader is given the charset explicitly rather than trusting the XML declaration of a file produced elsewhere.

The document is bounded by The Upload Limit and by nothing else, exactly as the text list is.

The Upload Limit

Property Default Meaning

spring.servlet.multipart.max-file-size (FCC_SERIAL_FILE_MAX_SIZE)

10MB

Largest file part accepted.

spring.servlet.multipart.max-request-size (same FCC_SERIAL_FILE_MAX_SIZE)

10MB

Largest whole multipart request — the file part plus the optional request part. One environment variable feeds both, so raising the limit raises it coherently.

The bound exists because the upload is read whole into memory (file.getBytes()) and turned into a list of up to ft.fcc.max-devices-per-test strings. This service has no authentication of its own — it trusts ui-backend — so an unbounded upload is a denial of service anybody on the docker network can trigger. 10 MB is far more than a 20 000-serial list needs; a 20-character serial per line is roughly 400 KB.

Exceeding it answers 413 / 9413, with the configured maximum in the message — read from the property above rather than from the exception, and omitted altogether when no source can state a figure, so it is prose to render and never a number to parse. See the multipart status codes for why the exception’s own figure cannot be trusted under Tomcat. Two neighbouring multipart mistakes have their own answers as well — a missing file part is 400 / 9400, and a part offered with a content type nothing accepts (curl’s -F default for the JSON request part) is 415 / 9415. All three exist because `FccExceptionHandler’s catch-all would otherwise mask Spring’s own status and answer 500; see the multipart status codes.

Why This Is Not a Port of the Legacy GroupUpdate View Builder

Anyone who knows the platform will ask it, so it is answered here rather than left to be rediscovered.

ft-iot-webservice already resolves a device set from saved conditions: GroupUpdateSqlBuilder.buildSql turns a saved view’s ViewCondition list into a SQL string, which createGroupUpdateView stores next to the view for later execution. It is a working, shipped component, and reusing it would have saved this service a DAO.

It was not reused, and the reasons below are facts about that code rather than an opinion about it. They are listed because they are also, one by one, the argument for a closed typed condition set: every one of them is something a free-form condition vocabulary makes possible and a closed one makes unrepresentable.

Fact Consequence

buildSql flattens the conditions into a literal SQL string and stores it.

The stored text is what later runs. Nothing re-derives it from the conditions, so the query and the view can diverge, and the SQL a customer is running is whatever the build that saved it produced.

Ten of the 28 ConditionType constants fall through to a default arm that emits plain equality (column = ?).

Greater, GreaterEqual, Less, LessEqual, Regexp, Contains, NotContains, InHierarchy, ColSpan and RowSpan all become =. No error, no log line, no difference a reader of the SQL could spot.

NotContains becomes = — an exclusion turning into its own inverse.

The worst of the ten by a distance: the operator asked for "everything except X" and got "only X". Applied to a compliance fleet that is not a narrower selection, it is the complement of the intended one.

InHierarchy degrades to a single-domain equality.

"This domain and everything under it" silently becomes "this domain only" — the devices of every sub-domain vanish from the selection.

A condition naming a TR-069 parameter path emits 1=1 plus a SQL comment carrying the intended operator and value, and does not filter at all.

A view built on "firmware parameter equals X" selects the entire group. The comment preserves the intent for a future implementation; the query does not honour it.

InList binds the whole comma-joined string as one parameter: column IN (?) with ? = "a,b,c".

That matches a device whose column literally equals the string a,b,c — in practice, nothing.

Its saved views live in iotw_view and the condition rows beside it, in `ui-backend’s own schema.

ft-fcc-service reads those two tables and nothing else there (the grant), through a fourth, lazy, read-only datasource added for this one purpose — and it translates only the part of the condition vocabulary it can prove it translates identically, refusing the rest by naming the column key and the operator (the subset). That refusal is the point: it is the thing the legacy builder’s reachable default: arm makes impossible.

This is not a swipe at GroupUpdate. That module serves a different product with a different tolerance: a group update that selects too many devices pushes a firmware, which is visible, reversible and noticed within the hour.

A compliance campaign that selects the wrong devices produces numbers, and the numbers go into a regulatory filing. There is no moment at which anybody notices, and there is no undo.

The conclusion the facts above support is therefore narrow and specific: a free-form condition vocabulary makes "the operator asked for X and got Y" unrepresentable as an error, because there is no place where an unsupported combination could be refused — default: is always reachable. A closed typed set has no default: arm to fall through: an unrecognised protocol name is a 400 listing the allowed set, an oversized list is a 400 naming the limit, and a condition nobody implemented cannot be expressed at all. That is the property this service needed, and it is not one the legacy builder can be patched into having.

How a Resolved Selection Becomes the Frozen Snapshot

FccTestService.replaceDevices, inside the single PUT /FCC/item/details transaction:

  1. fcc_test_device.deleteByTestId(testId);

  2. flush() — explicitly, because fcc_test_device(test_id, cpe_id) UNIQUE would otherwise reject a re-inserted device that Hibernate had not yet removed;

  3. FccTestDeviceDao.insertAll(testId, rows).

The fleet is replaced wholesale, never diffed. A diff would have to decide what a row’s identity is beyond (test_id, cpe_id), and there is nothing else it could be.

The insert is hand-written JDBC and not saveAll. FccTestDeviceEntity is mapped GenerationType.IDENTITY, which is correct for the schema and for single-row work, but Hibernate unconditionally disables JDBC insert batching for identity-generated entities — it has to round-trip for every generated key. A saveAll of a full fleet is therefore up to 20 000 individual statements no matter what hibernate.jdbc.batch_size or rewriteBatchedStatements say, because no batch is ever formed. FccTestDeviceDao forms it, sharing the caller’s transaction and connection, and fills the four audit columns exactly as AuditingEntityListener would have on a create.

Every stored value comes from the ACS row, never from the request. serial, model_id, domain_id and protocol are read off FtacsResolvedDeviceRow; a caller-supplied serial or modelId on FccDeviceRefRequest is accepted for round-tripping and discarded. FccTestDeviceDao.bind is also the one place cpe.protocol_id becomes a stored name — the raw id travels out of the ACS schema untouched, and the null-means-TR069 coercion is applied there and nowhere else.

domain_id is copied from the cpe row, not from the test: a device may sit in a sub-domain of the test’s own domain.

A Device With No Product Class Is Still a Device

Since the resolution read joins product_class LEFT ([every-join-is-left]), a device selected by id or by serial can come back with a null modelId, model and manufacturer. Three places had to be able to say so, and all three do:

  • fcc_test_device.model_id is nullable, so the snapshot stores it as it found it.

  • FccTestValidator.validateConnectionCeilings filters the nulls out before asking FccModelCapabilityProvider for capabilities — a null model id has no capability row by definition, and looking one up would be a lookup by a key that cannot exist.

  • FccDeviceResolveService.countsByModel gives it a bucket of its own, under a null key with null names, and neither drops it nor merges it into another.

That last one is the load-bearing case: dropping it would be the one behaviour a compliance fleet cannot have — a per-model breakdown that quietly adds up to less than the campaign it describes. The sum over countsByModel stays equal to totalCount, which is what an operator reading the wizard’s grid is entitled to assume.

The Model Set Is Frozen With the Devices

There is no fcc_test_model table, and there should not be one.

A model is fcc_test_device.model_id, so an immutable device set is already an immutable model set. A second table could only hold a copy of SELECT DISTINCT model_id FROM fcc_test_device WHERE test_id = ?, and a copy that can disagree with its source is worse than the join it saves. The per-model figures a report needs are a GROUP BY over the snapshot, and they are then guaranteed to describe the same devices the run measured.

The freeze itself is fcc_test.activation_count > 0 — see fcc_test_device — and it is compared on the resulting set of cpeId`s, after resolution, so a reworded `configs[] selection that resolves to the same devices is a legal no-op edit.

What the Snapshot Buys, and What It Costs

Buys: a regulatory report shows the fleet the campaign actually ran against, not the fleet as it looks today. ftacs is read-only and may be a different database instance, so a join at read time is not available in any case — and fcc_test_device.protocol (M3) is what lets "how many of these devices were never going to produce a measurement" be answered about a finished run, weeks later, without re-resolving anything.

A third reader since M4. Through M3 this snapshot had two readers — the save path that wrote it, and the per-device status screen (fcc_device_state, fcc_device_state). POST /FCC/reports/list, /reports/results, POST /reports/generate and POST /reports/export are a third: a report filtered by model, manufacturer, cpeId or serial resolves through fcc_test_device, never through ftacs.cpe, for the identical reason protocol was frozen in M3 — a finished run has to be reported against the devices it actually ran on, and a CPE re-provisioned into another model since then must not rewrite that history. See Model and Manufacturer Are Resolved Through the Frozen Fleet and Endpoints Implemented in M4.

Costs: the snapshot goes stale, deliberately, and two rules exist to live with that rather than to fix it.

  • A device that loses its serial after the fleet was frozen does not block an unrelated edit: the save-time serial check covers only devices being added, and the activation-time gate is the one that inspects the stored set.

  • protocol is never back-filled. Filling it in from a live table would be a snapshot rewritten with today’s facts, which is the one thing this table must not contain.