CPS Limitations

Jenkins pipelines run inside the Continuation-Passing Style interpreter provided by the workflow-cps plugin. It is a restricted Groovy dialect: every local variable and closure frame must be serialisable so the pipeline can be checkpointed and resumed. This page collects the rules that hurt in practice and the escape hatches used by this codebase.

What CPS changes

  • Every non-literal expression might be paused and re-entered. Anything on the local stack must serialise cleanly.

  • for (x in coll) { …​ } is not supported. Use .each { …​ }, .collect { …​ }, or a plain int i = 0; while (…​) loop.

  • try { …​ } catch { …​ } is allowed at the top level of a step body but not inside arbitrary non-serialisable closures.

  • synchronized blocks are not allowed. Shared state goes through file locks or atomic writes (utils.atomicWriteMeta).

  • Java / Groovy APIs that return non-serialisable types (java.io.File, java.util.regex.Matcher, groovy.json.internal.LazyMap) must be isolated behind a @NonCPS method.

The @NonCPS escape hatch

Annotate a method with @NonCPS and it runs under the normal Groovy interpreter, not under CPS. Its local stack is not checkpointable, but its return value — as long as it is serialisable — can cross back into the CPS world safely.

Common cases that need @NonCPS in this codebase:

  • File I/O (new File(…​).exists(), directory walks).

  • YAML / JSON parsing when the returned structure will be inspected (the LazyMap trap).

  • Regex Matcher objects and any =~ / ==~ that is not immediately consumed.

  • Closures over large collections (.collect, .findAll, .each with captured non-serialisable state).

  • HTML / text formatting helpers like utils.escapeHtml, utils.maskSensitiveValues, utils.parseEnvFileComments.

@NonCPS rules of engagement:

  1. It must return a plain Groovy / Java serialisable type.

  2. It should not call back into CPS-interpreted code.

  3. It must not rely on Jenkins-DSL-only calls (echo, sh, writeFile). Use them from the CPS caller instead.

Example pattern from utils.groovy:

@NonCPS
Set<String> extractEnvKeys(String content) {
    def keys = new LinkedHashSet<String>()
    content.eachLine { line ->
        def m = line =~ /^([A-Z_][A-Z0-9_]*)=/
        if (m.find()) keys << m.group(1)
    }
    return keys
}

The LazyMap trap

SnakeYAML (the engine behind readYaml) returns groovy.json.internal.LazyMap. That class is not Jenkins-serialisable, so passing it through anything that crosses a CPS checkpoint — stash, parallel, node(…​) — blows up with a serialisation error.

Fix: collapse the lazy tree into plain LinkedHashMap / ArrayList before it leaves the helper. utils.toBasic(Object) does exactly that:

def raw   = readYaml(file: dependenciesYaml)       // LazyMap
def clean = utils.toBasic(raw)                     // plain Map / List
writeJSON(file: output, json: clean)               // now safe

Any *Step.groovy that persists YAML-derived content to disk or passes it across node(…​) boundaries goes through toBasic.

Master side vs runner side

utils.exists(path) is @NonCPS and calls new File(path).exists(). That executes wherever the current node { …​ } block runs, but the @NonCPS frame itself evaluates on the master if the call sits outside a node(runner) { …​ } block. In practice:

  • Master-side file checks (under MASTER_ENVS_DIR) — use utils.exists freely.

  • Runner-side file checks (under AGENT_ENVS_DIR) — use fileExists inside the node(runner) { …​ } block. utils.exists will lie about those paths because master cannot see them.

No slashy regex in pipeline code

The Jenkins CPS parser rejects slashy regex (/\s+/, /\n/) in vars/*.groovy files — they fail at compile time. Use String regex instead:

// BAD — CPS rejects at load
line =~ /^([A-Z_]+)=/

// GOOD
line =~ '^([A-Z_]+)='

Active Choices script bodies are not parsed by CPS — they are strings embedded verbatim. Slashy regex works there, subject to the double-escape contract.

Typical serialisation errors

  • java.io.NotSerializableException: groovy.json.internal.LazyMap — run the value through utils.toBasic.

  • java.io.NotSerializableException: java.util.regex.Matcher — move the matcher logic into a @NonCPS helper that returns a String or boolean.

  • CpsCallableInvocation: …​ inside static initialisers — the static block tried to call a CPS helper. Make the helpers the static block depends on @NonCPS (this is why QAConfig.Limits.safeParseInt is @NonCPS).

Reading list