Release Workflow

vX.Y.Z — the production release identity. Created only by misc/release.sh tag --qa-approved, once, on main (new release) or on the relevant release/X.Y.x (LTS patch). changelog-check.yml requires a matching ## vX.Y.Z entry in CHANGELOG.md for every vX.Y.Z tag pushed.

Optional beta/RC tags (e.g. vX.Y.Z-rc.1) — not currently produced by any script in this repository; if a pre-release needs to be shared before tag --qa-approved runs, tag it manually following that pattern so it never collides with a production vX.Y.Z tag.

X.Y.Z-b0.0.N (from misc/version-string.sh) is not a release tag at all — it is the build/runtime version string a running deployment reports, where N is the BUILD counter for the current VERSION. It identifies a specific build of a version, not a production release; the production release identity is always the vX.Y.Z tag, never this string.

The patch number is not simply VERSION + 1. The matching release/X.Y.x LTS line may already have tagged patches that main never saw (they are not reachable from main, so VERSION on main does not know about them). Check git tag -l 'vX.Y.*' and take the next free number, or git push rejects the tag as already existing.

misc/hub/install.sh’s REPO_BRANCH must name a branch that exists, or misc/check-versions.sh fails the pre-commit hook and step 2’s commit is refused. On main that value is main; a bare X.Y.Z is only tolerated while a legacy branch of that name still exists, and Git Flow does not create one. Skip the sed if it already says main.

The generated docs — docs/antora.yml and the Antora pages under docs/modules/ROOT/pages/ — are refreshed by the pre-commit hook itself once VERSION is staged; do not edit them by hand. Without the hooks installed, run ./misc/md2adoc.sh yourself and include the result in the same commit.

Push main before the tag, because gh release create --target main resolves main server-side: run against an unpushed main it attaches the release to whatever commit GitHub still has, which is the release before this one. (changelog-check.yml is not the reason — it triggers on the tag push and checks the tag out directly, so it is indifferent to main’s state.)

Pushing straight to main needs the access to do it. This procedure is the one documented exception to the no-direct-pushes rule below, and nothing automates it. On a main with branch protection actually enforced you need push permission or a protection bypass; without either, stop here and take the version-bump and CHANGELOG commits through a PR, then tag the merge commit.

Merge the fix into develop as well, so it isn’t lost on the next cut.

If a release/X.Y.x line already exists for this version and is still supported, bring the fix onto it too, or every install pinned to that line ships without it. Cherry-pick the fix commit — that line normally carries its own patch tags and its own chore(version) bumps, so it has diverged from main and a merge would drag main’s version bump onto it. This is the same rule the old-version hotfix section states in the other direction: fixes move between lines by cherry-pick, not by merging one release line into another.

main — no direct pushes; only fast-forward/--no-ff merges performed by misc/release.sh tag, plus PRs for hotfix/. *What “performed by misc/release.sh tag” actually requires: that path merges release/X.Y.Z into main locally and then runs git push origin main — a direct push, from whoever’s account is running the release. The release operator’s account needs repo Admin permission — enforce_admins: false exempts admins from the “require a pull request” rule, which is the only way a direct git push can land on a protected branch. Anyone without that permission running release.sh tag fails at that push with the vX.Y.Z tag already created locally; recovery is a re-run once they have it (the path is resumable — it skips the tag and the GitHub Release if they already exist). One further documented exception: the manual finalization in Latest-version hotfix workflow above pushes the version-bump and CHANGELOG commits to main directly, and needs the same Admin permission. Required status check: commit-lint (Conventional Commits on the PR’s commits). changelog-check runs on the resulting vX.Y.Z tag push, not on the PR. docs-pdf.yml’s bot-driven version-bump-and-push step cannot bypass this (it runs as github-actions[bot]/GITHUB_TOKEN, not an Admin) — see the note below; this is deliberate, not a gap to fix.

develop — no direct pushes except the version-bump commits misc/release.sh cut/tag push as part of their own automation (same Admin requirement as main); all other changes land via feature//bugfix/ PRs with commit-lint required.

release/* (both release/X.Y.Z and release/X.Y.x, via the release-branches ruleset) — direct pushes restricted to release automation and hotfix merges (same Admin-role bypass); commit-lint and branch-sync-gate required on PRs into a release/** branch.

Conventional Commits are enforced by .github/workflows/commit-lint.yml on every PR into main, develop, or release/**, using misc/lib/conventional-commits.sh. Merge/fixup/squash commits are exempted.

CHANGELOG.md is generated by misc/release-notes/generate.sh, invoked by misc/release.sh tag on both the new-release and LTS paths. .github/workflows/changelog-check.yml fails a vX.Y.Z tag push if CHANGELOG.md has no matching ## vX.Y.Z entry — the signal that the tag was pushed by hand instead of through misc/release.sh tag.

Changelog fragments — a .changelogs/<slug>.yml file with title: and description: keys, for a notice a commit subject alone would misrepresent (a security advisory, an operator action). generate.sh folds every pending fragment into the entry and moves it to .changelogs/archive/, staging the move on both sides so it is consumed exactly once. It requires yq, and refuses to run if fragments are present without it rather than dropping them silently.

The one-time tag transition. Production tags are vX.Y.Z, but every tag in this repository’s history is a legacy bare X.Y.Z (1.6.9, 1.6.8, …). For the first Git-Flow release there is therefore no v* tag to bound the commit range, and generate.sh falls back to the newest legacy X.Y.Z tag reachable from HEAD, announcing that it did so. Every release after the first is bounded by a real vX.Y.Z. If neither shape exists — a genuinely fresh repository — generate.sh refuses instead of emitting the entire history, and names the two escape hatches: --since <ref> to bound it explicitly, or --allow-full-history to accept the full range on purpose.

VERSION/BUILD feed misc/version-string.sh, which prints the build-qualified runtime string X.Y.Z-b0.0.N (N = the BUILD counter for that VERSION). This string is what a running deployment reports for itself; it is never the production release identity, which is always the vX.Y.Z tag created by tag --qa-approved.

During QA, release/X.Y.Z publishes X.Y.Z as a pre-release (prerelease: -dev). The release commit removes prerelease:, so after tag the same branch publishes X.Y.Z as released. It is never deleted.

An LTS patch vX.Y.Z pushes its own release/X.Y.Z at the tag.

release/X.Y.x (LTS lines) and develop carry the NEXT version with prerelease: -dev and are not published:

cut refuses a version that already has its release/X.Y.Z or its tag; move develop to the next free version first.

Every step skips itself when it is already done. A re-run after the last bump changes nothing and does not release the next patch.

Every released version publishes its docs from its release/X.Y.Z branch (Antora, docs-friendly-tech.pages.dev/grafana-monitoring/<version>/); see Which ref carries which docs version above. release/X.Y.x lines and develop are not published.

Feature branches that add or change docs get a docs build (preview) so the new content is reviewable before merge.

VERSION and docs/antora.yml are kept in sync per branch (pre-commit version check); on the LTS tag path, misc/release.sh updates docs/antora.yml’s version: itself.

No uncommitted changes.

GitHub CLI (gh) installed and authenticated (checked by tag before any mutation runs).

…​ | bash -s — <ip> — latest release from main

…​ | bash -s — --branch <ref> <ip> — a specific branch. The default comes from misc/hub/install.sh’s REPO_BRANCH (see Files touched by misc/release.sh` below for what each path writes there). It must be a branch, not a tag: bootstrap checks it out as `origin/<ref>. Today that means main, a release/X.Y.x LTS line, or a surviving legacy X.Y.Z branch.

…​ | bash -s — --version-list — show all versions

Release lines pin the release. cut pins release/X.Y.Z to X.Y.Z, the LTS tag pins each patch, and misc/check-versions.sh (pre-commit) refuses a release/X.Y.Z whose pins or compose defaults differ from VERSION. A release/X.Y.x line keeps its last release’s pins while the next patch is developed (any X.Y.*, the same in both places). On any other branch the two places only have to agree with each other; develop’s value is whatever the last release merged back.

CI publishes the version tag from release lines and main only. Every build is tagged <branch> and <branch>-<sha>; on release/** and main it is also tagged with VERSION. develop never gets a version tag: its VERSION is the next version and can equal an LTS patch number. The workflows build on a change to VERSION or .env.example, so the commit that moves a pin builds the image it names.

A release does not ship pins to missing images. A tag the registry does not have fails compose’s pull and aborts up -d for every service, and an image build can fail on its own (Trivy). tag therefore stops until both builds for the release succeeded — when there is none, or the last one failed, it starts one for the branch head and waits for it. To retry by hand: gh run list --workflow alertmanager-config-ui.yml --branch <branch>, then gh workflow run alertmanager-config-ui.yml --ref <branch> (same for snmp-catcher.yml).

Feature, bugfix and develop deploys run their own build. On any branch other than main and release/*, the installer (ensure_mirror_pins → follow_branch_image_pins) points each own image at the branch’s tag when the registry has it, and otherwise keeps the pin. QA of a branch therefore tests that branch’s Config UI and Trap Catcher without editing .env. An air-gapped host, which cannot reach the registry, keeps the pin.

On the new release/X.Y.Z branch: BUILD reset to 0, misc/hub/install.sh’`s `REPO_BRANCH set to main, and our own images (ALERTMANAGER_CONFIG_UI_VERSION, SNMP_CATCHER_VERSION in prometheus-grafana-stack/.env.example and their compose defaults) pinned to X.Y.Z — see Our own images. After the push cut starts both image builds on the new branch itself (gh workflow run, so cut needs an authenticated gh too): the push that creates a branch does not pass the workflows’ path filter and would build nothing.

On develop: VERSION advanced to the next patch/minor/major, BUILD reset to 0, REPO_BRANCH set to main.

Both commits are made before either branch is pushed, so a failure in either one leaves nothing published and the retry is two plain pushes.

BUILD incremented by one and committed on the current release branch (VERSION is never touched).

CHANGELOG.md — a ## vX.Y.Z entry, generated by misc/release-notes/generate.sh, committed on release/X.Y.Z. The commit range is bounded by the newest vX.Y.Z tag on main (passed as --since), not by a tag reachable from the release branch: Git Flow puts the tag on main’s merge commit, which never reaches develop.

the vX.Y.Z git tag, on main.

a GitHub Release for vX.Y.Z.

docs/antora.yml — the prerelease: line is removed in a commit before the changelog commit, so the tag carries a released version.

the changelog pages, changelog-nav.adoc and release.json (see Changelog pages and release.json above), in the changelog commit.

develop — gets the release branch merged back, but keeps its own VERSION, BUILD and docs/antora.yml.

release/X.Y.x — created (or updated) from the vX.Y.Z tag, then moved to VERSION X.Y.(Z+1), BUILD 0, docs/antora.yml X.Y.(Z+1) with prerelease: -dev, and REPO_BRANCH release/X.Y.x. Its own-image pins stay at X.Y.Z while the next patch is developed: installs from the LTS line run released images, not development builds (check-versions accepts any X.Y.* there, and the LTS tag moves them to the patch).

before anything reaches main, the own-image pins are checked: a branch cut before the pins existed gets a chore(version): pin own images to X.Y.Z commit (pushed, built), and tag stops unless both image builds for the branch succeeded — starting them itself when there are none.

VERSION — the next patch on that line (or --version if given). The line normally has this value already; BUILD is reset to 0 only when VERSION changes.

docs/antora.yml — version: 'X.Y.Z' and no prerelease: line, if the file exists.

CHANGELOG.md — a ## vX.Y.Z entry, generated the same way, with the changelog pages and release.json.

the own-image pins — set to the new X.Y.Z in the same chore(version) commit (which branch-sync-gate does not forward-port). After the push, tag starts both image builds for that commit and waits for them (up to 30 minutes); it stops if either fails or never starts.

the vX.Y.Z git tag, on release/X.Y.x.

a GitHub Release for vX.Y.Z.

then release/X.Y.x again: VERSION X.Y.(Z+1), BUILD 0, docs/antora.yml X.Y.(Z+1) with prerelease: -dev, pushed.