Reference: Developer Documentation

Version 1.6.9 | Updated: September 04, 2026

Reference: Developer Documentation

For Developers

Documentation

This repository automatically maintains Antora-formatted documentation alongside the Markdown README.

Automatic Conversion

When you commit changes to README.md, a pre-commit hook automatically converts it to AsciiDoc format for Antora documentation. The converted file is placed in docs/modules/ROOT/pages/index.adoc and staged for the commit.

Setting Up Git Hooks

After cloning this repository, install the Git hooks:

./misc/install-hooks.sh

This installs a pre-commit hook that:

  • Detects when README.md is staged for commit

  • Converts it to AsciiDoc using pandoc

  • Stages the converted index.adoc file automatically

Requirements:

  • pandoc must be installed:

# macOS
brew install pandoc

# Ubuntu/Debian
sudo apt-get install pandoc

# CentOS/RHEL
sudo yum install pandoc
Manual Conversion

To manually convert the documentation:

./misc/md2adoc.sh
Antora Structure
docs/
├── antora.yml                    # Antora component descriptor
└── modules/
    └── ROOT/
        ├── nav.adoc              # Navigation menu
        └── pages/
            └── index.adoc        # Main documentation (auto-generated)

Container images

Every image this stack runs exists in our own registry, scanned, at the version we ship. Not for convenience: an install that pulls from six third-party registries is only as available as the least reliable of them, and versioned tags do disappear — Bitnami deleted every one of jmx-exporter’s, which is why that exporter is pinned by digest.

  • hub.friendly-tech.com/mirror/… — the third-party images (Prometheus, Grafana, Alertmanager, the exporters, nginx, mailpit). The Harbor project is public with automatic scanning, so pulling needs no credential.

  • hub.friendly-tech.com/monitoring/… — the two images we build ourselves, snmp-catcher and alertmanager-config-ui, tagged with the version branch that built them. This project is private, so an online install needs one docker login on the target host before deploying the stack. An air-gapped install needs none: the images come in the offline bundle like every other.

    Until 1.6.9 these two were built on the customer’s host at install time — the only images in the stack that were. That needed egress to PyPI and the Alpine mirrors exactly where it is least likely to exist, and a failed build left the install running without the service. Nothing is built on a target host any more.

Read-only account for the private project:

registry: hub.friendly-tech.com
username: ast
docker login hub.friendly-tech.com -u ast

It is guest on both projects — it can pull and nothing else; a push is rejected. Ask for the password rather than reusing a personal account.

Harbor locks an account after repeated failed logins, and then refuses the correct password from every host — docker login answers Get "https://…/v2/": unauthorized, which reads like a client or network problem and is not. If that happens, an administrator clears it by resetting the password.

How the mirror stays in step with what we install. The list is never maintained by hand. misc/list-stack-images.sh computes it from the same .env.example and compose files the runtime reads, and the Mirror stack images workflow (mirror-images.yml) runs on any change to those files — so bumping a version in .env.example is what syncs it. It copies with skopeo --all, keeping multi-arch images multi-arch, and fails the run rather than leaving the mirror partial. A weekly run catches anything lost to a registry cleanup.

bash misc/list-stack-images.sh            # what we ship, at its pinned version
bash misc/list-stack-images.sh --mirror   # the same, at our registry

What redirects a deployment to the mirror is a line in .env, not the compose file. The compose files keep naming the UPSTREAM image as their default — that is what tells the mirror workflow what to copy — and .env.example carries NAME_IMAGE=hub.friendly-tech.com/mirror/… for each of them. Since .env.example is only copied when there is no .env, a host installed before the mirror existed would keep asking for upstream names that the offline bundle does not contain — an air-gapped upgrade reaching for Docker Hub, on exactly the hosts that cannot reach it. So bootstrap and quick-deploy re-apply the pins on every run, the same way they refresh DOCKER_GID and CONTAINERD_SOCK. A pin already in .env is never touched, including a commented-out one — that is how you keep a single image on upstream.

The two images we build are published by their own workflows, which fire when their sources change (prometheus-grafana-stack/snmp-catcher/ and …/alertmanager-config-ui/).

Continuous Integration

Six workflows live in .github/workflows/:

  • Sandbox tests (sandbox-tests.yml) — runs the installer test suite in a container on every push to a 1.* branch and on every pull request.

  • Generate PDF Documentation (docs-pdf.yml) — rebuilds the PDF handouts.

  • Trigger Docs Portal Rebuild (trigger-docs-portal.yml) — notifies the portal repository when README.md, docs/* or VERSION changes. Note the portal builds only from 1. branches and tags, so a docs fix on main changes nothing.

  • Build SNMP Catcher / Build Alertmanager Config UI — publish those two images to Harbor, tagged by branch. latest is pushed only from main.

  • Fixture Drift (fixture-drift.yml) — weekly comparison of the sandbox fixtures against the reference host, so the mocks cannot go on replaying a world that has moved. Two jobs: record runs on a self-hosted runner on the reference host, because its ufw admits inbound ssh from an allowlist that hosted runners are not on, and a runner long-polls outbound instead; analyse runs hosted, because the secret scan and the sandbox suite need Docker and that runner’s account deliberately has none. It opens one rolling issue rather than committing, because a fixture change alters what every sandbox case believes about reality. Setup in misc/testing/sandbox/README.md.

Two things about triggers are worth knowing before adding a workflow here.

schedule runs on the default branch only. GitHub takes the workflow from the latest commit on main regardless of where it was developed, so a weekly job that lives only on a version branch never fires. That is why fixture-drift.yml is on main as well as on the development line.

workflow_dispatch is not limited to the default branch — it dispatches against any ref, and the version that runs is the one on that ref. Measured 05.08.2026: gh workflow run fixture-drift.yml --ref 1.6.9 ran seven hours before the file existed on main at all.

What does trip people up is registration, which is a different thing: GitHub does not know a workflow exists until some event has caused it to read the file. A workflow whose only triggers are schedule and workflow_dispatch gives it no such event on a version branch, so the API reports it as non-existent and there is nothing to dispatch. Push the file with any trigger that fires once — a temporary push: is enough — and dispatch works from then on, on any branch.

Development & Releases

The project uses version branches for development and releases. This describes what misc/release.sh does today.

A trunk-based model was agreed instead (DEV-1872, 23.07.2026) and is written up in docs/RELEASE-WORKFLOW.md — but the tooling has not been changed to match, so that document is the target and this section is the practice. Do not follow it expecting release.sh to behave that way.

Workflow:

1. Work on version branch (e.g., 1.6.9)
2. When ready: ./misc/release.sh [patch|minor|major]
   → Clears the prerelease marker in docs/antora.yml
   → Merges to main, tags, creates the GitHub Release with a changelog
   → DELETES the version branch, locally and on the remote
   → Creates the next version branch (e.g., 1.6.10) with the marker back
3. Continue development on new branch

Two steps in there surprise people, so they are spelled out rather than left in the script:

  • The version branch is deleted, both locally and on the remote. It has been merged and tagged by then, so nothing is lost — but anything still unpushed on it is, and a colleague tracking that branch will find it gone.

  • The prerelease: marker is stripped before the merge. Every new development branch carries prerelease: -dev in docs/antora.yml, and nothing used to remove it, so a released line kept publishing itself to the docs portal as X.Y.Z-dev, sorted below an older clean version and never becoming latest. If a release ever shows up on the portal with a -dev suffix, this is the step that did not run.

Version branch naming:

  • Branches: 1.6.9 (without v prefix)

  • Tags: 1.6.9 (without v prefix)

  • Releases: 1.6.9 (without v prefix)

Using specific versions:

# List available versions
curl -fsSL https://hub.friendly-tech.com/bootstrap/install.sh | sudo bash -s -- --version-list

# Install specific version
curl -fsSL https://hub.friendly-tech.com/bootstrap/install.sh | sudo bash -s -- stack --branch 1.6.9

tag 1.6.9 · commit a4c0a5e133c2cba0a8b461039e9634af33e3dd9b · page generated September 04, 2026