Rules for every module
The rules every module under tofu/ follows, their two deliberate exceptions, and how to run
one. Read this before writing or applying any module.
tofu/ covers provider-API resources Flux and Kustomize cannot own, because they live outside the
cluster: a DNS record, a registrar account, a mesh policy.
It is not used for anything Flux can reconcile. Pocket ID itself is a Flux-managed workload under
infra/auth/, not a tofu resource. The oidc module only registers its OIDC clients,
an operation against Pocket ID’s own API that no Kustomization can express.
Rules
Read-only against the secret stores. Never let a module write to one. Anything a module
mints becomes a sensitive output, filed by hand.
Exception: oidc and bifrost. Both write to Infisical, under a
machine identity scoped to /nodes/<hostname>/<app> and deliberately not the read-only identity
the cluster uses. oidc mints OIDC client secrets in Pocket ID, and the whole point of the module
is removing that hand-paste step for this one round trip. bifrost mints nothing anywhere: it
generates the gateway’s virtual keys locally, and exists because each one has to arrive at two
Infisical folders with the same value. Every other module stays read-only.
Provider tokens are never committed, in any form. A module’s tofu.<module> section of
config/sops/ops.sops.yaml holds two kinds of entry, and neither is a value. Identifying values that
grant nothing, such as a real public IP or an account ID, appear as TF_VAR_<name>: ....
Credentials appear as bare pass:// references that pass-cli run resolves after just tf has
loaded the section into the environment. No braces:
run resolves bare URIs and ignores braced ones, the inverse of inject. See
Secrets.
A genuinely non-identifying constant shared with other parts of the repository, such as the mesh
CIDR, is read straight from its committed source as a local rather than duplicated into
terraform.tfvars. The domain and the node addresses are identifying, so they come through
refs.env instead. See Domains.
State stays local and gitignored. tofu/**/.terraform/, tofu/**/*.tfstate* and
tofu/**/crash.log are all excluded. A module’s minted credentials can sit in state in plaintext
even when marked sensitive, which only suppresses console and plan output. Keep state on the
operator machine. .terraform.lock.hcl is the provider version lockfile and is committed.
Exception: b2. Local state makes a module non-portable, and for that one module
non-portable means broken. b2_bucket can only be imported by bucket id, and B2 bucket names are
globally unique, so a second operator machine starting from empty state does not adopt the bucket.
It fails the apply with duplicate_bucket_name. Its state lives in a B2 bucket instead, encrypted
client-side under a passphrase from Proton Pass, so the credential in it is ciphertext before it
ever reaches Backblaze. Every other module keeps its state local.
Verify provider resource and attribute names against current provider docs before the first apply.
Running a module
just tf init [<module>] # provider download only, unless the module has a backend
just tf plan <module>
just tf apply <module>
just tf init with no module argument inits every module under tofu/, and runs as part of
just ops setup. plan and apply compose both stores, in this order: the module’s section is
exported with sops -d --output-type dotenv --extract '["tofu"]["<module>"]', then
pass-cli run -- tofu <cmd> resolves the references in it. That needs a Proton Pass session
(pass-cli info) and the GPG smartcard present. Neither ever writes a value to disk.
init composes them too, but only for a module that ships a backend.tf, because initialising a
remote backend means authenticating against it. just tf init with no argument therefore asks for
the card and a Pass session as soon as one such module exists. The other modules still init on
nothing but a network connection.
One module needs more than a credential to apply: netbird writes account settings its own token
cannot reach at its normal role. See
Applying account settings.
Values another plane owns
A module never keeps its own copy of a value that already lives somewhere else. It declares a
reference in refs.env, which plan and apply resolve before running:
# <variable>=<repo-relative .sops file>#<sops --extract expression>
TF_VAR_edge_public_ip=config/sops/ops.sops.yaml#["nodes"]["ogma"]["ip"]
TF_VAR_domain=config/sops/cluster.sops.yaml#["stringData"]["DOMAIN"]
The expression goes to sops --extract verbatim, so one line shape reaches a node fact and a
Flux Secret alike. Who owns what:
| Value | Owner | Why |
|---|---|---|
| node addresses | config/sops/ops.sops.yaml, under nodes | roles/netbird writes each node’s mesh_ip back into it after a mesh join, so it is recorded where it is discovered |
| the domain and its subdomain labels | config/sops/cluster.sops.yaml | Flux substitutes the same keys into manifests, and Ansible reads them too |
| the backup bucket and its region | config/sops/cluster.sops.yaml | Flux substitutes both into K8up’s operator environment when it renders the HelmRelease, so that file has to hold them. b2 provisions what it names |
A value that is neither identifying nor secret needs no reference at all: tofu/netbird reads the
mesh CIDR straight out of the committed file that owns it, as a local.
refs.env holds only references, so it is committed in the clear even though both files it names
are encrypted. Both seal to the same operator key, so resolving one costs no extra card touch. A
module without the file is unaffected.
A variable in both refs.env and the module’s own section is not a harmless duplicate. The
section is exported after the refs, so its copy wins and the ref is silently dead. Keep each
value in exactly one of the two.
PUBLIC_IP and MESH_IP in config/sops/cluster.sops.yaml are the reverse direction and are a
deliberate second copy of what nodes records: Flux decrypts that file in-cluster and cannot read
config/sops/ops.sops.yaml at all. See
Why the operator store is separate.
The pre-commit tofu-validate hook only runs fmt and validate, never init, because a hook
that touches .terraform.lock.hcl fails pre-commit’s own “did this hook modify a file” check. Run
just tf init once locally before committing. CI runs init as its own step first. See
Checks and CI.
Modules
| Module | Manages |
|---|---|
bunny | Public DNS records in the existing Bunny DNS zone |
oidc | Pocket ID OIDC clients, writing the minted secret into Infisical |
netbird | Mesh access policy, account settings, and the internal DNS zone |
b2 | The Backblaze B2 bucket the restic repository lives in, and the application key K8up uses |
bifrost | The LLM gateway’s virtual keys, written into Infisical for both sides of each one |
What each touches. Every arrow into a secret store is a read except the ones marked in red, which
are the whole read-only rule and its two exceptions: oidc and bifrost write under a separate
identity, scoped to /nodes/<host>/<app>. Amber marks a third party this repository calls but does
not own, including the state bucket, which was created by hand.