Troubleshooting

Every entry here happened, either in this project's own history or in its end-to-end suite. They are the ones where the symptom points somewhere other than the cause.

An administrator is told they are not an administrator

you are not a member of directory-admins

…while being a member of directory-admins.

Cedar is default-deny, so an action no rule mentions is refused every time, and the refusal names the group rather than the gap. The case that produces it is an upgrade: Cardinal gains an action, the deployment keeps running its existing policy set, and nothing in that set mentions the new action.

The server lists ungoverned actions at startup for exactly this reason. Publish a policy set that names them.

The other cause is the step-up rule: administration requires a device-bound credential used within the administrative window, fifteen minutes unless configured. A synced passkey or a key touched this morning is refused, correctly, and the refusal says which window applies.

The application receives no cookies at all

Cookie is in authResponseHeaders and Cardinal is older than 0.6.2. Those versions send no Cookie header, and Traefik then forwards none, the application's own included. Upgrade Cardinal, or take Cookie out of the list until you do (forwardAuth).

Signing in lands on the console, not the site

Cardinal returns a browser to the site it came from only when that URL is https and on a registered application hostname or Cardinal's own host. A site the browser reached over plain HTTP, or a proxy that reports http in X-Forwarded-Proto because TLS ends at a load balancer in front of it, lands on the console's home instead (forwardAuth).

A header the application never receives

Cardinal sets it, Traefik drops it. Only the headers listed in authResponseHeaders are copied onto the original request, and anything missing from that list is discarded silently.

Cardinal shipped this bug itself: X-Auth-Request-Group-Ids was set on every response and listed nowhere, so no application ever saw the header the documentation told them to branch on. If a header from the table is missing, that list is the first place to look.

A policy rule that does nothing

A rule naming a group or application that does not exist never matches, and under a default-deny engine that is indistinguishable from the rule working.

cardinal policy test access.cedar -dsn "$DSN"

That reports every name the document mentions and the directory does not have. Publishing reports it too.

Activated, and not enforced

The console shows a version as active while this node still evaluates the previous one. Two causes:

• Seconds after an activation: each node checks on a short interval and the one you clicked on reloads immediately. Wait.
• The version does not compile. A set that fails to load is not swapped in, because swapping in nothing would deny everything and turn a bad publish into a total outage. The startup and reload logs say so, and the version list marks it.

The server refuses to start

• Missing migrations: run cardinal-server migrate. It is a separate step because applying a schema while other replicas serve the old one is how rolling deploys break.
• A migration this build cannot run against: the database carries a change declared breaking, and the reason was written into schema_migrations when it was applied, so an older binary can read it. Run the newer version.

A schema ahead of the binary is normal and warns rather than refusing.

A terminal sign-in waits and then gives up

Two different causes, and the message says which.

"nobody approved the terminal within two minutes" is the loopback handoff: approval comes back to a listener on the machine running the command, so opening the URL in a browser on another machine sends it to that machine's loopback, where nothing is listening. Either run it where the browser is, or use the other flow: -auth device prints a code you can approve from anywhere.

"nobody approved the terminal in time" is the device code, and it means exactly what it says: five minutes passed. Run the command again for a new one. Codes are deliberately short-lived, because a window that stays open is a window somebody can talk you into approving inside.

The command chooses a flow on its own and says which it picked. If it chose wrong (a multiplexer that outlives the SSH session it started in, a remote desktop where the browser really is here), -auth loopback and -auth device override it.

Passkey registration fails in the browser

WebAuthn needs a secure context and an origin that matches the configured relying party. localhost counts as secure without TLS; a bare IP address or a plain-HTTP hostname does not.

On the local stack this is why everything is served over HTTPS with a mkcert certificate rather than plain HTTP.

Traefik answers with a certificate for the wrong name

It serves its own self-signed default for a hostname the certificate does not cover, and the error names an internal Traefik hostname rather than the one that is missing: a genuinely confusing half hour. Reissue the certificate with every hostname the stack serves.

SCIM returns 403

Two things must both be true: the token carries the scim scope, and policy permits its owner to Provision. Neither alone is enough, and that is deliberate. See provisioning.

Mail is queued and nothing arrives

cardinal mail status      # what is queued, what is failing
cardinal mail test -to you@example.com

mail test prints what the relay actually said. An SMTP failure surfaced as "could not send" is a support ticket; the relay's own sentence is usually the answer.

A host behaves differently from its neighbours

cardinal-agent doctor
cardinal-agent status

doctor compares its own version against the server's, which is otherwise invisible, and reports missing prerequisites without changing anything.