Conformance

A conformance run acts as a patient, and then as an adversary, against your sandbox application. It watches your requests and webhook answers and grades eleven checks. Production doesn't require a run, but run one before your first production patient.

Before you start

• A separate application in sandbox status. A run sends corrupted webhook probes, so don't use an application that's live.
• A sandbox key. A production key gets 403 sandbox_only.
• Your webhook URL set, if you want the webhook checks graded.
• No other open run on the application. A second returns 409 run_in_progress.

Start a run

The response holds run_key, a sandbox key for your application. FinchNode counts its requests toward the run, and revokes it when the run ends. It appears once. Save it to a private temporary file outside your repository, and keep it out of logs and transcripts:

RUN_KEY_FILE=$(mktemp)
RESPONSE=$(curl -s --fail-with-body -X POST "https://api.finchnode.com/api/v1/conformance/runs" \
  -H "Authorization: Bearer $FINCHNODE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"scenario": "polypharmacy-senior"}') &&
  printf '%s' "$RESPONSE" | jq -er .run_key > "$RUN_KEY_FILE" &&
  printf '%s' "$RESPONSE" | jq '{id, status, phase, links}' ||
  printf '%s' "$RESPONSE" | jq .error

mktemp creates the file readable only by you. jq -e fails if there's no key, so a failed request never writes null in its place.

scenario is any record scenario. Add "sync_mode": "continuous" to grade the change feed too; that needs an application set to continuous.

Follow the run

1. Session. Your app creates a Connect session with run_key. FinchNode completes it as the patient. Up to 60 minutes.
2. Read. Your app reads the subject's records.
3. Pagination. Your app follows nextCursor to the end of a long category.
4. Rate limit. FinchNode answers up to two reads with 429 and Retry-After: 3. Retry each one, the same request, after at least 3 seconds.
5. Update. For continuous runs, the records change and records.updated arrives.
6. Revoke. The patient revokes consent and consent.revoked arrives. Make at most two more reads of the subject, then none for a full minute.
7. Probes. FinchNode sends five conformance.probe events: a valid one, a bad signature, another valid one, one signed an hour ago, and a replay of the first.

Poll GET /conformance/runs/{runId} for the phase and checks. Send record requests for the run's subject one at a time, so retries and cursor chains grade cleanly.

POST /conformance/runs/{runId}/advance grades the current phase on what it has seen and moves on. It can't end the session phase or interrupt probes being sent; both return 409 phase_not_advanceable. Advancing early can leave a MUST check inconclusive, which fails the run. POST /conformance/runs/{runId}/cancel ends the run.

Once the run ends, run_key stops working. Read the results with your own sandbox key.

Read the checks

Check	Level	Passes when
session.created	MUST	Your app created a session with the run key.
records.read	MUST	Your app read the subject after the session completed.
records.pagination	MUST	Each page carried the previous nextCursor, ending with hasMore: false.
ratelimit.retry_after	MUST	Your app retried each forced 429, the same request, after at least 3 seconds.
changes.incremental	SHOULD	After the update, your app read the change feed with a cursor and didn't re-read everything.
consent.fail_closed	MUST	After revocation, your app made at most two more reads of the subject, then none for a full minute. A 410 still counts as a read.
webhook.ack.valid	MUST	Your endpoint answered 2xx to the real consent.revoked.
webhook.ack.probe_control	MUST	Your endpoint answered 2xx to a valid probe.
webhook.reject.invalid_signature	MUST	Your endpoint answered 4xx other than 408 or 429 to a bad signature, then accepted the next valid probe.
webhook.reject.stale_timestamp	MUST	Your endpoint answered 4xx to a probe signed an hour ago.
webhook.replay.tolerated	SHOULD	Your endpoint answered the replayed probe with 2xx or 4xx.

A check is pending until the run decides it. Then it's pass, fail, skipped (with a reason), or inconclusive. MUST checks decide the verdict, and a failed or inconclusive one fails the run. SHOULD checks are reported only.

Share the report

links.report is a URL anyone can open without a key. It names your application and lists each check with counts and timings, never a record, subject, or key. The verdict is pass when every MUST check passed or was skipped. Reports are kept for 90 days.

A report describes behavior against synthetic data in the sandbox. It isn't a security assessment, a certification, or a statement about production readiness.