Conformance Test Suite v1.2

A conforming RCAN implementation MUST pass all tests at the level it claims. Levels are cumulative — an L3 implementation must also pass L1 and L2.

Level 1

L1 Core

RURI · mDNS · RBAC · Schema · §6 Audit

Level 2

L2 Safety

Safe-stop · Injection defense · HMAC audit chain · §16.2 Confidence gates

Level 3

L3 AI Accountability

§16.1 Model identity · §16.3 HiTL gates · §16.4 Thought log · Offline chain

L1 — Core

Every RCAN implementation must pass these baseline tests.

Test ID	Name	Area	Expected
`L1-RURI-001`	RURI format — valid standard form	RURI	✅ VALID
`L1-RURI-002`	RURI format — missing device-id segment	RURI	❌ INVALID · `INVALID_RURI`
`L1-RURI-003`	RURI format — with port and capability	RURI	✅ VALID
`L1-MDNS-001`	mDNS service type must be `_rcan._tcp`	Discovery	✅ Advertised correctly with required TXT keys
`L1-RBAC-001`	GUEST cannot issue COMMAND with `control` scope	RBAC	❌ DENIED · HTTP 403 · `INSUFFICIENT_PRIVILEGES`
`L1-AUDIT-001`	§6 Audit entry must contain all required fields	Audit	✅ `principal ruri timestamp_ms message_id event outcome`
`L1-SCHEMA-001`	Malformed message (missing payload) rejected before dispatch	Schema	❌ HTTP 400 / 422 · `SCHEMA_VALIDATION_FAILED`

RURI Pattern

rcan://<registry>/<manufacturer>/<model>/<device-id>
         ↑ lowercase hostname    ↑ ≥ 8 chars, lowercase + hyphens

Input	Result
`rcan://local.rcan/opencastor/rover/abc12345`	✅ VALID
`rcan://local.rcan/unitree/go2/a1b2c3d4:9000/teleop`	✅ VALID (with port + capability)
`rcan://local.rcan/opencastor/rover`	❌ missing device-id
`https://example.com/robot`	❌ wrong scheme
`rcan://UPPERCASE/mfr/mdl/12345678`	❌ uppercase not allowed
`rcan://local.rcan/mfr/mdl/short`	❌ device-id too short (<8 chars)

L2 — Safety

Builds on L1. Covers safety-critical behaviours from §14–§16.2 of the spec.

Test ID	Name	Area	Expected
`L2-SAFE-001`	COMMAND rejected when `estop_active=true`	Safe-stop	❌ REJECTED · `COMMAND_REJECTED / estop_active`
`L2-SAFE-002`	Heartbeat timeout triggers safe-stop	Network loss	State → `safe_stop` · audit: `NETWORK_LOSS_SAFE_STOP`
`L2-INJECT-001`	Payload containing "ignore previous instructions" is blocked	Injection defense	❌ BLOCKED · `COMMAND_REJECTED / prompt_injection`
`L2-HMAC-001`	Tampered audit record breaks HMAC chain	Audit integrity	Chain invalid from tampered record onward
`L2-CONF-001`	Confidence=0.3 blocked when gate `min_confidence=0.5 on_fail=block`	§16.2 Confidence gate	❌ REJECTED · `COMMAND_REJECTED / confidence_below_gate`
`L2-CONF-002`	Confidence=0.3 escalates when gate `on_fail=escalate`	§16.2 Confidence gate	Not dispatched → placed in HiTL queue

L3 — AI Accountability

Builds on L1 + L2. Covers §16 (AI Governance) of the spec.

Test ID	Name	Area	Expected
`L3-AI-001`	AI COMMAND audit must include `ai.provider` `ai.model` `ai.confidence`	§16.1 Model identity	All three fields present in audit entry
`L3-AI-002`	Audit missing `ai.model` is a conformance failure	§16.1 Model identity	❌ Non-conformant · `AUDIT_INCOMPLETE_AI_IDENTITY`
`L3-HITL-001`	HiTL gate: gated `action_type` must not dispatch immediately	§16.3 HiTL gates	Not dispatched → `queued_for_hitl=true`
`L3-HITL-002`	AUTHORIZE from OWNER allows dispatch; from GUEST is rejected	§16.3 HiTL authorization	OWNER → dispatch ✅ · GUEST → HTTP 403 ❌
`L3-THOUGHT-001`	Thought log: `reasoning` absent without `config` scope	§16.4 Thought log	Response contains `confidence`, excludes `reasoning`
`L3-CHAIN-001`	Exported JSONL + chain secret verifies offline	Offline chain verify	Chain valid, no network required

Running the Tests

L1 Live Checker

Run check_l1.py against your running RCAN endpoint:

# Basic check
python3 scripts/conformance/check_l1.py --host 127.0.0.1 --port 8080

# With auth token and verbose output
python3 scripts/conformance/check_l1.py \
  --host 192.168.1.42 \
  --port 8080 \
  --token <bearer-token> \
  --verbose

The script prints a per-test PASS / FAIL / SKIP table and a summary conformance badge. Exit code 0 = all passed.

Machine-Readable Test Cases

The full test suite is in scripts/conformance/rcan-conformance-v1.2.json. Drive your own harness against it:

import json

with open("scripts/conformance/rcan-conformance-v1.2.json") as f:
    suite = json.load(f)

for level, tests in suite["levels"].items():
    for t in tests:
        print(t["id"], t["name"])
        # run t["input"] against your implementation
        # compare result to t["expect"]

Conformance Badges

L1 CORE       ✅ RCAN Conformant v1.2
L2 SAFETY     ✅ RCAN Safety Conformant v1.2
L3 AI-ACCT    ✅ RCAN AI-Accountable v1.2

Relevant Error Codes

Code	Name	Description
1001	`INVALID_RURI`	Malformed RURI string
2002	`INSUFFICIENT_PRIVILEGES`	Role too low for operation
3003	`COMMAND_REJECTED`	Command failed safety / gate check
4001	`SAFETY_VIOLATION`	Action violates safety constraints
4002	`EMERGENCY_STOP`	Robot in emergency stop state
5001	`SCHEMA_VALIDATION_FAILED`	Message failed JSON schema check
5002	`AUDIT_INCOMPLETE_AI_IDENTITY`	AI audit entry missing model identity fields

→ Reference Implementations · View test files on GitHub