Safety constraints
at the protocol layer

Safety is a protocol constraint, not an application concern

A safety layer that wraps a model's output sits outside the execution boundary — it observes outputs but cannot constrain what the model attempts or when dispatch occurs. For text applications, this is an acceptable trade-off. For physical robots, the failure mode is different: a command that clears a post-generation filter but targets the wrong actuator causes physical harm before any check can intervene.

RCAN addresses this by making safety constraints structural at the message transport layer. Confidence thresholds and human authorization gates are declared in the robot's RCAN configuration and enforced at dispatch time — not as application-layer checks the AI agent reasons around, but as protocol invariants the runtime enforces before it dispatches a command to an actuator.

One implementation is OpenCastor, an open-source robot runtime by the same maintainer that implements RCAN on Raspberry Pi 5, Hailo-8 and OAK-D hardware. It is not the reference for RCAN. The mechanisms described here run on the maintainer's own robots; they have not been tested by an independent party.

Every RCAN action scope has a configurable minimum confidence threshold declared in the robot's config. If the AI model's reported confidence for a proposed action falls below that threshold, the protocol rejects dispatch and emits a CONFIDENCE_GATE_BLOCKED audit record. The threshold is per-scope, so fine-grained control (higher threshold for physical manipulation, lower for camera streaming) is expressed in configuration rather than code.

In a conforming runtime this gate runs before the command reaches the transport layer, so rephrasing, another code path in the runtime or a different tool call does not get a low-confidence MANIPULATE command past it. It cannot stop another process on the same machine from driving the actuators directly; closing that path is the deployment's job (Layer 3, the gateway, holds it only when the gateway's service account owns the device).

agent:
  confidence_gates:
    NAVIGATE: 0.85
    MANIPULATE: 0.90
    CAMERA_STREAM: 0.70
    ESTOP: 0.50     # lower threshold — emergency actions always considered

Action types declared as requiring human authorization in the RCAN config cannot be dispatched through a conforming runtime without a signed AUTHORIZE message from a principal holding OWNER or higher role. When a gated action is attempted, the protocol emits PENDING_AUTH status and the command waits. The AI agent cannot proceed, cannot re-issue the command, and cannot escalate its own role.

This supports EU AI Act Article 14 (human oversight) at the protocol layer; it does not by itself meet it. The human-machine interface is the PENDING_AUTH → AUTHORIZE flow, and the gate is a structural constraint on the transport, not a UI affordance that can be bypassed.

HiTL gate configuration supports scope-based and context-based conditions (e.g., require authorization for NAVIGATE only when operating in a human_proximate location class).

agent:
  hitl_gates:
    - scope: MANIPULATE
      reason: "Physical contact with environment — human confirmation required"
    - scope: NAVIGATE
      location_class: human_proximate
      reason: "Shared-space navigation above 0.5m/s"

Every RCAN command is recorded in an HMAC-SHA256 append-only audit chain at dispatch time — not reconstructed after the fact. Each record includes: principal identity, RURI, timestamp (millisecond precision), message_id, outcome, model provider, model identifier, inference confidence, inference latency, thought_id, escalation flag, and (from §16.6) a watermark token that marks the command as AI-generated.

Records are chained: each entry includes the SHA-256 hash of the previous record. Changing any record breaks the hash of every record after it, so the change shows up on verification. Two limits: whoever holds the HMAC key can recompute the whole chain, and a chain alone cannot show records removed from the end. Anchoring the latest hash somewhere the writer cannot rewrite (a signed checkpoint, a registry) narrows both. The chain provides records for the automatic event logging that EU AI Act Article 12 requires of high-risk systems.

Messages are signed with the robot's ML-DSA-65 identity key (see post-quantum identity below). The combination of HMAC chaining and per-message signing means the chain is both tamper-evident and attributable.

{
  "msg_id": "cmd_a3f9c1d2",
  "type": "COMMAND",
  "ruri": "rcan://lab.local/acme/bot-x1/00000001",
  "principal": "operator@acme.com",
  "scope": "NAVIGATE",
  "timestamp_ms": 1744329600000,
  "outcome": "ok",
  "ai_block": {
    "model_provider": "openai",
    "model_id": "gpt-4o",
    "inference_confidence": 0.91,
    "inference_latency_ms": 312,
    "thought_id": "thought_b4e87f23",
    "escalated": false,
    "watermark_token": "rcan-wm-v1:a3f9c1d2b4e87f23"
  },
  "delegation_chain": [],
  "chain_prev": "sha256:9f3a...",
  "chain_hash": "sha256:2c7d..."
}

The RCAN protocol binds each Robot Registry Number (RRN) to an ML-DSA-65 public key (CRYSTALS-Dilithium, NIST FIPS 204, standardised August 2024). Every RCAN message is signed with the robot's identity key. Verification requires only the robot's public key — no central server, no network connectivity.

ML-DSA-65 provides security against quantum adversaries. Classical HMAC-SHA256 is used for per-record chaining within audit sessions (fast, symmetric); ML-DSA-65 is used for per-message attribution (asymmetric, forward-secure, quantum-resistant). The combination is appropriate for systems with lifetimes measured in years operating in environments where key material may be exposed to future quantum attack.

OpenCastor uses the dilithium-py binding. Key generation, signing, and verification are implemented in castor/rcan/pqc.py.

When Robot B executes a command at the direction of Human A, routed through Robot A, the human provenance must be preserved in the audit record. RCAN's delegation_chain array carries a signed record for each hop: issuer RURI, human subject, timestamp, scope, and ML-DSA-65 signature. The receiving robot verifies each signature in the chain before dispatch.

The chain is limited to 4 hops maximum. Commands exceeding this limit are rejected with DELEGATION_CHAIN_EXCEEDED. The audit record serializes the full chain, giving auditors complete provenance from actuator back to originating human principal.

This matters for multi-robot deployments in regulated environments: a compromised Robot A cannot issue arbitrary commands to Robot B without the human subject in the delegation chain holding the required scope on Robot B's RBAC configuration.

{
  "type": "COMMAND",
  "scope": "MANIPULATE",
  "delegation_chain": [
    {
      "issuer_ruri": "rcan://lab.local/acme/bot-alpha/00000002",
      "human_subject": "operator@acme.com",
      "timestamp_ms": 1744329600000,
      "scope": "MANIPULATE",
      "signature": "mldsa65:3f9a..."
    }
  ]
}

RCAN's PHYSICAL_PRESENCE message type enables a robot to cryptographically attest that a human operator is physically proximate at the time of authorization — using on-device sensor data (camera, proximity sensor, or external beacon) combined with a signed timestamp. This attestation can be required as a precondition for certain HiTL gate approvals.

Physical presence verification is specific to embodied systems. It has no analogue in text AI safety architectures. For regulated environments where certain actions require in-person oversight (operating theatre robotics, industrial manipulation in human-shared zones), this mechanism provides a machine-verifiable record that the oversight requirement was met.

robot-memory.md is a structured YAML-fronted file maintained by OpenCastor's nightly analysis loop (autoDream). Each entry is an operational observation — hardware degradation, environmental conditions, learned behaviour adjustments — with a confidence score that decays at 0.05/day if not reinforced by new log evidence.

Entries with confidence below 0.30 are excluded from context injection; entries below 0.10 are pruned on the next write cycle. This produces a self-maintaining operational history that reflects current system state rather than accumulating stale observations. The schema is designed for EU AI Act Article 13 (transparency) and Article 17 (quality management) alignment: the confidence decay mechanism directly maps to Art. 17's requirement for systematic monitoring of system performance over time.

schema_version: "1.0"
rrn: RRN-000000000001
last_updated: 2026-04-10T02:00:00Z
entries:
  - id: mem-a3f9c1d2
    type: hardware_observation
    text: "Left wheel encoder intermittent under sustained load — prefer ≤0.3m/s"
    confidence: 0.92        # decays 0.05/day if not reinforced by log evidence
    first_seen: 2026-03-28T14:00:00Z
    last_reinforced: 2026-04-10T02:00:00Z
    observation_count: 14
    tags: [wheel, encoder, navigation]

Schema specification: RobotRegistryFoundation/rcan-spec — robot-memory-schema.md