Runtime Substrate and Workbench Surface Contract

Status: Derived implementation contract — required for platform completion; does not supersede frozen architecture

Date: 2026-07-28
Authority: ARCHITECTURE_LOCK.md and IMPLEMENTATION_CONTRACT_v1.1.md

1. Binding rules

1. Schemas are pure entity-based. Persisted records represent a business or
   platform entity, relationship, event, evidence artifact, policy, capability,
   execution, or versioned definition. They are never created merely because a
   sidebar label exists. There is no home, inbox, work-view-1, or
   finance-manager source-of-truth table.
2. Projections are RBAC-based. Gateway derives tenant, membership, role,
   permissions, policy, entitlement and permitted connector context from the
   authenticated identity. Projection Engine then applies role, department,
   industry, data availability, freshness and user preference. The browser may
   request a view; it cannot select a broader data scope.
3. A Workbench surface never maps one-to-one to a physical table. It resolves a
   versioned projection contract through Gateway and Projection Engine.
4. A physical table may be a primary source, but every projection may use
   multiple supporting sources, permission rules, connector availability and
   capability state.
5. Product never reads or writes platform tables directly. A mutation is a
   semantic command through Gateway → Governance → Hermes where required →
   Pipeline promotion → projection invalidation.
6. The following runtime guarantees are mandatory Base Spine/platform
   capabilities: event transport, idempotency, capability resolution,
   discovery, queues, retry/replay, sync/reconciliation, observability, cache
   discipline and object/evidence storage. Industry and connector extensions
   may add mappings; they must not redefine these guarantees.
7. The Twin is silent by default. A conversational intake may be offered in a
   contextual surface, but it neither becomes a mandatory chat panel nor writes
   truth directly. Submitted content is triaged, evidenced, governed where
   required, then promoted through the Pipeline.

2. Runtime substrate

flowchart LR
    I["Ingress: webhook, poll, stream, API, MCP, user intent"] --> V["Validate + archive raw artifact"]
    V --> E["Immutable event + idempotency/deduplication"]
    E --> B["Event bus / stream / queue"]
    B --> D["Discovery and capability resolution"]
    D --> G["Governance"]
    G --> H["Hermes / worker runtime"]
    H --> P["Provider or local execution"]
    P --> O["Outcome event"]
    O --> S["Sync, verify and reconcile"]
    S --> W["Pipeline promotion to Spine"]
    W --> X["Projection invalidation and Workbench"]

The Projection Engine and Workbench sit above this substrate. They can observe
its state through semantic projections; they do not manage queue delivery,
stream offsets, provider execution or retries directly.

3. Runtime guarantees and table-status map

The table names in the Required completion records column are the target
contract. A name is not evidence that a migration exists. The migration-035
status below is a repository reading, not a live database verification.

Runtime guarantee	Existing migration-035 evidence	Required completion records / service state	Release rule
Event bus and Pub/Sub	spine_event_types, spine_events, topics, subscriptions, deliveries, offsets, dead letters	publisher registry, subscription filters, per-tenant ordering and replay policy	Immutable envelope, delivery attempt tracking, isolated subscriber failure and poison-event quarantine.
Idempotency and deduplication	event idempotency_key; sync mutation idempotency key	spine_idempotency_keys, deduplication records, operation fingerprints	Same logical operation + scope + key produces one accepted effect. Provider timeout is reconciled before a retry.
Capability matrix	capabilities, versions, provider/policy bindings, runs and steps	implementation, requirement, role, connector and availability matrix records	Resolve capability by tenant/workspace/role/connector/auth/data/policy; installation expands implementations, never redefines base capability.
Discovery	discovery runs, assets, capability candidates, domain manifests	snapshots, discovered objects/fields/events/operations, classification and extension candidates	Discovery describes and maps a provider. It never mutates physical schema automatically.
Queues and workers	event deliveries and continuity retries; Cloudflare runtime bindings are outside schema	runtime queues, message/lease/attempt/dead-letter metadata and queue policies	User work queues and platform execution queues are separate planes.
Streaming	event consumer offsets	stream/partition/checkpoint/consumer-group metadata where a provider needs ordered replay	Use for ordered/high-volume changes; do not confuse it with Pub/Sub fan-out.
Webhook ingress	integration webhook endpoints	verified deliveries, signatures, classified events and failure records; raw payload in R2	Verify signature, archive raw payload, check replay/idempotency, then publish canonical event.
Polling ingress	connector cursors and sync runs	poll definitions, schedules, leases, results/failures	Advance a cursor only after emitted events are durably accepted.
Sync and reconciliation	sync runs, mutations, conflicts, verifications	definitions, checkpoints, pending mutations, dead letters and reconciliation records	An API success is not closure; re-ingest, compare expected/actual, reconcile, then close.
Replay and reprocessing	delivery dead letters and continuity retries	reprocess request/run/item, replay checkpoint and dead-letter resolution	Replay is a governed operation with dry-run/execution mode, target consumers, side-effect policy and verification method.
Logging and traceability	audit log, event correlation/causation, entity trace/correlation fields	structured log references, runtime errors, trace spans, causation/lineage records	Redacted, tenant-aware, queryable and retention-controlled. Logs are not audit lineage.
Observability and health	connection health and selected runtime state	service health, health checks, metrics, SLOs, incidents, alert rules/deliveries	Cover infrastructure plus connector, queue, capability, sync and projection freshness.
Cache discipline	projection snapshots; D1/KV bounded-state contract	cache namespace, invalidation rule, refresh and health metadata	Cache is reproducible, versioned and non-authoritative; never cache secrets.
Object/evidence store	evidence links and object references	blob/version/link/retention/access/integrity metadata, with bytes in R2	Object store owns bytes; Neon owns metadata/ownership; knowledge and vector indexes are derived.

4. Canonical runtime envelope

Every ingress path normalizes to an envelope equivalent to this contract before
it enters the event bus or queue:

interface SpineEventEnvelope<T = unknown> {
  eventId: string;
  eventType: string;
  eventVersion: string;
  tenantId: string;
  workspaceId?: string;
  source: {
    type: "webhook" | "poll" | "stream" | "api" | "mcp" | "user" | "agent" | "system";
    provider?: string;
    connectionId?: string;
    externalEventId?: string;
  };
  subject: { entityType?: string; entityId?: string; externalEntityId?: string };
  traceId: string;
  correlationId: string;
  causationId?: string;
  occurredAt: string;
  receivedAt: string;
  idempotencyKey: string;
  payload: T;
  provenance: { rawPayloadRef?: string; evidenceRefs: string[]; schemaVersion: string };
}

traceId covers the end-to-end operation, correlationId groups related work,
and causationId identifies the direct parent command/event. These identifiers
must flow from user intent or ingress through approval, capability execution,
provider response, re-ingestion, verification and final Spine mutation.

5. Reliability policy: until successful, safely

“Until successful” means retry until success, terminal non-retryable state,
policy expiration, dead letter, or required human intervention. It never means
an unbounded blind retry loop.

accepted → idempotency reserved → attempt
attempt → succeeded → verify → reconcile → close
attempt → retryable failure → backoff → scheduled attempt
attempt → ambiguous provider outcome → query/reconcile before retry
attempt → terminal failure | policy expired | retries exhausted → dead letter / escalation

Every retry policy needs maximum attempts and elapsed time, backoff/jitter,
retryable versus terminal errors, circuit-breaker thresholds, dead-letter
handling, and human escalation conditions. Provider mutations such as payment,
email, provisioning, deployment or record update must not be repeated until
the remote effect is proven absent.

6. Adaptive Spine expansion law

The Spine adapts to more data without flattening the base model or forcing
the Product to hard-code provider schemas:

Base schema module
→ connector discovery
→ versioned provider schema
→ approved mapping to canonical entity/relationship/field definition
→ additive domain extension candidate where no canonical mapping exists
→ governed approval and migration where physical expansion is required
→ pipeline promotion, provenance and projection recompilation

The Base Spine remains the minimum standard: identity, entity, work, event,
knowledge, evidence, memory, governance, capability and continuity. A new
industry, role, department or connector can add fields, entity subtypes,
relationships, projections and capabilities, but cannot weaken tenant
isolation, governance, provenance, idempotency or the Pipeline-writer law.

7. Surface backend contract

interface SurfaceBackendContract {
  navId: string;
  projectionId: string;
  primarySources: string[];
  supportingSources: string[];
  requiredContext: ["tenant", "membership", "role", "policy", "freshness"];
  commands: Array<{ capability: string; approval: "none" | "policy" | "human" }>;
  runtime?: {
    publishTopics?: string[];
    consumeTopics?: string[];
    queue?: string;
    syncDefinition?: string;
    idempotencyScope?: string;
    reprocessingPolicy?: string;
  };
}

Shared navigation and changing work surface

The sidebar structure is common. What changes is the Work projection:

Base manifest + active tenant + role + department + industry
+ connector/schema availability + policy + user preference
→ Projection Engine → permitted Workbench surface

The twelve role/domain projections are Workbench views—not user-facing “L1”
labels. A view definition stores filters, grouping, layout and capabilities; it
does not store the operational truth displayed in that view.

Canonical sidebar shell

The following navigation shell is common across the Product. RBAC determines
whether a user sees an item and whether it is read-only, operational, or
administrative. The Work branch is the adaptive branch: its actual views
are composed from role × department × industry × tenant policy × connector
availability, rather than being hard-coded per provider or user.

IW
├── Home
├── My Work
│   ├── Inbox
│   ├── Tasks
│   ├── My Queue
│   ├── Approvals
│   ├── Notifications
│   ├── Recent Work
│   └── Favorites
├── Communications
│   ├── Activity Feed
│   ├── Email
│   ├── Calendar
│   ├── Meetings
│   ├── Chat
│   └── Calls
├── Documents Shared
├── Personal
│   ├── Notes
│   ├── Knowledge Base
│   └── Brainstorming Sessions
├── Activity
├── Finance Manager
├── Work
│   └── Adaptive views: role × department × industry
│       ├── View 1
│       └── View N
├── Platform
│   ├── Integration Manager
│   ├── Connector Marketplace
│   ├── Activation Hub
│   ├── Capability Manager
│   ├── MCP Pool (internal Platform operators only)
│   ├── AI Models
│   ├── Knowledge
│   ├── Memory & Continuity
│   ├── Automation
│   ├── Pipelines
│   ├── Governance
│   └── Observability
└── Administration
    ├── Organizations
    ├── Users
    ├── Roles
    ├── Permissions
    ├── Audit
    ├── Billing
    ├── Deployments
    ├── API Keys
    └── Security

Chat is a communication or contextual-intake surface. It does not turn the
Silent Twin into a permanent sidebar destination, and any contextual Twin
engagement still uses the Four Buttons and governed Pipeline paths.

Surface groups

Group	Primary projection responsibility	Typical supporting truth
Home	Composite operational summary	events, work, approvals, notifications, calendar, timeline, readiness.
My Work / Inbox / Tasks / Queue	Prioritized work and attention	work items, queue membership read model, messages, approvals, notifications, exceptions.
Communications	Human and provider communication continuity	messages, threads, calendar, meetings, calls, evidence and entity links.
Documents / Notes / Knowledge	Document, evidence and derived knowledge access	blobs, documents, shares, evidence, knowledge/chunks, retrieval indexes.
Personal	User-scoped preferences, favorites and permitted context	memberships, favorites, tuning profile, notifications, permitted memory.
Brainstorming	Domain extension, not a replacement chat history	session, messages, canvas state, ideas/insights, decisions, promoted knowledge.
Organizational Memory / Company Wiki	First-class operating handbook projection	promoted memory, knowledge pages, policies, rituals, decisions, linked subpages, embedded views, and approved outcomes.
Work	Role/department/industry operational projection	canonical entities, work, signals, relationships, capabilities and governance.
Integration / Activation	Tenant connectivity and readiness	catalog, installation/connection, discovery, schemas, webhooks, sync, health, activation/hydration.
Capability / MCP / AI	Permitted technical capability availability	registry, implementations, bindings, policy, health and runs. MCP Pool remains internal.
Knowledge / Continuity / Automation / Pipelines	Platform operational control views	memory candidates/promotions, continuity, definitions/runs, event/queue/replay state.
Governance / Observability	Accountability and health projections	decisions, approvals, audit, traces, health, incidents, SLOs and freshness.
Administration	Tenant administration	organisations, users, roles, permissions, billing, deployments, API-key metadata and security events.

Canvas and intake

Canvas views may render permitted context as cards, tables, timelines, images,
documents, graph fragments and safely formatted JSON data. They are
representations of a projection, never direct database or object-store access.

Organizational memory is a canvas-capable projection surface. In practice it
behaves like a company wiki or operating handbook: document-like pages, linked
subpages, embedded operational views, and approved write-back to Spine when the
user and governance allow it. It is a projection over memory and knowledge, not
a second truth store.

A contextual intake/chat surface may accept text, files or links. It must:

1. archive the original artifact to the evidence/object-store path;
2. create an observation/event with tenant, provenance and idempotency;
3. have the Twin triage and propose classifications silently or on request;
4. use governance for consequential routing or actions; and
5. promote only validated outcomes to knowledge, work or canonical Spine truth.

8. Delivery checkpoint

Before a testing-environment event, provide evidence for:

1. one ingress-to-reconciliation trace with event, idempotency, approval,
   execution, re-ingestion and projection invalidation identifiers;
2. one retry/ambiguous-provider scenario that proves no duplicate side effect;
3. one dead-letter and governed reprocessing scenario;
4. one connector discovery-to-schema-mapping-to-projection scenario;
5. cross-tenant and cross-environment isolation for queue, cache, object,
   credential, webhook and projection access; and
6. a rendered Workbench showing shared navigation plus a role-specific Work
   projection, a canvas projection and explicit partial/degraded state.