Unclassified

Spine and Memory Model — Canonical vs Operational, Scopes, Promotion, Triage

Date: 2026-07-30
Status: Proposed — resolves D4 from RUNTIME_PROFILE_AND_ENTRY_CHANNEL_CONTRACT.md §9
Depends on: ARCHITECTURE_LOCK.md, CODEBASE_EXCAVATION.md §S8

1. The distinction

Two stores, two different kinds of truth, two different lifecycles. They are not tiers of the same thing.

	Operational Spine	Canonical Spine
Answers	"What is wired, and is it working?"	"What is true about the business?"
Contents	Connections, connectors, sync jobs, webhooks, schemas, capabilities, timeline	16 root concepts + 7 operational primitives
Lifecycle	High-churn; reflects runtime state	Append-only, versioned; reflects durable fact
Writer	The service that owns the runtime object	One writer — the Pipeline
Read by	Activation Hub, Integration Manager, Observability	Workbench projections, Twin, Capabilities
Store	D1 (edge-fast, tenant-partitioned)	Neon (durable, relational, versioned)
If lost	Rewire and resync	Unrecoverable

The clean formulation: Operational Spine is the state of the machine. Canonical Spine is the state of the business. Projections read canonical. Governance gates writes into canonical. Operational needs no governance gate because it holds no business truth.

Memory is a third thing, and §3 covers it — it is neither operational nor canonical, it is interpreted.

2. Code reality: the operational Spine is real; the canonical Spine does not run

This is the finding that resolves D4, and it is not what the documents imply.

2.1 Four implementations claiming Spine authority

Implementation	LOC	Store	Status
`services/spine/src/entity-store.ts` + `index.ts`	898	D1 (`DB`)	LIVE. 7 entity types: connections, connectors, sync-jobs, webhooks, schemas, capabilities, timeline-events. This is the Operational Spine, correctly built.
`services/spine/src/neon-workbench.ts`	558	Neon (`DATABASE_URL`)	LIVE. Tenant/actor-scoped Workbench projection. Named canonical by `CLAUDE.md`.
`services/spine/src/canonical-store.ts`	995	D1	DEAD. Zero importers. Declares the 16 root concepts + 7 primitives and states "This is the single source of truth… The pipeline is the sole writer." Its types were copy-pasted into `pipeline/src/canonical/entity-type-map.ts:16` — "Types duplicated from spine canonical-store to avoid cross-service dependency."
`services/pipeline/src/spine/index.ts`	2,773 (with `entity360-assembler`, `vault`, `crypto`)	Supabase	LEGACY. Self-labelled "the legacy writer (Supabase tables under 'spine'/'sales'/'cs' schemas)."

2.2 The canonical write path is bound to a placeholder

services/pipeline/src/canonical/canonical-writer.ts (296 LOC) is the intended canonical writer. It is invoked at services/pipeline/src/index.ts:1923:

// Write to canonical spine (16 root concepts + relationship graph)
if (env.CANONICAL_SPINE_DB) {
  const canonicalWriter = new CanonicalSpineWriter(env.CANONICAL_SPINE_DB);
  await canonicalWriter.write(rec, { create_relationships: true });
}

Its binding, in services/pipeline/wrangler.toml:118–120:

[[d1_databases]]
binding = "CANONICAL_SPINE_DB"
database_name = "integratewise-canonical-spine"
database_id = "YOUR_D1_DATABASE_ID_HERE"  # Create via: wrangler d1 create ...

The database ID is an unreplaced scaffold placeholder. The binding never resolves, if (env.CANONICAL_SPINE_DB) is always false, and every canonical Spine write is silently skipped. The surrounding try/catch logs to console.warn on failure — so even if it did resolve, failures would not surface.

The line immediately after is decisive:

// For writes, D1 is authoritative. Return success. (No legacy spine-v2 call for writes.)
return json({ success: true, written: entities.length, via: "d1-primary" });

The pipeline returns success for entities that were written only to the D1 entity cache (writeToEntityCache(env.CACHE_DB, …), database integratewise-spine-cache). A cache write is being reported as canonical persistence.

Conclusion: the Operational Spine is implemented and running. The Canonical Spine exists as an ontology, a dead 995-line store, and a writer bound to a non-existent database. Nothing durable is being written.

This also inverts the framing from CODEBASE_EXCAVATION.md §S8. There are not two competing canonical writers to choose between — there are zero working ones, plus a legacy Supabase writer that still runs.

2.3 Resolution of D4

The Canonical Spine is Neon. The Operational Spine is D1. services/spine owns both boundaries. The Pipeline is the only writer into canonical.

Concretely:

Provision integratewise-canonical-spine, or — preferred — drop CANONICAL_SPINE_DB entirely and have the Pipeline write canonical entities through the SPINE service binding to Neon. A second D1 database is not needed; it recreates the split.
Move canonical-store.ts's ontology into packages/spine-entities (which already exists and is already consumed by services/spine), and delete the copy-paste in pipeline/src/canonical/entity-type-map.ts. One definition of the 16 root concepts.
Change the console.warn swallow to a hard failure. A canonical write that fails must not return success: true.
Retire services/pipeline/src/spine/ once (1)–(3) land.
Rename for clarity: spine/src/entity-store.ts → operational; spine/src/neon-workbench.ts → canonical projection boundary. The word "Spine" alone should stop appearing unqualified in new code.

Item (3) is the highest-value single change in this document: it converts a silent data-loss path into a loud one.

3. Memory scopes

Memory is neither operational nor canonical. It is interpreted truth — derived from canonical facts and session evidence, carrying confidence and provenance, and therefore requiring governance that neither Spine needs.

3.1 Target model — four scopes, one dimension

Scope	Boundary	Governance	Store
Personal Memory	One user. Hard boundary — no other principal reads, including the org's own Twin acting for someone else.	User owns it. No approval needed.	`personal_memory`
Agent Memory	One agent/Twin instance within one tenant. Working state, learned routing, tool outcomes.	Not promotable directly to org. Must pass through triage as evidence.	`agent_memory`
Org Memory	Tenant-wide. The organisation's interpreted truth.	Governed. `staging` → approved → published.	`org_memory`
Conversational Memory	One session/thread. Ephemeral working context.	Decays. Never promoted directly.	`conversational_memory`

The rule that keeps this coherent: scope only ever widens through promotion, and promotion is always governed. Conversational and Agent memory are evidence, not truth. Personal memory never widens at all — it can only be cited by the user themselves.

3.2 Code reality: four competing vocabularies

Definition	Scopes	Status
`services/intelligence/migrations/d1_memory.sql`	`personal_memory`, `conversational_memory`	Tables exist. `personal_memory` correctly documented "Hard boundary — only this user reads", 8 categories, `governance_state`.
`services/intelligence/migrations/d1_knowledge.sql`	`org_memory`	Table exists, defaults to `governance_state = 'staging'`. Correct default.
`services/gateway/src/memory-governance.ts`	`MemoryScope = user \| team \| organization`; `MemoryType = conversation \| insight \| decision \| pattern \| workflow`	ORPHANED — not imported by `gateway/src/index.ts`. Introduces a `team` scope with no backing table anywhere in the repo.
`services/pipeline/src/spine/index.ts:1360`	`agent_memory`	Only definition in the repo. `CREATE TABLE IF NOT EXISTS agent_memory (kv_key, agent_id, session_id, type, …)` — inside the retired Supabase legacy writer.

Two structural problems.

Agent Memory exists only in retired infrastructure. If this repository is to own agent memory (per the operational/canonical repo split), the only implementation is in the Supabase writer scheduled for deletion. Agent memory needs a D1 table alongside personal_memory before pipeline/src/spine/ is retired, or the capability is lost.

org_memory is a cache being written as a store. d1_knowledge.sql:12 documents it as "Mirror of memory.org_memory for edge-fast reads." Yet six services write to it directly: pipeline/src/index.ts, pipeline/src/memory-promoter.ts, intelligence/src/memory/evolutionary-memory.ts, intelligence/src/handoff/outcome.ts, intelligence/src/workflows/triage-workflow.ts, mcp-connector/src/handlers/tools.ts.

This is the Spine problem repeated one layer up: a read-through cache treated as the system of record, with no single writer and no defined reconciliation against the Postgres memory.org_memory it claims to mirror. Either promote the D1 table to the store and delete the mirror language, or enforce a single writer and make the other five read-only. It cannot stay ambiguous.

There is also a schema-ownership split: org_memory is created in services/intelligence's migrations but written primarily by services/pipeline. Table ownership should follow write authority.

4. Promotion and Triage

4.1 The intended shape

Evidence                    Interpretation              Truth
────────                    ──────────────              ─────
conversational_memory  ┐
agent_memory           ├──► TRIAGE ──► candidate ──► PROMOTION GATE ──► org_memory
session events         │    (classify)               (govern)           (staging→approved)
canonical entities     ┘
                                                          │
                                            personal_memory (never promoted —
                                             widened only by the user, explicitly)

Triage classifies. The Promotion Gate governs. They are different responsibilities and must stay separate — a classifier that can also approve is not a gate.

4.2 Code reality: four governance models, two of them contradictory

Implementation	Decision vocabulary	Status
`pipeline/src/memory-promoter.ts` (415 LOC)	7 Promotion Signals — `important`, `repeated`, `strategic`, `behavioral`, `decision`, `preference`, `exception` → `approved` \| `staging`	LIVE. Queue-driven: `spine.written` → `MEMORY_PROMOTE` → `scoreForPromotion()` → governance gate → `org_memory` + `memory_promotion_audit` + AI Search index. Honours tenant `governanceDefaults.requiresApproval`. This is the good one.
`knowledge/src/synthesis/promotion-gate.ts`	`auto_approved` \| `hitl_required` \| `rejected` \| `deferred`	ORPHANED. Zero importers outside `synthesis/`.
`knowledge/src/triage-bot.ts`	Slack reactions ✅/❌/⏰	Neon + D1 + Workers AI.
`gateway/src/memory-governance.ts`	capture → triage → promote → decay	ORPHANED.

The contradiction, in one service:

knowledge/src/triage-bot.ts:13 — "The Triage Bot NEVER auto-approves. Every extraction requires customer review."
knowledge/src/synthesis/promotion-gate.ts:15 — "AUTO-APPROVE: High confidence + no governance flags + trusted domain."

Both live in services/knowledge. One of them is wrong. Given ARCHITECTURE_LOCK's "Governance is a hard gate" and the Twin's "never acts without explicit approval," the never-auto-approve position is the correct one for anything that will inform Twin reasoning or external action. memory-promoter.ts already implements the defensible middle: auto-promote to staging, which the Twin may not use until approved (its R11.3 comment) — approval automation without approval bypass.

The synthesis pipeline is built on retired infrastructure. synthesis/triage-bot.ts:4 — "Classifies raw CouchDB conversations"; synthesis/index.ts:14 — "CouchDB Session → TriageBot.classifySession() → ContinuityObjectFactory…"; synthesis/lineage-tracker.ts:279 — "Serialize lineage to a CouchDB-compatible document." CouchDB was superseded by the D1 memory tables (packages/coda-pack/template.ts:200 records the migration: "Rewrite continuity & intelligence caching from CouchDB to conversational, org, and personal memory tables"). The entire synthesis/ folder — triage-bot, promotion-gate, continuity-object-factory, lineage-tracker, knowledge-model — is a complete, unreferenced pipeline for a store that no longer exists.

Worth noting what is genuinely valuable in there and should be salvaged rather than deleted: lineage-tracker.ts and the UnderstandingObject provenance model have no equivalent in the live path. memory-promoter.ts writes an audit row but does not track lineage across promotions.

4.3 Resolution

pipeline/src/memory-promoter.ts is the single Promotion Gate. Delete knowledge/src/synthesis/promotion-gate.ts.
knowledge/src/triage-bot.ts is the single Triage classifier, retargeted from CouchDB sessions to conversational_memory + agent_memory + session events from services/continuity. Delete synthesis/triage-bot.ts.
Salvage lineage-tracker.ts into the promoter so promotions carry provenance, then delete the rest of synthesis/.
Delete gateway/src/memory-governance.ts. Its team scope has no store; if team memory is wanted, it is a new scope decision, not an orphaned type file.
Never auto-approve into approved. Auto-promotion may only reach staging. Make this a test, not a convention.
Add agent_memory as a D1 table in intelligence/migrations/d1_memory.sql alongside personal_memory, before pipeline/src/spine/ is retired.

5. Co-worker

This is already decided, already written down, and already orphaned.

packages/contracts/src/authority.ts (228 LOC) declares itself the "Canonical Authority Doctrine — supersedes all other architectural rules" and defines the boundary precisely:

Twin        = understands context
Co-worker   = performs preparation and bounded delegated work
Human       = owns decisions and authority

The Co-worker can say: "I found the relevant information, prepared the update, and recommend this action."
It cannot say: "I approved my recommendation and executed it."

And docs/wiki/21-adrs-glossary-invariants-and-superseded-concepts.md:66 records the decision as already taken:

"'Co-worker' as a second runtime → Twin is the contextual AI Co-worker behavior." — superseded concept

So the position is settled: Co-worker is a behavioural mode of the Twin, not a second runtime, not a service, not a memory scope. The doctrine matches the mechanisms already in place:

Co-worker behaviour	Existing primitive
Remembers your preferences	Personal Memory
Knows the org's decisions	Org Memory (promoted)
Picks up where you left off	Continuity
Learns from its own runs	Agent Memory
Prepares bounded work	Twin + Capability
Cannot approve its own work	Governance

5.1 The problem: the authority doctrine is in a dead package

packages/contracts has zero consumers (CODEBASE_EXCAVATION.md §3c). Nothing imports it. So the document that calls itself "supreme architectural principle… supersedes all other architectural rules" is enforced by nothing, and its neighbours share the fate:

File	Defines	Enforced by
`authority.ts`	Authority Doctrine, Twin/Co-worker boundary, approval lifecycle (`Draft → Proposed → …`), Triage Boundary	nothing
`surfaces/chat.ts`	"The Chat surface IS the Triage Hub" — nav id `chat`, route `/app/chat`, projection `platform.chat`	nothing
`surfaces/spine-access-plane.ts`, `platform.ts`, `admin.ts`, `iw-spine-network.ts`	Surface contracts	nothing
`marketplace/installation.ts`	Installation contract	nothing — `services/marketplace-installation` defines its own types

This matters more than the usual dead-package finding, because §4.2's contradiction has an authoritative answer sitting right here. authority.ts states the Triage Boundary explicitly:

The Triage Hub can receive, validate, classify, redact, deduplicate, link, quarantine, stage. It CANNOT independently decide that unverified material becomes organizational truth. Triage decides where a candidate should go for review, not whether it becomes final truth by itself.

That settles it: knowledge/src/synthesis/promotion-gate.ts's AUTO-APPROVE path violates the canonical doctrine. knowledge/src/triage-bot.ts's "NEVER auto-approves" is correct. The rule was written; it was just never wired to anything that could enforce it.

5.2 Resolution

Move authority.ts into packages/types (99 consumers) or packages/spine-schema, so the doctrine sits in a package that is actually imported.
Turn the Triage Boundary into a test, not prose. A unit test asserting that no promotion path can write governance_state = 'approved' without an approver identity is worth more than the doctrine as a string constant. Note that authority.ts currently exports the doctrine as a template literal — human-readable, machine-inert.
Adopt surfaces/chat.ts's definition: the Chat surface is the Triage Hub. This is a better answer than a separate "Triage" nav entry, and it connects the sidebar's Personal → Brainstorming Sessions to the triage pipeline.
Reconcile marketplace/installation.ts with the state machine's own types, or delete it (see RUNTIME_PROFILE_AND_ENTRY_CHANNEL_CONTRACT.md §4.1).
In user-facing copy, "Co-worker" is good positioning language. In code it remains Twin behaviour — never a type, table, service, or route.

Correction note: an earlier scan of this repository reported zero occurrences of "co-worker". That was a false negative caused by a case- and path-scoped grep. The term is present in 10 tracked files, including the two contract modules above. The conclusion was right; the evidence was wrong, and the real evidence is stronger.

6. What to fix, in order

#	Change	Why first
1	Make canonical write failure hard-fail instead of `console.warn` + `success: true`	Converts silent data loss into a visible error. One-line risk, highest value.
2	Route canonical writes through the `SPINE` binding to Neon; delete `CANONICAL_SPINE_DB`	Removes the placeholder binding and the second D1 canonical store
3	Move the 16-concept ontology to `packages/spine-entities`; delete the copy-paste in `pipeline/src/canonical/entity-type-map.ts`	One definition of canonical truth
4	Decide `org_memory`: store or cache. Enforce one writer.	Six writers into a documented cache is the §2 problem repeating
5	Add `agent_memory` to D1	Must land before `pipeline/src/spine/` is deleted or the capability is lost
6	Delete `synthesis/promotion-gate.ts` + `synthesis/triage-bot.ts`; salvage `lineage-tracker.ts`	Removes the contradictory governance rule
7	Retarget `knowledge/src/triage-bot.ts` off CouchDB onto the D1 memory tables	Makes triage real
8	Delete `gateway/src/memory-governance.ts` and `services/spine/src/canonical-store.ts`	~1,300 LOC of orphaned competing definitions
9	Retire `services/pipeline/src/spine/`	Now safe
10	Move `packages/contracts/src/authority.ts` into `packages/types`; make the Triage Boundary a test	The canonical doctrine is currently enforced by nothing (§5.1)

Steps 1–3 unblock the operational/canonical repo split in RUNTIME_PROFILE_AND_ENTRY_CHANNEL_CONTRACT.md §7. That split cannot be drawn while the canonical store is a placeholder.

7. Open decisions

#	Decision	Blocks
M1	Is `org_memory` (D1) the store, or a cache of Postgres `memory.org_memory`?	§3.2, step 4
M2	Does a `team` memory scope exist? (Currently a type with no table.)	§3.2, step 8
M3	May Agent Memory be cited by the Twin before triage, or only after promotion?	§3.1 — determines whether agent learning is immediate or governed
M4	Does Personal Memory ever widen to Org, and if so by what act?	§3.1 — the hard boundary is currently absolute
M5	Canonical Spine on Neon only, or Neon + a D1 read projection?	§2.3, step 2
M6	Is the Chat surface the Triage Hub, per `contracts/src/surfaces/chat.ts`?	§5.2, sidebar structure