{"slug":"architecture/SPINE_AND_MEMORY_MODEL","title":"Spine and Memory Model — Canonical vs Operational, Scopes, Promotion, Triage","section":"architecture","sectionLabel":"Architecture","status":"unclassified","type":"document","path":"architecture/SPINE_AND_MEMORY_MODEL.mdx","bytes":29676,"description":null,"updated":null,"source":"integratewise/Integratewise-docs-wiki","content":"\n**Date:** 2026-07-30\n**Status:** Proposed — resolves D4 from `RUNTIME_PROFILE_AND_ENTRY_CHANNEL_CONTRACT.md §9`\n**Depends on:** `ARCHITECTURE_LOCK.md`, `CODEBASE_EXCAVATION.md` §S8\n\n---\n\n## 1. The distinction\n\nTwo stores, two different kinds of truth, two different lifecycles. They are not tiers of the same thing.\n\n|           | **Operational Spine**                                                         | **Canonical Spine**                           |\n| --------- | ----------------------------------------------------------------------------- | --------------------------------------------- |\n| Answers   | \"What is wired, and is it working?\"                                           | \"What is true about the business?\"            |\n| Contents  | Connections, connectors, sync jobs, webhooks, schemas, capabilities, timeline | 16 root concepts + 7 operational primitives   |\n| Lifecycle | High-churn; reflects runtime state                                            | Append-only, versioned; reflects durable fact |\n| Writer    | The service that owns the runtime object                                      | **One** writer — the Pipeline                 |\n| Read by   | Activation Hub, Integration Manager, Observability                            | Workbench projections, Twin, Capabilities     |\n| Store     | D1 (edge-fast, tenant-partitioned)                                            | Neon (durable, relational, versioned)         |\n| If lost   | Rewire and resync                                                             | **Unrecoverable**                             |\n\nThe clean formulation: **Operational Spine is the state of the machine. Canonical Spine is the state of the business.** Projections read canonical. Governance gates writes into canonical. Operational needs no governance gate because it holds no business truth.\n\nMemory is a third thing, and §3 covers it — it is neither operational nor canonical, it is _interpreted_.\n\n---\n\n## 2. Code reality: the operational Spine is real; the canonical Spine does not run\n\nThis is the finding that resolves D4, and it is not what the documents imply.\n\n### 2.1 Four implementations claiming Spine authority\n\n| Implementation                                    | LOC                                                   | Store                 | Status                                                                                                                                                                                                                                                                                                                             |\n| ------------------------------------------------- | ----------------------------------------------------- | --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `services/spine/src/entity-store.ts` + `index.ts` | 898                                                   | D1 (`DB`)             | **LIVE.** 7 entity types: connections, connectors, sync-jobs, webhooks, schemas, capabilities, timeline-events. This is the **Operational Spine**, correctly built.                                                                                                                                                                |\n| `services/spine/src/neon-workbench.ts`            | 558                                                   | Neon (`DATABASE_URL`) | **LIVE.** Tenant/actor-scoped Workbench projection. Named canonical by `CLAUDE.md`.                                                                                                                                                                                                                                                |\n| `services/spine/src/canonical-store.ts`           | 995                                                   | D1                    | **DEAD.** Zero importers. Declares the 16 root concepts + 7 primitives and states _\"This is the single source of truth… The pipeline is the sole writer.\"_ Its types were **copy-pasted** into `pipeline/src/canonical/entity-type-map.ts:16` — _\"Types duplicated from spine canonical-store to avoid cross-service dependency.\"_ |\n| `services/pipeline/src/spine/index.ts`            | 2,773 (with `entity360-assembler`, `vault`, `crypto`) | Supabase              | **LEGACY.** Self-labelled _\"the legacy writer (Supabase tables under 'spine'/'sales'/'cs' schemas).\"_                                                                                                                                                                                                                              |\n\n### 2.2 The canonical write path is bound to a placeholder\n\n`services/pipeline/src/canonical/canonical-writer.ts` (296 LOC) is the intended canonical writer. It is invoked at `services/pipeline/src/index.ts:1923`:\n\n```ts\n// Write to canonical spine (16 root concepts + relationship graph)\nif (env.CANONICAL_SPINE_DB) {\n  const canonicalWriter = new CanonicalSpineWriter(env.CANONICAL_SPINE_DB);\n  await canonicalWriter.write(rec, { create_relationships: true });\n}\n```\n\nIts binding, in `services/pipeline/wrangler.toml:118–120`:\n\n```toml\n[[d1_databases]]\nbinding = \"CANONICAL_SPINE_DB\"\ndatabase_name = \"integratewise-canonical-spine\"\ndatabase_id = \"YOUR_D1_DATABASE_ID_HERE\"  # Create via: wrangler d1 create ...\n```\n\n**The database ID is an unreplaced scaffold placeholder.** The binding never resolves, `if (env.CANONICAL_SPINE_DB)` is always false, and every canonical Spine write is silently skipped. The surrounding `try/catch` logs to `console.warn` on failure — so even if it did resolve, failures would not surface.\n\nThe line immediately after is decisive:\n\n```ts\n// For writes, D1 is authoritative. Return success. (No legacy spine-v2 call for writes.)\nreturn json({ success: true, written: entities.length, via: \"d1-primary\" });\n```\n\nThe pipeline returns `success` for entities that were written **only to the D1 entity cache** (`writeToEntityCache(env.CACHE_DB, …)`, database `integratewise-spine-cache`). A cache write is being reported as canonical persistence.\n\n**Conclusion: the Operational Spine is implemented and running. The Canonical Spine exists as an ontology, a dead 995-line store, and a writer bound to a non-existent database. Nothing durable is being written.**\n\nThis also inverts the framing from `CODEBASE_EXCAVATION.md` §S8. There are not two competing canonical writers to choose between — there are **zero working ones**, plus a legacy Supabase writer that still runs.\n\n### 2.3 Resolution of D4\n\n> **The Canonical Spine is Neon. The Operational Spine is D1. `services/spine` owns both boundaries. The Pipeline is the only writer into canonical.**\n\nConcretely:\n\n1. Provision `integratewise-canonical-spine`, **or** — preferred — drop `CANONICAL_SPINE_DB` entirely and have the Pipeline write canonical entities **through the `SPINE` service binding** to Neon. A second D1 database is not needed; it recreates the split.\n2. Move `canonical-store.ts`'s ontology into `packages/spine-entities` (which already exists and is already consumed by `services/spine`), and delete the copy-paste in `pipeline/src/canonical/entity-type-map.ts`. One definition of the 16 root concepts.\n3. Change the `console.warn` swallow to a hard failure. A canonical write that fails must not return `success: true`.\n4. Retire `services/pipeline/src/spine/` once (1)–(3) land.\n5. Rename for clarity: `spine/src/entity-store.ts` → operational; `spine/src/neon-workbench.ts` → canonical projection boundary. The word \"Spine\" alone should stop appearing unqualified in new code.\n\nItem (3) is the highest-value single change in this document: it converts a silent data-loss path into a loud one.\n\n---\n\n## 3. Memory scopes\n\nMemory is neither operational nor canonical. It is **interpreted truth** — derived from canonical facts and session evidence, carrying confidence and provenance, and therefore requiring governance that neither Spine needs.\n\n### 3.1 Target model — four scopes, one dimension\n\n| Scope                     | Boundary                                                                                                  | Governance                                                            | Store                   |\n| ------------------------- | --------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------- | ----------------------- |\n| **Personal Memory**       | One user. Hard boundary — no other principal reads, including the org's own Twin acting for someone else. | User owns it. No approval needed.                                     | `personal_memory`       |\n| **Agent Memory**          | One agent/Twin instance within one tenant. Working state, learned routing, tool outcomes.                 | Not promotable directly to org. Must pass through triage as evidence. | `agent_memory`          |\n| **Org Memory**            | Tenant-wide. The organisation's interpreted truth.                                                        | **Governed.** `staging` → approved → published.                       | `org_memory`            |\n| **Conversational Memory** | One session/thread. Ephemeral working context.                                                            | Decays. Never promoted directly.                                      | `conversational_memory` |\n\nThe rule that keeps this coherent: **scope only ever widens through promotion, and promotion is always governed.** Conversational and Agent memory are evidence, not truth. Personal memory never widens at all — it can only be _cited_ by the user themselves.\n\n### 3.2 Code reality: four competing vocabularies\n\n| Definition                                          | Scopes                                                                                                                  | Status                                                                                                                                                          |\n| --------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `services/intelligence/migrations/d1_memory.sql`    | `personal_memory`, `conversational_memory`                                                                              | **Tables exist.** `personal_memory` correctly documented _\"Hard boundary — only this user reads\"_, 8 categories, `governance_state`.                            |\n| `services/intelligence/migrations/d1_knowledge.sql` | `org_memory`                                                                                                            | **Table exists**, defaults to `governance_state = 'staging'`. Correct default.                                                                                  |\n| `services/gateway/src/memory-governance.ts`         | `MemoryScope = user \\| team \\| organization`; `MemoryType = conversation \\| insight \\| decision \\| pattern \\| workflow` | **ORPHANED** — not imported by `gateway/src/index.ts`. Introduces a `team` scope with **no backing table anywhere in the repo**.                                |\n| `services/pipeline/src/spine/index.ts:1360`         | `agent_memory`                                                                                                          | Only definition in the repo. `CREATE TABLE IF NOT EXISTS agent_memory (kv_key, agent_id, session_id, type, …)` — **inside the retired Supabase legacy writer.** |\n\n**Two structural problems.**\n\n**Agent Memory exists only in retired infrastructure.** If this repository is to own agent memory (per the operational/canonical repo split), the only implementation is in the Supabase writer scheduled for deletion. Agent memory needs a D1 table alongside `personal_memory` before `pipeline/src/spine/` is retired, or the capability is lost.\n\n**`org_memory` is a cache being written as a store.** `d1_knowledge.sql:12` documents it as _\"Mirror of memory.org_memory for edge-fast reads.\"_ Yet **six services write to it directly**: `pipeline/src/index.ts`, `pipeline/src/memory-promoter.ts`, `intelligence/src/memory/evolutionary-memory.ts`, `intelligence/src/handoff/outcome.ts`, `intelligence/src/workflows/triage-workflow.ts`, `mcp-connector/src/handlers/tools.ts`.\n\nThis is the Spine problem repeated one layer up: a read-through cache treated as the system of record, with no single writer and no defined reconciliation against the Postgres `memory.org_memory` it claims to mirror. Either promote the D1 table to _the_ store and delete the mirror language, or enforce a single writer and make the other five read-only. It cannot stay ambiguous.\n\nThere is also a **schema-ownership split**: `org_memory` is created in `services/intelligence`'s migrations but written primarily by `services/pipeline`. Table ownership should follow write authority.\n\n---\n\n## 4. Promotion and Triage\n\n### 4.1 The intended shape\n\n```\nEvidence                    Interpretation              Truth\n────────                    ──────────────              ─────\nconversational_memory  ┐\nagent_memory           ├──► TRIAGE ──► candidate ──► PROMOTION GATE ──► org_memory\nsession events         │    (classify)               (govern)           (staging→approved)\ncanonical entities     ┘\n                                                          │\n                                            personal_memory (never promoted —\n                                             widened only by the user, explicitly)\n```\n\n**Triage classifies. The Promotion Gate governs. They are different responsibilities and must stay separate** — a classifier that can also approve is not a gate.\n\n### 4.2 Code reality: four governance models, two of them contradictory\n\n| Implementation                              | Decision vocabulary                                                                                                                       | Status                                                                                                                                                                                                                                            |\n| ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `pipeline/src/memory-promoter.ts` (415 LOC) | 7 Promotion Signals — `important`, `repeated`, `strategic`, `behavioral`, `decision`, `preference`, `exception` → `approved` \\| `staging` | **LIVE.** Queue-driven: `spine.written` → `MEMORY_PROMOTE` → `scoreForPromotion()` → governance gate → `org_memory` + `memory_promotion_audit` + AI Search index. Honours tenant `governanceDefaults.requiresApproval`. **This is the good one.** |\n| `knowledge/src/synthesis/promotion-gate.ts` | `auto_approved` \\| `hitl_required` \\| `rejected` \\| `deferred`                                                                            | **ORPHANED.** Zero importers outside `synthesis/`.                                                                                                                                                                                                |\n| `knowledge/src/triage-bot.ts`               | Slack reactions ✅/❌/⏰                                                                                                                  | Neon + D1 + Workers AI.                                                                                                                                                                                                                           |\n| `gateway/src/memory-governance.ts`          | capture → triage → promote → decay                                                                                                        | **ORPHANED.**                                                                                                                                                                                                                                     |\n\n**The contradiction, in one service:**\n\n- `knowledge/src/triage-bot.ts:13` — _\"The Triage Bot **NEVER auto-approves**. Every extraction requires customer review.\"_\n- `knowledge/src/synthesis/promotion-gate.ts:15` — _\"**AUTO-APPROVE**: High confidence + no governance flags + trusted domain.\"_\n\nBoth live in `services/knowledge`. One of them is wrong. Given `ARCHITECTURE_LOCK`'s _\"Governance is a hard gate\"_ and the Twin's _\"never acts without explicit approval,\"_ **the never-auto-approve position is the correct one** for anything that will inform Twin reasoning or external action. `memory-promoter.ts` already implements the defensible middle: auto-promote to `staging`, which the Twin **may not use** until approved (its R11.3 comment) — approval automation without approval bypass.\n\n**The synthesis pipeline is built on retired infrastructure.** `synthesis/triage-bot.ts:4` — _\"Classifies raw **CouchDB** conversations\"_; `synthesis/index.ts:14` — _\"CouchDB Session → TriageBot.classifySession() → ContinuityObjectFactory…\"_; `synthesis/lineage-tracker.ts:279` — _\"Serialize lineage to a **CouchDB**-compatible document.\"_ CouchDB was superseded by the D1 memory tables (`packages/coda-pack/template.ts:200` records the migration: _\"Rewrite continuity & intelligence caching from CouchDB to conversational, org, and personal memory tables\"_). The entire `synthesis/` folder — `triage-bot`, `promotion-gate`, `continuity-object-factory`, `lineage-tracker`, `knowledge-model` — is a complete, unreferenced pipeline for a store that no longer exists.\n\nWorth noting what is genuinely valuable in there and should be salvaged rather than deleted: `lineage-tracker.ts` and the `UnderstandingObject` provenance model have no equivalent in the live path. `memory-promoter.ts` writes an audit row but does not track lineage across promotions.\n\n### 4.3 Resolution\n\n1. **`pipeline/src/memory-promoter.ts` is the single Promotion Gate.** Delete `knowledge/src/synthesis/promotion-gate.ts`.\n2. **`knowledge/src/triage-bot.ts` is the single Triage classifier**, retargeted from CouchDB sessions to `conversational_memory` + `agent_memory` + session events from `services/continuity`. Delete `synthesis/triage-bot.ts`.\n3. **Salvage `lineage-tracker.ts`** into the promoter so promotions carry provenance, then delete the rest of `synthesis/`.\n4. **Delete `gateway/src/memory-governance.ts`.** Its `team` scope has no store; if team memory is wanted, it is a new scope decision, not an orphaned type file.\n5. **Never auto-approve into `approved`.** Auto-promotion may only reach `staging`. Make this a test, not a convention.\n6. **Add `agent_memory` as a D1 table** in `intelligence/migrations/d1_memory.sql` alongside `personal_memory`, before `pipeline/src/spine/` is retired.\n\n---\n\n## 5. Co-worker\n\n**This is already decided, already written down, and already orphaned.**\n\n`packages/contracts/src/authority.ts` (228 LOC) declares itself the _\"Canonical Authority Doctrine — supersedes all other architectural rules\"_ and defines the boundary precisely:\n\n```text\nTwin        = understands context\nCo-worker   = performs preparation and bounded delegated work\nHuman       = owns decisions and authority\n```\n\n> The Co-worker can say: _\"I found the relevant information, prepared the update, and recommend this action.\"_\n> It cannot say: _\"I approved my recommendation and executed it.\"_\n\nAnd `docs/wiki/21-adrs-glossary-invariants-and-superseded-concepts.md:66` records the decision as already taken:\n\n> _\"'Co-worker' as a second runtime → Twin is the contextual AI Co-worker behavior.\"_ — **superseded concept**\n\nSo the position is settled: **Co-worker is a behavioural mode of the Twin, not a second runtime, not a service, not a memory scope.** The doctrine matches the mechanisms already in place:\n\n| Co-worker behaviour         | Existing primitive    |\n| --------------------------- | --------------------- |\n| Remembers your preferences  | Personal Memory       |\n| Knows the org's decisions   | Org Memory (promoted) |\n| Picks up where you left off | Continuity            |\n| Learns from its own runs    | Agent Memory          |\n| Prepares bounded work       | Twin + Capability     |\n| Cannot approve its own work | Governance            |\n\n### 5.1 The problem: the authority doctrine is in a dead package\n\n`packages/contracts` has **zero consumers** (`CODEBASE_EXCAVATION.md` §3c). Nothing imports it. So the document that calls itself _\"supreme architectural principle… supersedes all other architectural rules\"_ is enforced by nothing, and its neighbours share the fate:\n\n| File                                                                               | Defines                                                                                                   | Enforced by                                                         |\n| ---------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |\n| `authority.ts`                                                                     | Authority Doctrine, Twin/Co-worker boundary, approval lifecycle (`Draft → Proposed → …`), Triage Boundary | nothing                                                             |\n| `surfaces/chat.ts`                                                                 | _\"The Chat surface IS the Triage Hub\"_ — nav id `chat`, route `/app/chat`, projection `platform.chat`     | nothing                                                             |\n| `surfaces/spine-access-plane.ts`, `platform.ts`, `admin.ts`, `iw-spine-network.ts` | Surface contracts                                                                                         | nothing                                                             |\n| `marketplace/installation.ts`                                                      | Installation contract                                                                                     | nothing — `services/marketplace-installation` defines its own types |\n\nThis matters more than the usual dead-package finding, because §4.2's contradiction has an authoritative answer sitting right here. `authority.ts` states the Triage Boundary explicitly:\n\n> The Triage Hub can receive, validate, classify, redact, deduplicate, link, quarantine, stage. **It CANNOT independently decide that unverified material becomes organizational truth.** Triage decides _where a candidate should go for review_, not whether it becomes final truth by itself.\n\nThat settles it: `knowledge/src/synthesis/promotion-gate.ts`'s `AUTO-APPROVE` path violates the canonical doctrine. `knowledge/src/triage-bot.ts`'s _\"NEVER auto-approves\"_ is correct. The rule was written; it was just never wired to anything that could enforce it.\n\n### 5.2 Resolution\n\n1. **Move `authority.ts` into `packages/types`** (99 consumers) or `packages/spine-schema`, so the doctrine sits in a package that is actually imported.\n2. **Turn the Triage Boundary into a test**, not prose. A unit test asserting that no promotion path can write `governance_state = 'approved'` without an approver identity is worth more than the doctrine as a string constant. Note that `authority.ts` currently exports the doctrine as a **template literal** — human-readable, machine-inert.\n3. **Adopt `surfaces/chat.ts`'s definition**: the Chat surface _is_ the Triage Hub. This is a better answer than a separate \"Triage\" nav entry, and it connects the sidebar's `Personal → Brainstorming Sessions` to the triage pipeline.\n4. **Reconcile `marketplace/installation.ts`** with the state machine's own types, or delete it (see `RUNTIME_PROFILE_AND_ENTRY_CHANNEL_CONTRACT.md §4.1`).\n5. In user-facing copy, \"Co-worker\" is good positioning language. In code it remains **Twin behaviour** — never a type, table, service, or route.\n\n**Correction note:** an earlier scan of this repository reported zero occurrences of \"co-worker\". That was a false negative caused by a case- and path-scoped grep. The term is present in 10 tracked files, including the two contract modules above. The conclusion was right; the evidence was wrong, and the real evidence is stronger.\n\n---\n\n## 6. What to fix, in order\n\n| #   | Change                                                                                                                          | Why first                                                                     |\n| --- | ------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- |\n| 1   | Make canonical write failure **hard-fail** instead of `console.warn` + `success: true`                                          | Converts silent data loss into a visible error. One-line risk, highest value. |\n| 2   | Route canonical writes through the `SPINE` binding to Neon; delete `CANONICAL_SPINE_DB`                                         | Removes the placeholder binding and the second D1 canonical store             |\n| 3   | Move the 16-concept ontology to `packages/spine-entities`; delete the copy-paste in `pipeline/src/canonical/entity-type-map.ts` | One definition of canonical truth                                             |\n| 4   | Decide `org_memory`: store or cache. Enforce one writer.                                                                        | Six writers into a documented cache is the §2 problem repeating               |\n| 5   | Add `agent_memory` to D1                                                                                                        | Must land before `pipeline/src/spine/` is deleted or the capability is lost   |\n| 6   | Delete `synthesis/promotion-gate.ts` + `synthesis/triage-bot.ts`; salvage `lineage-tracker.ts`                                  | Removes the contradictory governance rule                                     |\n| 7   | Retarget `knowledge/src/triage-bot.ts` off CouchDB onto the D1 memory tables                                                    | Makes triage real                                                             |\n| 8   | Delete `gateway/src/memory-governance.ts` and `services/spine/src/canonical-store.ts`                                           | ~1,300 LOC of orphaned competing definitions                                  |\n| 9   | Retire `services/pipeline/src/spine/`                                                                                           | Now safe                                                                      |\n| 10  | Move `packages/contracts/src/authority.ts` into `packages/types`; make the Triage Boundary a test                               | The canonical doctrine is currently enforced by nothing (§5.1)                |\n\nSteps 1–3 unblock the operational/canonical repo split in `RUNTIME_PROFILE_AND_ENTRY_CHANNEL_CONTRACT.md §7`. That split cannot be drawn while the canonical store is a placeholder.\n\n---\n\n## 7. Open decisions\n\n| #   | Decision                                                                      | Blocks                                                            |\n| --- | ----------------------------------------------------------------------------- | ----------------------------------------------------------------- |\n| M1  | Is `org_memory` (D1) the store, or a cache of Postgres `memory.org_memory`?   | §3.2, step 4                                                      |\n| M2  | Does a `team` memory scope exist? (Currently a type with no table.)           | §3.2, step 8                                                      |\n| M3  | May Agent Memory be cited by the Twin before triage, or only after promotion? | §3.1 — determines whether agent learning is immediate or governed |\n| M4  | Does Personal Memory ever widen to Org, and if so by what act?                | §3.1 — the hard boundary is currently absolute                    |\n| M5  | Canonical Spine on Neon only, or Neon + a D1 read projection?                 | §2.3, step 2                                                      |\n| M6  | Is the Chat surface the Triage Hub, per `contracts/src/surfaces/chat.ts`?     | §5.2, sidebar structure                                           |\n"}