{"slug":"architecture/END_TO_END_ARCHITECTURE","title":"IntegrateWise — End-to-End Architecture v3.7","section":"architecture","sectionLabel":"Architecture","status":"unclassified","type":"document","path":"architecture/END_TO_END_ARCHITECTURE.mdx","bytes":53618,"description":null,"updated":null,"source":"integratewise/Integratewise-docs-wiki","content":"\n> **Status: SUPERSEDED SYSTEMS-VIEW SNAPSHOT.** Its D1-as-canonical,\n> per-domain-partition and older Twin/runtime claims are retained only for\n> architecture genealogy. Use\n> [`INTEGRATEWISE_ARCHITECTURE.md`](INTEGRATEWISE_ARCHITECTURE.md) v2.2 for\n> current product-and-platform truth.\n\n## Systems View × User View\n\n**Version:** 3.7\n**Date:** June 2026\n**Original Status:** Architecture Locked\n\n**Canonical Reference:** [CANONICAL_PLATFORM_ARCHITECTURE.md](./CANONICAL_PLATFORM_ARCHITECTURE.md)\n\n> This document is downstream of the Canonical Platform Architecture. When this document contradicts the canonical architecture, the canonical architecture wins.\n\n---\n\n## Executive Summary\n\n> **Systems View:** A Cloudflare-native event-driven architecture where connectors feed a canonical Spine (D1), a selective Memory layer (Vectorize + D1), and a Twin runtime (Workers AI) that operates through an MCP tool surface and an ADK agent orchestrator — all connected via Queues, coordinated by Durable Objects, and cached at the edge (KV).\n\n> **User View:** Connect your tools once. The Workbench projects the right governed context for your role. The Twin speaks first with insights, proposes actions, waits for approval, hands off approved playbooks, and learns from outcomes — so week 6 is smarter than week 1.\n\n**Companion doctrine:** [WORKBENCH_DOCTRINE.md](WORKBENCH_DOCTRINE.md) defines every user, Twin, governance, knowledge, and domain surface as a governed Workbench projection over the Adaptive Spine. This end-to-end architecture inherits that rule: no Workbench owns canonical truth, and no Workbench is a system of record.\n\n---\n\n# PART I: SYSTEMS VIEW\n\n## How the Engine Runs\n\n---\n\n## 1. Service Topology\n\n```\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ EDGE LAYER │\n│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │\n│ │ Gateway │ │ Pages │ │ KV │ │ Vectorize │ │\n│ │ (Router) │ │ (SPA) │ │ (Cache) │ │ (Search) │ │\n│ └──────┬──────┘ └─────────────┘ └─────────────┘ └─────────────┘ │\n└─────────┼───────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ WORKER SERVICES │\n│ │\n│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │\n│ │ Auth │ │ Workspace │ │ Connector │ │ Pipeline │ │\n│ │ Service │ │ Service │ │ Sync │ │ Service │ │\n│ │ │ │ │ │ Service │ │ │ │\n│ │ JWT/RS256 │ │ Profile │ │ Nango/MCP │ │ Normalizer │ │\n│ │ Tenant │ │ Projection │ │ Scheduler │ │ 8-Stage │ │\n│ │ Session │ │ Context │ │ Hydration │ │ Sectorize │ │\n│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │\n│ │ │ │ │ │\n│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │\n│ │ Twin │ │ Memory │ │ Governance │ │ Execution │ │\n│ │ Service │ │ Service │ │ Service │ │ Service │ │\n│ │ │ │ │ │ │ │ │ │\n│ │ AI SDK │ │ Promotion │ │ Proposals │ │ Playbook │ │\n│ │ MCP Router │ │ Scoring │ │ HITL Queue │ │ Handoff │ │\n│ │ Context │ │ Continuity │ │ Policies │ │ Audit │ │\n│ │ Assembly │ │ Scope │ │ Audit │ │ Outcome │ │\n│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │\n└─────────┼────────────────┼────────────────┼────────────────┼───────────────┘\n │ │ │ │\n └────────────────┴────────────────┴────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ DURABLE OBJECTS │\n│ │\n│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │\n│ │ User │ │ Tenant │ │ Session │ │ Workflow │ │\n│ │ DO │ │ DO │ │ DO │ │ DO │ │\n│ │ │ │ │ │ │ │ │ │\n│ │ Identity │ │ Spine Config│ │ Chat State │ │ HITL State │ │\n│ │ Preferences │ │ Connectors │ │ Tool Calls │ │ Proposal │ │\n│ │ Continuity │ │ Schema │ │ Memory │ │ Progress │ │\n│ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │\n└─────────────────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ QUEUES │\n│ │\n│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │\n│ │ pipeline-process│ │ memory-promote │ │ connector-sync │ │\n│ │ │ │ │ │ │ │\n│ │ Raw payload → │ │ Spine entity → │ │ Schedule → │ │\n│ │ Normalizer │ │ Score → Memory │ │ Hydration job │ │\n│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │\n│ │\n│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │\n│ │ proposal-review │ │ execution-run │ │ outcome-feedback│ │\n│ │ │ │ │ │ │ │\n│ │ Human approval → │ │ Approved → │ │ Result → │ │\n│ │ Execute/Reject │ │ Playbook run │ │ Memory update │ │\n│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │\n└─────────────────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ D1 DATABASE │\n│ │\n│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │\n│ │ tenant_ │ │ decide_ │ │ build_ │ │ grow_ │ │\n│ │ spine_ │ │ data │ │ data │ │ data │ │\n│ │ config │ │ │ │ │ │ │ │\n│ │ │ │ Strategy │ │ Engineering │ │ Sales │ │\n│ │ Identity │ │ Planning │ │ Product │ │ Marketing │ │\n│ │ Schema │ │ Knowledge │ │ Delivery │ │ CS │ │\n│ │ Connectors │ │ Documents │ │ Projects │ │ Revenue │ │\n│ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │\n│ │\n│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │\n│ │ run_ │ │ cross_ │ │ memory_ │ │ governance_│ │\n│ │ data │ │ data │ │ index │ │ data │ │\n│ │ │ │ │ │ │ │ │ │\n│ │ Operations │ │ Cross-func │ │ Patterns │ │ Proposals │ │\n│ │ Finance │ │ Org-wide │ │ Decisions │ │ Decisions │ │\n│ │ HR │ │ Shared │ │ Preferences │ │ Policies │ │\n│ │ Compliance │ │ │ │ Lessons │ │ Audit │ │\n│ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │\n│ │\n│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │\n│ │ signal_ │ │ continuity_│ │ playbook_ │ │\n│ │ log │ │ state │ │ registry │ │\n│ │ │ │ │ │ │ │\n│ │ Changes │ │ Org scope │ │ Action │ │\n│ │ Anomalies │ │ Team scope │ │ Templates │ │\n│ │ Events │ │ Role scope │ │ Parameters │ │\n│ │ Patterns │ │ Personal │ │ Handoff │ │\n│ └─────────────┘ └─────────────┘ └─────────────┘ │\n└─────────────────────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## 2. Monorepo Package Structure\n\n```\nintegratewise-live/\n├── apps/\n│ ├── web/ # Next.js SPA (Pages deployment)\n│ ├── gateway/ # Cloudflare Worker (API router)\n│ └── marketing/ # Marketing site (separate Pages)\n│\n├── services/\n│ ├── auth/ # JWT, session, tenant context\n│ ├── workspace/ # Profile, projection, context assembly\n│ ├── connector-sync/ # Nango, MCP, hydration scheduler\n│ ├── pipeline/ # 8-stage normalizer, sectorize\n│ ├── twin/ # AI SDK, MCP router, context retrieval\n│ ├── memory/ # Promotion scoring, continuity scope\n│ ├── governance/ # Proposals, HITL, policies, audit\n│ ├── execution/ # Playbook handoff, run, audit\n│ └── workflow/ # (being retired — replaced by DO + Queues)\n│\n├── packages/\n│ ├── spine-model/ # Canonical entities, domain routing\n│ ├── projection-model/ # Context assembly, 6 dimensions\n│ ├── design-tokens/ # Color, typography, spacing system\n│ ├── shared-types/ # Cross-service TypeScript contracts\n│ └── test-utils/ # Playwright, test fixtures, helpers\n│\n├── infra/\n│ ├── wrangler/ # Per-service wrangler.toml configs\n│ ├── migrations/ # D1 schema migrations\n│ └── seeds/ # Development seed data\n│\n└── docs/ # Constitutional docs (CANON, etc.)\n```\n\n---\n\n## 3. API Contracts\n\n### 3.1 Gateway Router (Single Entry Point)\n\nAll requests flow through the Gateway Worker. No service is public-facing except Gateway.\n\n```\nPOST /api/v1/auth/login → Auth Service\nPOST /api/v1/auth/refresh → Auth Service\nGET /api/v1/workspace/profile → Workspace Service\nPOST /api/v1/workspace/initialize → Workspace Service\nPOST /api/v1/workspace/hydrate-connectors → Workspace Service → Queue: connector-sync\nGET /api/v1/workspace/engineering → Workspace Service → D1: build_data\nGET /api/v1/workspace/sales → Workspace Service → D1: grow_data\nPOST /api/v1/twin/chat → Twin Service\nPOST /api/v1/twin/brief → Twin Service\nPOST /api/v1/twin/reason → Twin Service\nGET /api/v1/proposals → Governance Service\nPOST /api/v1/proposals/:id/approve → Governance Service → Queue: execution-run\nPOST /api/v1/proposals/:id/reject → Governance Service\nGET /api/v1/executions → Execution Service\nGET /api/v1/memory → Memory Service\nGET /api/v1/continuity → Memory Service\n```\n\n### 3.2 Inter-Service Communication\n\nServices do not call each other directly. They communicate via D1 (state) and Queues (events).\n\n```\nDirect calls (anti-pattern): Service A → HTTP → Service B\nCorrect pattern: Service A → Queue → Service B (async)\n Service A → D1 read → State (sync, read-only)\n```\n\n---\n\n## 4. Queue Routing & Message Formats\n\n### 4.1 Queue Topology\n\n| Queue | Producer | Consumer | Message Format |\n| ------------------ | ---------------------------------- | ---------------------- | -------------------- |\n| `pipeline-process` | Connector-Sync, Gateway | Pipeline Service | `PipelineJob` |\n| `memory-promote` | Pipeline Service | Memory Service | `PromotionCandidate` |\n| `connector-sync` | Gateway (hydration trigger) | Connector-Sync Service | `HydrationRequest` |\n| `proposal-review` | Twin Service, Intelligence Service | Governance Service | `Proposal` |\n| `execution-run` | Governance Service (approved) | Execution Service | `ExecutionJob` |\n| `outcome-feedback` | Execution Service | Memory Service | `OutcomeEvent` |\n\n### 4.2 Message Schemas\n\n```typescript\n// PipelineJob\ninterface PipelineJob {\n job_id: string; // UUID\n tenant_id: string;\n connector_id: string; // \"github\", \"hubspot\", etc.\n provider: \"nango\" | \"mcp\" | \"ai\";\n payload: RawPayload; // Connector-specific raw data\n stage:\n | \"analyze\"\n | \"extract\"\n | \"classify\"\n | \"validate\"\n | \"resolve\"\n | \"transform\"\n | \"write\"\n | \"sectorize\";\n attempt: number; // Retry count\n created_at: ISO8601;\n}\n\n// PromotionCandidate\ninterface PromotionCandidate {\n candidate_id: string;\n tenant_id: string;\n entity_id: string; // Spine entity reference\n entity_type: CanonicalType;\n domain: DomainPartition;\n signals: SignalScore[]; // 7 signal scores\n composite_score: number; // 0–1\n promotion_reason: string[]; // Human-readable why\n created_at: ISO8601;\n}\n\n// HydrationRequest\ninterface HydrationRequest {\n request_id: string;\n tenant_id: string;\n user_id: string;\n type: \"creamy\" | \"full\" | \"delta\";\n connector_ids: string[]; // Which connectors to hydrate\n triggered_by: \"onboarding\" | \"scheduler\" | \"manual\" | \"webhook\";\n priority: \"critical\" | \"normal\" | \"background\";\n created_at: ISO8601;\n}\n\n// Proposal\ninterface Proposal {\n proposal_id: string;\n tenant_id: string;\n user_id: string;\n title: string;\n description: string;\n evidence: Evidence[]; // Links to Spine entities, signals\n reasoning_chain: string[]; // How the Twin arrived here\n action: PlaybookAction; // What would execute\n risk_level: \"low\" | \"medium\" | \"high\";\n auto_execute: false; // Always false (P7)\n status: \"pending\" | \"approved\" | \"rejected\" | \"modified\" | \"delegated\";\n created_at: ISO8601;\n}\n\n// ExecutionJob\ninterface ExecutionJob {\n job_id: string;\n tenant_id: string;\n proposal_id: string;\n playbook_id: string;\n action: PlaybookAction;\n parameters: Record;\n approved_by: string; // User ID who approved\n approved_at: ISO8601;\n status: \"ready\" | \"running\" | \"completed\" | \"failed\";\n}\n\n// OutcomeEvent\ninterface OutcomeEvent {\n event_id: string;\n tenant_id: string;\n execution_job_id: string;\n proposal_id: string;\n outcome: \"success\" | \"partial\" | \"failure\" | \"cancelled\";\n result_summary: string;\n lessons: string[]; // What the system learned\n memory_impact: number; // How much this should weight future proposals\n created_at: ISO8601;\n}\n```\n\n---\n\n## 5. D1 Schema Per Partition\n\n### 5.1 Core Tables (All Partitions)\n\nEvery partition table has these standard columns:\n\n```sql\nCREATE TABLE .entities (\n id TEXT PRIMARY KEY, -- Canonical UUID\n tenant_id TEXT NOT NULL,\n external_id TEXT, -- Provider-specific ID (e.g., GitHub issue #123)\n provider TEXT, -- \"github\", \"hubspot\", \"slack\"\n entity_type TEXT NOT NULL, -- \"repository\", \"task\", \"contact\", etc.\n domain TEXT NOT NULL, -- \"build_data\", \"grow_data\", etc.\n name TEXT,\n description TEXT,\n metadata JSON, -- Provider-specific fields (normalized)\n relationships JSON, -- Array of {entity_id, relation_type}\n signals JSON, -- Array of {signal_type, score, timestamp}\n created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,\n updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,\n INDEX tenant_entity (tenant_id, entity_type),\n INDEX external (tenant_id, provider, external_id)\n);\n```\n\n### 5.2 Tenant Spine Config\n\n```sql\nCREATE TABLE tenant_spine_config (\n tenant_id TEXT PRIMARY KEY,\n user_id TEXT NOT NULL,\n role TEXT NOT NULL, -- \"founder\", \"engineer\", \"sales\"\n department TEXT NOT NULL, -- \"engineering\", \"sales\", \"operations\"\n industry TEXT NOT NULL, -- \"saas\", \"fintech\", \"healthcare\"\n schema_version TEXT DEFAULT 'v1',\n schema JSON, -- Role×Industry-specific entity expectations\n connector_priorities JSON, -- Ordered list of connector IDs\n signal_types JSON, -- What signals to track\n memory_categories JSON, -- What memory types to form\n governance_defaults JSON, -- Default policies\n twin_context JSON, -- Boot context for Twin\n created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,\n updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP\n);\n```\n\n### 5.3 Memory Index\n\n```sql\nCREATE TABLE memory_index (\n id TEXT PRIMARY KEY,\n tenant_id TEXT NOT NULL,\n user_id TEXT, -- NULL = org-level\n scope TEXT NOT NULL, -- \"org\" | \"team\" | \"role\" | \"personal\"\n memory_type TEXT NOT NULL, -- \"pattern\" | \"decision\" | \"preference\" | \"exception\" | \"behavior\" | \"relationship\" | \"lesson\"\n source_entity_id TEXT, -- Link back to Spine\n source_entity_type TEXT,\n title TEXT NOT NULL,\n content TEXT NOT NULL,\n evidence JSON, -- Supporting entity IDs, signals\n confidence_score REAL, -- 0–1\n usage_count INTEGER DEFAULT 0, -- How often Twin referenced this\n last_accessed TIMESTAMP,\n created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,\n INDEX tenant_scope (tenant_id, scope, memory_type),\n INDEX source (source_entity_id)\n);\n```\n\n### 5.4 Governance Data\n\n```sql\nCREATE TABLE governance_data (\n id TEXT PRIMARY KEY,\n tenant_id TEXT NOT NULL,\n proposal_id TEXT NOT NULL,\n user_id TEXT NOT NULL, -- Who reviewed\n action TEXT NOT NULL, -- \"approve\" | \"reject\" | \"modify\" | \"delegate\" | \"schedule\"\n comment TEXT,\n modified_proposal JSON, -- If action = \"modify\"\n delegated_to TEXT, -- If action = \"delegate\"\n scheduled_for TIMESTAMP, -- If action = \"schedule\"\n created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP\n);\n```\n\n---\n\n## 6. MCP (Model Context Protocol) Architecture\n\n### 6.1 MCP Server Surface\n\nThe Twin does not call APIs directly. It calls **MCP tools** that abstract the Spine.\n\n```\n┌─────────────────────────────────────────────────────────────┐\n│ MCP TOOL SURFACE │\n│ │\n│ spine.read → Read entities from D1 partition │\n│ spine.search → Semantic search via Vectorize │\n│ spine.query → Structured query across partitions │\n│ spine.relationships → Get related entities │\n│ │\n│ signal.list → List signals for tenant/user │\n│ signal.subscribe → Subscribe to signal type │\n│ signal.acknowledge → Mark signal as seen │\n│ │\n│ memory.retrieve → Get memory by scope/type │\n│ memory.search → Semantic memory search │\n│ memory.promote → Manually promote entity to memory │\n│ │\n│ proposal.create → Create proposal (Twin internal) │\n│ proposal.list → List pending proposals │\n│ proposal.get → Get proposal details │\n│ │\n│ execution.run → Execute approved playbook (gated) │\n│ execution.status → Check execution status │\n│ execution.audit → Get execution history │\n│ │\n│ continuity.get → Get continuity state for scope │\n│ continuity.update → Update continuity (system only) │\n│ │\n│ connector.list → List connected connectors │\n│ connector.status → Get connector health │\n│ connector.sync → Trigger manual sync │\n└─────────────────────────────────────────────────────────────┘\n```\n\n### 6.2 MCP Router\n\n```\nTwin (AI SDK) → MCP Client → MCP Router (Worker) → Tool Handler → D1 / Queue / Vectorize\n```\n\nThe MCP Router is a Cloudflare Worker that:\n\n1. Authenticates the tool call (tenant-scoped)\n2. Validates parameters against schema\n3. Routes to the correct handler (D1 read, Queue write, Vectorize search)\n4. Returns structured response to the Twin\n5. Logs all calls for audit\n\n### 6.3 Tool Permission Model\n\n```\nspine.* → Read-only (Twin cannot write to Spine directly)\nsignal.* → Read + acknowledge (Twin can mark signals seen)\nmemory.* → Read + promote (Twin can suggest promotion, system scores)\nproposal.* → Read + create (Twin creates proposals, human approves)\nexecution.* → Read only (Twin cannot execute — governance service does)\ncontinuity.* → Read only (Twin reads continuity, system updates)\nconnector.* → Read + sync trigger (Twin can request sync, scheduler decides)\n```\n\n---\n\n## 7. ADK (Agent Development Kit)\n\n### 7.1 Agent Primitives\n\nThe Twin is not a monolithic agent. It is composed of **agent primitives** orchestrated by the ADK.\n\n```\n┌─────────────────────────────────────────────────────────────┐\n│ ADK ORCHESTRATOR │\n│ │\n│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │\n│ │ Context │ │ Reasoner │ │ Proposer │ │\n│ │ Agent │ │ Agent │ │ Agent │ │\n│ │ │ │ │ │ │ │\n│ │ Assembles │ │ Analyzes │ │ Generates │ │\n│ │ Spine + │ │ patterns, │ │ proposals │ │\n│ │ Memory + │ │ identifies │ │ with │ │\n│ │ Continuity │ │ insights │ │ evidence │ │\n│ └─────────────┘ └─────────────┘ └─────────────┘ │\n│ │\n│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │\n│ │ Brief │ │ Chat │ │ Handoff │ │\n│ │ Agent │ │ Agent │ │ Agent │ │\n│ │ │ │ │ │ │ │\n│ │ Morning │ │ Conversa- │ │ Transfers │ │\n│ │ brief, │ │ tional │ │ to human │ │\n│ │ daily │ │ Q&A │ │ or playbook │ │\n│ │ summary │ │ │ │ │ │\n│ └─────────────┘ └─────────────┘ └─────────────┘ │\n│ │\n│ ┌─────────────┐ │\n│ │ Session │ │\n│ │ Manager │ │\n│ │ │ │\n│ │ Manages │ │\n│ │ long-running│ │\n│ │ sessions, │ │\n│ │ tool call │ │\n│ │ history, │ │\n│ │ memory │ │\n│ └─────────────┘ │\n└─────────────────────────────────────────────────────────────┘\n```\n\n### 7.2 Agent Handoff Protocol\n\n```\nUser Query\n ↓\nADK Router → Intent Classification\n ↓\n[Chat intent] → Chat Agent → Context Agent → Answer\n[Brief intent] → Brief Agent → Context Agent → Summary\n[Reason intent] → Reasoner Agent → Context Agent → Insights\n[Proposal intent] → Proposer Agent → Context Agent → Proposal → Queue: proposal-review\n[Execution intent] → Handoff Agent → Governance Service → Human Review\n```\n\n### 7.3 Session Persistence\n\n```\nSession DO (Durable Object) stores:\n - Chat history\n - Tool call log\n - Agent handoff chain\n - User preferences learned in-session\n - Pending proposals created\n\nSession → Memory promotion: At session end, triaged insights promoted to Memory Index\n```\n\n---\n\n## 8. Spine Cache (KV Strategy)\n\n### 8.1 What Gets Cached\n\n| Cache Key | Source | TTL | Invalidation |\n| -------------------------------------- | ---------------------- | ------ | ---------------------------- |\n| `projection:::` | D1 projection query | 5 min | D1 write to partition |\n| `twin_context::` | Context assembly | 2 min | Memory update, signal change |\n| `spine_schema:` | tenant_spine_config | 1 hour | Schema update |\n| `connector_status:` | Connector health check | 30 sec | Sync completion |\n| `prompt_cache:` | Compiled system prompt | 1 hour | Schema version change |\n| `feature_flags:` | Governance defaults | 5 min | Policy update |\n\n### 8.2 Cache Invalidation Rules\n\n```\nD1 Write (any partition) → Invalidate projection cache for tenant\nMemory Update → Invalidate twin_context cache\nSignal New → Invalidate twin_context cache\nSchema Update → Invalidate spine_schema + prompt_cache\nPolicy Change → Invalidate feature_flags + governance cache\n```\n\n### 8.3 Cache-Aside Pattern\n\n```\nRead: Check KV → Hit → Return\n → Miss → Query D1 → Write KV → Return\n\nWrite: Write D1 → Invalidate KV → (Async) Warm KV with new value\n```\n\n---\n\n## 9. Request Flow: Gateway → Service → D1\n\n### 9.1 Authenticated Request Flow\n\n```\nUser Request\n ↓\nCloudflare Access / Gateway JWT (RS256)\n ↓\nGateway Worker\n ↓\n[Route Resolution] → Service Worker\n ↓\nService Worker\n ↓\n[Tenant Context Injection] → D1 binding (tenant-scoped)\n ↓\nD1 Query (partitioned by tenant_id)\n ↓\nResponse → Service Worker → Gateway Worker → User\n```\n\n### 9.2 Async Event Flow (Queue)\n\n```\nWebhook / Scheduler / Manual Trigger\n ↓\nGateway Worker (enqueue)\n ↓\nCloudflare Queue\n ↓\nConsumer Worker (Service)\n ↓\nD1 Write / Queue Forward\n ↓\nNext Queue Consumer\n```\n\n### 9.3 Twin Request Flow\n\n```\nUser Chat Message\n ↓\nGateway → Twin Service\n ↓\nSession DO (retrieve history)\n ↓\nADK Router (intent classification)\n ↓\nContext Agent → MCP Router\n ↓\n[MCP Tool Call] → D1 / Vectorize / Queue\n ↓\nCompiled Context → AI SDK (Workers AI)\n ↓\nLLM Response\n ↓\n[If Proposal] → Queue: proposal-review\n ↓\nResponse + Session DO Update\n ↓\nGateway → User\n```\n\n---\n\n## 10. Infrastructure Stack\n\n| Layer | Technology | Purpose |\n| ----------------- | ------------------------------------- | -------------------------------------------------- |\n| **Edge** | Cloudflare Pages | SPA hosting, marketing site |\n| **Edge** | Cloudflare Workers | API Gateway, service workers |\n| **Edge** | Cloudflare KV | Projection cache, prompt cache, feature flags |\n| **Edge** | Cloudflare Vectorize | Semantic search, memory retrieval |\n| **Compute** | Cloudflare Workers | Stateless service logic |\n| **Compute** | Cloudflare Durable Objects | Stateful sessions, tenant state, workflow progress |\n| **Compute** | Cloudflare Workflows | Long-running orchestration, HITL state machines |\n| **Coordination** | Cloudflare Queues | Async job routing, service decoupling |\n| **State** | Cloudflare D1 | Authoritative operational state (all partitions) |\n| **Files** | Cloudflare R2 | Artifacts, document exports, backups |\n| **AI** | Cloudflare AI SDK / Workers AI | Twin runtime, embeddings, inference |\n| **Auth** | Cloudflare Access + Gateway JWT | RS256, tenant-scoped, session management |\n| **Observability** | Cloudflare Analytics + Custom Logging | Request tracing, queue depth, D1 metrics |\n\n### Absolute Prohibitions\n\nThese are not preferences. They are architectural axioms. No exceptions. No compatibility layers. No migration paths. No edge cases.\n\n| Technology | Status | Why |\n| --------------------------------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------ |\n| **Supabase** | ❌ ABSOLUTELY PROHIBITED | Not in the stack. Not for auth. Not for anything. D1 is the only database. |\n| **PostgreSQL / Neon** | ❌ ABSOLUTELY PROHIBITED | No direct reads. No alternate writers. No fallback. D1 only. |\n| **CouchDB** | ❌ ABSOLUTELY PROHIBITED | No document databases. D1 handles all structured data. |\n| **n8n** | ❌ ABSOLUTELY PROHIBITED | Not in product runtime. Not for workflows. Cloudflare Workflows + Queues + DO only. |\n| **OpenWebUI** | ❌ ABSOLUTELY PROHIBITED | Not for Twin orchestration. Not for LLM UI. Twin runs on Workers AI / AI SDK only. |\n| **Redis** | ❌ ABSOLUTELY PROHIBITED | No external caches. Cloudflare KV is the only cache layer. |\n| **VPS / EC2 / VMs** | ❌ ABSOLUTELY PROHIBITED | No virtual servers in product runtime. Workers + DO + Workflows only. |\n| **LangChain / LangSmith** | ❌ ABSOLUTELY PROHIBITED | No external LLM frameworks. AI SDK + MCP + ADK only. |\n| **Any non-Cloudflare AI runtime** | ❌ ABSOLUTELY PROHIBITED | Twin runs on Cloudflare Workers AI. No OpenAI API calls from Twin. No Anthropic direct. No external orchestration. |\n\n**The Rule:** If it is not in the Cloudflare Developer Platform (Workers, Pages, D1, Durable Objects, Queues, KV, R2, Vectorize, Workflows, AI SDK), it is not in the product.\n\n---\n\n# PART II: USER VIEW\n\n## How People Experience the System\n\n---\n\n## Workbench Projection Doctrine in the End-to-End Flow\n\nEvery user-facing moment below happens through a Workbench projection:\n\n- **Human / Domain Workbench:** the accountable operating surface where people understand, decide, collaborate, approve, and act.\n- **Twin Workbench:** the reasoning surface where the Twin observes, explains, drafts, and proposes.\n- **Governance Workbench:** the hard-gate control plane for approvals, evidence, lineage, policy, and audit.\n- **Knowledge / Memory Workbench:** the retrieval and institutional-memory projection for documentation, lessons, playbooks, and promoted knowledge.\n\nThe Workbench layer does not own data. It renders governed context assembled from the Spine, Memory, Governance, and Continuity services. Applications and UI components are replaceable capabilities; the Workbench contract remains stable.\n\n---\n\n## 1. The Personas\n\n### 1.1 The Founder (Primary)\n\n> **\"I need to know what's happening across my company without asking 5 people for updates.\"**\n\n- Connects: GitHub, HubSpot, Slack, Notion\n- Needs: Engineering + Sales + Cross-functional view\n- Uses: Morning brief, situation proposals, governance audit\n- Expects: The AI knows the business context, speaks first, never hallucinates\n\n### 1.2 The Engineering Lead\n\n> **\"I want to know what my team shipped, what's blocked, and what needs my attention.\"**\n\n- Connects: GitHub, Jira, Slack\n- Needs: Build_data projection, PR health, blocked signals\n- Uses: Engineering workbench, Twin chat about repos, execution proposals\n- Expects: Accurate technical context, no made-up PRs\n\n### 1.3 The Sales Manager\n\n> **\"I need to see pipeline health, stalled deals, and what my team should do next.\"**\n\n- Connects: HubSpot, Slack, Calendar\n- Needs: Grow_data projection, opportunity signals, activity patterns\n- Uses: Sales workbench, deal proposals, follow-up reminders\n- Expects: Revenue-aware insights, not generic sales advice\n\n### 1.4 The Operations Person\n\n> **\"I need to see cross-functional blockers and ensure nothing falls through cracks.\"**\n\n- Connects: All tools\n- Needs: Cross_data + Run_data, signal triage, governance queue\n- Uses: Operations workbench, proposal review, execution audit\n- Expects: Complete organizational context, governed action flow\n\n---\n\n## 2. The Day 1 Experience\n\n### Step 1: Welcome (30 seconds)\n\n```\nScreen: \"IntegrateWise — Your AI knows your business\"\nAction: User clicks \"Get Started\"\nSystem: Creates tenant, generates session\n```\n\n### Step 2: Identity (1 minute)\n\n```\nScreen: \"Tell us about your role\"\nFields: Role (Founder / Engineer / Sales / Operations)\n Department (Engineering / Sales / Operations / Cross-functional)\n Industry (SaaS / Fintech / Healthcare / Other)\nSystem: Writes tenant_spine_config to D1\n → Schema auto-selected based on Role×Industry\n → Connector priorities auto-ranked\n → Governance defaults loaded\n```\n\n### Step 3: Goals (1 minute)\n\n```\nScreen: \"What are you trying to achieve?\"\nOptions: \"Ship faster\", \"Close more deals\", \"Reduce churn\",\n \"Improve team alignment\", \"Automate reporting\"\nSystem: Adds goals to tenant_spine_config.twin_context\n → Twin will reference these in briefs and proposals\n```\n\n### Step 4: Connectors (2 minutes)\n\n```\nScreen: Connector Tray — 5 recommended connectors based on Role×Industry\nExample for Founder/SaaS: GitHub, HubSpot, Slack, Notion, Intercom\nAction: User clicks \"Connect\" on each → OAuth via Nango\nSystem: Stores tokens in Nango vault\n → Triggers creamy hydration\n → Shows \"Connecting...\" progress\n```\n\n### Step 5: Hydration (0–2 hours)\n\n```\nSystem: Enqueues creamy hydration jobs\nQueue: connector-sync → pipeline-process\nPipeline: 8-stage normalizer → D1 build_data/grow_data/etc.\nResult: Essential reality loaded\n```\n\n### Step 6: Workspace (First View)\n\n```\nScreen: L1 Human / Domain Workbench — Role×Department×Industry hydrated\nHeader: \"Good morning, [Name] — here's your [Department] workspace\"\nNavigation: Engineering | Sales | Operations | Cross-functional (based on role)\nContent: Empty states with honest messaging\n → \"Connect GitHub to see repository health\"\n → \"Connect HubSpot to see pipeline activity\"\nSystem: ProjectionContext drives all UI components\n → Single authority: one department value everywhere\n```\n\n### Step 7: Twin First Interaction\n\n```\nUser: \"What changed in engineering this week?\"\nTwin: \"3 PRs merged, 2 blocked >48h, 1 new contributor.\n Here's the breakdown...\"\nSource: D1 build_data (direct read, not GitHub API)\n```\n\n---\n\n## 3. The Day 7 Experience (When Working)\n\n### Morning Brief (Automatic)\n\n```\nTime: 9:00 AM (user timezone)\nChannel: In-app notification + Email optional\nContent:\n \"Good morning. Here's what changed since yesterday:\n\n Engineering:\n • 2 PRs merged (repo: api-gateway, auth-service)\n • 1 PR blocked >48h — assigned to [Name]\n • 1 new issue: performance regression in staging\n\n Sales:\n • 3 new leads from HubSpot\n • 1 deal stalled at negotiation (Acme Corp, $45K)\n • Follow-up due: 2 contacts from last week\n\n Signals:\n • ⚠️ Velocity dropped 15% vs last week\n • ℹ️ New contributor joined engineering team\"\n\nSystem: Brief Agent → Context Agent → MCP spine.read + signal.list\n → D1 direct read → AI SDK → Formatted brief\n```\n\n### Twin Chat (On Demand)\n\n```\nUser: \"Why is the Acme Corp deal stalled?\"\nTwin: \"The deal moved to negotiation 12 days ago.\n Last activity: email sent 5 days ago, no reply.\n Similar deals at this stage typically close in 8 days.\n Proposal: Schedule follow-up call. [View Proposal →]\"\n\nSystem: Chat Agent → Context Agent → MCP spine.search + memory.retrieve\n → D1 + Vectorize → AI SDK → Answer with evidence links\n```\n\n### Proposal Review (Governance)\n\n```\nScreen: L5 Operations — Proposals Queue\nItem: \"Schedule follow-up with Acme Corp\"\nEvidence: Deal record, email timeline, velocity comparison\nReasoning: \"Stalled >5 days. Historical pattern: 60% of stalled\n deals at this stage close after follow-up.\"\nActions: [Approve] [Reject] [Modify] [Delegate] [Schedule]\n\nUser clicks [Approve]\nSystem: Queue: proposal-review → execution-run\n → Execution Service → Playbook: \"hubspot-deal-followup\"\n → Result: Task created in HubSpot, assigned to owner\n → Outcome logged → Queue: outcome-feedback → Memory update\n```\n\n---\n\n## 4. The Day 30 Experience (Compounding)\n\n### What the User Notices\n\n```\nWeek 1: \"It knows my tools. It answers from my data.\"\nWeek 2: \"It remembered that I prefer morning briefs at 8am, not 9am.\"\nWeek 3: \"It proposed something I was about to do myself.\"\nWeek 4: \"It caught a pattern I missed — 3 deals stalling at the same stage.\"\nWeek 6: \"It feels like it works here.\"\n```\n\n### What the System Did\n\n```\nWeek 1: Creamy → Full hydration. Basic projection active.\nWeek 2: Memory formation started. First preferences captured.\nWeek 3: Continuity scope built. Twin references past decisions.\nWeek 4: Pattern detection. Cross-entity signals identified.\nWeek 6: Proposal quality improved. Outcome feedback loop active.\n → Proposals weighted by historical success\n → Lessons from failed proposals incorporated\n → Twin speaks more precisely to this user's style\n```\n\n---\n\n## 5. The Governance Experience\n\n### The User Sees\n\n```\nL5 Operations:\n [3] Proposals pending your review\n [1] Situation requires attention\n [12] Signals today (2 new)\n\nL6 Execution:\n [2] Running — \"HubSpot follow-up\", \"Slack notification\"\n [5] Completed today\n [0] Failed (last 7 days)\n\nL7 Governance:\n [24] Decided this month (18 approved, 4 rejected, 2 modified)\n [3] Active policies\n [1] Audit trail export ready\n```\n\n### The System Does\n\n```\nProposal created → Queue: proposal-review → Governance DO\nGovernance DO → User notification → WebSocket push\nUser action → Governance DO update → Queue: execution-run (if approved)\nExecution DO → Playbook run → Outcome capture → Queue: outcome-feedback\nMemory Service → Update weights → Continuity scope refresh\n```\n\n---\n\n## 6. The Memory Experience\n\n### User View: \"It remembers\"\n\n```\nUser: \"What did we decide about the pricing model last quarter?\"\nTwin: \"In March, you approved a tiered pricing proposal.\n Key decision: $49/$99/$249 tiers.\n Outcome: 23% increase in ACV.\n Lesson: Mid-tier converted 40% better than expected.\n [View full memory →]\"\n\nUser: \"Who handled the Acme Corp integration?\"\nTwin: \"[Name] led the integration in Q2.\n Key contacts: [CTO email], [Engineering lead].\n Technical notes: OAuth 2.0 + webhook setup.\n [View relationship map →]\"\n```\n\n### System View: Memory Formation\n\n```\nSpine entity created → Signal scoring (7 dimensions)\nScore > threshold → Queue: memory-promote\nMemory Service → Scope assignment (org/team/role/personal)\nVectorize embedding → Semantic index\nTwin retrieval → Usage count ++ → Confidence refinement\n```\n\n---\n\n# PART III: THE BRIDGE\n\n## How Systems Enable the User Experience\n\n---\n\n## Mapping: User Moment → System Action\n\n| User Moment | System Action | Technologies |\n| ------------------------------ | ------------------------------------------------------------------------ | ------------------------- |\n| \"Connect my tools\" | OAuth → Nango vault → token storage | Nango, Gateway |\n| \"What changed this week?\" | Brief Agent → Context Agent → MCP spine.read → D1 → AI SDK | Twin, MCP, D1, Workers AI |\n| \"Why is this stalled?\" | Chat Agent → MCP spine.search + memory.retrieve → D1 + Vectorize | Twin, MCP, Vectorize |\n| \"Approve this proposal\" | Governance DO → approved handoff package → customer-owned execution path | DO, Queues, Handoff |\n| \"It remembered my preference\" | Session DO → Memory promotion → Continuity scope update | DO, Memory, D1 |\n| \"It proposed before I asked\" | Signal detection → Reasoner Agent → Proposer Agent → Queue | Pipeline, Twin, Queues |\n| \"Week 6 is better than Week 1\" | Outcome feedback → Weight update → Proposal quality improvement | Execution, Memory, AI SDK |\n\n---\n\n## The Complete Loop\n\n```\nUSER SYSTEM\n──── ──────\n\"Connect tools\" → Gateway → Nango → OAuth → Token vault\n ↓\n\"See workspace\" → Workbench projection → ProjectionContext → D1\n ↓\n\"Ask question\" → Twin Service → ADK Router → Context Agent\n ↓\n → MCP Router → spine.read / memory.retrieve\n ↓\n → D1 / Vectorize → Compiled context\n ↓\n → AI SDK (Workers AI) → Response\n ↓\n\"Get proposal\" → Proposer Agent → Proposal → Queue: proposal-review\n ↓\n\"Review & approve\" → Governance Service → Governance DO\n ↓\n → Approved handoff package → customer-owned runtime\n ↓\n → Playbook execution outside IW product plane\n ↓\n\"See result\" → Execution DO → Outcome capture\n ↓\n → Queue: outcome-feedback → Memory Service\n ↓\n → Memory update → Continuity scope refresh\n ↓\n\"Next week, better\" → Weighted proposals → Improved Twin responses\n```\n\n---\n\n## The Architectural Promise\n\n> **Systems View:** Every user action flows through a governed, auditable, event-driven pipeline. No direct API calls. No stateless execution. No bypasses.\n\n> **User View:** The Workbench projects governed context, the Twin reasons over it, governance protects consequence, and outcomes improve future context. I never have to re-explain the business.\n\n> **The Bridge:** The same architecture that makes the system governable (Spine, Memory, Queues, MCP, Handoff, and Workbench projections) is what makes the user experience feel continuous and intelligent.\n\n---\n\n_This document is the canonical end-to-end architecture of IntegrateWise v3.7. It specifies both the engine (Systems View) and the experience (Workbench View), with the bridge between them. Implementation details, current build status, and remaining work are tracked in the engineering backlog and CI pipeline._\n\n---\n\n## Appendix: Doc ↔ Code Reconciliation (as of 2026-06-15)\n\n> This doc is the **canonical/target** architecture. The running code is consolidated under\n> v3.6/v3.7 (see `AGENTS.md`). The logical services below map to the deployed workers as follows;\n> reconcile names over time rather than treating the divergence as drift.\n\n| Doc (logical) service | Deployed worker (actual) |\n| ---------------------- | --------------------------------------------------------------------------------- |\n| Auth Service | `services/gateway` (`src/auth.ts`) |\n| Workspace Service | `services/gateway` (`src/workspace-spine.ts`, `spine-schema.ts`) |\n| Connector-Sync Service | `services/connector-sync` (DO-alarm scheduler) + `services/connector` |\n| Pipeline Service | `services/pipeline` (8-stage normalizer consolidated) |\n| Twin Service | `services/iw-agent-runtime` (Cloudflare Agents SDK: `TwinAgent`, `TenantBrainDO`) |\n| Memory Service | `services/pipeline` (`memory-promoter.ts`) + `services/continuity` |\n| Governance Service | `services/intelligence` (`action-loop.ts`, `governance_data`) |\n| Execution Service | `services/intelligence` (playbook hand-off) |\n| `apps/gateway` | `services/gateway` (gateway is a service, not an app) |\n\n**Known terminology deltas to reconcile:**\n\n- \"ADK (Agent Development Kit)\" in this doc = the **Cloudflare Agents SDK** runtime in `services/iw-agent-runtime`.\n- `apps/web` is a **Vite + React SPA** on Cloudflare Pages (this doc says \"Next.js\"); update whichever is authoritative.\n- Memory tables in code are `conversational_memory` / `org_memory` / `personal_memory` (+ `memory_promotion_audit`); `memory_index` here is the logical/target shape.\n- Per-domain D1 partitions (`decide_data | build_data | grow_data | run_data | cross_data`) are live and match `getDomainForType()`.\n"}