{"slug":"architecture/USER_SYSTEM_JOURNEY_BLUEPRINT","title":"User Journey ≠ System Journey — Canonical Blueprint","section":"architecture","sectionLabel":"Architecture","status":"superseded","type":"document","path":"architecture/USER_SYSTEM_JOURNEY_BLUEPRINT.mdx","bytes":22276,"description":null,"updated":null,"source":"integratewise/Integratewise-docs-wiki","content":"\n## v3.7 — Frontend Layered Stack & Backend Stages Map\n\n**Date:** June 18, 2026 \n**Status:** SUPERSEDED — historical journey and layer mapping\n**Companion doctrines:** [WORKBENCH_DOCTRINE.md](WORKBENCH_DOCTRINE.md), [DETAILED_USER_FLOWS_AND_SYSTEM_PATHS.md](DETAILED_USER_FLOWS_AND_SYSTEM_PATHS.md)\n\n**Canonical Reference:** [CANONICAL_PLATFORM_ARCHITECTURE.md](./CANONICAL_PLATFORM_ARCHITECTURE.md)\n\n> Retained for engineering lineage only. The current Product is one Workbench,\n> the Twin is contextual and silent, and Neon—not D1—is canonical truth. Use\n> `ARCHITECTURE_LOCK.md`,\n> `CONSOLIDATED_PRODUCT_PLATFORM_CONTRACT_v1.2.md` and\n> `FRONTEND_PLATFORM_API_AND_DISCOVERY_SPEC.md`.\n\n---\n\n## 1. The Fundamental Model (User Spec)\n\nAll layers below are Workbench projections over governed Adaptive Spine context. No layer owns canonical truth; each layer renders the right projection for the user, Twin, governance, knowledge, or domain purpose.\n\nEvery user gets exactly two views — never more:\n\n- **Personal** (always `PERSONAL` domain)\n- **One Work workbench** (chosen once during onboarding — e.g. `CUSTOMER_SUCCESS`, `SALES`, `REVOPS`, `MARKETING`, `PRODUCT_ENGINEERING`, `BIZOPS`, etc.)\n\n### Differentiation rules (locked):\n\n- **L1 (main workbenches) + Memory layer (L4)** are differentiated by the chosen domain.\n- **Everything else** (L2 overlay, L3 Twin, L5, L6, L7) is shared.\n\n_Data Invariant:_ Data always comes from D1 Spine projections (never raw tool data). The chosen domain decides which `${domain}_data` partition (e.g. `grow_data`, `build_data`) you see in L1.\n\n---\n\n## 2. High-Level Flow\n\n```text\nL0 Onboarding (choose useCase + department → domain)\n ↓\nAppShell (restores or sets activeDomain + WORK/PERSONAL preference)\n ↓\nWorkspaceShellNew\n ├── Personal view → PERSONAL domain content\n └── Work view → chosen domain content (L1)\n │\n ├── L2 Overlay (⌘J drawer) — always available on top\n ├── L3 Twin (shared chat/reasoning)\n ├── L4 Memory Hub (differentiated)\n ├── L5 Operations (shared)\n ├── L6 Execution (shared)\n └── L7 Governance (shared, HARD GATE)\n```\n\nBackend system stages run underneath (S1–S12).\n\n## 2.5 Architecture Diagram (v3.7 Canonical)\n\n```mermaid\nflowchart TB\n %% User Surface\n subgraph User[\"👤 User Surfaces (L0–L7)\"]\n direction TB\n L0[\"L0 Onboarding
(domain selection)\"]\n L05[\"L0.5 Always-On
(background DO crons)\"]\n L1[\"L1 Workbench
(domain-specific dashboards)\"]\n L2[\"L2 Overlay
⌘J Intelligence Drawer\"]\n L3[\"L3 Twin Workbench
(CF-native chat + voice)\"]\n L4[\"L4 Knowledge
(Memory Hub)\"]\n L5[\"L5 Operations
(Signals + Proposals)\"]\n L6[\"L6 Execution
(Handoff + Playbooks)\"]\n L7[\"L7 Governance
(HARD GATE Approvals)\"]\n\n L0 --> L1\n L1 --> L2 & L3 & L4 & L5 & L6 & L7\n end\n\n %% Gateway\n GW[\"🚪 Gateway
(services/gateway)
JWT verify • Rate limit • D1-direct projections\"]\n\n User -->|Browser / MCP| GW\n\n %% Core Services\n subgraph Services[\"⚙️ Cloudflare Workers Services\"]\n direction LR\n Pipeline[\"services/pipeline
Normalizer (8-stage) + Spine Writer\"]\n Connector[\"services/connector
Loader + MCP Bridge + Sync\"]\n Intelligence[\"services/intelligence
Think • Act • Govern\"]\n Knowledge[\"services/knowledge
KB Ingestion + Org Memory\"]\n AgentRuntime[\"services/iw-agent-runtime
TwinAgent DO + TenantBrainDO
(Cloudflare Agents SDK)\"]\n end\n\n GW --> Services\n Services -->|write entities| Data\n\n %% Data Plane\n subgraph Data[\"🗄️ Cloudflare Data Plane (Single Source of Truth)\"]\n direction TB\n D1[\"D1 (integratewise-spine-cache)
tenant_spine_config
decide_data | build_data | grow_data | run_data | cross_data\"]\n KV[\"KV (hot cache + signals)\"]\n Vectorize[\"Vectorize + AI Search
(semantic knowledge)\"]\n DOs[\"Durable Objects
Twin sessions • FolderWatcher • HITL\"]\n R2[\"R2 (artifacts, docs)\"]\n Queues[\"Queues
pipeline-process • intelligence-events • ...\"]\n end\n\n %% External World\n subgraph External[\"🔌 External World\"]\n Connectors[\"Connectors (Nango, direct MCP, APIs)\"]\n Execution[\"Execution Providers
(customer local agents, APIs, Playwright)\"]\n end\n\n Connectors -->|raw data| Connector\n AgentRuntime -->|propose + handoff| L6\n L7 -.->|x-approval-token| L6\n L6 -->|playbook / outcome| Execution\n Execution -->|feedback| Knowledge\n\n %% Twin Context Flow\n L3 -.->|assembleContinuity| Knowledge\n Knowledge -.->|hydrate| AgentRuntime\n Data -.->|D1-direct projections| GW\n\n %% Key Flows\n Pipeline --> Data\n Connector --> Pipeline\n Intelligence --> L2 & L5 & L7\n AgentRuntime --> L3\n L4 <--> Knowledge\n\n classDef layer fill:#e8f5e9,stroke:#2e7d32\n classDef service fill:#e3f2fd,stroke:#1565c0\n classDef data fill:#fff3e0,stroke:#e65100\n classDef gate fill:#ffebee,stroke:#c62828\n\n class L0,L05,L1,L2,L3,L4,L5,L6,L7 layer\n class Pipeline,Connector,Intelligence,Knowledge,AgentRuntime, GW service\n class D1,KV,Vectorize,DOs,R2,Queues data\n class L7 gate\n```\n\n**Legend**\n\n- **User Layers (left)**: What the human sees and interacts with.\n- **Services (center)**: The active engine (all Cloudflare Workers).\n- **Data Plane (bottom-right)**: 100% Cloudflare (D1 is the Spine of truth).\n- **Flows**: Ingestion (right→left), Intelligence + Twin (center), Governance gate (red), Execution handoff (external).\n\nThis diagram reflects the locked v3.7 state (DECISION 22/23 + L0–L7 + S1–S12).\n\nSee the companion documents [WORKBENCH_DOCTRINE.md](WORKBENCH_DOCTRINE.md) and [DETAILED_USER_FLOWS_AND_SYSTEM_PATHS.md](DETAILED_USER_FLOWS_AND_SYSTEM_PATHS.md) for the projection doctrine, exhaustive step-by-step user flows per layer, system stage details (S1–S12), end-to-end examples (onboarding, intelligence→governance→execution loop, Twin hydration), and data invariants.\n\nBackend system stages run underneath (S1–S12).\n\n---\n\n## 3. Detailed Layer Breakdown\n\n| Layer | User Intent | What You See | Domain Handling | Backend Stages | Current Status / How It Is Wired |\n| :------- | :---------------------------------------- | :-------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------ | :-------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| **L0** | Onboarding | 4-step wizard (Welcome → Profile → Goals → Connect) | `resolveDomain` (useCase, department) produces the single work domain (no default for work) | S1–S3 (connect + initial hydration) | **Fully implemented.** Calls initializeWorkspace({ domain }). Tracks loading timeouts for initial loaders. Active domain is written to profile + localStorage and JWT metadata. |\n| **L0.5** | Always-On Operating Layer | No direct UI (background) | N/A (tenant + domain context) | Background jobs, crons, DO alarms | **Non-UI background layer.** Powered by stateful TwinAgent (TwinAgent DO daily reflex cron + TenantBrainDO DO per-tenant 15-min HITL timeout). |\n| **L1** | User Workbench | Domain-specific dashboards (e.g. Account Success, Sales Pipeline, Personal Hub, BizOps) | Strictly differentiated. `activeDomain` + `contentDomain` decides the L1 surface | S5 Spine Write + S4 Entity Resolution + projections | **Most mature layer.** Uses `DOMAIN_CONTENT_MAP` + per-domain modules (success/, salesops/, personal/, bizops/). Fetches pass `?workDomain=DOMAIN`. Design tokens (forest/gold) enforced. |\n| **L2** | Intelligence Overlay | `⌘J` sliding drawer/panel (signals, insights, think, govern, context, evidence) | Shared on top of current L1 view. Filters by active domain | S6 Intelligence | **Present and functional.** Cognitive signals strip + insight panel. Uses `cognitive-overlay-panel-provider`. |\n| **L3** | Twin Workbench | Chat interface + reasoning surface | Shared (Twin sees the user's full context) | S10 Twin Runtime | **Native Cloudflare DOs.** Shipped as `TwinWorkbench` + `use-twin-chat` hook (SSE). No OpenWebUI, no Agent Zero. Context seeds from L1 entities + L2 insights. |\n| **L4** | Knowledge Workbench (\"Remember\") | Memory Hub: semantic search, org memory, documents, AI sessions, promoted knowledge | Differentiated (memory layer) | S7 Memory | **Routed shell.** Shipped as `memory-hub.tsx` in `UnifiedShell`. Calls `/api/v1/knowledge` endpoints. Accepts `domain` prop and queries workspace-partitioned governance memory. |\n| **L5** | Operations Workbench (\"Plan\") | Situations board, signals stream, proposal pipeline | Shared (but can scope to current work view) | S6 Intelligence | **Routed shell.** Shipped as `operations-workbench.tsx`. Pulls situations/signals/proposals via `/api/v1/intelligence` + `listProposalsForOps`. |\n| **L6** | Execution / Handoff Workbench (\"Execute\") | Ready queue (approved), running executions, history + writeback audit | Shared | S11 Execution + S12 Writeback | **Routed shell.** Shipped as `execution-workbench.tsx`. Shows approved, executing, completed proposals. Governance gate compliance checked. 24h reminder tracking logic. |\n| **L7** | Governance Workbench (\"Govern\") | Approval queue (HITL), policies, full audit trail | Shared + HARD GATE | S8 Governance (HARD GATE) | **Routed shell.** Shipped as `governance-workbench.tsx`. Approve/reject actions, policy configuration. This is the HARD GATE — no token, no execution. |\n\n---\n\n## 4. How the Layers Are Actually Wired Today\n\n1. `AppShell` decides domain + `workspacePreference` (`\"PERSONAL\"` or `\"WORK\"`) from onboarding or restored D1 profile.\n2. `WorkspaceShellNew` computes `contentDomain = activeView === \"PERSONAL\" ? \"PERSONAL\" : activeDomain`.\n3. `ContentRouter` looks up `DOMAIN_CONTENT_MAP[contentDomain][moduleId]`.\n4. Every domain map is augmented with shared `LAYER_MODULES`:\n\n```javascript\nconst LAYER_MODULES = {\n twin,\n memory,\n operations,\n execution,\n governance,\n};\nconst withLayerModules = (map) => ({ ...map, ...LAYER_MODULES });\n```\n\n5. L4–L7 all wrap content in the common `LayerShell` (consistent header, badge \"L4\"/\"L5\"/\"L6\"/\"L7\", system wire label, design tokens).\n\n---\n\n## 5. Backend System Stages (the invisible engine)\n\n| Stage | Purpose | Key Component |\n| :------ | :--------------------- | :----------------------------------------------------------------------------- |\n| **S1** | Connectivity | Connectors / Nango / MCP |\n| **S2** | Loader | Ingest raw data |\n| **S3** | Normalizer (8-stage) | `services/pipeline` |\n| **S4** | Entity Resolution | Spine deduping |\n| **S5** | Spine Write | D1 `*_data` partitions (the source of truth) |\n| **S6** | Intelligence | Signals, think, proposals (`services/intelligence`) |\n| **S7** | Memory | Org + personal + conversational (`services/knowledge` + `services/continuity`) |\n| **S8** | Governance (HARD GATE) | Approvals, policies, audit |\n| **S9** | Continuity Bridge | Assembled context |\n| **S10** | Twin Runtime | `iw-agent-runtime` (DOs) |\n| **S11** | Execution | Handoff / playbook dispatch |\n| **S12** | Writeback | Record outcomes |\n\n_Important:_ The UI (all layers) only ever reads projections assembled from the Spine. Raw data is never shown.\n\n---\n\n## 6. Key Invariants\n\n- **HARD GATE:** L7 is the only place that can mint an approval. L6 execution surface is dead without it.\n- **Single Source of Truth:** D1 (`tenant_spine_config` + domain partitions). Gateway serves many reads directly.\n- **Design System:** `--iw-forest` / `--iw-gold` + `iw-` classes (old blue/purple leaks have been cleaned in active paths).\n- **Customer Zero Verification:** 7 real checkpoints (identity, connectors, hydration, spine, memory, continuity, twin) — all D1 queries, visible in admin.\n- **No direct execution in product** except for IntegrateWise's own ops (Customer Zero).\n\n---\n\n## 7. Current Maturity (June 2026)\n\n- **L0, L1, L2, L3:** Quite solid and in active use.\n- **L4–L7:** \"Routed shells\" — they exist, use consistent `LayerShell`, call real APIs (`intelligence/proposals`, `knowledge/*`, etc.), and are available in every workbench. They are functional but lighter on custom domain-specific UI than L1.\n- **L0.5:** Running in production background (DOs + crons).\n\n_This is the actual as-built architecture after the recent fixes for domain scoping, design tokens, and layer consistency._\n\n---\n\n## 8. Exact Data Flow: L7 (Governance) → L6 (Execution)\n\nEvery execution proposal inside the product follows a governed, immutable lifecycle to maintain context integrity and enforce the execution boundary:\n\n1. **Proposal Draft / Pending:** When the Twin or background intelligence normalizers detect a high-risk situation, a proposal is written to the D1 `governance_data` table in a `pending_review` status.\n2. **Governance Workbench (L7) Approval:**\n - The operator opens L7 and clicks \"Approve\".\n - The web app sends a POST request to `/api/v1/intelligence/proposals/:id/approve` handled by `approveProposal()` in [services/intelligence/src/govern/proposal.ts](../../services/intelligence/src/govern/proposal.ts).\n - The system checks `canTransition(status, 'approved')` to enforce valid state transition.\n - The status is updated to `approved`, the decision is cryptographically signed using a `signatureKey` via `logDecision()`, and written to the Spine audit logs.\n3. **Execution Package Minting:**\n - Upon approval, the system automatically mints a first-class **Execution Package** inside the D1 `execution_packages` table.\n - This package captures the exact workflows (steps), tools (schemas), and context JSON payload required to execute the action.\n4. **Execution Handoff (L6):**\n - **Customer Zero Flow (Internal Hermes Execution):**\n - `executeAction()` triggers asynchronously.\n - The proposal's state is set to `executing` in D1.\n - The system resolves the target entity type domain (e.g. `grow`, `build`) and executes direct SQLite SQL queries (`INSERT`/`UPDATE`) on the domain partitions (e.g., `grow_data`, `build_data`).\n - Relationships are recorded in `link_data`.\n - Execution outcomes and lineage are recorded in `recordLineage()`.\n - The proposal and execution package statuses transition to `completed`.\n - **External Customer Flow (Handoff):**\n - The Execution Package is exported in a vendor-neutral canonical JSON format.\n - The customer's execution agent (e.g., OpenClaw, crewAI, local CLI) fetches the package and updates status to `HANDED_OFF`.\n - The execution runs inside the customer's firewall.\n - Upon completion, the external runtime reports results back via a POST request to `/api/v1/handoff/outcome` containing the governance token. The gateway verifies the token, resolves status to `COMPLETED`, and writes the execution memory back to the Spine.\n\n---\n\n## 9. Context Hydration: How the Twin (L3) Gets Context\n\nThe Twin (`iw-agent-runtime`) is stateful and runs entirely inside Cloudflare Durable Objects. When a user sends a chat message, the Twin hydrates its context using a multi-step grounding pipeline before invoking the LLM:\n\n```mermaid\ngraph TD\n A[User Message] --> B[Continuity Retrieval]\n B --> C{Activation Check}\n C -- Failed --> D[Block with Hydration Message]\n C -- Passed --> E[Parallel Context Search]\n E --> F[aiSearch: semantic knowledge]\n E --> G[conversationalSearch: past sessions]\n E --> H[mcpSearch: connector assets]\n A --> I{Engineering Query?}\n I -- Yes --> J[spineReadEngineering: D1 build_data]\n I -- No --> K[Skip]\n F & G & H & J --> L[Combine & Slice Context]\n L --> M[DO SQLite: Warm history turns]\n M --> N[Learned Profile: Daily reflections]\n N --> O[Construct System Prompt]\n O --> P[gatewayChat: LLM Inference]\n```\n\n1. **Step 1: Continuity Assembly & Activation Gate:**\n - The Twin calls `assembleContinuity()` to fetch the user's conversational, organizational, and personal memory.\n - `evaluateActivation()` verifies if the tenant's memory database has passed the minimum threshold (e.g., 3+ total memories, 1+ promoted). If not, the Twin refuses to answer, providing a status message about current connectors and hydration progress to prevent hallucinations.\n2. **Step 2: Parallel Context Search:**\n - The Twin performs parallel queries against:\n - `aiSearch`: Semantic chunks from the knowledge service.\n - `conversationalSearch`: Past chat memories in the hot cache.\n - `mcpSearch`: Connector resource schemas.\n3. **Step 3: Engineering Intent Detection:**\n - If the query contains engineering-related terms (e.g., repository, pull request, merge, sprint, commit), the Twin calls `spineReadEngineering()`.\n - This performs a D1-direct read on the `build_data` partition (never querying live APIs), appending PRs, active repos, and blockers to the prompt context.\n4. **Step 4: DO SQLite Warm Conversation Turns:**\n - The DO queries its local SQLite database `conversation_turns` table to fetch the 10 most recent message exchanges in the current session.\n5. **Step 5: Evolving Personal Profile:**\n - The Twin loads its `learned_profile` block, which contains communication styles, risk tolerance signals, and success patterns extracted by the `dailyReflection` cron job.\n6. **Step 6: Prompt Assembly & Gateway Chat:**\n - The Twin formats the retrieved context block, profile, and system prompt, and calls `gatewayChat()` to route the request through OpenRouter/OpenAI (with Workers AI fallback).\n7. **Step 7: Memory Event Emission:**\n - The turn is written to local DO SQLite, and a session event is dispatched asynchronously to the continuity pipeline (`emitTriageInput`) to enrich the memory substrate.\n\n---\n\n## 10. Customer Zero Verification: The 7-Stage Checklist\n\nThe Customer Zero Verification state (implemented in [services/gateway/src/workspace-spine.ts](../../services/gateway/src/workspace-spine.ts)) is a D1-direct checklist used to verify that a tenant is fully hydrated and functioning:\n\n| Stage | Checkpoint Label | D1 Verification Query | Target Done Condition |\n| :---- | :--------------- | :----------------------------------------------------- | :------------------------------------------------ |\n| **1** | Identity Schema | `SELECT schema_version FROM tenant_spine_config` | Row exists and `schema_version` is populated |\n| **2** | Connectors | `SELECT connected_connectors FROM tenant_spine_config` | At least one connector is recorded as connected |\n| **3** | Hydration | Derived from latest connector jobs | Initial \"creamy\" ingestion finished successfully |\n| **4** | Spine | `SELECT COUNT(*) FROM build_data / grow_data / ...` | Total count of entities across all partitions > 0 |\n| **5** | Memory | `SELECT COUNT(*) FROM memory_promotion_audit` | Audit records and promoted count > 0 |\n| **6** | Continuity | `SELECT topic FROM org_memory` | Promoted org memory context records are populated |\n| **7** | Twin | Computed based on memories & thresholds | `promotedCount >= 1` and `totalMemories >= 3` |\n"}