{"slug":"internal/architecture/INTEGRATEWISE_ARCHITECTURE_REAL","title":"IntegrateWise — Accurate Architecture, Boundary & Product Surfaces","section":"internal","sectionLabel":"Internal","status":"unclassified","type":"document","path":"internal/architecture/INTEGRATEWISE_ARCHITECTURE_REAL.mdx","bytes":30661,"description":null,"updated":null,"source":"integratewise/Integratewise-docs-wiki","content":"\n> **Status:** Live system audit — May 2026  \n> **Scope:** What is real, deployed, and user-facing today. Not aspirational.  \n> **Version:** 1.0 (post-verification)\n\n---\n\n## 1. Architecture — What Is Real\n\n### 1.1 Core Thesis (Implemented)\n\nIntegrateWise is an operating environment, not a chatbot or dashboard. The implemented thesis:\n\n- **One Spine** holds canonical context across all connected tools\n- **MCP is the protocol boundary** for all machine-to-machine communication\n- **Governance is structural**, not a policy wrapper: AI proposes → human approves → execution happens in the user's own stack\n- **Memory survives model/provider switches** because it lives in the Spine, not the context window\n\n### 1.2 Worker Fleet (Deployed)\n\n| Worker                          | Role                                                               | Status          |\n| ------------------------------- | ------------------------------------------------------------------ | --------------- |\n| `integratewise-gateway`         | API gateway, auth, rate limiting, routing                          | ✅ Live         |\n| `integratewise-pipeline`        | 8-stage normalizer, Spine write, queue consumer                    | ✅ Live         |\n| `integratewise-mcp-connector`   | MCP protocol server (17 tools)                                     | ✅ Live         |\n| `integratewise-intelligence`    | Consolidated v3.6 worker (govern + proposals + signals)            | ✅ Live         |\n| `integratewise-loader`          | Universal intake membrane, 8-stage pipeline                        | ✅ Live         |\n| `integratewise-connector`       | 30+ tool connectors (bridge layer)                                 | ✅ Live         |\n| `integratewise-connector-sync`  | Sync orchestration                                                 | ✅ Live         |\n| `integratewise-knowledge`       | KB ingestion, semantic search, org memory                          | ✅ Live         |\n| `integratewise-think`           | Twin reasoning, session persistence                                | ✅ Live         |\n| `integratewise-webhook-ingress` | External events entry point                                        | ✅ Live         |\n| `integratewise-billing`         | Stripe + Razorpay                                                  | ✅ Live         |\n| `integratewise-tenants`         | Tenant management                                                  | ✅ Live         |\n| `integratewise-store`           | KV/D1 operations                                                   | ✅ Live         |\n| `integratewise-continuity`      | Session continuity (migrating to Spine DB)                         | ⚠️ In migration |\n| `integratewise-workflow`        | Cloudflare Workflows/Queues orchestration (NOT n8n)                | ✅ Live         |\n| `integratewise-l2`              | Entity 360, cognitive overlays                                     | ✅ Live         |\n| `integratewise-admin`           | Admin utilities                                                    | ✅ Live         |\n| `integratewise-hermes`          | Hermes agent service binding                                       | ✅ Live         |\n| `integratewise-govern`          | Standalone governance (deprecated, consolidated into intelligence) | ⚠️ Deprecated   |\n| `integratewise-spine-v2`        | Deprecated (consolidated into pipeline)                            | ❌ Retired      |\n| `integratewise-agents`          | Deprecated (consolidated into intelligence)                        | ❌ Retired      |\n\n**Total:** 20 live workers, 2 deprecated, 2 retired.\n\n### 1.3 MCP Pipeline (Real)\n\n```\nExternal Tool\n     │\n     ▼\n    MCP ──── Loader (fetch) ──── Normalizer (transform) ──── Spine\n```\n\n**What is implemented:**\n\n- `services/mcp-connector/src/handlers/tools.ts` exposes **17 real tools**:\n  - `kb.*` (7): write_session_summary, write_article, get_artifact, list_recent, search, topic_upsert, topic_list\n  - `memory.*` (5): write_conversational, read_conversational, upsert_org, search_org, propose\n  - `figma.*` (5): get_file, get_components, get_node, export_image, get_comments\n- Invoke handlers (lines 1121–1942) make actual KV/D1/Spine DB calls\n- `services/loader/src/pipeline-stages.ts` implements **8 real stages**:\n  1. **Analyzer**: SHA-256 fingerprint\n  2. **Classifier**: data_kind detection, PII regex scan\n  3. **Filter**: D1 dedup (`processed_fingerprints`), tenant schema gating\n  4. **Refiner**: composite splitting, cross-ref extraction\n  5. **Extractor**: canonical field mapping across 12 domains\n  6. **Validator**: required-field checks\n     7/8. **Write Plan + Execution**: generates and executes write plan\n\n**What is NOT implemented:**\n\n- \"LLM-in-the-loop at stages 2, 3, 4, 6\" — these are heuristic (regex, keyword scoring), no LLM calls inside the normalizer\n- Stale `mcp-tools.json` still references `analyze_with_python` (ZERO PYTHON violation, not active)\n\n### 1.4 Spine (Real)\n\nThe Spine is **one logical layer across multiple physical stores**:\n\n| Store                 | Role                                | Data                                                                                                                         | Access Pattern                                             |\n| --------------------- | ----------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- |\n| **Spine DB Postgres** | Canonical context, long-term memory | `public.entities`, `memory.org_memory`, `memory.conversational_memory`, pgvector embeddings                                  | Worker-only writes. Frontend reads via Gateway.            |\n| **Cloudflare D1**     | Hot cache, governance, fast queries | `proposals`, `governance_audit_log`, `processed_fingerprints`, `personal_memory` (D1 only), `conversational_memory` (mirror) | Readable by multiple workers. Writable by pipeline/govern. |\n| **Cloudflare KV**     | Hot entity cache                    | Serialized entity blobs                                                                                                      | TTL 60–300s                                                |\n| **Cloudflare R2**     | Blob storage                        | Large artifacts, exports                                                                                                     | Direct signed URLs                                         |\n| **Redis (Upstash)**   | Session state, rate limit counters  | Ephemeral operational state                                                                                                  | Fast ephemeral                                             |\n\n**Critical rule:** Spine DB is accessed **only through Cloudflare Workers** for writes. D1 is the external-readable hot cache.\n\n**Reality check:** Frontend has two direct Spine DB bypasses — `useSpine DBEntities` and `useDemoData` — that call RPC directly from browser. These are known deviations.\n\n### 1.5 Memory Layers (Partially Real)\n\n| Layer                     | Spine DB                                                  | D1                                           | Status            |\n| ------------------------- | --------------------------------------------------------- | -------------------------------------------- | ----------------- |\n| **Conversational Memory** | ✅ `memory.conversational_memory`                         | ✅ Mirror table                              | **Real**          |\n| **Organizational Memory** | ✅ `memory.org_memory` (pgvector, RLS, governance states) | ❌ Not mirrored                              | **Real**          |\n| **Personal Memory**       | ❌ **Not present**                                        | ✅ `personal_memory` (uncommitted migration) | **D1 only — gap** |\n\n**Governance states in `org_memory`:** `staging` → `approved` → `published` → `archived`. Promotion requires explicit action.\n\n### 1.6 Governance (Real)\n\nImplemented in D1 (`services/govern/src/proposal.ts` + `services/govern/src/audit.ts`):\n\n- **Proposals table**: 27 columns, full lifecycle (`draft` → `pending_review` → `approved` → `executing` → `completed`/`failed`)\n- **Audit log**: HMAC SHA-256 signed entries\n- **API surface**: `POST /v1/proposals`, `POST /v1/proposals/:id/approve`, `POST /v1/proposals/:id/reject`, `GET /v1/audit`\n- **Continuity writeback**: Approval updates Spine DB `org_memory` via `writebackContinuity`\n- **Status**: Standalone govern worker is deprecated (v3.6 consolidated into `integratewise-intelligence`). Code is real but migrating.\n\n### 1.7 Auth (Real)\n\n- **Spine DB JWT is canonical** across all three workspaces\n- One identity: `app.integratewise.ai`, `ops.integratewise.ai`, `knowledge.integratewise.ai`\n- Gateway validates JWT, routes to appropriate worker\n- Known issue: Gateway returns 500/1101 with fake/test tokens (pre-existing, not a blocker)\n\n---\n\n## 2. Boundary — What IntegrateWise Is vs Is Not\n\n### 2.1 Inside the Boundary (We Do)\n\n| Capability                                                                | Implementation                                    |\n| ------------------------------------------------------------------------- | ------------------------------------------------- |\n| **Ingest** any structured or semi-structured data from connected tools    | Loader + 8-stage normalizer                       |\n| **Normalize** heterogeneous data into canonical Spine entities            | Normalizer (heuristic stages, compounding writes) |\n| **Reason** over the Spine to detect risk, opportunity, and anomalies      | Twin (Think worker), Intelligence worker          |\n| **Propose** actions, memory promotions, and operational changes           | Proposal engine with confidence scoring           |\n| **Govern** every proposal through HITL approval with audit lineage        | Govern engine, D1 proposals + audit log           |\n| **Project** Spine truth into user-facing surfaces                         | React/Vue components, MCP tool calls              |\n| **Remember** conversationally, organizationally, and (D1-only) personally | Spine DB `memory.*` + D1 mirrors                  |\n| **Search** semantically across org memory                                 | pgvector + Knowledge worker                       |\n| **Maintain continuity** across sessions, models, and providers            | Spine persistence, memory layers                  |\n| **Hand off** execution intent to the user's own stack                     | Twin → user agentic claw (UI, not API execution)  |\n\n### 2.2 Outside the Boundary (We Do NOT)\n\n| Capability                                         | Why Not                                  | What Happens Instead                                                                      |\n| -------------------------------------------------- | ---------------------------------------- | ----------------------------------------------------------------------------------------- |\n| **Execute actions directly in external tools**     | Security, liability, trust               | Twin proposes → user approves → user executes in their own tool via deep link             |\n| **Write directly to Spine DB from frontend**       | Architecture violation                   | All writes: Frontend → Gateway → Worker → Spine DB. (Two known bypasses exist.)           |\n| **Use Python anywhere**                            | Architecture lock (TypeScript/Node only) | Zero Python in production.                                                                |\n| **Use Firestore as canonical state**               | Firestore is v0/GCP ops monitoring only  | All canonical state in Spine DB (memory) + D1 (governance cache).                         |\n| **Store credentials, tokens, or API keys in code** | Hard security rule                       | Secrets in Worker env bindings, Doppler, or KV — never in repo.                           |\n| **Deploy to production automatically**             | Explicit human gate                      | `wrangler deploy` only on instruction. OAuth session (connect@integratewise.ai) required. |\n| **Delete archived repos**                          | Never delete                             | ARCHIVE = no new development. Repos persist for history.                                  |\n\n### 2.3 The Handoff Model (Boundary in Practice)\n\n```\n┌─────────────────────────────────────────────────────────────────┐\n│                     INSIDE INTEGRATEWISE                        │\n│  ┌──────────┐    ┌──────────┐    ┌──────────┐                  │\n│  │  Twin    │───→│ Proposal │───→│  HITL    │                  │\n│  │ Reasons  │    │ Engine   │    │ Approval │                  │\n│  └──────────┘    └──────────┘    └────┬─────┘                  │\n│                                        │                        │\n│                              ┌─────────▼──────────┐             │\n│                              │   Audit + Lineage  │             │\n│                              │   (D1, HMAC-signed)│             │\n│                              └────────────────────┘             │\n└─────────────────────────────────────────────────────────────────┘\n                              │\n                              ▼ HANDOFF\n┌─────────────────────────────────────────────────────────────────┐\n│                   OUTSIDE INTEGRATEWISE                         │\n│  ┌──────────────────────────────────────────────────────┐      │\n│  │  User's Agentic Claw (their tools, their creds)      │      │\n│  │  - Click deep link to HubSpot / Jira / Stripe        │      │\n│  │  - Execute in native tool UI                         │      │\n│  │  - MCP re-reads result → Spine updates (async)       │      │\n│  └──────────────────────────────────────────────────────┘      │\n└─────────────────────────────────────────────────────────────────┘\n```\n\n**The rule:** IntegrateWise reasons, plans, and prepares. The user executes. Execution never happens inside our workers against external APIs.\n\n---\n\n## 3. Productised Solutions — What the User Gets\n\n### 3.1 Three Workspace Architecture (Real vs Target)\n\nThe product is delivered through **three workspaces**. Today they are at different maturity levels:\n\n| Workspace        | URL                          | Role                                                                           | Status                                                    |\n| ---------------- | ---------------------------- | ------------------------------------------------------------------------------ | --------------------------------------------------------- |\n| **Intelligence** | `app.integratewise.ai`       | Read + Decide. At-risk briefs, entity 360, signal detection, Twin reasoning.   | ✅ **Live and working**                                   |\n| **Execution**    | `ops.integratewise.ai`       | Act + Log. Governance queue, department playbooks, KPIs, action logs.          | ⚠️ **Rebuilding — Firestore → MCP migration in progress** |\n| **Knowledge**    | `knowledge.integratewise.ai` | Remember + Govern. Org memory search, semantic discovery, doctrine management. | ⚠️ **Partial — org_memory works, surface incomplete**     |\n\n### 3.2 Intelligence Surface (Live)\n\n**What the user sees:**\n\n- **Unified Shell**: One `AppShell` with Personal/Work toggle\n- **At-Risk Brief**: CS view showing accounts ranked by risk (Critical/High/Medium/Low)\n- **Entity 360**: HubSpot + Jira + Gmail merged into one Person/Company view\n- **Twin Sidebar**: Embedded at bottom/right for insights, suggestions, approvals inline\n- **Proposals Inline**: Pending Twin proposals appear in-context with Approve/Reject buttons\n- **Signals**: Live signal stream (`useSignals`) with auto-refresh (30s)\n\n**Data flow:**\n\n```\napp.integratewise.ai\n    → Gateway (/api/v1/intelligence/*)\n    → Intelligence Worker (/v1/proposals, /v1/signals)\n    → D1 (proposals, signals) + Spine DB (entities, memory)\n```\n\n**Key components:**\n\n- `CSAtRiskView` — live signals + proposals matched to accounts\n- `useProposals` — React Query hook, 30s refresh, approve/reject mutations\n- `useSignals` — live signal stream\n\n### 3.3 Execution Surface (In Rebuild)\n\n**What the user will see (target):**\n\n- **Universal Shell**: `/execution/:dept` — one layout, six panels, configured per department\n- **Six Panels**:\n  1. **Playbook** — Standard operating procedures, runbooks, decision trees\n  2. **Tools** — Deep links to native tools (HubSpot, Jira, Stripe, etc.)\n  3. **KPIs** — Metrics and health indicators for the department\n  4. **Workflows** — Active Cloudflare Workflows, triggers, automations\n  5. **Actions** — Governance queue: pending proposals with approve/reject\n  6. **Log** — Audit trail of all decisions and executions\n\n**Current state:**\n\n- `integratewise-ops/src/pages/cs.vue` — CS prototype with 4 tabs (Playbook, KPI, Actions, Log)\n- Uses `useMcpTool` composables to call MCP (`proposal.list`, `signal.list`)\n- Hardcoded `demo-tenant`, no real auth yet\n- **Data layer migration needed**: `useCollection` still reads Firestore (wrong)\n\n**Architecture rule for Execution:** MCP-only. No direct Spine DB. No direct connectors. Every read goes through `integratewise-mcp-connector`.\n\n### 3.4 Knowledge Surface (Partial)\n\n**What works today:**\n\n- `memory.org_memory` table with pgvector search\n- Categories: `doctrine`, `architecture`, `decision`, `workflow`, `policy`, `insight`, `lesson_learned`, `pattern`\n- Governance states: `staging`, `approved`, `published`, `archived`\n- MCP tools: `memory.search_org`, `memory.upsert_org`\n\n**What is missing:**\n\n- Dedicated Knowledge workspace UI (not a separate route yet)\n- `memory.personal_memory` in Spine DB (only exists in D1 migration)\n- Full \"Memory View Layer\" as described in early architecture docs\n\n### 3.5 Product Modules (Feature Matrix)\n\n| Module                | Intelligence    | Execution          | Knowledge        | Status          |\n| --------------------- | --------------- | ------------------ | ---------------- | --------------- |\n| Entity 360            | ✅ View         | ⚠️ Deep links only | ❌               | Live            |\n| At-Risk Brief         | ✅ Full         | ❌                 | ❌               | Live            |\n| Proposals / HITL      | ✅ Inline       | ✅ Actions panel   | ❌               | Live            |\n| Governance Audit      | ❌              | ✅ Log panel       | ❌               | Live            |\n| Org Memory Search     | ⚠️ Embedded     | ❌                 | ⚠️ MCP tool only | Partial         |\n| Personal Memory       | ❌              | ❌                 | ❌               | Not in Spine DB |\n| Conversational Memory | ✅ Twin uses it | ❌                 | ❌               | Live            |\n| Semantic Search       | ⚠️ Backend only | ❌                 | ⚠️ Backend only  | Partial         |\n| Department Playbooks  | ❌              | ⚠️ CS prototype    | ❌               | In build        |\n| Tool Deep Links       | ⚠️ Inline       | ✅ Tools panel     | ❌               | Partial         |\n| Workflow Triggers     | ❌              | ⚠️ CF Workflows    | ❌               | Partial         |\n\n---\n\n## 4. User Views — What the User Actually Sees\n\n### 4.1 Authenticated Shell (`app.integratewise.ai/app`)\n\n```\n┌─────────────────────────────────────────────────────────────┐\n│  [Logo]  Personal | Work        [Search]    [User] [Bell]   │  ← Top Nav\n├─────────────────────────────────────────────────────────────┤\n│                                                             │\n│  ┌─────────────────────────────────────────────────────┐   │\n│  │                                                     │   │\n│  │              WORKBENCH CONTENT AREA                 │   │\n│  │                                                     │   │\n│  │  - At-Risk Accounts                                 │   │\n│  │  - Entity 360 Cards                                 │   │\n│  │  - Knowledge Modules (embedded)                     │   │\n│  │                                                     │   │\n│  └─────────────────────────────────────────────────────┘   │\n│                                                             │\n│  ┌─────────────────────────────────────────────────────┐   │\n│  │  🤖 TWIN SIDEBAR                                    │   │  ← Bottom/Right\n│  │  \"3 accounts at risk. Approve draft email?\"         │   │\n│  │  [Approve] [Reject] [Explain]                       │   │\n│  └─────────────────────────────────────────────────────┘   │\n│                                                             │\n└─────────────────────────────────────────────────────────────┘\n```\n\n**Reality:** This is **one shell**, not three surfaces. The Twin is a sidebar. Memory is embedded modules. There is no separate \"Twin Workbench\" route or \"Memory View Layer\" route.\n\n### 4.2 Execution Shell (`ops.integratewise.ai/execution/:dept` — Target)\n\n```\n┌─────────────────────────────────────────────────────────────┐\n│  [IW]  Execution: Customer Success        [User] [Settings] │\n├─────────────────────────────────────────────────────────────┤\n│  [Playbook] [Tools] [KPIs] [Workflows] [Actions] [Log]      │  ← 6 Panels\n├─────────────────────────────────────────────────────────────┤\n│                                                             │\n│  ┌──────────────┐  ┌─────────────────────────────────────┐ │\n│  │ PLAYBOOK     │  │  Content for selected panel         │ │\n│  │ - Runbook A  │  │                                     │ │\n│  │ - Runbook B  │  │  (Playbook: SOPs, decision trees)   │ │\n│  │ - Decision C │  │  (Tools: Deep links to HubSpot...)  │ │\n│  └──────────────┘  │  (KPIs: Metrics cards)              │ │\n│                    │  (Workflows: CF Workflow trigger list) │ │\n│                    │  (Actions: Proposal queue)          │ │\n│                    │  (Log: Audit trail)                 │ │\n│                    └─────────────────────────────────────┘ │\n│                                                             │\n└─────────────────────────────────────────────────────────────┘\n```\n\n**Current state:** Only CS department prototype exists (`cs.vue`). Other departments not built.\n\n### 4.3 Proposal Flow (User Journey)\n\n```\nUser sees at-risk account in Intelligence view\n              │\n              ▼\nTwin generates proposal: \"Draft retention email to Acme Corp\"\n              │\n              ▼\nProposal appears in Intelligence view (inline) AND Execution Actions panel\n              │\n              ▼\nUser clicks [Approve] or [Reject]\n              │\n              ▼\nGateway → Intelligence Worker → D1 (update proposal + audit log)\n              │\n              ▼\nContinuity writeback: Spine DB org_memory updated (async, non-blocking)\n              │\n              ▼\nUser executes in their own HubSpot/Gmail (deep link, not API call)\n              │\n              ▼\nConnector re-reads result via MCP → Spine updates → loop continues\n```\n\n### 4.4 Memory Interaction (User Journey)\n\n```\nUser asks Twin: \"What did we decide about pricing last quarter?\"\n              │\n              ▼\nTwin calls memory.search_org (MCP tool)\n              │\n              ▼\nMCP Connector → Knowledge Worker → Spine DB pgvector search\n              │\n              ▼\nReturns ranked org_memory entries (doctrine, decision, insight)\n              │\n              ▼\nTwin surfaces answer + citations\n              │\n              ▼\nUser says: \"Promote this to approved doctrine\"\n              │\n              ▼\nProposal created → HITL approval → governance_state = 'approved'\n```\n\n---\n\n## 5. Honest Gap List\n\n| Gap                                                               | Impact                                        | Path Forward                                                                        |\n| ----------------------------------------------------------------- | --------------------------------------------- | ----------------------------------------------------------------------------------- |\n| `personal_memory` missing from Spine DB                           | Personal Memory feature blocked               | Add table to `memory.*` schema, migrate D1 data                                     |\n| Three surfaces = one shell today                                  | Architecture doc overstates product structure | Build Execution Shell (`/execution/:dept`) and Knowledge surface as separate routes |\n| Frontend Spine DB bypasses (`useSpine DBEntities`, `useDemoData`) | Security posture weakened                     | Route through Gateway or document as intentional                                    |\n| Normalizer stages 2–6 are heuristic, not LLM                      | \"AI-native\" claim is overstated for ingestion | Document as roadmap, not current capability                                         |\n| `mcp-tools.json` stale (python reference)                         | Confuses new developers                       | Delete or sync with live `tools.ts`                                                 |\n| `spine_entities` → `public.entities` naming drift                 | Documentation mismatch                        | Update all docs to use `entities`                                                   |\n| Execution Layer reads Firestore                                   | Wrong data source for ops                     | Complete MCP migration (`useMcpTool` composables)                                   |\n| Govern worker deprecated but code duplicated                      | Maintenance burden                            | Finish consolidation into `integratewise-intelligence`                              |\n| No standalone Memory View Layer route                             | Knowledge surface invisible to users          | Build `/knowledge` route with org_memory search UI                                  |\n\n---\n\n## 6. Technology Stack (Verified)\n\n| Layer             | Technology                                                          |\n| ----------------- | ------------------------------------------------------------------- |\n| **Frontend**      | React 18 (Intelligence), Vue 3 (Execution)                          |\n| **API / Workers** | Cloudflare Workers, Hono, TypeScript                                |\n| **Protocol**      | MCP (Model Context Protocol)                                        |\n| **Canonical DB**  | Spine DB Postgres (memory, entities)                                |\n| **Hot Cache**     | Cloudflare D1 (proposals, audit, fingerprints)                      |\n| **Session Cache** | Cloudflare KV                                                       |\n| **Queue**         | Cloudflare Queues (pipeline-process, knowledge-ingest, etc.)        |\n| **Search**        | Spine DB pgvector (1536-dim embeddings)                             |\n| **Auth**          | Spine DB JWT                                                        |\n| **Blob**          | Cloudflare R2                                                       |\n| **Build**         | Vite, pnpm, Turbo                                                   |\n| **Banned**        | Python, Firestore (canonical), direct Spine DB writes from frontend |\n\n---\n\n_This document was produced by codebase audit on May 2026. Every claim is backed by file-level evidence. When reality changes, this document should be updated._\n"}