{"slug":"business/ENTITY_DEDUP_AND_IDENTITY_RESOLUTION","title":"Current Surface-Binding Doctrine","section":"business","sectionLabel":"Business","status":"unclassified","type":"document","path":"business/ENTITY_DEDUP_AND_IDENTITY_RESOLUTION.mdx","bytes":3534,"description":null,"updated":null,"source":"integratewise/Integratewise-docs-wiki","content":"## Current Surface-Binding Doctrine\n\n- Operational Workbench = the primary work surface; left sidebar nav; Twin sidebar pops out when needed.\n- Twin Workbench = full AI ecosystem surface (skills, knowledge, agents, prompts, conversational library).\n- Governance is embedded at every layer — not a separate workbench.\n- OpenWebUI is not the customer-facing product shell.\n- Twin / OpenWebUI runtime surfaces use the canonical Forest + Paper system language.\n- Midnight Executive is investor-deck / fundraising language only.\n\nUse older L1/L2 / workspace / knowledge-UI language in this file through that boundary if any historical phrasing remains.\n\n---\n\n# Entity Deduplication & Identity Resolution\n\n## Overview\n\nS7.5 Identity Resolution runs after S7 Sanity and before S8 Sectorize in the 8-stage normalizer pipeline. It detects potential cross-source duplicates but NEVER auto-merges. All merge decisions require human review (HITL).\n\n## How It Works\n\n### Detection (Automatic)\n\n1. Entity passes through S1-S7 of the pipeline\n2. S7.5 generates a signature from identifying fields (email, domain, name, phone)\n3. Signature stored in D1 `entity_signatures` table\n4. Query for matches: same tenant + entity type + different source system\n5. If match found with sufficient confidence → create `merge_candidate` record\n6. Entity ALWAYS written to Spine as separate record (never blocked)\n\n### Supported Entity Types\n\naccount, contact, lead, ticket, opportunity, people_team\n\n### Matching Logic\n\n| Field  | Weight | Match Type          |\n| ------ | ------ | ------------------- |\n| email  | High   | Exact (normalized)  |\n| domain | Medium | Exact               |\n| name   | Medium | Fuzzy (normalized)  |\n| phone  | Low    | Exact (digits only) |\n\n### Confidence Scoring\n\n- Email exact match: 0.95\n- Domain + name match: 0.80\n- Name + phone match: 0.75\n- Domain only: 0.50\n\n## Resolution (Human)\n\n### API Endpoints\n\n| Method | Path                                         | Action                        |\n| ------ | -------------------------------------------- | ----------------------------- |\n| GET    | `/api/identity/candidates`                   | List pending candidates       |\n| GET    | `/api/identity/candidates/:id`               | Single with entity comparison |\n| POST   | `/api/identity/candidates/:id/merge`         | Merge two records             |\n| POST   | `/api/identity/candidates/:id/keep-separate` | Not duplicates                |\n| POST   | `/api/identity/candidates/:id/defer`         | Revisit later                 |\n| GET    | `/api/identity/stats`                        | Signature + candidate counts  |\n\n### Merge Action\n\nWhen merged:\n\n1. Primary record kept as canonical\n2. Secondary record's unique fields merged (if merge_fields strategy)\n3. Merge logged to Spine DB `audit_log`\n4. Candidate status → \"merged\"\n\n### Keep Separate\n\nWhen kept separate:\n\n1. Candidate status → \"rejected\"\n2. Future matching suppressed for this pair\n\n## D1 Schema\n\n```sql\n-- entity_signatures: one per entity per source\nCREATE TABLE entity_signatures (\n  tenant_id TEXT, entity_type TEXT, source_system TEXT,\n  external_id TEXT, signature TEXT, email TEXT,\n  domain TEXT, name TEXT, phone TEXT, created_at TEXT\n);\n\n-- merge_candidates: pending HITL review\nCREATE TABLE merge_candidates (\n  tenant_id TEXT, entity_type TEXT,\n  source_a TEXT, external_id_a TEXT,\n  source_b TEXT, external_id_b TEXT,\n  confidence REAL, matched_fields TEXT,\n  status TEXT DEFAULT 'pending',\n  resolved_by TEXT, resolved_at TEXT\n);\n```\n"}