Salesforce Private AI Duplicate Detection with Qwen
Duplicate records are a persistent Salesforce data-quality challenge. They commonly appear following migrations, acquisitions, system integrations, bulk data loads and years of inconsistent data entry.
Salesforce provides native Matching Rules and Duplicate Rules, and several commercial products extend those capabilities. However, organizations may still face additional licensing, proprietary matching engines, manual review processes or dependencies on products such as Agentforce and Data 360.
I wanted to explore another architectural option:
Can Salesforce users find and review duplicate Accounts through natural language while keeping the model, security controls and duplicate decisions under customer control?
The result is a working Salesforce-native Private AI Duplicate Assistant built using Qwen, Amazon Bedrock, Lightning Web Components, Apex, metadata-validated dynamic SOQL, deterministic matching and controlled merge execution.
Watch the demonstration
Link to the demostration
The video demonstrates the complete journey from a natural-language request to validated Account retrieval, explainable duplicate groups, interactive review, merge simulation and controlled execution.
The user experience
A Salesforce user can enter a request such as:
“Find duplicate Accounts created this year in Canada.”
The user does not need to create a report, write SOQL, export records or leave Salesforce.
The AI Duplicate Assistant interprets the request, validates it and displays the results as interactive duplicate groups.
The reviewer can examine:
Confidence scores
Matching explanations
Side-by-side field values
Field conflicts
Recommended master records
Account hierarchy
Related-record impact
Projected surviving values
Merge simulation
Saved resolution plans
Why Qwen?
For this proof of concept, I selected Qwen 3.6 35B A3B.
Qwen provides strong natural-language understanding, instruction following and structured JSON-generation capabilities. These characteristics make it suitable for interpreting a business request and converting it into a constrained machine-readable intent.
The model is used as a semantic parser not as the Salesforce execution engine.
Qwen does not receive unrestricted Salesforce credentials, directly execute SOQL, calculate the final duplicate score or merge records.
The architecture is also model-agnostic. Qwen can be replaced by another compatible model without redesigning the Lightning user experience or Salesforce validation layer.
Local proof-of-concept architecture
For the original local PoC, Qwen ran through LM Studio on a developer workstation.
The local architecture included:
A Lightning Web Component for natural-language requests and interactive review.
An Apex controller and integration service.
A Salesforce Named Credential and External Credential.
A temporary Cloudflare Quick Tunnel.
A local cloudflared tunnel agent.
A PowerShell gateway for authentication, prompting and response normalization.
LM Studio exposing an OpenAI-compatible REST endpoint.
Qwen running locally on the workstation.
Salesforce cannot directly call a laptop’s localhost endpoint. The temporary Cloudflare tunnel therefore provided an HTTPS route to the local gateway.
This approach was useful for development and validation, but a temporary tunnel is not the recommended production topology.
Alternative local or self-hosted runtimes include Ollama, llama.cpp, vLLM and NVIDIA NIM. These can run on a workstation, GPU server, Docker environment, Kubernetes platform or on-premises infrastructure.
Customer-controlled AWS architecture
For the cloud implementation, the laptop, PowerShell gateway, LM Studio runtime and temporary tunnel were replaced by Amazon Bedrock.
Salesforce authenticates through a Named Credential and External Credential. The model request is sent to a regional Amazon Bedrock Runtime endpoint.
AWS IAM provides least-privilege authorization, while Amazon Bedrock provides managed inference for Qwen.
Additional AWS controls can include:
AWS KMS encryption
CloudWatch monitoring
CloudTrail auditing
Regional model deployment
Service-control policies
Customer-defined logging and retention requirements
The model endpoint operates within the customer’s AWS environment. Salesforce information does not need to be submitted to a shared consumer chatbot or uncontrolled AI endpoint.
Other hosting alternatives include Amazon SageMaker, EC2, EKS, Microsoft Foundry, Azure Machine Learning, AKS, Google Vertex AI, GKE and customer-managed Kubernetes with vLLM or NVIDIA NIM.
The Salesforce trust boundary
The most important architectural decision is that the language model never receives authority to execute arbitrary Salesforce operations.
Qwen produces constrained JSON containing information such as:
Primary object
Account filters
Related-record conditions
Requested fields
Minimum confidence
Permitted operation
Apex treats this response as untrusted input.
Using Salesforce Schema Describe, the validation layer verifies:
Object availability
Field existence
CRUD and field-level security
Queryability and filterability
Supported operators
Salesforce datatypes
Picklist values
Approved relationships
Candidate limits
Permitted operations
Only after validation succeeds does Apex construct the SOQL query.
The query executes in user mode, allowing Salesforce security, field access and sharing to remain authoritative.
Explainable duplicate scoring
The model does not decide whether two Accounts are duplicates.
After candidate retrieval, deterministic Apex logic currently evaluates:
Normalized phone number: 40%
Normalized website domain: 30%
Fuzzy Account-name similarity: 20%
Fuzzy billing-address similarity: 10%
The matcher also requires corroborating evidence. One weak similarity is not enough to create a duplicate group.
Candidate blocking reduces unnecessary pairwise comparisons using phone numbers, websites, Account-name prefixes, city and postal-code characteristics.
Each duplicate group receives a confidence score and an explanation of why the records were matched.
Human-in-the-loop merge governance
The data steward reviews the evidence before any transaction occurs.
The application supports Account hierarchy review, related-record impact, field-conflict identification, recommended master selection and merge simulation.
The current PoC supports a controlled merge for two or three reviewed Accounts.
Before execution, Apex reloads the records, validates the selected master, confirms the duplicate members and evaluates safety conditions.
The LLM never performs the merge.
What about millions of records?
The current synchronous PoC intentionally limits candidate retrieval to 500 Accounts.
An enterprise implementation would not send five million records to the LLM, browser or a single Apex transaction.
The validated request would instead create a persistent asynchronous analysis job. Records could be partitioned using:
Migration batch
Source system
Geography
Account type
Creation date
External identifiers
Normalized blocking keys
Processing could use Batch Apex, Queueable Apex, Platform Events, Salesforce APIs or a customer-hosted matching service.
Users would review prioritized duplicate groups—not millions of raw records.
High-confidence, low-risk groups could follow governed bulk approval. Medium-confidence groups could require data-steward review. High-impact groups involving Opportunities, Contracts, Cases or complex hierarchies could be escalated.
Proof of concept and future scope
The current implementation proves the end-to-end architecture for a bounded Salesforce dataset.
Future production-engineering capabilities could include:
Millions-of-record processing
Persistent analysis jobs
Dynamic standard and custom matching fields
Configurable algorithms and weights
Data-steward work queues
Bulk approval policies
Merge rollback
Operational monitoring
Accuracy benchmarking
False-positive and false-negative evaluation
Production concurrency and failure recovery
This is not a claim that AI-powered Salesforce deduplication has never been attempted.
It is an independently deployable Private AI reference architecture whose model, validation, matching and governance logic can be inspected, tested and adapted by the customer.
Qwen provides natural-language intelligence. Apex provides the zero-trust validation boundary. Salesforce metadata provides runtime governance. Deterministic scoring provides explainable evidence. The Lightning Web Component provides human-in-the-loop review. And the controlled service layer protects the final transaction.
Discuss the architecture & Use Case
If your organization is dealing with Salesforce duplicates following a migration, acquisition, integration or bulk data load, contact me to discuss the architecture, customer-hosting options and enterprise implementation approach.
Contact info: contactus@advantex.io
Comments