SfSkills Domains Docs GitHub

Home / data / lead-data-import-and-dedup

lead-data-import-and-dedup

Lead data import quality and deduplication strategy: matching rule design for Leads, Data Import Wizard dedup behavior, Web-to-Lead data quality controls, cross-object Lead-to-Contact fuzzy dedup, and enrichment patterns.

Open the skill folder on GitHub

Lead Data Import and Dedup

Use this skill when leads are entering Salesforce from file imports, web forms, or marketing integrations and you need to prevent duplicates, detect existing duplicates, or enrich incoming data before it pollutes the CRM. It covers matching rule design specific to Lead objects, the limitations of Data Import Wizard dedup, Web-to-Lead bypass behavior, and the constraints of native vs. third-party cross-object matching. It does not cover large-scale batch merge execution or Data Loader mechanics.

Before Starting

Gather this context before designing any lead import or dedup workflow:

Core Concepts

Standard Lead Matching Rule and Its Algorithms

The Standard Lead Matching Rule (enabled by default in all Salesforce orgs) uses three matching methods on different field groups:

Field(s)Matching MethodBehavior
First Name, Last NameJaro-Winkler fuzzyTolerates transpositions and minor misspellings
Last Name (phonetic)Metaphone 3Catches phonetic variants (e.g., "Smith" vs. "Smyth")
EmailExactCase-insensitive exact match only
PhoneNormalized exactStrips formatting characters before comparison
CompanyJaro-Winkler fuzzyTolerates abbreviations and minor differences

A match is declared when the combined score across enabled fields exceeds the rule's configured threshold. Practitioners often assume the standard rule catches all near-duplicates; in practice it misses records with mismatched email domains and correct names, or same-email leads with different company names when Email is weighted too low.

Custom matching rules can extend this by adding additional field comparisons, but each rule still uses only the built-in algorithm options — there is no way to inject custom similarity logic through native configuration alone.

Data Import Wizard Dedup: Single-Field Exact Match Only

The Data Import Wizard's built-in dedup capability is significantly more limited than Duplicate Rules:

For list imports where data quality is poor (name variants, missing email, abbreviated company), the Data Import Wizard dedup will miss a high percentage of true duplicates. A pre-import cleansing step (normalizing email, standardizing company name) dramatically improves match rates before the wizard runs.

Web-to-Lead Bypass Behavior

Web-to-Lead is a POST-based lead capture mechanism that inserts records via an internal Salesforce pathway that does not enforce Duplicate Rule blocking:

The practical implication: if your org relies on Duplicate Rule blocking to prevent web form spam or duplicate capture, the rule will silently fail for every form submission. The correct design pattern is a post-insert Apex trigger or a Flow on Lead creation that checks for duplicates and either merges, alerts, or routes the record — rather than relying on blocking.

Cross-Object Lead-to-Contact Dedup

Salesforce ships a standard "Lead-to-Contact" matching rule that checks incoming Lead records against existing Contacts using Email (exact) and Name (fuzzy). This is the native mechanism for detecting leads that already exist as contacts in the org before conversion.

Native limitations:

For true fuzzy cross-object dedup at volume, third-party tools (DemandTools, Cloudingo, Plauti) are the standard recommendation.

Common Patterns

Pre-Import Normalization + Data Import Wizard

When to use: CSV-based list imports from trade shows, marketing events, or vendor lists where email is present but data quality varies.

How it works:

  1. Before uploading to Salesforce, run the CSV through a normalization script or spreadsheet formula:
    • Lowercase and trim all email addresses.
    • Standardize phone numbers to E.164 format.
    • Strip common company suffixes ("Inc.", "LLC", "Ltd") from Company field before import if exact matching will be used.
  2. Remove exact-email duplicates within the import file itself before uploading (VLOOKUP or pandas.drop_duplicates on Email).
  3. Use Data Import Wizard with Email as the match field and "Update existing records" mode. This performs an exact-match upsert on Email — records with matching email are updated rather than inserted as new.
  4. After import, run a SOQL query to find leads with no email that were inserted without dedup protection. Route these to a data steward for manual review.

Why not skip normalization: The Data Import Wizard's exact-match on Email will miss John@example.com vs. john@example.com unless email is lowercased. Without stripping suffixes, "Acme Inc." and "Acme" are treated as different companies.

Apex Trigger Post-Insert Duplicate Check for Web-to-Lead

When to use: The org receives leads via Web-to-Lead or API, Duplicate Rule blocking does not apply to this channel, and the business requires real-time duplicate detection and routing.

How it works:

  1. Write an after-insert Apex trigger on Lead.
  2. In the trigger, call Datacloud.DuplicateRule.findDuplicates() to detect whether the new lead matches existing leads or contacts.
  3. If a match is found above confidence threshold: set a custom Duplicate_Status__c field, create a Task for the owning rep, or — for automated programs — invoke a Flow to auto-assign the record to a dedup queue.
  4. Do not auto-merge from an after-insert trigger: Database.merge() in a trigger context that fired on insert can cause recursive trigger execution and governor limit issues. Queue merges to a batch job instead.
trigger LeadDuplicateCheck on Lead (after insert) {
    List<SObject> newLeads = new List<SObject>(Trigger.new);
    Datacloud.FindDuplicatesResult[] results =
        Datacloud.DuplicateRule.findDuplicates(newLeads);

    List<Lead> toUpdate = new List<Lead>();
    for (Integer i = 0; i < results.size(); i++) {
        Integer matchCount = 0;
        for (Datacloud.DuplicateResult dr : results[i].getDuplicateResults()) {
            matchCount += dr.getMatchResults().size();
        }
        if (matchCount > 0) {
            Lead l = new Lead(Id = Trigger.new[i].Id);
            l.Duplicate_Status__c = 'Potential Duplicate';
            toUpdate.add(l);
        }
    }
    if (!toUpdate.isEmpty()) {
        update toUpdate;
    }
}

Why not use a Flow trigger alone: Flow duplicate detection using the standard Duplicate Rule action works for record-created flows, but the "Block" action in Flow context also does not prevent the insert if the lead was created via API or Web-to-Lead. The Apex approach gives programmatic control over the response.

Alert-Mode Duplicate Rule + Duplicate Record Set Review Queue

When to use: The org wants to capture all suspected duplicates for human review without blocking any lead creation (e.g., for marketing orgs where every lead must be captured).

How it works:

  1. Set all active Lead Duplicate Rules to "Alert" action (not "Block").
  2. Enable "Report" on the rule — this creates a Duplicate Record Set for every flagged record.
  3. Create a List View or Report on Duplicate Record Sets filtered to Lead object.
  4. Assign a data steward or use a Flow to route high-confidence duplicates (e.g., exact email match) to a dedup queue automatically.
  5. Set up a weekly Duplicate Jobs run (if available in the org's edition) to surface existing duplicates that predate the rule.

Why not Block mode: Block mode prevents data entry through the standard UI but silently fails for Web-to-Lead, API inserts, and Apex DML unless the calling code explicitly checks the AllowSave duplicate rule options. Alert-mode with a review queue captures everything without losing leads.

Decision Guidance

SituationRecommended ApproachReason
CSV import, email present, <50K recordsData Import Wizard with Email match field + pre-normalizationSimple, no-code, sufficient for exact email dedup
CSV import, no reliable email, name+company matching neededExternal pre-matching (Python/Excel) then upload pre-deduped fileDIW does not support multi-field fuzzy matching
Web-to-Lead receiving duplicatesAlert-mode Duplicate Rule + after-insert Apex or Flow for routingBlocking rules do not stop Web-to-Lead inserts
API-based lead integration creating duplicatesAfter-insert Apex trigger calling findDuplicates()Integration code bypasses blocking rules
Leads matching existing Contacts before conversionStandard Lead-to-Contact Matching Rule (Alert mode) + review queueNative cross-object detection, manual or Flow-based resolution
Lead-to-Contact fuzzy dedup at volume (>500K contacts)Third-party tool (DemandTools or Cloudingo)Native rules have performance impact and algorithm limitations at this scale
Web form spam or bot-generated leadsLead Assignment Rule to quarantine queue + Apex blacklist checkDedup rules are not designed for spam filtering
  1. Identify the data entry channel — determine whether leads are coming from CSV file imports, Web-to-Lead forms, API/integration sync, or a combination. Each channel has different dedup capabilities. Document the channel and estimated volume.
  2. Audit existing Duplicate and Matching Rules — check Setup > Duplicate Rules and Matching Rules for the Lead object. Confirm whether the Standard Lead Matching Rule is active, whether any rule is set to Block vs. Alert, and whether the Lead-to-Contact rule is enabled. Note how many of the 5-active-rule slots are consumed.
  3. Design the matching and dedup strategy per channel — for CSV imports, choose the Data Import Wizard dedup field and plan a pre-import normalization step. For Web-to-Lead or API, confirm that Alert mode is configured and design an Apex trigger or Flow to handle post-insert routing. For cross-object matching, determine whether native Lead-to-Contact rules are sufficient or whether third-party tooling is required.
  4. Implement and test in sandbox — deploy matching rule and duplicate rule configuration changes, any Apex trigger, and any Flow. Test with sample records that represent real-world data quality (missing email, name variants, different phone formats). Confirm that Duplicate Record Sets are created as expected for Alert-mode rules.
  5. Execute the import or enable the channel — run the CSV import or activate the Web-to-Lead form. For imports, review the post-import duplicate summary from the wizard and run a SOQL query on DuplicateRecordItem to assess how many flagged records require review.
  6. Work the Duplicate Record Set queue — assign the Duplicate Record Set list view to a data steward. Merge or discard flagged records. For automated programs, configure a scheduled Flow or Apex batch to auto-merge high-confidence duplicates (exact email match) and route low-confidence ones to a queue.
  7. Establish ongoing prevention controls — after the import cleanup is complete, confirm that Duplicate Rules are active and set to Alert or Block as appropriate. Schedule a recurring Duplicate Jobs run (if available) to catch drift. Document the dedup field choice and rule configuration for future imports.

Review Checklist

Salesforce-Specific Gotchas

  1. Blocking Duplicate Rules do not block Web-to-Lead or API inserts — a Duplicate Rule with action "Block" will evaluate and create a Duplicate Record Set but will not prevent the record from being inserted when the lead arrives via Web-to-Lead POST, REST API, or Apex DML. This silently creates duplicates that the admin believes are being blocked.
  2. Data Import Wizard dedup is single-field exact match only — practitioners assume the wizard uses the same fuzzy matching as the Standard Matching Rule. It does not. Choosing "Name" as the match field performs a case-insensitive exact match on the full name string. Missing email addresses result in those records always being inserted as new.
  3. 5 active Duplicate Rules per object is a hard limit — attempting to activate a sixth Duplicate Rule on Lead will fail. Orgs with managed packages that include dedup rules may hit this limit unexpectedly. Audit active rules before designing new ones.

Output Artifacts

ArtifactDescription
Matching rule and duplicate rule configuration specWhich rules to enable, field coverage, action (Alert/Block), and active rule slot inventory
Pre-import normalization script or spreadsheet formulaPython or Excel steps to normalize email, phone, and company before CSV upload
Apex after-insert trigger for post-insert duplicate detectionTrigger calling Datacloud.DuplicateRule.findDuplicates() and setting a status field
Duplicate Record Set list view configList view on DuplicateRecordSet filtered to Lead, for data steward review queue
Post-import SOQL query setQueries on DuplicateRecordItem and Lead to confirm dedup coverage after import

Package files