SfSkills Domains Docs GitHub

Home / data / ai-training-data-preparation

ai-training-data-preparation

Judging whether Salesforce data is fit to train an Einstein model: feature field selection, outcome definition, data quality and fill-rate thresholds, leakage detection. Trigger keywords: Einstein Discovery data requirements, training data for Einstein, ML feature engineering, Einstein Prediction Builder data prep, AI model training data. NOT for authoring the story in CRM Analytics Studio — use admin/einstein-discovery-setup. NOT for setting up the prediction itself — use agentforce/einstein-prediction-builder.

Open the skill folder on GitHub

AI Training Data Preparation

Use this skill when setting up data for a Salesforce predictive model: an Einstein Studio predictive model built in Model Builder on Data Cloud data, an Einstein Discovery story in CRM Analytics, or an Einstein Prediction Builder prediction on Salesforce objects. It covers outcome design, fill rates, leakage, cardinality, and class balance, and ships a checker that audits a training extract before anything trains.

Grounding note (2026-10-03): the numeric requirements that a fetched official source states belong to Einstein Studio predictive models (Data Cloud guide, "Create Predictive AI Models From Scratch"). The Einstein Discovery and Prediction Builder thresholds in earlier versions of this skill come from Salesforce Help pages, which do not fetch; they are kept below with UNVERIFIED markers.

Before Starting

Gather this context before working on anything in this domain:

Questions to Ask Before Configuring

Each question traces to a gotcha in references/gotchas.md.

QuestionWhy it mattersWhat a good answer addsWhat proper configuration adds over just doing it
"What exactly is the outcome, and when in the record's life is it known?"Fields set at or after the outcome leak it, and a model that looks too accurate is the symptom (Gotcha 1)An outcome definition plus a "known before" date for every predictorAccuracy in training is the accuracy users get on open records
"How many rows and columns will the training data have?"Einstein Studio predictive models need 400 to 20 million rows and 3 to 50 columns (Gotcha 2)Counts from the actual extract, not the objectThe model trains on the first attempt instead of failing setup
"Which predictors are free-text or ID-like with many distinct values?"Einstein Studio supports up to 100 categories per variable, and high-cardinality fields rarely help (Gotcha 3)A list of fields to bucket, drop, or consolidatePredictors the model can use, and explanations people can read
"What fills the gaps in sparse fields, and why are they sparse?"Nulls become an Unspecified category, and dropping too many rows hides real patterns (Gotcha 4)A per-field rule: impute, bucket, or excludeSparse fields help or are removed on purpose, not by accident
"Is the type, data source, and goal final?"After an Einstein Studio model is created, Choose Type, Select Data, and Set Goal can't be edited (Gotcha 5)Sign-off on type, source, and goal before Save & TrainNo rebuild because the outcome direction was wrong
"How rare is the positive outcome?"A rare outcome lets a model look accurate while predicting the majority class (Gotcha 6)Class counts and a threshold planThe business sees useful recall, not a flat "no" for everyone

Core Concepts

Data Requirements by Product

Einstein Studio predictive models (grounded). The data source must have at least 400 and at most 20 million rows, and at least 3 columns (1 outcome plus 2 others) and at most 50. Supported use cases are regression (continuous numbers) and binary classification (two-value text outcomes). "Created models in Einstein Studio support up to 100 categories per variable"; categories with fewer than 25 observations can be consolidated into Other, and null values go into a category called Unspecified.

Einstein Discovery. Earlier versions of this skill said Einstein Discovery requires a minimum of 400 rows where the outcome is populated, and that fields with a fill rate below about 70% are silently dropped from feature selection. UNVERIFIED (2026-10-03): both figures come from Salesforce Help and were not found in a fetched source; the Analytics Platform Setup Guide points to "Einstein Discovery Limits" without listing them. Treat 70% as a review threshold, not a platform rule.

Einstein Prediction Builder. Earlier versions said EPB requires at least 200 records where the outcome is true and 200 where it is false, with 400 total recommended. UNVERIFIED (2026-10-03): not found in a fetched source. The Metadata API types behind predictions are MLDataDefinition (includedFields, excludedFields, trainingFilter, scoringFilter, segmentFilter; entityDeveloperName and type can't be updated after creation) and MLPredictionDefinition (predictionField, pushbackField, status Enabled, Disabled, or Draft).

Outcome Field Design and Leakage

"Leakage occurs when the data used to train your model includes one or more variables that contain the information that you're trying to predict. This can result in models that are extremely accurate when, in actuality, they are problematic." The outcome must be unknown at the time the predictors were captured. Fields created or populated as a result of the outcome (a "Closed Won Reason" filled at close) are proxies. For Einstein Studio models, an accuracy rating of "too high means the model is perfect or nearly perfect, which indicates potential data leakage or overfitting."

Earlier versions said Einstein Discovery flags predictors with over 30% correlation to the outcome as leakage candidates. UNVERIFIED (2026-10-03): not found in a fetched source. The Data Cloud glossary does say correlation "is quantified as a percentage" and is "not causation," so a high correlation is a prompt to investigate, not proof.

Choosing Between the Products

Common Patterns

Pattern: Outcome and Predictor Audit Before Training

When to use: Before creating any model, story, or prediction.

How it works:

  1. Extract the candidate training rows to CSV with the outcome and every candidate predictor.
  2. Run python3 scripts/check_ai_training_data_preparation.py --manifest-dir <folder> --outcome <field> --type binary|regression.
  3. Resolve every ERROR (row and column limits, outcome shape, predictors identical to the outcome).
  4. Review every WARN: post-outcome field names, high cardinality, low fill, constant columns, rare positive class.
  5. Record which fields were dropped and why.

Why not skip this: A leaking field or an unusable categorical can make a model look excellent in training and useless on open records.

Pattern: Feature Field Selection Matrix

When to use: When curating which fields to include as predictors.

How it works: Build a field inventory with: fill rate, when the field is populated relative to the outcome, distinct-value count, and correlation-to-outcome suspicion. Exclude post-outcome fields, formulas derived from the outcome, ID-like fields, and fields you cannot fill. Collapse collinear fields ("customers who live in the city of Tampa also live in the state of Florida") to one.

Decision Guidance

SituationRecommended ApproachReason
Regression or binary outcome on Data Cloud dataEinstein Studio predictive modelGrounded requirements: 400 to 20 million rows, 3 to 50 columns
Need stories, what-if analysis in CRM AnalyticsEinstein DiscoveryRequires a CRM Analytics license and a dataset
Prediction on a Salesforce object configured in SetupEinstein Prediction BuilderData definition and prediction deploy as MLDataDefinition and MLPredictionDefinition
Fewer than 400 training rowsFix data volume firstBelow the Einstein Studio minimum; results would not train
Categorical predictor with more than 100 valuesBucket or consolidate firstEinstein Studio supports up to 100 categories per variable
Model accuracy rated "too high"Hunt for leakage before activatingThe Data Cloud guide names leakage or overfitting as the cause
  1. Confirm product scope with the Decision Guidance table, based on outcome type, data location, and license.
  2. Define the outcome and the date each predictor becomes known; list post-outcome fields to exclude.
  3. Extract and audit the training data to CSV and run scripts/check_ai_training_data_preparation.py with --outcome and --type; fix every ERROR.
  4. Remediate fill-rate, cardinality, and collinearity warnings: impute, bucket, consolidate, or exclude, and write down each decision.
  5. Configure training: for Einstein Studio, confirm type, data, and goal before Save & Train because they can't be edited later; for Prediction Builder, set includedFields and excludedFields explicitly and deploy with status Draft (references/metadata-examples.md).
  6. Validate the model: check the accuracy rating (performant, too low, too high), top predictors, and confirm no excluded or leaking field appears among them.

Review Checklist

Run through these before marking work in this area complete:

Salesforce-Specific Gotchas

  1. Too-high accuracy is a warning: Einstein Studio rates accuracy as performant, too low, or too high; "too high" indicates potential leakage or overfitting.
  2. Setup choices freeze: After an Einstein Studio model is created, Choose Type, Select Data, and Set Goal can't be edited.
  3. Einstein Discovery reads from its story source: Stories read an AnalyticsDataset, LiveDataset, or Report source, not the object directly. A field added to the object after the dataflow or recipe was built is not in a dataset source until the dataflow or recipe includes it and runs.

Output Artifacts

ArtifactDescription
Training extract auditChecker output: row and column limits, outcome shape, leakage, cardinality, fill, balance
Outcome leakage checklistDocuments whether each high-correlation field was populated before or after the predicted event
Product decision recordDocuments which product was selected and why
Prediction Builder metadataMLDataDefinition and MLPredictionDefinition files with explicit included and excluded fields

Package files