SfSkills Domains Docs GitHub

Home / devops / continuous-integration-testing

continuous-integration-testing

Use when configuring Apex test execution in CI/CD pipelines, choosing test levels for deployments, parsing test results, or troubleshooting code coverage in automated builds. Triggers: 'RunLocalTests', 'RunSpecifiedTests', 'sf project deploy', 'code coverage CI', 'JUnit test results'. NOT for standing up a GitHub Actions workflow — use devops/github-actions-for-salesforce. NOT for Apex test class design — use apex/test-class-standards. NOT for LWC Jest testing — use lwc/lwc-testing.

Open the skill folder on GitHub

Continuous Integration Testing

Use this skill when Apex tests must run as part of an automated CI/CD pipeline rather than manually through the Developer Console or Setup UI. The objective is a pipeline configuration that executes the right test level, collects accurate coverage numbers, produces machine-readable results, and fails the build when coverage or test outcomes do not meet deployment thresholds.

Before Starting

Gather this context before working on anything in this domain:

Core Concepts

Test Levels Control What Runs

The sf project deploy start and sf apex run test commands accept a --test-level flag with four values:

LevelBehavior
NoTestRunSkip tests entirely. Only valid for non-production targets or metadata-only changes.
RunSpecifiedTestsRun only the named test classes. Salesforce enforces 75% coverage per class and per trigger in the deployment package — stricter than org-wide.
RunLocalTestsRun all tests in the org except managed package tests. Coverage is calculated org-wide at 75%.
RunAllTestsInOrgRun everything including managed package tests. Rarely appropriate in CI because managed package test failures block your deployment.

The critical nuance: RunSpecifiedTests applies coverage requirements per-component in the package, not org-wide. A single under-covered class in your changeset will fail deployment even if the org is at 90% overall.

Coverage Collection Is Not Coverage Enforcement

The --code-coverage flag on sf apex run test instructs the platform to collect coverage data and include it in the result payload. It does not enforce any threshold. Your pipeline script must parse the coverage result and fail the build if coverage is below the team standard. Salesforce only enforces the 75% threshold during actual deployment, not during standalone test runs.

JUnit Output for CI Report Ingestion

Most CI platforms can parse JUnit XML to display test results in their UI. Use --result-format junit on sf apex run test to produce this format. The output file can be declared as a test artifact in GitHub Actions, GitLab CI, or Jenkins to get per-test pass/fail reporting in the build dashboard.

--result-format accepts four values, not just junit: human (the default terminal-readable table), tap (Test Anything Protocol), junit (the XML most dashboards ingest), and json (the full machine-readable payload you parse in a coverage-gate script). Pick junit for dashboard artifacts and json when the pipeline script needs to read coverage numbers programmatically.

For per-test-method coverage rather than only aggregate per-class numbers, add --detailed-coverage alongside --code-coverage. This surfaces which specific test method exercised each class, which is useful when diagnosing why a class sits just under the gate — but it does not change the enforcement behavior, only the level of detail in the report.

Asynchronous Test Execution and Polling

By default, sf apex run test with --synchronous runs tests in a single synchronous request, which is limited to a smaller set of tests. Without --synchronous, tests run asynchronously and the CLI polls for completion when --wait is specified. A known platform behavior: combining --code-coverage with --wait on asynchronous runs can return 0% coverage. The workaround is to let the run complete, then retrieve results separately with sf apex get test --test-run-id <id> --code-coverage.

The UI Reporting Surfaces the Pipeline Replaces

Before wiring coverage into CI, know the two in-org UI surfaces the same numbers come from — they are where developers cross-check a pipeline result or investigate a coverage drop by hand:

These UI surfaces do not enforce anything beyond the platform's own 75% deploy gate; they are for inspection. The pipeline is still where enforcement lives.

Common Patterns

Pattern: Two-Stage Deploy with Validation Then Quick-Deploy

When to use: Production deployments where you want a CI validation gate before the actual deployment window.

How it works:

  1. Run sf project deploy validate --test-level RunLocalTests --target-org prod during CI. This runs all local tests and validates coverage without actually deploying.
  2. Capture the validation ID from the output.
  3. During the deployment window, run sf project deploy quick --job-id <validation-id> --target-org prod to deploy without re-running tests.

Why not the alternative: Running sf project deploy start directly means tests and deployment happen atomically. If CI validates in the morning and deployment is approved in the afternoon, validation-then-quick-deploy avoids re-running the full test suite.

Pattern: RunSpecifiedTests with Explicit Coverage Gate

When to use: Large orgs where RunLocalTests takes too long (30+ minutes) and the team wants fast CI feedback.

How it works:

  1. Determine which test classes cover the changed Apex classes. Many teams maintain a mapping file or naming convention (MyClass -> MyClassTest).
  2. Run sf project deploy start --test-level RunSpecifiedTests --tests MyClassTest,MyServiceTest --target-org target.
  3. Parse the deployment result for per-class coverage. Fail the build if any component in the package is below 75%.

Why not the alternative: RunLocalTests in a large org can take over an hour. RunSpecifiedTests gives targeted feedback in minutes but requires the team to maintain accurate test-to-class mappings.

Pattern: Standalone Test Run with Separate Coverage Retrieval

When to use: When the pipeline needs test results and coverage data but is not performing a deployment (e.g., a PR validation build against a sandbox).

How it works:

  1. Run sf apex run test --test-level RunLocalTests --result-format junit --output-dir ./test-results --wait 30 --target-org sandbox.
  2. Run sf apex get test --test-run-id <id> --code-coverage --result-format json to retrieve coverage separately.
  3. Parse the JSON coverage output and enforce the team threshold.

Why not the alternative: Combining --code-coverage with --wait on the initial run can return 0% coverage due to a known CLI behavior. Separating the retrieval step avoids this.

Decision Guidance

SituationRecommended Test LevelReason
Production deployment, no managed packagesRunLocalTestsCovers all local code, enforces 75% org-wide
Production deployment, managed packages presentRunLocalTestsAvoids failures from managed package tests you cannot control
Fast PR validation, known test mappingRunSpecifiedTestsMinutes instead of hours; per-class 75% is still enforced
Sandbox validation, exploratoryRunLocalTestsBroad coverage check without deployment risk
Metadata-only change (labels, layouts)NoTestRunNo Apex involved; tests add no value
Full regression before releaseRunAllTestsInOrgOnly when you explicitly want to verify managed package compatibility

Step-by-step instructions for configuring CI testing in a Salesforce pipeline:

  1. Authenticate the CI runner -- configure JWT-based or auth-URL-based authentication so the pipeline can connect to the target org without interactive login. Store the connected app consumer key and server key as CI secrets.
  2. Choose the test level -- use the decision table above. Default to RunLocalTests for production, RunSpecifiedTests for PR builds when test mappings exist, and NoTestRun for metadata-only changesets.
  3. Configure the deploy or test command -- use sf project deploy validate for production validation or sf apex run test for sandbox-only test runs. Always include --wait with a timeout (e.g., --wait 60) so the pipeline blocks until tests finish.
  4. Capture results in JUnit format -- add --result-format junit --output-dir ./test-results and publish the output directory as a CI artifact. Most CI platforms auto-detect JUnit XML.
  5. Retrieve coverage separately if needed -- if using sf apex run test with --code-coverage, verify coverage numbers are non-zero. If they report 0%, fall back to sf apex get test --test-run-id <id> --code-coverage as a second step.
  6. Enforce coverage thresholds in script -- parse the coverage JSON and fail the build if org-wide or per-class coverage falls below the team-defined minimum. Do not rely solely on Salesforce's 75% deployment gate.
  7. Report results -- publish JUnit artifacts, log coverage percentages, and surface failures clearly in the CI dashboard.

Review Checklist

Run through these before marking work in this area complete:

Salesforce-Specific Gotchas

Non-obvious platform behaviors that cause real production problems:

  1. --code-coverage with --wait returns 0% -- when running asynchronous tests, combining these flags can result in coverage data showing 0% even when tests pass. The platform has not finished aggregating coverage by the time the CLI reads the result. Fix: retrieve coverage separately using sf apex get test --test-run-id <id> --code-coverage.
  2. RunSpecifiedTests enforces per-class coverage, not org-wide -- unlike RunLocalTests, which applies the 75% threshold org-wide, RunSpecifiedTests requires 75% coverage on every individual class and trigger in the deployment package. A single under-covered class blocks the entire deployment.
  3. Parallel test execution is opt-in per class -- by default, Salesforce runs test classes serially in a test run. The @isTest(isParallel=true) annotation opts a class into parallel execution, which can reduce total run time but introduces risk if tests share mutable state through SeeAllData or poorly isolated setup.
  4. Developer Console coverage highlighting goes stale on edit -- the line-level covered/uncovered highlighting in the Tests tab dims the moment you edit the class, signaling the coverage is no longer valid. A developer who reads dimmed highlighting as "still covered" is looking at numbers from before their change. Rerun the tests before trusting it.
  5. Setup test history is retained only 30 days -- results on the Application Test Execution / View Test History page disappear 30 days after they finish (or sooner if cleared). Do not treat it as a durable coverage-trend record — capture JUnit/JSON artifacts in the pipeline if you need history.

Output Artifacts

ArtifactDescription
CI pipeline configurationYAML or Jenkinsfile with correct sf CLI commands, test levels, and coverage gates
JUnit XML test resultsMachine-readable test output for CI dashboard integration
Coverage enforcement scriptShell or Python script that parses coverage JSON and exits non-zero below threshold

Package files