SfSkills Domains Docs GitHub

Home / apex / long-running-process-orchestration

long-running-process-orchestration

Use when designing or reviewing multi-step Apex orchestration that spans multiple transactions, checkpoints state across async boundaries, or recovers from partial failure in a long-running workflow.

Open the skill folder on GitHub

Long-Running Process Orchestration

Use this skill when a business process must span multiple Apex transactions, cannot complete within a single execution context, and requires state preservation, progress tracking, and recovery from partial failure. The skill covers Queueable chaining with bounded depth, Platform Event state machines, Finalizer-based recovery, and Continuation for async UI callouts.

Before Starting

Gather this context before working on anything in this domain:

Core Concepts

Why Single Transactions Are Insufficient

Salesforce imposes hard transaction limits per execution context: 60 000 ms CPU (asynchronous), 100 SOQL queries, 150 DML statements, 12 MB heap (asynchronous), a maximum of 100 callouts, and a cumulative callout timeout of 120 seconds per transaction.

That 120-second figure is the one to design against, and it is routinely mis-stated as 10 seconds. Ten seconds is the default timeout for a single callout, and it is not a ceiling: HttpRequest.setTimeout() accepts 1–120,000 ms, in both synchronous and asynchronous Apex. So one callout may legitimately run for the full two minutes, or twelve callouts may run for ten seconds each — what you cannot do is exceed 120 seconds summed across every callout in the transaction. A process that must send 20 batched callouts against a slow service, process thousands of records in sequential dependency steps, or wait for external system responses cannot fit in one transaction. Multi-step orchestration is the platform-endorsed pattern for these cases: each step runs in its own transaction with a full allocation of governor limits, and state advances through controlled async handoffs.

Queueable Chaining As A Step Sequencer

The most direct Apex orchestration mechanism is chaining Queueable jobs. Each Queueable's execute() method does its work and enqueues the next job in the sequence, passing forward a state object through the constructor. Key constraints that shape this pattern:

The Finalizer Interface For Cross-Step Recovery

The System.Finalizer interface runs in a guaranteed separate transaction after its parent Queueable completes — regardless of whether the parent succeeded or threw an uncaught exception. This is the only safe mechanism for:

The FinalizerContext provides getResult() (SUCCESS or UNHANDLED_EXCEPTION), getException(), and getJobId(). A Finalizer may enqueue one additional Queueable. System.attachFinalizer() must be called early in the parent's execute() before any code that could throw (Apex Developer Guide — Transaction Finalizers).

Platform Event State Machine

When a process needs to cross system boundaries, tolerate external latency, or decouple step producers from step consumers, a Platform Event state machine is more flexible than direct Queueable chaining. The pattern:

  1. Each step completion publishes a StepCompleted__e (or domain-specific) Platform Event with the next-step identifier and carry-forward state as payload fields.
  2. An Apex after insert trigger on that event type reads the next-step field and dispatches a Queueable for the next step.
  3. Progress and replay position are tracked on a Custom Object record (the "process instance"), which also serves as the operational dashboard.

Platform Events decouple publish time from subscribe time, survive transaction rollbacks (events published in a rolled-back DML transaction are still delivered — this is a feature, not a bug), and allow external systems to participate as steps in the process (Platform Events Developer Guide).

Progress Tracking And Observability

A long-running process is operationally incomplete without a way to observe where each instance is and why it stopped. Checkpoint patterns:

Step start/end timestamps enable latency monitoring per step, which surfaces performance regressions before they cause timeouts.

Continuation For Long-Running LWC Callouts

When the long-running operation is a callout initiated from Lightning UI, the Continuation class provides an async callout mechanism that does not block the user session. The controller initiates the callout by returning a Continuation object; Salesforce processes the callout asynchronously (up to 120 seconds) and invokes a callback method with the response. This is distinct from Queueable: Continuation is UI-initiated, synchronous from the user's perspective, and limited to callout contexts in LWC/Aura/Visualforce controllers (Apex Developer Guide — Continuation).

Common Patterns

Bounded Queueable Chain With State Object

When to use: A sequential process of 3–10 steps where each step is a distinct DML or callout operation, steps are ordered and have no parallelism, and the total chain depth fits within platform limits.

How it works:

  1. Define a serializable ProcessState inner class or standalone class carrying: currentStep (integer or enum), recordId, retryCount, and any payload fields needed by subsequent steps.
  2. Each Queueable receives a ProcessState instance, executes its step, updates the Custom Object checkpoint record, then evaluates whether to chain:
public class StepOneQueueable implements Queueable {
    private final ProcessState state;

    public StepOneQueueable(ProcessState state) {
        this.state = state;
    }

    public void execute(QueueableContext ctx) {
        System.attachFinalizer(new StepFinalizer(state));

        // Do step work
        doStepOneWork(state.recordId);

        // Update checkpoint
        updateCheckpoint(state.recordId, 'StepOne', 'Complete');

        // Chain to next step
        state.currentStep = 2;
        AsyncOptions opts = new AsyncOptions();
        opts.MaximumQueueableStackDepth = 5;
        System.enqueueJob(new StepTwoQueueable(state), opts);
    }
}
  1. The Finalizer handles failure — it increments retryCount on the state object and re-enqueues the same step if retries remain, or marks the process instance as failed.

Why not the alternative: Placing all steps in a single Queueable with a switch statement runs the risk of hitting governor limits mid-step with no safe boundary. Separate Queueables give each step its own full transaction allocation.

Platform Event State Machine With Process Instance Record

When to use: The process involves external system responses between steps, steps may run minutes or hours apart, or you need strict decoupling between step producers and consumers.

How it works:

  1. Create a Process Instance Custom Object (OrchestrationRun__c) with fields: Status__c (picklist: Pending/Running/Complete/Failed), CurrentStep__c (text), RetryCount__c, LastError__c.
  2. When launching the process, insert the record with Status__c = 'Running' and CurrentStep__c = 'StepOne', then publish a StepAdvance__e platform event with RunId__c = record.Id and Step__c = 'StepOne'.
  3. The platform event trigger reads Step__c and enqueues the appropriate Queueable:
trigger StepAdvanceTrigger on StepAdvance__e (after insert) {
    for (StepAdvance__e evt : Trigger.new) {
        if (evt.Step__c == 'StepOne') {
            System.enqueueJob(new StepOneWorker(evt.RunId__c));
        } else if (evt.Step__c == 'StepTwo') {
            System.enqueueJob(new StepTwoWorker(evt.RunId__c));
        }
        // Additional steps...
    }
}
  1. At the end of each worker, it updates OrchestrationRun__c and publishes the next StepAdvance__e.
  2. On failure, the Finalizer updates Status__c = 'Failed' and LastError__c.

Why not the alternative: Queueable chaining alone cannot resume after an external system delay of hours without consuming a Queueable slot waiting for the response. Platform Events decouple the step trigger from the response and survive org restarts.

Hybrid: Queueable Chain Within Steps, Platform Events Between Phases

When to use: The orchestration has distinct phases (e.g., Validation, Processing, Notification) where each phase consists of sequential sub-steps, and phases are separated by external events or significant time gaps.

How it works: Use a Queueable chain for the sub-steps within a phase; use a Platform Event to signal phase completion and trigger the next phase. The Custom Object record tracks both phase and sub-step. This hybrid keeps chain depth low per phase while maintaining decoupled phase transitions.

Decision Guidance

SituationRecommended ApproachReason
3–7 sequential steps, no external waits, completes in minutesBounded Queueable chainSimpler, no event infrastructure needed
Steps separated by external system responses or human actionsPlatform Event state machineDecouples step timing from Apex transaction lifecycle
Fan-out required (one step spawns N parallel workers)Platform Events or BatchQueueable allows only one child per execution
Step failure should auto-retry up to N timesFinalizer-based retryOnly mechanism that handles all failure modes including system-level exceptions
Process is UI-initiated and involves a long calloutContinuationDesigned for async LWC/Aura callouts up to 120s
Step volume exceeds Queueable chain depth limitsBatch Apex for heavy step, Queueable to orchestrate transitionsBatch handles large data volume; Queueable manages step transitions
Operations team needs live process visibilityCustom Object checkpoint + Platform Event broadcastEnables query-based dashboards and external monitoring
Process has 10+ steps or irregular branchingPlatform Event state machine with dispatch tableCleaner than deeply nested Queueable chain logic

Step-by-step instructions for an AI agent or practitioner activating this skill:

  1. Gather context — identify all steps, their sequencing rules, governor-limit risks per step, callout needs, and failure recovery expectations. Confirm whether the process is purely Apex or has external participants.
  2. Choose the primary pattern — use the Decision Guidance table above. For simple sequential processes, start with Queueable chaining; for externally-triggered or time-separated steps, use the Platform Event state machine.
  3. Design the state model — define the Custom Object (or state class) that will carry process instance data across transactions. Include step identifier, retry count, timestamps, and error message fields.
  4. Implement step boundaries — each step should be a separate Queueable class. Attach a Finalizer in every Queueable that has side effects. Pass state only through constructor fields — not static variables.
  5. Implement the checkpoint mechanism — update the Custom Object record at the start and end of each step. Publish a progress Platform Event if real-time dashboards are required.
  6. Validate — run the checker script, verify governor limit exposure per step in test scenarios, confirm Finalizer coverage, and ensure chain depth respects MaximumQueueableStackDepth.
  7. Review against checklist — confirm all items in the Review Checklist are satisfied before marking implementation complete.

Review Checklist

Salesforce-Specific Gotchas

  1. MaximumQueueableStackDepth does not propagate — setting the option on the first enqueueJob call does not carry forward to child jobs. Every subsequent enqueueJob in the chain must re-set it explicitly, or the depth guard is silently absent on all jobs beyond the first.
  2. Platform Events published in a rolled-back DML transaction are still delivered — this is by design. A Queueable that fails mid-transaction but already published a step-advance event will advance the state machine while leaving the DML changes rolled back, causing state inconsistency. Publish step-advance events only after confirming the DML in that step succeeded.
  3. Finalizer failures are silent unless monitored — the Finalizer runs in a separate transaction with its own governor limits. If the Finalizer itself hits a limit or throws, it fails without surfacing to the original job's error context. Log from the Finalizer to a Custom Object or named credential endpoint so these failures are observable.
  4. System.AsyncInfo.getCurrentQueueableStackDepth() returns 0 in synchronous test context — test methods that call Queueables synchronously via Test.stopTest() will always see a depth of 0, so depth-guard branches that skip enqueuing will never be exercised in unit tests. Add an explicit integration test or set a low MaximumQueueableStackDepth in test context to cover boundary logic.
  5. Continuation is LWC/Aura/VF only — attempting to create a Continuation from a trigger, Queueable, Batch class, or @AuraEnabled method that is not a controller action throws a runtime exception. The async callout pattern for non-UI contexts is always Queueable with Database.AllowsCallouts.

Output Artifacts

ArtifactDescription
Orchestration pattern recommendationDecision on Queueable chain vs Platform Event state machine vs hybrid, with rationale
State model designCustom Object schema and state class fields required for this process
Step scaffoldQueueable class skeletons with Finalizer, state passing, and checkpoint update
Platform Event trigger dispatch tableThin trigger routing step-advance events to the correct Queueable worker
Review findingsAssessment of an existing multi-step async implementation against the review checklist

Dependencies

Package files