Skip to main content

Configure the Batch Engine

Use the Batch Engine to process Payload Jobs asynchronously in controlled batches.

Configure the Batch Engine

Use the Batch Engine when Payload work should be queued and processed in batches instead of running immediately in the current transaction.

This is useful when the Flow, user action, or process can hand off the work and review the resulting Jobs later. The caller creates the work, Payloads records the Jobs, and the Batch Engine processes those Jobs in controlled groups.

The main benefit is separation. The process that decides work should happen does not also need to wait for every external callout, mapping, and target write to finish.

When to use it

Use Batch Engine when you need to:

  • process a larger number of Payload Jobs asynchronously

  • control how many Jobs are handled in each batch execution

  • avoid making the calling Flow wait for each Payload to finish

  • separate the process that creates work from the process that executes work

  • monitor queued and completed batch work from one place

For Flow setup, see Use Payloads in Flow.

For small interactive actions, immediate or queueable execution may be simpler. Batch Engine is most useful when you expect queued work, grouped processing, or a need to monitor the queue separately from the process that created it.

How Batch Size works

Batch Size controls how many Jobs Payloads should process in each batch execution.

A smaller Batch Size gives each execution less work to do. A larger Batch Size processes more Jobs at once, but each batch execution has more work to complete inside Salesforce limits.

Start with a conservative Batch Size, then adjust after reviewing Job metrics and batch behaviour. If Jobs are complex, perform callouts, run Data Queries, or write several Data Target records, a smaller batch may be more reliable than pushing more work into each execution.

What happens at runtime

When a Payload runs through the Batch Engine, Payloads creates Jobs and marks them as batch work.

Those Jobs stay visible in the Jobs tab. A Job can remain New while it is waiting to be processed. After the Batch Engine processes it, the Job should move to Completed or Failed.

Use the Job record to review the same runtime details you would check for any other Payload run: inputs, request data, response data, target output, error message, and metrics. The Batch Engine controls when the Job is processed; the Job still tells you what happened during the run.

Monitor batch work

Open the Batch Engine tab to review current and recent batch activity.

Use it to check whether batch work is waiting, running, or completed. If Jobs stay New longer than expected, check the Batch Engine tab before changing the Payload configuration. The issue may be queue processing rather than the Payload definition itself.

The Batch Engine tab showing service status, thread count, queued jobs, processing jobs, and current workloads.

The Batch Engine tab is the queue-level view for batch processing, separate from individual Job records.

The Batch Engine tab is the operational view for batch processing. Use it to answer three questions:

  • Is there batch work waiting to be processed?

  • Is the Batch Engine currently running?

  • Did recent batch work complete or fail?

Start here when a Flow used Batch Engine firing mode but the resulting Jobs have not moved from New to Completed or Failed.

Review batch Job cards

Batch Job cards group related batch activity so you can see what Payload work was queued and how it moved through processing.

Open a batch entry when you need more detail about the Jobs it processed. Then open the individual Job records for request, response, target output, error, and metrics detail.

Use the Batch Engine tab for queue-level triage. Use Job records for run-level diagnosis. That distinction helps avoid changing a Payload when the real issue is that queued work has not been processed yet.

When Jobs stay New

A Job in New status is not automatically wrong. It may simply be waiting for the Batch Engine to process it.

If Jobs stay New longer than expected, check:

  • the Payload was started with Batch Engine firing mode

  • Batch Size is reasonable for the amount of work

  • the Batch Engine tab shows active or recent processing

  • Salesforce scheduled, queueable, or batch Apex limits are not preventing work from running

  • failed batch work has not left a group of Jobs unprocessed

Do not keep re-running the same Payload until you understand whether the original Jobs are queued or stuck. Re-running can create duplicate external requests and make the queue harder to reason about.

What to check

After testing a batch run, confirm:

  • the expected Jobs were created

  • the Jobs are marked as batch work

  • Batch Size matches the volume and complexity of the Payload work

  • completed Jobs have the expected request, response, and target output

  • failed Jobs have clear error messages

  • metrics do not show unexpected query, DML, CPU, heap, or callout pressure

For Job-level review, see Understand Job records.

Did this answer your question?