Batch processing patterns for workflow automation at scale

This guide explains how batch processing in workflow orchestration differs from stream processing and presents three patterns—chunking, rate limiting, and per-item error handling with retry—for handling large loads without hitting memory limits or API quotas. It describes how n8n's Split Out, Aggregate, Loop Over Items, HTTP Request, Wait, and Remove Duplicates nodes support these patterns, and notes n8n was tested with up to 220 workflow executions per second on a single c5a.large instance running a two-node workflow. It also covers idempotency, checkpointing, and choosing batch sizes, plus sub-workflows and queue mode for scaling.