Orchestrate Parallel Sub-Workflows and Wait for Completion
detail.loadingPreview
This n8n workflow pattern demonstrates how to trigger multiple sub-workflows asynchronously and then pseudo-synchronously wait for all of them to complete before proceeding. It's ideal for complex, multi-step processes that can be broken down into independent tasks.
About This Workflow
This sophisticated n8n workflow is designed to orchestrate parallel execution of sub-workflows. It begins by simulating a set of tasks that need to be processed independently. The core functionality lies in initiating multiple instances of a sub-workflow asynchronously via webhooks. Once triggered, the parent workflow enters a waiting state, utilizing a webhook callback mechanism. Each sub-workflow, upon completion, sends a callback notification to the parent, signaling its completion and providing an identifier. The parent workflow meticulously tracks these callbacks, updating a 'finishedSet'. It then checks if all initiated sub-workflows have reported back. This pattern effectively simulates synchronous execution of parallel tasks, allowing for complex process management and robust error handling.
Key Features
- Asynchronous Sub-Workflow Initiation: Trigger multiple sub-workflows concurrently without blocking the main workflow.
- Pseudo-Synchronous Waiting: Effectively pause the main workflow until all parallel sub-tasks report completion.
- Webhook-Based Callback: Utilize webhooks for reliable communication between sub-workflows and the parent workflow.
- Dynamic Task Tracking: Dynamically manage and track the completion status of each parallel task.
- Error Handling and Retries: Built-in mechanisms for retrying failed callback requests to ensure workflow resilience.
How To Use
- Set up the Sub-Workflow: Create a separate n8n workflow that will act as the sub-workflow. This sub-workflow should be designed to be triggered by a webhook and should have a webhook endpoint configured to send a callback to the parent workflow upon completion.
- Configure the Parent Workflow:
- Use a
manualTriggeror your preferred trigger to start the parent workflow. - Employ a
codenode (e.g., 'Simulate Multi-Item for Parallel Processing') to define the items/tasks that will be processed in parallel. Each item should have a unique identifier (e.g.,requestId). - Use a
splitInBatchesnode (or equivalent) to prepare items for parallel execution (though in this example, thecodenode effectively provides the items). - For each item, use an
httpRequestnode (e.g., 'Start Sub-Workflow via Webhook') to trigger the sub-workflow, passing the item's identifier and the parent workflow's resume URL (using$execution.resumeUrl) in the headers. - Initialize a
finishedSetarray in asetnode for tracking. - Implement an
ifnode to check if the number of finished items ($json.finishedSet.length) equals the total number of initiated items ($($('Simulate Multi-Item for Parallel Processing').all()).length). - If not all items are finished, use a
waitnode (e.g., 'Webhook Callback Wait') configured toresume: 'webhook'to wait for callback notifications. - When a callback is received, use a
codenode (e.g., 'Update finishedSet') to add the finished item's ID to thefinishedSet. - If all items are finished, proceed with the rest of your parent workflow. Use a
respondToWebhooknode in the sub-workflow to send the callback back to the parent.
- Use a
Apps Used
Workflow JSON
{
"id": "8d4e7618-921b-494a-9f00-ff01636cef23",
"name": "Orchestrate Parallel Sub-Workflows and Wait for Completion",
"nodes": 13,
"category": "DevOps",
"status": "active",
"version": "1.0.0"
}Note: This is a sample preview. The full workflow JSON contains node configurations, credentials placeholders, and execution logic.
Get This Workflow
ID: 8d4e7618-921b...
About the Author
SaaS_Connector
Integration Guru
Connecting CRM, Notion, and Slack to automate your life.
Statistics
Related Workflows
Discover more workflows you might like
Automated PR Merged QA Notifications
Streamline your QA process with this automated workflow that notifies your team upon successful Pull Request merges. Leverage AI and vector stores to enrich notifications and ensure seamless integration into your development pipeline.
Visualize Your n8n Workflows: Interactive Dashboard with Mermaid.js
Gain unparalleled visibility into your n8n automation landscape. This workflow transforms your n8n instance into a dynamic, interactive dashboard, leveraging Mermaid.js to visualize all your workflows in one accessible place.
Effortless Bug Reporting: Slack Slash Command to Linear Issue
Streamline your bug reporting process by instantly creating Linear issues directly from Slack using a simple slash command. This workflow enhances team collaboration by providing immediate feedback and a structured approach to logging defects, saving valuable time for development and QA teams.