Articles

Implementation

How to Run a Recoverable n8n AI Workflow with Queue Mode, Error Routing, and Safe PostgreSQL Writes

This guide connects n8n queue-mode execution to a stable operation identity, atomic PostgreSQL writes and a recovery procedure that operators can verify safely.

October 14, 202621 min read
Rows of drive bays in a server rack with green status lights.
Photo by Domaintechnik on Unsplash

When a worker stops beside the database write

An operations team receives a request through an n8n workflow. The workflow assigns the request to an AI step, structures the result and writes an approved business effect to PostgreSQL. Production executions run on queue workers rather than in the main n8n process.

One worker stops near the database write. The execution record may show where processing reached, but the operator still has a more important question: did PostgreSQL commit the operation before the worker disappeared?

Retrying immediately could be harmless, or it could create a duplicate effect. Treating the execution as lost could also leave a committed record without the follow-up handling it needs. The operator requires a stable identity that connects the incoming request, saved n8n execution, PostgreSQL row and exception process.

The practical answer is to combine:

  • queue-mode execution with consistent process configuration;

  • retained failed-execution evidence;

  • a stable operation ID created outside the AI output;

  • a PostgreSQL unique constraint and atomic write boundary;

  • an error route that supplies an execution reference;

  • bounded retries only before a write is known to have started;

  • controlled failure scenarios; and

  • a recovery procedure that checks the database before reprocessing.

This is a setup for an existing self-hosted n8n environment. It does not cover fresh installation, infrastructure provisioning or a complete high-availability design.

Confirm the setup and limits before changing anything

Queue mode introduces multiple n8n processes and a message broker. The main process, workers, Redis and n8n persistence must already exist or be available to the team responsible for the deployment. The operator also needs access to change workflow settings, restart affected processes and test failures without exposing production records.

Use a separate PostgreSQL credential for the downstream business write. Its permissions should match the chosen SQL rather than granting broad access. PostgreSQL requires INSERT for inserts, additional UPDATE privileges when ON CONFLICT DO UPDATE is used, and SELECT privileges for values read by the conflict action or RETURNING clause. The exact implications are documented in PostgreSQL INSERT.

This guide does not cover Redis, PostgreSQL or n8n installation; infrastructure migration; Kubernetes; general backup design; full disaster recovery; production security hardening; AI model selection; prompt evaluation; observability-platform deployment; pricing estimates; or performance benchmarks. It defines the contract with an error workflow without reproducing that separate workflow in detail.

Preflight checklist

  • Record the installed n8n version and the versions running on the main process and every worker.

  • Confirm that an existing self-hosted main process and worker environment is available.

  • Confirm that the main process and workers can reach Redis and n8n persistence.

  • Confirm access to restart every n8n process affected by configuration changes.

  • Verify that all relevant processes use the required shared n8n encryption key.

  • Prepare an AI-service credential for the selected OpenAI node operation.

  • Create or select a downstream PostgreSQL credential limited to the example schema, with SELECT and INSERT plus UPDATE only if the chosen conflict policy needs it.

  • Confirm permission to edit credentials, environment settings, workflow settings, execution retention and error routing.

  • Reserve a non-production environment or controlled data set for interruption, duplicate-delivery and database-outage scenarios.

Map the execution path and its permission boundaries

In n8n queue mode, the main instance receives triggers and creates executions. It passes an execution ID through Redis. A worker retrieves the workflow information from n8n persistence, runs the nodes and writes the execution result back. Redis then notifies the main process that execution has finished.

For this example, the complete path is:

  1. The main n8n process receives a request and creates an execution.

  2. Redis makes that execution available to a worker.

  3. The worker loads workflow and execution information from n8n persistence.

  4. The workflow validates the input and assigns or accepts a stable operation ID.

  5. The OpenAI node performs the bounded AI-assisted task using its own credential.

  6. The workflow explicitly rejoins the original operation context with the AI result.

  7. The Postgres node sends a parameterized, constraint-protected write to the downstream database.

  8. n8n saves the execution outcome according to the configured retention policy.

  9. A failed execution invokes the selected error workflow when n8n records it as a workflow failure.

  10. The exception owner uses the execution reference to inspect saved data and recover the operation ID.

Keep two databases conceptually separate. n8n persistence contains workflow, credential and execution state used by n8n. Downstream PostgreSQL contains the business effect created by this workflow. They may be operated on related infrastructure, but a row in one does not prove a corresponding state in the other.

Assign access by component:

  • The main process and workers need Redis and n8n persistence connectivity.

  • Processes that must use stored credentials need the same n8n encryption key. n8n documents that workers must share the main instance’s key so that they can access encrypted credentials.

  • The OpenAI node uses an AI-service credential. It does not need the downstream database credential.

  • The Postgres node uses a schema-limited database credential. It does not need administrative database rights.

  • The human exception owner needs permission to inspect relevant executions and query the operation record, but not necessarily to alter committed business data.

Name an owner and a fallback owner for exceptions before activation. An alert without someone authorized to inspect the execution and database state is not a recovery path.

Configure queue execution and retain enough recovery evidence

Apply queue and execution settings as a controlled configuration change. Variable names and availability can change, so compare these examples with the documentation for the installed n8n version.

  1. Use queue execution consistently. Set EXECUTIONS_MODE=queue on the main process and workers. Configure the Redis host, port, database, username, password and TLS settings that match your deployment. The queue-mode variable reference lists the supported variables.

  2. Distribute the encryption key securely. Set the same N8N_ENCRYPTION_KEY value on the main process and every worker that needs stored credentials. Handle it as a secret rather than placing its literal value in a general deployment manifest.

  3. Start workers with explicit capacity. Workers are started with the n8n worker command. Set concurrency deliberately rather than allowing an unnoticed default to define database and API pressure. n8n documents a default worker concurrency of 10 and recommends 5 or higher, while also warning that many low-concurrency workers can exhaust database connection pools. Capacity still needs to match the actual database, Redis and API limits.

  4. Enable readiness evidence where useful. Setting QUEUE_HEALTH_CHECK_ACTIVE=true exposes worker health and readiness endpoints. Readiness checks include the worker’s database and Redis connections. Protect or restrict operational endpoints according to the deployment’s network policy.

  5. Allow planned shutdowns to drain. N8N_GRACEFUL_SHUTDOWN_TIMEOUT controls how long a worker can finish running jobs during shutdown. Choose a value in relation to normal execution duration and the deployment manager’s termination window. The older QUEUE_WORKER_TIMEOUT is deprecated, so do not copy it from an outdated configuration.

  6. Bound execution and AI waiting time. Review EXECUTIONS_TIMEOUT, EXECUTIONS_TIMEOUT_MAX and N8N_AI_TIMEOUT_MAX. The execution environment settings distinguish workflow timeouts in seconds from the AI-node HTTP timeout in milliseconds. Align them so that one layer does not terminate work unexpectedly while another still assumes it can finish.

  7. Retain failed execution data. Set EXECUTIONS_DATA_SAVE_ON_ERROR=all for the recovery design. Decide separately whether successful execution data should be saved. Keeping all successes can help comparison and audit, but increases storage and cleanup load.

  8. Set a deliberate retention window. Configure pruning, maximum age and maximum retained count so failed records remain available through the team’s response window. Relevant settings include EXECUTIONS_DATA_PRUNE, EXECUTIONS_DATA_MAX_AGE and EXECUTIONS_DATA_PRUNE_MAX_COUNT. A recovery process cannot depend on an execution that is deleted before the exception is reviewed.

  9. Check storage and display boundaries. Execution-data storage can use the database or other documented modes. The maximum display size can omit large data from the execution interface even though n8n can still load the full data for retrying or resuming. Avoid putting unnecessary prompt or payload content into execution data solely for recovery.

  10. Restart and check every affected process. Confirm that the main process and all workers loaded the intended configuration. Keep process versions aligned. If the deployment uses multi-main mode, n8n explicitly requires all main and worker processes to run the same n8n version.

Queue lock duration, renewal time and stalled-job checks are version-sensitive operational controls. Do not infer from them that an abruptly interrupted execution will always be redelivered, resumed, marked failed or sent to the error workflow. Record the behavior of the installed version during the controlled verification described later.

Queue mode also has deployment boundaries. n8n does not support its distributed setup over SQLite, and filesystem binary-data storage is not supported for persisted binary data in queue mode. If the workflow handles binary data or large webhook responses, review the relevant storage and version requirements before applying this reference path.

Build the example around one stable operation identity

Assume the workflow receives an internal request that must be classified by an AI step before one business effect is recorded. Define the input contract before adding nodes:

{
  "operation_id": "req_01JEXAMPLE7F4M2",
  "source_ref": "intake-84521",
  "received_at": "2026-10-05T12:00:00Z",
  "workflow_version": "intake-write-v3",
  "request_text": "Controlled example input"
}

The caller may provide operation_id if it already has a durable request identifier. Otherwise, create it once at ingress. Never regenerate it during retry or reprocessing. Its value becomes the idempotency key used by PostgreSQL.

Build the node sequence as follows:

  1. Trigger: Receive the request through the existing trigger.

  2. Validate input: Require operation_id, source_ref, received_at, workflow_version and the fields needed for classification. Stop before the AI call when the identity contract is invalid.

  3. Preserve context: Create a structured object such as operation_context containing the stable fields. Keep this object independent of prompt text and model output.

  4. Run the AI operation: Use a supported text operation in the n8n OpenAI node, such as Generate a Model Response, with the designated OpenAI credential. OpenAI node V2 is documented as available from n8n 1.117.0, so verify node and operation availability on the installed version.

  5. Validate the AI result: Check that the fields required by the database write exist and have acceptable types. A model response should not control the operation ID, table name, SQL or credential selection.

  6. Remap explicitly: Build the Postgres input from the preserved operation_context plus the validated AI fields. Set operation_id from the original context, not from model-generated content.

  7. Execute the atomic write: Pass fixed SQL and query parameters to the Postgres node.

  8. Return a classified outcome: Distinguish a newly accepted operation from an already existing operation using the query result.

Expressions will depend on the exact node output. The important ownership rule is stable: preserve identity before the AI node and map it back after that node. Do not rely on a model to repeat an identifier exactly.

Because the documented Error Trigger payload does not establish that arbitrary business fields are forwarded, retain failed execution data. The error route can then pass the documented execution ID or URL to the exception owner, who opens the saved execution and recovers operation_id from the preserved context.

Make PostgreSQL enforce one atomic business effect

Workflow branching alone cannot guarantee idempotency under concurrent or repeated delivery. Put the rule in PostgreSQL as a unique constraint.

A compact reference schema is:

CREATE TABLE workflow_operations (
  operation_id text PRIMARY KEY,
  source_ref text NOT NULL,
  received_at timestamptz NOT NULL,
  workflow_version text NOT NULL,
  ai_result jsonb NOT NULL,
  status text NOT NULL CHECK (status IN ('committed')),
  committed_at timestamptz NOT NULL DEFAULT current_timestamp
);

CREATE TABLE business_effects (
  operation_id text PRIMARY KEY
    REFERENCES workflow_operations(operation_id),
  effect_payload jsonb NOT NULL
);

The same operation_id is used from first submission through every retry and approved reprocessing attempt. Do not delete or replace the key merely to make a failed execution run again.

In the Postgres node, choose Execute Query and use query parameters rather than concatenating input into SQL. n8n documents $1, $2 and similar placeholders plus a Query Parameters option, and states that it sanitizes data supplied through those parameters.

The following statement accepts one operation and creates its related effect within one PostgreSQL statement:

WITH accepted AS (
  INSERT INTO workflow_operations (
    operation_id,
    source_ref,
    received_at,
    workflow_version,
    ai_result,
    status
  )
  VALUES ($1, $2, $3, $4, $5::jsonb, 'committed')
  ON CONFLICT (operation_id) DO NOTHING
  RETURNING operation_id
),
created_effect AS (
  INSERT INTO business_effects (
    operation_id,
    effect_payload
  )
  SELECT operation_id, $6::jsonb
  FROM accepted
  RETURNING operation_id
)
SELECT
  $1::text AS operation_id,
  EXISTS (SELECT 1 FROM accepted) AS newly_accepted;

Supply the parameters in this order:

  1. stable operation ID;

  2. source reference;

  3. received timestamp;

  4. workflow version;

  5. validated AI result as JSON;

  6. business-effect payload as JSON.

If newly_accepted is true, this statement inserted the operation and its related effect. If the related effect insert fails, the statement does not leave the operation insert committed by itself. If newly_accepted is false, the unique constraint prevented this statement from inserting another operation or business effect. Query PostgreSQL separately by operation ID when the operator needs to inspect the pre-existing row and its related effect.

DO NOTHING is safer when a repeated request must never rewrite the accepted result. Choose DO UPDATE only when the business rule explicitly permits replay to modify an existing record. PostgreSQL documents that ON CONFLICT DO UPDATE provides an atomic insert-or-update outcome, but it also requires the relevant UPDATE and SELECT privileges and may silently broaden the effects of a retry if the update rule is poorly constrained.

For a different design that sends multiple queries or incoming items through one Postgres node, the node’s Transaction query-batching option executes the grouped queries in a transaction and rolls them back when one fails. Keep every related business effect inside the same database transaction whenever partial completion would be invalid.

Route failures with enough context for an operator to act

Select an error workflow in the main workflow’s settings. The separate workflow must begin with n8n’s Error Trigger. According to the error-handling documentation, its default failure data can include:

  • workflow ID and name;

  • execution ID and URL when the execution was saved;

  • retryOf when the execution is a retry;

  • the last node executed;

  • execution mode; and

  • error message and stack information.

Use a restrained routing contract. A notification can contain:

{
  "summary": "Workflow execution requires inspection",
  "workflow_id": "1",
  "workflow_name": "AI-assisted intake",
  "execution_id": "231",
  "execution_url": "https://n8n.example.com/execution/231",
  "retry_of": "34",
  "last_node": "Commit business effect",
  "owner": "operations-on-call"
}

Keep the alert summary fixed and safe. Do not forward raw prompts, request text, stack traces or database payloads into chat or email merely because the Error Trigger exposes them. The exception owner can inspect sensitive details in the controlled execution interface if authorized.

The owner’s first task is correlation, not retry. Open the saved execution from its URL or ID, recover the preserved operation_id, and use that key to inspect PostgreSQL. If the failure occurred in the trigger itself, the documented error payload may not contain an execution ID or URL because the main workflow did not execute. Route that case as an intake or activation failure and do not claim that a business write was attempted.

Detailed construction of the companion error workflow is outside this guide. Its contract here is to notify the named owner, retain the execution reference and avoid taking an automatic database action without first establishing the operation ID and write state.

Choose when to retry and when to inspect

The Postgres write is the decision boundary. Before it, some transient failures can be retried without risking a duplicate database effect. At or after it, uncertainty about commit status must be resolved through the stable operation key.

n8n documents Retry On Fail for integration-node failures and a configurable delay between attempts. Its rate-limit guidance also describes Loop Over Items plus Wait for paced requests. Confirm the available attempt-limit and delay controls in the installed n8n version before enabling them.

An eligible example is an AI-service rate-limit response that occurs before the Postgres node begins. Use a finite limit, a deliberate delay and a terminal exception when attempts are exhausted. Avoid retrying validation failures, permission errors or malformed requests as though they were temporary.

If PostgreSQL is unavailable before any write is sent, the operation may be retried after service recovery, but it should still use the same idempotency key. If the connection fails while or after sending the statement, the outcome is ambiguous. Treat it as inspection work rather than assuming rollback or commit.

Worker interruption is also an inspection case unless the installed version’s observed execution state and the database evidence remove the ambiguity. Queue lease and stalled-job settings do not by themselves prove what happened to the business write.

Choose the recovery path

Bounded automatic retry

Use this only for a documented transient failure where the downstream write is known not to have started. Apply a finite attempt limit and deliberate delay whose controls have been confirmed for the installed n8n version.

Inspect, then reprocess

Use this for worker interruption, an ambiguous write outcome, database connection loss near the write, exhausted retries or any unknown commit status. Correlate the saved execution with the stable operation key, inspect PostgreSQL and reprocess only through the idempotent path.

Verify the invariants with controlled failure scenarios

Run these scenarios with controlled data. For every test, record the operation ID, relevant execution IDs, observed worker or queue state and the result of a PostgreSQL lookup such as:

SELECT
  o.operation_id,
  o.status,
  o.committed_at,
  e.operation_id IS NOT NULL AS has_business_effect
FROM workflow_operations o
LEFT JOIN business_effects e
  ON e.operation_id = o.operation_id
WHERE o.operation_id = $1;

The acceptance invariants are:

  • no more than one committed business effect for one operation ID;

  • no operation row without its required related effect;

  • a recoverable execution reference when n8n saved one; and

  • an actionable exception when automatic handling ends.

1. Successful execution

Action: Submit a new controlled operation ID and allow the workflow to complete.

Inspect in n8n: Confirm the completed execution contains the original operation context, the validated AI result and the database outcome.

Inspect in PostgreSQL: Query by operation ID. Expect one operation row and one related business-effect row.

Accept when: The write is complete and both sides refer to the same operation ID.

2. Duplicate submission

Action: Submit the same operation again with the same operation ID.

Inspect in n8n: Confirm the Postgres result returns newly_accepted=false, showing that this statement did not insert another operation or business effect.

Inspect in PostgreSQL: Run the separate lookup by operation ID and confirm that there is still one operation row and one related effect.

Accept when: Repeated delivery does not produce another business effect or rewrite the accepted row under the DO NOTHING policy.

3. Failure before the write

Action: Cause a controlled failure after preserving context but before the Postgres node, for example by rejecting an intentionally invalid AI-result shape.

Inspect in n8n: Confirm that the failed execution is saved and the error route receives an execution reference when applicable.

Inspect in PostgreSQL: Query the operation ID and confirm that no row exists.

Accept when: No business effect exists and the exception owner can identify the operation from the saved execution.

4. Interruption around the write boundary

Action: In the controlled environment, interrupt the selected worker around the Postgres step according to the team’s approved test method.

Inspect in n8n: Record the actual execution status, queue behavior, any linked retry and whether the error workflow activates. Do not assume that the execution will resume, be redelivered or fail in a particular way.

Inspect in PostgreSQL: Query the stable operation ID before deciding whether to reprocess.

Accept when: The database contains either one complete effect or no effect. Reprocessing through the same key must not create a duplicate. Preserve the observed n8n behavior as version-specific operating evidence.

5. PostgreSQL unavailable

Action: Use the controlled test environment to make the downstream database unavailable before the write.

Inspect in n8n: Confirm that the Postgres node fails, the execution evidence is retained and the exception route provides its documented reference.

Inspect in PostgreSQL: After restoring access, query the operation ID before retrying.

Accept when: No partial related write exists, and any reprocessing uses the original operation ID.

6. Error-routing activation

Action: Use an intentional workflow failure, such as a Stop And Error node in the controlled test path.

Inspect in n8n: Confirm that the selected error workflow starts and receives workflow identity, the saved execution reference when available and the last-node context.

Inspect the notification: Confirm that it contains the fixed safe summary and intended identifiers without raw prompts, workload content or stack traces.

Accept when: The named owner can open the saved execution, recover the operation ID and begin inspection without guessing which request failed.

Treat these as acceptance criteria for your test run, not as predicted results for every n8n version. Keep the observed interruption behavior with the deployment’s operational documentation.

Recover an ambiguous execution without bypassing idempotency

When a write outcome is unknown or automatic attempts are exhausted, use the following procedure:

  1. Stop unsafe repetition. Pause the affected workflow, intake route or retry mechanism as narrowly as possible. Avoid stopping unrelated processing when operations are independently keyed.

  2. Capture identifiers. Record the workflow ID, execution ID or URL, retry linkage, last executed node and stable operation ID. Recover the operation ID from saved execution data if the alert does not contain it.

  3. Query PostgreSQL by operation ID. Inspect the operation row and every related effect covered by the transaction boundary.

  4. Classify the state. Mark it as committed, uncommitted, partial or indeterminate based on evidence rather than the last visible n8n node.

  5. Handle a committed operation. Do not replay it under a new key. Confirm that the related effect is complete. If the committed business result is wrong, obtain the required approval and use the system’s defined correction or compensation process.

  6. Handle an uncommitted operation. Reprocess it through the original workflow path with the same idempotency key after the underlying cause is resolved.

  7. Handle a partial operation. Keep processing paused for that operation. A partial state means the intended transaction boundary was absent, bypassed or changed. Correct or compensate only through an approved database and business procedure, then repair the write design before resuming.

  8. Handle an indeterminate operation. Escalate to the database or service owner. Do not convert uncertainty into a retry by generating a fresh operation ID.

  9. Verify the invariant. Confirm that the operation has at most one complete business effect and that related rows agree.

  10. Resume and document. Restore the paused route, record the evidence and decision, and retain the execution references for the team’s operating record.

A database transaction rollback concerns writes that did not commit. A business compensation concerns a validly committed effect that must later be reversed or corrected. Do not describe compensation as a rollback, and do not delete the idempotency record merely to make replay easier.

Keep the recovery path usable as the system changes

Assign recurring ownership rather than treating recovery as a one-time workflow setting:

  • The platform owner monitors Redis connectivity, queue depth, worker readiness and process configuration.

  • The workflow owner reviews failed executions and workflow changes.

  • The exception owner responds to ambiguous operations and exhausted retries.

  • The database owner controls schema, constraints, permissions and approved corrections.

  • The security or credential owner coordinates key and credential rotation.

Retention should cover the real exception-response window while still pruning old execution data. Longer retention improves the chance of reconstructing an event, but increases storage and administrative load. Saving all successful executions provides more comparison evidence at an additional cost, so choose it deliberately.

Re-run the relevant controlled scenarios after:

  • an n8n upgrade;

  • worker-count or concurrency changes;

  • Redis topology or credential changes;

  • n8n encryption-key or integration-credential rotation;

  • a change to the OpenAI node or selected operation;

  • modifications to the operation-context mapping;

  • SQL, constraint or PostgreSQL permission changes; and

  • schema migrations affecting either table in the atomic boundary.

Queue mode adds Redis, worker processes and distributed-state handling. More workers can increase compute, Redis and database load. Many workers with poorly chosen concurrency can pressure connection pools. Longer execution retention consumes more storage. These tradeoffs depend on workload and infrastructure, so size them from the deployment’s own observations rather than a generic benchmark.

For a broader view of execution-state ownership, see Monitoring the AI operating layer. The setup here remains narrower: it is a recoverable write path, not a complete high-availability, security-hardening, backup or disaster-recovery design.

Next step: prove one write can be recovered safely

Choose one existing workflow write in a controlled environment. Give it a stable operation ID, protect that ID with a PostgreSQL unique constraint, keep related effects inside one transaction boundary and submit the same operation twice.

Then inspect the saved n8n execution evidence and query PostgreSQL by that operation ID. The observable result should be no more than one committed business effect, with enough information to explain what happened. That replaces the opening operator’s uncertainty with a checkable recovery decision rather than an unsafe guess.

Questions and answers

Does queue mode prevent duplicate PostgreSQL writes?

No. Queue mode changes how n8n distributes executions across its main process, Redis and workers. Duplicate protection should be enforced separately with a stable idempotency key, a PostgreSQL unique constraint and a conflict policy that every retry uses.

Should every failed node be retried automatically?

No. Reserve bounded automatic retry for a documented transient failure where the database write is known not to have started. If commit status is uncertain, inspect the saved execution and query PostgreSQL by operation ID before reprocessing.

Can an n8n error workflow identify the affected business operation directly?

The documented Error Trigger payload includes workflow and execution details, but it does not establish that arbitrary business fields are included. Save failed execution data and use its execution ID or URL to recover the stable operation ID before making a recovery decision.

What proves that the recovery design is working?

Controlled tests should show no more than one committed business effect for each operation ID, no partial related writes, a usable execution reference when n8n saves one, and an actionable exception when automatic handling ends.