Skip to content
Factflow

Queue diagnostics

When a pipeline seems stuck — messages accepted but no progress — the queue is the first place to look. This guide walks through the diagnostic tools.

Health check

Section titled “Health check”

Fast path — does the broker even respond?

Terminal window
factflow system health --debug

Look for the queue component. Status healthy means connection is live.

Full round-trip diagnostic

Section titled “Full round-trip diagnostic”

Publish a test message, subscribe, verify receipt. Runs against every configured queue:

Terminal window
factflow system mq-diagnostic

Or via API:

Terminal window
curl -X POST http://localhost:8000/api/v1/system/mq-diagnostic

Expected output: one line per queue, all ok. If any fail, the error payload names the queue and the failure class.

Provider-specific issues

Section titled “Provider-specific issues”

Artemis (STOMP) — header timestamps

Section titled “Artemis (STOMP) — header timestamps”

Artemis rejects STOMP frames with malformed timestamps silently (no error, message just disappears). Factflow’s StompNamingStrategy formats timestamps correctly; custom subscribers must match.

Format: strict ISO-8601 with timezone, e.g. 2026-04-22T18:30:00.123456+00:00.

RabbitMQ (AMQP) — name rules

Section titled “RabbitMQ (AMQP) — name rules”

AMQP queue names must match [a-zA-Z0-9_\-.:]+. Special characters cause the broker to reject publishes. AmqpNamingStrategy validates before publish.

Pulsar — topic topology

Section titled “Pulsar — topic topology”

Pulsar has different semantics around ordering and subscription types. PulsarNamingStrategy handles the topology mapping. If you hit Pulsar-specific issues, the factflow-infra reference has the provider’s quirks.

Execution isolation

Section titled “Execution isolation”

Every queue name is wrapped by ExecutionScopedQueue — prefixed with the execution id. A message published for execution A under route web_scraper is NOT visible to execution B’s web_scraper processor, even though they share the route name in config.

If you expect execution B to pick up something and it doesn’t: verify the executions aren’t using the same broker instance but different execution ids (which is correct — that’s what isolation does).

Stuck message — common diagnoses

Section titled “Stuck message — common diagnoses”

1. Processor isn’t subscribed

Section titled “1. Processor isn’t subscribed”
Terminal window
factflow pipeline list --execution EXEC_ID

Every route should show a processor. If a route is missing, the orchestrator failed to start that processor. Check factflow execution get EXEC_ID for errors.

2. Backpressure applied

Section titled “2. Backpressure applied”

factflow system metrics shows in_flight count. If in_flight == concurrency for the affected route, the processor is saturated. Options: raise concurrency in YAML, find the slow adapter, reduce upstream rate.

3. Circuit breaker open

Section titled “3. Circuit breaker open”

The engine’s per-adapter circuit breaker may have opened after repeated failures. factflow lineage failures --execution EXEC_ID --adapter X shows the pattern.

4. Route is globally paused

Section titled “4. Route is globally paused”
Terminal window
factflow pipeline list

Shows paused state per route. Unstick with:

Terminal window
factflow pipeline resume ROUTE_ID

5. Broker queue depth

Section titled “5. Broker queue depth”

Direct broker inspection tools (Artemis web console, RabbitMQ management UI) show queue depth. If messages are piling up without being consumed, see #1 and #2.

Section titled “Related”