Adapter catalog

Every adapter the engine discovers and registers, grouped by package. The always-current list with full JSON-Schema config is the registry itself — GET /api/v1/adapters (and GET /api/v1/adapters/{type} for one adapter's schema). When this page and the registry disagree, the registry wins.

Type names are how you reference an adapter in pipeline YAML (type: web_scraper). Config classes are Pydantic models in the owning package; query the registry for the full field schema.

Webscraper (factflow-webscraper)

`type:`	Purpose	Config class
`sitemap_parser`	Fetch a sitemap XML, extract URLs	`SitemapParserConfig`
`url_expander`	Fan-out: URL list → one message per URL	`URLExpanderConfig`
`web_scraper`	HTTP fetch with adaptive rate limiting	`WebScraperConfig`
`web_crawler`	JS-rendered fetch via crawl4ai; detects and fans out PDF links	`WebCrawlerConfig`
`pdf_fetcher`	Download a crawl-discovered PDF, stage it for conversion	`PdfFetcherConfig`
`web_content_storage`	Persist HTML + metadata	`WebContentStorageConfig`

pdf_fetcher consumes the PDF links web_crawler fans out, downloads the bytes, and stages them so document_converter produces markdown beside the HTML crawl output. Key fields: max_pdf_size_mb, request_timeout, rate_limit.

Markdown (factflow-markdown)

`type:`	Purpose	Config class
`storage_retriever`	Read bytes from storage by key	`StorageRetrieverConfig`
`html_to_markdown`	GitHub-flavoured HTML → markdown	`HtmlToMarkdownConfig`
`smart_segmenter`	Token-aware markdown splitting	`SmartSegmenterConfig`
`segment_publisher`	Fan-out: one message per segment	`SegmentPublisherConfig`
`segment_storage`	Persist segments	`SegmentStorageConfig`
`markdown_storage_writer`	Persist canonical markdown	`StorageWriterConfig`

factflow_markdown also defines a metadata_extractor alias for the same adapter, but it is registered by register_markdown_adapters(), which only the package's own e2e tests call — auto-discovery does not pick it up, so a running server does not have it and a pipeline naming it fails config validation. Use html_to_markdown.

Note markdown_storage_writer uses StorageWriterConfig (the shared writer model), not a markdown-specific class.

Embeddings (factflow-embeddings)

`type:`	Purpose	Config class
`embedding_generator`	Generate embeddings (multi-model), write to storage	`EmbeddingGeneratorConfig`

Translator (factflow-translator)

`type:`	Purpose	Config class
`llm_translator`	LLM translation with markdown preservation	`LLMTranslatorConfig`

Hygiene (`factflow-hygiene`)

Composed in the scribe pipeline, upstream of synthesis. Order matters: pii_redactor → sensitivity_classifier → anti_bac_gate → sc_redactor.

`type:`	Purpose	Config class
`pii_redactor`	Redact structured PII (fnr, IBAN, …)	`PIIRedactorConfig`
`sensitivity_classifier`	LLM sensitivity-tier classification	`SensitivityClassifierConfig`
`anti_bac_gate`	LLM gate that quarantines broken-access-control / org-invariant violations; blanks the body on a fired flag	`AntiBacGateConfig`
`sc_redactor`	Blank strictly-confidential bodies	`SCRedactorConfig`

anti_bac_gate keys: llm_profile, content_field, origin_field, source_ref_field, content_cutoff. It caches verdicts by (content_hash, model_profile), so weekly re-runs and replay are deterministic.

Knowledge (factflow-knowledge)

`type:`	Purpose	Config class
`concept_detection`	Taxonomy-guided concept extraction	`ConceptDetectionConfig`
`concept_map_writer`	Persist the concept map	`ConceptMapWriterConfig`
`concept_fanout`	Fan out concepts for consolidation	`ConceptFanoutConfig`
`concept_source_loader`	Load source content per concept	`ConceptSourceLoaderConfig`
`batch_synthesizer`	LLM synthesis across sources	`BatchSynthesizerConfig`
`structural_validator`	Citation / length / diversity checks	`StructuralValidatorConfig`
`avalon_writer`	Post consolidated map to Avalon	`AvalonWriterConfig`
`concept_map_updater`	Update stored concept map	`ConceptMapUpdaterConfig`
`knowledge_diff`	Compare maps, emit structured diff	`KnowledgeDiffConfig`
`scribe_diff_writer`	Persist the diff	`ScribeDiffWriterConfig`
`storage_enumerator`	Enumerate stored artefacts	`StorageEnumeratorConfig`

SharePoint (factflow-sharepoint)

`type:`	Purpose	Config class
`sharepoint_enumerator`	List files across SharePoint sites (Graph)	`SharePointEnumeratorConfig`
`sharepoint_fetcher`	Download each file	`SharePointFetcherConfig`
`document_converter`	Convert Office docs / PDFs → markdown	`DocumentConverterConfig`

Boost (factflow-boost)

Boost owns two adapter families: the conversation processor (enumerate → compact → filter → deduplicate → cluster → catalog) and the routines renderer. Most processing adapters pair with a matching storage writer that persists their output stage.

Conversation processor

`type:`	Purpose	Config class
`boost_enumerator`	Walk an export, emit per-package messages	`BoostEnumeratorConfig`
`boost_batch_processor`	Process every file in a package in one invocation	`BoostBatchProcessorConfig`
`boost_compact`	Extract and compact content from individual ActionMeta files	`BoostCompactConfig`
`boost_stored_keys_fanout`	Fan-out: one message per key written by the batch processor	`BoostStoredKeysFanoutConfig`
`boost_filter`	Drop excluded conversations	`BoostFilterConfig`
`boost_norwegian_filter`	LLM filter dropping non-Norwegian conversations	`BoostNorwegianFilterConfig`
`boost_filtered_collector`	Fan-in: aggregate filtered messages per package	`BoostFilteredCollectorConfig`
`boost_deduplicate`	Near-duplicate removal	`BoostDeduplicateConfig`
`boost_clustering`	Group similar conversations (embedding-based)	`BoostClusteringConfig`
`boost_catalog`	LLM-render the catalogue	`BoostCatalogConfig`
`boost_storage_writer`	Persist compacted content + metadata sidecar	`BoostStorageWriterConfig`
`boost_deduplicate_writer`	Persist deduplicated output	`BoostDeduplicateStorageWriterConfig`
`boost_clustering_writer`	Persist clustering output	`BoostClusteringStorageWriterConfig`
`boost_catalog_writer`	Persist catalogue output	`BoostCatalogStorageWriterConfig`

Routines

`type:`	Purpose	Config class
`boost_routine_discovery`	Discover export dirs, fan-out one message per export	`BoostRoutineDiscoveryConfig`
`boost_routine_indexer`	Parse export `data.json`, fan-out one message per root intent	`BoostRoutineIndexerConfig`
`boost_routine_renderer`	Render a routine to markdown	`BoostRoutineRendererConfig`
`boost_routine_writer`	Write rendered markdown to storage	`BoostRoutineWriterConfig`

Replay (factflow-replay)

Replay isn't composed into YAML as a type: — it's invoked via POST /executions/{id}/replay or the CLI. See the Replay guide.

Confluence (factflow-confluence)

A three-stage ingest chain: crawl a discovery unit, fetch each page, then fetch its attachments. The client is built from env-driven settings and lazily created on first use, so none of these take connection config in YAML.

`type:`	Purpose	Config class
`confluence_space_crawler`	Enumerates a discovery unit and fans out one message per page	`ConfluenceCrawlerConfig`
`confluence_page_fetcher`	Stores a page's HTML and fans out one descriptor per attachment	`ConfluencePageFetcherConfig`
`confluence_attachment_fetcher`	Downloads an attachment, preserving its filename so the converter can infer format	`ConfluenceAttachmentFetcherConfig`

confluence_page_fetcher is 1→N: the stored body goes to the markdown chain and each attachment to the attachment fetcher. Fan-out is routed by an explicit destination on each emitted message — outbound when conditions are not evaluated for emit_messages — so each message carries its own destination queue. A 404 at either fetch stage means the page or attachment was deleted between crawl and fetch; it is skipped, not an error.

Simulator (`factflow-simulator`)

Synthetic load for exercising the queue subsystem at volume. Not part of any production pipeline — these carry no business logic and take no dependencies (no network, no LLM, no storage), so a message lost or run twice under them can only have come from message delivery. Used by test-cli scenario s14; see Verifying delivery at volume.

`type:`	Purpose	Config class
`synthetic_fanout`	Emits `count` synthetic children to a destination through the real fan-out path	`SyntheticFanoutConfig`
`synthetic_sink`	Terminal stage; holds its handler slot for `work_ms` and records nothing	`SyntheticSinkConfig`

Configure synthetic_fanout twice for a two-level spread (waves, then work items). That keeps each lineage pre-registration transaction bounded to one wave instead of the whole run, which is also the shape real pipelines have — sitemap → URLs → documents.

Introspection at runtime

The registry is the source of truth. Dump it with full config schemas:

curl http://localhost:8000/api/v1/adapters
curl http://localhost:8000/api/v1/adapters/{type}

GET /api/v1/adapters/{type} returns the adapter's config class as JSON Schema — usable for IDE autocomplete and tooling.

Not adapters: `monitoring:` block strings

This catalog enumerates registry adapters — the type: values you put in pipeline routes. The shipped scribe and avalon configs (config/pipelines/scribe/*.yaml) also carry a monitoring: block whose string values look like types but are not registered adapters: console (a metrics.exporters[].type), queue_connectivity and storage_accessibility (health_checks.checks[].type). They aren't in the adapter registry and won't appear in GET /api/v1/adapters. Don't reference them as route type: values.

Pipeline YAML reference
Writing a new adapter
Per-package references under Reference → Packages