Migrating from the centralized Catalog Service to the decentralized catalog

A migration guide for networks currently publishing through CATALG's centralized publish/subscribe API, moving to the decentralized, self-hosted catalog model.

This migration needs no new integration code from you.

• Provider Node: publishing becomes hosting files and running the existing catalog-publisher tool — no catalog/publish client to write.
• Discovery Service builder: the crawling and signature-verification logic you'd otherwise have to build is already done for you by the catalogcrawler plugin — you configure it, not code it (if you're using DISCOVR, even that's already wired up).
• Consumer Node: no change at all — you still call discover against the same Discovery Service as before; this migration is entirely behind that API and invisible to you.

This is a configuration and infrastructure change for two of the three actors, and a non-event for the third — not a development project for anyone.

Full specifications: NFH-014 — Decentralized Catalog Publishing and Discovery defines the new model end to end; NFH-003 — The Beckn Protocol Stack is the updated stack definition with the Cataloging Service actor removed. Both are linked again, at the relevant section, throughout this guide.

If you're a Provider Node, here's what changes — in three steps

This is the question most PNs land on this page to answer. In full, it's §1 below — but the short version is:

1. Host your catalog files on storage you already have (S3, a CDN, GitHub Pages, an ngrok tunnel for testing) instead of calling a network API.
2. Generate and sign them with the catalog-publisher tool instead of hand-building a catalog/publish request — it produces the baseline/change file and catalog index for you.
3. Set one field — meta.catalog_index_urls — on the Registry record you already have. That's the entire "registration" step; there's no separate publish-side onboarding.

No server to stand up, no subscription list to manage, no ACK/NACK handshake to get right. See Test it yourself with starter-kit below to see this end to end before touching your own integration.

Executive Summary

If you're publishing catalogs today via catalog/publish with ACK/NACK responses, managing subscriptions with catalog/subscription, or running a Discovery Service that subscribes to a central Cataloging Service (CS) — this guide is for you. NFH fabric's catalog model has moved to a decentralized shape: a Provider Node (PN) self-hosts and self-signs its own catalog files and a catalog index on infrastructure it controls, and a Discovery Service (DS) crawls, independently verifies, and indexes that data directly. There is no shared, centrally-run relay in the path anymore.

Nothing about your catalog content changes — the Catalog, Resource, and Offer schemas are exactly what they were. What changes is how a catalog gets from a PN to a DS. This document is the specification for that new model: NFH-014 — Decentralized Catalog Publishing and Discovery, which also updates NFH-003 — The Beckn Protocol Stack to remove the Cataloging Service actor entirely.

The legacy API is not being switched off out from under you. beckn.yaml's Cataloging Service endpoint group (/catalog/publish, /catalog/subscription, /catalog/pull, /catalog/push, /catalog/search, and their callbacks) is marked deprecated: true, not removed. Its actual retirement date and forced-cutover path are not yet defined; this guide covers the migration you can start today, on your own timeline.

The conceptual shift

Before: a Provider Node called a network API (catalog/publish) and got an ACK/NACK back. A central Cataloging Service stored the catalog, handled Discovery Service subscriptions, and served catalog/pull/catalog/search to consumers.

Now: a Provider Node publishes immutable, signed JSON files to storage it already controls (any CDN, object store, or static host). A Discovery Service resolves the PN's Registry record, crawls that storage directly, verifies every file's signature and digest, and builds its own index. There is no central service to call, subscribe to, or wait on.

1. If you are a Provider Node (publisher)

Short version: pick some storage, generate and sign your catalog files with the catalog-publisher tool, and set one field on a Registry record you already have. There's no server to stand up, no subscription list to manage, and no ACK/NACK handshake to get right.

This assumes you're already onboarded to the NFH fabric — a Registry identity and registered signing key are prerequisites to publishing, not something this migration sets up for you. If you're not there yet, start with Onboarding on the NFH fabric.

1. Pick storage you already have. Any static host works — S3, a CDN, GitHub Pages, even an ngrok tunnel for local testing. You're not building a new service; you're pointing a publishing tool at a folder.
2. Generate and sign your catalog with catalog-publisher. This is a plugin for beckn-onix that diffs your submission against what you last published, produces a fresh baseline or an incremental change file as appropriate, computes digests, signs everything with your Registry-registered key, and writes the resulting catalog index. See Catalog Publishing and Discovery → Publishing tooling and the starter-kit's Catalog Publishing walkthrough for a working, runnable example.
3. Set meta.catalog_index_urls on your existing Registry Subscriber record (a list of {url} entries, not a single string) — that's the entire "registration" step. No separate pointer file, no new registry to onboard into.

Everything else — subscriptions, a central Cataloging Service, waiting on on_publish callbacks — simply isn't part of this model anymore, so there's nothing to configure for it; see What you no longer need below for what to actually delete from your existing integration.

Full step-by-step walkthrough: Catalog Publishing and Discovery → Quickstart.

2. If you are a Discovery Service builder

Replace your subscription setup and catalog/push receiver with a crawler — and in practice, you don't need to build that crawler yourself. The four things a conformant DS must do (enumerate PNs, resolve their Registry records, fetch and verify their catalog index and files, and never index anything unverified) are exactly what the catalogcrawler plugin for beckn-onix already does, out of the box, as a scheduled background job inside your ONIX deployment. For most DS builders, this whole section reduces to: run ONIX with the catalogcrawler plugin configured and pointed at your own /catalog/push-equivalent ingestion endpoint — there's no separate crawl service to design or write.

This isn't specific to any one Discovery Service product. If you're building your own DS — your own indexing, ranking, and /discover API — you still don't need to hand-write the crawl/verify logic: catalogcrawler is a standalone, deployment-agnostic plugin whose only integration contract is discoveryPushUrl, an HTTP endpoint on your service that receives resolved, verified catalogs. Point it at your own ingestion endpoint and it does the crawling for you, regardless of what you build behind that endpoint. Writing your own crawler from scratch is only necessary if this plugin's behavior genuinely doesn't fit your case (see its README for exactly what it does and doesn't cover).

What the crawler plugin handles for you:

1. Enumerates the Provider Nodes relevant to your network — either by direct Registry enumeration, or via a registry plugin (e.g. dediregistry) that answers "which index URIs are relevant to me" in one call. You configure which networkId(s) to watch (networks in its config), or hand it a fixed staticIndexUrls list; it does the enumeration.
2. Resolves each PN's Registry manifest and Subscriber record, and reads meta.catalog_index_urls — again, no code you write; this is the plugin's discovery source.
3. Fetches and verifies the catalog index, then fetches only the baseline or change files its own persisted per-catalog cursor calls for — it never re-fetches what it's already verified and applied, and resumes correctly after a restart (progress is stored in Postgres).
4. Never indexes anything unverified — a missing, invalid, or mismatched signature or digest gets the entry parked (retried on a bounded schedule, then surfaced as abandoned if it never clears), not silently indexed or silently dropped; the plugin never deletes previously-indexed content just because an entry disappeared from a later crawl.

The interface between the crawler and your Discovery Service

Your integration work is entirely on the receiving side, and it's a single HTTP contract: the plugin POSTs each resolved, verified catalog to whatever URL you set as discoveryPushUrl, as a catalog/push-shaped Beckn payload — context.action: "catalog/push", message.catalogs[] holding the fully-resolved catalog (masters already merged into overlays, so you receive self-contained resources, not raw fragments), and a matching message.publishDirectives[] entry carrying catalogType and updateMode. Your endpoint just needs to respond HTTP 200 to acknowledge the push (indexing can happen asynchronously after that); anything else is treated as a failed push and retried on the plugin's normal retry/park schedule. There's no separate ACK/NACK envelope to parse — the HTTP status code alone is the signal.

This is the same internal interface DISCOVR — NFH fabric's reference Discovery Service — implements at its own /catalog/push endpoint (see DISCOVR → Catalog indexing): DISCOVR is simply a catalogcrawler deployment with its own discoveryPushUrl pointed at itself. Nothing about this endpoint is Beckn-protocol-mandated or crawler-mandated — it's just the shape this plugin happens to push in — so building your own DS means implementing a POST receiver with this same request shape, not designing a new protocol between your crawler and your indexer.

See the catalogcrawler plugin README for the full config reference (networks, staticIndexUrls, discoveryPushUrl, polling intervals, retry/park behavior) and its /crawl/trigger and /crawl/status operational endpoints.

Full crawl sequence, verification rules, and error-handling table (what the plugin implements under the hood, useful if you're auditing its behavior or building a crawler from scratch instead): Catalog Publishing and Discovery → Hosting & Discovery.

How a DS notices a brand-new publisher

There's no separate "onboarding event" to subscribe to, because the old catalog/subscription step doesn't exist to replace. A DS's own periodic enumeration pass — re-querying the Registry for participants relevant to the networkId(s) it cares about (§Enumerating PNs, above) — is what surfaces a PN that registered since the last pass. Once that PN's Registry record carries meta.catalog_index_urls, it becomes a normal candidate the very next time the DS enumerates: there's no separate "new publisher" code path, and no faster way to be noticed than making sure both the Registry record and the index it points to are already in place before you expect to be discoverable — that gap is exactly what Before you're crawlable covers. A PN that wants to be picked up sooner than a DS's own enumeration cadence can additionally emit an out-of-band change signal to invite an earlier crawl (§10.6 of NFH-014) — this is an optional accelerator, never a substitute for the DS still fully verifying everything on its own terms.

What you no longer need

• A catalog/publish call to a shared, network-facing Cataloging Service, with an ACK/NACK response. Files replace the round-trip; local validation happens up front, before you ever write a file to storage.
• catalog/subscription CRUD. A crawler's scope is its own configuration now — you don't manage subscriber lists.
• catalog/search. Removed from the publish/pull surface; a Discovery Service may still offer search over its own index, but that's not something you interact with as a publisher.
• catalog/push/on_pull callbacks. Consolidated into the crawler pulling from a PN and pushing into the Discovery Service's own index — a DS never receives a callback for this.
• Restricted catalogs, download gates, authMethods as an access gate. Catalog access is uniformly public, unconditionally — any party with a file's URL can fetch it. If you relied on publishDirectives.visibleTo as an access gate, its replacement (networkIds in the index) is a relevance filter only, never an access control.

Field-by-field mapping

Old (centralized CATALG / DISCOVR)	New (decentralized)
catalog/publish with ACK/NACK	Files saved to storage; validation happens up front, locally
publishDirectives.visibleTo	Per-catalog networkIds in the index — relevance filter, not access gate
publishDirectives.updateMode: MERGE	A change file (id-keyed upserts/removals)
publishDirectives.updateMode: FULL	A fresh baseline
catalog/pull, mode FULL	The baseline file
catalog/pull, mode INCREMENTAL	Change files after the crawler's cursor
downloadManifest (checksum, sizeBytes)	digest/size in the index, verified against each self-signed file
Subscription filters (networkIds, schemaTypes)	Crawler-side filtering on the index
Subscription CRUD (catalog/subscription)	Not needed — a crawler's scope is its own config
catalog/search	Removed from this surface
catalog/push	Crawler pull, with an optional out-of-band change signal as an accelerator
on_pull callback	Consolidated into the Discovery Service's own crawl loop
subscriberId	The PN's domain, resolved via its Registry record
Restricted catalogs / download gate / authMethods as access control	Removed. Catalogs are public-only; no per-catalog auth exists in this RFC
Offer-only catalogs, query-time attachment	Unchanged — still resolved behind /discover
Master/Regular resolution (resourceDirectives[].extends)	Unchanged in shape — resolved by the Discovery Service at index time instead of centrally at publish time

What stays exactly the same

• Your Catalog/Resource/Offer JSON content and its schema.
• catalogType: MASTER/REGULAR and resourceDirectives[].extends.
• Offer-only catalogs and query-time attachment behind /discover.
• The Beckn HTTP Signature discipline itself — you're still signing with your Registry-registered key; you're just signing files instead of API requests.

Test it yourself with starter-kit

Before touching your production integration, run the whole loop locally with the starter-kit repository — it already runs the target-state model end to end:

1. Clone starter-kit and bring up the stack (docker compose -f docker-compose-generic.yml up) — see its Quick Start.
2. Set up the ngrok tunnel so your published files are actually reachable from outside your machine — see its Public-Internet Testing section. This step is easy to skip and is the most common reason a locally-published catalog never shows up in discover.
3. Trigger catalog/publish against onix-bpp (the Catalog Publishing walkthrough) and inspect generic-devkit/data/beckn/ to see the signed baseline and index it wrote.
4. Send a discover request from the BAP collection and confirm your catalog comes back in the results — remember a crawler runs on its own schedule, so allow for the same propagation delay you'd see against a real network, not an instant reflection of what you just published.
5. Confirm meta.catalog_index_urls is actually set on your node's Registry record — the starter-kit README calls this out as the field most commonly left unset, with no immediate feedback when it is.

This is the same shape a real migration follows — only the storage and Registry identity are throwaway.

Reference material

• NFH-014 — Decentralized Catalog Publishing and Discovery — the full protocol specification: schemas, versioning, error flows, and security considerations.
• NFH-003 — The Beckn Protocol Stack — the updated stack definition with the Cataloging Service actor removed.
• starter-kit — a runnable, end-to-end example of catalog-publisher publishing signed files and a crawler-fed Discovery Service consuming them.
• catalog-publisher plugin README — the publishing tool's design and configuration reference.
• catalogcrawler plugin README — the Discovery-side crawler plugin's design and configuration reference.
• Catalog Publishing and Discovery — the full guide for the new decentralized model.