Cowrie Research

sqd-primary-sources-2026-05-14

3 min read

SQD primary-source snapshot

Fetched: 2026-05-14 UTC

1) Current architecture overview (llms-full.txt)

Source: https://sqd.dev/llms-full.txt

Selected excerpts:

SQD (formerly Subsquid) is a blockchain data infrastructure company that provides validated, real-time onchain data from 225+ networks through a single streaming HTTP API.

SQD’s architecture has five components:

  1. SQD Network (Decentralized Data Lake)
  2. Portal API (Data Access Layer)
  3. Squid SDK (ETL Framework)
  4. Pipes SDK (Composable Streaming Pipelines)
  5. SQD Cloud (Managed Platform)

Additional signals from the source:

  • This page is the clearest high-level decomposition of SQD into protocol/network, access layer, SDKs, and managed platform.
  • The explicit Decentralized Data Lake framing matters because it separates storage/replication from the query and developer layers above it.
  • The former Subsquid branding is still operationally relevant because repos and package namespaces remain under subsquid.

2) Portal and network description (llms-full.txt)

Source: https://sqd.dev/llms-full.txt

Selected excerpts:

Portal host: https://portal.sqd.dev. This single host serves three different things depending on path:

  • Landing page at the root
  • Portal app at /app
  • HTTP streaming API at /datasets/{network}/stream

SQD Network (Decentralized Data Lake)

  • 2,000+ worker nodes storing and serving blockchain data
  • Data replicated across independent operators (10–34x replication range)
  • Data stored in immutable Parquet format for reproducibility

Additional signals from the source:

  • Portal is a meaningful chokepoint because the same host fronts landing, control dashboard, and production streaming API.
  • The docs foreground immutable Parquet-backed storage and replication rather than archive-node RPC semantics.
  • This makes SQD more comparable to a replicated data substrate plus gateway layer than to a simple RPC vendor.

3) Squid-from-scratch docs

Source: https://docs.sqd.dev/en/sdk/squid-sdk/how-to-start/squid-from-scratch.md

Selected excerpts:

Build a Squid SDK indexer from scratch — learn how to compose evm-processor, typeorm-store, and graphql-server NPM packages without a starter template.

Suppose the task is to track transfers of USDT on Ethereum, then save the resulting data to PostgreSQL and serve it as a GraphQL API.

const processor = new EvmBatchProcessor() .setGateway(‘https://v2.archive.subsquid.io/network/ethereum-mainnet’)

Additional signals from the source:

  • The docs make the application-facing developer flow explicit: indexer packages plus database/store plus GraphQL serving.
  • The setGateway(...) pattern shows the user-facing query surface is still gateway-mediated even when the lower storage plane is decentralized.
  • The older archive.subsquid.io naming is another reminder that the product stack spans multiple branding eras.

4) Public Squid SDK README

Source: https://raw.githubusercontent.com/subsquid/squid-sdk/master/README.md

Selected excerpts:

Squid SDK is a TypeScript ETL toolkit for blockchain data

Subsquid SDK stands apart from the competition by

  • Being a toolkit (rather than an indexing app like TheGraph or Ponder)
  • Native support for sourcing the data from Subsquid Network.

Subsquid Network is a decentralized data lake and query engine, that allows to granularly select and stream subset of block data to lightweight clients while providing game changing performance over traditional RPC API.

Additional signals from the source:

  • The README draws a sharp boundary between toolkit and fully managed indexing app, which helps preserve SQD’s modularity in the corpus.
  • The repo confirms the network/SDK split in public code, not only in marketing pages.
  • The strongest lower-layer comparison point is the move from RPC polling to subset block streaming from a dedicated data network.

Notes

  • SQD is worth cataloging because it turns blockchain data infrastructure into separate layers: replicated custody, streaming gateways, developer SDKs, and managed deployment.
  • The main control surfaces to track are portal routing, schema normalization, cloud deployment dependence, and the relationship between decentralized worker storage and centralized or semi-centralized access points.
  • Best comparison set for next pass: Firehose, Substreams, Goldsky, Envio/HyperSync, Apibara, and archive-RPC providers.

Graph View