> ## Documentation Index
> Fetch the complete documentation index at: https://goldrush.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Pipeline API Overview

> The GoldRush Pipeline API is a managed real-time data pipeline service that streams blockchain data from Covalent's database directly to your own infrastructure - databases, warehouses, queues, object storage, or webhooks. Configure pipelines through the GoldRush Platform with ABI decoding, SQL transforms, and protocol-native schemas built in.

The Pipeline API pushes structured blockchain data directly to your infrastructure. Instead of polling an API, you configure a pipeline once and data flows continuously into your database, warehouse, queue, or webhook - decoded, transformed, and ready to query.

## Architecture

```
                                                                                                          ╔═ Your Destination ═══╗
                          ╔═ GoldRush Pipeline ═══════════════════════════════════════════╗               ║                      ║░
                          ║                                                               ║░              ║   · ClickHouse       ║░
╔══════════╗              ║   ╔═══════════════╗    ╔══════════════╗    ╔════════════════╗ ║░              ║   · Postgres         ║░
║ Covalent ║░             ║   ║ Normalization ║ ─▶ ║ ABI Decoding ║ ─▶ ║ SQL Transforms ║ ║░              ║   · Kafka            ║░
║ Database ║░ ──stream──▶ ║   ╚═══════════════╝    ╚══════════════╝    ╚════════════════╝ ║░ ──deliver──▶ ║   · S3 / GCS / R2    ║░
╚══════════╝░             ║                                                               ║░              ║   · SQS              ║░
 ░░░░░░░░░░░░             ╚═══════════════════════════════════════════════════════════════╝░              ║   · Webhook          ║░
                           ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░              ║                      ║░
                                                                                                          ╚══════════════════════╝░
                                                                                                           ░░░░░░░░░░░░░░░░░░░░░░░░
```

## How It Works

<Steps>
  <Step title="Create a New Pipeline">
    Log in to the [GoldRush Platform](https://goldrush.dev/platform/) and navigate to **Manage Pipelines**. Click **Create New Pipeline**.
  </Step>

  <Step title="Select your Destination Type">
    Select a destination type - ClickHouse, Postgres, Kafka, Object Storage (S3/GCS/R2), AWS SQS, or Webhook - and provide your connection credentials. You can also test if the Pipeline API can connect to your destination successfully.
  </Step>

  <Step title="Select your Data Object Type">
    Choose a chain and data type. Pick from event logs, transactions, token transfers, or protocol-specific data streams like HyperCore fills or Solana DEX trades.
  </Step>

  <Step title="Select your Data Range">
    Configure the block range for your pipeline. Your pipeline can run in both bounded (with start/end block heights) as well as unbounded mode for continuously delivering data.
  </Step>

  <Step title="Apply SQL transforms and ABI decoding">
    Optionally add SQL transforms to filter rows, select specific columns, or compute derived fields before data reaches your destination. Aditionally you can also provide ABI
    definitions to decode raw event logs or call data into structured, typed columns.
  </Step>

  <Step title="Deploy and monitor">
    Review your configuration, deploy the pipeline, and monitor throughput and status from the Pipelines dashboard. Data begins flowing in real time.
  </Step>
</Steps>

## Destination Types

<CardGroup cols={3}>
  <Card title="ClickHouse" href="/goldrush-pipeline-api/destinations/clickhouse">
    Stream data into ClickHouse for high-performance analytical queries over large volumes of blockchain data.
  </Card>

  <Card title="Postgres" href="/goldrush-pipeline-api/destinations/postgres">
    Push decoded blockchain data into Postgres for application backends and transactional workloads.
  </Card>

  <Card title="Kafka" href="/goldrush-pipeline-api/destinations/kafka">
    Publish raw data to Kafka topics for downstream consumers, stream processing, and event-driven architectures.
  </Card>

  <Card title="Object Storage" href="/goldrush-pipeline-api/destinations/object-storage">
    Write data to S3, GCS, or Cloudflare R2 as Parquet or JSON files for data lake and batch processing workflows.
  </Card>

  <Card title="AWS SQS" href="/goldrush-pipeline-api/destinations/sqs">
    Deliver events to SQS queues for reliable, decoupled message processing with at-least-once delivery.
  </Card>

  <Card title="Webhook" href="/goldrush-pipeline-api/destinations/webhook">
    Receive HTTP POST callbacks for each event, enabling lightweight integrations without managing infrastructure.
  </Card>
</CardGroup>

## Key Features

<CardGroup cols={3}>
  <Card title="ABI Decoding" href="/goldrush-pipeline-api/abi-decoding/overview">
    Automatically decode raw EVM event logs into structured, typed columns using standard Solidity ABI definitions.
  </Card>

  <Card title="SQL Transforms" href="/goldrush-pipeline-api/sql-transforms">
    Apply SQL expressions to filter rows, select specific columns, or compute derived fields before data reaches your destination. Reduce storage costs and processing overhead at the source.
  </Card>

  <Card title="Protocol-Native Schemas" href="/goldrush-pipeline-api/normalizers/overview">
    Purpose-built normalizers for HyperCore and Solana DEX data that produce clean, protocol-aware tables and not raw byte arrays.
  </Card>
</CardGroup>

## Manage pipelines programmatically

Pipelines can be managed through the [GoldRush Platform UI](https://goldrush.dev/platform/) or the [Pipeline REST API](/goldrush-pipeline-api/rest-api/overview). The REST API exposes the same operations - create, update, pause, resume, delete, and inspect status, logs, metrics, and destination health - and is authenticated with a [ServiceKey](/goldrush-pipeline-api/service-keys).

## Supported Chains

| Chain                 | Status |
| --------------------- | ------ |
| Ethereum              | Live   |
| Polygon               | Live   |
| BNB Smart Chain (BSC) | Live   |
| Avalanche C-Chain     | Live   |
| Optimism              | Live   |
| Fantom                | Live   |
| ADI Chain             | Live   |
| Arbitrum              | Live   |
| Axie/Ronin            | Live   |
| Base                  | Live   |
| Blast                 | Live   |
| Celo                  | Live   |
| HyperCore             | Live   |
| HyperEVM              | Live   |
| Ink                   | Live   |
| Monad                 | Live   |
| Moonbeam              | Live   |
| Scroll                | Live   |
| Solana                | Live   |
| zkSync Era            | Live   |

<Info>
  For the full list of supported chains and available data entities, see the [Supported Chains](/goldrush-pipeline-api/supported-chains) page.
</Info>

## Pipeline API vs Foundational and Streaming

Each GoldRush API serves a different access pattern. Choose based on how your application consumes data.

|                   | Foundational API                                   | Streaming API                                  | Pipeline API                                       |
| ----------------- | -------------------------------------------------- | ---------------------------------------------- | -------------------------------------------------- |
| **Pattern**       | Pull (REST)                                        | Push (WebSocket)                               | Push (to your infra)                               |
| **Best for**      | On-demand queries, wallet lookups, historical data | Real-time UI updates, trading bots, live feeds | Data warehousing, analytics, ETL, backend indexing |
| **Data delivery** | Request/response                                   | WebSocket subscription                         | Continuous destination delivery                    |
| **Destination**   | Your application                                   | Your application                               | Your database, warehouse, queue, or webhook        |
| **Transforms**    | None (query parameters only)                       | None (filter on subscribe)                     | SQL transforms, ABI decoding, normalizers          |

<Tip>
  Use the **Foundational API** when you need data on demand. Use the **Streaming API** when you need real-time events in your application. Use the **Pipeline API** when you need blockchain data flowing continuously into your own infrastructure.
</Tip>