Skip to content

Latest commit

 

History

History
53 lines (36 loc) · 3.5 KB

introduction.mdx

File metadata and controls

53 lines (36 loc) · 3.5 KB
Raw
title description
PeerDB
Welcome to PeerDB docs. Here you'll find everything you need to get started with PeerDB.
PeerDB is now natively integrated with ClickHouse Cloud!

We recently announced the Postgres CDC connector in ClickPipes, now available in Private Preview. This connector is fully powered by PeerDB.

As a ClickHouse Cloud customer, you get a seamless, native experience for moving data from your Postgres databases to ClickHouse Cloud. Simply navigate to the Data Sources tab and add a Postgres source to start ingesting data.

👉 You can sign up to the private preview by following this link. Our team will provide access within a day or 2.

What is PeerDB?

At PeerDB, we are building a fast, simple and the most cost effective way to stream data from Postgres to Data Warehouses, Queues and Storage engines. If you are running Postgres at the heart of your data-stack and move data at scale from Postgres to any of the above targets, PeerDB can provide value.

We support different modes of streaming - log based (CDC), cursor based (timestamp or integer) and XMIN based. Performance wise, we are 10x faster than existing tools. Features wise, we support native Postgres features such as comprehensive set of data-types incl. jsonb/arrays/postgis, efficiently streaming toast columns, schema changes and so on.

PeerDB is Free and Open

PeerDB is free to use and deploy. It's licensed under Elastic License 2.0 (ELv2). Here goes the link to our github repo: https://github.com/PeerDB-io/peerdb

Why PeerDB?

Existing ETL tools primarily focus on supporting a wide range of data-stores. However, they fall short in providing a rich experience for any two specific data-stores. This becomes evident when your workloads need scale or have demanding feature requirements. It is common for such users to try out these tools and fail – tools not meeting their performance and reliability SLAs or lacking the required features. Such users resort to developing their own in-house solutions, investing a lot of time and resources.

Data-store nativity at it’s core, enabling scalable ETL

PeerDB takes a data-store first approach to ETL. It supports a set of highly adopted stores, implements multiple infrastructural and data-store native optimizations, providing a highly scalable and a feature-rich ETL experience. For example, in a sync from Postgres to BigQuery or Snowflake, PeerDB is 10 times faster than other tools. We are database experts and believe that an ETL tool should be datastore centric, than a hodge-podge of too many connectors.

Postgres-compatible SQL interface to do ETL

The Postgres-compatible SQL interface for ETL is unique to PeerDB and enables you to operate in a language you are familiar with. You can do ETL the same way you work with your databases.

You can use Postgres’ eco-system to manage your ETL —

  1. Client tools like pgadmin, psql to run SQL commands.
  2. BI tools like grafana, tableau to visually monitor syncs and transforms.
  3. Database migration and versioning tools like Flyway to manage your ETL.
  4. Any language (Python, Go, Node.JS etc) and Scheduler (AirFlow) for development.
  5. And many more
Get started with PeerDB in less than 5 minutes. Learn about the architecture of PeerDB.