Greengage Database (GPDB) is an advanced, fully featured, open source data warehouse, based on PostgreSQL. It provides powerful and rapid analytics on petabyte scale data volumes. Uniquely geared toward big data analytics, Greengage Database is powered by the world’s most advanced cost-based query optimizer delivering high analytical query performance on large data volumes.

The Greengage project is released under the Apache 2 license. We want to thank all our past and present community contributors and are really interested in all new potential contributions. For the Greengage Database community no contribution is too small, we encourage all types of contributions.

Overview

A Greengage cluster consists of a master server, and multiple segment servers. All user data resides in the segments, the master contains only metadata. The master server, and all the segments, share the same schema.

Users always connect to the master server, which divides up the query into fragments that are executed in the segments, and collects the results.

More information can be found on the project website.

Building Greengage Database with GPORCA

GPORCA is a cost-based optimizer which is used by Greengage Database in conjunction with the PostgreSQL planner. It is also known as just ORCA, and Pivotal Optimizer. The code for GPORCA resides src/backend/gporca. It is built automatically by default.

Installing dependencies (for macOS developers)

Follow these macOS steps for getting your system ready for GPDB

Installing dependencies (for Linux developers)

Follow appropriate linux steps for getting your system ready for GPDB

Downloading submodules

Make sure that you download submodules. To do this, use git clone --recurse-submodules when downloading the source code. If you want to update the submodules, run:

git submodule update --init --recursive --force

Build the database

The recommended way to build the database is to use build system located in the gpAux directory, which is also used for CI testing and building packages.

To create optimized release build, use the following:

make GPROOT=~/build PARALLEL_MAKE_OPTS=-j8 dist -C gpAux

To run regression tests, a debug build with debug extensions is required. It can be built using the following command:

make GPROOT=~/build PARALLEL_MAKE_OPTS=-j8 devel -C gpAux

Load the greengage environment into your current shell:

source ~/build/greengage-db-devel/greengage_path.sh

Start demo cluster:

make create-demo-cluster

To use the demo cluster, source the environment variables from gpdemo-env.sh, which contains PGPORT and MASTER_DATA_DIRECTORY values:

source gpAux/gpdemo/gpdemo-env.sh

The directory, the TCP ports, the number of segments, and the existence of standbys for segments and coordinator for the demo cluster can be changed when starting the demo cluster. Instead of make create-demo-cluster, consider:

DATADIRS=/tmp/gpdb-cluster PORT_BASE=5555 NUM_PRIMARY_MIRROR_PAIRS=1 WITH_MIRRORS=false make create-demo-cluster

If you want to clean all generated files:

make distclean

Running tests

• By default, tests use the GPORCA optimizer:

make installcheck-world

• To turn GPORCA off and use the Postgres planner for query optimization:

PGOPTIONS='-c optimizer=off' make installcheck-world

• The TCP port for the regression test can be changed:

PGPORT=5555 make installcheck-world

• The top-level target installcheck-world will run all regression tests in GPDB against the running cluster. For testing individual parts, the respective targets can be run separately.

• The PostgreSQL check target does not work. Setting up a Greengage cluster is more complicated than a single-node PostgreSQL installation, and no-one's done the work to have make check create a cluster. Create a cluster manually or use gpAux/gpdemo/ (example below) and run the toplevel make installcheck-world against that. Patches are welcome!

• The PostgreSQL installcheck target does not work either, because some tests are known to fail with Greengage. The installcheck-good schedule in src/test/regress excludes those tests.

• When adding a new test, please add it to one of the GPDB-specific tests, in greengage_schedule, rather than the PostgreSQL tests inherited from the upstream. We try to keep the upstream tests identical to the upstream versions, to make merging with newer PostgreSQL releases easier.

Alternative Configurations

Internally, the configure script is used to adapt the build system to the characteristics of the host machine. For finer control over the components that are built, configuration options must be specified explicitly. The configuration options in effect are recorded in the file config.log and may be inspected with the following command:

head config.log

Building GPDB without GPORCA

Currently, GPDB is built with GPORCA by default. If you want to build GPDB without GPORCA, configure requires --disable-orca flag to be set.

Building GPDB with gpperfmon enabled

gpperfmon tracks a variety of queries, statistics, system properties, and metrics. To build with it enabled, change your configure to have an additional option --enable-gpperfmon

See more information about gpperfmon here

gpperfmon is dependent on several libraries like apr, apu, and libsigar

Building GPDB with Python3 enabled

GPDB supports Python3 with plpython3u UDF

See how to enable Python3 for details.

Building GPDB client tools on Windows

See Building GPDB client tools on Windows for details.

Development with Docker

See README.docker.md.

We provide a docker image with all dependencies required to compile and test GPDB (See Usage).

Development with Vagrant

There is a Vagrant-based quickstart guide for developers.

Code layout

The directory layout of the repository follows the same general layout as upstream PostgreSQL. There are changes compared to PostgreSQL throughout the codebase, but a few larger additions worth noting:

• gpMgmt/

  Contains Greengage-specific command-line tools for managing the cluster. Scripts like gpinit, gpstart, gpstop live here. They are mostly written in Python.

• gpAux/

  Contains Greengage-specific release management scripts, and vendored dependencies. Some additional directories are submodules and will be made available over time.

• gpcontrib/

  Much like the PostgreSQL contrib/ directory, this directory contains extensions such as gpfdist and gpmapreduce which are Greengage-specific.

• doc/

  In PostgreSQL, the user manual lives here. In Greengage, the user manual is maintained separately and only the reference pages used to build man pages are here.

• ci/

  Contains configuration files for the GPDB continuous integration system.

• src/backend/cdb/

  Contains larger Greengage-specific backend modules. For example, communication between segments, turning plans into parallelizable plans, mirroring, distributed transaction and snapshot management, etc. cdb stands for Cluster Database - it was a workname used in the early days. That name is no longer used, but the cdb prefix remains.

• src/backend/gpopt/

  Contains the so-called translator library, for using the GPORCA optimizer with Greengage. The translator library is written in C++ code, and contains glue code for translating plans and queries between the DXL format used by GPORCA, and the PostgreSQL internal representation.

• src/backend/gporca/

  Contains the GPORCA optimizer code and tests. This is written in C++. See README.md for more information and how to unit-test GPORCA.

• src/backend/fts/

  FTS is a process that runs in the master node, and periodically polls the segments to maintain the status of each segment.

Contributing

See CONTRIBUTING.md