Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: README update to help new users #370

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Docs: README update to help new users #370

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3c3607d
Update README.md
ETHenzlere Oct 1, 2023
5739de6
Merge branch 'cmu-db:main' into docs/documentation-update
ETHenzlere Oct 3, 2023
44fac74
Split Prerequisites into two sections
ETHenzlere Oct 3, 2023
f6a9465
Merge branch 'main' into docs/documentation-update
bpkroth Oct 9, 2023
1605b46
Merge branch 'main' into docs/documentation-update
apavlo Oct 19, 2023
5da6c30
PR comments + Formatting
ETHenzlere Oct 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
199 changes: 113 additions & 86 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,33 @@ BenchBase (formerly [OLTPBench](https://github.com/oltpbenchmark/oltpbench/)) is

**Table of Contents**

- [Quickstart](#quickstart)
- [Description](#description)
- [Usage Guide](#usage-guide)
- [Contributing](#contributing)
- [Known Issues](#known-issues)
- [Credits](#credits)
- [Citing This Repository](#citing-this-repository)
- [Prerequisites](#prerequisites)
- [Quickstart](#quickstart)
- [Description](#description)
- [Usage Guide](#usage-guide)
- [Contributing](#contributing)
- [Known Issues](#known-issues)
- [Credits](#credits)
- [Citing This Repository](#citing-this-repository)

---

## Prerequisites

### Non-Containerized

Java 17 needs to be installed on the machine in order to run the benchbase commands.

Java can be downloaded from the [official download page.](https://www.oracle.com/java/technologies/downloads/#java17)

### Containerized

Benchbase supports the usage inside of docker containers.

Docker can be downloaded from the [official download page.](https://docs.docker.com/get-docker/)

> See the [docker section](#how-to-use-with-docker) for more details on how to use benchbase with docker.

## Quickstart

To clone and build BenchBase using the `postgres` profile,
Expand Down Expand Up @@ -60,24 +77,24 @@ features, e.g., per-transaction-type latency and throughput logs.

The BenchBase framework has the following benchmarks:

* [AuctionMark](https://github.com/cmu-db/benchbase/wiki/AuctionMark)
* [CH-benCHmark](https://github.com/cmu-db/benchbase/wiki/CH-benCHmark)
* [Epinions.com](https://github.com/cmu-db/benchbase/wiki/epinions)
* hyadapt -- pending configuration files
* [NoOp](https://github.com/cmu-db/benchbase/wiki/NoOp)
* [OT-Metrics](https://github.com/cmu-db/benchbase/wiki/OT-Metrics)
* [Resource Stresser](https://github.com/cmu-db/benchbase/wiki/Resource-Stresser)
* [SEATS](https://github.com/cmu-db/benchbase/wiki/Seats)
* [SIBench](https://github.com/cmu-db/benchbase/wiki/SIBench)
* [SmallBank](https://github.com/cmu-db/benchbase/wiki/SmallBank)
* [TATP](https://github.com/cmu-db/benchbase/wiki/TATP)
* [TPC-C](https://github.com/cmu-db/benchbase/wiki/TPC-C)
* [TPC-H](https://github.com/cmu-db/benchbase/wiki/TPC-H)
* TPC-DS -- pending configuration files
* [Twitter](https://github.com/cmu-db/benchbase/wiki/Twitter)
* [Voter](https://github.com/cmu-db/benchbase/wiki/Voter)
* [Wikipedia](https://github.com/cmu-db/benchbase/wiki/Wikipedia)
* [YCSB](https://github.com/cmu-db/benchbase/wiki/YCSB)
- [AuctionMark](https://github.com/cmu-db/benchbase/wiki/AuctionMark)
- [CH-benCHmark](https://github.com/cmu-db/benchbase/wiki/CH-benCHmark)
- [Epinions.com](https://github.com/cmu-db/benchbase/wiki/epinions)
- hyadapt -- pending configuration files
- [NoOp](https://github.com/cmu-db/benchbase/wiki/NoOp)
- [OT-Metrics](https://github.com/cmu-db/benchbase/wiki/OT-Metrics)
- [Resource Stresser](https://github.com/cmu-db/benchbase/wiki/Resource-Stresser)
- [SEATS](https://github.com/cmu-db/benchbase/wiki/Seats)
- [SIBench](https://github.com/cmu-db/benchbase/wiki/SIBench)
- [SmallBank](https://github.com/cmu-db/benchbase/wiki/SmallBank)
- [TATP](https://github.com/cmu-db/benchbase/wiki/TATP)
- [TPC-C](https://github.com/cmu-db/benchbase/wiki/TPC-C)
- [TPC-H](https://github.com/cmu-db/benchbase/wiki/TPC-H)
- TPC-DS -- pending configuration files
- [Twitter](https://github.com/cmu-db/benchbase/wiki/Twitter)
- [Voter](https://github.com/cmu-db/benchbase/wiki/Voter)
- [Wikipedia](https://github.com/cmu-db/benchbase/wiki/Wikipedia)
- [YCSB](https://github.com/cmu-db/benchbase/wiki/YCSB)

This framework is design to allow for easy extension. We provide stub code that a contributor can use to include a new
benchmark, leveraging all the system features (logging, controlled speed, controlled mixture, etc.)
Expand All @@ -87,31 +104,36 @@ benchmark, leveraging all the system features (logging, controlled speed, contro
## Usage Guide

### How to Build
Run the following command to build the distribution for a given database specified as the profile name (`-P`). The following profiles are currently supported: `postgres`, `mysql`, `mariadb`, `sqlite`, `cockroachdb`, `phoenix`, and `spanner`.

Run the following command to build the distribution for a given database specified as the profile name (`-P`). The following profiles are currently supported: `postgres`, `mysql`, `mariadb`, `sqlite`, `cockroachdb`, `phoenix`, and `spanner`.

```bash
./mvnw clean package -P <profile name>
```

The following files will be placed in the `./target` folder:

* `benchbase-<profile name>.tgz`
* `benchbase-<profile name>.zip`
- `benchbase-<profile name>.tgz`
- `benchbase-<profile name>.zip`

### How to Run
Once you build and unpack the distribution, you can run `benchbase` just like any other executable jar. The following examples assume you are running from the root of the expanded `.zip` or `.tgz` distribution. If you attempt to run `benchbase` outside of the distribution structure you may encounter a variety of errors including `java.lang.NoClassDefFoundError`.

Once you build and unpack the distribution, you can run `benchbase` just like any other executable jar. The following examples assume you are running from the root of the expanded `.zip` or `.tgz` distribution. If you attempt to run `benchbase` outside of the distribution structure you may encounter a variety of errors including `java.lang.NoClassDefFoundError`.

To bring up help contents:

```bash
java -jar benchbase.jar -h
```

To execute the `tpcc` benchmark:

```bash
java -jar benchbase.jar -b tpcc -c config/postgres/sample_tpcc_config.xml --create=true --load=true --execute=true
```

For composite benchmarks like `chbenchmark`, which require multiple schemas to be created and loaded, you can provide a comma separated list:

```bash
java -jar benchbase.jar -b tpcc,chbenchmark -c config/postgres/sample_chbenchmark_config.xml --create=true --load=true --execute=true
```
Expand Down Expand Up @@ -151,6 +173,11 @@ mvn clean compile exec:java -P postgres -Dexec.args="-b tpcc -c config/postgres/

this is equivalent to the steps above but eliminates the need to first package and then extract the distribution.

### How to configure Connections & Workloads

The configuration files are located in the `/config` folder and can be used to adapt the benchbase config to the user's system of choice.
A more detailed explanation can be found [here](/config/README.md).

### How to Enable Logging

To enable logging, e.g., for the PostgreSQL JDBC driver, add the following JVM property when starting...
Expand All @@ -168,46 +195,46 @@ To modify the logging level you can update [`logging.properties`](src/main/resou
./mvnw -B release:perform
```

### How use with Docker
### How to use with Docker

- Build or pull a dev image to help building from source:
- Build or pull a dev image to help building from source:

```sh
./docker/benchbase/build-dev-image.sh
./docker/benchbase/run-dev-image.sh
```
```sh
./docker/benchbase/build-dev-image.sh
./docker/benchbase/run-dev-image.sh
```

or
or

```sh
docker run -it --rm --pull \
-v /path/to/benchbase-source:/benchbase \
-v $HOME/.m2:/home/containeruser/.m2 \
benchbase.azure.cr.io/benchbase-dev
```
```sh
docker run -it --rm --pull \
-v /path/to/benchbase-source:/benchbase \
-v $HOME/.m2:/home/containeruser/.m2 \
benchbase.azure.cr.io/benchbase-dev
```

- Build the full image:
- Build the full image:

```sh
# build an image with all profiles
./docker/benchbase/build-full-image.sh
```sh
# build an image with all profiles
./docker/benchbase/build-full-image.sh

# or if you only want to build some of them
BENCHBASE_PROFILES='postgres mysql' ./docker/benchbase/build-full-image.sh
```
# or if you only want to build some of them
BENCHBASE_PROFILES='postgres mysql' ./docker/benchbase/build-full-image.sh
```

- Run the image for a given profile:
- Run the image for a given profile:

```sh
BENCHBASE_PROFILE='postgres' ./docker/benchbase/run-full-image.sh --help # or other benchbase args as before
```
```sh
BENCHBASE_PROFILE='postgres' ./docker/benchbase/run-full-image.sh --help # or other benchbase args as before
```

or
or

```sh
docker run -it --rm --env BENCHBASE_PROFILE='postgres' \
-v results:/benchbase/results benchbase.azurecr.io/benchbase --help # or other benchbase args as before
```
```sh
docker run -it --rm --env BENCHBASE_PROFILE='postgres' \
-v results:/benchbase/results benchbase.azurecr.io/benchbase --help # or other benchbase args as before
```

> See the [docker/benchbase/README.md](./docker/benchbase/) for further details.

Expand All @@ -223,9 +250,9 @@ Please see the existing MySQL and PostgreSQL code for an example.

We welcome all contributions! Please open a pull request. Common contributions may include:

- Adding support for a new DBMS.
- Adding more tests of existing benchmarks.
- Fixing any bugs or known issues.
- Adding support for a new DBMS.
- Adding more tests of existing benchmarks.
- Fixing any bugs or known issues.

## Known Issues

Expand All @@ -239,31 +266,31 @@ The original OLTPBench code was largely written by the authors of the original p

A significant portion of the modernization was contributed by [Tim Veil @ Cockroach Labs](https://github.com/timveil-cockroach), including but not limited to:

* Built with and for Java ~~11~~ 17.
* Migration from Ant to Maven.
* Reorganized project to fit Maven structure.
* Removed static `lib` directory and dependencies.
* Updated required dependencies and removed unused or unwanted dependencies.
* Moved all non `.java` files to standard Maven `resources` directory.
* Shipped with [Maven Wrapper](https://maven.apache.org/wrapper).
* Improved packaging and versioning.
* Moved to Calendar Versioning (https://calver.org/).
* Project is now distributed as a `.tgz` or `.zip` with an executable `.jar`.
* All code updated to read `resources` from inside `.jar` instead of directory.
* Moved from direct dependence on Log4J to SLF4J.
* Reorganized and renamed many files for clarity and consistency.
* Applied countless fixes based on "Static Analysis".
* JDK migrations (boxing, un-boxing, etc.).
* Implemented `try-with-resources` for all `java.lang.AutoCloseable` instances.
* Removed calls to `printStackTrace()` or `System.out.println` in favor of proper logging.
* Reformatted code and cleaned up imports.
* Removed all calls to `assert`.
* Removed various forms of dead code and stale configurations.
* Removed calls to `commit()` during `Loader` operations.
* Refactored `Worker` and `Loader` usage of `Connection` objects and cleaned up transaction handling.
* Introduced [Dependabot](https://dependabot.com/) to keep Maven dependencies up to date.
* Simplified output flags by removing most of them, generally leaving the reporting functionality enabled by default.
* Provided an alternate `Catalog` that can be populated directly from the configured Benchmark database. The old catalog was proxied through `HSQLDB` -- this remains an option for DBMSes that may have incomplete catalog support.
- Built with and for Java ~~11~~ 17.
- Migration from Ant to Maven.
- Reorganized project to fit Maven structure.
- Removed static `lib` directory and dependencies.
- Updated required dependencies and removed unused or unwanted dependencies.
- Moved all non `.java` files to standard Maven `resources` directory.
- Shipped with [Maven Wrapper](https://maven.apache.org/wrapper).
- Improved packaging and versioning.
- Moved to Calendar Versioning (https://calver.org/).
- Project is now distributed as a `.tgz` or `.zip` with an executable `.jar`.
- All code updated to read `resources` from inside `.jar` instead of directory.
- Moved from direct dependence on Log4J to SLF4J.
- Reorganized and renamed many files for clarity and consistency.
- Applied countless fixes based on "Static Analysis".
- JDK migrations (boxing, un-boxing, etc.).
- Implemented `try-with-resources` for all `java.lang.AutoCloseable` instances.
- Removed calls to `printStackTrace()` or `System.out.println` in favor of proper logging.
- Reformatted code and cleaned up imports.
- Removed all calls to `assert`.
- Removed various forms of dead code and stale configurations.
- Removed calls to `commit()` during `Loader` operations.
- Refactored `Worker` and `Loader` usage of `Connection` objects and cleaned up transaction handling.
- Introduced [Dependabot](https://dependabot.com/) to keep Maven dependencies up to date.
- Simplified output flags by removing most of them, generally leaving the reporting functionality enabled by default.
- Provided an alternate `Catalog` that can be populated directly from the configured Benchmark database. The old catalog was proxied through `HSQLDB` -- this remains an option for DBMSes that may have incomplete catalog support.

## Citing This Repository

Expand Down
69 changes: 69 additions & 0 deletions config/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Config

The config files are duplicated in a seperate folder for each of the supported database systems.

For example, the postgres/sibench config file has the following pieces:

## Connection Details

The connection details allow for the configuration of the connection to the database via JDBC.

```xml
<!-- Connection details -->
<type>POSTGRES</type>
<driver>org.postgresql.Driver</driver>
<url>jdbc:postgresql://localhost:5432/benchbase?sslmode=disable&amp;ApplicationName=sibench&amp;reWriteBatchedInserts=true</url>
<username>admin</username>
<password>password</password>
<isolation>TRANSACTION_SERIALIZABLE</isolation>
<batchsize>128</batchsize>
```

## Workload

- `scalefactor` allows to upscale the workload by the specified factor
- `time` allows to modify the runtime of the workload. It is given in **seconds**
- `weights` define the weight of each [transaction Type](#procedures). The weights are defined as **percentages**

```xml

<scalefactor>1</scalefactor>

<!-- The workload -->
<terminals>1</terminals>
<works>
<work>
<time>60</time>
<rate>unlimited</rate>
<weights>50,50</weights>
</work>
</works>
```

## Procedures

Each workload specifies certain procedures which can be removed by deleting the `transactionType` from the config. By default, all transaction types of the workload are listed in an unmodified config file.

```xml
<!-- SIBENCH Procedures declaration -->
<transactiontypes>
<transactiontype>
<name>MinRecord</name>
</transactiontype>
<transactiontype>
<name>UpdateRecord</name>
</transactiontype>
</transactiontypes>
</parameters>

```

## Workload Specific Configs

### Templated

The templated workload demands a path to a file that holds the templated queries.

```xml
<query_templates_file>data/templated/example.xml</query_templates_file>
```