-
Notifications
You must be signed in to change notification settings - Fork 187
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce templated benchmarks. #301
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
anjagruenheid
changed the title
Draft for templated benchmarks.
Introduce templated benchmarks.
Apr 20, 2023
bpkroth
reviewed
Apr 21, 2023
bpkroth
reviewed
Apr 21, 2023
bpkroth
reviewed
Apr 21, 2023
src/main/java/com/oltpbenchmark/benchmarks/templated/TemplatedBenchmark.java
Show resolved
Hide resolved
bpkroth
reviewed
Apr 21, 2023
src/main/java/com/oltpbenchmark/benchmarks/templated/TemplatedBenchmark.java
Outdated
Show resolved
Hide resolved
bpkroth
reviewed
Apr 21, 2023
src/main/java/com/oltpbenchmark/benchmarks/templated/util/TraceTransactionGenerator.java
Outdated
Show resolved
Hide resolved
bpkroth
reviewed
Apr 21, 2023
src/main/java/com/oltpbenchmark/benchmarks/templated/util/GenericQueryOperation.java
Outdated
Show resolved
Hide resolved
bpkroth
reviewed
Apr 21, 2023
src/main/java/com/oltpbenchmark/benchmarks/templated/TemplatedWorker.java
Outdated
Show resolved
Hide resolved
bpkroth
reviewed
Apr 21, 2023
src/main/java/com/oltpbenchmark/benchmarks/templated/TemplatedBenchmark.java
Outdated
Show resolved
Hide resolved
bpkroth
reviewed
Apr 21, 2023
src/main/java/com/oltpbenchmark/benchmarks/templated/TemplatedBenchmark.java
Show resolved
Hide resolved
bpkroth
reviewed
Apr 21, 2023
…Worker.java Co-authored-by: Brian Kroth <[email protected]>
…ricQueryOperation.java Co-authored-by: Brian Kroth <[email protected]>
…id/benchbase into templated_benchmarks
…eTransactionGenerator.java Co-authored-by: Brian Kroth <[email protected]>
bpkroth
reviewed
Apr 27, 2023
bpkroth
reviewed
Sep 26, 2023
src/main/java/com/oltpbenchmark/benchmarks/templated/TemplatedBenchmark.java
Show resolved
Hide resolved
Templated benchmarks improvements
bpkroth
reviewed
Oct 2, 2023
bpkroth
approved these changes
Oct 2, 2023
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is ready to go.
It's got lots of tests, both unit and functional for several major DBs now.
The only remaining issue was support for NULLable fields and we filed #366 to track that in future work.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces templated benchmarks for BenchBase. In essence, users can define their own query templates that are parametrized with one or more values at runtime. If multiple parameter combinations are possible, we will continue to cycle through all available combinations.
A templated benchmark has the structure
where $ParameterType is the integer java.sql.Types value (i.e., 4 for integer, 16 for boolean etc.) and each value tag within 'parameter_values' contains the values for one instantiation of the parameters set in $SQLQuery. The SQL query string is read as a PreparedStatement, i.e., parameters are defined in the string via a '?' placeholder. An example for a templated benchmark can be found in data/templated/example.xml. The file path for the XML template has to be defined in the workload configuration using the 'query_templates_file' tag. An example configuration can be found in config/sqlserver/sample_template_config.xml. The example can be executed if a loaded TPC-C instance is used as JDBC endpoint.
Templated benchmarks are instantiated using 'templated' as benchmark class when running BenchBase via the command line.