OLTPBenchmark Support for Splice Machine

[bpmcpherson]

I'm looking to run some benchmarks on Splice Machine in order to do performance comparisons between it and a few other database products. OLTPBenchmark allows me to perform TPC-C on a number of existing products, but I was curious to know if it would be possible to either utilize the system as-is (given that Apache Derby is the general front-face of the product) or if it would be possible to implement at some point in the future?

Based on a presentation from September 2015 (see below), an older version of Splice Machine (before it's Spark integration) was able to perform a TPC-C benchmark. Assuming that there's little that needs to change on the front-end, I feel this would be a useful addition to the over-all software package. Additionally, there does exist a benchmark for TPC-H, although it's implementation differs from what I see from the various implementations in OLTPBenchmark.

Splice_Machine_Strata_NY_TPC-C_Benchmarks.pdf

[apavlo]

I have no experience with SpliceMachine. I also do not have time to add support for this DBMS. We usually take PRs from vendors to add them to the framework. Perhaps @jleach4 would know whether OLTP-Bench works?

[jleach4]

@apavlo @bpmcpherson ,

I have not tried Splice Machine on OLTPbenchmark unfortunately. We have a resource running OLTP benchmarks currently in our utility we forked from TPCC, I will ask him if he would be willing to generate a pull request so we can be added to the framework.

[bpmcpherson]

@apavlo @jleach4

Thank you both for the prompt feedback. I'm eager to see what results from this. Meanwhile, I will continue to look into TPC-C and other benchmarks for the platform so that I can have a rounded analysis.

[bpmcpherson]

@apavlo

Just to have a follow-up to this, I decided to try my hand at implementing Splice Machine and VoltDB into OLTPBench. I have used Issue Ticket #124 in addition to my own findings to make the necessary changes to allow these two products to work. This includes creating a configuration file, a ddl sql file, and editing the tpcc-dialects and DatabaseType.java files to accommodate these additions. Both feature JDBC drivers, which is how I am connecting.

However, I am encountering a SQLFeatureNotSupportedException (see below) when trying to load data into the database for VoltDB (I had a similar encounter with Splice Machine before encountering a strange JDBC connection error and have not gone to address at this time). Does this mean there are other files which need to be altered?

Provided below are text versions of files which I have created or edited. If I have overlooked something or need to provide additionally information, please let me know. Thanks.

VoltDB
volttpcc_config.txt
tpcc-voltdb-ddl.txt
OLTPBenchmark VoltDB Populate Database Error.docx

Splice Machine
splicetpcc_config.txt
tpcc-splice-ddl.txt

OLTPBenchmark
DatabaseType.txt
tpcc-dialects.txt

[apavlo]

Sorry, this slipped through the cracks on me because I was swamped with teaching / writing.

Are you still stuck on this? The SQLFeatureNotSupportedException error looks to be specific to VoltDB.

I will say that VoltDB has their own TPC-C implementation that you may want to use:

https://github.com/VoltDB/voltdb/tree/master/tests/test_apps/tpcc

[bpmcpherson]

It's not a problem. I have continued with trying to get both VoltDB and Splice Machine to work. I was still encountering the SQLFeatureNotSupportedException and had looked towards VoltDB's own implementation in the hopes of discovering what they are doing differently. I may have to simply use it for the time being.

Regarding Splice Machine, the error I am encountering is in regards to the Timestamp when trying to load the data. The error I'm getting (which I will have to provide when I'm at my personal computer) is that it's getting unknown data type in the Timestamp. I'm not sure if this has to do with how Splice Machine handles Timestamps or it is something else entirely.

Update: Here's what happens when I try to load the database. While the message below features a Date data type, it has happened with Timestamp as well. It is my assumption that in order to fix this issue, I'm going to have to address how Timestamps/Dates are created in TPCCBenchmark.java and see if I can provide it the right data.

Splice Machine Unknown Data Date Error.txt

[apavlo]

@bpmcpherson See #185