Auto-tuning RocksDB

Abstract

RocksDB is one of the most widely used embeddable persistent key-value stores available open-source. Its configurability, performance and workload flexibility have been essential factors that differentiate it from contenders. The data structure, Log Structured Merge Trees (LSM-trees), used in RocksDB differs from the more traditional B+ tree especially by offering better write throughput. However, the LSM-trees themselves do not provide a full-grown solution to all workloads, hence why there exist so many different databases implementing their own versions of the data structure.

Auto-tuning databases is in the wind, with examples like Oracle Autonomous Database and Peloton offering next to no configuration. RocksDB has also recently received tuning features like dynamically changing level sizes for Leveled Compaction and an auto-tuning rate limiter. However, RocksDB is known for dominating background activity by default and requires configuration for optimal performance for different workloads. This thesis evaluates an implementation of a compaction auto-tuner for RocksDB and presenting positive write performance gains during high write load. The research did also attract positive attention from the RocksDB developers at Facebook.