Auto-tuning RocksDB
Abstract
RocksDB is one of the most widely used embeddable persistent key-value storesavailable open-source. Its configurability, performance and workload flexibility havebeen essential factors that differentiate it from contenders. The data structure,Log Structured Merge Trees (LSM-trees), used in RocksDB differs from the moretraditional B+ tree especially by offering better write throughput. However, theLSM-trees themselves do not provide a full-grown solution to all workloads, hencewhy there exist so many different databases implementing their own versions of thedata structure.
Auto-tuning databases is in the wind, with examples like Oracle AutonomousDatabase and Peloton offering next to no configuration. RocksDB has alsorecently received tuning features like dynamically changing level sizes for LeveledCompaction and an auto-tuning rate limiter. However, RocksDB is known fordominating background activity by default and requires configuration for optimalperformance for different workloads. This thesis evaluates an implementation ofa compaction auto-tuner for RocksDB and presenting positive write performancegains during high write load. The research did also attract positive attention fromthe RocksDB developers at Facebook.