QCLab: a framework for query compilation on modern hardware platforms
Loading...
Date
2022
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
As modern in-memory database systems achieve higher and higher processing
speeds, the performance of memory becomes an increasingly limiting factor. Although there has been significant progress, the bottleneck only has shifted. While
earlier systems were optimized for memory latencies, current systems are rather
affected by the limited memory bandwidth.
Query compilation is a proven technique to address bandwidth limitations.
It translates queries via Just-In-Time compilation to native programs for the target
hardware. The compiled queries execute with very high efficiency and only
with a bare minimum of communication via memory. Despite these important
improvements, the benefit of query compilation in certain scenarios is limited.
On the one hand query compilers typically use standard compiler technology
with relatively long compilation times. Therefore the overall execution time can be
prolonged by the additional compilation time. On the other hand, not all emerging
database technology is compatible with the approach. Query compilation uses a
tuple-at-a-time processing style that departs from the column-at-a-time or vector-at-
a-time approaches that in-memory systems typically use. Especially data-parallel
processing techniques, e.g. SIMD or coprocessing-techniques, are challenging to
use in combination with the approach.
This work presents QCLab, a framework for query compilation on modern hardware
platforms. The framework contains several new query compilation techniques
that allow us to address the mentioned shortcomings and ultimately to extend the
benefit of query compilation to new workloads and platforms. The techniques
cover three aspects: compilation, communication, and processing. Together they
serve as basis for building highly efficient query compilers. The techniques make
efficient use of communication channels and of the large processing capacities
of modern systems. They were designed for practical use and enable efficient
processing, even when workload characteristics are challenging.
Description
Table of contents
Keywords
Query compilation, Modern hardware, Parallel data processing