Authors: | Noll, Stefan |
Title: | Resource-efficient processing of large data volumes |
Language (ISO): | en |
Abstract: | The complex system environment of data processing applications makes it very challenging to achieve high resource efficiency. In this thesis, we develop solutions that improve resource efficiency at multiple system levels by focusing on three scenarios that are relevant—but not limited—to database management systems. First, we address the challenge of understanding complex systems by analyzing memory access characteristics via efficient memory tracing. Second, we leverage information about memory access characteristics to optimize the cache usage of algorithms and to avoid cache pollution by applying hardware-based cache partitioning. Third, after optimizing resource usage within a multicore processor, we optimize resource usage across multiple computer systems by addressing the problem of resource contention for bulk loading, i.e., ingesting large volumes of data into the system. We develop a distributed bulk loading mechanism, which utilizes network bandwidth and compute power more efficiently and improves both bulk loading throughput and query processing performance. |
Subject Headings: | Resource efficiency Main-memory database systems Memory tracing CPU cache partitioning Bulk loading |
Subject Headings (RSWK): | Ressourceneffizienz Datenbanksystem Ablaufverfolgung Cache-Speicher |
URI: | http://hdl.handle.net/2003/40058 http://dx.doi.org/10.17877/DE290R-21938 |
Issue Date: | 2021 |
Appears in Collections: | LS 06 Datenbanken und Informationssysteme |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
noll_phd_thesis.pdf | DNB | 2.33 MB | Adobe PDF | View/Open |
This item is protected by original copyright |
This item is protected by original copyright rightsstatements.org