The Merlin OMS Benchmark - Definition, Implementations and Results

Abstract

The bottleneck of highly integrated software development environments (SDEs) could still be the use of an underlying central data store. An environment is called highly integrated, if the included tools support most of the life-cycle phases and especially inform, analyse and check document interdependencies and even propagate changes on basically any level of granularity. In that case a central data store for documents and document interdependencies is the necessary key information base. A number of so-called object management systems (OMSs) have recently been developed to address the needs of SDEs. In order to support an environment builder to select the most appropriate one, this paper presents a new benchmark for such databases. We precisely define that benchmark in an abstract way in terms of its conceptual schema, possible initial database states and the benchmark operations. The benchmark was implemented twice to allow for a comparison of the OMSs GRAS and GemStone. We describe these two implementations as well as the results we gained when we performed the benchmark.