Chinese Journal of Computers   Full Text
Title	An Efficient Checkpointing Scheme for Update Intensive Applications in Main Memory Database Systems
Authors	QIN Xiong-Pai1),2) XIAO Yan-Qin1),2) CAO Wei1),2) WANG Shan1),2)
Address	1)(Key Laboratory of Data Engineering and Knowledge Engineering (Renmin University of China), MOE, Beijing 100872) 2)(School of Information, Renmin University of China, Beijing 100872)
Year	2009
Issue	No.11(2200—2210)
Abstract &   Background	Abstract Four requirements for checkpointing in update intensive main memory database systems are identified, namely how checkpointing interferes with normal transaction processing, how checkpointing handles skewed access pattern, whether checkpointing supports fast restart, and whether checkpointing provides system availability during recovery. A partition based transaction-consistent checkpoint scheme is proposed in this paper, the scheme uses a tuple-level dynamic multi-versioning concurrency control protocol to avoid the lock conflicts between read only transactions and update transactions, thus achieves high system throughput. Checkpointing is implemented as data partition read only transactions, and incurs little interfering with normal transaction processing, furthermore transaction-consistent checkpoint requires writing out only redo log records of committed transactions, during recovery only one pass of log scanning is needed. Priority-based on demand partition recovery gives high attention to data access demands of executing transactions, provides system availability during recovery, which is critical for high performance applications. Finally, two level version management and dynamic version sharing reduce space overhead to an acceptable level. Experiment results demonstrate effectiveness and efficiency of the proposed scheme. Keywords main memory database systems; dynamic multi-versioning; transaction-consistent checkpoint; tuple level Background In update intensive applications, main memory database systems produce large volume of log records, periodically checkpointing is necessary to reclaim log disk space, in the meantime, vulnerability of physical main memory makes checkpointing important for the recoverability of database systems. Fuzzy checkpointing variants for main memory database systems have been deeply investigated since 1980s. Little could be done to improve them. The major disadvantage of fuzzy checkpointing is that two passes of log scanning are needed to restore the database, and new transactions cannot get executed until all log records are processed. Consistent checkpointing is regaining research effort in recent years mainly due to larger and cheaper memory, consistent checkpointing has several nice properties, it enables redo only logging which speedups system restart, partition based checkpointing supports partition recovery which is critical for system availability during recovery. The capacity of memory gets larger, it is reasonable to use some memory in multi versioning for consistent checkpointing. A partition based transaction consistent checkpointing scheme (PB-TCC-DV) is proposed. A tuple-level dynamic multi-versioning concurrency control protocol is used to schedule transactions. Under the control of the protocol, checkpointing activities incurs little interfering with normal transaction processing, the system achieves high throughput. The two-level version management scheme and dynamic version sharing technique are used to further reduce space overhead. The experiment results show that PB-TCC-DV checkpointing scheme outperforms Fuzzy checkpointing by about 27% with 10.99% space overhead incurred. The scheme excels in terms of interfere with normal transaction processing, supporting fast restart, providing system availability during recovery, as well as handling skew access pattern. The research is supported by the National Natural Science Foundation of China under Grant No. 60496325 and No. 60503038. Also it is partially supported by the international cooperation program with HP Lab China (The Project of Large Scale Data Management). The authors are exploring novel, effective, and efficient ways to mange large scale of data under both the rapid improvements of hardware and the more and more challenging demands of database applications. The team has made much progresses and achievements in this research area. They have setup a main memory database test bed that is based on MonetDB as well as a simulation environment, and have published a number of papers which are involved in database performance behaviors on different hardware architecture (CPUs), query processing techniques for new hardware, domain benchmark for main memory database systems, parallel recovery of main memory database and so on. The research in this paper is focusing on looking for a checkpointing scheme with high efficiency for update intensive main memory database systems. The experiment results show that the scheme is superior over fuzzy checkpointing scheme with acceptable space overhead.