Chinese Journal of Computers   Full Text
Title	Evaluating GUI Systems Based on the Analysis of Process Execution Behaviors
Authors	SONG Bo1)，2) CHEN Ming-Yu1) FAN Jian-Ping1)
Address	1)(Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190) 2)(Graduate University of Chinese Academy of Sciences, Beijing 100049)
Year	2009
Issue	No.7(1393—1403)
Abstract &   Background	Abstract Traditional methodology with throughput as a performance metric is not appropriate for evaluating interactive applications on GUI-based systems, because non-determinate user input time takes up most of the total execution time. In this case, user’s perception is more important. In multi-user shared GUI systems, the resources available to a single user are limited, and the processing latency for user requests may be prolonged. So the subtraction of user input time from total time, namely the “real execution time”, can truly reflect the user-perceived performance. It is a difficult problem to extract the real executing time from the end-to-end time. This paper introduces a novel method of evaluating GUI systems based on the analysis of process execution behaviors, and presents a sectional maximum correlation algorithm, which is able to extract the real executing time accurately. In order to record kernel traces, a tool called Pro is designed and implemented, which can record pieces of trace produced during kernel profiling quickly and accurately, while introduces a quite small extra cost of time and kernel space. Four GUI-based applications on systems with different memory sizes are analyzed, and the experiment results fully demonstrate the veracity and the efficiency of this method. Keywords real execution time; GUI system; kernel profiling; process behaviors contrasting; sectional maximum correlation Background The traditional system evaluating methodology which focuses on throughput-related metrics is appropriate for benchmarking computing dense systems. However, it has great limitation for analyzing the performance of interactive applications on GUI-based systems, because on-determinate user response time takes up most of the total execution time. In multi-user shared GUI systems, the resources available to each user are quite limited, and the system response time for one request can be user-perceived. User interface studies indicate that a user’s perception should be the arbiter of performance. Interests in the research of quantifying interactive performance have been really recent. Endo et al. put forward event-latency as an evaluation metric in 1996. This metric was accepted by the subsequent research and much work has been done on extracting event-processing latency from the total time, e.g., the idle algorithm by Endo et al. and the VNCPlay tool by Zeldovic et al. However, to the best of our knowledge, there have been no methods that can do accurate extraction. In the paper, the authors use the “real execution time”, which is the sum of all event-latency, as a parameter to evaluate how efficiently interactive applications work on GUI-based systems. The authors introduce a novel method of evaluating GUI systems based on the analysis of process executing behaviors, and present a sectional maximum correlation algorithm, which can accurately extract the real executing time from the total time. Experiment results fully demonstrate the veracity and the efficiency of this method. This work is supported by the National Natural Science Foundation of China under grant No.60633040 with the title of “Hyper Parallel Computer Architecture Research”.