Chinese Journal of Computers   Full Text
Title	The Key Theorem and the Bounds on the Rate of Uniform Convergence of Statistical Learning Theory on Quasi-Probability Spaces
Authors	HA Ming-Hu1) FENG Zhi-Fang2) SONG Shi-Ji3) GAO Lin-Qing1)
Address	1)(College of Mathematics and Computer Sciences, Hebei University, Baoding, Hebei 071002) 2)(Department of Mathematics and Information Sciences, Langfang Normal School, Langfang, Hebei 065000) 3)(Department of Automation, Tsinghua University, Beijing 100084)
Year	2008
Issue	No.3(476—485)
Abstract &   Background	Abstract Some properties of quasi-probability are further discussed. The definitions and properties of quasi-random variable and its distribution function, expected value and variance are then presented. Markov inequality, Chebyshev’s inequality and the Khinchine’s law of large numbers on quasi-probability spaces are also proved. Then the key theorem of learning theory on quasi-probability spaces is proved, and the bounds on the rate of uniform convergence of learning process on quasi-probability spaces are constructed. The investigations will help lay essential theoretical foundations for the systematic and comprehensive development of the quasi-statistical learning theory. keywords quasi-probability; empirical risk functional; expected risk functional; key theorem; bounds on the rate of uniform convergence background This work is supported by the National Natural Science Foundation of China "Statistical Learning Theory Based on Uncertain Samples" (grant No.60573069 ) and "Study of Uncertain Statistical Learning Theory" (grant No.60773062). Statistical Learning Theory or SLT, which deals mainly with the statistical learning principles when samples are limited, is proposed in the 1970s by Vladimir N.Vapnik. It provides important theoretical framework for studying the theory and methods of machine learning when samples are limited, and its kernel idea is to control generalization ability of learning machine by capacity control. SLT provides a solid theoretical foundation for people to study the problem of learning machine when samples are limited, also Support Vector Machine is a new and generic method of learning machine which is developed from this theoretical foundation, and is successfully applied in biological information technology, images graphics processing and economic forecasting. At present, SLT and SVM have become a hot research issue in the field of machine learning. Though SLT is generally acknowledged as a better learning theory handling a small sample in academe, there exist some shortcomings yet. For example, SLT is provided on the probability (probability measure, a kind of special measure) spaces. As we all know, probability is a set function which satisfies additive or accountable additive on probability spaces. This condition is so strict that sometimes it can’t be satisfied in practical application. In other words, there exist a large number of non-additive set functions. This paper puts forward the concept of Uncertain Statistical Learning Theory. Nowadays, the authors have finished the basic theory research of uncertain Statistical Learning Theory, and have got a series of results on construction and applications of the multistage Support Vector Machine, the Support Vector Machine inverse problem and fuzzy optimization problems. In the paper, the authors further discuss quasi-probability’s proposition, and prove the key theorem of learning theory and the bounds on the rate on uniform convergence of learning process based on quasi-probability space. They expand to scope of theory and application of SLT and build the theoretical foundation of study of SLT on quasi-probability space.