Chinese Journal of Computers   Full Text
Title	The Relation Between Intrinsic Complexity and Generalization of a Model and the Geometric Curvature
Authors	LU Zi-Ang LUO Si-Wei YANG Jian LIU Yun-Hui ZOU Qi
Address	(School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044)
Year	2007
Issue	No.7(1094—1103)
Abstract &   Background	Abstract The paper uses the conception of curvature from the point of view of differential geometry to explore the intrinsic model complexity that is free of reparametrization; and then through theoretical analysis, shows that the Gauss-Kroneker curvature can describe the whole properties of the statistical manifold, thus gives the relation between curvature and the volume of the manifold. An algorithm is proposed based on study of the solution locus in the neighborhood of the expectation of parameters to calculate the curvature of the model. This paper proves that the future residual that is qualified to measure the generalizability can be expressed by using the intrinsic curvature array of model, from which a new model selection criterion GKCIC is given. It not only considers the factors such as the number of parameters, sample size and functional form, but also with very clear and intuitive geometric understanding of model selection. The geometrical method of the statistical manifold is compared with the statistical learning theory, in particular, the VC dimension versus the Gauss-Kroneker curvature. By running the algorithm on synthetic and real datasets, the author argue that the GKCIC work efficiently. keywords model selection; generalizability; intrinsic complexity; statistical manifold; Gauss-Kroneker curvature background The work is supported by the National Natural Science Foundation of China under grant 60373029, Research Fund for the Doctoral Program of Higher Education of China (20050004001) and Co-construction Project of Key Subject of Beijing. Neural computation is an intercrossed science involving cognitive science, artificial intelligence, neural network and so on. Among which, effective coding is an important theory for comprehending neural system function. The theory was brought forward in 1961, but it is not developed extensively until man understands more deeply toward neural system in 1990’s. Therefore, it is of great sense tracing research in this field. Using mathematic tools as differential geometry and statistics, the project is to advance new cognitive model fitting theoretical framework of effective coding in human vision cognitive system better. The model is built from viewpoints of differential manifold, information geometry and statistics. Through analyzing cooperation mechanism among multi visual areas, the authors construct a hierarchical and parallel effective coding model, which can realize side feedback and side inhibition to some extent. In this way, the model approaches information processing mechanism of human brain more closely. The research results of above theory and model are to be validated by being applied into practical system. The project adopts the method combining cognitive science and neural science. With deeper exploration of human behavior, it is sure to have broad application space. This project has been studied for about 4 years, and of more than 30 papers has been published.