Chinese Journal of Computers   Full Text
Title	General Framework, Mathematical Model, Current Activities and Open Issues for Media Adaptation in MPEG-21 DIA
Authors	HUANG Tian-Yun
Address	(School of Computer Science and Technology, Southwest University for Nationalities, Chengdu 610041)
Year	2008
Issue	No.7(1185—1199)
Abstract &   Background	Abstract A growing variety of terminals and heterogeneous networks with dynamic throughputs make it a challenge in multimedia contents consumption. Media adaptation has been introduced in order to maximize user experiences and ensure quality of service (QoS) in universal media access environment. Based on the relations among entities such as resources, adaptation operators and utility in MPEG-21 digital items adaptation (DIA), and the general media adaptation framework, a mixed-variable constrained optimization model is proposed in this paper. The model unifies researches in this field, brings together the problem formulation and solution in a consistent way. A hierarchical adaptation structure is also constructed, from the viewpoints of image understanding and video analysis, and applied to classify some representative media adaptation applications. Leading pursuits such as image adaptation, scalable video transcoding, bit rate adaptation, video objects (VOs) adaptation, multi-modality conversion and fusion of mixed medias, etc., are illustrated by examples. Hotspots such as semantic abstraction of media contents, measurements of subjective quality, and maximizing of user experiences, etc., are also addressed. Keywords universal media access; media adaptation; MPEG-21 DIA; entity; abstract model; transcoding; bit rate adaptation; modality conversion Background In order to meet the requirements of different users and all kinds of terminals, to access media contents widely distributed over networks, the concept of universal media access (UMA) was introduced in recent years, which had greatly enhanced research activities in this field, and accelerated standardization progresses such as MPEG-7 and MPEG-21. However, in heterogeneous network environment, how to maximize user experiences on media consumption and guarantee quality of service (QoS) based on user preferences and terminal capacities? There still exist many challenges. Meanwhile, media formats and contents are continuously increasing and becoming more abundant, media adaptation has become necessary. Digital items adaptation (DIA) has been proposed in MPEG-21 in order to satisfy the requirements of UMA, and has attracted more attentions in recent years. People have devoted in this field for years, such as JPEG images or MPEG I frames adaptation in size and color space, MPEG video transcoding to different formats (e.g., MPEG-2 to MPEG-4, Real media to AVI) or into smaller size (e.g., DCT coefficient requantization or dropping, frame dropping), etc. Probably the most salient work is bit rate adaptation for adaptive delivery of media content according to the network conditions, especially streaming of MPEG-4 finer granularity scalability (FGS), MPEG-4 video objects (VOs) over Internet and wireless, etc. But notice that almost all researchers are concentrated on constructing their own mathematical models and try to find the optimal solutions in different ways, e.g., by dynamic programming or some elaborate heuristics. These researches only partly solved media adaptation problem under different constraints, there is yet no consistent way to describe, analysis and solve the problem in the media adaptation framework. Aimed at a unified solution for media adaptation, the author proposes a mixed-variable constrained optimization model in this paper, based on analysis of the relations among entities in MPEG-21 DIA. The model unified current researches in media adaptation, and can be solved by the direct searches. The author also proposes a hierarchical media adaptation structure, from the viewpoints of image understanding and video analysis. Leading research activities such as FD/CD transcoding, full scalable and object encoding video adaptation, multi-modality conversion and fusion, etc., are illustrated by examples. Hotspots in media adaptation such as semantic abstraction of media contents, measurements on subjective quality, etc., are also addressed in this paper. A thorough analysis of algorithms to solve the model had been conducted in related paper. The author became interested in video traffic analysis, optimal video transmission and quality of streaming video since he was pursuing his Ph.D. degree in UESTC in 1999. He had published one book (Video traffic analysis and QoS management) and more than ten papers in journals and conferences. He holds a digital media lab in SWUN, and made some contributions to media adaptation with his students in recent years, such as rate control of FGS video, prioritized discarding of EL layers in video objects over the Diffserv, energy-aware FGS video streaming over wireless, etc. Now, he is dedicated in the assessment and measurement on subjective quality of scalable videos, and tries to apply it to wireless video streaming. This work is partly supported by the research fund for the State Ethnic Affairs Commission of China (05XN09)—"Research on the Key Issues of Video Streaming over Wireless".