A major research goal during the last decade has been to create photo-realistic interactive synthetic worlds. Despite much effort, however, synthetic worlds remain artificial environments, and have limited usage in human life. Nowadays there are several Virtual Reality applications emerging for entertainment, medical and telepresence purposes. All these applications are dealing with homogenous synthetic worlds which are very far from representing the real world.
One major problem is the restrictive human machine interfaces like gloves, magnetic trackers, and head mounted displays. Such interfaces with excessive connections are hampering the interactivity. Another major issue is the rendering of photo realistic, detailed and interactive environments in real-time. Although the current computer graphics technology can model, animate and render synthetic articulated figures with deformable surfaces with near photo-realism in real-time, we cannot say the same about rendering of a complex synthetic world.
Until now I presented the problems of the current Virtual Reality technology. There is a new approach, called Augmented Reality, to overcome the limitations of the Virtual Reality. We can summarize the Augmented Reality as a combination of distinct technologies spanning from virtual reality to computer vision. By definition Augmented Reality enhances user's view of the real world with visual information from a computer. In case of interactions between real and synthetic worlds, the computers perception should be enhanced also.
In the first phase of my thesis I will use and improve Augmented Reality technology to develop new interaction techniques. A major issue is to develop a robust computer vision module to sample the real world. This vision system should obtain essential data, motion data for the humans on the scene etc., from the real world and transform it into a machine understandable form.
The second step will be to process this data in real-time. I will use high level action recognition tools which are partially developed during my Diploma work at LIG to process the data for human users. I will analyse other techniques to obtain data from the real world and to process it.
The third step is the visual and audio feedback from the computer to the user. Currently there are two popular approaches: Using head mounted display or a large video monitor ``magic mirror metaphor''. I will consider both approaches for different types of applications. The Audio feedback will be developed to improve the realism. The video output of a typical Augmented Reality application is a mixture of real and virtual worlds. The video sequence from the real world is blended with the computer rendered virtual scene. In my thesis I will emphasise on exact representation of the real objects/humans in the virtual world.
The second phase of my thesis will be to develop a software controlling module for autonomous virtual humans. At LIG several researchers are developing autonomous virtual humans with synthetic perception modules. I will develop an interaction driven control module to develop applications like Interactive Drama. This control module will let the participants to change the storyline of a drama.
The work will be coordinated with the current on going projects: VLNET (Telepresence) and VISTA (Interactive Drama).
In my thesis the main objective is to let humans to interact with autonomous virtual objects and humans in a natural way. This will lead development of new human machine interfaces for several purposes like, education, entertainment and telepresence.