In his 1965 book, Understanding Media, Marshall McLuhan wrote, "The hybrid or meeting of two media is a moment of truth and revelation of which new form is born...The crossings of media release great force." In many ways, this sort of bringing together is exactly what researchers have done with virtual reality. The technologies of computer graphics, data communications, and computer programming have been loosely melded with the technologies of the telephone, the television, and the video game. The result is a single, ever-changing technology, immensely superior to each of its predecessors.

Although the technology is mature enough to have different applications, there are key issues to be resolved for its use for medical applications.

1. Objectives of the project

2. Key issues of the VREPAR project

3. A framework for future researches in VR human issues

1. Objectives of the project

It is precisely one of the goals of the project to overcome those shortcomings in order to develop a demonstrator of a virtual environment that can be effectively used by end users. The main goal of the project is to build a demonstrator of a virtual reality system (Virtual Reality All-purpose Modular System - VRAMS), based on a modular architecture, to be used for psychoneurophysiological assessment and rehabilitation.

Various studies aimed at the development of virtual reality systems for this type of application have been under way for some time now, but the existing systems have a series of problems that limit their real possibilities of use:

With the development of VRAMS, VREPAR has the following objectives:

a) To develop a virtual reality system for the medical market that can be marketed at a price which is accessible to its possible end-users (hospitals, universities and research centres) and which would have the modular, connectability and interoperability characteristics that the existing systems lack;

b) To develop three hardware/software modules for the application of VRAMS in psychoneurophysiological assessment and rehabilitation. The chosen development areas are eating disorders (bulimia, anorexia and obesity), movement disorders (Parkinson's disease and torsion dystonia) and stroke disorders (unilateral neglect and hemiparesis).

c) To define reference standards and parameters relating to technological and experimental factors, which can also be used by third parties. In particular, VREPAR aims at:

- defining a standard for software development and a series of hardware specifications to be used in the development of further VRAMS modules;

- identifying the factors affecting individual experiences in a virtual environment;

2. Key issues for the VREPAR project

Undoubtedly the construction of different VR modules brings out many interesting and challenging design questions. The following key points have emerged from our survey of PC related market that are relevant for VREPAR Project's aims:

A suitable approach is in considering these points not as constraints for our research but to turn them into advantages for the successful outcome of the project. The instability of the market can drive us towards the adoption of a standard highly inter-operable and multi-platform like VRML. It may be widely diffusable because almost all VR packages support the possibility of importing/converting VRML worlds. In terms of low cost technology it satisfies the requirements under the most convincing circumstances because a VRML application is unbounded from platform specifications and available to immediate exploration in the Internet which we can consider under this perspective the most promising method for low-cost VR. The Internet exploitation opens a wide set of possibility linked to the envisaging of medical practices as activities freed from time and location constraints. VR could assist highly demanded operations such as tele-diagnostics, tele-therapy and many others both as one-to one or as collaborative processes among different roles and actors opening exciting new perspectives for medical VR.

But the problem of VREPAR is not just a technical problem of assembling highly technical devices, it is also a problem of rendering the efficacy and successful outcome of the project with the intrinsic guarantee that any VR interaction treatment whether be diagnostic or therapeutic must definitely be not harmful and cause any danger to patients and users. Like for any other medical aim a specific concern on the absence of collateral effects and accurate testing on the result of somministration is mandatory due to the high responsibility of dealing primarily with human health. There is a literature on the effects that VR interaction may cause to users, these effects while being limited to special sectors of population can appear and undermine the clinical benefit with unpredictable and unwanted results. So in this perspective it has to be considered one of the key issues an accurate and attentive consideration of human factors involved, this has to be achieved at this stage of the project were hardware software characteristics must be able to set their use to the mandatory concern of human health and safety.

Professional literature on Human Computer Interaction - HCI specifically requires from the prototyping phase special consideration of these issues which are not ignored in the project and will be attentively treated as the work will progress.

Another consideration can be made for what concerns the more accurate deepening and tuning of the different modules to the specific cognitive and social contexts of use.

3. A framework for future researches in VR human issues

As underlined before, the problem of VREPAR is not just a technical problem of assembling highly technical devices. A main aim of the project is to assure that VR interaction treatment, whether be diagnostic or therapeutic, is not harmful and cause any danger to patients and users. To establish a framework for the future research in VR human issues, a method is needed to quantify the temporal performance of virtual reality systems, ensuring the HCS characteristics can cope with them since the prototyping phase. Infact, as we seen in the human factor part, many of the undesirable effects of exposures to virtual reality environments can be ascribed to limitations in the temporal performance of the system hardware. These limitations result in lags between movements of the head or hands and corresponding movements of displayed images.

A mathematical model is described below which could provide the basis for such a framework. The inputs to the model are either the angular displacements of tracked objects, or recorded head motions (Lewis and Griffin, 1996).

3.1 Input output models of the temporal performance of the VR systems

3.2 Applications of systems models

3.3 Recommendation for the design of applications


3.1 Input-output models of the temporal performance of virtual reality systems

Transfer function models are used to represent the sampling delays in a head position sensor, image processor and display. Transfer function models can also be used to represent the tracking behaviour of users and their vestibular and ocular responses to head and image motions (Lewis and Griffin, 1996). The dynamic of the various components of the model are discussed in below.

The transfer functions of the system components can be combined into the input-output model shown in Figure 2. The primary inputs to the model are the displacements of the visual target. A head motion time history may also be input directly to the model, in which case the elements of the human operator tracking model, Hh(s) and R(f) would be set to zero.


3.1.1 Modelling the temporal characteristics of the human operator


The head tracking response

The human operator in a continuous tracking task can be modelled as a linear transfer function, Hh(f), where:

where Oh(f) and I(f) are frequency domain representations of the head position output, oh(t), and the target position input, i(t). From an engineering point of view a human operator is a non-linear control element, but Krendel and McRuer (1965) showed that, with particular inputs, subjects responded similarly to equivalent linear systems. However, to completely describe the system output it was necessary to include a "remnant" component, r(t). The remnant comprises a relatively small signal which is added to the output of the linear model to account for differences between the response of the operator and the time-invariant linear relationship between the system input and both the tracking error and the system output. This type of model is referred to as a quasi-linear model of the human operator. The remnant is modelled as normally distributed broad-band random noise.

So and Griffin (1995a) have measured closed-loop transfer functions for head-aiming. A cross-hair aiming reticle was displayed in the centre of a monocular helmet mounted display. A circular target was driven by independent random functions in both the pitch and the yaw axes. The head coupled system had an inherent lag of approximately 40 ms between head movements and the corresponding movement on the display (comprising the lag in the head position sensor, the computation time and the update rate of the display). An additional time delay, which was varied between 0 ms and 160 ms, was imposed by a computer.

Figure 3 shows mean head tracking transfer functions in the pitch and yaw axes measured with five values of imposed system lag. At frequencies below 0.4 Hz the modulus (i.e. the gain) of the human operator transfer functions were all close to unity but at higher frequencies the gain increased with increasing imposed display lag. The increased gain at higher frequencies was believed to be a strategy used by the subjects to compensate for the increased lag in the system. The phase lags reflect delays in the operator's response, relative to the target motion. The phase lags at around 0.1 Hz were shown to decrease significantly with increasing lag, but the reduction was not sufficient to compensate for the imposed display lag. The phase response at 0.1 Hz was important since a large proportion of the energy in the target motion was around this frequency. Although there was a decrease in the phase lag of the subjects with increasing system lag at low frequencies, the phase lags above 0.5 Hz increased with increasing system lag. This is a typical response for a system consisting of a lead-lag filter and a lag: lead generation at low frequencies is accompanied by increases in response lag at higher frequencies, and a consequent reduction in tracking bandwidth (McRuer, 1973).

The vestibulo-ocular reflex

The vestibulo-ocular reflex (VOR) induces movements of the eyes which compensate for movements of the head. The VOR stabilises the eyes in space during body motion. This makes it possible to view objects which are fixed in space without smearing of the retinal image. The frequency response characteristics of the compensatory eye movements induced by angular oscillation in yaw have been measured by, for example, Benson and Barnes (1978). Barnes (1980) has modelled the slow phase (compensatory) eye movements evoked by yaw axis rotation by the following transfer function:

where s is the complex radial frequency; is the angular displacement of the eye; is the angular displacement of the head; T=0.005 s; TA=0.2 s; TB=15 s; TC= 0.125 s; TD= 0.002 s; TAD=80 s and KC=0.7.

The pursuit reflex

The VOR can be overridden at low frequencies by a pursuit tracking reflex. Benson and Barnes (1978) have discussed methods for modelling this visually-driven response. The pursuit reflex has been shown to break down when the velocity of a viewed object is greater than 40 to 60 degrees per second or the frequency of the movement is above about 1 Hz (Benson and Barnes, 1978).


3.1.2 Modelling the temporal characteristics of system components


Head position sensor

Typical head pointing systems sample the angular displacement of the head with sample rates between 30 and 60 samples per second. The output signal of the head pointing system, (s), in response to head displacement, (s), can be represented by:

where *(s) is the signal (s) sampled at discrete intervals of T seconds and is given by:

If the system operates at a sampling rate of 30 samples per second, T= 2/s = 0.033 s. Hdelay(s) represents a time delay of Tseconds and is equivalent to:

Hhold(s) is a zero-order hold and is equivalent to:

Image processing

The image processor is a computer which takes the head angle measured by the head pointing system and renders the images which are displayed on the head-mounted display at appropriate locations in the video frame. The computation will impose a time delay and a sample-and-hold effect. Hence the transfer function, Hc(s), has the same form as that of the head pointing system. The computation time, T, is the inverse of the system frame rate.

Head-mounted display

The response and frame rate of the display combine to produce a sample-and-hold effect, hence the transfer function representing the effect of the display on the position of a moving image will be approximated by:

with a frame rate of 60 frames per second T= 2/s = 0.0167 s. Hhold(s) is a zero-order hold given by:

Predictive filter

The characteristics and implementation of phase-lead filters for predicting future head motions have been discussed by So (1995) and So and Griffin (1996). Alternative methods are cited by Wioka (1995). Alternative head position prediction algorithms may be implemented in the model to determine the extent to which they reduce the effects of system lags.


3.2 Application of system models


Evaluation of visual-vestibular interaction

Perception of rotational self-motion may be examined using models such as that proposed by Zacharias and Young (1981). The inputs to this model are the head velocities sensed by the semi-circular canals, ves, and by peripheral vision (i.e. the world-stabilised background image), vis. the extent of potential cue conflicts between visual and vestibular perception of self-motion may be derived from the difference between the two signals.

Evaluation of visual-vestibular interaction in the perception of angular self-motion.

This figure shows how that generic model may be adapted to evaluate potential cue conflicts. The figure also shows the velocity difference between the head velocity and image velocity time histories. In this example the system update rate was 10 Hz and the lag in the head pointing system was 15 ms. A conflict index was derived from the relative magnitude of the velocity difference and the head velocity signal. The conflict index was calculated from:

With the fast head motion the system lag induces a relatively large velocity difference, resulting in a conflict index of 0.81. With the slower, continuous head motions the velocity difference is smaller compared with the head velocity, resulting in a conflict index of 0.19.

The above example illustrates the importance of the interaction between the system lags in an immersive virtual environment and the characteristics of the head movements made by the users. The head motion characteristics need to be taken into account when optimising the system for a particular application.

Registration errors

The model can be used to evaluate dynamic errors in image registration, between the real world seen through a head-mounted display, and overlaid virtual objects. The figure below shows how the generic model may be adapted to evaluate image registration errors in augmented velocity tasks.

Evaluation of tracking performance and registration errors.


3.3 Recommendations for the design of applications

It might be possible to reduce side effects of exposures to virtual reality environments by either optimising the design of the system (i.e. the software and the hardware) or by implementing procedures to manage exposures. Recommendations for managing exposures to flight simulators and virtual reality environments have previously been made by Frank et al (1983) and McCauley and Sharkey (1992).

The users of virtual reality applications designed for assessment and rehabilitation may have disabilities which increase their susceptibility to certain side-effects. Many of the reported effects on performance and well-being have been ascribed to distortions in the relationship between the movements of users, and the visual feedback of those movements. These distortions arise because of limitations in the spatial and temporal performance of virtual reality systems, or because image motions are presented which give the illusion of body motion in the absence of real motion.

The proposed VRAMS modules (at least the Eating Disorders one) will deliberately create further distortions in order to augment the stimulation and feedback provided by the visual images. These distortions may be expected to give rise to additional problems. Special precautions therefore need to be taken to ensure the safety and effectiveness of such virtual reality applications. Section makes specific recommendations for managing exposures to virtual reality environments for assessment and rehabilitation.

The design of virtual reality applications for assessment and rehabilitation may also require special precautions to ensure the effectiveness of the tasks presented to subjects. Section makes design recommendations based on current knowledge of the effects of system characteristics on the performance of tasks and the incidence of side-effects.


3.3.1 Managing exposures to virtual reality environments

3.3.2 Recommendations for the design of the VREPAR modules

The VRAMS modules that will be developed within the VREPAR project all involve some degree of manual control, or manipulation of virtual objects. The eating disorders module is a fully immersive virtual reality application in which users move through a virtual environment, observing and interacting with virtual objects. The stroke disorders module may utilise a fixed wide field-of-view display to present moving scenes which could induce a strong sense of vection. Some special considerations which should be given to the implementation of these three features are detailed below.

Interactive, immersive virtual environments

Manual control or manipulation of virtual objects

Presentation of large moving scenes in the absence of body motion

For any questions or requests, please contact