THE AIMS OF THE PROJECT
In his 1965 book, Understanding Media, Marshall McLuhan wrote, "The hybrid or meeting of two media is a moment of truth and revelation of which new form is born...The crossings of media release great force." In many ways, this sort of bringing together is exactly what researchers have done with virtual reality. The technologies of computer graphics, data communications, and computer programming have been loosely melded with the technologies of the telephone, the television, and the video game. The result is a single, ever-changing technology, immensely superior to each of its predecessors.
Although the technology is mature enough to have different applications, there are key issues to be resolved for its use for medical applications.
1. Objectives of the project
2. Key issues of the VREPAR project
3. A framework for future researches in VR human issues
It is precisely one of the goals of the project to overcome those shortcomings in order to develop a demonstrator of a virtual environment that can be effectively used by end users. The main goal of the project is to build a demonstrator of a virtual reality system (Virtual Reality All-purpose Modular System - VRAMS), based on a modular architecture, to be used for psychoneurophysiological assessment and rehabilitation.
Various studies aimed at the development of virtual reality systems for this type of application have been under way for some time now, but the existing systems have a series of problems that limit their real possibilities of use:
With the development of VRAMS, VREPAR has the following objectives:
a) To develop a virtual reality system for the medical market that can be marketed at a price which is accessible to its possible end-users (hospitals, universities and research centres) and which would have the modular, connectability and interoperability characteristics that the existing systems lack;
b) To develop three hardware/software modules for the application of VRAMS in psychoneurophysiological assessment and rehabilitation. The chosen development areas are eating disorders (bulimia, anorexia and obesity), movement disorders (Parkinson's disease and torsion dystonia) and stroke disorders (unilateral neglect and hemiparesis).
c) To define reference standards and parameters relating to technological and experimental factors, which can also be used by third parties. In particular, VREPAR aims at:
- defining a standard for software development and a series of hardware specifications to be used in the development of further VRAMS modules;
- identifying the factors affecting individual experiences in a virtual environment;
issues for the VREPAR project
Undoubtedly the construction of different VR modules brings out many interesting and
challenging design questions. The following key points have emerged from our survey of PC
related market that are relevant for VREPAR Project's aims:
A suitable approach is in considering these points not as constraints for our research
but to turn them into advantages for the successful outcome of the project. The
instability of the market can drive us towards the adoption of a standard highly
inter-operable and multi-platform like VRML. It may be widely diffusable because almost
all VR packages support the possibility of importing/converting VRML worlds. In terms of
low cost technology it satisfies the requirements under the most convincing circumstances
because a VRML application is unbounded from platform specifications and available to
immediate exploration in the Internet which we can consider under this perspective the
most promising method for low-cost VR. The Internet exploitation opens a wide set of
possibility linked to the envisaging of medical practices as activities freed from time
and location constraints. VR could assist highly demanded operations such as
tele-diagnostics, tele-therapy and many others both as one-to one or as collaborative
processes among different roles and actors opening exciting new perspectives for medical
But the problem of VREPAR is not just a technical problem of assembling highly technical devices, it is also a problem of rendering the efficacy and successful outcome of the project with the intrinsic guarantee that any VR interaction treatment whether be diagnostic or therapeutic must definitely be not harmful and cause any danger to patients and users. Like for any other medical aim a specific concern on the absence of collateral effects and accurate testing on the result of somministration is mandatory due to the high responsibility of dealing primarily with human health. There is a literature on the effects that VR interaction may cause to users, these effects while being limited to special sectors of population can appear and undermine the clinical benefit with unpredictable and unwanted results. So in this perspective it has to be considered one of the key issues an accurate and attentive consideration of human factors involved, this has to be achieved at this stage of the project were hardware software characteristics must be able to set their use to the mandatory concern of human health and safety.
Professional literature on Human Computer Interaction - HCI specifically requires from the prototyping phase special consideration of these issues which are not ignored in the project and will be attentively treated as the work will progress.
Another consideration can be made for what concerns the more accurate deepening and tuning of the different modules to the specific cognitive and social contexts of use.
3. A framework for future
researches in VR human issues
As underlined before, the problem of VREPAR is not just a technical problem of assembling highly technical devices. A main aim of the project is to assure that VR interaction treatment, whether be diagnostic or therapeutic, is not harmful and cause any danger to patients and users. To establish a framework for the future research in VR human issues, a method is needed to quantify the temporal performance of virtual reality systems, ensuring the HCS characteristics can cope with them since the prototyping phase. Infact, as we seen in the human factor part, many of the undesirable effects of exposures to virtual reality environments can be ascribed to limitations in the temporal performance of the system hardware. These limitations result in lags between movements of the head or hands and corresponding movements of displayed images.
A mathematical model is described below which could provide the basis for such a framework. The inputs to the model are either the angular displacements of tracked objects, or recorded head motions (Lewis and Griffin, 1996).
3.1 Input output models of the temporal performance of the VR systems
3.2 Applications of systems models
3.3 Recommendation for the design of applications
Transfer function models are used to represent the sampling delays in a head position sensor, image processor and display. Transfer function models can also be used to represent the tracking behaviour of users and their vestibular and ocular responses to head and image motions (Lewis and Griffin, 1996). The dynamic of the various components of the model are discussed in below.
The transfer functions of the system components can be combined into the input-output
model shown in Figure 2. The primary inputs to the model are the displacements of the
visual target. A head motion time history may also be input directly to the model, in
which case the elements of the human operator tracking model, Hh(s) and R(f)
would be set to zero.
3.1.1 Modelling the temporal characteristics of the
The head tracking response
The human operator in a continuous tracking task can be modelled as a linear transfer
function, Hh(f), where:
where Oh(f) and I(f) are frequency domain representations of the head position output, oh(t), and the target position input, i(t). From an engineering point of view a human operator is a non-linear control element, but Krendel and McRuer (1965) showed that, with particular inputs, subjects responded similarly to equivalent linear systems. However, to completely describe the system output it was necessary to include a "remnant" component, r(t). The remnant comprises a relatively small signal which is added to the output of the linear model to account for differences between the response of the operator and the time-invariant linear relationship between the system input and both the tracking error and the system output. This type of model is referred to as a quasi-linear model of the human operator. The remnant is modelled as normally distributed broad-band random noise.
So and Griffin (1995a) have measured closed-loop transfer functions for head-aiming. A cross-hair aiming reticle was displayed in the centre of a monocular helmet mounted display. A circular target was driven by independent random functions in both the pitch and the yaw axes. The head coupled system had an inherent lag of approximately 40 ms between head movements and the corresponding movement on the display (comprising the lag in the head position sensor, the computation time and the update rate of the display). An additional time delay, which was varied between 0 ms and 160 ms, was imposed by a computer.
Figure 3 shows mean head tracking transfer functions in the pitch and yaw axes measured
with five values of imposed system lag. At frequencies below 0.4 Hz the modulus (i.e. the
gain) of the human operator transfer functions were all close to unity but at higher
frequencies the gain increased with increasing imposed display lag. The increased gain at
higher frequencies was believed to be a strategy used by the subjects to compensate for
the increased lag in the system. The phase lags reflect delays in the operator's response,
relative to the target motion. The phase lags at around 0.1 Hz were shown to decrease
significantly with increasing lag, but the reduction was not sufficient to compensate for
the imposed display lag. The phase response at 0.1 Hz was important since a large
proportion of the energy in the target motion was around this frequency. Although there
was a decrease in the phase lag of the subjects with increasing system lag at low
frequencies, the phase lags above 0.5 Hz increased with increasing system lag. This is a
typical response for a system consisting of a lead-lag filter and a lag: lead generation
at low frequencies is accompanied by increases in response lag at higher frequencies, and
a consequent reduction in tracking bandwidth (McRuer, 1973).
The vestibulo-ocular reflex
The vestibulo-ocular reflex (VOR) induces movements of the eyes which compensate for
movements of the head. The VOR stabilises the eyes in space during body motion. This makes
it possible to view objects which are fixed in space without smearing of the retinal
image. The frequency response characteristics of the compensatory eye movements induced by
angular oscillation in yaw have been measured by, for example, Benson and Barnes (1978).
Barnes (1980) has modelled the slow phase (compensatory) eye movements evoked by yaw axis
rotation by the following transfer function:
where s is the complex radial frequency; is the angular displacement of the eye; is the
angular displacement of the head; T=0.005 s; TA=0.2 s; TB=15
s; TC= 0.125 s; TD= 0.002 s; TAD=80
s and KC=0.7.
The pursuit reflex
The VOR can be overridden at low frequencies by a pursuit tracking reflex. Benson and
Barnes (1978) have discussed methods for modelling this visually-driven response. The
pursuit reflex has been shown to break down when the velocity of a viewed object is
greater than 40 to 60 degrees per second or the frequency of the movement is above about 1
Hz (Benson and Barnes, 1978).
3.1.2 Modelling the temporal characteristics of system
Head position sensor
Typical head pointing systems sample the angular displacement of the head with sample
rates between 30 and 60 samples per second. The output signal of the head pointing system,
(s), in response to head displacement, (s), can be represented by:
where *(s) is the signal (s) sampled at discrete intervals of T seconds and is given by:
If the system operates at a sampling rate of 30 samples per second, T= 2/s
= 0.033 s. Hdelay(s) represents a time delay of Tseconds and is
Hhold(s) is a zero-order hold and is equivalent to:
The image processor is a computer which takes the head angle measured by the head
pointing system and renders the images which are displayed on the head-mounted display at
appropriate locations in the video frame. The computation will impose a time delay and a
sample-and-hold effect. Hence the transfer function, Hc(s), has the same
form as that of the head pointing system. The computation time, T, is the inverse
of the system frame rate.
The response and frame rate of the display combine to produce a sample-and-hold effect,
hence the transfer function representing the effect of the display on the position of a
moving image will be approximated by:
with a frame rate of 60 frames per second T= 2/s = 0.0167 s. Hhold(s) is a zero-order hold given by:
The characteristics and implementation of phase-lead filters for predicting future head
motions have been discussed by So (1995) and So and Griffin (1996). Alternative methods
are cited by Wioka (1995). Alternative head position prediction algorithms may be
implemented in the model to determine the extent to which they reduce the effects of
Evaluation of visual-vestibular interaction
Perception of rotational self-motion may be examined using models such as that proposed by Zacharias and Young (1981). The inputs to this model are the head velocities sensed by the semi-circular canals, ves, and by peripheral vision (i.e. the world-stabilised background image), vis. the extent of potential cue conflicts between visual and vestibular perception of self-motion may be derived from the difference between the two signals.
Evaluation of visual-vestibular interaction in the perception of
This figure shows how that generic model may be adapted to evaluate potential cue
conflicts. The figure also shows the velocity difference between the head velocity and
image velocity time histories. In this example the system update rate was 10 Hz and the
lag in the head pointing system was 15 ms. A conflict index was derived from the relative
magnitude of the velocity difference and the head velocity signal. The conflict index was
With the fast head motion the system lag induces a relatively large velocity difference, resulting in a conflict index of 0.81. With the slower, continuous head motions the velocity difference is smaller compared with the head velocity, resulting in a conflict index of 0.19.
The above example illustrates the importance of the interaction between the system lags
in an immersive virtual environment and the characteristics of the head movements made by
the users. The head motion characteristics need to be taken into account when optimising
the system for a particular application.
The model can be used to evaluate dynamic errors in image registration, between the real world seen through a head-mounted display, and overlaid virtual objects. The figure below shows how the generic model may be adapted to evaluate image registration errors in augmented velocity tasks.
Evaluation of tracking performance and registration errors.
It might be possible to reduce side effects of exposures to virtual reality
environments by either optimising the design of the system (i.e. the software and the
hardware) or by implementing procedures to manage exposures. Recommendations for managing
exposures to flight simulators and virtual reality environments have previously been made
by Frank et al (1983) and McCauley and Sharkey (1992).
The users of virtual reality applications designed for assessment and rehabilitation
may have disabilities which increase their susceptibility to certain side-effects. Many of
the reported effects on performance and well-being have been ascribed to distortions in
the relationship between the movements of users, and the visual feedback of those
movements. These distortions arise because of limitations in the spatial and temporal
performance of virtual reality systems, or because image motions are presented which give
the illusion of body motion in the absence of real motion.
The proposed VRAMS modules (at least the Eating Disorders one) will deliberately create
further distortions in order to augment the stimulation and feedback provided by the
visual images. These distortions may be expected to give rise to additional problems.
Special precautions therefore need to be taken to ensure the safety and effectiveness of
such virtual reality applications. Section 18.104.22.168 makes specific recommendations for
managing exposures to virtual reality environments for assessment and rehabilitation.
The design of virtual reality applications for assessment and rehabilitation may also require special precautions to ensure the effectiveness of the tasks presented to subjects. Section 22.214.171.124 makes design recommendations based on current knowledge of the effects of system characteristics on the performance of tasks and the incidence of side-effects.
3.3.1 Managing exposures to virtual reality
3.3.2 Recommendations for the design of the VREPAR
The VRAMS modules that will be developed within the VREPAR project all involve some
degree of manual control, or manipulation of virtual objects. The eating disorders module
is a fully immersive virtual reality application in which users move through a virtual
environment, observing and interacting with virtual objects. The stroke disorders module
may utilise a fixed wide field-of-view display to present moving scenes which could induce
a strong sense of vection. Some special considerations which should be given to the
implementation of these three features are detailed below.
Interactive, immersive virtual environments
Manual control or manipulation of virtual objects
Presentation of large moving scenes in the absence of body motion
For any questions or requests, please contact firstname.lastname@example.org