DAVE WARNER

MEDICAL NEUROSCIENTIST

HUMAN PERFORMANCE INSTITUTE

LOMA LINDA UNIVERSITY MEDICAL CENTER

11406 LOMA LINDA DRIVE

LOMA LINDA CA. 92354

USA

VOICE 909-799-6190

FAX 909-799-6106

EMAIL davew@well.sf.ca.us

EDUCATION

Bachelor of Arts: Physical Science ( special emphasis in physics, mathematics and philosophy ) 1986. San Diego State University

Current enrollment in MD./Ph.D. degree program. Loma Linda University (expected graduation M.D. spring 1995, Ph.D. spring 96)

INTERESTS -INTENSE THIRST FOR KNOWLEDGE

Physics - Mathematics - Philosophy - Physiology - Cognitive Neuroscience - Perceptual Psychophysics - Bio-Cybernetics - Human Performance

Since I started the MD. Ph.D. program in 1988, my research efforts have focused on advanced instrumentation and new methods of analysis which can be applied to evaluating various aspects of human function. I have been working with several classes of advanced human computer interface technologies including Data gloves, pressure sensors, surface EMG, EEG, EOG, (direct bioelectric signals), a new form of force transducer and complete sensory emersion systems of virtual reality. My specific research is involved in studying the flow of information between the human and the computer. The intent of this effort is to identify methods and techniques which optimize information flow between humans and computers. It is postulated that an optimal mapping of interactive interface technologies to the human nervous systems capacity to transduce, assimilate and respond intelligently to information in an integrative-multisensory interaction will fundamentally change the way that humans interact with information systems.

When I first started my work here at Loma Linda University Dr. Doug Will demonstrated his commitment to bringing innovative research to LLU by providing me with a research scholarship to pursue these nontraditional, yet promising, methods and technologies. From the philosophic orientation of general systems theories and human information sciences we are exploring relevant issues in interactive human computer interface design such as cognitive neuroscience, perceptual psychophysics and bio-cybernetics The intent of our efforts is to create interactive systems for quantitative assessment of human performance, augmentative communication, environmental controls for the disabled, medical informatics and integrated interactive educational multimedia.

"INTERACTIVE TECHNOLOGIES IN HEALTH CARE AND EDUCATION"

In the past 3 years our work in demonstrating socially responsible applications for advanced interface technologies, developed for enhanced human computer interaction, has caught the attention of several professional societies from quite diverse areas, i.e. the entertainment industry, the aerospace industry, the communication industry, educational technologies industry, along with different fields of medicine. We have been and hope to continue to be clear leaders in this type of integrative effort.

"APPLYING INFORMATION TECHNOLOGY TO IMPROVE QUALITY OF LIFE"

Quality of life has become an issue of increasing concern among health care providers. Off the shelf technologies exist that can be modified or adapted to enhance human performance in every day life. Through the collaboration of a multi-disciplinary group of physicians, engineers, clinicians and patients we are finding creative ways to increase the quality of life.

The primary role that I play is one of a general medical-scientist liaison between the medical community and high-tech development companies. My role is to provide an application assessment of new technologies for possible medical and educational uses. My experience in the area of interactive human computer interaction combined with my current status in clinical medicine allows me the unique opportunity to facilitate the rapid exchange of relevant information between the high-tech industry and the medical community in general.. I am particularly active in technology transfer of aerospace and other defense derived technologies to the fields of health care and education. Specifically, advanced instrumentation for the acquisition and analysis of medically relevant biological signals, informatic systems which augment both the general flow of medical information and provide decision support for the health care professional, public accesses health information databases designed to empower the average citizen to become more involved in their own health care and advanced training technologies which will adaptivly optimize interactive educational systems to the capacity of the user.

PAPERS PRESENTED

Chaotropic dynamical analysis of the EEG.

The VPL Data Glove as an instrument for quantitative motion analysis.

The Neurorehabilitation Workstation.

Remapping the Human-Computer Interface for Perceptualization of Medical Information.

Medical Rehabilitation, Cyber-Style

Integrative Use of Computer Graphics in a Medical University,

Dynamical Analysis of EEG: Evidence for a Low‑Dimensional Attractor in Absence Epilepsy.

Quantitative analysis of tremor and chorea using the VPL Data Glove.

The Data Glove for precise quantitative measurement of upper motor neuron (UMN) function in amyotrophic lateral sclerosis (ALS).

The VPL Data Glove as a tool for hand rehabilitation and communication

Quantitative Motion Analysis Instrumentation for Movement Related Potentials

Quantitative motion analysis of the hand using the data glove.

Compressed Dimensional Array: a topographic technique for EEG analysis.

Re-enabling Technologies Immediate Applications for Virtual Reality Interfaces.

INVITED LECTURES

Computer Enhanced Perception: Medical Applications of Multisensory Natural User Interfaces.

8Th Annual Pacific Northwest Computer Graphics Conference. Portland Organ 1990

Computer Interface Technology for Dynamic Characterization of Neurological Disorders

Neurological Institute Colombia University New York 1990

Immediate Medical applications of virtual reality technologies

Distinguished Lecture Series San Francisco University 1991

Medical Applications of Virtual Reality Technology

2Ed International CyberArts Conference Pasadena Ca. 1991

Virtual Reality: Applications in Medicine

National Computer Graphics Association Anaheim 1992

Medical Applications of Emerging Interface Technologies

CUBE 92 Lawrence Livermore 1992

Real Medical Applications of Virtual Reality Technology

Imagina 93 Monte Carlo Monaco 1993

Human-Computer Interface Technologies in Rehabilitation

California Medical Association Anaheim Ca 1993

Virtual Reality Applications in Medicine and Health Care

IEEE Student Association Walla Walla WA. 1993

Advanced Interactive Technologies in Medicine and Education

4th National Conference of College Teaching and Learning Jacksonville Fl 1993

Interactive Technologies in Medicine.

SUNY Conference on Computers in Health Care Syracuse NY. 1993

Integrated Use Of Computer Graphics In A Medical University

SIGGRAPH 93 Anaheim Ca 1993

ACCOMPLISHMENTS

President/Founder of "VEKTORS" High tech awareness club for students with magnitude and direction. San Diego Mesa College 1983-84

President/Founder of Human-Space Interface a Human factors in space technology awareness group. San Diego State University 1985-87

Program Advisor for Medicine Meets Virtual Reality conference

San Diego 1992 and 1994

Member of editorial board for Virtual Reality Systems Magazine

Member of program committee for the VRIAS 93 conference. The first annual IEEE conference of Engineering and Virtual Reality

Director/Founder SAMARITAN PROJECT an organization dedicated to the intelligent implementation of interactive information technologies in Health Care and Education. Loma Linda University 1993 to present.

Director/Founder HUMAN PERFORMANCE INSTITUTE a multidisipline research support network at Loma Linda University Medical Center which acts as a surrogate administration for projects designed to study any aspect of human performance in the context of health care, education, communication and enriching recreation.

DAVE WARNER

MEDICAL NEUROSCIENTIST

HUMAN PERFORMANCE INSTITUTE

LOMA LINDA UNIVERSITY MEDICAL CENTER

11406 LOMA LINDA DRIVE

LOMA LINDA CA. 92354

USA

VOICE 909-799-6190

FAX 909-799-6106

EMAIL davew@well.sf.ca.us

SAMARITAN

PROJECT

LEADING BY EXAMPLE

INTELLIGENTLY IMPLEMENTING

INTERACTIVE INFORMATION TECHNOLOGY

in

HEALTH CARE

AND

EDUCATION

to

IMPROVE QUALITY OF LIFE

MISSION:

To improve quality of life by intelligent use of interactive information technology in the fields of Health Care and Education.

METHODS:

Provide the knowledge and resources required to empower people to make a positive difference with information technology.

Lead in the development of socially responsible applications of information technology.

Guide and assist in the intelligent implementation of advanced information technologies.

Identify, acquire and implement advanced information technology in exemplary applications.

Actively share information and experience with others who are willing to become involved in a socially responsible utilization of advanced information technology.

It is the intent of this project to seek out key areas where information technology can be effectively utilized to improve quality of life, i.e. in health care and/or education, and then actively participate in facilitating implementation.

WHAT IS INTERACTIVE INFORMATION TECHNOLOGY?

Information technology is not new; clay tablets and quill pens are information technologies. The radio, television and telephone are information technologies. Computers and advanced interfaces are only the most resent additions.

Interactive information technology is any technology which augments our human ability to dynamically create, express, retrieve, analyze, process, communicate, or experience information.

WHAT HAVE WE DONE WITH THIS TECHNOLOGY SO FAR ?

For the past 4 years a small, dedicated group of socially conscientious technologists have been actively implementing off the shelf technologies that were developed for the military, entertainment, aerospace industries in medical and educational applications.

The primary role that we play is one of a general medical-scientist liaisons between the medical community and high-tech development companies, specifically for the purpose of technology transfer and application assessment of new technologies for possible medical uses. Our experience in the area of human computer interaction combined with our current access to clinical medicine allows us the unique opportunity to facilitate the rapid exchange of relevant information between the high-tech industry and the medical community in general. We are particularly active in technology transfer of aerospace and other defense derived technologies to more socially useful applications.

Project areas currently active include;

1. Advanced instrumentation for the acquisition and analysis of medically relevant biological signals

2. Advanced informatic systems which augment the general flow of medical information and provides decision support for the health care professional.

3. Public access to health information databases designed to empower the average citizen to become more involved in their own health care.

4. Advanced training technologies which will allow the rapid dispersion of newly developed techniques.

5. New interface devices for persons with disabilities.

6. Educational systems that adapt to the users ability to learn.

HOW CAN YOU HELP ?

The finacial status of the Samaritan Project is completely dependent on donations from people who see what we are doing and provide support so that these efforts will continue. Donations to the Samaritan Project are managed through LOMA LINDA UNIVERSITY, a non-profit organization, which has established a special account for the project. All contributions to the Samaritan Project are tax deductible and a letter of verification will be provided to the contributor by the University

WHAT WILL THE FUNDS BE USED FOR?

The funds for the Samaritan Project are to support the following:

1. EQUIPMENT (computers, communication technologies, interface devices, research instruments....)

2. SUPPLIES ( office supplies, film, video tapes, paper, computer disks.....)

3. PERSONNEL ( wages for part time/full time/contract workers , room and board for volunteers...)

4. EDUCATIONAL ACTIVITIES (travel, tuition, expenses for key persons involved with the Samaritan project to attend/participate in academic and professional functions...)

5. TRAVEL EXPENSES (for attending professional meetings, to network with strategic partners, for invited lectures .....)

Contributions may be sent to:

SAMARITAN PROJECT

C/O DAVE WARNER

LOMA LINDA UNIVERSITY

11406 LOMA LINDA DRIVE

LOMA LINDA CA 92354

PHONE 909-799-6190

FAX 909-799-6106

EMAIL davew@well.sf.ca.us

BIO-CYBERNETICS

A Biologically Responsive Interactive Interface

"Adventures In The Next Paradigm Of Human Computer Interaction"

Dave Warner, Jeff Sale, Todd Anderson, Jo Johanson

Human Performance Institute

Dept. of Rehabilitation Engineering

Loma Linda University Medical Center

The capacity of computers to receive, process, and transmit massive amounts of information is continually increasing. Current attempts to develop new human-computer interface technologies have given us devices such as gloves, motion trackers, 3-D sound and graphics. Such devices greatly enhance our ability to interact with this increasing flow of information. Interactive interface technologies emerging from the next paradigm of human-computer interaction are directly sensing bio-electric signals (from eye, muscle and brain activity) as inputs and rendering information in ways that take advantage of psycho-physiologic signal processing of the human nervous system (perceptual psychophysics). The next paradigm of human-computer interface will optimize the technology to the physiology -- a biologically responsive interactive interface.

"BIOCYBERNETICS"

INTERACTIVE INFORMATION TECHNOLOGY

Interactive information technology is any technology which augments our ability to create / express / retrieve / analyze / process / communicate / experience information in an interactive mode. Biocybernetics optimizes the interactive interface, promising a technology that can profoundly improve the quality of life of real people today. The next paradigm of interface technology is based on new theories of human-computer interaction which are physiologically and cognitively oriented. This emerging paradigm of human computer interaction incorporates multi-sense rendering technologies, giving sustained perceptual effects, and natural user interface devices which measure multiple physiological parameters simultaneously and use them as inputs. Biologically optimized interactive information technology has the potential to facilitate effective communication. This increase in effectiveness will impact both human-computer and human-human communication, "enhanced expressivity". Work in human-computer interaction is an ongoing endeavor in many areas. These efforts have captured the attention of several professional societies; the entertainment industry, the aerospace industry, communications and educational technologies industries, as well as medicine. These diverse areas will all be impacted in multiple ways by advances in technologies that enhance human-computer interaction.

Optimizing the human computer interface will rely on the knowledge base of physiology and neuroscience, that is, the more we know about the way we acquire information physiologically the more we know the optimum way for a human to interact with intelligent information systems. The next paradigm will see the "THINNING" of the human-computer interface to a biological sheer as the interface will map very close to the human body.

PHYSIOLOGICALLY ORIENTED INTERFACE DESIGN

Knowledge of sensory physiology and perceptual psychophysics is being used to optimize our future interactions with the computer. By increasing the number and variation of simultaneous sensory inputs, we can make the body an integral part of the information system, "a sensorial combinetric integrator". We can then identify the optimal perceptual state space parameters in which information can best be rendered. That is what types of information are best rendered to each specific sense modality, "a sense specific optimization of rendered information. Research in human sensory physiology, specifically sensory transduction mechanisms, shows us that there are designs in our nervous systems optimized for feature extraction of spatially rendered data, temporally rendered data, and textures. Models of information processing based on the capacity of these neurophysiological structures to process information will help our efforts to enhance perception of complex relationships by integrating visual, binaural, and tactile modalities. Then by using the natural bioelectric energy as a signal source for input; electroencephalography, electroocculography, and electromyography (brain, eye and muscle) we can generate highly interactive systems in which these biological signals initiate specific events. Such a real-time analysis enables multi-modal feedback and closed-loop interactions.

"BIOCYBERNETIC CONTROLLER"

Interactive interface technology renders content specific information onto multiple human sensory systems giving a sustained perceptual effect, while monitoring human response, in the form of physiometric gestures, speech, eye movements and various other inputs. Such quantitative measurement of activity during purposeful tasks allows us to quantitatively characterize individual cognitive styles. This capability promises to be a powerful tool for characterizing the complex nature of normal and impaired human performance. The systems of the future will monitor a user's actions, learn from them, and adapt by varying aspects of the system's configuration to optimize performance. By immersion of external senses and iterative interaction with biosignal triggered events complex tasks are more readily achieved.

This paradigm shift of mass communication and information technologies is providing an exciting opportunity to facilitate the rapid exchange of relevant information thereby increasing the individual productivity of persons involved in the information industry. Areas such as computer-supported cooperative work, knowledge engineering, expert systems, interactive attentional training, and adaptive task analysis will be changed fundamentally by this increase in informatic ability. The psycho-social implications of this technologically mediated human-computer and human-human communication are quite profound. Providing the knowledge and technology required to empower people to make a positive difference with information technology could foster the development an attitude of social responsibility towards the usage of this technology and may be a profound step forward in modern social development. Applications which are intended to improve quality of life, such as, applications in medicine, education, recreation and communication must become a social priority.

USING TECHNOLOGY TO IMPROVE QUALITY OF LIFE

The potential of this technological capability to improve quality of life can be best understood when it is actualized into the lives of real people with real needs. The Human Performance Institute at Loma Linda University Medical Center is an interdisciplinary research center which is leading the effort to utilize the latest in human computer interface technology to "make the world a better place". The primary research areas are in developing interactive interfaces which enable severely disabled individuals to lead productive lives, and in the design of environmental systems which support experiential interaction with information systems in such a way as to help maintain a state of general good health.

The following are real world cases that demonstrate the utility of this technology to change the future of disabled individuals.

-Crystal, an 18 month old "C1 quadriplegic" (complete paralysis from the neck down, requires a respirator in order to breathe) was the first person to use this biocybernetic technology in a medical setting. Processing of electropotential changes along the eye and adjacent muscles into a biological signal enabled this child to interact in real time with the displays on the monitor, in short, "her eyes became her hands" in generating commands to the screen. The activity was direct, the implications profound: She was able to enter into a unified feedback loop where direct real time response to a physiological signal was used to modify and improve that psycho-physiological source. In this case, her capacity to learn and interact with the world willfully was restored.

-Andy, a 10 year old C2 quadriplegic whose speech is confined to the breathing patterns of his respirator to such an extent that it requires better than a minute to make a verbal request found himself in a spatialized environment where commands from facial muscles enabled him to "fly around" in a 3d computer environment. This was the first time in 5 years where he was able to willfully control something in his environment without the aid of others.

- A 17 year old car accident victim who was motivated to rehabilitate his impaired psycho motor skills through an "air guitar" interactive system which converted the weak bioelectric signals from his impaired muscles into "rock and roll" music.

We have also developed the BioCar, a primitive yet functional demonstration of telerobotic devices under direct biocybernetic control. The BioCar is a simple demonstration of how the biosignals can be used to control objects within an environment. For this demonstration a remote control car from Radio Shack was modified so that it can be controlled from the parallel port of a standard IBM compatible PC. Since there are only seven discrete functions (there is no proportional control) that the car can perform (forward, forward left, forward right, stop, reverse, reverse left and reverse right) then it takes a minimum of three sets of electrodes to control all of the functions (2³=8). The BioCar software is responsible for interpreting the bioelectric signals from the user and sending commands to the remote control car.

-Michael, a 27 year old engineer recently paralyzed in an auto accident was able to navigate the BioCar through a very complicated course using the muscles of his face and arms. The same system that allowed him to control this toy car could be easily adapted to control his wheel chair or some type of robotic arm. The potential to empower the disabled to become functional members of society can be realized through biocybernetic interface design.

The next effort of our lab was to expand the utility of this biocybernetic controller. We modified a nintendo game to accept commands from our system as if they were coming from the regular hand controller. This simple modification allows disabled children to use whatever muscle activity they have control of to play the same games as normal children. This generalized biocybernetic controller opens up an enormous resource of compelling games which can be integrated into rehabilitative therapy. From the control of virtually nothing to really something, we can get coordinated motion from patients at a much earlier time. Instead of some arbitrary task, they can work with computer generated objects that have specific motions associated with them; getting the associated feedback of watching themselves pick up a virtual object even though you may lack the physical strength to pick up a real object.

Future efforts will focus on adapting the biocybernetic controller beyond games and toys to functional information systems. The capacity to operate interactive educational multimedia systems will open a whole new area where human expressivity can be optimized in applications that customize an educational environment to the capabilities of an individual.

CYBERNETIC HEDONISM

The other focus of our efforts is in developing highly interactive, biocybernetic systems where biological signals can modify an environmental chambers' parameters allowing the user to bioelectrically interface with spatialized environments. We believe that such physiologically modulated environmental systems may have a health preserving function. Interfaces to control stimulation can adaptivly utilize any biosignal. The result is the capacity to create a stimulus regime that accelerates relaxation and facilitates stress reduction. This is an application of wellness maintenance technology.

"The Nirvana Express"

THE MICROSCOPE OF THE MIND

The goal is to extend these environmental control systems into new methods of investigative research. Such as a test of basic cognitive functionality or the capacity to maintain attentional focus necessary to complete an iterative series of cognitive tasks. Data fusion of sensor data with user interaction parameters will allow meaningful correlation's to be made across various performance modalities. A goal of this application is to seek to identify a qualitative difference between the two performance/behavior states and then investigate various methods of quantifying that difference in a way that can be generalized.

It is postulated a difference will be seen in the modulation of some of the natural rhythms. It is also postulated that a cognitively induced modification would be consistent in an individual but would most likely be different between individuals. The psycho-social-behavioral nature of individuals factors into initial assessment of their cognitive function. Other indicators of cognitive function are short-intermediate-long term memory, sound judgment and the ability to identify similarities in related objects. Performance of these cognitive functions is a strong indicator of the biologic health of the brain. Poor performance is highly correlated with organic brain dysfunction.

THE POTENTIAL OF THIS NEW PARADIGM OF BIOCYBERNETICS IS LIMITED ONLY BY THE IMAGINATION(and funding) OF THE USERS

Send all responses to:

Dave Warner

Medical Neuroscientist

Human Performance Institute

Loma Linda University Medical Center

11406 Loma Linda Drive

Loma Linda Ca. 92354

909-799-6190 Voice

909-799-6106 Fax

davew@well.sf.ca.us Email

Special thanks to Dave Gilsdorf and Patrick Keller for their ongoing efforts in making the world a better place

REAL MEDICAL APPLICATIONS

OF

VIRTUAL REALITY TECHNOLOGY

Dave Warner

Medical Neuroscientist

Human Performance Institute

Loma Linda University Medical Center

Virtual Reality technologies are technologies which support an experiential interaction with in a computationally sustained environment. Virtual Reality technologies represent a fundamentally new way for humans and computers to interact. For not only do these technologies translate natural human actions of communication, such as speaking eye-movements and body gestures into computer commands , but they also render information to the human in multiple sensory modalities, that is spatialized audio, 3D graphics and various somato-sensory forms.

To date the major effort of companies developing these technologies has been primarily , to cater to the military, entertainment and construction industries.

This is very unfortunate. While no one will argue the economics' of this bias, there is a strong humanitarian consideration that is being neglected. It is true that these technologies will profoundly impact the entertainment industry, and they should, for interactive media is truly a vehicle for participatory governments of the future. However, if we miss this opportunity to fully exploit the positive humanitarian potentials in EDUCATION and MEDICINE this period in history will be looked upon as one of the truly great missed opportunities in techno-social evolution.

Here at the Center for Really Neat Research in Loma Linda University we have a moral imperative to "Dare to Care" and an operational mode of action of "Lead by Example" in our efforts to fuse High Tech with High Touch in the fields of medicine and education.

The following are some examples of our efforts:

Immediate Medical Applications for Virtual Reality Interfaces

Normal people are naturally enabled. They are born with the capacity to interact with the world and willfully manipulate their environments. Disabled people have lost the capacity for such interaction and manipulation through either trauma or disease. Advanced human-computer interface technology that has been developed as natural user interfaces for interaction with virtual reality has immediate application in re-enabling the disabled persons. While virtual reality promises to solve many problems in the future, the immediate application of these advanced interfaces can improve the lives of millions of today. At the Loma Linda University Medical Center we have had many successes in utilizing these technologies. The utility of these devices has already been demonstrated as augmentative communication devices, as environmental controllers, as therapeutic tools in rehabilitation and as tools for quantitative assessment for diagnostic evaluation. Patients who have lost the ability to communicate verbally have successfully used an instrumented glove configured in a gesture to speech mode. Spinal cord injury, stroke and traumatic brain injury patients have virtual reality technology to manipulate virtual objects and practice specific skilled motor tasks. Quadriplegics have used physiological input devices to move objects on the screen with only their eyes and to play virtual instruments merely by contracting face and neck muscles. These are just a few examples of immediate uses for this promising technology that can profoundly improve the quality of life of real people today.

The Virtual Reality technologies also are showing great promise in the field of psychiatry. In a recent experiment a real-time performance animation system was used to encode facial expressions of an actor and then generate a 3D talking head with realistic facial expressions that could interact with children that were in the hospital. This Virtual Teacher taught several classes on anatomy to a group assembled in a classroom and then made individual bedside appearances in the children's room over the hospital television system. The reaction of the kids (and the doctors) was overwhelming. The potential for a system such as this to augment the quality of life of hospital bound children is profound.

Now for some Theory.

Virtual Reality is a paradigm shift in the way we think about mass communication and information technologies. Consider the following:

Remapping the Human-Computer Interface

for Optimized

Perceptualization of Medical Information

In the distant past Medicine was an art, the practice of medicine was guided mostly by refined heuristics and intuition. All the external senses were used in the evaluation of the patient. Visual, auditory, tactile, olfactory and gustatory cues were all integrated to give the healer a perception of what to do. With the development of science and technology the practice of medicine has slowly shifted from being intuition based to being guided by the results of objective tests. In many ways this is progress, though in other ways we seem to have forsaken our own senses in favor of machines, thus removing ourselves from the determination of the problem. The ever increasing ability of technology to quantitate complex physiological parameters and to image volumetric anatomical structures are taking us to a point where we will soon be unable to assimilate all the available information through the traditional means (i.e., numbers and graphs). Recent attempts to solve this problem have focused primarily on advanced visualization techniques. While much progress has been made in this field, the visual sense is finite and is reaching its saturation level.

Enter Virtual Reality. Virtual reality technology is primarily interface technology that renders computer information onto multiple human sensory systems to give a sustained perceptual effect (i.e., a sensation with a context) while monitoring human response in the form of gestures, speech, eye movements, brain waves and other inputs. This interface also allows for a natural interaction with abstract data sets providing an integrated experiential encounter with information. This new technology provides us with the capacity to move into a new paradigm, a paradigm where the physiological integration of a pansensory rendering of medically relevant information provides an enhanced capability to discriminate between classes of complex dynamic interactions involved in pathophysiological processes.

Much attention has been given to enhanced visualization techniques. Dynamic volumetric stereoscopic rendering methods have greatly enhanced our capacity for visual assessment of medical information. We need however to be careful that we do not become photo-chauvinistic and forget that we have other senses. There are relevant concepts from sensory physiology that are now within the resolution of the interface technology. This new technology increases the number and variation of simultaneous sensory inputs, thus making the body a sensorial combinetric integrator. A good working knowledge of sensory physiology and perceptual psychophysics can help us optimize our future interactions with the computer. Aside from the basic neuroscience issues of modality, duration, intensity, distribution, frequency, spatial displacement, contrast, inhibition, threshold, adaptation, transduction, conductance and transmission (to name a few) we must identify the optimal perceptual state space parameters with in which information can best be rendered. We must also identify which types of information are best rendered by each specific sense modality.

New technologies and techniques have recently become available that allow for the rendering of data via auditory means. Not only can we now represent any data set in the form of sound but we can also spatialize the displacement of multiple sound sources giving us simultaneous exposure to different dynamic data sets. In these spatialized environments we can shift our attentional focus from source to source for real time comparison of multiple sets of data. Devices now exist which can stimulate the sensation of pressure, vibration, texture and temperature. This is a relatively untouched field as far as abstract data representation is concerned. These modalities combined with somatotopic placement also provide for spatial coding of the rendered information. The implementation of vision, hearing and touch technologies allow for simultaneous sensation of multiple independent and dynamic data sets that can be integrated physiologically into a single perceptual state.

Yet to be fully embraced by the virtual reality community are the olfactory and gustatory senses, smell and taste. While their current integration is questionable, their potential impact is quite profound. Recent work in olfactory science has identified at least 30 basic smells. Technologies under current development will be able to deliver quantified combinations of these smells for a wide range of distinct perceptual states.

In the area of taste the development of automated food processing will eventually allow for the doctor to get a taste of complex data. The use of smell and taste to help convey the state of complex systems may seem like quite a reach of the imagination. However, the possibility that these senses may help discern subtle changes in complex systems warrants investigation.

We are embarking on an adventure that promises to change our relationship with the computer forever. With the immersion of all the external senses into virtual reality, our ability to perceptualize medically relevant information in an interactive mode will greatly enhance our capacity for improvisational investigation (stand up research). This is truly a paradigm shift and the beginning of a new era of computer assisted medicine.

THE NEUROREHABILITATION WORKSTATION:

A CLINICAL APPLICATION FOR MACHINE-RESIDENT INTELLIGENCE

Dave Warner, Jeff Sale, Stephen Price, Doug Will

Human Performance Institute

Loma Linda University Medical Center

ABSTRACT

The Neurorehabilitation Workstation is described. The need to maintain a clinical perspective motivates the comprehensive nature of the system, which integrates multiple data acquisition devices, interface technologies, advanced analytical techniques, and multi-sensory rendering capabilities. Emphasis is placed on machine-resident intelligence embedded at several levels.

INTRODUCTION

The field of Rehabilitation applies techniques and resources from many disciplines and is constantly seeking to improve the measurement of human performance and the assessment of therapeutic efficacy. We have had considerable success recently in our attempts to transfer new technologies into the clinical setting for such purposes. Devices such as gloves to measure hand motion dynamics, surface EOG and EMG sensors for eye movement and muscle contraction, and lightweight pressure sensor arrays for gait analysis show great promise in therapy. At the same time, our efforts to make these transfers permanent have been impeded by the lack of standard platforms, interfaces, inaccessible file formats, as well as the medical community's lack of time, technical expertise, and adequate budgets. Until now no cost-effective solution appeared possible.

Recent developments in human-computer interface hardware and software, data analysis, and expert systems suggest this is no longer the case. We are currently exploring a solution, the Neurorehabilitation Workstation (NRW), which integrates these technologies and methods into a comprehensive system designed specifically for the clinic. In addition, we hope it may be generic enough to act as a standard for other similar applications.

The success of the NRW depends on four things; modular design (for distributed processing and adaptability), integration of several data input devices into a single platform within a common interface protocol, implementation of machine-resident intelligence (neural nets, fuzzy logic) on several levels, and creation of a development environment driven by clinical needs. We detail aspects of these features below.

DATA INPUT

A necessary feature of the NRW is the integration of a variety of data input devices into a single system to include EEG, EMG, EOG, ECG, dynamic bend sensors, pressure sensors, audio and video digitizers, etc. The resulting capacity for data fusion allows for meaningful correlations to be made across various performance modalities. The devices and their hardware boards connect to an external module, and a high speed bus will route the data both to a central multi-tasking server and to the rendering subsystem for immediate feedback. The server should be intelligent enough to automatically implement a custom configuration of input device parameters, interface functionality, and relevant records based on the device(s) connected and the identity of the operator(s) and patient(s) currently at the system.

DATA MANAGEMENT

The maintenance of medical record integrity is a significant issue. Such integrity is achieved through security protocols, standardized data formats, error handling, and semi-automated database archiving. The data management subsystem tasks also include linking the device data with the patient record and specifying sensor-specific data formats and structures.

INTERACTION MODALITIES/METHODOLOGIES

The user interface will be based on new theories of human-computer interaction methodologies , computer-supported cooperative work, knowledge engineering, expert systems, and adaptive task analysis The system will monitor a user's actions, learn from them, and adapt by varying aspects of the system's configuration to optimize performance. Adaptable on-line knowledge-based help using text, graphics, and animated tutorials provide interactive learning and navigation.

DATA ANALYSIS