ATON Project: CREATE/UCSB Tasks
Introduction
The ATON project is a collaboration between the Computer
Vision and Robotics Research Laboratory at the University of California,
San Diego, the
Center for Research
in Electronic Art Technology (CREATE) at the University of California,
Santa Barbara, and the
CalTrans Test-Bed
Center For Interoperability . The project involves topics as diverse as robotics, computer
vision, distributed multimedia processing, and virtual reality. The tasks
being undertaken at the CREATE Lab are described here.
Virtual Environment Technology
Virtual reality technology
(also known as telepresence, immersive user interfaces, or virtual environments:
VE) is finding more and more widespread application as a fundamental user
interface metaphor, moving beyond its original uses in simulation-based
environments. This user interface paradigm will be of great use to the
intelligent vehicle initiative, and the federal operation test, and also
as a front-end to pure-simulation-based systems.
The VE-related work
we are proposing for this topic center around three main topic areas:
1) Distributed programming infrastructure for
VE systems
2) Multimedia output for VE systems
3) Navigation and interaction in VE systems
Background: The DRIVE System and Project
The development of the
Distributed Real-time Interactive Virtual Environment (DRIVE) system has
been motivated by the desire for a portable, scalable, and easily programmable
immersive multimedia user interface system. The DRIVE system architecture
is based on a distributed, real-time, object-oriented database that represents
the virtual "objects" in one or more virtual worlds. The database is connected
to (visual and/or aural) renderers (one or more per user), user input "vehicles,"
and software applications that can change the state of objects in a world.
DRIVE is a collaborative project of UCSB's CREATE lab and the Swedish Institute
of Computer Science. In addition to the DRIVE system, we have used a number
of other VE software packages, including MultiGen, Erdas Imagine, CosmoWorlds,
Maya, and others. We expect to extent this list to new packages in the
future, and to integrate them into our work.
Previous work at CREATE
has focused on VE systems architecture, complex world-building tasks, and
simple distributed processing for real-time multimedia applications. In
the current project, we will extend and integrate these results with a
state-of-the-art communications infrastructure for mobile interfaces, and
middle-level software APIs and support libraries for transparent distribution
of multimedia processing and I/O functionality.
Task Area 1: Distributed programming infrastructure
for VE systems
Introduction
A great deal of effort
has been invested over the last decade in scalable distributed computing
on multiprocessor systems. Recently, the CORBA (Common Object Request Broker
Architecture) standard has gained wide acceptance as a framework for interprocess
communication and coordination over (potentially) wide-area heterogeneous
networks. Several software suppliers now have CORBA-compliant software
packages (Object Request Brokers or ORBs) that allow programs written in
a variety of object-oriented languages (e.g., C++, Java, or Smalltalk)
running on a wide range of hardware platforms to interoperate.
Given these developments,
it is now possible to perform wide-area distributed processing with little
of the programming overhead associated with traditional multi-tier applications.
This advance in technology has lead to a number of new application architectures,
and to the widespread use of CORBA in many application domains, even areas
where real-time interaction and controlled latency are required. Based
on our previous work, however, we have experienced a number of problems
in exact CORBA compliance by commercial ORBs, and widely varying performance
characteristics.
In this task area, we
propose (a) to develop a comprehensive software package to test ORBs for
their standards compliance and performance characteristics, (b) to create
an infrastructure that can support complex multi-user distributed VE systems
with (wireless) mobile participants, and(c) to develop a high-level programmers
interface (API) for distributed multimedia programming.
Task 1A: CORBA Interoperability and Performance
Measurements
In this task, we will
develop and document a suite of programs for measuring the standards compliance
and relative performance of different CORBA ORB implementations, language
bindings, and delivery systems. We will deliver a portable and well-documented
suite of programs for testing ORBs, and publish our results for several
off-the-shelf commercial and public-domain ORBs.
The task is to develop
a software framework for evaluating the interoperability and performance
of various CORBA implementations -- both from the point of view of the
ORB and the host language mapping. We intend to deploy a test suite on
several different ORBs (e.g., Iona/Orbix, Imprise/VisiBroker, and DNS/SmalltalkBroker)
using different hardware/operating system platforms (e.g., SGI, Sun, and
Mac) and development languages (e.g., Java, C++, and Smalltalk).
The research contribution
will be the test suite software and potential insights into the relative
performance of various ORBs and language interfaces relative to one another.
We intend to distribute this software suite into the public domain for
use by other researchers, by those evaluating specific ORBs, and by ORB
developers and vendors.
Task 1B: Distributed Programming over Wireless
Networks
Our VE work requires
a low-level communications infrastructure for mobile computing and I/O.
This will likely be based on one of the new standards for medium-range
(on the order of tens of meters) high-speed communication. The two candidates
here are the 2 Mb/sec "BlueTooth" consortium technology, and the recently
standardized IEEE-802.11b technology (with CCK modulation), which offers
up to 11 Mb/sec.
In this task, we will
integrate such a wireless network with high-performance distributed programming
libraries based on CORBA and direct socket connections. We are interested
in both problems related to the wireless hardware infrastructure, and scalability
issues of medium-scale wireless LANs (with tens of nodes in a relatively
small volume).
Our research contribution
will be in the evaluation of new wireless LAN technologies, and hopefully
in the development of ORB-based multi-users wireless applications frameworks.
These will have obvious potential commercial spin-offs.
Task 1C: An API for High-Performance Distributed
Programming
The third software component
for supporting our VE applications should provide a uniform LAN- or WAN-oriented
distributed computing API based on multi-vendor standards such as CORBA
and Java RMI. This effort will deliver a high-level API for distributed
multimedia signal processing that is independent of the external data representation,
remote function call discipline, and transport details. This API should
be tested via implementations on top of CORBA IDL, Java RMI, and Smalltalk
sockets.
The design and implementation
of this API will be the primary research result from task area 1; we hope
to deliver a working, scalable, and effective high-level API for a range
of multimedia application areas that can be "hosted" over CORBA, native
sockets, or other low-level communications infrastructures.
Previous Work
Related projects undertaken
by the partners have focused on standards-based distributed processing
using off-the-shelf technologies such as CORBA. We have collected significant
experience in both CORBA-based real-time applications and in distributed
computing using "home-grown" socket-based object-oriented data and processing
distribution frameworks.
Deliverables
The results of the VE
infrastructure task will be both software and documentation. We will deliver
the CORBA interoperability and performance test suites and results of our
tests, the support software developed to support the wireless LANs, and
lastly, the newly developed MM VE API and implementations of it on the
basis of CORBA, Java RMI, and native sockets.
Task Area 2: Multimedia Output for VE Systems
In order to use VE systems
more effectively, we need to integrate immersive output devices such as
stereo-optic displays (head-mounted or goggle) and flexible spatialized
sound output. Several solutions to the first problem are available off-the-shelf
with the advent of low-cost head-mounted stereo-optic displays such as
the new generation of immersive "goggles" and 3-D screen-based shutter
glasses. To integrate these we need to customize the DRIVE visual renderers
for the new output devices. We also intend to use an alternative output
system based on the EON VE delivery software.
For increased realism
of VE worlds, we require real-time multi-channel spatialized sound processing
so that sounds can appear to emanate from their "virtual positions" and
can be tracked by the user in harmony with the visual cues provided by
the optical renderers. This kind of "aural renderer" has been difficult
in the past because of the different latency introduced by the visual and
aural output systems, the network overhead, high levels of latency jitter,
and the complexity of high-quality models of sound spatialization. For
maximum flexibility and scalability, we will develop a stand-alone spatial
sound output system, and provide several kinds of interfaces to it: CORBA
IDL, direct calls from the APIs of the VE delivery systems we choose (e.g.,
EON and/or MultiGen), and a low-level socket-based protocol for use by
other programs.
The system will support
the description of (a) sound "worlds" in terms of the architectural acoustical
properties of a space (e.g., a concert hall or an automobile passenger
compartment), and (b) the radiation characteristics of the sound sources
in that space. The aural renderers for each of the (possibly many) VE users
in that space would then perform the real-time digital sound mixing, filtering,
and reverberation process (aural rendering) for that user's particular
position in the world. Because the description of the space and its sound
sources is kept abstract, different aural renderers can support users with
differing sound output hardware (e.g., one user listening over headphones
and another with an eight-channel surround sound loudspeaker system) co-habitating
the same VE world.
The research contribution
of this work will be the new abstractions for sound and acoustical space
description, and the real-time distributed aural rendering techniques we
intend to develop. Potential commercial products include the description
layer (e.g., for scalable computer games) and the several kinds of aural
renderers we develop.
Previous Work
Previous and in-progress
work at the partner labs has centered on the development of VE systems
with diverse input/output media, and on complex world-building tasks (see
the DRIVE project). We also have several past and current parallel projects
related to spatial sound synthesis, processing, and distribution (performance).
Deliverables
The results of the MM-output
task are to be working telepresence systems that demonstrate various modes
of navigation and presentation within virtual worlds based on several kinds
of models. We will construct navigation software ("vehicles") that use
several different interaction devices. The DRIVE renderers will incorporate
multi-channel spatial sound and limited-bandwidth video feeds. We will
describe these results in a written report that includes sample output
from the various virtual worlds.
Task Area 3: Navigation and Interaction in VE
Systems
The final task is to
develop a framework for using a variety of input devices for navigating
within VE worlds and for interacting with virtual objects. This will include
input devices with six degrees of freedom (i.e., 3-D position and orientation)
such as wands and gloves, and also application-specific I/O devices (e.g.,
car-like input devices or virtual musical instruments).
To integrate this kind
of hardware into DRIVE and other VE delivery systems, we need to write
"gesture-mapping" software for the input devices to map input signals into
user movement (navigation) in a VE world or interaction with virtual objects.
To use these with other VE delivery systems (such as off-the-shelf commercial
systems), we need to be able to program the "glue" code to interface between
the hardware input drivers and the vendor-provided APIs of the VE systems
(e.g., EON and/or MultiGen).
Given the basic level
of input gesture mapping, we can then experiment with higher-level gestures
and virtual interaction, such as realistic driving interaction (coordinated
input from several devices), virtual models of complex interactive objects
(e.g., musical instruments), and reactive virtual objects.
The research content
of this work relates to the abstractions for gestural input and the framework
for building new vehicle programs for new kinds of navigation and interaction
in VE systems.
Previous Work
In our previous VE systems,
we have restricted ourselves to standard input devices. Several of our
other projects, however, have developed alternative input schemes based
on using the MIDI (Musical Instruments Digital Interface) software interface
to communicate between programs, and to drive VE systems with MIDI-derived
data.
Deliverables
There will be both low-level
and high-level deliverables from this task. The simple driver-level "glue"
code and DRIVE "vehicles" will be provided, and will also be used for the
higher-level research into complex physical interactions in VE systems.
References and Previous Work
CORBA/Distributed Processing
"OMG CORBA Event Services
Specification." OMG document 97-12-11. Available from www.omg.org.
"OMG CORBA Time Service
Specification." OMG document 97-12-21. Available from www.omg.org.
"OMG CORBA Telecommunications
Domain (Audio/Video Streams) Specification." OMG document 98-07-13. Available
from www.omg.org.
"OMG CORBA Wireless
Access and Terminal Mobility RFP." OMG document telecom/99-05-05. Available
from www.omg.org.
"OMG CORBA Mgmt. of
Event Networks RFP." OMG document telecom/98-09-05. Available from www.omg.org.
"OMG CORBA Draft Adopted
Real-time CORBA 1.0 Specification." OMG document ptc/99-05-03 . Available
from www.omg.org.
Pope, Stephen T. "The
Siren Music and Sound Package for Squeak Smalltalk." See http://www.create.ucsb.edu/Siren/index.html.
Distributed VE
Frécon, Emmanuel
and Mårten Stenius. "DIVE: A scaleable network architecture for distributed
virtual environments." Available at http://www.sics.se/~emmanuel/publications/dsej/.
See also http://www.sics.se/publications/dive.html
Hagsand, Olof. "Interactive
MultiUser VEs in the DIVE System." IEEE Multimedia Magazine, 3(1), 1996.
"Standard for information
technology, protocols for distributed interactive simulation." DIS-ANSI/IEEE
Standard 1278-1993, American National Standards Institute, 1993.
Pope, Stephen T. et
al. "CREATE DRIVE Project: Distributed Real-Time Interactive Virtual Environments:
Phase 2 Final Report." See http://www.create.ucsb.edu/drive/phase2/report.html.
Spatial Sound
Adler, D. "Virtual Audio
- Three-Dimensional Audio in Virtual Environments." SICS Internal Report,
ftp://ftp.sics.se/pub/SICS-reports/Reports/SICS-T--96-03--SE.ps.Z, 1996,
ISRN: SICS-T--96/03-SE.
Kendall, Gary. "A 3-D
Sound Primer: Directional Hearing and Stereo Reproduction." Computer Music
Journal 19:4, Winter, 1995.
Kendall, Gary. "The
Decorrelation Of Audio Signals and its Impact on Spatial Imagery." Computer
Music Journal 19:4, Winter, 1995.
MacDonald, Alistair.
"Performance Practice in the Presentation of Electroacoustic Music." Computer
Music Journal 19:4, Winter, 1995.
Malham , David G. and
Andrew Myatt. "3-D Sound Spatialization using Ambisonic Techniques." Computer
Music Journal 19:4, Winter, 1995.
Pope, Stephen T. "The
Use of 3-D Audio in a Synthetic Environment." in Proceedings of the 1993
International Computer Music Conference.
Pope, Stephen T. "Sound
and Music Processing in SuperCollider." See http://www.create.ucsb.edu/htmls/sc.book.html
Roads, Curtis, Stephen
T. Pope, Giovanni De Poli, and Aldo Piccialli, eds. "Musical Signal Processing."
Swets and Zeitlinger, 1997.
Rocchesso, Davide. "The
Ball within the Box: A Sound-processing Metaphor." Computer Music Journal
19:4, Winter, 1995.
I/O Devices
Collected articles in
two "Computer Music Journal" special issues on interactive performance
interfaces, Vol. 14 Nrs 1 and 2, Ed. By Stephen T. Pope. MIT Press, 1990.
Pope, Stephen T. "Real-Time
Performance via User Interfaces to Musical Structures." in Proceedings
of the International Workshop in Man-Machine Interaction in Live Performance,
Pisa, Italy, June, 1991. Reprinted in INTERFACE 22(3): 195-212.
Figures
DRIVE Architecture
See figures in http://www.create.ucsb.edu/drive/phase2/paths.html
Examples of DIVE Worlds
See http://www.sics.se/dive/demos/images/
CORBA Infrastructure
See figures in http://pent21.infosys.tuwien.ac.at/Research/Corba/OMG/arch2.htm
URL = http://www.create.ucsb.edu/aton/overview.html
Created: 1999.10.08; LastEditDate: 2000.11.02