$Revision: 89 $
Copyright © 2004–2006 Infiscape Corporation
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with the Invariant Sections being Appendix H, GNU Free Documentation License, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in Appendix H, GNU Free Documentation License.
Some of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and Infiscape Corporation was aware of the trademark claim, the designations have been printed in caps or initial caps.
$Date: 2007-04-09 09:48:38 -0500 (Mon, 09 Apr 2007) $
Table of Contents
List of Figures
List of Tables
List of Examples
simulated_digital_device.jdef)position_proxy.jdef)input_device.jdef)<help>
Tagsurface_viewport.jdef)cluster_manager.jdef)position_proxy.jdef)position_proxy.jdef)user.jdef)This book explains the process of configuring VR Juggler and its component modules. It is written for VR Juggler 2.0. Throughout, we will make reference to the modules JCCL and Gadgeteer, both of which are at Version 1.0 with the release of VR Juggler 2.0. Readers who are not familiar with the modular structure of VR Juggler are referred to the VR Juggler Project website, specifically the high-level description of the Juggler Suite. Understanding this will be vital to understanding how to configure the various software components utilized by VR Juggler.
Table of Contents
Since its inception, VR Juggler has been designed to be highly flexible and dynamic. At the heart of this is the high degree of configurability present in VR Juggler. In spite of being a fundamental aspect of using VR Juggler, configuration of VR Juggler has always been difficult. Through this book, we will present VR Juggler configuration in a way that will make the process more approachable and that will enhance understanding of VR Juggler as a whole. We begin by examining the basic aspects of the VR Juggler configuration system, encapsulated within the JCCL module, and we then provide high-level explanations of how to configure general pieces of VR Juggler. At the end of this book, the reader will find appendices that explain the configuration of specific devices. The appendices include tips and troubleshooting points to help with commonly encountered complexities.
A key feature of VR Juggler is its portability, both in terms of operating systems and in terms of immersive system. As a virtual platform for the development of virtual reality (VR) applications, VR Juggler facilitates the transition of VR applications from the desktop to the immersive system. It also aids in the migration of the application across a variety of immersive systems. VR Juggler achieves this degree of portability by separating the details of the immersive system from the application. Hence, those details can change without requiring any changes to the application.
To handle variations in immersive visualization systems, VR Juggler has been designed to be highly configurable. VR Juggler is not written to work with any specific type or classification of immersive system. Instead, it handles the general aspects common to all immersive systems in code while offloading the device-specific pieces to run-time configuration. The components of VR Juggler have been parameterized to ensure portability and scalability across a staggering variety of display and input hardware combinations.
Component parameterization is captured within configuration elements, or simply “config elements.” Understanding config elements is vital to understanding the process of configuring VR Juggler. Throughout this text, we will discuss config elements and their utility as the means for configuring VR Juggler.
A configuration element is the fundamental unit of configuration within VR Juggler. Each config element is composed of one or more properties and is identified by a user-specified name. Each property has a type such as string, integer, pointer, etc., and properties can have either a fixed number of values or a variable number (zero or more) of values. The conceptual structure of a simple config element is shown in Figure 1.1, “Simple Configuration Element”.
In the figure, we see that the property named Size has two values: width and height. The valid type for both values of this property is integer. In general, all the values of a given property will have the same type. While there are exceptions to this, we will not go into such details at this time. In nearly all current cases, remember that the values of a property all share the same type.
The list of valid types for properties is shown in Table 1.1, “Valid Property Types”. A basic understanding of the valid types and their run-time interpretation in C++ can be useful when entering values. For example, note that the property type “integer” can be interpreted as a wide variety of C++ integer types. In some cases, the component being configured may limit the range of valid integer values that can be accepted from a config element. Hence, it can be important to know what the range of valid values is for a given property.
Table 1.1. Valid Property Types
| Type | Valid C++ Interpretation |
|---|---|
| integer | int, unsigned int, long, unsigned long, long long, unsigned long long, short, unsigned short, char, unsigned char |
| float | float, double |
| boolean | bool, integer |
| string | std::string, char* |
| config element pointer | std::string, char* |
| config element | jccl::ConfigElementPtr |
The concept of a config element “pointer” reduces to simply the name of another config element. At run-time, the loaded config elements are stored in a table that is indexed by element name. This means that within a given configuration, all the elements must have unique names.
Note that the last entry in Table 1.1, “Valid Property Types” is “config element.” This means that config elements can be nested within other config elements. A nested config element is often called an embedded config element. There is no limit to the depth of config element nesting, but in general, config elements are rarely nested more than one level deep. Doing so can complicate user understanding of the configuration process. It is hoped, however, that continued improvements to the VR Juggler configuration editor will alleviate confusion related to nested config elements. For now, it is sufficient to understand that a nested config element is just another property type.
The specification of property types, the number of allowed values, and other structural aspects of config elements are defined by configuration definition , or simply “config definitions.” The relationship between config elements and config definitions is quite similar to that of objects and classes in object-oriented languages. A class defines structural and behavioral information, and an object is an instance of a class. At any time, there can be many instances of a single class, each referenced by a different variable. Similarly, a config definition defines structural information, and a config element is an “instance” of the definition. There can be many config elements for a given definition, and each config element is identified by its unique name. Config definitions even support a limited form of multiple inheritance.
Config definitions are found in files that are named based
on the config element type defined therein and have the extension
.jdef. To allow versioning of config
elements, a given config definition may have many versions, each
of which is identified by an integer value that is incremented
each time a structural change is made to the definition. Config
definitions are loaded by scanning directories listed in an
environment variable. VR Juggler performs this scan before any
attempt to load a config file is made to ensure that the run-time
environment knows about all existing config definitions.
As one would expect, configuration elements are what provide the run-time configuration information for VR Juggler. A casual observer would say that VR Juggler is configured by the configuration files that are loaded at run time. In reality, VR Juggler is reconfigured by the individual config elements that are processed. As a whole, VR Juggler starts out with no configuration. With the help of the Configuration Manager, the VR Juggler kernel main loop performs what is known as run-time reconfiguration wherein the config elements read from the configuration file(s) given at run time are processed. This means that VR Juggler dynamically reconfigures itself over and over again until the kernel determines that no more reconfiguration can occur.
The Configuration Manager maintains three lists of config elements:
The Configuration Manager delivers individual config elements from the pending list to those entities within VR Juggler that accept them. These entities are known as config element handlers, and they normally register themselves with the Configuration Manager when they are created.
For each config element in the pending list, the Configuration Manager tests to see if the dependencies of the element are satisfied. With the presence of config element pointers, one config element may depend on one or more other config elements being processed first. In other words, if config element A depends on config element B, then B must be in the active list before A can be given to a handler. If it is determined that the dependencies for the config element are met, then the Configuration Manager delivers the config element to its handler. If the dependencies are not met, then the config element goes back into the pending list, and another attempt to process it will be made later.
When a handler receives a config element, it attempts to read the values from the individual properties and configure (or de-configure) itself. If the handler concludes that its configuration procedure was successful, the Configuration Manager moves the config element from the pending list to the active list. If the handler cannot process the config element correctly, the config element goes back into the pending list so that another attempt to process it can be made later.
In typical usage, the startup process of a VR Juggler application shows the reconfiguration process. Attempts are made to process config elements until the pending list reaches a state where it is considered “stale.” This means that a sufficient number of reconfiguration attempts were made with no config elements being processed. When all config elements are processed, a message similar to the following will be printed to the console where the application was executed:
[0002315/000] DBG:ConfigManager::pendingNeedsChecked: Pending list is now
STALE: 0 items still in the pending list
[0002315/000] DBG:---- Pending list: 0 items ----
----------------------------------
If there should be one or more config elements that are left unprocessed for some reason, then output similar to the following will be printed:
[0002333/000] DBG:ConfigManager::pendingNeedsChecked: Pending list is now
STALE: 1 items still in the pending list
NOTE: These items have been specified in the configuration
but have not been loaded.
This may be a problem in the configuration OR
it may be waiting for more configuration information.
[0002333/000] DBG:---- Pending list: 1 items ----
ADD -->My IS-900 type: intersense
----------------------------------
Most people getting started with VR Juggler interpret one of the above messages to mean that something about the application has “stopped.” In reality, nothing has stopped permanently. The kernel main loop is still running, but there is nothing new in the pending config element list to process. The above message may be printed again later if new configuration elements are added to or removed from the Configuration Manager.
Neither of the above output messages is an error message. Rather, both forms of the output state that the reconfiguration process has reached a point where there is nothing new to process. The longer form indicates one of two things: either the configuration had extra, unused config elements or the configuration is incomplete. In the latter case, that usually means that a config element did not have one or more dependencies met. As we will see later in this chapter, it is possible to satisfy missing dependencies without restarting the application.
The output from the VR Juggler configuration process can sometimes be misleading. Due to its dynamic nature, attempts and reattempts are made to configure components within VR Juggler. Thus, if something in the configuration fails early in the process, it may cause problems much later on. It is therefore a very common mistake to assume that the last message printed to the console, no matter what it says, is where a problem occurred. If some part of the configuration fails, there may be other error messages or warning messages printed early in the output that explain what caused the later failure. As a rule of thumb, if a problem occurs with the execution of a VR Juggler application, scroll up to see if there are errors or warnings printed to the console before concluding that the last message is related to the fatal error.
VR Juggler configuration files, or simply “config
files,” are those files that end with the extension
.jconf. They contain zero or more configuration
elements, and they can reference other configuration files. The files
themselves are stored in a format called XML, a text-based, and therefore
cross-platform, file format.
In VR Juggler terminology, a configuration is a
collection of config elements that provide a complete set of
parameters needed to execute a VR Juggler application successfully. A
configuration can be defined by one or more
.jconf files, and .jconf
files can be mixed and matched to give different configurations. The
concept of a configuration is especially important within the context
of the VR Juggler configuration editor,
VRJConfig. We discuss
VRJConfig more in the following
section.
Ultimately, the concepts surrounding VR Juggler configurations are hierarchical in nature. Each piece of the configuration puzzle is composed of smaller pieces. The following summarizes the hierarchy starting from the smallest piece and going up from there:
Value: A storage unit capable of holding a single datum of any type.
Property: A property has zero or more values of the same type.
Configuration element: A config element is composed of one or more properties.
Configuration file: A config file contains zero or more config elements.
Configuration: A configuration is made up of one or more config files that together provide a comprehensive collection of settings for parameterized components in VR Juggler.
Since .jconf files are stored as plain
XML, they can be edited manually using a standard text editor. While
sometimes quick and convenient, this method is not recommended
because it is very easy to make mistakes that break the configuration
or that violate the XML syntax. Furthermore, editing configuration
files by hand with a text editor retains the old mindset of testing
configurations by running an application, shutting it down, changing
the configuration, and running the application again. VR Juggler
offers much more power and efficiency in terms of configuration
editing than this. In this section, we introduce
VRJConfig, the VR Juggler configuration
editor, and we explain how it can be used for static and dynamic
configuration editing. Since the theme of this book is configuring VR
Juggler, it is important for us to explain
VRJConfig. However, an in-depth usage
guide is beyond the current scope of this document. We will explain
the concepts present in VRJConfig and
provide enough explanation of its usage to get readers comfortable
working with it.
VRJConfig is the Java®-based graphical user interface (GUI) for creating and editing VR Juggler configurations. A screen shot of the VRJConfig Configuration Editor is shown in Figure 1.2, “VRJConfig”. It is based on Tweek, an implementation of a distributed model/view/controller system. Without going into great detail about the Tweek Java® API, it is sufficient to state VRJConfig exists as a collection of JavaBeans™ loaded by and executed within the Tweek JavaBean™ Loader.
The VRJConfig Configuration Editor represents the low-level, all-purpose editor. It is capable of editing any config element. The representation of a config element provided by this editor shows both the property values and the structure of the element.
It is important to understand, however, that
VRJConfig strives to provide an
interface for configuration editing as opposed to the lower level
concept of property editing. As was noted earlier,
.jconf files can be edited manually using an
ordinary text editor. This approach would be considered property
editing because the user concentrates on the values of individual
properties without necessarily considering the whole config
element let alone the whole configuration. With the
VRJConfig Configuration Editor, the
intention is to provide more of a big-picture view of a collection
of config elements as a complete configuration rather than as an
assortment of disparate property values.
Before we dive into the use of VRJConfig, we will explain more about the purpose of VRJConfig and how it relates to VR Juggler applications written in C++ or Python. We have just stated that VRJConfig strives to provide an interface for editing configurations. The name VRJConfig is a catch-all name for various components. The VRJConfig Configuration Editor was already shown above. The other two components are the VRJConfig Control Panel and the VRJConfig Config Definition Editor. These two components will be examined later.
In the most abstract terms, the purpose of VRJConfig is to provide a valid configuration for use by VR Juggler. There is an informal contract between VRJConfig and VR Juggler[1]. The contract is simple: VRJConfig guarantees that it will output valid XML describing a correct configuration of VR Juggler. Note that completeness of the configuration is not included in the contract because the notion of a “complete configuration” is not defined within the scope of VR Juggler. VR Juggler can run with no configuration at all, it can run with a fully immersive, multi-screen, cluster configuration, or somewhere in between. Hence, completeness depends on the needs of each user and his or her execution environment.
Unfortunately, the implementation of the contract is quite complex. As of VR Juggler 2.0 Beta 1, only the valid XML aspect is fully implemented. VRJConfig validates the config files and config definition files using the XML Schemata that describe those file formats. This ensures that the input to VRJConfig and the output it creates are valid XML and are valid in terms of the respective file formats. Therefore, VR Juggler is free from re-validating the input.
Users who edit configuration files and config definition files by hand using a text editor do not get the benefit of validation through the XML Schemata. Therefore, it is possible for VR Juggler to fail to load a given file or to crash altogether if the input is not valid. Manual editing, therefore, is highly discouraged. At the very least, users should load manually modified files into VRJConfig to ensure that they are valid XML.
The VRJConfig Configuration Editor utilizes a multi-document interface (MDI) paradigm. Each internal window is called a configuration context, or simply a “context.” Each context should be treated as a complete configuration to achieve proper use of VRJConfig. Throughout the discussion of configuration contexts, the main toolbar at the top of the VRJConfig Configuration Editor panel will be important. It is shown again in Figure 1.3, “VRJConfig Configuration Editor Main Toolbar” to distinguish it from the full editor GUI.
A new config context can be created either by opening
an existing .jconf file or by creating
a new .jconf file. In either case, a
new internal window will open within the
VRJConfig Configuration Editor.
The main toolbar has buttons for creating a new .jconf file
and for opening an existing .jconf file. They are
highlighted in Figure 1.4, “VRJConfig Configuration Editor New and Open
Buttons”.
Once the context is created, the view of VRJConfig will be similar to what was shown earlier in Figure 1.2, “VRJConfig”.
The purpose of a configuration context within
VRJConfig is to facilitate the
notion of editing a configuration as a whole. As such,
adding more .jconf files to an open
context is a common operation. Each context has its own
toolbar that is independent of the main
VRJConfig Configuration Editor
toolbar, and this toolbar is what must be used to extend or
modify the state of a given configuration context. The
context-specific toolbar is shown in Figure 1.5, “VRJConfig Configuration Context Toolbar”.
On the left side of the toolbar, we see the usual
icons for file creation, opening, and saving. To add a new
.jconf file to the existing config
context, either the or the
button must be used. Using the
or the
button from the main
VRJConfig toolbar will result in
the creation of a new configuration context.
On the right side of the context-specific toolbar, we
see a button with an icon that is a downward pointing arrow.
This button is only active when there are at least two
.jconf files open in the
VRJConfig Configuration Editor as
a whole. Clicking the button reveals a special menu shown in
Figure 1.6, “VRJConfig Context Modification Menu”. In the
menu, the selected .jconf files are
those that are open in the current context. Selecting a file
from the menu will add it to the current context.
De-selecting a file removes it from the context. In this
way, it is possible to build up config contexts using
different config files.
With multiple configuration contexts open, it is often desirable to move or copy config elements around. The MDI paradigm helps make this possible by allowing multiple windows to be visible at once, and cut/copy/paste support within the VRJConfig Configuration Editor completes the picture. Note, however, that the granularity of cut/copy/paste operations—including drag and drop—is at the level of config elements. Individual properties cannot be copied between contexts or between config elements.
On the left side of each config context window is a tree view of the config element hierarchy (as seen in Figure 1.7, “VRJConfig Config Element Navigator Panel”). This hierarchy is a dynamically updating organizational model of all the config elements in the current configuration context. Within the tree, names shown in bold text are categories. Names shown in non-bold text are config element names. As config elements are added to and removed from the current context, the tree structure will change. Empty categories are not shown.
The config element hierarchy also depicts the hierarchy inherent in config elements. That is, the nesting of config elements within config elements is shown, too. This allows quick, direct access to nested config elements for property editing purposes.
Adding new config elements to a configuration context is easy. Simply click the button shown above the config element hierarchy; it is always active. When clicked, a dialog box is opened that is used for choosing the type of config element to add to the context. The list, shown in Figure 1.8, “Config Element Chooser”, displays the human-friendly names of all known config element types. Entering text into the field labeled Config Element Type causes the selection to jump to the first element matching the entered text. This search is a rudimentary, exact-match type of search.
For a more advanced search capability, click the tab labeled Search. The dialog box will change to what is shown in Figure 1.9, “Config Element Search Dialog”. Initially, the config element list is empty, but as soon as text is entered into the search field, all elements whose human-friendly identifier match the entered text will be displayed. This search supports sub-string matches of search terms. The Search tab is useful to users who know exactly what they want or who have an idea of what to look for. In Figure 1.9, “Config Element Search Dialog”, the user started entering the word window as a search term, and all three possibilities for window-related config elements were immediately displayed.
Once the element type is chosen, click the button. A new element will be created in the current context with all of its property values initialized to their default setting. If multiple files are open within the context, another dialog box will be opened asking the user for a destination file for the new element. The first thing to do after creating a new config element is to set its name. In the config element hierarchy, new elements are created with a simple, unique name based on the human-readable description of the element type. To rename the element, either double-click on its name or right-click on the name and choose the item from the pop-up menu.
In some cases, it is necessary to triple-click on the name of a config element in the hierarchy tree. This is because Java® sometimes interprets a double-click within a tree as an expand/collapse command for that branch of the tree. To get the correct behavior every time, right-click on the name of the config element to get the pop-up menu and choose .
To remove a config element, click on its name within the hierarchy and click the button. Otherwise, right-click on the element and choose the item from the pop-up menu.
The largest panel in the configuration context window is devoted to editing configuration elements. The full config element selected in the element tree is shown in this panel. From this panel, the individual property values can be edited. The view shown in this panel is designed to illustrate the hierarchical structure of a config element. Panels are nested within panels to show the nesting of config elements within config elements. Multi-value properties are rendered using a layout that shows data grouping as opposed to nesting.
Editing properties with a fixed number of values is straightforward. Those that have a multiple values may need to be expanded to show the full list, but no extra steps are required to insert entries that can be edited. VRJConfig will always show the full number of values for this variety of property. For new config elements, it will fill in the default setting for each of the property's values.
Properties with a variable number of values are a little more tricky. This variety of property can have zero or more values set, and the default case is to have zero values set. As such, new values must be added before editing can take place. In Figure 1.10, “Editing a Property with a Variable Number of Values”, we see two highlighted buttons: one for adding a new value and one for removing an existing value. The button is always at the top center of the inset panel while the button appears on the right side of the inset panel. There will be one button per value present in the config element.
The values of a property with a variable number of values may be reordered using VRJConfig. Depending on the interpretation config element, this may be useful for changing the processing order of the values. An example of this is shown in Figure 1.11, “Changing the Order of Property Values”. The highlighted and buttons move the associated value up or down one position in the ordering of the property's values.
Editing a static configuration represents the usual configuration editing process. This means that configuration files are loaded from permanent into VRJConfig. When the editing is complete, the files are saved back to permanent storage to be loaded at a later time by VR Juggler. Following this procedure stays with the traditional configuration process, depicted as follows:
Create or edit a configuration.
Run an application to test the configuration.
Exit the application and go back to step 1 if more changes are required.
This process does not harness the full potential of VR Juggler. Through its run-time reconfiguration capabilities, VR Juggler can be reconfigured on the fly using VRJConfig as a remote configuration editing tool. That is, VRJConfig can connect to a running application and edit its configuration without requiring the application to be shut down and restarted. A full discussion of this topic is postponed until Chapter 5, Remote Run-Time Reconfiguration Using VRJConfig.
Besides the low-level Configuration Editor, VRJConfig includes a component known as the “Control Panel.” It is named this way because it is styled after the Microsoft Windows® Control Panel. Through the Control Panel, users are presented with high-level, or “custom,” editors that are designed for specific configuration tasks. Furthermore, the Control Panel provides configuration creation tools in the form of wizards that combine multiple high-level editors into a step-by-step configuration process. Whereas the Configuration Editor is capable of editing any type of config element, customized code must be written for use by the Control Panel to edit specific element types. As of this writing, editors exist for the Ascension Flock of Birds®, the Ascension MotionStar Wireless®, the Fakespace Pinch™ Gloves, and the InterSense IS-900™ devices. There is also an editor that presents a directed graph representation of the all the proxies in the config context.
The VRJConfig Control Panel is loaded by clicking on the Control Panel JavaBean™, which is nested under the VRJConfig category. When it is first loaded, the Control Panel appears as shown in Figure 1.12, “VRJConfig Control Panel Start Screen”. The individual panels within the Control Panel are a mixture of category icons and editor or wizard icons. Clicking an icon either navigates to a sub-category or opens a high-level editor or a wizard, depending on what the icon represents. The and buttons (in the upper right-hand corner) allow navigation through the panels.
The Control Panel is organized in a hierarchical manner.
This is similar to the way that the low-level Configuration Editor
organizes config elements in its navigator panel (refer back to
Figure 1.7, “VRJConfig Config Element Navigator Panel”). The two hierarchies
are similar, but the Control Panel hierarchy is not as dynamic as
the Configuration Editor's navigator panel. Indeed, the Control
Panel hierarchy is defined statically by the Control Panel
configuration file (interested readers can see it in
$VJ_BASE_DIR/share/vrjuggler/data/ControlPanel.xml)
in order to avoid requiring many mouse button clicks just to get
to the desired editor or wizard.
Similar to the low-level Configuration Editor, the Control Panel must be told what files will be edited. This is done by opening an existing configuration file. Note that the tool bar used by the Control Panel has the same buttons as the low-level Configuration Editor, shown earlier in Figure 1.3, “VRJConfig Configuration Editor Main Toolbar”. This indicates that the Control Panel has the same basic functionality for opening configuration contexts, but with the Control Panel, there is only a single context.
After opening a configuration file, navigate through the categories to find the editors that correspond to the contents of the file. The config elements within the file that have high-level editors will be displayed as icons within the Control Panel. Clicking on the icon will open the high-level editor for that config element. The changes made in the high-level editor can be undone by clicking the button or by closing the high-level editor and clicking the button in the Control Panel toolbar. Some examples of high-level editors can be found in Appendix F, VRJConfig Custom Editors by Infiscape.
As noted above, VR Juggler configuration files and config definition files use XML. Among other benefits, using XML allows for (relatively) simple updating and migration using XSL Transforms (XSLT). Prior to using XML for VR Juggler configuration files, there was either no way to automate configuration migration or a script had to be written in a language such as Python or Perl. XSLT offers the VR Juggler developers a comparatively convenient mechanism to implement configuration migration and update code. All that is needed to run the XSLT program on a configuration file or config definition file is an XSLT processor, which most modern operating systems include by default. Examples of XSLT processors are xsltproc (part of the GNOME library libxslt), Xalan-J, Xalan-C++, Saxon, and MSXSL.
VRJConfig makes use of XSLT internally to update config element versions automatically. The XSLT used for this purpose is found in the config definition files that come with VR Juggler. Each definition version, except the first, includes an “upgrade transform” to handle migrating from the previous definition version. When a config file is loaded, its version is tested to determine if it is up to date with the corresponding config definition. If it is not, VRJConfig will prompt the user to determine if it can proceed with the automatic upgrade. After doing so, users must be sure to save the config file.
With this introductory information complete, we can now move on to configuring VR Juggler. The remainder of this book is devoted to specific configuration topics. In general, configuring VR Juggler means configuring two large pieces: inputs and displays. In Chapter 2, Configuring Input, we explain the process of configuring input devices, both physical and simulated. In Chapter 3, Configuring Displays, we move on to the topic of configuring displays. The appendices of this book are devoted to configuration of specific input devices. Those readers who are already familiar with the general configuration of inputs to VR Juggler can refer to the appendices to find device-specific configuration instructions and tips.
Table of Contents
Input is critical to all immersive applications. Without it, achieving a sense of immersion is nearly impossible. VR Juggler provides a very advanced, robust, and flexible input management system through the Gadgeteer Input Manager. The flexibility of the Input Manager is due in no small part to the parameterization of components such as device drivers and input device proxies, but there is no denying the fact that configuring input for VR Juggler is difficult. Evolving features of VRJConfig are simplifying the process, but a thorough understanding of the input configuration process is invaluable regardless of features and capabilities offered by VRJConfig.
In this chapter, we will provide all the information needed to understand the configuration of input within VR Juggler. Fortunately, the design of the Input Manager reduces the process of input configuration to the same three conceptual steps regardless of what input device is being used:
Configure the input device, physical or simulator
Configure one or more proxies that point at the physical device
Configure zero or more proxy aliases
In the real world, there may be as many as five actual steps, but the above three sum up the overall process to follow every time input is configured.
The release of VR Juggler 2.0 Alpha 1 in March 2003 brought with it a new feature: device driver plug-ins. Instead of statically compiling all device drivers into the Input Manager, device drivers could now be loaded on demand. In order for this to occur correctly, however, the Input Manager now had to be told what driver(s) to load. This is achieved by configuring the Input Manager using a config element.
The Input Manager config element has three properties: a driver search path, a driver DLL name, and a directory to scan for drivers to load. All three properties may have zero or more values, but at least one must have a value in order for the config element to be useful. Each of the three is explained in the following subsections.
The driver search path provides the Input Manager with a list of directories where driver DLLs may be found. This property has a default setting determined at the time when the Input Manager was compiled. Based on the current list of supported operating systems, the default search path will be one of three values:
$VJ_BASE_DIR/lib/gadgeteer/drivers:
The default for Linux, Microsoft Windows®, Mac OS
X, FreeBSD, and other platforms where only one binary type
is supported.
On Microsoft
Windows®, the default is actually
%VJ_BASE_DIR%\lib\gadgeteer\drivers.
Platform-specific path details such as this may be
omitted in this document for brevity. Bear in mind that
internal handling of paths by VR Juggler and its
components is always done correctly for a given operating
system.
$VJ_BASE_DIR/lib32/gadgeteer/drivers:
The default for an N32 build on the IRIX® operating
system.
$VJ_BASE_DIR/lib64/gadgeteer/drivers:
The default for an N64 build on the IRIX® operating
system.
Since the base collection of device drivers that come with Gadgeteer are guaranteed to appear in the default search path, there is usually no need to set a value for the Input Manager driver search path property. The search path is configurable to allow users to tell the Input Manager where to find drivers they have written themselves that appear outside the VR Juggler installation hierarchy. When one or more values are provided through the driver search path property, the default search path is always appended to the end of the list. This ensures that user-specified directories will always be searched first when loading drivers.
When setting the driver search
path, environment variables are allowed. As with all uses of
environment variables in VR Juggler configuration files, they must
be specified using either the form ${ENV_VAR} or
$(ENV_VAR). Using $ENV_VAR or
%ENV_VAR% will not work in this case or any other
case. Any environment variable may be used.
The driver DLL
name property is used to name specific device drivers to load. The
name of the DLL to load must be specified in a special
platform-agnostic manner. Essentially, this form boils down to
removing the file extension from the DLL name. For example, the
Ascension MotionStar
Wireless® DLL may be named
MotionStar_drv.so,
MotionStar_drv.dylib, or
MotionStar_drv.dll depending on the host
operating system. Note that all three have a common base name:
MotionStar_drv. In the configuration of the
Input Manager, this common base name is the string to set for the
property value. This aids with portability of configuration files
and (we hope) reduces the mental overhead required to configure
the Input Manager on different operating systems where conventions
will almost certainly vary.
The driver scan path provides a mechanism for loading all the device drivers found in the named directory or directories. This behavior is very similar to the old Input Manager that knew about all the device drivers, but in this case, the drivers are not statically compiled into the Input Manager. Use of this property is helpful when there is no need or no desire to load drivers individually. Doing so increases the memory overhead of the Input Manager, but it dramatically reduces the possibility for configuration errors related to driver loading.
Try to avoid using the driver scan path with the loading of individually named DLLs. Doing so could lead to unpredictable, undesirable behavior if the scanning process finds a DLL already identified by the naming of a specific driver DLL.
As with the driver search path, environment variables can be
used in the string value(s) provided for this property.
Furthermore, there is no need to specify a platform-specific file
extension for use when scanning a directory for device drivers.
The file extension is determined at compile time based on the
target operating system. The search criterion used by the Input
Manager is that the files match the pattern
*_drv.<ext> where
<ext> is the platform-specific file
extension set at compile time.
The configuration process for input devices depends on the specific device being configured. In general, there is very little overlap among the configurations of the various device drivers that come with Gadgeteer. There are some exceptions such as the need to name a serial port and baud setting for some drivers, but this is hardly enough to provide the basis for a general explanation of how to configure input devices for use with VR Juggler. Instead, we will concentrate on the two categories of devices supported by Gadgeteer: physical devices and simulator devices. The term physical device is somewhat ambiguous because simulator devices get their input from a keyboard and mouse, and a keyboard and mouse are certainly physical input devices. The critical distinction is that simulator devices mimic the behavior of physical devices such as six-degree-of-freedom (6DOF) trackers using input from a keyboard and/or a mouse. As such, the configuration of simulator devices is very different from that of physical devices, and that difference is worth deeper examination.
A key aspect that is common to all input devices handled by the Input Manager is that they define input sources of one or more input categories. The input categories currently supported by Gadgeteer are the following:
Analog: Data in a continuous range with well-defined minimum and maximum values. Applications receive the data as normalized values in the range 0.0 to 1.0 inclusive.
Command: Discrete command input, often in the form of recognized spoken commands or pre-defined gestures.
Digital: Discrete on/off input, usually corresponding to simple button presses and releases.
Digital glove: Distinct combinations of fingers.
Gesture glove: Recognizable hand gestures based on knuckle angles.
Keyboard/mouse input handler: Source of keyboard and mouse events from the native windowing system.
Positional: Multi‒degree-of-freedom tracker data. Up to six degrees of freedom are supported.
String: Arbitrary sequences of characters, usually corresponding to spoken words or phrases.
Thus, a given device, physical or simulator, will fall into one or more of those categories. That same device may provide multiple input sources for a given input category. For example, a game pad may have eight buttons and two joysticks that operate in two axes. That game pad defines eight digital input sources (from the eight buttons) and four analog input sources (from the two axes of the two joysticks). We will speak of input sources more abstractly in the following sections, so it is important to understand what we mean by this term before proceeding.
Physical devices are those that provide what is traditionally considered immersive input. Typically, such devices have multiple degrees of freedom and/or provide specific information about what the user is doing. For example, a glove device could provide data identifying a specific hand gesture that the user is making. By attaching a 6DOF tracker sensor to the glove, both the position of the hand and the gesture it is making would be available to an immersive application. Placing tracker sensors all over the user's body provides full-body position and orientation information.
In terms of the Gadgeteer Input Manager, physically devices have thus far been characterized as requiring a device driver plug-in. The driver implements in software the communication protocol with the hardware so that data can be read from the device and interpreted before being passed on to the VR Juggler application. As of this writing, it is certainly true that each supported physical device has its own device driver plug-in. Though this could change in the future, it will be useful to remember this fact for the purposes of the topics presented here.
The process of configuring a physical device means configuring its device driver. As noted above, there is very little commonality in the process of configuring specific hardware devices, but it is vital to understand the importance of properly configuring a device driver as part of the overall input configuration. If the driver itself is not configured correctly, there is no way that the rest of the input chain can function properly. In other words, if the driver cannot read data from the input device, there will be nothing to pass on to the VR Juggler application. For instructions and tips on specific device configurations, refer to the appendices at the end of this book.
Since all physical devices do have a device driver plug-in, proper configuration of the Input Manager is absolutely critical. Configuration elements for device drivers are handled by the drivers themselves, and if the Input Manager does not load a required device driver, the config element cannot be processed[2]. As such, it is very important to remember to configure the Input Manager as part of configuring a physical device.
If individual devices are configured in separate
.jconf files, include an Input Manager
config element for loading the device driver in the same file
as the driver config element. The Input Manager can be
configured repeatedly, so there is no problem with having
multiple Input Manager config elements as part of a complete
configuration. Each Input Manager config element must have a
separate name, however, or else one config element may
overwrite another in the Configuration Manager's pending list.
The config element names can be made unique by including the
name of the driver being loaded. For example, a
.jconf file loading the IBox driver could
include an Input Manager config element named “Input
Manager IBox.”
The configuration of simulator devices is much more involved than that required for physical devices. All simulator devices read data from a traditional desktop keyboard and mouse and translate the data into information that mimics the behavior of the various device types supported by the Input Manager[3]. To configure a simulator device, there are three pieces that must be configured: a keyboard/mouse input handler, an input window, and the actual simulator device. We explain this in more detail below.
Keyboard and mouse input data are received using what are known as keyboard/mouse input handlers in Gadgeteer terminology. Keyboard/mouse input handlers are tied to a window in a given windowing system. That is, a desktop window of some form must be open and have focus in order for Gadgeteer to be able to receive keyboard and mouse input data. There are two options for achieving this when configuring VR Juggler: plain input windows that display nothing and graphics windows that display the three-dimensional scene. Both of these can be configured to feed keyboard and mouse activity into Gadgeteer keyboard/mouse input handlers. As such, we can think of both types of windows as event sources for keyboard/mouse input handlers. For the remainder of this book, we will normally refer to “keyboard/mouse input handlers” (the Gadgeteer device type) and to “event sources” (desktop windows that feed keyboard and mouse events into Gadgeteer keyboard/mouse input handlers). When necessary, we will refer specifically to plain input windows or to graphics windows. The use of graphics windows as event sources is discussed in more detail in the section called “Graphics Window Input”. For now, we will concentrate on the concept of event sources in general.
This property defines the relationship between mouse movement within the event source and the number of key presses registered by the event source. The notion of a key press in the context of a mouse is counter-intuitive, and an in-depth explanation of what this means is beyond the scope of this document. We must explain, however, that keyboard/mouse input handlers record input using the abstract concept of a key press. This corresponds to an event raised by the windowing system and handled by event sources for keyboard/mouse input handlers. A keyboard/mouse input handler records how many times an event was raised since the last time its data sample buffers were retrieved by the Input Manager. In the context of mouse movement, the interpretation of how much the mouse moved is based on the number of mouse motion events raised rather than the number of pixels the mouse pointer would have moved physically. Such behavior is necessary for recording input when the mouse is locked to the center of a window. Setting the mouse sensitivity tells the keyboard/mouse input handler how to interpret the mouse motion events. A value of 1.0 for this property indicates that one pixel of movement will correspond to one key press. A value of 0.1 says that ten pixels of movement translate to one key press. The default value is 1.0.
This property is used for configuring a cluster of graphics nodes. The topic of cluster configuration is addressed in Chapter 4, Configuring a Cluster.
Before we explain the use of keyboard/mouse input handler as data sources for simulator devices, we should note that keyboard/mouse input handlers can be used as input devices for VR Juggler applications. As we will see in the section called “Device Proxies”, there is a proxy type and corresponding device interface through which applications can receive keyboard and mouse input. In this case, there is no special device driver to load.
As stated above, input windows are normal desktop windows, but they display nothing. Their sole purpose is to handle keyboard and mouse events from the native windowing system and feed those events into a Gadgeteer keyboard/mouse input handler.
This two-valued property sets the origin for the window when it is opened.
In VR Juggler, the lower left-hand corner of the display is the origin for positioning windows. This is in contrast to the way some windowing systems work wherein the display origin is the upper left-hand corner.
This two-valued property defines the dimensions of the window in pixels.
The “display number” property is used with windowing systems that define multiple displays on which windows can be opened. As of this writing, there is one such windowing system supported: the X Window System. The value set for this property is used as an index into an array of available displays defined in another config element called the Display System. The default value is 0, and this property will be ignored when using windowing systems that do not support multiple displays.
This property points to another config element
of type keyboard_mouse_device. The
value is the configured name of the keyboard/mouse
input handler to which events will be fed. This
property must be set to point to a valid config
element of a keyboard/mouse input handler in order for
the input window to be useful.
The “lock key” property defines a
key that, when pressed, causes the mouse pointer to be
locked to the center of the window. Such a capability
is useful when it is necessary for the mouse to remain
within the borders of the window at all times. The
lock key also unlocks the mouse pointer when it is in
the locked state. The default value is
KEY_NONE which indicates that there
is no key that toggles mouse pointer locking.
Related to the lock key, the “start locked” property indicates where the mouse pointer should be locked to the center of the window as soon as it opens. This setting defaults to false.
Each input window object handles events from the windowing system in its own thread. Depending on the nature of the windowing system, the event handling loop may never block on its own. To prevent the input window thread from starving all the other threads, this property sets a sleep time (in milliseconds) for the input window thread. Adjustment of this property may be necessary to get optimal behavior on a given computer. The default setting causes the input window sample loop to sleep 75 milliseconds between each event handling pass.
With an event source configured, we can now configure a simulator device. The simulator device type to configure will be based on what is needed by the application. There are five simulator device types from which we can choose:
Simulated analog device: Provides a variable number of analog input sources in a configurable range based on two key presses (increment the value, decrement the value).
Simulated digital device: Provides a variable number of digital (on/off) inputs based on a single key press.
Simulated digital glove device: Provides a pair of digital inputs representing finger combinations from the left and right hand in a manner similar to a pair of Fakespace Pinch™ Gloves.
Simulated glove gesture device: Provides a variable number of gesture inputs in a manner similar to a Virtual Technologies CyberGlove.
Simulated positional device: Provides a single of 6DOF positional input source based on a combination of twelve key presses.
Below, we review each of these in detail and explain how to configure them in a manner that makes them easier for VR Juggler application users to understand. Before doing so, however, we must note that the config element for each simulated device type has a property for a keyboard/mouse input handler proxy. Device proxies will be explained in detail in the section called “Device Proxies”, but for now, it will be helpful to keep in mind that the simulated device types do not refer directly to a keyboard/mouse input handler. Instead, an extra level of indirection is introduced; namely, a proxy to the actual keyboard/mouse input handler.
Furthermore, all simulated device types have key presses configured as key/modifier pairs. On a standard keyboard, there are alphanumeric keys (A‒Z, 0‒9) and modifier keys (CTRL, ALT, and SHIFT). A key/modifier pair is thus a combination of an alphanumeric key and a modifier key. Since mouse input is also considered a key press in the eyes of the Input Manager, a key/modifier pair may also be the combination of mouse movement and a modifier key or the combination of a mouse button press and a modifier key.
Simulated analog devices provide input values from a continuous, configurable range. At the application level, the input will be received as values normalized to occur in the range 0.0 and 1.0 inclusive. Each configured simulated analog device can provide zero or more analog input sources, each configured as a pair of key presses: one key press to increment the value and one to decrement it.
This property specifies a pointer to another config element of type keyboard_mouse_proxy that in turn points to the keyboard/mouse input handler from which keyboard and mouse input will be read.
This property defines a list of key/modifier pairs that are used to identify the key presses for incrementing the value of the analog input sources. There may be zero or more of these, and the amount must match the number of decrement key presses defined. The number of key presses defined indicates the number of analog data sources that this device has.
This property is the complement to the “increment key” property. It too specifies a list of key/modifier pairs that are used to identify the key presses for decrementing the value of the analog input sources. There may be zero or more of these, and the amount must match the number of increment key presses defined. The number of key presses defined indicates the number of analog data sources that this device has.
The “delta” property defines the change in the analog value per key press.
This property sets the minimum possible value for the device as a floating-point value. The default is 0.0.
This property sets the maximum possible value for the device as a floating-point value. The default is 255.0.
This property sets the starting value for the device as a floating-point value. This must be in the range [min, max]. The default is 0.0.
Note that a single simulated analog device shares the settings for the delta, range minimum, range maximum, and initial value across all its input sources. If there is a need to vary these settings, multiple simulated analog devices must be configured.
Simulated digital devices translate keyboard events and mouse events (both motion events and button presses) into digital input. A single simulated digital device may provide many digital input sources.
This property specifies a pointer to another config element of type keyboard_mouse_proxy that in turn points to the keyboard/mouse input handler from which keyboard and mouse input will be read.
The “digital button key” property defines a list of key/modifier pairs that provide digital input sources. There may be zero or more key/modifier pairs, and the number defined specifies the number of input sources that this device has.
Simulated digital glove devices mimic the behavior of gloves that have distinct gestures or finger combinations. Such gloves are usually in the same vein as the Fakespace Pinch™ Glove where the glove sends a signal indicating which fingers are pressed together or are pressing a pad on the base of the palm. Gestures are identified as being either on or off. This is use is different enough from gesture recognition devices that a separate device type was created for it, but it is worth noting that a “digital” glove could act as a gesture recognition glove with the correct information provided to the Input Manager.