ABSTRACT
Platform independent user interface builders allow GUI applications
to be created on one platform; that one set of specifications
and source code can then be automatically compiled for several
other platforms without modification (potentially). The implications
for minimizing software development, porting, and maintenance
issues are monumental. I evaluated four commercially available
tools with rigorous and objective processes and evaluation forms,
and provide those results along with opinionated insights. I determined
which of these tools and methodologies are viable for creating
portable, medium- to high-complexity GUI applications. This is
a fast moving area and the impact of these technologies can shape
the nature of creating software for open systems today and into
the future.
1 Introduction
This document describes the first phase of an ongoing project
to form GEIS' new, more efficient software development environment
which is critical to maintaining a leadership position in the
information services market. This was carried out at the University
of Maryland Human-Computer Interaction Laboratory by faculty research
assistant Richard Chimera and supervised by Dr. Ben Shneiderman.
Phase I was the evaluation of commercially available user interface
builders that support multiple hardware, operating system, and
windowing system environments, or "platforms." Thus
we have the term platform independent user interface builders,
or PIUIB. (In clear context the words "platform independent"
may be dropped or the words "builder" and "tool"
interchanged for the sake of clarity through brevity.) Working
with the User Interface Group and others at GEIS, the HCIL was
charged to review available PIUIBs, select the elite few to evaluate
rigorously and objectively, and to recommend if and which PIUIB
GEIS should adopt.
1.1 Motivation
In GEIS' current software development environment, which is true
of almost every software producer, each product supported requires
a unique team of developers for each platform on which the software
is delivered. It is very costly to maintain products when capability
changes must be performed on multiple sets of code. This duplication
of effort costs money and limits the number of products offered.
A new development environment that supports platform independence
will not only improve the quality, usability, and consistency
of the interfaces GEIS designs for their products, it will help
to significantly reduce costs and "time to market" and
therefore allow them to offer a broader range of services to customers.
By being a leader in this technology as it reaches viability,
GEIS may significantly increase its advantage over competitors
who do not inculcate PIUIBs into their software development environment.
There are more benefits of using today's PIUIBs. In contrast to
early entrants into this field that were limited by what has become
known as the "lowest common denominator" effect--provide
only the functionality common to all platforms, the leading products
today incorporate a superset of the functionality found
on individual platforms. This is an advantage because a good concept
found on only one platform is abstracted and offered on all platforms
by the PIUIB. Another benefit is that new look-and-feel's or new
releases to current ones are automatically supported by the PIUIB
with no modification to application source code (though recompilation
would be necessary).
1.2 Perspective
The most important aspects of a platform independent user interface
builder concern software construction methodology/infrastructure
and portability issues. Classic user interface issues, which transcend
those first two, are next in importance. In practice, these three
issues are often inseparable in that the salient features of a
PIUIB are its user interface objects and construction methodology
as supported by the underlying software engineering and infrastructure
and porting paradigms. Phase I of this project looked at the PIUIBs
in the large while keeping an eye to smaller concerns.
Evaluations were executed from the perspective of learning a tool's
big picture and enough of the details in order to implement the
front end to a typical organization communication application--a
subset of GEIS' Business Talk 2000 product. The application required
creating associated windows, dialog boxes, hierarchical menus,
buttons, scrolling lists, editable textviews, and icons/graphic
labels.
2 Project Components and Processes
2.1 Selection Process
The selection process started with collecting information about
possible products from trade magazines, colleagues, and marketing
material from vendors. We collected information on over 50 products,
some of which did not cover all the platforms in which we were
interested (appendix A). The products were rated via 19 selection
criteria -- four "must-have" attributes, ten important
attributes, and five attributes that would be major pluses.
Must-have Attributes
1. Generates interfaces for at least two platforms differentiated by hardware and operating system; at least one platform is not UNIX.
2. Has facilities for prototyping GUIs "easily" (we have in mind an interactive design tool with widget set).
3. "Chameleon" behavior, which is defined as the ability to conform to the various look & feel standards on each platform.
4. Company's track record. Issues include how long company has
existed as software developer, how long current product has existed,
installed user base, number of employees, revision frequency,
etc. This is mostly to capture highly negative features of the
company, and is more subjective and less researched than it should
be.
Important Attributes
5. Ability to build own widget primitives from supplied widgets.
6. Can develop on HP/Unix platform.
7. Generates interfaces for Windows 3.X.
8. Prototypes not necessarily throw-away, they can be extended to be an "efficient" final application.
9. Uses direct manipulation as the main building process.
10. Can merge application code into iterative design versions easily.
11. Provides platform-specific industry-standard widget combinations for high-level objects, i.e. dialog boxes.
12. Ability to create own icons.
13. Maintains and uses user developed widget collections as templates or libraries.
14. Performance speed good; interoperability within platform such
as cut&paste, access to operating system and native window
system, etc.
Major Pluses
15. Uses OOP and can generate C++ code.
16. Allows for surface level end-user customization.
17. Updates for package itself are backwards compatible.
18. General user satisfaction with building and execution issues.
19. Reference checks.
The must-have selection criteria forced the bulk of the products to drop out of contention mostly due to a lack of platform coverage or weak GUI facilities. This narrowed the pool down to eight products worthy of further investigation: Aspect, CommonView, GUILD, JAM, Open Interface, OpenUI, WNDX, and XVT. Since this was still too large a group to evaluate deeply and they all satisfied the must-have criteria, we rated these eight tools on a four point scale for seven of the ten important criteria (appendix B). This, along with our desire to evaluate tools representing differing methodologies and to favor tools with object-oriented design and an interactive widget layout feature, led us to the tools we evaluated: Open Interface, XVT, GUILD, and OpenUI.
We knew this type of evaluation would carry on into the future and would be of importance to many divisions at GEIS and many professional communities, thus we realized the power of making the review process as objective as possible. Review objectivity would allow many things:
Of all the project components, the most important was the actual
evaluation of a PIUIB and this is given its own section below.
2.2 Learning and Use
A two week period was allocated to learn and use each PIUIB. Only
one at a time was attended so as to minimize confusion over PIUIB
characteristic attribution. The author was the sole reviewer for
all five tools. The various parts of the installation, tutorial,
prototype creation, and porting were timed to the minute with
a stopwatch and rated on various scales. In most cases, the first
platform used was UNIX/Motif, and then the prototype was ported
to the PC/Windows 3.0 platform.
The learning and use of the PIUIB consisted of these phases:
1. Open the shrink wrap of the "Read This First" document or installation instructions and understand them.
2. Install all software.
3. Read tutorial and follow interactively with the tool as requested.
4. If necessary read additional documents to achieve a comfortable level of general knowledge of the tool; this was the case for all tools. This could involve further issues on one or more of the interactive widget layout program (all PIUIBs reviewed provided one), user's guide, or programmer's guide.
5. The heart of the process -- build the application prototype. The prototype was specified by GEIS personnel via both a text description and a set of Windows look & feel screen snapshots of the existing Business Talk 2000 product. Functionality included:
6. Test the prototype within the interactive layout program itself, if possible.
7. Compile and run the prototype, go back to modify the prototype, and recompile to determine handling of iterative development.
8. Stop the building process with 2.5 days left. Install the PIUIB on the second platform. Transfer the prototype specifications to the second platform and load into the interactive layout program to corroborate its basic functioning and to check that the prototype specifications were parsed and laid out correctly. Then compile and run the prototype on the second platform to finish the port. One day was allocated for the installation and complete port to the second platform.
9. Spend the last 1.5 days filling out the evaluation forms and
essays described in the next section.
2.3 Evaluation
We wanted to get answers to these characteristics of a PIUIB:
The primary difficulty with these questions is to find their answers
in terms of objective definitions and measures. After much searching
through the user interface, human factors, and software engineering
and metrics literature, we discovered that Dr. Deborah Hix and
her colleagues at Virginia Tech had created an evaluation tool
very close to what we wanted. It is a tool for evaluating user
interface development tools in general. It does not specifically
address issues concerning platform independence or traversal.
It was designed and validated as being a reviewer-independent
form that could be used to compare different UI development tools
running on different platforms reviewed by different people.
2.3.1 Dr. Hix's Evaluation Tool
Dr. Hix's evaluation tool is best described by her own account
as reported in Communications of the ACM, March 1991. Hix's tool
has been used by over 100 organizations since its inception two
years ago. This section is not meant to replace her own introduction
and descriptions, but to provide some context and explanation
for how we have decided to use the tool in our evaluations. Dr.
Hix's evaluation tool consists of 35 spreadsheet-like pages with
issues as rows and evaluation criteria as columns. Rows are organized
by user interface construction content areas such as menus, text,
graphics, devices, etc. Columns are organized by implementation
techniques for affecting the current issue (row), binary values
for direct manipulation, text-coded language, and other techniques.
There is an additional column organization set aside to rate the
ease of performing the current task with the tool, which uses
a three-level scale. The tool covers a wide range of topics in
order to touch on as many areas as is tractable that are relevant
to building user interfaces. Naturally, the tool does not fit
perfectly with our needs; it covers some areas in which we are
not interested as well as not providing enough coverage in some
areas in which we have high interest.
Dr. Hix's evaluation tool constitutes a general methodology for
quantitatively evaluating user interface development tools.
Again, we feel an important part of this project is to create
and use an evaluation process that is as objective as possible.
This will be critical for future comparative or replicated research--if
evaluations of other tools are performed by the same reviewer,
if the same tools are evaluated by a different reviewer(s), if
a new version of the same tools is evaluated by the same or different
reviewer(s), etc. This quantitative approach can even be used
to compare objectively these platform independent GUI builders
with platform specific GUI builders, avoiding much of the concern
as to whether apples are being compared to oranges.
We have shaped the Hix tool to better suit our needs. We have
eliminated entire pages/forms that have little or no relevance
to our current evaluation (issues of animation, multimedia, etc.).
For pages of the tool in which we have interest but the form itself
does not provide the appropriate perspective or coverage, we have
instead composed small essays on the topic using the form as a
structuring guide. We use the remainder of the pages in standard
fashion. This involves assigning weights to each line item so
as to specify the importance of that line item in relation to
other line items in calculations of summary totals. We defined
a weight of 1 to mean that we are not interested in that line
item. A weight of 3 indicates that we are interested in that item,
but we are not evaluating it specifically in building the prototype.
We rated the item as well as we could from information (including
users/references) provided about the tool by the company and other
reviews of the tool. A weight of 5 indicates that we are interested
in the item and are evaluating it in building the prototype. The
pruned Hix tool we used appears in appendix C with its own glossary
at the end. In a few cases, we have redefined a term as given
in Hix's glossary, some only slightly and some with a related
but arbitrary complete redefinition. Our new term definitions
follow Hix's glossary.
The Hix tool uses a spreadsheet to input data and create summary
reports using various macros they have defined. It is important
to note that a current limitation of the Hix tool is that if any
item on a page is used, the entire page is used in calculations.
This may show up as low marks for a certain area simply because
many items on the page were not weighted highly thus reducing
the sum potential. Though not yet investigated, we could create
our own macros to have the important items carry a more representative
weight or eliminate unwanted items with respect to summary analyses.
2.3.2 Additional Evaluation Forms
The Hix tool doesn't address all our needs, so we used additional
evaluation techniques. One was structured essay questions, most
based on the Hix questions that did not cover deeply enough issues
of which we were concerned. Another was a free-form catch-all
essay of unusually good or bad aspects of the PIUIB. Another technique
was a modified QUIS (Questionnaire for User Interaction Satisfaction)
form, which collects subjective, emotional data in an objective
manner. We added questions to the QUIS where it is weak with respect
to this project, e.g., questions regarding objects and models
supplied by the PIUIB, its use of direct manipulation, and porting
procedures. These supplemental forms are in appendix D.
Hix's tool does not address time issues and we wanted measures
of effort that were as objective as possible. Therefore we created
a Hix-like form as another supplemental evaluation technique.
This page was also spreadsheet-like with rows that covered the
the majority of the install, learn, and build process and columns
for preparatory time, performance time, and times for finding,
reading, and applying help. Additional columns were for ease of
doing the activity and for the porting issues of look, feel, doing,
and help. Our Hix-like form is in appendix D, also.
As a pilot test of the critical phases of the building process
and evaluation process and forms, the prototype building and evaluation
phases were performed once with a non-platform independent user
interface building tool with which the author was already familiar,
Developer's Guide by Sun Microsystems. Minor changes were made
to our Hix-like evaluation form.
The creation of the evaluation process and forms was the most
difficult aspect of this project, and unfortunately was somewhat
disappointing in terms of focused results and analysis. The current
evaluation forms are an excellent start and provide data that
are quite valid. There should be wider and deeper coverage for
issues peculiar to platform independence as well as the interrelated
roles of the interactive editor, resource language, and API, among
other lesser deficiencies. The more useful evaluation results
are the author's detailed experiences, essays, anecdotes, and
recommendations.
3 Evaluation Results Summary
It is difficult to digest all the evaluation data collected during
the reviews. Table 3.1 in section 3.1 attempts to summarize crucial
aspects by combining relevant evaluation data, deriving numerical
values for the twelve evaluation criteria, and weighting and summing
them to produce a final rating value. Brief comparative analyses
that support table 3.1 are in section 3.2. No statistical analysis
has been performed on the evaluation data because five is too
small a sample size.
The detailed evaluations in descriptive form for Open Interface,
XVT, GUILD, and OpenUI are chapters 4 through 7, respectively.
The detailed, raw evaluation data in its original form for Open
Interface, XVT, GUILD, and OpenUI are appendices E-1 through E-4
respectively.
3.1 Evaluation Results
I conclude that Open Interface's power, methodology, and maturity
suits GEIS' technical needs and recommend its adoption. XVT is
a solid tool with minimal surprises but is too conservative and
lacks broad GUI coverage for full-scale use by GEIS. GUILD and
OpenUI are not suitable for GEIS due to their esoteric flavor
or narrow characteristics.
It is most interesting to note that the clear frontrunner, Open
Interface, began as an internal effort and was later cleaned up
and made into a commercial product. Perhaps the fact that their
own personnel had to use their PIUIB to create a variety of
platform independent client applications is a significant reason
for the superior power, methodology, extensibility, and documentation
of their tool compared to the other tools reviewed.
The rows of table 3.1 list items of general information, prototype
building milestones and timing data, and the twelve evaluation
criteria; the columns are the five tools. Most items should be
self-explanatory. "Port Install & Load Time" is
the time it took to install the PIUIB on the second platform and
load the prototype specification file into the interactive layout
editor to see how everything looks. Most PIUIBs didn't make it
through a complete compile of the prototype on the ported platform
for various reasons, among them that the HCIL has outdated compilers
for the PC which are not supported by the PIUIBs. However, loading
the prototype specification file into the interactive layout editor
is a reasonable approximation under the circumstances.
3.2 Brief Comparative Analyses
This section provides supporting material for the values found
in table 3.1.
Ease of Learning
XVT was the easiest tool to learn because it has the least amount
of functionality which is presented in a straightforward manner.
Open Interface and OpenUI use object-oriented techniques and are
more difficult to learn than XVT. Open Interface is harder because
it has so much functionality to learn. OpenUI uses its own version
of message passing and further abstractions to invoke functionality
which requires time to learn its specifics. GUILD is very hard
to learn because it uses strict, pure object-oriented communication
yet the documentation is so lacking that the mechanism specifics
are basically unknown.
Ease of Use
Once learned and a certain comfort level is reached, the PIUIBs
can be used similarly effectively and straightforwardly. XVT loses
a point for not having any testing within their interactive design
tool and forces the user to invoke a shell, compile, and then
test. Both GUILD and OpenUI lose a rating point for not describing
their message passing mechanisms clearly and for making certain
actions needlessly obtuse. OpenUI loses an additional point for
its abstract widget layout mechanism of using a logical grid system
that, while not misguided, at least ended up in the wrong form.
Documentation
Open Interface's documentation is well organized, has excellent
presentation style, and is voluminous. XVT had occasions where
related issues were omitted or full disclosure was not provided
or referenced. OpenUI's approach was rather simplistic and gives
one the feeling there should be a further, large companion document
for programmers. As noted earlier, GUILD's major downfall is its
documentation which is sorely lacking purposeful direction, coverage,
and explanation of the large (and probably powerful) object messages
set.
Online Help Facilities
GUILD clearly had the best online help facilities but they weren't
enough to make up for what the documentation lacks in high-level
explanations. It was context sensitive, indexed, and had hypertext
features. XVT's online help was good but not very context sensitive.
Open Interface openly admitted there was no online help available
nor is it (straightforwardly) producible for built applications.
The curious thing about OpenUI is that it has what seems to be
elaborate help mechanisms for the application authors, yet don't
provide any online help for their own tool!
Technical Support
XVT's technical support was efficient and professional; the trouble
call was received, delegated, and responded to within two hours.
GUILD's technical support was even more conscientious, but one
got the feeling there's only a handful of programmers who must
also double-duty as technical support. OpenUI had competent and
reasonably efficient tech support. Open Interface gave a cold,
standoffish reception and took overnight to initially respond
to the trouble call and yet another day to answer it.
Range and Capabilities of Supplied Widget Set
Open Interface has a vastly superior widget set compared to all
others reviewed, including two hierarchy browsing widgets. XVT's
widget set is slightly above average though more programmer intervention
is necessary than should be to make everything work. GUILD also
has a slightly above average widget set that functions reasonably
with promising graphics features. OpenUI has the weakest widget
set of the PIUIBs evaluated.
Prototype Evolves into Final Application Program
Open Interface handles evolution the best, most notably by leaving
intact code modified by the user as it regenerates the same file
in response to additions/changes made in the interactive layout
editor. GUILD and OpenUI handle evolution well due to their extremely
minimal use of C code. XVT has absolutely no mechanisms for incorporating
evolutionary changes, all code merging must be done by hand. All
PIUIBs treat their end product as the final application program
not a throw-away prototype, whether or not the evolution aspect
is handled well.
Conformity to Style Guide of Each Platform
The PIUIBs conformed to the platforms' various look & feel
standards. The only significant departures were by Open Interface
in the menu behavior. Menu items did not unhighlight properly
in Motif or OpenLook leaving a rather ugly menu bar. It is because
Open Interface emulates look & feels rather than use their
APIs that this happens, but it should be easily fixed.
Briefly described, there are two ways to have an application to
be in conformance with a look & feel standard. The obvious
way is to use the look & feel's API which naturally guarantees
perfect conformity for primitive objects. The other way is to
use your own routines to emulate the look and the feel of a standard.
This is a long and difficult process, but has one big payoff.
There can be a significant gain in speed if the emulation routines
are efficient and go down to the most elementary display routine
level, therefore bypassing potentially many layers of library
calls. For example, a Motif application using the Motif API is
built upon many layers: Xlib, Xt, and Motif. Adding to that the
UIB's API is yet another layer. But with emulation, the Xt and
Motif layer are eliminated which can produce a significant speed
gain.
Chameleon Ability
All PIUIBs have the ability to take on multiple look & feel's.
Open Interface has the unique ability to change interactively
to any look & feel it supports within the design
tool (except for the Macintosh look & feel on non-Macintosh
platforms due to legal concerns)! Open Interface emulates the
look & feel standards rather than using their APIs so that
it can do this.
Speed and Size of Final Application Program
No attempts were made to measure speed of execution of the prototype.
Running applications were only completed on Unix which is less
affected by application size than other platforms.
Interoperability: conformity to industry standard on each platform
All the PIUIBs' interactive design tools used platform standards
to perform cut and paste routines. This was the only interoperability
test conducted.
Subjective Opinion
It is this author's opinion that Open Interface is the best all-around
PIUIB for creating typical office information and communication
GUI programs that are robust, versatile, highly interactive, and
maintainable. XVT is the second best tool limited by its conservative
offerings. Both GUILD and OpenUI need to have their documentation
overhauled and their message passing mechanisms made coherent.
Until that time, they would be difficult to use to build medium-
to high-complexity GUI programs.
4 Open Interface Evaluation
4.1 Executive Summary
Open Interface version 2.0 by Neuron Data is the frontrunner,
enjoying a large lead over the other PIUIBs. Open Interface is
strong in terms of software infrastructure, methodology, documentation,
polished presentation, and completeness. Its strengths include
advanced object-oriented design; the largest and most useful widget
set; powerful and malleable interactive layout features; printing,
string, and memory management support; and extensibility. Its
weaknesses include menu display glitches and other small deviations
from look & feel standards (due to its decision to emulate,
not to use, the APIs of the standard window systems), and the
company's consistently poor attitude towards customers.
4.2 Installation and Learning
The installation from tape was smooth and the instructions long-winded
but clear. Open Interface has an extra step that requires one
to run an install script that completes the environment and tool
setup. They provided manual checks to make sure that the installation
was completed correctly.
The tutorial ("primer") sections they provided were
clear, organized, and followed well the principle of progressive
disclosure. The documentation used a very convenient and informative
presentation style for advancing the example tutorial code. For
new code that was inserted, a bold typeface was used. Deletions
from the previous code were printed with a strikethru typeface.
When lines were moved, the old location used a strikethru, underline,
and bold typeface; the new location used both an underline and
bold typeface. With these presentation styles, the changes in
the code necessary to create the new tutorial functionality were
easily identified and the learning process was accelerated.
4.3 Documentation, Online Help, and Tech Support
When the product arrives, one gets a sense of completeness as
the two volume, 2,000 page documentation set makes a resounding
thud on the desk. Then one gets a sense of professionalism as
the documentation organization and presentation are absorbed.
The unique presentation style of the tutorial text, as mentioned
earlier, was outstanding.
There is no online help facility in Open Interface. As for technical
support, they receive a low rating for taking too long to get
back to me after the initial call and were somewhat cold and uncaring
in tone. The sales staff treated me in a similar manner.
4.4 Interactive Design Tool
OpenEdit is a powerful interactive design editor with which to
build graphical user interfaces. It provides the largest widget
set of the PIUIBs reviewed including a hierarchy browser and overview
widgets. OpenEdit itself is written using Open Interface, a clever
advertisement for the power of their tool. (Note: all PIUIBs reviewed
wrote their interactive editor using their own tool!) There is
an abundance of options and controls to choose among to further
specify the behavior of various widgets. Indeed, sometimes almost
an entire OpenEdit window changes form to accommodate specifying
certain options and characteristics, and one can overlook the
radio buttons that trigger this. Though after using OpenEdit for
a while, this inconspicuousness is not a problem.
4.4.1 Windows
The module is at the highest level of Open Interface specification
and corresponds directly to a window in an application. For every
window a new module needs to be created, which is a slight design
drawback since the module should be able to encapsulate any and
all functionality for an abstract part of an application that
may happen to need two or more windows. Below a module come the
various encapsulation objects for menus, dialogs, buttons, strings,
etc. There are check boxes in OpenEdit for window attributes such
as zoom boxes, resize capabilities, etc. Automatic MDI functionality
is possible with Open Interface.
OpenEdit takes navigation among objects in a window one step further
than most PIUIBs. It supports the concept of a visit order that
is independent of both geometry and creation order of objects,
two common ways other tools set the visit order. OpenEdit provides
a scrolling list of object names that is manipulated to specify
the visit order. The last item wraps around to visit the first
item, and navigation can go in reverse order with the back-tab
(Shift Tab) key.
4.4.2 Menus
The way menus are defined is at once powerful and cumbersome.
The power includes ways to specify hierarchical menus; checkable
menu items, both independent and one-of-a-group items; menu mnemonics;
menu separators; as well as allowing icons to appear in menu labels
between the menu text and keyboard equivalent label. This is powerful
functionality and should not be overshadowed by the longer explanation
of the menu problems immediately following.
The cumbersome aspect is with keyboard equivalents. Firstly, for
a menu item to have a keyboard equivalent involves defining two
objects. Each keyboard specification (such as ^O or F8) must be
defined as an object. Menu items take lists of keyboard objects
so each keyboard object must be put into a list object that can
be associated with a menu item. Though this is the most general
approach, allowing any of N keyboard equivalents to invoke one
menu item, most applications need only one keyboard equivalent
so that the extra list object is a frustration. Secondly, the
string that is displayed as the keyboard equivalent label must
be supplied rather than automatically generated by OpenEdit. More
than a nuisance, it is a look & feel portability problem because
each displays the same keyboard equivalent differently. This can
be worked around using the Open Interface string resource, but
modifications would involve manipulations in two logical places
rather than one. Thirdly, the checkable menu items and radio menu
items only had their look specifiable within OpenEdit, the feel
must be programmed. In other words, when any type of checkable
menu item is chosen, several lines of code must be written to
invoke the proper Open Interface functions that turn off an on
check marks.
Finally, there were inexcusable display glitches of all menu bar
items and menu items in the Motif look & feel. Whereas this
is not a problem with the methodology like the first three issues,
it is ugly and brings in to doubt Open Interface's ability to
emulate every aspect of any look & feel.
4.4.3 Dialogs
Open Interface treats dialog boxes as just another widget. They
offer good access to various attributes that control the behavior
of a dialog box such as whether it is modal, its size, its location,
of course its contents, etc.
4.4.4 Icons
OpenEdit has its own icon editor that allows an icon to be designed
and built once and automatically ported to all platforms. The
icon editor is fully functional and it took only ten minutes to
create a mail in-box icon. Icons can also be embedded in menu
item and button labels smoothly.
4.4.5 Other Notable Widgets
Open Interface provides a multiple row and column scrolling list
object with variable size rows and columns. This object supplies
all scrolling list functionality and handles textual or image
data. The application prototype required the use of this type
of object in single column mode only, thus its full functionality
was only read.
Two more powerful widgets are a hierarchy browser and hierarchy
overview widgets. Naturally they can be used together to provide
a form of fisheye view that is very helpful for visualizing large
hierarchies. These widgets are used quite appropriately in OpenEdit
itself to view the application hierarchy being created. This is
not only useful but almost necessary since Open Interface uses
hierarchic object-oriented design to a large extent.
4.4.6 Notes on Building the Application
Everything encountered in the prototype specifications was straightforwardly
handled in Open Interface and OpenEdit. Everything but menu item
keyboard equivalents was handled quickly and practically effortlessly.
Icons in menu items were a breeze. Arranging, grouping, and aligning
widgets was simple. Password masking is a simple check box. "Combo
boxes" are a supplied widget. A makefile can be generated
for any platform, and the one I generated for HP/Unix compiled
the first time without modification. It was a pleasure to use
OpenEdit and Open Interface to build the prototype.
4.4.7 Miscellaneous
Another excellent aspect of OpenEdit is the ability to specify
resize behavior for any object. This is done with the concept
of lock bars. There are four lock bars external to an object--one
on each side of the rectangular bounding box-- and two lock bars
internal to an object--one vertical and one horizontal. The presence
of a lock bar keeps display location (external) or display size
(internal) constant. For instance, if the vertical internal lock
bar is present and the horizontal internal lock bar is not present,
a window resize will change the size of the object in the horizontal
direction and not in the vertical direction. I could not specify
that only one of the four external lock bars be present and I
was not sure if this was a bug or a feature of OpenEdit. Objects
could not be locked to one another, only to window or panel borders,
but this combined with object grouping would seem to cover a majority
of application needs.
OpenEdit was robust and never crashed or lost changes. This cannot
be said of any other tool reviewed. The full evaluation
data collected are appendix E-1.
4.5 Prototype Evolution
Code generation and regeneration capabilities are strong. OpenEdit
generates stylistically clean code with good comments, and supplies
appropriate switch statement cases for events of interest to an
object. The regeneration of code is very carefully done so as
not to wipe out additions to previously generated code that performs
application-specific functionality. Complete makefiles are also
generated, adding new files and dependencies when appropriate.
Evolutionary development of an application is supported superlatively
by Open Interface.
4.6 Look & Feel Conformity
This is one area in which Open Interface needs some work. They
do a rather good job of emulating every popular look & feel
standard to the point where one can see any look & feel on
any platform! That's right, see Windows while the host computer
is in OpenLook and vice versa. That is the good news.
However, there are two quite noticeable deviations from look &
feel standards that were discovered. One was menu display glitches
of the Motif style. Menu items both in the menu bar and pull-down
menus themselves did not unhighlight properly, leaving ugly remnants
of the 3-D effect in the menu items and menu bars. One would assume
this will be fixed in future releases of Open Interface. The other
deviation is the feel of the OpenLook submenus. Open Interface
interprets the OpenLook guidelines differently than the popular
OpenWindows implementation of OpenLook does. Open Interface forces
the mouse to touch the triangle that represents the submenu rather
than having a sweep right gesture of N pixels activate the submenu.
While this may not violate the explicit OpenLook guidelines, it
remains a curiosity as to why they would choose to do it differently
than OpenWindows.
4.7 Extensibility
Extensibility is a key feature of Open Interface. They allow for
subclassing of existing widgets with the API, and provide mechanisms
for importing them to be first class objects in OpenEdit, as well.
4.8 Non-GUI Features
Open Interface provides strong abstractions for string, printing,
and memory management support. The only file support is prepared
file/directory choosing dialog boxes.
4.9 Porting Issues
Though we do not have a compiler that Open Interface supports
on the PC, I was able to install Open Interface without a hitch
and load my application specifications into OpenEdit and see that
everything was there and looked as it should for the Windows 3.0
look & feel. When I interactively switched to the Motif and
OpenLook standards, the color and shading palettes were a bit
wacky and the screen was quite hideous and barely legible with
red and greened checkered patterns, etc. There is probably a straightforward
fix for that situation, but my evaluation time was used up by
that point.
5 XVT Evaluation
5.1 Executive Summary
XVT version 3.0 by XVT Software, the grandfather of tools and
companies in this field, falls in line behind Open Interface.
It is a solid tool with good interactive layout features; file,
printing, string, and memory management support; and an above
average widget set. Some standards bodies consider some aspects
of XVT in some decisions. It doesn't offer much in terms of methodology,
iterative development, or extensibility.
XVT is the first commercially successful PIUIB, thus it seems
to be the benchmark by which all other PIUIBs are judged. For
low-complexity GUIs, this is okay and XVT should be commended;
but almost no one cares about low-complexity GUIs anymore, sophisticated
GUIs are becoming the norm for market leading companies. For the
medium- and high-complexity GUIs, it is time to find a better
benchmark. XVT's conservative offerings and lack of methodology
and infrastructure are goals beyond which PIUIBs should aim.
XVT uses the greatest common denominator approach to platform
independence. Only when all platforms have a similar functionality
does XVT provide it. (Note: most others in the community name
this the "lowest common denominator" approach because
they just can't use the word "greatest" when describing
a negative feature.) XVT leaves the methodology rather open for
the programmer to do much using normal C code practices. There
is no object-oriented design and only primitive structures for
the programmer to handle user interaction.
5.2 Installation and Learning
The installation of XVT went very smoothly. The preparatory reading
lasted only 45 minutes and there was no post tape-loading steps
to take. The tutorial for the XVT system lasted 60 minutes and
the tutorial for XVT-Design (the interactive editor) lasted only
90 minutes. In other words, the methodology is simple, the interactive
editor is simple. An additional 50 minutes was spent becoming
comfortable with XVT-Design and another 90 minutes was spent getting
comfortable with XVT before embarking on prototype development.
5.3 Documentation, Online Help, and Tech Support
XVT's documentation was sufficient but not great. It lacked coverage
for explanation of items in the proper places and didn't much
mention associated objects or concepts when appropriate. The online
help facility was utilized throughout XVT-Design, but was not
very context sensitive. It provided what I was looking for about
half the time. XVT's technical support was the best of any PIUIB
evaluated. It was highly organized, the problem is assigned to
an appropriate person and they respond immediately to tell you
they're on the problem and to find out additional information
if needed. I was contacted initially within two hours. My problem
was discovered and fixed by the next day. They offered a patch
to be sent out to me but I decided to work around my original
problem.
5.4 Interactive Design Tool
XVT-Design version 1.1 is the interactive editor for XVT. It is
the least powerful and most weakly coupled to the total PIUIB
environment of the interactive editors evaluated. It has good
WYSIWYG layout features and provides nice alignment capabilities.
It does not provide interface testing capabilities and only a
minimum of C code generation and regeneration. It also lacks access
to event specifications and color control which must be done using
XVT's Universal Resource Language (URL). Windows, dialogs, and
menus are given priority treatment by XVT, one stubs file for
each object type instantiated is generated and modified by the
programmer.
5.4.1 Windows
Windows are one of the three major objects in XVT. The minimal
attributes that are shared by all look & feel standards are
provided.
All widgets are implemented as windows, which may slow down a
complex application. There is no encapsulation from this fact
for the programmer, window controlling routines are used to manipulate
many aspects of widgets, I feel abstractions should be provided
here.
5.4.2 Menus
Menus are the second of the three main XVT objects. Menu creation
seems to be XVT-Design's best feature; ironically, the evaluated
version crashed when twenty menu items were exceeded. The menu
power was all there, including automatically generated labels
for keyboard equivalents. The process of creating menus was clear
and concise. Four default menus were provided with a menu bar:
File, Edit, Font, and Help; these can be deleted. There was no
ability to include a graphic in a menu item.
5.4.3 Dialogs
Dialogs are the third of the three main objects provided by XVT.
Dialogs have minimal controls for themselves, only location, size,
initial visibility, and whether they are modal. There is more
functionality provided for the objects that go into a dialog.
Alignment of objects is done rather well. The concept of a visit
order among dialog objects is also handled well, it is a separate
list with good editing features to control the list/visit order.
5.4.4 Icons
XVT defers the issue of icons to be created with platform specific
graphic editors. This is not a very forward-thinking approach
to building platform independent GUIs.
5.4.5 Other Notable Widgets
There really is not much extra to note about XVT because of their
decision to provide only greatest common denominator objects.
5.4.6 Notes on Building the Application
XVT handles the look of widgets fine, but sometimes does not handle
the feel. A case in point is radio buttons. They can be laid out
with XVT-Design straightforwardly, but they won't function as
radio buttons without several lines of source code created by
the programmer. It is beyond my understanding why XVT does not
provide for the entire radio button functionality without making
programming necessary.
Events are handled at the window level. During the definition
of an object, a unique identifier is issued automatically by XVT
and the application author assigns a #define string for it. This
is basically a low-level identification code that can be compared
to part of an event structure that is passed during runtime. The
other critical part of the event structure is the event type (window
create, mouse click, menu, etc.). It is the responsibility of
the application code to determine what kind of event it was, whose
low-level identification code it is, and do the application-specific
operation. This method is not elegant or well-structured. One
potentially powerful and unique aspect of XVT's handling of events
is that certain events can be sent to the application before
getting sent to XVT's event dispatcher.
5.4.7 Miscellaneous
Mouse events are not handled perfectly. There are two situations
that need extra special care and bookkeeping by the programmer:
double clicks and mouse-down, mouse-up pairs when the mouse has
left a window. It is possible for both these to be missed, misinterpreted,
or for the code to become quite messy. It is not necessary to
go into the details here; suffice it to say these should have
been handled better by XVT.
5.5 Prototype Evolution
XVT-Design does not provide support for prototype evolution; however,
it will not overwrite files with the same name as those it wants
to (re)generate. Thus the burden is put onto the programmer to
move any files that are expected to be regenerated, and then merge
by hand the new code with the old code that has programmer modifications
in it.
5.6 Look & Feel Conformity
XVT provides very good conformity with the various look &
feel standards because it uses their APIs to create applications.
Because XVT provides no extraordinary widgets for any particular
platform, there is no situation that can arise that would put
a look & feel standard issue in conflict with what XVT wants
to do.
5.7 Extensibility
There is no extensibility capability with XVT's own mechanisms.
5.8 Non-GUI Features
The operating system features, file, printing, string, and memory
resources, are all very strong in XVT. This shows that XVT understands
there is more to a GUI than just what appears on the screen. These
powerful capabilities are what keeps XVT near the top end of PIUIBs.
5.9 Porting Issues
Because XVT has been around so long, it was the only PIUIB that
supported our out-dated PC compilers. It took me over 8 hours
to compile and link my application due to the fact that I needed
to specify that the link should be case insensitive which was
mentioned nowhere in XVT documentation. Once running, the application
fared well and conformed to the Windows look & feel.
6 GUILD Evaluation
6.1 Executive Summary
GUILD version 2.1 is produced and sold by Guild Products. It offers
an all-encompassing object-oriented approach, good interactive
layout and widget set features, strong database and variable substitution
facilities, and dynamic handling of multiple types of very useful
and powerful variables. However, the weaknesses of documentation;
coverage by the object and message passing systems; and lack of
file, printing, and memory management support relegate GUILD to
its lower standing.
6.2 Installation and Learning
Installation of GUILD was straightforward and simple requiring
45 minutes to read the instructions and about 10 minutes to load
from diskette. The first platform used for GUILD was PC/Windows;
I intended to port to the Macintosh platform but time was used
up before the port attempts.
Initial learning was quite difficult and I spent 7.5 hours getting
comfortable. GUILD didn't supply a tutorial proper, rather it
gave a brief description of most of the interactive editor functions
that could be followed online. This approach lacked goals and
at this point I still didn't know how to create a minimal application.
I went next to reading about their resource language and object-oriented
approach which took a long time. The approach is interesting and
viable, but the documentation did not cover enough issues of use,
caveats, and possible conflicts among messages. I had to go back
to the documentation after starting the prototype build more often
with GUILD than with any other PIUIB evaluated.
6.3 Documentation, Online Help, and Tech Support
As mentioned throughout this report, GUILD's documentation was
lacking in coverage and direction. They may have put more effort
into the online help facility for their interactive editor because
the online help was superlative. It had a highly context sensitive
providing the most specific information possible first. It also
had hypertext features and a good index which allowed me to investigate
associated topics as desired. However, the online help facility
was not enough to explain everything I needed (online help probably
shouldn't try to be everything), and thus didn't make up for the
lacking hardcopy documentation.
GUILD's tech support was very conscientious. However, I got the
feeling that I was being taken care of by one or two of the main
developers. This could be good since I'd be getting the most correct
information most quickly. This could be bad since my impression
from one of their demos is that there are only a handful of developers
in the first place and they are handling the tech support duties
which can obviously take time away from forward progress on GUILD
development. My impressions of the quickness and quality for tech
support were positive.
6.4 Interactive Design Tool
GUILD has a mostly well-designed interactive editor (three actually,
one each for windows, menus, and dialogs/controls). In addition
to items covered in the immediately following sections, it provides
a very strong database and variable substitution mechanisms that
provide a higher level of flexibility for interface specifications
than in the other PIUIBs evaluated. However, confusion was the
norm for specifying which object is sending which message to which
other object, I feel the interface could be done more clearly
for this capability. GUILD also provides an interactive debugger,
Snooper, that one can control to show various levels of detail
during program execution including messages, variable modifications,
etc. I found Snooper to be quite helpful in understanding what
was going on during execution. GUILD also provides good graphics
capabilities.
6.4.1 Windows
GUILD provided very good window attribute controls, though the
interactive editor did not provide access to all that the resource
language could do. Window objects are used to create both windows
and dialog boxes.
6.4.2 Menus
Menu creation was clear and quick. There was good access to mnemonics
and keyboard accelerators. It was also possible to put an icon
into a menu item by declaring a background bitmap for it. All
in all a powerful and simple menu generation mechanism.
6.4.3 Dialogs
Dialogs take a broader meaning within GUILD. In order to get any
control into the interface, a dialog object must be used; in essence
it is like a panel object. For a dialog box, one would create
a window for it with a dialog object and controls.
There was a good widget set for GUILD with the exception of providing
a multi-line text object which was still on the list of things
to do. The widget alignment features was very good.
6.4.4 Icons
GUILD provides a good graphic editor that can be used for creating
icons. GUILD uses a vector graphic image approach which allows
for animation, arbitrary zooming, and good platform independence.
Again, bitmaps can be used as the background for windows, dialogs,
or menu items.
6.4.5 Other Notable Widgets
The only other item worth mentioning is GUILD's excellent online
help creation capability. It provides features for hypertext links,
many styles, and keyword and indexing of which end-users can take
advantage. Integrating the help for any object is very simple
which allows for easy creation of context sensitive help.
6.4.6 Notes on Building the Application
Compiling the application was reasonable. However, it took me
a long time to discover a bug in my specifications in which I
had conflicting flags or message passing constructs. It took the
tech support personnel most of two days to figure out the problem.
This points to the possibility of a design problem in these constructs
if it is so easy to tell the interface to do conflicting things.
6.4.7 Miscellaneous
Message passing is viable, buts needs to be better documented
for typical uses, conflicts with other typical and non-typical
situations, and indexed.
The variable substitution mechanisms seem to provide a level of flexibility not possible with the other PIUIBs. Substituted strings are provided by all PIUIBs through one mechanism or another. GUILD provides six types of variables that allow for much more flexibility (straight from the documentation):
6.5 Prototype Evolution
No C code is generated, so prototype evolution is not made difficult.
The more that functionality is performed with message passing
rather than GUILD API calls, the easier will be prototype evolution.
6.6 Look & Feel Conformity
GUILD exhibited high conformance to the Windows look & feel
which was the only standard in which GUILD was used. This makes
sense since GUILD uses the look & feel APIs to create the
interfaces.
6.7 Extensibility
GUILD does not provide any subclassing mechanism, so that kind
of extensibility is not possible. GUILD does provide you with
the GUILD specification of their own interactive editor such that
this could be modified to provide additional or different functionality
so long as it used legal GUILD constructions. It may be that C
code would need to be added to certain functional additions or
modifications performed in this way. This level of extensibility
is cute, but not very powerful.
6.8 Non-GUI Features
GUILD provides no file, printing, or memory management support
which detracts considerably from its usefulness. Though it should
be noted that many file needs could be met by GUILD's strong database
capabilities and memory management is only needed for C code which
can be minimal in GUILD applications. As mentioned before, on
the plus side GUILD does have strong database and variable substitution
facilities.
6.9 Porting Issues
I did not attempt to port the prototype built with GUILD to another
platform due to time constraints. However, similar to the case
made for prototype evolution, it seems that in theory a port to
another platform would be easy proportional to the amount of C
code written.
7 OpenUI Evaluation
7.1 Executive Summary
OpenUI version 1.30 by Open Software Associates rounds out the
evaluations. It uses an object-oriented approach that is reasonably
powerful but can be extended in many circumstances. Its interactive
layout mechanism is so abstract it is a negative feature. The
lack of a strong widget set and file, printing, and memory management
support compounds the negatives.
OpenUI seems to be the least grounded in a GUI and software engineering
perspective. It is leaning towards database and OLTP domains.
7.2 Installation and Learning
It took 23 minutes to read and understand the installation instructions
and another 20 minutes to load from tape. A small battery of tests
and environment setup was completed in another 20 minutes. The
initial learning period consisted of 1 hour 40 minutes following
the tutorial and an additional 2 hours 20 minutes reading up on
the interactive editor, ouibldr, and object-oriented methodology
used by OpenUI.
7.3 Documentation, Online Help, and Tech Support
OpenUI's documentation seems usually to suggest or give the flavor
of a description rather than to give direction, answer, or explain.
Many times I would pick up the set of manuals and leaf through
them recognizing none had the answer for which I was looking.
OpenUI provided no online help for their tool, even though OpenUI
itself provides a capable help authoring system with hypertext
links, keywords, etc.
7.4 Interactive Design Tool
The interactive design tool, ouibldr, supplies reasonable power
and a below average widget set. However, it uses a logical grid
to lay down widgets in an attempt to be resolution independent.
Instead, it is far too cumbersome to work with comfortably and
creatively. Another strange choice of behavior in ouibldr is that
placed objects are always in test mode except for the double-click
event which will invoke the attribute editor. However, the actions
associated with the objects do not execute while in test mode.
A companion program, ouishow, is used for this purpose. This method
does not make much sense to me, either do it or don't do it, but
don't do it halfway. This methodology got me in to trouble when
I created a modal dialog box but couldn't get out of it because
the OK and Cancel buttons weren't specified with actions to close
the window! It took me over an hour to find in the documentation
that hitting the escape key will exit the window.
Another oddity about ouibldr is that it uses a textual form-based
style that doesn't seem to "fit" well in a GUI-based
application. All attribute changes are made by typing text or
in a few cases choosing text items from a set of choices. This
textual form-based approach takes longer than other methods, is
somewhat frustrating and tedious, and leaves one wondering why.
There really is no point in following the structure of the previous
chapters in explaining the various window, menu, dialog, etc.
features of ouibldr because its methodology is quite limited compared
to the other PIUIBs evaluated. Only if OpenUI was being compared
to lesser competitors or platform dependent builders would such
an exercise be worthwhile.
8 Acknowledgements
This project was jointly inspired by Ben Shneiderman of University
of Maryland, Ren Stimart, Manager of the User Interface Group
at GEIS, and myself. Ren also supplied much insight to criteria
for selection, process, and evaluation of the PIUIBs. Cathy O'Donnell,
Senior Specialist in the User Interface Group at GEIS, helped
with much of the less glamorous but very important, voluminous
details of this project, including reviews of this report. This
project would not have been possible without the continued support
and encouragement of Lew Priven, Vice President of Engineering
and Application Development at GEIS, and Don Deutsch, Manager
of Industry Applications and Services at GEIS.
I would also like to extend my gratitude to each of the five companies for supplying our lab with free, full-privilege copies of the tools evaluated (or extended evaluation periods).