Engineering Large-scale Hypermedia Systems: Lessons from 24 years of Hypertext R&D

Robert M. Akscyn


The aim of this tutorial is to impart to researchers and practitioners in the field of hypertext lessons learned from experience with developing and maintaining hypertext systems (and technology) over a several decade period.

Large scale systems often introduce design issues that do not arise at small scale. Usually there are 'breakpoints' -- thresholds of scale where design principles that previously sufficed are no longer functional. It can be difficult to surmount these breakpoints - not only because a 'paradigm shif't in architecture may be required - but the designers may have to forgo features they previously thought indispensable. As a result the design tradeoffs become more agonizing.

This tutorial describes the 'lessons learned' from research and development of software and hardware architectures, systems, and applications -- conducted at Knowledge Systems during the past 20 years.

The focus of the tutorial is on the problematics of building, maintaining, and operating large-scale hypertext systems to support a wide-variety of applications - and approaches to tackling these problematics. Example application areas include database development, knowledge management, on-line/off-line document development, digital libraries, and software engineering.

Tutorial outline

    Review of History of Field of Hypertext

Overview of course
    Objectives for the tutorial

A framework for evaluating hypertext system designs

Types of scalability:
    Transaction Rates
    Response Rate

Human factor issues in highly-scaled environments
    The "Need for Speed"!
       Rapid response (< 1/10 second)
       Homogeneous layouts
       Few contexts (i.e., less modes)
       Directness of user interface
    Multiple, interconnected screens
    Integrated editing (text/graphic/images)
    Integration of browsing/editing
    'Uphill Representation'
    Multi-level indexing
    Structure-based searching
    Version management

Example Highly-scaled Application areas:
    Knowledge Management
    Digital Libraries
    Software Engineering
    Massive Intelligent Agent Systems
    Computer-Support Collaborative Work
      Artifact-Based Collaboration

Future of Hypertext: some possibilities:
    'Hypertext Machines'
       Example: PetaPlex Architecture
       Project history
         National Security Agency
       Current state/systems
          Virginia Tech PetaPlex
       Experience/Lessons learned
       Future work
      "SuperPrograms" - immense-scale 'hypertext software'
          Digital Library 'Utilities'

Summary of "Lessons Learned"

Tutorial history

I have presented a number of tutorials on hypertext and digital libraries at the following conference series:
   ACM Hypertext XX (including European side) (1991,1992)
   ACM Multimedia series (1993/1995/1996)
   ACM Digital Libraries (1998)
   Hypermedia '94 (Johannesburg, South Africa)
as well as presented full-day, industry-oriented tutorial series on hypertext in Paris, London, and Milan (1990)

Presenter biography

I have been the President of Knowledge Systems for the past 20 years - since founding, along with my colleague Don McCracken, a 1981 spin-off from the Computer Science Department at Carnegie Mellon University. My focus during this period has been the development of software and hardware architectures and products - -to support the development and maintenance of large-scale hypertext systems.

In recent years my emphasis has been on the evolution of massively-parallel 'hypertext machines' -- designed to support very rapid response for large numbers of concurrent users. In the late 90's I develop the "PetaPlex" architecture for the National Security Agency for a system to store 20 petabytes of hypertext-structured material. My company is now commercializing that work.

A specific theme that permeates my work -- has been the issue of system response time - believing that the usability of hypertext systems is strongly related to the ability of users to navigate quickly (e.g., response times on the order of 1/10 second or less)

Another highlight of my work during this twenty year period -- has been the exploration of hypertext as a representation medium for software -- in which large-scale, hypertext structures are directly executable (as well as browsable). This has lead to the belief that there is significant potential for the development of 'superprograms' -- software systems with billions of executable components housed in massively-parallel clusters, as a means of truly augmenting human intelligence in the day-to-day performance of knowledge work (i.e., what Doug Engelbart has been encouraging us to do all along).

Relevant references

Fox, Ed., Akscyn, R., Leggett, J. and Furuta, R. (Editors), Digital Libraries (Special Issue of) Communications of the ACM, April, 1995.

Akscyn, R., "Constructing a World-Scale Digital Library" White paper for NIST ATP Focussed Program Development Workshop: "Networking, Telecommunications and Information Technology: The Requirements of U.S. Industry", August 30-31, 1994

Akscyn, R. , "Reengineering the Field: Hypertext in the 21st Century". Proceedings of Hypermedia '94, Johannesburg, South Africa. Mar 94

Akscyn, R. and McCracken, D. Design of Hypermedia Script Languages: The KMS Experience, Proceedings of Hypertext '93, ACM NY 1993

Akscyn, R., and D. McCracken, "PLEXUS: A Hypermedia Architecture for Large-scale Digital Libraries " Proceedings of SIGDOC '93, ACM SIGDOC, Oct 1993, Kitchener, Ontario

Akscyn, R. (Ed.) "ACM Hypertext Compendium", [A hypertext database containing over 130 articles from the field of hypertext], ACM Press 1991

Yoder, E., Akscyn, R., and D. McCracken. "Collaboration in KMS, A Shared Hypermedia Environment." In Proceedings of CHI '89, ACM SIGCHI, May 1989, Austin, TX.

Akscyn,R. and Yoder, E. Editors. "Hypertext on Hypertext," the KMS hypertext version of the Communications of the ACM July 1988 special issue on hypertext. Association for Computing Machinery, New York, NY.

Akscyn, R., McCracken, D. and E. Yoder, "KMS: A Distributed Hypermedia System for Managing Knowledge in Organizations," Communications of the ACM, July 1988.

Akscyn, R., Yoder, E. and D. McCracken, "The Data Model is the Heart of Interface Design." In Proceedings of CHI '88, ACM SIGCHI, May 1988, Washington, D.C.

Akscyn, R., McCracken, D. and E. Yoder, "KMS: A Distributed Hypermedia System for Sharing Knowledge in Organizations," Proceedings of Hypertext '87, University of North Carolina, Chapel Hill, NC, November 13-15, 1987.

Akscyn, R., "The Next Computer Revolution?", Pittsburgh High Technology, 2(6), November/December 1987.

Akscyn, R., "The Hypertext Approach to Managing Knowledge in Organizations," Proceedings of the Fifth Corporate Electronic Publishing Seminar (CEPS), Chicago, IL, March 1987.

Akscyn, R. and D. McCracken, "The ZOG Approach to Database Management," Proceedings of the Trends and Applications Conference: Making Database Work, Gaithersburg, Maryland, May 1984.

Akscyn, R. and D. McCracken, "ZOG and the USS CARL VINSON: Lessons in System Development," Proceedings of the First IFIP Conference on Human- Computer Interaction (Interact '84), London, U.K., September 1984.

Yoder, E., McCracken, D., and R. Akscyn, "Instrumenting a Human-Computer Interface for Development and Evaluation," Proceedings of the First IFIP Conference on Human-Computer Interaction (Interact '84), London, U.K., September 1984.

Akscyn, R., and D. McCracken, "The ZOG/VINSON Project: An Experiment in Direct Technology Transfer," Proceedings of IEEE COMPCON'84 Fall Conference on the Small Computer Revolution, September, 1984.

McCracken, D. and R. Akscyn, "Experience with the ZOG Human-Computer Interface System," International Journal of Man-Machine Studies, Vol. 21, 1984, pp. 293 -310. 21 pages.

Robertson, C.K. and R. Akscyn, "Experimental Evaluation of Tools for Teaching the ZOG Frame Editor," Proceedings of the International Conference on Man/ Machine Systems, Manchester, U.K., July 1982. 10 pages.

Newell, A., McCracken, D., Robertson, G., and R. Akscyn, "ZOG and the USS CARL VINSON," Computer Science Research Review, Carnegie-Mellon University, 1981, pp. 95-118.

Back to Main Tutorials Page