TiChi Introduction

Fitt's Law
Object-Action Interface
Prescriptive Theories
Fisheye strategy
Conceptual, semantic,
    syntactic, & lexical

Direct Manipulation

Information Processing
Hacker's Action Theory
Attention & Memory
Andersen's ACT-R
Knowledge & Mental
Social & Cultural

Theories in Computer human interaction

Printer Friendly Version

Objects-Actions Interface model
Amir Khella
October 2002


"Innovative designers may recognize opportunities for novel widgets that provide a closer match between the screen representation and the user's workplace"

Ben Shneiderman, Designing the user interface


    There exists two basic interaction models for any given system :

  • Object-Action model : The user first selects an object and then selects the action to be performed on the selected object
  • Action-Object model : The user first selects an action to be performed and then selects the objects on which this action will be performed.

    As Graphical User Interfaces (GUI) are replacing the text mode command based languages, the concern is now shifted to the visual representation and display of the user's tasks objects and actions. In early days' command based language, users used to maintain a big load of device dependent syntactic details in their memory. Sometimes this load becomes a burden on performing a certain task since the users shift their mind mode from the task domain to the tool domain to remember the appropriate actions to be performed to get the required results. This problem arises sometimes in command based languages when users try to perform a task through a sequence of commands. If some of these commands are not frequently used, users will be forced to shift their mind mode from the task they are performing to the context of the operating system and try to remember the appropriate command and even sometimes consulting the manual pages.

    In the following sections, we discuss the different aspects of the Object-Action Interface (OAI) model and the different impacts of this model on the way users interact with a certain system.



    There has been always distinction between those two models in several domains starting long ago by compiler designers who always wanted to distinguish between syntax, which is the different tokens parsed from a source code file, and the semantics, which are the actual operations invoked by that text. Different areas use the syntactic-semantic model of human behavior, such as programming, database manipulation facilities and direct manipulation (Shneiderman 1980,1981)

    The OAI model is also in harmony with the software engineering model of Object Oriented programming model that has been popular in the past decades.


    To distinguish the difference between the two models, we will study the course of interaction of a user using a command based system (UNIX) versus using a direct manipulation GUI environment (WINDOWS). The task is to copy a file from a directory (Unix) or folder (Windows) to another location. In the command based system which is based on the Action-Object model, the user starts by specifying the action to be performed, which is "copy" on our case, next he specifies the objects on which this action will be performed, which are the file name and the destination folder. In contrast, a user on a GUI based environment like windows chooses the object by clicking on the file name,  and then performs the copy action by dragging the file from the current folder to the new folder.

       We notice that the Object-Action model  maps to real life environment. If you want to move a paper from a folder to another, you simply start by selecting the paper you will move not by initiating a move action and then selecting the paper.


Syntactic knowledge:

        By syntactic knowledge we mean the information necessary to be maintained and memorized by a user to be able to use a certain system efficiently. For example a user of a certain programming language can use it efficiently only when he knows a good deal of commands and syntax specific to this language. This kind of knowledge has the following drawbacks:

  • Places a considerable memory load on the user, which is hard to retain over time unless that knowledge is applied frequently. Recognition  is usually preferred over recall. Most of the time, it is better to provide visual clues in the interface that make the user recall the underlying functionality.
  • Hard to provide hierarchical or modular structure to cope with complexity of the system. For example, a user using a mail system can have several  modes of termination : ENTER to terminate a paragraph, CTRL-D to terminate a letter, CTRL-C to cancel a letter, Q to quit the system, and so on .  A novice user might be confused by those different modes of operations having a pretty similar syntactic structure.
  • Syntactic knowledge is system dependent. It's hard to apply previous system knowledge to the new one. This goes from using different keyboard layout to the different syntax necessary to perform each task (The reason behind that is that the user build a mental model of a system while interacting with it. Shifting to a new system, supposedly similar in functionality, the user tried to apply the previous model to that system and faces frustration when the results he gets are not the ones he is expecting.)

The OAI model:

    In the Action-Object model, the computer is seen as a tool to perform different action. Whereas in the Object-Action model, the user gains a great sense of control from the feeling of a direct involvement. The computer in this case is seen as a medium through which different tools are represented, which is isomorphic to interacting with objects in the real world.

    Designing an OAI model starts with examining and understanding the tasks to be performed by the system. The domain of tasks include the universe of objects within which the user works to accomplish a certain goal as well as the domain of all possible actions performed by the user. Once these tasks objects and actions are agreed upon, the designer starts by creating an isomorphic representation of the corresponding interface objects and actions.


Figure from Designing the user interface by Ben Shneiderman

    The figure above shows the designer mapping from the real world universe of objects and intentions to the interface world universe of metaphors and plans. The interface actions are usually performed by pointing device or keyboard and hence have to be visual to the user so that the later can decompose his plan into steps of actions such as pointing, clicking, dragging, etc...

    One major factor that governs the design of direct manipulation interfaces is that they have to provide a snapshot of the real world situation through the objects provided to the user. The biggest advantage of direct manipulation systems, besides being easy to learn and providing immediate feedback, is that they minimize the syntactic details and hence minimize the memory load of the user. Someone who knows the objects and tasks domains can learn easily and perform well on such an interface.

Task hierarchies of objects and actions:

        Tasks include hierarchies of objects and actions at different high and low levels. For a certain user, these hierarchies might not be perfect, but since they are comprehensible, they provide a great deal of usefulness.

  • For the user:

        Hierarchical decomposition of a complex task into several simpler tasks has been a successful way in problem solving specially when dealing with large complex problems. Most real world entities and objects have this property of being built of hierarchical simple structures. Similarly, intentions can be decomposed into a plan of small action steps. People learn tasks actions and objects through their course of life while interacting with these objects and learning about their properties, functionalities and limitations. It is easy to conclude that people learn these issues independently from the underlying implementation on a certain system. We note that a user has to be first proficient in the task domain before using an interface to accomplish those real-world tasks.

  • For the designer:

        The following steps are recommended (Shneiderman) in order to build correct tasks hierarchies by designers for a system:

           1 - Know about the users and their tasks (Interviewing users, reading workbooks and taking training sessions)

           2 - Generate hierarchies of tasks and objects to model the users' tasks

           3 - Design interface objects and actions that metaphorically map to the real world universe


Interface hierarchies of objects and actions:

        Similar to the task domain, the interface domain contains hierarchies of objects and tasks at different levels.

  • Interface Objects:

            Users interacting with a computer get to understand some high level concepts relevant to that system. As an example, they learn that computer stores information, that these information are stored in files contained within a hierarchy of directories, and that each file has its own attributes like name, size, date, etc ...

  • Interface Actions:

           These are also hierarchies of lower levels actions. A high level plan is to create a text file might involve mid-level actions such as creating a file, inserting text and saving that file. The mid-level action of saving a file the file can be decomposed into lower level actions such as storing the file with a backup copy and may be applying the access control rights. Further lower level actions might involve choosing the name of the file, the location folder to be saved in, dealing with errors such as space shortage, and so on.

  • For the user:

            There are several ways users learn interface objects and actions such as demonstrations, sessions, or trial and error sessions. When these objects and actions have logical structure that can be related to other familiar task objects and actions, this knowledge becomes stable in the user's memory.

  • For the designer:

            The OAI model helps a designer to understand the complex processes that a user has to perform in order to successfully use an interface to perform a certain task. Designers model the interface actions and objects based on familiar example and then fine tune these models to fit the task and the user.

Limitations and challenges:

    People are different, and individuals have their own way in performing a particular task. Of course there are lots of standards governing the way people accomplishing certain missions or activities but the whole picture for everyone is pretty much diverse. It is true that direct manipulation environments are proven to be efficient for most users but they have their limitations. It's hard to apply a series of actions, that you already performed on an object,  to another set of objects (what command line users call "batching") .. It is hard to integrate batching techniques in visual environments since those techniques rely mainly on the concept of applying a sequence of actions on some objects, some of which exist only as intermediate results from an action in the same sequence.

    It's also hard in the object-action model to perform pipelining . In a pipeline, outputs of previous action-object command can be fed to another action and so on . Constructing a pipeline in Unix is simple and proven very efficient ,again because of the nature of the model used. In an objects-actions model environment, it is hard to construct pipelines and the way people accomplish such tasks is creating intermediate objects and applying subsequent actions on these objects.

    One last challenge appears when users start to become proficient using a certain system, and their performance curve start to raise. At a certain time, in a visual environment, and for certain natures of tasks, objects-action model will perform poorly. One of the examples is programmers using a visual tool to design an object oriented program by visually drawing classes, objects and relationships. A skilled programmer is much faster typing than using a mouse. shifting hands between the keyboard and mouse (to draw an object and name it) introduces certain delay .