TiChi Introduction

Fitt's Law
Object-Action Interface
Prescriptive Theories
Fisheye strategy
Conceptual, semantic,
    syntactic, & lexical

Direct Manipulation

Information Processing
Hacker's Action Theory
Attention & Memory
Andersen's ACT-R
Knowledge & Mental
Social & Cultural

Theories in Computer human interaction

Printer Friendly Version

Knowledge and Mental Models in HCI
Amir Khella
September 2002


"In interacting with the environment, with others, and with the artifacts of technology, people form internal, mental models of themselves and of the things with which they are interacting. These models provide predictive and explanatory power for understanding the interaction."

-Norman (in Gentner & Stevens, 1983)


    In Kenneth Craik 's words, mental models are representations in the mind of real or imaginary situations. Conceptually, the mind constructs a small scale model of reality and uses it to reason, to underlie explanations and to anticipate events. These models can be constructed from perception, imagination, or interpretation of discourse. A mental model represents explicitly what is true, but not what is false. The greater number of mental models a task suggests, and the greater the complexity of every model, the poorer performance is. 

    These models are more than just pictures or images, sometimes the model itself cannot be visualized or the image of the model depends on underlying models. Models can also represent abstract notions like negation or ownership which are impossible to visualize.


    Some of the characteristics of mental models are:



    -Constantly evolving

    -Not accurate representation (contain errors and uncertainty measures)

    -Provide a simple representation of a complex phenomena

    -Can be represented by a set of if-then-else rules


    The theory of mental models demolished an assumption which was until then prevalent in psychological theories of reasoning: that humans employ a kind of mental logic, which is similar to the propositional logic employed by logicians, when making inferences about the world.



     Mental models scope is wide enough to range from naive models of natural phenomena such as electricity to instructional models of scientific concepts used in teaching. Since human thought is involved in every day's activities,mental models are applicable to almost every human interaction with nature, devices, and even interaction with other individuals.

    The theory of knowledge representation and mental models is applicable in designing every day's things. By knowing what users know about the system and how they can infer the system functionality from the provided interface, it will be possible to predict and improve the learning curve as well as users errors and the ease of use of that system and finally to design interfaces that support the acquisition of appropriate user model.




    As an example of mental models and their influence in every day's interaction, let's take a look at the sketch by M.C. Escher shown below.


Click on the thumbnail above for the full size version


    At a first glance,  the drawing looks perfectly normal ... As you focus on the structure inside the red circle, something starts to feel confusing. For a while, it's hard to figure out the illogic part of it since the walking men on the top direct the eye through the ascending and descending stairs which, as part seems natural but the mind starts to refuse some concept about what is perceived.  The reason behind this confusion is that the mental model of doing an action stairs on stairs leads to a different state (going upstairs brings you to a higher level and going downstairs brings you to a lower level) and since this concept is violated in the above model, the mind starts having the confusion and we find words popping out "something's wrong...", "this doesn't make sense",etc...


    Another example would be considering users making the transition to Windows 95. Mac users were used to the concept of a recycle bin sitting on their desktop. These users did not have problems with the newly introduced recycle bin in that version of windows since they had already built and experienced the right mental model for it. In contrast, users making the transition from older versions of windows did not have that model since it was not present in early versions of windows.



Knowledge representation:

        Knowledge is stored in in memory in a highly organized fashion. It usually takes the mind a fraction of a second to answer some question such as what is the capital of Paris or who is the president of the United States .Through this document, we will refer to the two expressions "knowledge representation" and "mental representations" as being synonyms.





figure 1 : Johnson-Laird proposition of the three types of mental representations


    Let's take an example to state the nature of arbitrariness of language like representation versus the isomorphism of picture representation and then see how mental model falls somewhere between those two representations. Children who have never seen an elephant before would not necessarily recognize an elephant in front of them when all the information stored in their brain about it is the description their mothers provided them with. The mapping in this case is totally arbitrary. Meanwhile, a child who has already seen a picture of an elephant would probably recognize it in real life. Picture like representation is isomorphic mapping from the mental model to the real life phenomenon. Furthermore this representation preserves features and relationships.

    Although mental imagery is a rich representation, strong claims exist that prepositional representation is the basis for mental models. Johnson Laird (1983) described them as mental echo of the real world information encoded in verbal format. This representation is mapped to mental models through procedural semantics. During this process, the mind acquires information about the new phenomena and searched previously stored models for matching semantics. If no model was found,  a new model will be constructed and stored with the relevant semantics .


(For more information about mental procedural mapping click here)



Mental models:


    The theory of mental models was formulated in the early 40's by Kenneth Craik. He sought to provide a general explanation of the human thought based on the assertion that humans represent  the world they interact with through mental models. Johnson Laird (1983) based his theory on Craik's assumption stating  that an individual holds a working model  of a certain phenomenon in order to understand it. Mental models are not necessarily a visual representation of the real life case,  neither they are more complex representations. Laird argues that the abundance of representation details does not imply its usefulness. The only constraint for a mental model is that it has a similar structure to the phenomenon it represents. An ideal mental model explains all the aspects of the phenomenon the individual is interacting with. A referentially isomorphic mapping exists between the mental model and the real one.

    The mental models theory demolished the old psychological theories assumptions that human beings use some kind of prepositional logic to make inferences about the world. Laird tries in his new theory to unravel the puzzling facts discovered by empirical observations that while individuals are capable of logical inference, yet the logical reasoning rules cannot apply on the decisions and the answers they deliver. People tend to solve problems by using prior information and knowledge about similar problems if a similarity exists between the structure of the two models.


 Structural and Functional mental models:

        In the early 1980's, two basic mental models were identified, which are Structural and functional models. Structural models define facts the user has about how a certain system works. Its basic advantage is that the knowledge of how a device or system works can predict the effect of any possible sequence of actions, meanwhile constructing such a model in mind involves a great deal of effort .On the other hand, functional models, also called task-action mapping models,  are procedural knowledge about how to use the system. The main advantage of functional models is that they can be constructed from existing knowledge about a similar domain or system. Structural models are context free while functional models are context sensitive.


Applicability in HCI :


    From an HCI perspective, users form mental models by interacting with a certain computer system. The content and structure of mental models are influenced by selecting which information about a certain system is presented to the user and how it is presented. The interpretation of these models specifies how users interact with that system. Some major questions in this domain arise such as : To what extent the form of representation used in the interface affects the way the user solves a certain problem? Furthermore, Is it possible to develop interfaces that facilitate problem solving and support creativity? Does a graphical programming environment support innovation because it provides information in a format that is closer to the user's mental representation of the problem?


Mental Models in HCI:



    Several theories exist relating different models of users, designers and systems. They proposed four basic models of models that affect the way users interact with a system which are User's model of the system which is the model constructed at the users' side through their interaction with the target system,

the system's model of the user which is the model constructed inside the system as it runs through different sources of information such as profiles, user settings, logs, and even errors. The third model is the conceptual model which is an accurate and consistent representation of the target system held by the designer or an expert user, and the last model is the designer's model of the user's model which is basically constructed before the system exists by looking at similar systems or prototype or by cognitive models or task analysis.



       Several factors influence the way these models are built and maintained. At the users' side : their physical and sensory abilities, their previous experience dealing with similar systems, their domain knowledge and finally ergonomics  and environments in which users live.

    At the designers' side, the need is to influence the user's model to perceive the conceptual model underlying the relevant aspects of the system. This can be accomplished using metaphor, graphics, icons,  language, documentations and tutorials. It is important that all these materials collaborate together to encourage the same model.



    As an example of how users build mental models, let's take an example of a windows typical user exposed to a Unix environment for the first time and as a task in hand, he has to type a document on Emacs as opposed to his favorite windows text editor. The user makes a typo and without hesitating presses his fingers on the Control and the Z buttons since these are the keys he always used as a keyboard shortcut for UNDO command. The user gets frustrated as the Emacs editor completely disappears from the screen and he got back to the Unix prompt with no single notification message. The fact that the user has been working on windows builds a mental model for the UNDO command in almost all windows programs and associates this model with the action of pressing CTRL-Z, not knowing that these actions will cause a completely different action in Unix environment (which is running Emacs as a background process and the only way to bring it back is to type "fg" at the Unix prompt.



Design considerations:

    As stated above, systems should be designed to help users form the correct productive mental models. Common design methods include the following factors:

  • Affordance:      Clues provided by certain objects properties on how this object will be used and manipulated.

  • Simplicity:        Frequently accessed function should be easily accessible. A simple interface should be simple and transparent enough for the user to concentrate on the actual task in hand .

  • Familiarity:       As mental models are built upon prior knowledge, it's important to use this fact in designing a system. Relying on the familiarity of a user with an old, frequently used system gains user trust and help accomplishing a large number of tasks. Metaphors in user interface design are an example of applying the familiarity factor within the system.

  • Availability:      Since recognition is always better than recall, an efficient interface should always provide cues and visual elements to relieve the user from the memory load necessary to recall the functionality of the system.

  • Flexibility:        The user should be able to use any object, in any sequence, at any time.

  • Feedback:        Complete and continuous feedback from the system through the course of action of the user. Fast feedback helps assessing the correctness of the sequence of actions.

Limitations and challenges:

    The biggest limitation on the mental models theory is to capture and validate these models. Studies have shown that these models are built "on-the-fly", moreover, other studies have shown that asking a subject about his mental model can modify the mental model itself (Rogers, et.al. 1992).


"Descriptions of the formation of mental models rely on a variety of abstract concepts and processes such as schemas. Consequently, they seem to be an abstraction composed of abstractions. They are highly subjective.  Most studies obtain descriptions of high-level performance that are difficult to link to the building blocks of mental structures and processes.  Most studies of mental models also elicit mental model information after formation. Few studies map specific elements of a stimulus/system (including cues) to mental model component formation. (Roger, et.al. 1992)

 There are marked differences in the validation criteria accepted by the cognitive science community and that for HCI practitioners. Cognitive science demands statistically significant study with strict controls applied from the scientific method. HCI studies of mental models, and usability in general, are usually less robust than cognitive science studies. They have a different purpose, commercial feasibility. Introspection is commonly used. (Rogers, et.al., 1992)"

    Quoted from Mental Models and Usability, Depaul University, Cognative Psychology 404




Web Accessibility