AAAI Spring Symposium on Metacognition in Computation

Paper List

Abstract. This paper takes a cursory examination of some of the research roots concerning the topic of metacognition in computation. Various disciplines have examined the many phenomena of metacognition and have produced numerous results, both positive and negative. I discuss some of these aspects of cognition about cognition and the results concerning them from the point of view of the psychologist and the computer scientist, and I attempt to place them in the context of computational theories. We examine metacognition with respect to both problem solving and to comprehension processes of cognition. The history is limited to the 20th century.

Abstract. By employing hybrid dynamical systems-oriented techniques for reasoning about dynamical systems, it is possible to formalize some typically informal meta-intelligence about realtime intelligence of embodied agents. Furthermore, this meta-reasoning could be straightforwardly implemented in an embodied agent, forming a basis for meta-intelligent planning or deeper logical reflection. This paper concretely illustrates the underlying concepts, discussing a specific dynamical system for navigation intelligence, a specific system for meta-level reasoning, and a hypothetical case of their integration in an embodied agent. The paper also suggests that the fundamental ideas generalize to other, similarly expressed intelligence models, and that some high-level meta-reasoning over dynamical intelligence could thus be straightforwardly reduced to meta-reasoning over logical representations.

Abstract. Nesting of computational constructs is prevalent in computers. If one had a rigorous and general formal model of cognition, a high-level programmable and computable schema, then it would be possible to provide a cognitive AI system with that schema, let the system apply the schema to its own cognition as a substitution instance, thus turning the system into a metacognitive system. Concerns would still include infinite nesting and `first person' grounding.

Abstract. Reasoning with limited computational resources (such as time or memory) is an important problem, in particular in cognitive embedded systems. Classical logic is usually considered inappropriate for this purpose as no guarantees regarding deadlines can be made. One of the more interesting approaches to address this problem is built around the concept of active logics. Although a step in the right direction, active logics still do not offer the ultimate solution.
Our work is based on the assumption that Labeled Deductive Systems offer appropriate metamathematical methodology to study the problem. As a first step, we have shown that the LDS-based approach is strictly more expressive than active logics. We have also implemented a prototype automatic theorem prover for LDS-based systems.

Abstract. An empirically based cognitive model of real-world decision making was implemented in Shruti, a system capable of rapid, parallel relational reasoning. The system effectively simulates a two-tiered strategy associated with proficient decisions makers: Recognitional or reflexive activation of expectations and associated responses, accompanied by an optional, recursive process of critiquing and correcting, regulated by the stakes of the problem, the time available, and the remaining uncertainty. The model and implementation are inconsistent with the conventional claim that decision makers fall back on formal analytical methods when pattern recognition fails. Instead, they learn simple metacognitive strategies to leverage reflexive knowledge in novel situations. In addition, the model suggests that the development of executive attention functions (metacognitive strategies) may be necessary for, and integral to, the development of working memory, or dynamic access to long term memory, and that strategies developed for uncertainty handling may accelerate the reflexive learning of remotely connected concepts.

Abstract. To construct a perpetual self-aware cognitive agent that can continuously operate with independence, an introspective machine must be produced. To assemble such a agent, it is necessary to perform a full integration of cognition (planning, understanding, and learning) and metacognition (control and monitoring of cognition) with intelligent behaviors. The failure to do this completely is why similar more limited efforts have not succeeded in the past. As a start toward this goal, we performed an integration of an introspective multistrategy learning system with a nonlinear state-space planning agent using the wumpus world as environment. In this integration I show how the resultant system we call INTRO can generate its own goals. I use this system to discuss issues of self-awareness by machine.

Abstract. Robust intelligent systems will require a capacity for metacognitive reasoning, where intelligent systems monitor and reflect on their own reasoning processes. A large-scale study of human strategic reasoning indicates that rich representational models of commonsense psychology are available to enable human metacognition. In this paper, we argue that large-scale formalizations of commonsense psychology enable metacognitive reasoning in intelligent systems. We describe our progress toward developing 30 integrated axiomatic theories of commonsense psychology, and discuss the central representational challenges that have arisen in this work to date.

Abstract. While research in metacognition has grown significantly in the past 10 years, there has been a relative lack of research devoted to the focused study of the interactions between metacognition and affective processes. Computational models represent a useful tool which can help remedy this situation by constructing causal models of demonstrated correlational relationships, and by generating empirical hypotheses which can be verified experimentally. In this paper we describe enhancements to an existing cognitive–affective architecture that will enable it to perform a subset of metacognitive functions. We focus on modeling the role of a specific metacognitive factor, the feeling of confidence (FOC), and the anxiety-linked metacognitive strategy of emotion-focused coping.

Abstract. In this paper, we present an approach for dropping and reconsidering intentions, wherein concurrent actions and results are allowed, in the framework of the time-sensitive and contradiction-tolerant active logic. In this approach, a metacognitive process strives to dynamically mark intentions as achievable, unachievable or achieved, drop futile or achieved intentions and create alternative intentions for currently unachievable intentions when possible. Since, this process runs concurrently (and shares resources) with the cognitive activities of the agent, the amount of resources available for the process depends on real-time conditions. Therefore, when and whether intentions are dropped or reconsidered depends on the conditions and resources available at run-time.

Abstract. Current knowledge acquisition tools are oblivious to the process or strategy that the user may be following in entering new knowledge and unaware of their progress during a session. Users have to make up for these shortcomings by keeping track of the status, progress, potential problems and possible courses of actions by themselves. We present a novel extension to existing systems that 1) keeps track of past problem solving episodes and relates them to user entered knowledge, 2) assesses the current status of the knowledge and the problem solving using such relations, and 3) provides assistance to the user based on the assessment. We applied the approach in developing an intelligent assistant for decision making tasks. The resulting interaction shows that the system helps the user understand the progress and guides the knowledge authoring process in terms of making the knowledge more useful, adapting the knowledge to dynamic changes over time, and making the overall problem solving more successful.

Abstract. Automated decision-making is a significant concern for the AI community and especially for multi-agent systems. Although it has long been known among scholars of rhetoric that human decision-making can be systematically influenced by skillful argumentation, there seems to be a lack of formalizations which handle the impact rhetoric has on the concealment of logical fallacies to the human mind. In this paper, we highlight the need of metacognition for the successful formal representation and interpretation of human argumentation and thus successful automated decision-making. The relevance of such investigations is illustrated with a real-world example taken from the discourse of neuroscience.

Abstract. How can self-awareness emerge in a distributed system with no central control? How can such awareness feed back in a decentralized way to control the system's behavior? Many people have written about how self-awareness might come about in the brain. In this paper, I examine mechanisms for self-awareness and control in two other decentralized biological systems: the immune system and ant colonies. I then attempt to isolate some principles common to both systems. Finally, I discuss ways in which these mechanisms can serve as inspiration for the design of artificial intelligence systems with sophisticated abilities for distributed self-awareness and self-control.

Abstract. We describe a computational framework designed to provide adaptive support for learning from problem solving activities that make worked-out examples available. This framework targets several meta-cognitive skills required to learn effectively in this type of instructional setting, including explanation-based-learning-of-correctness and min-analogy. The generated interventions are based on an assessment of a student's knowledge and meta-cognitive skills provided by the framework's student model, and thus are tailored to that student's needs.

Abstract. The analysis of natural-language text involves many different kinds of processes that might be described in multiple ways. One way to describe these processes is in terms of the semantics of their requirements and results. Such a description makes it possible to view these processes as analogous to inference rules in a theorem-proving system. This analogy is useful for metacognition because there is existing theory and infrastructure for manipulating inference rules. We describe a taxonomy of text extraction tasks that we have represented as inference rules. We also describe a working system that encodes the behavior of text analysis components as a graph of inferences. This representation is currently used to present browsable explanations of text extraction to a user; in future work, we expect to perform additional automated reasoning over this encoding of text analysis processes.

Abstract. In this paper we describe how we have integrated deliberation scheduling into the CIRCA Adaptive Mission Planner. We present results on the performance of several agents in an example scenario from the UAV domain. This paper shows how the qualitatively different behaviors of different deliberation managers affects mission performance.

Abstract. This paper motivates and defines a notion of explicit self-awareness, one that implies human-like scope of the self-model, and an explicit internal representation susceptible to general inference methods and permitting overt communication about the self. The features proposed for KR&R supporting explicit self-awareness include NL-like expressiveness, autoepistemic inference grounded in a computable notion of knowing/believing, certain metasyntactic devices, and an ability to abstract and summarize stories. A small preliminary example of self-awareness involving knowledge of knowledge categories is attached as an appendix.

Abstract. Metacognitive reasoning in computational systems will be enabled by the development of formal theories that have broad coverage over mental states and processes as well as inferential competency. In this paper we evaluate the inferential competency of an existing formal theory of commonsense human memory by attempting to use it to validate the appropriateness of a commonsense memory strategy. We formulate a particular memory strategy (to create an associated obstacle) as a theorem in first-order predicate calculus. We then attempt to validate this strategy by showing that it is entailed by the axioms of the theory we evaluated. These axioms were encoded into the syntax of an automated reasoning system, which was used to automatically generate inferences and search for formal proofs.

Abstract. Developing and testing intelligent agents is a complex task that is both time-consuming and costly. This is especially true for agents whose behavior is judged not only based on the final states they achieve, but also by the methods used to accomplish their task. In this paper, we examine methods for ensuring an agent upholds constraints particular to a given domain. We explore two significant projects dealing with this problem, and we determine that two properties are crucial to success in complex domains. First, we must seek efficient methods of representing domain constraints and testing potential actions for consistency. Second, behavior must be assessed at run-time, as opposed to only during a planning phase. Finally, we explore how abstract behavior representations might be used to satisfy our first desired property, and then explore how these representations could be used by an agent to assess its own behavior at runtime. We end the paper with a brief discussion of the current state of our project and our plans for future work.