All Publications    |    Journal Articles    |    Book Chapters    |    Refereed Conference Papers

Journal Articles


Fu, W.-T., D'Andrea, L., & Bertel, S. (2013). Effects of collaborative technologies on communication and performance in a remote spatial orientation task. Spatial Cognition and Computation. [PDF] [Abstract]

An experiment was conducted to examine the impact of communication methods (text-only, audio-only, and audio-plus-video) on communication patterns and effectiveness in a 2-person remote spatial orientation task. The task required a pair of participants to figure out the cardinal direction of a target object by communicating spatial information and perspectives. Results showed that overall effectiveness in the audio-only condition was better than the text-only and audio-plus-video conditions, and communication patterns were more predictive of errors than individual differences in spatial abilities. Discourse analysis showed that participants in the audio-plusvideo condition performed less mental transformation of spatial information when communicating, which led to more interpretation errors by the listener. Participants in the text-only conditions performed less confirmation and made more errors by misreading their own display. Results suggested that speakers in the audio-plusvideo condition minimized effort by communicating spatial information based on their own perspective but speakers in the audio-only and text-only conditions would more likely communicate transformed spatial information. Analysis of gestures in the audioplus- video condition confirmed that iconic gestures tended to co-occur with spatial transformation. Iconic gesture rates were negatively correlated with transformation errors, indicating that iconic gestures more likely co-occurred with successful communication of spatial transformation. Results show that when visual interactive feedback is available, speakers tend to adopt egocentric spatial perspectives to minimize effort in mental transformation and rely on the feedback to ensure that the hearer correctly interprets the information. When visual interactive feedback is not available, speakers will put more effort in transforming spatial information to help the hearer to understand the information. The current result demonstrated that allowing two persons to see and communicate with each other during a remote spatial reasoning task can lead to more errors because of the use of a suboptimal communication strategy.

Fu, W.-T., Lee, H., Boot, W., & Kramer, A. (2013). Bridging across cognitive training and brain plasticity: a neurally inspired computational model of interactive skill learning. WIREs Cognitive Science, 4, 225-236. [PDF] [Abstract]

This article reviews recent empirical and brain imaging data on effects of cognitive training methods on complex interactive skill learning, and presents a neurally inspired computational model that characterizes the effects of these training methods. In particular, the article focuses on research that shows that variable priority training (VPT), which requires learners to shift their priorities to different task components during training, often leads to better acquisition and retention of skills than fixed priority training (FPT). However, there is only weak evidence that shows that VPT can enhance transfer of complex interactive skills to untrained situations. Brain imaging studies show that VPT leads to significantly lower activations and a higher reduction of activities in attentional control areas after training than FPT. Research also shows that the volume of the striatum predicts the learning effects, but only in VPT. The computational model, developed based on learning mechanisms at the neural level, bridges across the empirical and the braining imaging results by explaining the effects of VPT and FPT at both the behavioral and neural levels. The results were discussed in the context of previous findings on cognitive training


Fu, W.-T. & Dong, W. (2012). From Collaborative Indexing to Knowledge Exploration: A Social Learning Model. IEEE Intelligent Systems. [PDF] [Abstract]

The World Wide Web has evolved from a read-only information resource to a participatory environment that lets people share, explore, and learn through multiple forms of user-generated content (such as blogs and photos). A social information system, for example, is a form of social learning in which multiple users engage in knowledge exploration. Learning in such situations involves finding and evaluating relevant documents related to the topic, comprehending and extracting information from the documents, and integrating the information with existing knowledge. This form of knowledge exploration becomes social when users share found documents through systems such as Delicious (formerly, a site that lets users collaboratively index documents with short "tags" and share them with other users. Such collaborative indexing using social tags can not only provide structures to information on the Web but can also act as navigational cues for other users to find relevant information. We focus on these kinds of collaborative or social tagging systems. Researchers have argued that social tagging systems can effectively improve knowledge exploration and sense-making activities,1,2 and studies have analyzed the knowledge structures in these systems, making them an ideal testbed for social learning. We describe a social learning model that characterizes the iterative process of knowledge exploration and learning activities. We also present results from an empirical study that directly tested the social learning model.


Kannampallil, T. G., Waicekauskas, K., Morrow, D. G., Kopren, K. M., & Fu, W.-T. (2011). External Tools for Collaborative Medication Scheduling. Cognition, Technology, & Work.

Fu, W.-T. (2011). A Dynamic Context Model of Interactive Behavior. Cognitive Science. [PDF] [Abstract]

A dynamic context model of interactive behavior was developed to explain results from two experiments that tested the effects of interaction costs on encoding strategies, cognitive representations, and response selection processes in a decision-making and a judgment task. The model assumes that the dynamic context defined by the mixes of internal and external representations and processes are sensitive to the interaction cost imposed by the task environment. The model predicts that changes in the dynamic context may lead to systematic biases in cognitive representations and processes that eventually influence decision-making and judgment outcomes. Consistent with the predictions by the model, results from the experiments showed that as interaction costs increased, encoding strategies and cognitive representations shifted from perception-based to memory-based. Memory-based comparisons of the stimuli enhanced the similarity and dominance effects, and led to stronger systematic biases in response outcomes in a choice task. However, in a judgment task, memory-based representations enhanced only the dominance effects. Results suggested that systematic response biases in the dominance context were caused by biases in the cognitive representations of the stimuli, but response biases in the similarity context were caused by biases in the comparison process induced by the choice task. Results suggest that changes in interaction costs not only change whether information was assessed from the external world or from memory but also introduce systematic biases in the cognitive representation of the information, which act as biased inputs to the subsequent decision-making and judgment processes. Results are consistent with the idea of interactive cognition, which proposes that representations and processes are contingent on the dynamic context defined by the information flow between the external task environment and internal cognition.


Fu, W.-T., & Dong, W. (in press). From Collaborative Indexing to Knowledge Exploration: A Social Learning Model. IEEE Intelligent Systems.

Fu, W.-T., Kannampalli, T. G., Kang, R., & He, J. (2010). Semantic imitation in social tagging. ACM Transactions on Computer-Human Interaction. [PDF] [Abstract]

We present a semantic imitation model of social tagging and exploratory search based on theories of cognitive sciences. The model assumes that social tags evoke a spontaneous tag-based topic inference process that primes the semantic interpretation of resource contents during exploratory search, and the semantic priming of existing tags in turn influences future tag choices. The model predicts that (1) users who can see tags created by others tend to create tags that are semantically similar to these existing tags, demonstrating the social influence of tag choices; and (2) users who have similar information goals tend to create tags that are semantically similar, but this effect is mediated by the semantic representation and interpretation of social tags. Results from the experiment comparing tagging behavior between a social group (where participants can see tags created by others) and a nominal group (where participants cannot see tags created by others) confirmed these predictions. The current results highlight the critical role of human semantic representations and interpretation processes in the analysis of large-scale social information systems. The model implies that analysis at both the individual and social levels are important for understanding the active, dynamic processes between human knowledge structures and external folksonomies. Implications on how social tagging systems can facilitate exploratory search, interactive information retrievals, knowledge exchange, and other higher-level cognitive and learning activities are discussed.


Fu, W.-T., Anderson, J. (2008). Dual Learning Processes in Interactive Skill Acquisition. Journal of Experimental Psychology, Applied, 14 (2), 179-191.[PDF] [Abstract]

Acquisition of interactive skills involves the use of internal and external cues. Experiment 1 showed that when actions were interdependent, learning was effective with and without external cues in the single-task condition but was effective only with the presence of external cues in the dual-task condition. In the dual-task condition, actions closer to the feedback were learned faster than actions farther away but this difference was reversed in the single-task condition. Experiment 2 tested how knowledge acquired in single and dual-task conditions would transfer to a new reward structure. Results confirmed the two forms of learning mediated by the secondary task: A declarative memory encoding process that simultaneously assigned credits to actions and a reinforcement-learning process that slowly propagated credits backward from the feedback. The results showed that both forms of learning were engaged during training, but only at the response selection stage, one form of knowledge may dominate over the other depending on the availability of attentional resources.

Fu, W.-T., Anderson, J. (2008), Solving the Credit Assignment Problem: Explicit and Implicit Learning of Action Sequences with Probabilistic Outcomes. Psychological Research, 72 (3), 321-330. [PDF] [Abstract]

In most problem-solving activities, feedback is received at the end of an action sequence. This creates a credit-assignment problem where the learner must associate the feedback with earlier actions, and the interdependencies of actions require the learner to remember past choices of actions. In two studies, we investigated the nature of explicit and implicit learning processes in the credit-assignment problem using a probabilistic sequential choice task with and without a secondary memory task. We found that when explicit learning was dominant, learning was faster to select the better option in their Wrst choices than in the last choices. When implicit reinforcement learning was dominant, learning was faster to select the better option in their last choices than in their Wrst choices. Consistent with the probability learning and sequence-learning literature, the results show that credit assignment involves two processes: an explicit memory encoding process that requires memory rehearsals and an implicit reinforcement-learning process that propagates credits backwards to previous choices.

Fu, W.-T. (2008). Is a single-bladed knife enough to dissect cognition? Cognitive Science, 32 (1), 155-161. [PDF] [Abstract]

Griffiths, Christian, and Kalish (this issue) present an iterative-learning paradigm applying aBayesian model to understand inductive biases in categorization. The authors argue that the paradigm is useful as an exploratory tool to understand inductive biases in situations where little is known about the task. It is argued that a theory developed only at the computational level is much like a single-bladed knife that is only useful in highly idealized situations. To be useful as a general tool that cuts through the complex fabric of cognition, we need at least two-bladed scissors that combine both computational and psychological constraints to characterize human behavior. To temper its sometimes expansive claims, it is time to show what a Bayesian model cannot explain. Insight as to how human reality may differ from the Bayesian predictions may shed more light on human cognition than the simpler focus on what the Bayesian approach can explain. There remains much to be done in terms of integrating Bayesian approaches and other approaches in modeling human cognition.


Fu, W.-T., Pirolli, P. (2007), SNIF-ACT: A Model of Information-Seeking Behavior in the World Wide Web. Human-Computer Interaction, 22, 355-412. [PDF] [Abstract]

How do humans or animals adapt to a new environment? After years of research, it is embarrassing how little we understand the underlying processes of adaptation: just look at how difficult it is to build a robot that learns to navigate in a new environment or to teach someone to master a second language. It is amazing how seagulls and vultures have learned to be landfill scavengers in the last century and be able to sort through human garbage to dig out edible morsels. At the time this chapter is written, an alligator is found in the city park of Los Angeles, outwitting licensed hunters who tried to trap the alligator for over 2 months. The ability to adapt to new environments goes beyond hardwired processes and relies on the ability to acquire new knowledge of the environment. An important step in the adaptation process is to sample the effects of possible actions and world states so that the right set of actions can be chosen to attain important goals in the new environment.


Fu, W.-T., Gray, W. D. (2006), Suboptimal tradeoffs in information-seeking. Cognitive Psychology, 52, 195-242. [PDF] [Abstract]

Explicit information-seeking actions are needed to evaluate alternative actions in problem-solving tasks. Information-seeking costs are often traded off against the utility of information. We present three experiments that show how subjects adapt to the cost and information structures of environments in a map-navigation task. We found that subjects often stabilize at suboptimal levels of performance. A Bayesian satisficing model (BSM) is proposed and implemented in the ACT-R architecture to predict information-seeking behavior. The BSM uses a local decision rule and a global Bayesian learning mechanism to decide when to stop seeking information. The model matched the human data well, suggesting that adaptation to cost and information structures can be achieved by a simple local decision rule. The local decision rule, however, often limits exploration of the environment and leads to suboptimal performance. We propose that suboptimal performance is an emergent property of the dynamic interactions between cognition and the environment.

Fu, W.-T., Anderson, J. R. (2006), From recurrent choice to skilled learning: A reinforcement learning model. Journal of Experimental Psychology: General, 135(2), 184-206. [PDF] [Abstract]

The authors propose a reinforcement-learning mechanism as a model for recurrent choice and extend it to account for skill learning. The model was inspired by recent research in neurophysiological studies of the basal ganglia and provides an integrated explanation of recurrent choice behavior and skill learning. The behavior includes effects of differential probabilities, magnitudes, variabilities, and delay of reinforcement. The model can also produce the violation of independence, preference reversals, and the goal gradient of reinforcement in maze learning. An experiment was conducted to study learning of action sequences in a multistep task. The fit of the model to the data demonstrated its ability to account for complex skill learning. The advantages of incorporating the mechanism into a larger cognitive architecture are discussed.

Fu, W.-T., Bothell, D., Douglass, S., Haimson, C., Sohn, M.-H., Anderson, J. A. (2006), Toward a real-time model-based training system. Interacting with Computers, 18(6), 1216-1230. [PDF] [Abstract]

This article describes the development of a real-time model-based training system that provides adaptive ''over-the-shoulder'' (OTS) instructions to trainees as they learn to perform an Anti-Air Warfare Coordinator (AAWC) task. The long-term goal is to develop a system that will provide real-time instructional materials based on learners’ actions, so that eventually the initial set of instructions on a task can be strengthened, complemented, or overridden at different stages of training. The training system is based on the ACT-R architecture, which serves as the theoretical background for the cognitive model that monitors the learning process of the trainee. An experiment was designed to study the impact of OTS instructions on learning. Results showed that while OTS instructions facilitated short-term learning, (a) they took time away from the processing of current information, (b) their effects tended to decay rapidly in initial stages of training, and (c) their effects on training diminished when the OTS instructions were proceduralized in later stages of training. A cognitive model that learned from both theupfront and OTS instructions was created and provided good fits to the learning and performance data collected from human participants. Our results suggest that to fully capture the symbiotic performance between humans and intelligent training systems, it is important to closely monitor the learning process of the trainee so that instructional interventions can be delivered effectively at different stages of training. We proposed that such a flexible system can be developed based on an adaptive cognitive model that provides real-time predictions on learning and performance.

Gray, W. D., Sims, C. R., Fu, W.-T., Schoelles, M. J. (2006), The soft constraints hypothesis: A rational analysis approach to resource allocation for interactive behavior. Psychological Review, 113(3), 461-482. [PDF] [Abstract]

Soft constraints hypothesis (SCH) is a rational analysis approach that holds that the mixture of perceptual-motor and cognitive resources allocated for interactive behavior is adjusted based on temporal cost-benefit tradeoffs. Alternative approaches maintain that cognitive resources are in some sense protected or conserved in that greater amounts of perceptual-motor effort will be expended to conserve lesser amounts of cognitive effort. One alternative, the minimum memory hypothesis (MMH), holds that people favor strategies that minimize the use of memory. SCH is compared with MMH across 3 experiments and with predictions of an Ideal Performer Model that uses ACT-R's memory system in a reinforcement learning approach that maximizes expected utility by minimizing time. Model and data support the SCH view of resource allocation; at the under 1000-ms level of analysis, mixtures of cognitive and perceptual-motor resources are adjusted based on their cost-benefit tradeoffs for interactive behavior.


Fu, W.-T., Gray, W. D. (2004), Resolving the paradox of the active user: Stable suboptimal performance in interactive tasks. Cognitive Science, 28(6), 901-935. [PDF] [Abstract]

This paper brings the intellectual tools of cognitive science to bear on resolving the "paradox of the active user" [Interfacing Thought: Cognitive Aspects of Human–Computer Interaction, Cambridge, MIT Press, MA, USA]—the persistent use of inefficient procedures in interactive tasks by experienced or even expert users when demonstrably more efficient procedures exist. The goal of this paper is to understand the roots of this paradox by finding regularities in these inefficient procedures. We examine three very different data sets. For each data set, we first satisfy ourselves that the preferred procedures used by some subjects are indeed less efficient than the recommended procedures. We then amass evidence, for each set, and conclude that when a preferred procedure is used instead of a more efficient, recommended procedure, the preferred procedure tends to have two major characteristics: (1) the preferred procedure is a well-practiced, generic procedure that is applicable either within the same task environment in different contexts or across different task environments, and (2) the preferred procedure is composed of interactive components that bring fast, incremental feedback on the external problem states. The support amassed for these characteristics leads to a new understanding of the paradox. In interactive tasks, people are biased towards the use of general procedures that start with interactive actions. These actions require much less cognitive effort as each action results in an immediate change to the external display that, in turn, cues the next action. Unfortunately for the users, the bias to use interactive unit tasks leads to a path that requires more effort in the long run. Our data suggest that interactive behavior is composed of a series of distributed choices; that is, people seldom make a once-and-for-all decision on procedures. This series of biased selection of interactive unit tasks often leads to a stable suboptimal level of performance.

Gray, W. D., Fu, W.-T. (2004), Soft constraints in interactive behavior: The case of ignoring perfect knowledge in-the-world for imperfect knowledge in-the-head. Cognitive Science, 28(3), 359-382. [PDF] [Abstract]

Constraints and dependencies among the elements of embodied cognition form patterns or microstrategies of interactive behavior. Hard constraints determine which microstrategies are possible. Soft constraints determine which of the possible microstrategies are most likely to be selected. When selection is non-deliberate or automatic the least effort microstrategy is chosen. In calculating the effort required to execute a microstrategy each of the three types of operations, memory retrieval, perception, and action, are given equal weight; that is, perceptual-motor activity does not have a privileged status with respect to memory. Soft constraints can work contrary to the designer's intentions by making the access of perfect knowledge in-the-world more effortful than the access of imperfect knowledge in-the-head. These implications of soft constraints are tested in two experiments. In experiment 1 we varied the perceptual-motor effort of accessing knowledge in-the-world as well as the effort of retrieving items from memory. In experiment 2 we replicated one of the experiment 1 conditions to collect eye movement data. The results suggest that milliseconds matter. Soft constraints lead to a reliance on knowledge in-the-head even when the absolute difference in perceptual-motor versus memory retrieval effort is small, and even when relying on memory leads to a higher error rate and lower performance. We discuss the implications of soft constraints for routine interactive behavior, accounts of embodied cognition, and tool and interface design.


Fu, W.-T. (2001), ACT-PRO action protocol analyzer: A tool for analyzing discrete action protocols. Behavior Research Methods, Instruments, & Computers, 33 (2), 149-158. [PDF] [Abstract]

This article presents a top-down approach for analyzing sequential events in behavioral data. Analysis of behavioral sequential data often entails identifying patterns specified by the researchers. Algorithms were developed and applied to analyze a kind of behavioral data, called discrete action protocol data. Discrete action protocols consist of discrete user actions, such as mouse clicks and keypresses. Unfortunately, the process of analyzing the huge volume of actions (typically, >105) is very labor intensive. To facilitate this process, we developed an action protocol analyzer (ACT-PRO) that provides two levels of pattern matching. Level one uses formal grammars to identify sequential patterns. Level two matches these patterns to a hierarchical structure. ACT-PRO can be used to determine how well data fit the patterns specified by an experimenter. Complementarily, it can be used to focus an experimenter's attention on data that do not fit the prespecified patterns.