General Research Interests
I am interested in understanding adaptive human performance and strategic adaptation. How do cognitive characteristics, task characteristics, and context shape human performance? And how well can people optimize their performance given these constraints?
Although interested in these broader questions across domains, two domains have received special attention: multitasking and vision (loss). In the multitasking domain, I study how well people can optimize the way they interleave their attention between tasks given the task characteristics and the priorities that they set (or the rewards that they receive). In vision, I have studied how people who have lost part of their vision due to macular degeneration can adapt to this physiological change. How does this influence the way that they look around in the world and how can they get better at this?
In my research I use a combination of empirical studies and formal modeling. Observations that are made in studies are formalized in theories as developed in computational cognitive models. I have also used these models to develop adaptive "user models", which can dynamically adapt the state of a system to the performance and characteristics of individual users. In my work I combine insights from, and publish in, the fields of cognitive science, experimental psychology, vision, human factors, human-computer interaction, and computer science.
Below are listed some specific projects that I am or was involved in. If you are a student, there are almost always opportunities to do a research project related to each of these themes.Ongoing projects
Understanding human behavior in (semi-)autonomous vehicles
Technological development of autonomous vehicles is in full swing. Although the technology is promising, less is known about human behavior. Automation research suggests that human behavior changes when automation is introduced. We try to understand how human behavior changes when the car is automated. Several projects in my lab investigate aspects of human behavior in automated settings. This includes the PhD research or Remo van der Heiden (of whom I am primary supervisor) and my own Marie Curie supported research.
Representative publicationsJournal papers
Video clips describing (part of) this work
On my personal Youtube channel, I post video clips describing my research. Here are clips about my work on autonomous driving:
Adaptive visual strategies for information maximization: comparing healthy adults with adults with macular degeneration
(initiated as post-doc research at Smith Kettlewell eye-research institute)
Age-related macular degeneration (AMD) is one of the leading causes of reduced visual function that cannot be corrected optically. In the United States alone, it affects 6.5% of the population over the age of 40 (Klein, Chou, Klein, Zhang, Meuer, & Saaddine, 2011). However, it is a condition that becomes more pervasive with age, and by the age of 80, around a third of all adults has some form of AMD (Friedman et al., 2004). In AMD, a scotoma (or blindspot) develops, typically around the fovea. Due to the scotoma, people have to change their visual strategies for locating information in the world. No longer can they use their high-resolution fovea, which they used for decades and to which the visual and cognitive system was adapted.
In the absence of foveal vision, most individuals learn to use an eccentric pseudo fovea, called the preferred retinal locus (PRL), which is typically located just outside the scotoma. But many of these individuals have difficulty directing the PRL to effectively scan items of interest, and have difficulty reading (e.g., Nilsson, Frennesson, & Nilsson, 2003; Seiple, Grant, & Szlyk, 2011). Developing a better understanding of how and why these adaptive strategies work for some and not others, has the potential to inform eye movement training methods for all people with AMD, such that their sight can improve.
To work towards this larger goal, I examine strategies for moving the eyes efficiently both in healthy adults and adults with AMD. The research uses a combination of empirical studies (with healthy adults and a patient population) and computational cognitive modeling.
Representative publicationsJournal papers
Understanding Strategic Adaptation in Dual-Task Situations as Cognitively Bounded Rational Behavior
(initiated as PhD research, University College London)
I successfully defended my thesis in April 2012. I received the 2014 Briggs Award for the Best Doctoral Dissertation in Applied Experimental/Engineering Psychology, awarded by the APA 21 division (site)
This is the abstract of my thesis:
"In this thesis I explored when people interleave attention in dual-task settings. The hypothesis is that people try to perform in a cognitively bounded rational way. Performance is limited by constraints that come from the task environment and cognition. If, given these constraints, multiple strategies for interleaving tasks are available, then people will interleave tasks in a way that aligns with their local priority objective (Chapter 3), or which maximizes the value of an objective payoff function that evaluates performance (Chapter 4). This hypothesis was tested using a combination of experimental studies and computational cognitive models. Across a series of studies, the interplay between different constraints was investigated. In Chapters 5 and 6, I developed mathematical models to study what task combinations in general allowed for "ideal payoff manipulations" to study task interleaving. The work contributed to the existing literature in four ways: (1) it provided an overarching theory of skilled human dual-task performance and tested this in relatively applied settings, (2) the theory was formalized in computational cognitive models that can predict performance of unobserved strategies and that can bracket the (optimal) performance space, (3) linear and logarithmic tasks were identified as an ideal combination for achieving ideal payoff manipulations, and (4) results demonstrated that in multitasking situations attention is not necessarily interleaved solely at chunk boundaries and other "natural breakpoints", but that this depends on a person's priorities. The work has implications for driver distraction research, in that it helps in systematically understanding the performance trade-offs that people face when multitasking. Moreover, the modeling framework could be used for model-based evaluation of new mobile interfaces. Finally, the demonstration that priorities can strongly influence multitasking performance highlights the importance of public safety campaigns that emphasize awareness of driver safety. Limitations and further implications are discussed. "
Video clips describing (part of) this work
On my personal Youtube channel, I post video clips describing my research. Here are clips about my work on multitasking and efficiency:
Understanding how conversations while driving can be made safer
(With colleagues at Microsoft Research)
Talking while you are driving can be distracting. However, not all conversations are equally distracting. In this work we try to understand what makes conversations distracting. We then try to use these insights to try and make in-car conversations safer. In the study described below participants drive in a driving simulator. While they drive, they also perform a conversation task with a remote partner. Unknown to the driver, we share sounds from the driver's context (e.g., car honks, sirens) with the remote caller. We found that sharing these sounds changes the remote caller's perception of how busy the driver is. However, this only had a modest effect on the conversation and on driving performance. This highlights a need for better training or information of how to then act on this information to make conversations less distracting to the driver.
(of course, if you find yourself in a situation like this, the safest thing is to hang up the phone and post-pone the call to a later, safer time!)
You can watch a video of this setup by clicking on the following link (to video).
When, what, and how much to reward in reinforcement learning models of cognition
(initiated as MSc thesis research, Rensselaer Polytechnic Institute)
In this project we investigated how choices for reinforcement learning models about when, what, and how much to reward in the model influences subsequent behavior.
User modeling for training recommendation in a depression prevention game
(initiated as Junior Researcher, University of Groningen, The Netherlands)
I worked as a junior researcher of the Cognitive modeling research group of the Artificial Intelligence Department of the University of Groningen, and developed a user model for a serious game. Within this game players learned to cope with depression. My model used principles from cognitive science and AI to capture the learned social skills of the user automatically.
As a result, each game can adapt to characteristics of the individual user. This individual approach is very important, as depression is characterized by several characteristics, of which a depressed person most of the time only has a subset. The intensity of those characteristics differs for each person.
Personalization of a Virtual Museum Tour using Eye-gaze
(initiated as BSc project, University of Groningen, The Netherlands)
For my BSc project, I developed a system that tracked people's fixations while they were watching digital paintings. Based on the fixations, a "virtual tour guide" would play audio snipets that explained aspects of the art work - especially those that the user fixated most on. Overall goal was to find a way in which a tour through a digital museum could be personalized.