Tuning the senses: How the pupil shapes vision at the earliest stage

The pupil responds reflexively to changes in brightness and focal distance to maintain the smallest pupil (and thus the highest visual acuity) that still allows sufficient light to reach the retina. The pupil also responds to a wide variety of cognitive processes, but the functions of these cognitive responses are still poorly understood. Here I propose that cognitive pupil responses, like their reflexive counterparts, serve to optimize vision. Specifically, an emphasis on central vision over peripheral vision results in pupil constriction, which matches the fact that central vision benefits most from increased visual acuity. Furthermore, an intention to act with bright stimuli results in preparatory pupil constriction, which allows the pupil to respond quickly when that bright stimulus is subsequently brought into view. More generally, cognitively driven pupil responses are likely a form of sensory tuning: a subtle adjustment of the eyes to optimize their properties for the current situation and the immediate future.


Introduction
Vision scientists traditionally distinguish perception from sensation. Perception refers to the brain's interpretation of sensory input, and as such would be affected by cognitive factors such as prediction, attention, and reward. In contrast, sensation refers to how (photo)receptors respond to sensory input, and as such would be unaffected by cognition. But how independent is visual sensation from cognition, really?
The key tenet of this review is that we actively tune our senses, and specifically our eyes, to optimize their properties for the current situation and the immediate future. Saccadic eye movements are a familiar example of this: We shift our gaze to bring relevant information into central vision (Kowler, 2011). This allows us to sense relevant information with the central part of the retina, which contains a dense network of cone photoreceptors that provides high-acuity vision (Curcio, Sloan, Packer, Hendrickson, & Kalina, 1987).
But the muscles of the eye allow for many more movements than only those that shift gaze in space.
For example, the curvature of the lens can increase (accomodation) to shift the point of focus from far to near (Brown, 1973). And the eyes can rotate clockwise or counterclockwise (when viewed from the front); such torsional eye movements may serve little purpose in humans or other animals with frontally placed eyes, but they stabilize gaze in animals with lateral eyes (Banks, Sprague, Schmoll, Parnell, & Love, 2015). In total, the eye is controlled by fourteen muscles: six that rotate the eye, five that control the eye lid, one that controls lens accomodation, and two that control pupil size; together, these muscles provide the eye with an incredible freedom of movement and shape.
In this review, I will focus on pupil responses, which profoundly affect how visual information falls onto the retina. I will start by introducing the three main factors that cause the pupil to constrict (become smaller) or dilate (become bigger): light, focal distance, and arousal (Section 2). I will then describe how pupil size is related to visual cognition, with a focus on visual attention, working memory, and mental imagery (Section 3) and also to eye movements (Section 4). Next, I will describe how pupil responses affect the way that light falls onto the retina, which in turn affects the way that visual input is processed by visual brain areas, which in turn affects affects visually guided behavior and subjective visual experience (Section 5). Finally, I will outline a general theory of sensory tuning based on the findings reviewed in this article; this theory will focus on vision and pupil size, but I will end by proposing that sensory tuning is a general principle of sensation and perception.

Pupil responses
There are three broad classes of pupil responses. These differ primarily in the stimulus that triggers 1. Introduction 3 of 27 the response, but they are also controlled by partly distinct neural pathways (see Mathôt, 2018 for a detailed discussion).

The pupil light response
When you walk from a shady office into the bright outdoors, your pupils rapidly constrict; this is the pupil light response (PLR), a large response that can change the pupil from its maximum (±8 mm in humans) to its minimum size (±2 mm in humans), thus changing the amount of light that enters the eye by a factor of about 16. Following exposure to light, the pupil starts to constrict with a latency of 200 -250 ms (Ellis, 1981), reaching its minimum size after about 1,000 -2,000 ms (the exact latencies depend mostly on the strength of the stimulus). This initial constriction is ballistic in the sense that it occurs even in response to very brief flashes of light, in which case the pupil starts to constrict only after the light has already been extinguished. The initial constriction is driven largely by rods and cones, the same photoreceptors that also mediate regular (image-forming) vision; consequently, the initial constriction has many of the same properties as regular vision, including a fast response profile that is dominated by input from central vision (Crawford, 1936;Hong, Narkiewicz, & Kardon, 2001).
Rods and cones respond vigorously to changes in brightness, but quickly adapt when brightness remains constant (Nakatani & Yau, 1988). Therefore, if the PLR were only driven by rods and cones, increases in brightness would cause a transient pupil constriction, but this constriction would not be maintained. The fact that pupils can stay constricted indefinitely is due to a different class of photoreceptors: intrinsically photosensitive retinal ganglion cells (ipRGCs) (reviewed in Do, 2019).
Like regular retinal ganglion cells, ipRGCs receive input from rods and cones. But in addition they are also photosensitive themselves, through a photopigment called melanopsin. The melanopsin response is slow, with a latency of up to 10 seconds, does not show adaptation, and has a peak sensitivity to bluish light, somewhat in between the peak sensitivities of rods and S (blue) cones (Markwell, Feigl, & Zele, 2010). This melanopsin response is what keeps your pupils constricted throughout the day.
The PLR relies on a subcortical, parasympathetic pathway that carries luminance information from photoreceptors in the retina, via the Pretectal Olivary Nucleus (PON) and the Edinger-Westphal Nucleus (EWN), and back towards the eye, where contraction of the iris sphincter muscle results in pupil constriction (Kardon, 2005;McDougal & Gamlin, 2008). The cognitive influences on the PLR that I will discuss in Sections 3 and 4 likely reflect a modulation of this subcortical pathway by cortical brain areas; however, the exact mechanisms behind this modulation are still unclear.

The pupil near response
When you shift focus from an object that is far away towards an object that is nearby, three different 1. Introduction 4 of 27 eye movements occur in concert: vergence, an inward rotation of the eyes that brings the nearby object in central vision for both eyes; accomodation, an increase in lens curvature that brings the nearby object into focus; and the pupil near response (PNR), a pronounced pupil constriction (Mays & Gamlin, 1995;McDougal & Gamlin, 2008). Together with brightness, focal distance is the main determinant of pupil size.
The neural pathway that drives the PNR is less well-understood than that of the PLR. Possibly, cortical areas, including the frontal eye fields in the macaque brain or its homologue in the human brain, project to the EWN. From there, the pathway would be identical to that of the PLR (McDougal & Gamlin, 2008).

The psychosensory pupil response
There are many psychological processes that are accompanied by pupil dilation: arousal, emotion (positive and negative), mental effort, working-memory load, motor preparation, and many more (reviewed in Beatty, 1982;Beatty & Lucero-Wagoner, 2000;Goldwater, 1972;Laeng & Alnaes, 2019;Loewenfeld, 1958). These processes are varied, yet all have in common that they are characterized by a general increase in cognitive activity, which has been dubbed the 'intensity dimension of thought' (cf. Just & Carpenter, 1993), and that they are all accompanied by a slight dilation of the pupil. I refer to this phenomenon as the psychosensory pupil response (PPR). Other authors have used different terms, such as reflex dilation, effort-related dilation, and arousal-related dilation; all of these refer to the same phenomenon.
Several authors have argued that the PPR is a non-functional epiphenomenon, and that its interest for psychologists lies solely in the fact that it can be used as a reporter variable for various cognitive processes (e.g. Beatty & Lucero-Wagoner, 2000) . However, as I will discuss in Section 6, I think it is more fruitful to view the PPR as a subtle form of sensory tuning that adapts vision to the needs of the situation.
The PPR relies on a subcortical, sympathetic pathway that projects from several brain areas that reflect arousal (notably the hypothalamus, locus coeruleus [LC], and the superior colliculus [SC]) to the eye, where contraction of the iris dilator muscle triggers pupil dilation (Kardon, 2005;McDougal & Gamlin, 2008).

Pupil responses and visual cognition
The way in which visual input is processed depends on many cognitive factors, including visual attention, visual working memory, and visual mental imagery. This is the domain of visual cognition (Cavanagh, 2011). The fact that many of these same cognitive factors also affect pupil size highlights that pupil responses are an integral part of visual cognition.

Covert attention towards the periphery (attentional breadth)
The term "attentional breadth" refers to how diffusely attention is spread across the visual field. A broad attentional focus encompasses much of the visual periphery, and contrasts with a narrow focus of attention on central vision. (This terminology implicitly characterizes attention as a zoomlight that changes size while remaining centered on central vision, rather than as a spotlight that moves around in space. This is a simplified characterization, but useful for the present purpose.) Central and peripheral vision differ in many ways. These differences already start at the level of the retina (Curcio et al., 1987). Specifically, the distribution of cones is much denser in the fovea than in the retinal periphery; this is especially the case for red-and green-sensitive cones, but to some extent also for blue-sensitive cones. In contrast, rods are distributed more uniformly across the retina, with a peak density in the dorsal retina, which corresponds to the lower visual field. Given the many differences between central and peripheral vision, and assuming that pupil size adapts to the demands of the situation, the question arises whether pupil size varies as a function of whether attention is narrowly focused on central vision, or rather is diffusely spread across peripheral vision; that is, does attentional breadth affect pupil size?
To address this question, Daniels, Nichols, Seifert, & Hock (2012) presented an array of stimuli to participants. Some of these stimuli were near (but just outside of) central vision, while other stimuli were placed further into peripheral vision. Participants were instructed to shift their focus of attention between the central and the peripheral stimuli, as indicated by rhythmic changes in color of a central fixation dot. Crucially, Daniels et al. (2012) found that these rhythmic changes in attentional breadth also induced rhythmic changes in pupil size, suggesting that pupil size is affected by attentional breadth (see also Brocher, Harbecke, Graf, Memmert, & Hüttermann, 2018; . More recently, we conducted an experiment designed to address some of the limitations of previous work (Ivanov, Lazovic, & Mathôt, 2019) . Specifically, we directly measured pupil-size changes in response to shifts of attention, rather than conducting a time-frequency analysis as done by Daniels et al. (2012). In addition, we kept the task (unlike , and difficulty and visual stimulation (unlike Brocher et al., 2018) constant across all conditions. Participants saw patches of tilted lines (gabor patches), three on each side of fixation, at various eccentricities ( Figure 1a).
Participants indicated the orientation of two targets, which were tilted gabor patches, one on each side of fixation. A pre-cue indicated whether the targets would be presented at the near, medium, or far eccentricity. Crucially, we found that pupil size increased with increasing eccentricity of the attended location (Figure 1b). In summary, the studies by Daniels et al. (2012), Brocher et al. (2018), and our own  suggest that the size of the pupil flexibly adapts to attentional breadth; specifically, the pupil is larger when attention is diffusely spread across a large part of the visual field, as compared to when attention is narrowly and centrally focused.

Attention to, working memory of, and imagery of bright or dark stimuli
The pupil light response (PLR; see Section 2) was traditionally considered a low-level reflex to light, and not as something that is susceptible to cognitive influences. The PLR is indeed reflexive in the sense that it is a stereotyped response that is automatically triggered by light: If someone shines a light in your eye (and assuming that you are neurologically intact), your pupils will always constrict and never dilate, and this constriction will always have the same stereotyped profile as described in Section 2. However, cognitive factors can increase or decrease the strength of the PLR, and even induce a (weak) PLR-like response in the absence of direct visual stimulation.
In one experiment, we tested whether reflexive shifts of attention towards bright or dark objects affect pupil size (Mathôt, Dalmaijer, Grainger, & Van der Stigchel, 2014; see also Binda, Pereverzeva, & Murray, 2013;Mathôt, van der Linden, Grainger, & Vitu, 2013;Naber, Alvarez, & Nakayama, 2013;Unsworth & Robison, 2017). Participants fixated in the center of a display that was horizontally divided into a bright and a dark half. Two patches of lines were presented, one on the bright side of the screen, and one on the dark side. Next, one of these patches seemed to move for 50 ms (through a continuous phase change that induces a motion signal), which captures attention reflexively. Crucially, we found that when attention was captured towards the bright side of the screen, the pupil was smaller than when attention was captured towards the dark side. That is, even 3. Pupil responses and visual cognition when visual input and eye position are controlled, covert spatial attention towards brightness or darkness affects pupil size.
In the study described above, attention was directed to a bright or dark location in space. However, even when bright and dark stimuli overlap in space, selectively attending to either stimulus affects the size of the pupil (i.e. feature-based, as opposed to spatial, attention). In a recent study, Turi, Burr, & Binda (2018) presented two superimposed fields of random dots moving in opposite directions ( Figure 2a). One field consisted of bright dots; the other field consisted of dark dots (see also . Perceptually, this type of stimulus gives the impression of a rotating cylinder, where the direction of rotation depends on which field is attended, and thus perceived as being in front. Based on the direction of rotation that participants reported, the authors could therefore determine whether participants were attending to the bright or the dark field. The authors found that feature-based attention towards brightness or darkness affects pupil size, in line with similar studies on spatial attention ( Figure 2b). Strikingly, the authors also found strong-but-systematic individual differences in the strength of this effect; specifically, pupil size was affected most strongly in participants who scored high on autistic traits (Figure 2c), presumably because these participants tend to focus more strongly on details (in this case the front surface of the cylinder) rather than on the whole. This finding illustrates that pupil size reflects not only basic perceptual processes, but also individual differences in visual cognition. Turi, Burr, and Binda (2018). a) Participants viewed two superimposed fields of dots, one black and one white, moving in opposite directions. Subjectively, either the black (blue line) or white (red line) field is perceived as being in front, and perception frequently switches between the two. b) When the black field is perceived as being in front, the pupil is larger than when the white field is perceived as being in front. c) The pupil-size difference (black -white) is larger for participants who score high on autistic traits. [Figure adapted from: Turi et al. (2018). License: CC-by.] In the studies described above, bright or dark stimuli were always visible to the participant; therefore, changes in pupil size resulted from an interaction between visual input and cognitive processes. But 3. Pupil responses and visual cognition 8 of 27 are cognitive processes by themselves sufficient to elicit a PLR, even in the absence of visual stimuli? Several recent studies have shown that this is indeed possible. In a series of experiments, we asked participants to maintain both a dark and a bright stimulus in visual working memory (Husta, Dalmaijer, Belopolsky, & Mathot, 2019; see also Zokaei, Board, Manohar, & Nobre, 2019) . After the stimuli had been removed, a retro-cue indicated which of the two stimuli would be probed later.

Figure 2. Schematic paradigm and results for
Crucially, we found that when the dark stimulus was cued, pupil size was larger than when the bright stimulus was cued. This shows that, even when there is no direct visual stimulation, a mental representation of brightness or darkness is sufficient to (slightly) change the size of the pupil. Other studies, which used mental imagery (Laeng & Sulutvedt, 2014) or word comprehension (Mathôt, Grainger, & Strijkers, 2017) to elicit a mental representation of brightness or darkness, have found similar results.
In summary, the PLR is a reflex that is modulated by visual cognition. That is, a flash light of light always triggers a reflexive pupil constriction, but the strength of this constriction is modulated by visual cognition. A small PLR-like response can even be elicited by a mental representation of brightness or darkness in the absence of visual stimulation.

Mental imagery of stimuli that are nearby or far away
So far, most studies that have looked at interactions between visual cognition and pupil size have focused on the PLR. However, there is some evidence that the pupil near response (PNR) is affected by some of the same cognitive factors that also affect the PLR. The most compelling evidence so far comes from a study by Sulutvedt, Mannix, & Laeng (2018), in which participants were first shown an object, which was subsequently removed from the display. Next, participants were asked to imagine the object placed either nearby or far away. Crucially, the authors found that the pupils were smaller when participants imagined nearby objects than when they imagined objects that were far away, suggesting that mental imagery can trigger a weak PNR, similar to what has been found for the PLR (Laeng & Sulutvedt, 2014). Future studies will need to replicate and extend this initial finding to establish more firmly whether the PNR is indeed susceptible to cognitive influences.

Pupil responses and spatial eye movements
Spatial eye movements shift gaze from one location to another; such eye movements contrast with non-spatial eye movements, such as accomodation, pupil responses, and torsional eye movements, which change the properties of the eye in different ways. The two most-studied types of spatial eye movements are saccadic eye movements, which shift gaze between objects, and smooth pursuit eye movements, which track moving objects (Kowler, 2011). There are strong connections between pupil responses and spatial eye movements, both in terms of overlapping neural pathways (Wang & Munoz, 2015) and in the sense that eye movements are often accompanied by changes in pupil size.

Pupil constriction after blinks and saccadic eye movements
Saccadic eye movements are followed by a pronounced pupil constriction that resembles the response to a brief flash of light: the pupils start to constrict with a latency of about 200 -250 ms after the eye movement, and it takes about three seconds before they have regained their original size (Knapen et al., 2016;Mathôt et al., 2015a;Zuber, Stark, & Lorber, 1966) . Eye blinks trigger a very similar pupil response (Knapen et al., 2016) .
It is not entirely clear what triggers these pupil responses to blinks and saccadic eye movements.
But the main driving force may be visual change, which causes a transient pupil constriction, even without changes in overall luminance (Sahraie & Barbur, 1997;Slooter & van Norren, 1980;Ukai, 1985;Van de Kraats, Smit, & Slooter, 1977) . For example, when you look at a checkerboard that inverses polarity (all white tiles become black and vice versa), the pupils briefly constrict (Slooter & van Norren, 1980). Eye movements are accompanied by large shifts of visual input across the retina; and blinks are accompanied by a brief-but-severe blanking of visual input. The hypothesis that these visual changes are what trigger pupil constriction after blinks and eye movements is supported by the finding that, when additional visual change is introduced during an eye movement (in the form of an intrasaccadic percept), the subsequent pupil constriction also becomes more pronounced (Mathôt et al., 2015a).

Preparation of saccadic eye movements towards bright or dark stimuli
As discussed in Section 2, when a light is switched on, the pupils constrict with a latency of 200 -250 ms (Ellis, 1981). However, in this situation you are a passive receiver of brightness changes, and this is far from typical of daily life. More commonly, you actively control brightness changes by making eye movements towards bright or dark objects, in which case the visual system can anticipate the changes in luminance before they occur, effectively reducing the latency of the PLR.
To test whether preparation plays a role in the PLR, we performed a simple experiment in which participants initially fixated at the center of a display that was bright on one side, and dark on the other (Mathôt et al., 2015b). Next, a cue instructed participants to make an eye movement either towards the left or towards the right. In one condition of the experiment, the display flipped as soon as the eyes set in motion; that is, the side of the display that was initially dark became bright, and vice versa. This allowed us to dissociate the preparatory component of the PLR (driven by the presaccadic brightness) from its reactive component (driven by the post-saccadic brightness). Crucially, we found that the pupil started to respond to the pre-saccadic brightness almost immediately when 4. Pupil responses and spatial eye movements the eyes set in motion; this preparatory response then gradually dissipated, until after 450 ms the pupils mostly responded to the post-saccadic brightness. This finding suggests that the PLR is not a passive response, but rather is prepared along with (or rather, as part of) saccadic eye movements (see also Ebitz, Pearson, & Platt, 2014).

Exploration, exploitation, and object-based attention
The adaptive-gain theory (AGT) is an influential framework that links behavior to pupil size and activity in the Locus Coeruleus (LC), a brain-stem area (Aston- Jones & Cohen, 2005). Specifically, exploration refers to a mode of behavior that is characterized by distractibility and frequent switching between tasks. Exploration would be accompanied by elevated tonic (sustained) firing of the LC, but reduced phasic (event-related) firing; analogously, exploration would be accompanied by large pupils that are not very reactive to stimuli (i.e. pupil responses would reflect LC firing rates). In contrast, exploitation refers to a mode of behavior that is characterized by focus on a single task. Exploitation would be accompanied by reduced tonic and increased phasic firing of the LC, and analogously by medium-to-small pupils that are highly reactive to stimuli. Simply put, the AGT posits that the LC is a neural control center for behavior, and that pupil size is useful as a marker of LC activity. This theory provides a useful framework, although the link between LC activity and pupil size is likely much more complex than this simplified view suggests (Joshi & Gold, 2019).
Most studies on exploration and exploitation have used game-like tasks, such as the Wisconsin Card Sorting Test (Pajkossy, Szőllősi, Demeter, & Racsmány, 2017) or a modified version of the Iowa Gambling task (Jepma & Nieuwenhuis, 2011) . In these tasks, modes of behavior are inferred from how participants play the game; for example, switching from one deck of cards to another would be indicative of exploration, whereas sticking to the same deck would be indicative of exploitation.
However, the terms exploration and exploitation are also directly applicable to eye movements. In this context, exploitation would refer to within-object eye movements that inspect different parts of a person, object, or text; for example, an eye movement from 'this word' to 'this word' would reflect exploitation. In contrast, exploration would refer to between-object eye movements that carry gaze from one object to another; for example, an eye movement that shifts gaze away from this text to check for notifications on your smartphone would reflect exploration.
We recently conducted an experiment to test whether the predictions of the AGT hold up in the context of eye movements in a visual-search task (Mathôt & Regnath, 2019). In our study, participants searched for a target letter among a large number (143) of distractor letters. The search display was divided into four randomly generated regions that were defined by color (Figure 3a).
These regions were irrelevant to the search-task, but we nevertheless predicted that participants would be sensitive to the boundaries between these regions. 4. Pupil responses and spatial eye movements We found that participants tended to search within regions for longer than would be expected by chance; that is, participants first made exploitation-like eye movements within regions, before making exploration-like eye movements that carried gaze from one region to another. Crucially, we also found that these exploration-like eye movements were preceded by a slight pupil dilation (Figure 3b).
Although these findings are correlational (we did not manipulate whether participants made within-or between-region eye movements), and should be replicated, this pattern of results is consistent with the AGT.

Effects of pupil size on visual processing
Most of the research reviewed so far has looked at pupil size as a function of visual input, cognitive factors, or a combination of both. However, the relationship between pupil size and visual input is bidirectional: pupil size also affects how visual input is processed.

Effects of pupil size on detection and discrimination performance
When considering the effect of pupil size on performance on visual tasks, it is useful to distinguish discrimination tasks from detection tasks. In a discrimination task, the goal is to identify a stimulus; a 5. Effects of pupil size on visual processing 12 of 27 prototypical example of a discrimination task is reading. In a detection task, the goal is to detect the presence of a faint stimulus without indicating its identity; a prototypical example of a detection task is driving through a thick fog, in which case the driver needs to respond to any kind of stimulus that might suddenly emerge from the fog.
Small pupils are generally advantageous for discrimination tasks (Campbell & Gregory, 1960;Woodhouse, 1975). This is because the eye's lens suffers from imperfections that distort the image in various ways, for example by blurring the image and by having a slightly different focal distance for different wavelengths of light (Liang & Williams, 1997). These optical distortions become less severe with decreasing pupil size, and this leads to measurable improvements in discrimination performance. For example, human-factors research has shown that it is easier to discriminate letters when they are presented against a bright background (Buchner, Mayr, & Brandt, 2009;Dobres, Chahine, & Reimer, 2017;Piepenbrock et al., 2014a); this so-called positive-polarity advantage is likely due in large part to the fact that a bright background induces small pupils (Piepenbrock et al., 2014b).
Large pupils are generally advantageous for detection tasks, especially when the goal is to detect faint stimuli that are at the threshold of detectability. This is because large pupils allow more light into the eye, thus increasing the signal; that is, large pupils make it easier to distinguish something from nothing. Although surprisingly few studies have directly investigated this large-pupil advantage for detection, we recently found that it is easier to detect a faint stimulus presented at an unpredictable location when pupils are large, as compared to when they are small, at least when pupil size is manipulated through the brightness of the visual periphery .
The simple narrative in which large pupils benefit detection, whereas small pupils benefit discrimination, becomes more complex when considering additional factors. For example, the fact that large pupils increase the amount of light that enters the eye means that large pupils also increase the amount of retinal light scatter (Lombardo & Lombardo, 2010). Retinal light scatter creates a diffuse (unfocused) veil of light across the retina, which could hinder the detection of faint stimuli. In other words, in situations where retinal light scatter is a prominent factor (for example because there is a bright source of light somewhere, resulting in so-called discomfort glare), detection of faint stimuli might actually benefit from small, rather than large, pupils. Additional complicating factors are dark adaptation, which affects the relative extent to which vision is based on input from rods or cones, and (related) whether stimuli are presented in peripheral or central vision (see Kalloniatis & Luu, 1995 for a review of optical factors) .
In summary, small pupils improve visual acuity, and are therefore generally advantageous for discrimination tasks. In contrast, large pupils improve visual sensitivity, and are therefore generally advantageous for detection tasks. However, the effect of pupil size on visual performance depends 5. Effects of pupil size on visual processing 13 of 27 on many complex interactions between the size of the pupil, the observer's goals, the state of retina, and the environment, thus allowing for many exceptions to this general rule.

Effects of pupil size on brightness perception
When the pupil dilates, more light enters the eye. Then why do you not perceive a brightening of the world whenever your pupils dilate? The primary reason is that subjective brightness perception relies strongly on indirect clues, such as context (is an object in the shadows?) and world knowledge (refrigerators tend to be white). But what if these clues are not available? Does pupil size then affect subjective brightness perception, and, if so, how?
We recently conducted a series of experiments to test this (Wardhani, Boehler, & Mathôt, 2019) . In one of these experiments, participants were first instructed to remember the brightness of a reference stimulus. Next, we presented a task-irrelevant blue or red stimulus for 10 s; the shades of blue and red were equiluminant (as determined with a separate procedure before the experiment), but a prolonged blue inducer, as compared to a red inducer, strongly activates the ipRGCs (see Section 2), resulting in a slight sustained pupil constriction (Do, 2019). Finally, we presented a tester stimulus, and participants indicated whether the tester was brighter or darker than the referent.
Finally, we determined how bright the tester needed to be in order to be perceived as equally bright as the referent, as a function of whether the pupil was small (blue inducer) or large (red inducer).
Crucially, we found that the brightness of the tester was underestimated when the pupil was large, as compared to small. In other words, even though more light enters the eye when the pupil dilates, stimuli are perceived as less bright.
One interpretation of this result is that subjective brightness perception takes into account information about pupil size, either through proprioception (information about bodily states) or a corollary discharge (a copy of the motor commands that control pupil size, sometimes also called an efference copy) (reviewed in Sommer & Wurtz, 2008). The notion that visual input is combined with proprioception or an efference copy is commonly used to explain visual stability across eye movements (see Wurtz, 2008;Mathôt & Theeuwes, 2011); that is, eye movements dramatically change how the world is projected onto the retina, yet these retinal changes are not perceived as movement out there, presumably in part because the visual system relies on prioprioception and a corollary discharge to distinguish self-generated movement from movement in the world. Possibly, a similar mechanism allows us to distinguish self-generated changes in retinal illumination from changes in brightness in the world; specifically, an increase in retinal illumination would be discounted when accompanied by pupil dilation. Our finding that stimuli are subjectively perceived as less bright with increasing pupil size could indicate an overcompensation for the increased retinal illumination that accompanies pupil dilation.

Effects of pupil size on visual processing
In summary, our initial studies on the relationship between subjective brightness perception and pupil size suggest that larger pupils may lead to an underestimation of brightness. However, this result should be replicated and verified with different methods to manipulate pupil size.

Effects of pupil size on visuocortical processing
Changes in pupil size dramatically affect how light enters the eye, and therefore should, in some way, affect how the brain processes this information. However, exactly how pupil size affects visuocortical processing is not entirely clear.
Several studies have looked at correlations between pupil size and activity in visual cortex using functional magnetic resonance imaging (fMRI; e.g. DiNuzzo et al., 2019;Murphy, O'Connell, O'Sullivan, Robertson, & Balsters, 2014). Strikingly, these studies found that larger pupils are associated with decreased activity in visual cortex. For example, we recently looked at the correlation between pupil size and cortical activity while participants were watching a movie in an fMRI scanner (Mathôt & Hanke, 2019). We found that activity in visual cortex correlated negatively with pupil size. Importantly, this negative correlation appeared not to be driven by luminance, but rather by the amount of visual change in the video. That is, bright scenes lead to smaller pupils than dark scenes, but do not lead to increased visuocortical activity, presumably because the brightness of visual input is normalized at a very early level of processing (Carandini & Heeger, 2012) ; however, scenes with lots of movement (or other kinds of visual change) lead to both smaller pupils and increased visuocortical activity. A tentative interpretation of this finding is that the strong negative correlation between pupil size and visuocortical activity does not reflect a causal link, but rather is mediated by third factors, notably the amount of change in visual input. Thigpen, Bradley, & Keil (2018) looked at the link between pupil size and cortical activity using a different method. They recorded pupil size and electroencephalography (EEG) while participants viewed flickering stimuli, which results in rhythmic neural activity, or steady-state Visual Evoked Potentials (ssVEPs). The power of ssVEPs is a general measure of the strength of neural processing; for example, attended stimuli elicit stronger ssVEPs than unattended stimuli (Morgan, Hansen, & Hillyard, 1996). Thigpen et al. (2018) found that natural fluctuations in pupil size did not correlate with ssVEP strength, consistent with the idea that changes in retinal illumination are normalized and therefore do not affect cortical processing.
The only study so far to have looked at how pupil size causally affects cortical processing is by Bombeke, Duthoo, Mueller, Hopf, & Boehler (2016), who also used EEG. In one of their experiments, pupil size was manipulated by having participants covertly attend to a bright or a dark stimulus in the periphery, while maintaining central fixation (cf. Binda et al., 2013;Mathôt et al., 2013) . Next, a taskirrelevant stimulus was briefly presented either in the upper or the lower visual field. The authors then 5. Effects of pupil size on visual processing 15 of 27 looked at the C1, an event-related-potential (ERP) component that is believed to reflect the very earliest stage of visual processing. Crucially, Bombeke et al. (2016) found that large pupils resulted in an attenuated C1. They interpreted this finding in terms of visual acuity, such that large pupils would blur the input of the C1-eliciting stimulus, thus attenuating cortical responses. However, the C1-eliciting stimulus was presented in the visual periphery, where acuity is already poor, and the induced pupil-size changes were minute (0.2 mm in diameter in one experiment, and only 0.02 mm in diameter in another), which would result in only a very slight blurring of visual input; in other words, it is not clear whether the attenuation of the C1 could indeed have been due to their pupil-size manipulation, or whether it was inadvertently driven by some other aspect of the paradigm.
Why has a clear link between pupil size and visuocortical activity proven so elusive? Plausibly, the effects of pupil size on visual perception, and thus visuocortical activity, are most pronounced when stimuli are near the threshold of perception. That is, small pupils enhance perception of fine detail, but this effect is only evident for stimuli that are near the threshold of discriminability, such as very small letters Piepenbrock et al., 2014a) . And large pupils enhance detection of faint stimuli, but (although direct evidence for this is missing) this effect may again only be evident for stimuli that are near the threshold of detectability. This provides a clear direction for future studies, which should: a) directly manipulate pupil size, rather than rely on spontaneous fluctuations, and b) use stimuli that are near the threshold of discriminability or detectability.
In summary, there is a strong negative correlation between pupil size and activity in visual cortex (DiNuzzo et al., 2019;Murphy et al., 2014) . However, this correlation may be driven by mediating factors, such as changes in visual input, rather than reflect a causal link between pupil size and visuocortical activity. So far, there has not been a conclusive demonstration of such a causal link (but see Bombeke et al., 2016), although future studies, using near-threshold stimuli and effective manipulations of pupil size, may reveal this link.

A theory of sensory tuning
The main tenet of this article is that changes in pupil size reflect an adaptation of the senses to meet the demands of the current situation and the immediate future: sensory tuning. How do the findings reviewed above fit into this general framework? Is there evidence that sensory tuning also applies to non-visual modalities? And does a theory of sensory tuning lead to falsifiable predictions?

Pupil responses likely improve the quality of vision
The beneficial effects of the pupil light response (PLR) and pupil near response (PNR) are reasonably well understood (Sections 2 and 3; see also Mathôt, 2018) . Large pupils allow more light to enter the eye, thus improving vision in darkness, where visual sensitivity is limited by the available 6. A theory of sensory tuning 16 of 27 light ; therefore, pupils dilate in darkness. In contrast, small pupils focus light more sharply, thus improving visual acuity (Campbell & Gregory, 1960;Woodhouse, 1975) ; therefore, pupils constrict in brightness, when sensitivity is not limited by the available light.
Focusing on a nearby object places additional demands on visual acuity, because depth of field decreases dramatically with focal distance; that is, if you focus on an object that is very nearby (say at 20 cm), then an object that is slightly further away (say at 25 cm) is already considerably out of focus. Small pupils increase depth of field by improving focus for all distances (Campbell, 1957;Charman & Whitefoot, 1977), and this is likely why near-focus is accompanied by pupil constriction.
Whether cognitively driven pupil responses also improve the quality of vision is less clear; several authors have even argued that they do not, that cognitive effects on pupil size are too small to be behaviorally relevant, and must therefore be epiphenomenal (Beatty & Lucero-Wagoner, 2000;. However, a careful consideration of how cognitive factors modulate pupil size under various circumstances suggests differently. As reviewed in Section 3, the pupil is smaller when attention is focused centrally, as compared to when attention is spread diffusely across peripheral vision (Brocher et al., 2018;Daniels et al., 2012;Ivanov et al., 2019). This effect of attentional breadth on pupil size matches the properties of the retina: cone density is far higher in the central retina than in the peripheral retina (Curcio et al., 1987), and visual acuity in central vision is consequently far superior to that in peripheral vision. The beneficial effect of small pupils on visual acuity is therefore most useful for central vision; for peripheral vision, where visual acuity is limited by the properties of the retina rather than by the focus of the lens, the benefit of small pupils is likely marginal.
A similar argument applies to the finding that the pupil is smaller when people engage in exploitation behavior, as compared to exploration behavior (Section 4; Aston- Jones & Cohen, 2005;Jepma & Nieuwenhuis, 2011), and that the pupil is smaller when the level of arousal level is intermediate, as compared to high (Section 2; e.g. Bradley, Miccoli, Escrig, & Lang, 2008). (Low arousal is associated with drowsiness.) Exploitation and intermediate arousal are characterized by a narrow focus on a single task, and thus by a narrow, central focus of attention, whereas exploration and high arousal are characterized by distractability, and thus by a broader, peripheral focus of attention. Therefore, the effect of attentional breadth on pupil size, the effect of exploration/ exploitation on pupil size, and the effect of arousal on pupil size may all reflect the same basic principle: Small pupils are most advantageous for (attention to) central vision, whereas large pupils are most advantageous for (attention to) peripheral vision.
As reviewed in Section 4, the pupil constricts in preparation of an eye movement towards a bright stimulus (Ebitz et al., 2014;Mathôt et al., 2015b) . Here, the benefit may be one of timing: by 6. A theory of sensory tuning preparing a pupil constriction along with an eye movement, the latency of the PLR effectively decreases by about 100 ms, as compared to the response to a light stimulus during passive viewing.
This latency decrease may allow the visual system to adapt more rapidly to the changes in brightness that occur across eye movements, as gaze shifts from dark to bright objects and back again (Mathôt & Van der Stigchel, 2015).
The effects of attention towards (Section 2; Binda et al., 2013;Mathôt et al., 2013;Naber et al., 2013;Unsworth & Robison, 2017), and visual working memory (VWM) of (Husta et al., 2019;Zokaei et al., 2019), bright and dark stimuli may be an indirect form of preparation, and in this sense be related to the effect of eye-movement preparation. Specifically, a covert shift of attention towards a bright object is in some ways (Craighero & Rizzolatti, 2005;Rizzolatti, Riggio, Dascola, & Umiltá, 1987), though likely not all ways (Casteau & Smith, 2019), similar to programming an eye movement to that object without actually executing that movement. Similarly, when you keep a stimulus in VWM, you generally do this with the intention to interact with that stimulus; for example, you may keep a visual representation of a white shirt in VWM to search for that shirt in your closet. The exact relationship between eye-movement preparation, visual attention, and VWM is still debated (Casteau & Smith, 2019;Xu, 2017), but for the present purpose the key observation is that they can all be characterized as an intention to act; this may explain why these cognitive processes are all accompanied by pupil constriction when the target stimulus is bright, and pupil dilation when the target stimulus is dark.
Despite the fact that many cognitive effects on pupil size can be understood as subtle forms of sensory tuning, some effects still remain mysterious. For example, pupil dilation in response to increased memory load (Kahneman & Beatty, 1966) , listening effort (Zekveld, Kramer, & Festen, 2010), or cognitive load more generally (Just & Carpenter, 1993) does not serve any obvious function. Perhaps these kinds of cognitive effects are indeed epiphenomenal, possibly resulting from overlap with other cognitive processes for which pupil dilation is functional.
To summarize, two key observations may explain many, though not all, cognitively driven pupil responses. First, an emphasis on central vision over peripheral vision results in pupil constriction, and this matches the fact that central vision benefits most from the increased visual acuity provided by small pupils. Second, an intention to act with bright stimuli results in pupil constriction, and this reduces the latency of the PLR when that bright stimulus is subsequently brought into view.
Together, this pattern suggests that cognitively driven pupil responses improve the quality of vision in subtle ways that meet the demands of the situation; that is, cognitively driven pupil responses, like their reflexive counterparts, are a form of sensory tuning.
Sensory tuning in the auditory modality 6. A theory of sensory tuning So far, I have focused on vision. However, I propose that sensory tuning is a general principle of sensation and perception that also affects other sensory modalities, such as hearing.
The stapedius (or accoustic) reflex is a contraction of the muscles of the middle ear. This reflex reduces auditory sensitivity to low-pitch sounds, while leaving sensitivity to high-pitch sounds mostly intact (Borg, 1968), and is triggered by loud sounds and the act of speaking (Borg & Zakrisson, 1975). The stapedius reflex is a form of sensory tuning that optimizes the auditory sense depending on the situation. For example, speaking generates low-frequency vibrations that propagate through the skull, and that overpower the mid-range frequencies of actual speech. By contracting the middle ear, thus filtering out low-frequency vibrations, the stapedius reflex makes it easier to hear yourself speaking (Borg & Zakrisson, 1975). But in other situations, when there are no irrelevant lowfrequency sounds, the inner ear relaxes, thus allowing you to also hear low frequencies.

Predictions based on a theory of sensory tuning
There are many degrees of freedom when assigning functions to pupil responses: It is easy to come up with convincing post-hoc explanations for why the pupil responds when and as it does. A stronger test of a theory of sensory tuning is to make (and test) predictions about how the pupil should respond in so-far untested situations.
A first prediction follows from the hypothesis that cognitively driven pupil dilation reflects an emphasis on peripheral vision, at the expense of central vision. Central vision is severely impaired for people who suffer from macular degeneration (Ferris, 1983), and most people with this condition develop a strategy where they consistently use a specific part of their peripheral vision (the so-called preferred retinal locus, or PRL) as a stand-in for their impaired central vision (Fletcher & Schuchard, 1997).
Crucially, for people who suffer from macular degeneration, there is no clear-cut distinction between central and peripheral vision. Consequently, I predict that people who suffer from macular degeneration should show markedly reduced cognitively driven pupil dilation in situations where nonvisually impaired people do show such dilation, for example when switching from an exploitation to an exploration mode of behavior.
A second prediction follows from the finding that the pupil constricts when maintaining a bright stimulus in visual working memory (VWM; Husta et al., 2019;Zokaei et al., 2019) , and the hypothesis that this reflects an indirect intention to act upon that stimulus. A prominent notion in the field of VWM is that VWM items can be in different states (reviewed in Olivers, Peters, Houtkamp, & Roelfsema, 2011;Wolff, Jochim, Akyürek, & Stokes, 2017;Zokaei, Ning, Manohar, Feredoes, & Husain, 2014). Items that are likely to be acted upon in the immediate future are kept in a so-called active or prioritized state; the number of items that can simultaneously be in an active state may be more limited than the capacity of VWM, although the extreme view that only a single item can be 6. A theory of sensory tuning active at a time (Olivers et al., 2011) is likely too restrictive (Zhou, Lorist, & Mathôt, 2019). In contrast to this active state, items that should be remembered, but are unlikely to be acted upon in the immediate future, are kept in a so-called silent or accessory state. Crucially, I predict that the brightness of VWM items should only affect pupil size when these items are in an active state. This prediction can be tested by having participants memorize bright and dark items, and using various techniques to control the state of VWM items (reviewed in Zokaei et al., 2014) .
A third prediction follows from the hypothesis that sensory tuning is a general principle that applies also to the auditory modality. As described above, the stapedius reflex reduces or increases sensitivity to low-frequency sounds, depending on the situation (Borg, 1968). However, it is still an open question whether the stapedius reflex is modulated by cognitive factors, in the same way that the pupil light response is. (Although Jones, Greene, & Ahroon (2019) recently showed that the stapedius reflex is likely not susceptible to classical conditioning.) This could be tested in an experiment in which participants hear two simultaneous streams of sounds, one with a high pitch, and one with a low pitch. Participants would attend to one of the streams. Crucially, I predict that the middle ear should contract when participants attend to the high-pitch stream, as compared to when participants attend to the low-pitch stream.
To summarize, the theory of sensory tuning that I have put forward in this article leads to falsifiable predictions, three of which I have outlined above.

Conclusion
Sensation is often described as a passive response of receptors to external stimulation. According to this view, cognitive processes affect sensory processing only at a later, perceptual stage, when processes such as attention and prediction shape how sensory information is processed (Cavanagh, 2011). The key tenet of this review is that the classic dichotomy between sensation and perception is far too restrictive, and that cognition affects sensation at every stage; our senses are active organs with substantial freedom of movement, and we continuously tune our senses to meet the demands of the current situation and the immediate future.
In this review, I have focused on changes in pupil size as one specific form of sensory tuning. I have reviewed the various ways in which pupil size is affected by sensory input (Section 2), by cognitive processes (Sections 3 and 4), and by the interaction between them. I have proposed that cognitive processes that are characterized by an emphasis on central vision over peripheral vision are accompanied by pupil constriction (Section 6), because central vision benefits from the high visual acuity offered by small pupils, whereas peripheral vision does not, or hardly (Section 5). In addition, cognitive processes that are characterized by an intention to interact with bright stimuli are accompanied by preparatory pupil constriction, so as to reduce the latency of the pupil light response 7. Conclusion 20 of 27 when that bright stimulus is subsequently brought into view (Section 6).
Although the focus of this review has been on vision, I have proposed that sensory tuning is a general principle that applies to all sensory modalities. As one example from the auditory modality, I have discussed the stapedius (acoustic) reflex (Section 6; Borg, 1968). The stapedius reflex is a contraction of muscles in the middle ear, which results in a reduced sensitivy to low-frequency sounds. This allows the ear to be selectively sensitive to low-frequency or high-frequency sounds, as the situation demands.
To summarize, our eyes are controlled by fourteen muscles that move and shape the eyes so that we can sense light in different ways that are optimized for different situations. Pupil responses are but one fascinating example of such sensory tuning.