Center for Neurobiology of Vision
858.453.4100 ext 1014 | office
Sergei Gepshtein is a scientist specializing in perceptual psychology and sensorimotor neuroscience. Much of his research dwells on the interface between two aspects of visual perception: the entry process called "early vision" and the constructive process called "perceptual organization."
Sergei studies computational principles and biological mechanisms underlying these processes, in particular how visual systems organize information for the perception of motion and change.
He also studies sensorimotor integration, which is a field of research concerned with how we use visual and haptic information to guide action. A key questions is how vision helps us to plan actions prospectively, for many steps ahead, in view of the dynamic nature of the environment, its varying risks and uncertainties.
As a scientist and a principal investigator at the Salk Institute for Biological Studies, he uses experimental and computational methods to characterize neuronal mechanisms of sensation, perception, and action.
As a member of the board of directors of the Academy of Neuroscience for Architecture, and as founding member of the Worldbuilding Institute (originally 5D Institute), Sergei is increasingly engaged in translational and applied research, which includes design and analysis of the built environment and sensory media.
In 2017, he founded the Center of Spatial Perception & Concrete Experience (SPaCE) at the University of Southern California’s School of Cinematic Arts, for which he serves as the director, with the goal to develop new forms of physical, immersive, and environmental “spatial media,” including augmented reality, virtual reality, and beyond.
As an educator, Sergei helps to develop and teach new "art-science" curricula connecting scientific and design disiplines at the University of Southern California (Los Angeles), the New School for Architecture and Design (San Diego), and the University of California (San Diego).
Eyewitness misidentification accounts for 70% of verified erroneous convictions. To address this alarming phenomenon, research has focused on factors that influence likelihood of correct identification, such as the manner in which a lineup is conducted. Traditional lineups rely on overt eyewitness responses that confound two covert factors: strength of recognition memory and the criterion for deciding what memory strength is sufficient for identification.
Here we describe a lineup that permits estimation of memory strength independent of decision criterion. Our procedure employs powerful techniques developed in studies of perception and memory: perceptual scaling and signal detection analysis. Using these tools, we scale memory strengths elicited by lineup faces, and quantify performance of a binary classifier tasked with distinguishing perpetrator from innocent suspect. This approach reveals structure of memory inaccessible using traditional lineups and renders accurate identifications uninfluenced by decision bias. The approach furthermore yields a quantitative index of individual eyewitness performance.
Gepshtein S, Wang Y, He F, Diep D, Albright TD (2020). A perceptual scaling approach to eyewitness identification. Nature Communications, in press.
"Besides its traditional reliance on the tacit knowledge of timeless practices of construction, architecture relies largely on theories and findings of other areas of research and knowledge, instead of possessing an independent theoretical foundation of its own. During the past decades, architecture has been viewed from various theoretical perspectives, provided by, for instance, psychology, psychoanalysis, structural linguistics and anthropology as well as deconstructionist and phenomenological philosophies, just to name a few." -Juhani Pallasmaa
Today this sweeping array of theories and perspectives is being expanded again by the disciplines allied under the loosely defined umbrella of neuroscience. Several families of concepts, paradigms, and methods from the neurosciences appear to be perfectly suitable for investigating what the architect would call the "human response to the built environment." These overlapping families include systems neuroscience and affective neuroscience, sensorimotor psychophysics and experimental phenomenology, to mention only some of the contenders. It is far from clear, however, which ideas will stick and what shape they will take in the new context. Just as it has happened to many prior imports to architecture, hard-won scientific knowledge may remain a foreign entity within the living body of architecture, supplying occasional metaphor and inspiring freewheeling speculation. Or the sciences may retain their natal rigor and invigorate architectural theory and practice by helping architects to test some old ideas and possibly rid their discipline of unbuttoned preconceptions, some of which had already been subjected to incisive analytical scrutiny...
Gepshtein S & Snider J (2019). Neuroscience for architecture: The evolving science of perceptual meaning. Proceedings of the National Academy of Sciences, USA 116 (29) 14404-14406.
Let us recall that the philosophical discipline of phenomenology and the scientific discipline of experimental phenomenology have a common origin. This origin is the descriptive psychology introduced by Franz Brentano and promulgated by the "Brentano circle" to the threshold of a new approach in the philosophy of mind and a new approach in empirical psychology. The aspiration of descriptive psychology was to develop a "science of mental phenomena." This aspiration became figurative in philosophical phenomenology and literal in experimental phenomenology, but the common origin left its mark on both disciplines. From the outset, their common goal was to investigate the human mind from an adamantly first-person perspective.
Let us also recall that the scientific study of the mind was the goal of another discipline that made no commitment to the first-person perspective. This discipline is sensory psychophysics, conceived just before Brentano's descriptive psychology and dedicated to investigating mental phenomena from the third-person perspective of natural science...
Gepshtein S (2019). On some paradoxes of current perceptual theories. In Bianchi I & Davies R (Eds.) Paolo Bozzi's Experimental Phenomenology, 514-527.
Cortical sensory neurons are characterized by selectivity to stimulation. This selectivity was originally viewed as a part of the fundamental "receptive field" characteristic of neurons. This view was later challenged by evidence that receptive fields are modulated by stimuli outside of the classical receptive field. Here we show that even this modified view of selectivity needs revision. We measured spatial frequency selectivity of neurons in cortical area MT of alert monkeys and found that their selectivity strongly depends on luminance contrast, shifting to higher spatial frequencies as contrast increases. The changes of preferred spatial frequency are large at low temporal frequency and they decrease monotonically as temporal frequency increases. That is, even interactions among basic stimulus dimensions of luminance contrast, spatial frequency and temporal frequency strongly influence neuronal selectivity. This dynamic nature of neuronal selectivity is inconsistent with the notion of stimulus preference as a stable characteristic of cortical neurons.
Pawar AS, Gepshtein S, Savel'ev S & Albright TD (2019). Mechanisms of spatiotemporal selectivity in cortical area MT. Neuron 101 (3), 514-527.
In the course of perceptual organization, incomplete optical stimulation can evoke the experience of complete objects with distinct perceptual identities. According to a well-known principle of perceptual organization, stimulus parts separated by shorter spatial distances are more likely to appear as parts of the same perceptual identity. Whereas this principle of proximity has been confirmed in many studies of perceptual grouping in static displays, we show that it does not generalize to perception of object identity in dynamic displays, where the parts are separated by spatial and temporal distances.
We use ambiguous displays which contain multiple moving parts and which can be perceived two ways: as two large objects that gradually change their size or as multiple smaller objects that rotate independent of one another. Grouping over long and short distances corresponds to the perception of the respectively large and small objects. We find that grouping over long distances is often preferred to grouping over short distances, against predictions of the proximity principle. Even though these effects are observed at high luminance contrast, we show that they are consistent with results obtained at the threshold of luminance contrast, in agreement with predictions of a theory of efficient motion measurement. This is evidence that the perception of object identity can be explained by a computational principle of neural economy rather than by the empirical principle of proximity.
Zharikova A, Gepshtein S & van Leeuwen C (2017). Paradoxical perception of object identity in visual motion. Vision Research, 136, 1-14.
Sensory systems adapt to environmental change. It has been argued that adaptation should have the effect of optimizing sensitivity to the new environment. Here we consider a framework in which this premise is made concrete using an economic normative theory of visual motion perception. In this framework, visual systems adapt to the environment by reallocating their limited neural resources. The allocation is optimal when uncertainties about different aspects of stimulation are balanced. This theory makes predictions about visual sensitivity as a function of environmental statistics. Adaptive optimization of the visual system should be manifested as a change in sensitivity for an observer and for the underlying motion-sensitive neurons. We review evidence supporting these predictions and examine effects of adaptation on the neuronal representation of visual motion.
Gepshtein S & Albright TD (2017). Adaptive Optimization of Visual Sensitivity. Journal of the Indian Institute of Science, 97 (4), 423-434.
The future is uncertain because some forthcoming events are unpredictable and also because our ability to foresee the myriad consequences of our own actions is limited. Here we studied how humans select actions under such extrinsic and intrinsic uncertainty, in view of an exponentially expanding number of prospects on a branching multivalued visual stimulus.
A triangular grid of disks of different sizes scrolled down a touchscreen at a variable speed. The larger disks represented larger rewards. The task was to maximize the cumulative reward by touching one disk at a time in a rapid sequence, forming an upward path across the grid, while every step along the path constrained the part of the grid accessible in the future. This task captured some of the complexity of natural behavior in the risky and dynamic world, where ongoing decisions alter the landscape of future rewards. By comparing human behavior with behavior of ideal actors, we identified the strategies used by humans in terms of how far into the future they looked (their "depth of computation") and how often they attempted to incorporate new information about the future rewards (their "recalculation period").
We found that, for a given task difficulty, humans traded off their depth of computation for the recalculation period. The form of this tradeoff was consistent with a complete, brute-force exploration of all possible paths up to a resource-limited finite depth. A step-by-step analysis of the human behavior revealed that participants took into account very fine distinctions between the future rewards and that they abstained from some simple heuristics in assessment of the alternative paths, such as seeking only the largest disks or avoiding the smaller disks. The participants preferred to reduce their depth of computation or increase the recalculation period rather than sacrifice the precision of computation.
Snider J, Lee D, Poizner H & Gepshtein S (2015). Prospective optimization with limited resources. PLoS Computational Biology, 11 (9): e1004501. doi:10.1371/journal.pcbi.1004501.
Visual adaptation is expected to improve visual performance in the new environment. The expectation has been contradicted by evidence that adaptation sometimes decreases sensitivity for the adapting stimuli, and sometimes it changes sensitivity for stimuli very different from the adapting ones. We hypothesize that this pattern of results can be explained by a process that optimizes sensitivity for many stimuli, rather than changing sensitivity only for those stimuli whose statistics have changed. To test this hypothesis, we measured visual sensitivity across a broad range of spatiotemporal modulations of luminance, while varying the distribution of stimulus speeds. The manipulation of stimulus statistics caused a large-scale reorganization of visual sensitivity, forming the orderly pattern of sensitivity gains and losses. This pattern is predicted by a theory of distribution of receptive field characteristics in the visual system.
Gepshtein S, Lesmes LA & Albright TD (2013). Sensory adaptation as optimal resource allocation. Proceedings of the National Academy of Sciences, USA 110 (11), 4368-4373.
Jurica P, Gepshtein S, Tyukin I & van Leeuwen C (2013). Sensory optimization by stochastic tuning. Psychological Review 120 (4) 798-816. doi: 10.1037/a0034192.
Individually, visual neurons are each selective for several aspects of stimulation, such as stimulus location, frequency content, and speed. Collectively, the neurons implement the visual system's preferential sensitivity to some stimuli over others, manifested in behavioral sensitivity functions. We ask how the individual neurons are coordinated to optimize visual sensitivity. We model synaptic plasticity in a generic neural circuit, and find that stochastic changes in strengths of synaptic connections entail fluctuations in parameters of neural receptive fields. The fluctuations correlate with uncertainty of sensory measurement in individual neurons: the higher the uncertainty the larger the amplitude of fluctuation. We show that this simple relationship is sufficient for the stochastic fluctuations to steer sensitivities of neurons toward a characteristic distribution, from which follows a sensitivity function observed in human psychophysics, and which is predicted by a theory of optimal allocation of receptive fields. The optimal allocation arises in our simulations without supervision or feedback about system performance and independently of coupling between neurons, making the system highly adaptive and sensitive to prevailing stimulation.
Gepshtein S (2010). Two psychologies of perception and the prospect of their synthesis. Philosophical Psychology 23 (2), 217-281.
Two traditions have had a great impact on the theoretical and experimental research of perception. One tradition is statistical, stretching from Fechner's enunciation of psychophysics in 1860 to the modern view of perception as statistical decision making. The other tradition is phenomenological, from Brentano's "empirical standpoint" of 1874 to the Gestalt movement and the modern work on perceptual organization. Each tradition has at its core a distinctive assumption about the indivisible constituents of perception: the just-noticeable differences of sensation in the tradition of Fechner vs. the phenomenological Gestalts in the tradition of Brentano. But some key results from the two traditions can be explained and connected using an approach that is neither statistical nor phenomenological. This approach rests on a basic property of any information exchange: a principle of measurement formulated in 1946 by Gabor as a part of his quantal theory of information. Here the indivisible components are units (quanta) of information that remain invariant under changes of precision of measurement. This approach helped to understand how sensory measurements are implemented by single neural cells. But recent analyses suggest that this approach has the power to explain larger-scale characteristics of sensory systems.
Nikolaev AR, Gepshtein S, Gong P & van Leeuwen C (2010). Duration of coherence intervals in electrical brain activity in perceptual organization. Cerebral Cortex 20 (2), 365-382.
We investigated the relationship between visual experience and temporal intervals of synchronized brain activity. Using high-density scalp electroencephalography, we examined how synchronized activity depends on visual stimulus information and on individual observer sensitivity. In a perceptual grouping task, we varied the ambiguity of visual stimuli and estimated observer sensitivity to this variation. We found that durations of synchronized activity in the beta frequency band were associated with both stimulus ambiguity and sensitivity: the lower the stimulus ambiguity and the higher individual observer sensitivity the longer were the episodes of synchronized activity. Durations of synchronized activity intervals followed an extreme value distribution, indicating that they were limited by the slowest mechanism among the multiple neural mechanisms engaged in the perceptual task. Because the degree of stimulus ambiguity is (inversely) related to the amount of stimulus information, the durations of synchronous episodes reflect the amount of stimulus information processed in the task. We therefore interpreted our results as evidence that the alternating episodes of desynchronized and synchronized electrical brain activity reflect, respectively, the processing of information within local regions and the transfer of information across regions.
Gepshtein S & Kubovy M (2007). The lawful perception of apparent motion. Journal of Vision 7 (8):9, 1-15.
Visual apparent motion is the experience of motion from the successive stimulation of separate spatial locations. How spatial and temporal distances interact to determine the strength of apparent motion has been controversial. Some studies report space-time coupling: If we increase spatial or temporal distance between successive stimuli, we must also increase the other distance between them to maintain a constant strength of apparent motion (Korte's third law of motion). Other studies report space-time tradeoff: If we increase one of these distances, we must decrease the other to maintain a constant strength of apparent motion. In this article, we resolve the controversy. Starting from a normative theory of motion measurement and data on human spatiotemporal sensitivity, we conjecture that both coupling and tradeoff should occur, but at different speeds. We confirm the prediction in two experiments, using suprathreshold multistable apparent-motion displays called motion lattices. Our results show a smooth transition between the tradeoff and coupling as a function of speed: Tradeoff occurs at low speeds and coupling occurs at high speeds. From our data, we reconstruct the suprathreshold equivalence contours that are analogous to isosensitivity contours obtained at the threshold of visibility.
Gepshtein S, Tyukin I & Kubovy M (2007). The economics of motion perception and invariants of visual sensitivity. Journal of Vision 7 (8):8, 1-18.
Neural systems face the challenge of optimizing their performance with limited resources, just as economic systems do. Here, we use tools of neoclassical economic theory to explore how a frugal visual system should use a limited number of neurons to optimize perception of motion. The theory prescribes that vision should allocate its resources to different conditions of stimulation according to the degree of balance between measurement uncertainties and stimulus uncertainties. We find that human vision approximately follows the optimal prescription. The equilibrium theory explains why human visual sensitivity is distributed the way it is and why qualitatively different regimes of apparent motion are observed at different speeds. The theory offers a new normative framework for understanding the mechanisms of visual sensitivity at the threshold of visibility and above the threshold and predicts large-scale changes in visual sensitivity in response to changes in the statistics of stimulation and system goals.
Trommershäuser J, Gepshtein S, Maloney LT, Landy MS & Banks MS (2005). Optimal compensation for changes in task relevant movement variability. Journal of Neuroscience 25 (31), 7169-7178.
Effective movement planning should take into account the consequences of possible errors in executing a planned movement. These errors can result from either sensory uncertainty or variability in movement planning and production. We examined the ability of humans to compensate for variability in sensory estimation and movement production under conditions in which variability is increased artificially by the experimenter. Subjects rapidly pointed at a target region that had an adjacent penalty region. Target and penalty hits yielded monetary rewards and losses. We manipulated the task-relevant variability by perturbing visual feedback of finger position during the movement. The feedback was shifted in a random direction with a random amplitude in each trial, causing an increase in the task-relevant variability. Subjects were unable to counteract this form of perturbation. Rewards and penalties were based on the perturbed, visually specified finger position. Subjects rapidly acquired an estimate of their new variability in <120 trials and adjusted their aim points accordingly. We compared subjects' performance to the performance of an optimal movement planner maximizing expected gain. Their performance was consistent with that expected from an optimal movement planner that perfectly compensated for externally imposed changes in task-relevant variability. When exposed to novel stimulus configurations, aim points shifted in the first trial without showing any detectable trend across trials. These results indicate that subjects are capable of changing their pointing strategy in the presence of externally imposed noise. Furthermore, they manage to update their estimate of task-relevant variability and to transfer this estimate to novel stimulus configurations.
Banks MS, Gepshtein S & Landy MS (2004). Why is spatial stereoresolution so low? Journal of Neuroscience 24 (9), 2077-2089.
Spatial stereoresolution (the finest detectable modulation of binocular disparity) is much poorer than luminance resolution (finest detectable luminance variation). In a series of psychophysical experiments, we examined four factors that could cause low stereoresolution: (1) the sampling properties of the stimulus, (2) the disparity gradient limit, (3) low-pass spatial filtering by mechanisms early in the visual process, and (4) the method by which binocular matches are computed. Our experimental results reveal the contributions of the first three factors. A theoretical analysis of binocular matching by interocular correlation reveals the contribution of the fourth: the highest attainable stereoresolution may be limited by (1) the smallest useful correlation window in the visual system, and (2) a matching process that estimates the disparity of image patches and assumes that disparity is constant across the patch. Both properties are observed in disparity-selective neurons in area V1 of the primate (Nienborg et al., 2004).
Gepshtein S & Banks MS (2003). Viewing geometry determines how vision and touch combine in size perception. Current Biology 13 (6), 483-488.
Vision and haptics have different limitations and advantages because they obtain information by different methods. If the brain combined information from the two senses optimally, it would rely more on the one providing more precise information for the current task. In this study, human observers judged the distance between two parallel surfaces in two within-modality experiments (vision-alone and haptics-alone) and in an intermodality experiment (vision and haptics together). We find that the combined size estimates are finer than it is possible with either vision or haptics alone. Indeed, the combined estimates approach statistical optimality.
Gepshtein S & Kubovy M (2000). The emergence of visual objects in space-time. Proceedings of the National Academy of Sciences, USA 97 (14), 8186-8191.
It is is natural to think that in perceiving dynamic scenes, vision takes a series of snapshots. Motion perception can ensue when the snapshots are different. The snapshot metaphor suggests two questions: (i) How does the visual system put together elements within each snapshot to form objects? This is the spatial grouping problem. (ii) When the snapshots are different, how does the visual system know which element in one snapshot corresponds to which element in the next? This is the temporal grouping problem. The snapshot metaphor is a caricature of the dominant model in the field (the sequential model) according to which spatial and temporal grouping are independent. The model we propose here is an interactive model, according to which the two grouping mechanisms are not separable.
Double-click the blue markers [+] for further detail.[+] Solid field of visibility: first model and test
Visual systems are unequally sensitive to different features of the world. One characteristic of the unequal sensitivity is the contrast sensitivity function (CSF) commonly measured with periodic stimuli, such as luminance gratings, viewed over a fixed distance. For a fixed viewing distance, the CSF describes how different amounts of luminance contrast are required to make the stimulus just visible for different spatial frequencies of luminance modulation in the stimulus. When the spatial frequency is fixed but the viewing distance is varied, a similar profile of contrast sensitivity is expected as a function of viewing distance.
Using this method, we predict the range of viewing distances over which the stimulus is visible in any direction. A complex optical environment that contains multiple patterns can be described in terms of multiple corresponding solid regions of visibility that may overlap or nest in one another. This structure is usefully summarized as a 'solid field of visibility' in which the value of visibility varies smoothly across location. This description constitutes a sensory counterpart of the light field, which is a radiometric description of the amount of light flowing in every direction through every point in space. We derived a simplified model of the solid field of visibility using an established characterization of human spatiotemporal CSF (Kelly, 1979).
We tested the model using large-scale robotics at the UCLA Architectural Robotics Laboratory. A large screen and a projector were mounted on the end arms of two computer-controlled industrial robots, capable of moving synchronously along linear tracks. An integrated custom-built suite of hardware and software coordinated the positions of robots and presented visual stimuli on the screen. Observers were seated between the tracks and performed the tasks of detection and direction discrimination by means of a wireless computer mouse. First, we confirmed that visibility of static stimuli was confined to the solid spatial regions predicted by the model. Second, the boundaries of visibility were tested for stimuli projected on moving screens. Visibility was found to change across location as predicted by the model, sometimes changing from zero to full visibility over a distance of half a meter, indicating a sharp boundary of the solid field of visibility. Such abrupt spatial transitions of visibility have immediate applications in design of immersive environments: fully virtual and mixed. Knowing the solid field of visibility of an environment allows one to selectively enlarge or diminish the solid regions of visibility for specific parts of the environment and thus control user engagement.
Gepshtein S, Lynn G and McDowell A | SfN 2016, Session Mechanisms of Color, Contrast, and Form Perception, Saturday Nov 12, 2016 1:00 PM - 5:00 PM
We developed a new task to identify the strategies employed by humans to plan risky actions for multiple future steps using incomplete and rapidly varying information. A triangular lattice of disks of different value scrolls down on a touch screen at different speeds. By touching disks in a rapid sequence, subjects choose an upward path through the lattice to maximize the cumulative reward. A path is a sequence of binary choices made by selecting between the two disks directly above the current disk.
We parameterize human strategies by depth of computation (d) and recalculation period (r). Depth d is the number of rows subjects use to evaluate potential paths. Period r is how often subjects reassess their planned behavior. For example, a subject may look ahead for two rows (d=2) and develop a plan of action. She may then reassess the plan after 1 step (r=1) or play out the whole plan (r=2). The lattices were optimized so different strategies led to paths in different directions, allowing to estimate (r, d) strategies from individual choices. We also tested whether subjects used simple heuristics, such as 'seek out the large' or 'avoid the small.'
In the initial study (Snider et al, 2015, PLoS Comp Biol, e1004501) subjects exchanged deeper computation for less frequent reassessment. Surprisingly, there was no evidence that subjects relied on heuristics or pruning. They behaved as if they exhaustively performed all the sums along an exponential number of paths. When time pressure increased, subjects chose to decrease d, but kept the exhaustive computations.
How can such computations be performed? We hypothesize that the parallel organization of the visual system enables the comprehensive evaluation of potential paths, taking advantage of the analogue representation of value by disk size. We can interfere with the visual computation by representing value by numerals that engage memory based serial cortical mechanisms. Indeed, the numerical task decreased depth d by about one row and forced subjects to rely on heuristics. In the fast numerical condition, they only aimed for the largest disks.
In a more rigorous approach, choice behavior is parameterized using such machine learning algorithms as Partially Observable Markov Decision Process. The data requirements for estimating parameters in such models are staggering. We are partnering with a game company to create a mobile version of the task that entertains subjects and increases user pools from dozens to perhaps millions, to satisfy the data hungry models. We present simulations of how we store data and analyze subjects' choice behavior at the scale of Big Data.
Snider J and Gepshtein S | SfN 2016, Nanosymposium Visual Cognition: Decision Making, Tuesday Nov 15, 2016, 8:00 AM - 12:00 PM
Full title: MT neurons have different tuning properties at contrast threshold and above
Background: Behavioral studies of visual perception often employ stimuli near the threshold of visibility (for example by measuring contrast sensitivity of humans and animals) and also well above the threshold. But the neural mechanisms underlying visual perception are most commonly studied using suprathreshold stimuli, under the assumption that many characteristics of neuronal responses at the threshold and above the threshold are similar to one another. Here we investigate neuronal behavior across the full range of luminance contrasts and ask whether cell tuning at the threshold can be accurately predicted from suprathreshold contrast measurements.
Methods: We measured responses of 80 cells in the middle temporal (MT) area of the visual cortex of one alert macaque monkey engaged in a fixation task. The stimuli were sinusoidal gratings at multiple contrasts (0.5-100%) at 5 spatial frequencies (SF) and 1-3 temporal frequencies (TF). For each cell, we obtained response functions and contrast sensitivity functions (CSF). We defined the response function as the firing rate measured at a fixed contrast for multiple SFs. We defined CSF as the contrast at which the firing rate was one standard deviation above the resting firing rate for multiple SFs. We compared the SF peaks of response functions for multiple contrasts with the SF peaks of contrast sensitivity within cells.
Results: A previous study concluded that response functions at 25 and 50% contrasts in anesthetized cats were an accurate predictor of the sensitivity function in MT cells (Movshon et al., 1978). At low contrasts, we found that peaks of CSF and response functions matched only in 25% of all cases and mostly at lower contrasts (below 15%). At high contrasts, the peak of the response function tended to drift toward low or high SF. We modeled frequency tuning of the network using a canonic inhibition-stabilized circuit (arXiv:1410.4237). At low stimulus contrasts, in the linear regime, such a circuit behaved as a linear filter intrinsically tuned to stimulus frequency. At high contrasts, in the nonlinear regime, the intrinsic tuning of the circuit changed and its frequency tuning shifted similar to that of MT neurons. Compared to the contrast normalization models, in which visual cortical computation are represented by banks of frequency-tuned linear filters with a separate stage of response nonlinearity (Carandini & Heeger, 2012), this model dynamically changes its tuning properties as a function of contrast.
Conclusions: MT neurons have different tuning properties at the threshold of luminance contrast and above the threshold. This behavior is predicted by a model of canonic cortical computation.
Pawar A, Gepshtein, S and Albright TD | SfN 2016, Session Visual Motion, Monday Nov 14, 2016 8:00 AM - 12:00 PM
Visual perception is adaptive: it depends on the previous visual stimulation. The adaptive change is mediated by synaptic plasticity of individual neural cells whose behavior is stochastic. Here we show how the stochastic activity of individual cells leads to stochastic updating of their tuning, and how these changes are sufficient to explain some previously puzzling results from behavioral and physiological studies of visual perception
Gepshtein S, Jurica P, Tyukin I, van Leeuwen C and Albright TD | early report: SfN 2012
The spatiotemporal contrast sensitivity function (the 'Kelly function') provides a broad summary of human visual sensitivity used in basic and clinical studies of vision. We ask which features of the Kelly function are invariant across tasks and measurement procedures. Knowing which aspects of sensitivity are invariant facilitates the comprehensive assessment of contrast sensitivity and the changes of sensitivity caused by adaptation or disease. We isolate those aspects of the Kelly function that do not vary across observers, tasks, and experimental procedures. This allows us to advance specific prescriptions for efficient estimation of sensitivity. In particular, we propose that the method of estimation ought to incorporate measurement of the width (or the slope) of the underlying psychometric functions, or no assumption should be made about the width.
Laddis PA, Albright TD, Gepshtein S
The spatiotemporal contrast sensitivity function describes visual sensitivity to moving or flickering gratings across the entire range of visible spatial and temporal frequencies of luminance modulation. In spite of its value for assessment of spatiotemporal vision, the long testing time required for assaying the entire function has often forced researchers to confine measurements to representative sections of spatiotemporal sensitivity: spatial, sampled at a fixed temporal frequency, or temporal, sampled at a fixed spatial frequency. Here we present a novel adaptive method that accelerates the measurement by using Bayesian adaptive inference from the information gained from multiple sections of the spatiotemporal function. The new procedure evaluates the expected gain of information about parameters of sensitivity within every section and selects the stimulus that maximizes the expected gain across several sections. We validated the new procedure in computational and psychophysical experiments. In a direction discrimination task, we used drifting grating stimuli that spanned a broad range of spatial (0.5-8 cycles/deg) and temporal frequencies (0.25-24 Hz) of luminance modulation. Within 300-500 trials (15-25 minutes of run time) the new procedure provided estimates of sensitivity at the accuracy of 10% and the precision of 0.2-0.3 decimal log units.
Lesmes LA, Gepshtein S, Lu Z-L, Albright TD
The symposium organized by Sergei Gepshtein (Salk Institute) and Alex McDowell (USC) celebrated the rapidly growing interaction between two communities: researchers engaged in the scientific study of human perception and action and the practitioners of interactive and immersive narrative media technologies. Leading scientists and artists discussed human behavior and conscious experience in face of physical, social, and imagined realities represented in purely virtual worlds, as well as in the 'mixed' worlds that interlace the physical and virtual realities. The symposium consisted of a series of sessions each featuring two speakers: a scientist and an artist or immersive-reality practitioner. The speakers first presented their approaches and then reviewed the existing and prospective links between their domains of expertise. Each session was followed by an extensive discussion. An on-line publication featuring footage from the event is forthcoming.
Gepshtein S and McDowell A | preview
Until very recently, research on perceptual organization has been primarily descriptive. The result was a taxonomy of phenomena with little attempt to identify underlying mechanisms or develop predictive models. The situation has changed in recent years. New experimental methods have been introduced to measure the organizational processes in vision and other sensory modalities, and new predictive computational theories have been developed. This Handbook is an organized survey of the many new approaches to the study of perceptual (mainly visual) organization with an emphasis on computational and mathematical approaches. With chapters written by leading authorities, the Handbook describes modern experimental and computational methods that not only contribute to deciphering the mechanisms of the classical phenomena of perceptual organization but also open new perspectives in what is sometimes called the neo-Gestalt approach to perception. The intended audience includes researchers in psychology, neural science, computer science, and philosophy as well as graduate and advanced undergraduate students in these fields.
Gepshtein S, Singh M, and Maloney LT | preview
|Solid field of sensitivity: perceptual structure of immersive space
Keynote at VI International Conference on Spatial Cognition | Rome, September 10, 2015
| WEAVING MERCURY: Art and Science in Perception
Livestream panel at FMX conference on animation, effects, games and transmedia | Stuttgart, May 5, 2015
|Seen and unseen: Could there ever be a "cinema without cuts"?
Scientific American Blogs | April 29, 2014
|How the movies of tomorrow will play with your mind
Pacific Standard | April 29, 2014
|The visual system as economist: neural resource allocation in visual adaptation
Medical Xpress | April 1, 2013
|Despite what you may think, your brain is a mathematical genius
ScienceNewsline | April 10, 2013
|Brain waves challenge area-specific view of brain activity [video 1 2]
KU Leuven | March 20, 2013