This PLOS Computational Biology neurotheory paper with Christopher Rozell and Ilya Nemenman was my first step into neuroscience and has a special place in my heart. For context, neurons come in two main types: excitatory and inhibitory. The interplay between these types of neurons shapes computation in the brain. Despite brain sizes varying by several orders of magnitude across species, the ratio of excitatory and inhibitory sub-populations (E:I ratio) remains relatively constant, and we don’t know why. This paper offers a potential answer to this puzzle. We use a computational model which simulates the visual brain (visual cortex) and which contains both excitatory and inhibitory neurons. We limit the total number of neurons (i.e., brain volume) available to the model and simulate it at different E:I ratios (including those that cannot be found in nature and thus experimentally studied) and track its performance across different measures that capture how well it encodes visual information and how much metabolic energy the neurons consume (i.e., ATP molecules). The performance of our model at different E:I ratios reveals an optimal E:I ratio, where performance is optimal concurrently for both performance measures (best encoding of visual information, lowest metabolic energy consumption). Let’s unpack what we did and what we found…
1. Background
The brain is small but energy hungry, accounting for $\approx$3% of our body weight but consuming $\approx$15-20% of all the energy we expend. Energy consumption is determined by the activity of neuron populations i.e., spikes emitted by neurons to transform information and transmit it to each other. Where does the brain spend all this energy? At a high level, doing things essential for our survival. For example, our eyes, ears, noses, skin etc. are instruments that collect an immense amount of sensory information about our environment, and the brain must process all this information into a form which can guide our behavior and ensure survival. With $\approx$50% of the brain’s activity and energy consumption estimated to involve sensory information processing, sensory neuroscience emerges as an undeniably compelling area of research.
Against this backdrop, the efficient coding hypothesis (Barlow, 1961) suggests that sensory areas of the brain develop a neural code capable of representing information efficiently i.e., with a minimal amount of neural activity. It also predicts that the neural code for sensory information processing should be adapted to naturalistic stimuli, or rather statistical regularities present in stimuli within naturalistic environments. Among sensory modalities, the neural code underlying vision remains an attractive unsolved puzzle for experimental and computational neuroscientists, even after over a century of inquiry.
Computational/Theoretical models are useful because their observable (mathematical) nuts and bolts can offer normative accounts about the inner workings (e.g., the neural code) of a system (e.g., the visual brain), that are consistent with experimental observations.
However, their true utility lies in allowing computational scientists/theorists to make predictions that may give rise to novel experiments that test the ground truth in richer ways than before, and in simulating system configurations that cannot be studied experimentally (e.g., if they don’t exist in nature). Computational/Theoretical models may lead experimental research (e.g., predictions about gravitational lensing made by the theory of relativity were tested several years after being made) or their development may be led by experimental results that are not normatively explained. Regardless of whether they lead or lag experimental research, the interplay between computational/theoretical and experimental science has accelerated progress in many fields, and visual neuroscience is no exception.
Computational/Theoretical models don’t necessarily attempt to capture every little aspect of the system whose behavior they model. Despite Their development is as much an art, guided by intuition, as it is a science guided by mathematical fits to experimental data.
Their development is often guided by intuition and experimental data
They connect experimental observations with a normative understanding (correct or not) of how the system works, using mathematics as the language to build such bri
eloped at the computational, algorithmic and implementational levels, as outlined by Marr.
The sparse coding model of V1, a computational model realizing the efficient coding hypothesis by codifying energy-efficiency of the neural code into the computational objective, was a hugly significant forward step for the computational understanding of how the visual system works. The most compelling feature of this model is that when it is shown natural images, its neurons learn to respond to the same kinds of basic shapes as real V1 neurons in the brain. Despite this compelling success in translating a conceptual hypothesis into a mathematical form, which behaves much like the real brain does, the model is limited in its algorithmic and implementation fidelity w.r.t biology
A model of visa *efficient coding hypothesis** One of the earliest computational models realizing this hypothesis was a model of visual processing in early visual cortex/area V1, proposed by Olshausen & Field in 1996. The sparse coding model of V1 incorporated energy-efficiency of the neural code as a prior in a Bayesian setting. Crucially, it showed that the
Something about Marr’s levels of implementation Computational, Algorithmic and Implementational. Make sure i don’t lean too much into Marr, since there are lots of critics. Link this article as a tempering note 1.2 Sparse coding models of visual cortex (area V1) (Olshausen & Field 1996)
1.3 LCA: A biophysically plausible circuit implementing sparse coding (Rozell 2008)
1.4 Excitatory and Inhibitory Neurons in the Brain: Biology & Modeling
1.5 Biological Plausibility of Sparse Coding Models of V1
2. Our Approach and Results
2.1 Limiting brain size with a volumetric constraint
2.2 How did we do? Measuring a computational model’s performance
2.3 What should happen in Biology according to our model?
2.4 What really happens in Biology?