The Biological Complexity Monster
For about a decade I have seen the same pattern repeat itself. A circuit designer builds a Spike-Timing-Dependant-Plasticty (STDP) neuron. They justify their efforts because STDP is popular and by saying “that is what the brain does”. This is correct–STDP does occur in the brain. The problem is that neurons do much more than STDP. STDP, as you will see in the article, is the tip of the proverbial iceberg. Many take a route of piling on complexity, thinking that if only they could simulate enough of the biological complexity, that perhaps something important might pop out. What level of biological realism will it take to make a useful brain-inspired-computer? The answer is that biological realism has nothing to do with utility. Biology may inspire, but it does not select. The purpose is to make useful technology. Useful means that people will use it to solve their problems. This is why Knowm Inc. is focused on primary performance benchmarking rather than biological mimicry. When we choose method A over method B, it is because method A performs quantitatively better across a suite of benchmark learning tasks.
To put STDP into context, I offer below a (highly) simplified glimpse into the eyes of the biological complexity monster we call ‘neural plasticity’. Every person in the field of ‘Brain-Inspired Computing’ needs to stare this nightmare directly in the face. If you are not turned into stone, paralyzed by the endless complexity, I hope you walk away with a more carefully considered guiding principle than strict biological mimicry. Perhaps you may even start to see some unifying physical principles.
This material was gathered as part of a document that Hal Greenwald and I prepared for the DARPA SyNAPSE program back in 2010.
1: Canonical STDP is the tip of the iceberg
Canonical STDP is that the strength of the participating synapse is increased (LTP) or decreased (LTD) via precise temporal correlations. However, it may be more accurate to view this process in another way. Positive and negative timing correlations leads to the activation of NMDA receptors. This in turn leads to a cascade of events in the post-synaptic and pre-synaptic neuron which can effect potentially all functional properties of a neuron. Of particular interest are the numerous retrograde effects and the separate “LTP” and “LTD” pathways leading to almost opposite effects.
Figure 1: A simplified composite sketch of plasticity effects in the pre- and post-synaptic neurons following NMDA receptor activation. The activation of NMDA receptors can lead to a cascade of effects on the pre and post-synaptic cell, including the release of neurotrophic factors (NT). NF’s, in turn, are implicated in a vast number of cellular functions including and extending vastly beyond synaptic plasticity.
2: At least five basic forms of activity-dependent plasticity exist
Whereas modification to the synapse is generally regarded as a post-synaptic property, the whole picture is vastly more complicated and includes activity-dependent modifications to:
- Pre-synaptic neurotransmitter release probability
- Post-synaptic strength (“potency”)
- Axonal restructuring
- Dendritic restructuring
- Neuronal excitability
It is worth consideration that the five properties mentioned constitute a large proportion of the functional properties available to a neuron. It is possible that all functional properties of a neuron are under activity-dependent control.
3: Introduction to Long Term Potentiation (LTP)
LTP depends on increases in calcium ion concentrations, which are mediated by NMDA receptors. NMDA receptors are voltage-gated glutamate receptors; opening these ion channels requires both glutamate to bind to its receptors and sufficient depolarization to enable the removal of a Mg2+ ion that gates the channel. For this reason, they have been viewed as coincidence/correlation detectors for mediating LTP. Early LTP lasts 1-3 hours; extending this to a day or longer requires protein synthesis and gene transcription. LTP leads to an increase in the number of AMPA receptors at synapses and modifies their properties via phosphorylation. Such structural changes lead to enlarged synapses.
4: Long Term Depression (LTD) is different from and an essential complement to LTP
The problem with only having potentiation is that it is inherently unstable; correlated activity leads to increased synaptic strength, which leads to even more correlated activity. Stent (1973) modified Hebb’s postulate to say that when a neuron is not consistently active when a postsynaptic neuron generates an action potential, the synapse is weakened. However, it is not the case that LTD simply occurs in the absence of potentiation or as a result of renormalization of synaptic weights when other competing synapses undergo potentiation. It was once thought that the effects of monocular deprivation in kittens could be explained by such renormalization; however, blocking action potentials by injecting action potentials into the deprived eye prevented LTD. Low-frequency stimulation (typically 0.5-5 Hz for 15 minutes, but it depends on the history of recent activity) leads to NMDA activation, which increases post-synaptic Ca2+ concentrations; preventing the increased [Ca2+] prevents LTD. Internal calcium stores may also be relevant when NMDA receptors are not optimally stimulated, although these could also cause LTD to spread to neighboring synapses. LTP can also be mediated by metabotropic glutamate receptors and endocannabinoid receptors.
5: Forms of activity-dependent plasticity seem to be positively correlated
Temporally correlated spikes have been show to induced LTP and LTD (STP-LTP and STP-LTD) through multiple pathways.
NMDA receptor activation leading to an increase in post-synaptic potency also induces the synthesis and release of neurotropic factors, particularly brain derived neurotrophic factor (BDNF). BDNF has been shown to increase the pre-synaptic release probability and is transported retrogradely causing the stabilization of axon branches and the induction of LTP at the pre-synaptic neurons input synapses. BDNF treatment promotes the maturation of presynaptic terminals in dissociated neuronal culture and increases the number of synapses per axon terminal in vivo. Increase of post-synaptic Ca2+, caused by NMDAR activation via STDP and tetanus induction protocols is correlated with dendrite branch stabilization via the expression of CaMKII, as well as in increase in pre-synaptic neuron excitability. NMDAR activation leading to increased neural excitability can occur in the absence of LTP, suggesting separate downstream mechanisms. The immediate changes of intrinsic excitability points to the existence of rapid cytoplasmic signaling mechanisms that cause global modulation of ion channels inthe neuronal membrane.
Similar and opposite mechanisms exist for the LTD pathway. Contingent on NMDAR activation and weak post-synaptic Ca2+ concentration, a decrease in post-synaptic synaptic strength occurs, which is correlated with a decrease in pre-synaptic release probability. Endocannabinoids released postsynaptically act on presynaptically located endocannabinoid receptors and briefly reduce neurotransmitter release. Endocannabinoids have also been shown to decrease the excitability of pre- synaptic neurons. Again contingent on NMDAR activation, post-synaptic release of NO can trigger a retrograde transport mechanism that leads to LTD of input synapses of the pre-synaptic neuron. NO can disrupt axon growth and act as a retrograde signal to increase neuronal apoptosis (cell suicide).
To generalize all of the above findings, it could be said that STD-LTP (STD-LTD) causes the post- synaptic dendrite to stabilize (destabilize?), the pre-synaptic neuron to increase (decrease) both its excitability and transmission probability, as well as causing LTP (LTD) in the pre-synaptic neurons, stabilizing (destabilizing?) the pre-synaptic axons and increasing (decreasing) the neurons chances of survival.
6: Homeostatic and Meta Plasticity
Homeostatic plasticity, or synaptic scaling, modifies all of a neuron’s synapses in response to prolonged changes in activity. Synaptic scaling is separate from LTP and LTD mechanisms because it does not depend on NMDA receptors. Metaplasticity (“plasticity of plasticity”) also serves to preserve network stability by changing plasticity thresholds so that LTP and LTD become easier or harder to induce over time depending on recent activity. According to this idea, how stimulating a synapse will change its strength cannot be predicted unless its prior history is known. Intrinsic plasticity is another stability mechanism that operates at the neuron level rather than the synapse level and involves changes in the number, properties, and distributions of ion channels. Intrinsic plasticity can alter plasticity thresholds and can affect spike timing in addition to spike rate. In vivo, spontaneous activity can reverse STDP-induced synaptic changes, which could explain why visual adaptation disappears quickly in humans.
7: Influences of single spikes on plasticity depend on short-term changes in the dynamic state of the synapse.
A single synapse can be potentiated or depressed depending on its recent history. Prolonged high frequency presynaptic stimulation or low frequency presynaptic stimulation with postsynaptic depolarization induce LTP. Prolonged low-frequency presynaptic stimulation induces LTD. Pairedpulses separated by less than 20 ms typically produce depression, possibly due to sodium channel inactivation or because the vesicles at the presynaptic terminal are depleted. Paired pulses separated by 20-500 ms produce potentiation, possibly because of leftover calcium. When neurons in Layer 2/3 of visual cortex were stimulated with complex spike trains formed of two spike pairs, the first pair had a greater influence on the sign and magnitude of subsequent changes; modeling the interactions between spikes can improve predictions of subsequent synaptic modifications. Synapses with a high probability of vesicle release tend to depress, whereas depressed synapses tend to increase their likelihood of firing. Glia may also play a role by controlling the rate at which neurotransmitters are cleared from the synaptic cleft and by releasing additional modulators to regulate neurotransmitter release. Within a single neuron, synapses can have different forms of plasticity, and knowing which synapses have released vesicles may be more informative than knowing that the neuron has fired an action potential; for this reason, Abbott & Regehr (2004) proposed the use of transmission-triggered averages based on individual synapses rather than spike-triggered averages. A low probability of neurotransmitter release can make a synapse act like a high-pass temporal filter, and a high probability of initial release can make it act like a lowpass filter because subsequent stimulation yields decreased responses (due to adaptation).
8: Axon pruning results from active competition and requires pre and post-synaptic activation
Studies of the developing visual cortex show that axonal pruning is the result of a competitive process that requires pre-synaptic activation. When TTX was injected into the deprived eye of kittens during monocular deprivation experiments, the normal depression response was absent. This experiment has been confirmed with the mouse.
Following the formation of initial synaptic contacts between motoneurons and target muscles, the initiation of synaptic transmission activates the muscle and results in overt embryonic movements. Chronic blockade of this activity during the cell death period with specific drugs or toxins that cause paralysis prevents the death of all motoneurons.
Experiments with the neuromuscular junction illustrate clearly a competition for post-synaptic activation. If portions of the post-synaptic cell is prevented from firing, the axons innervating will prune. However, if all the post-synaptic receptors (the whole cell) is blocked, the axons do not prune. This indicates that the post-synaptic cell actually has to fire for a pruning event to take place. An axon’s terminals on one post-synaptic cell cannot compete against themselves but rather serve as a competitive unit vying against the cartels of other axons. In both neurons and glia, the final outcome of this competitive process is the survival of optimal numbers of cells for innervation or myelinations.
9: The flow of nerve growth factors (NGF) from axon terminals to cell body is necessary and sufficient to support distal axon extension
Neuronal cell bodies survive when only their terminals are treated with NGF indicating that target derived NGF available only to neurites can generate and retrogradely transport the signaling required for cell body. Importantly, if NGF is removed from the distal axons but left in the medium bathing the cell bodies, the distal axons retract and degenerate, but the neurons survives. Therefore, NGF signaling initiated at distal axons is both necessary and sufficient to support distal axon extension, whereas NGF signaling in cell bodies alone cannot support distal axon extension. NGF’s are also implicated in LTP.
10: Neurotrophic factors are heavily implicated in synaptic plasticity (and almost everything else)
NT receptors are present on both post- and presynaptic sites of the synaptic cleft. In-vitro studies have shown that NT’s, and in particular BDNF, have rapid effects on synaptic transmission. Numerous studies have confirmed that NTFs have important functions in synaptic plasticity, meaning that these synaptotophins regulate the functional strength of synaptic transmission including LTP. Studies have shown NT production during synaptic plasticity events. NT, in turn, influence substantial aspects of neural function, as can be seen from the chart below.
Figure 2: Non survival Function of Neurotrophic Factors. Red indicates the trophic factor has been implicated in anterograde transport, blue for retrograde transport. Green indicated that the respective function has been linked to the neurotrophin. Composite generated From Squire, et. al, P. 461-462
11: Dendrites are plastic, they can learn, and are capable of contextual logic
In addition to changing the mean firing rate, regenerative electrical activity in dendrites can modify the temporal structure of firing patterns of neurons. These changes of the shape of the f/I curve can be viewed as computations performed by the neuron. Shifts of the f/I curve can be interpreted as additions or subtractions, changes of its slope constitute multiplications or divisions, and changes in the firing pattern in response to coincident distal and proximal inputs can be viewed as a logical AND operation.
Dendritic electrical properties can change in an activity-dependent manner with a time course of milliseconds up to hours and perhaps even days, which means there exist dendritic as well as synaptic learning rules. Dendritic spikes in oblique apical dendrites of hippocampal pyramidal cells are highly plastic and subject to neuromodulation, providing a novel mechanism for memory
Dendritic spikes evoked in distal apical dendrites of hippocampal pyramidal neurons by synaptic stimulation generally fail to propagate to the soma. However, when synapses that are more proximal are also activated, the depolarization enables forward propagation towards the soma. This is especially interesting as the two location are innervated by functionally different inputs: a direct input from the entorhinal cortex to the distal apical dendrites and the Schaffer collateral input to the more proximal apical regions.
Network activity in different functional states (theta activity versus sharp waves) determines the mode of dendritic integration, thus enabling or disabling the propagation of dendritic spikes to the soma.
local modulatory synaptic input, which may up- or down-regulate excitability in different dendritic branches, may set different rules for plasticity in different dendritic compartments.
12: Dendritic spikes can trigger synaptic plasticity
Even if dendritic spikes do not forward propagate reliably and therefore fail to contribute directly to axonal output, they can trigger local synaptic plasticity and dendritic release of neurotransmitters.
13: Plasticity can be synapse specific
Plasticity rules are clearly not exclusively determined by dendrites but also by mechanisms residing in the synapse itself. For example, excitatory inputs of different origin that converge onto neighboring dendritic locations of lateral amygdala projection neurons have different forms of plasticity. Inputs from the thalamus readily undergo typical STDP, whereas cortical afferent synapses do not.Connections originating from the same somatosensory cortical L2/3 pyramidal neuronal type have distinct plasticity rules and mechanisms depending on the interneuronal type they target
14: Critical periods are induced by experience and can be prolonged
In cats that are raised in the dark until well past the end of the critical period, cortical neurons continue to be driven binocularly and their responses remain weak. When these animals are finally allowed visual experience, the responses of theses neurons gradually increase in strength and the relative representation of the left and right eyes is shaped by the animals experience. In monocularly deprived cats, the non-deprived eye inputs become predominant. Thus, without experience-driven input, the mechanisms that control ocular representation in the cortex remain in an uncommitted state, waiting for instructions for a prolonged, if not indefinite, period of time.
15: Axonal and dendritic remodeling is linked to synaptic plasticity
Imaging studies, both in culture and in vivo indicate that when filopodia contact potential synaptic partners, they can lose their motility and become stabilized. These contacts are then thought to transform into synaptic structures.
The assembly of synapses occurs very rapidly. The time required from initial contact to establishment of a functional synapse is only in the range of 1-2 hours, with the formation of a presynaptic terminal occurring in only 10-20 minutes and the recruitment of post-synaptic neurotransmitter receptors lagging somewhat behind.
Long-term imaging experiments have shown that in naive adult mice a subset of axonal branches in the cortex can undergo structural rearrangements over lengths of tens of micrometers over several days.
As these retractions and elongations are associated with turnover of numerous boutons, they are likely to be associated with synapse formation and elimination. Axons from layer 6 pyramidal cells are rich in small terminaux boutons. Terminaux boutons appear and disappear at higher rates, with more than 50% turnover in a month.
It has been shown that presynaptic activity and glutamate can trigger spine growth. New protrusions seem to preferentially grow towards axonal boutons with active synapses, often ignoring other potential presynaptic elements in the direct vicinity of the dendrite. Glutamate receptor antagonists decrease arbor growth and cause a corresponding decrease in arbor dynamics. These data suggest an intimate association between synaptic transmission, branch dynamics and dendritic arbor development. These observations suggest that glutamate may act as a trophic factor to guide new spines to active boutons. However, it is likely that numerous other cell-autonomous and secreted molecules also contribute to guide new spine growth.
Under baseline conditions, spines that appear are likely to disappear again within a few days. Conversely, spines that persist for over a week are likely to persist for months. There is a relationship between spine stability and spine size; transient spines are typically small, whereas persistent spines are large. These observations suggest that new spines initiate a competition that leads to the pruning of one of the synapses at a later stage.
In a study to test the hypothesis that experience-dependent spine growth might contribute to response potentiation, a subset of whiskers from a mouse were trimmed in a chessboard fashion and structural plasticity was analyzed. Induction of plasticity was found to stabilize new spines (13–15%, compared with 5% under baseline conditions) on pyramidal neurons over 2–3 weeks after whisker trimming.
Training in a forelimb reaching task rapidly induced the growth of new dendritic spines in the motor cortex, which were preferentially stabilized by subsequent training sessions. Spine addition and subtraction, with synapse formation and elimination, are likely to contribute to experience-dependent rewiring of cortical circuits.
The growth and maintenance of dendritic filopodia on retinal ganglion cells is regulated by acetylcholine during the peak of cholinergic synapse formation, and subsequently by glutamate during the peak of glutamatergic synapse formation. Thus neurotransmission mediated by established synapses may promote the growth and maintenance of nearby filopodia and facilitate further synapse formation where functional synaptic contacts exist.
In the retinotectal systems of X. laevis and zebrafish larvae, glutamate receptor blockade markedly decreases the branch lifetime of retinal ganglion cell axons and tectal dendrites, indicating that synaptic transmission promotes branch stability.
Neural activity can modulate the organization of the local cytoskeleton to stabilize neuronal branches. Lohmann et al. Recorded local, spontaneous rises in calcium in retina explants. These rises depend on neurotransmission and calcium-induced calcium release (CICR) from intracellular stores in retinal ganglion cell dendrites, and they serve to maintain the retinal ganglion cell dendrites, as perturbing the spontaneous local calcium activity by pharmacological blockage of the CICR signaling pathway leads to dendrite retraction within a few minutes.
Dendrite retraction can be prevented by focal calcium uncaging, which raises intracellular calcium locally. The stabilizing effect of calcium is rapid and remains within the dendritic segments where the calcium level is elevated. Since LTP requires elevated post-synaptic calcium, it is reasonable to related LTP with dendritic stabilization.
Increased Calcium concentration brought on by NMDA receptor activation leads to expression of CaMKII. Due to its ability for autophosphorylation, CaMK activity can outlast the intracellular calcium transient that is needed to activate it. This property is important for the induction of synaptic plasticity. Pharmacological inhibition of CaMKII blocks the induction of long-term potentiation. Upon activation, CaMKII phosphorylates postsynaptic glutamate receptors and thus changes the electrical properties of the synapse. Wu and colleagues showed that CaMKII expression is necessary and sufficient for the transition of tectal cell dendrites from the immature, dynamic state into the mature, stabilized state. Blocking endogenous CaMKII either pharmacologically or by expressing an inhibitory peptide maintains neurons in their rapid growth phase, so that dendritic arbors grow larger than normal.
The relationship between synaptogenesis and axon development was recently examined by Alsina et al. by time-lapse imaging of X. laevis retinal ganglion cells. The authors observed marked synapse and axon branch remodeling over 24-h recording periods. In comparison to stable branches, branches that were eventually eliminated had, on average, fewer synapses per axon terminal, suggesting that synapse formation and stabilization leads to the stabilization of axon branches.
Segregation of retinal anglion cell axons from opposite eyes in doubly innervated X. laevis tectum is mainly accounted for by the preferential retraction of axon branches from territories dominated by the other eye, and NMDA receptor blockade reduces branch retraction, suggesting that NMDA receptor activation selectively destabilizes axon branches.
Newly formed dendrite branches are highly unstable, suggesting that BDNF promotes dendritic arbor growth by accelerating branch dynamics rather than by promoting branch stability. Thus activity- dependent neurotrophin signaling can coordinate the exploratory behavior of axon and dendrite branches to favor the formation of new branches where appropriate synaptic contacts have been established.
NMDAR antagonists significantly reduced branch dynamics, again indicating that arbor elaboration requires some sort of testing. Blocking AMPARs does not significantly reduce arbor growth rates or branch dynamics in young neurons.
A model that accommodates much of these data is that during developmental periods of synaptogenesis, synaptic activity increases the emergence of fine dendritic branches. Newly extended branches on axons and dendrites form synapses with only NMDARs. As these synapses mature due to LTP, AMPARs are recruited to the synaptic sites. Mature AMPAR-containing synapses stabilize the branches on which the synapses are located. Stabilized branches then add new branches. These branches in turn establish new NMDAR-only synapses, which are either stabilized through the addition of AMPARs or retracted.
16: Axonal and dendritic remodeling can be rapid and is linked to activity
Dendritic and axonal arbors follow the same pattern of arbor development: short branches are rapidly added and retracted from axon arbors with a lifetime of about 10 minutes.
Retinal ganglion cell axons in living Xenopus larvae add 150% and retract 137% of the initial branch tip number in 2 h. Their post-synaptic partners, the tectal neuron dendrites, add 180% and retract 124% of the initial branch tip number during the same period.
Dendritic filopodia have average lifetimes on the order of minutes. Widely thought to be potential synapse precursors, motile filopodia may have a crucial role in regulating the rate of synapse turnover during development.
Post-synaptic densities marked by green fluorescent protein-tagged PSD-95, a post-synaptic density protein, appear, disappear and move along the dendritic shafts within minutes of imaging in neonatal brain slices. In cultured hippocampal neurons, more than 20% of the post-synaptic densities turn over in a 24-h period.
17: LTP can be reversed quickly via spontaneous activity and stabilized with delayed bursts.
In a study of the retinotectal system of Xenopus tadpoles it was found that when neurons were allowed to fire spontaneously, the effects of LTP and LTD were rapidly lost. Bursting spikes triggered from random flashes on the retinae were particularly effective at reversing LTP and LTD. Because the bursting spikes triggered by random flashes are largely uncorrelated with the activity of the specific retinotectal connection under examination, the reversal of synaptic modification appeared to result largely from uncorrelated post synaptic spiking induced by other converging inputs on the tectal neuron.
Preventing the reversal due to spontaneous or background noise could be attained by spaced applications of LTP/LTD inductions. In contrast to the “massed” TBS induction (60 bursts), three episodes of TBS (20 bursts each) within the 20-min period counteracted the effect of spontaneous activity, resulting in persistent synaptic potentiation. Residual potentiation resulting from each TBS episode accumulated when the subsequent TBS episode was applied before the reversal was completed.
18: LTP and LTD can spread
The traditional Hebbian view of plasticity is that individual synapses are independent and that only those with correlated activity are modified. There is substantial evidence that other synapses within the same neural network are also modified. LTP and LTD can spread to neighboring synapses. LTP can spread laterally within presynaptic neurons and can backpropagate from postsynaptic neurons. Importantly, lateral or forward propagation from postsynaptic units has not been observed. LTD can spread to other synapses within the presynaptic neuron, can spread laterally to other inputs on the same postsynaptic cell, and can backpropagate from postsynaptic to presynaptic cells. Forward propagation of LTD from postsynaptic neurons has not been observed. When LTP was induced with tetanic stimulation, some unstimulated synapses showed evidence of depression (“heterosynaptic LTD”). The implications are that LTP and LTD can spread via long-range signaling through extracellular space and this spread is restricted to a retrograde direction.
The target cell-specific presynaptic lateral spread of LTP is reminiscent of the target cell-specific action of brain-derived neurotrophic factor (BDNF) found in these hippocampal cultures, where BDNF- induced potentiation of glutamatergic synaptic transmission was observed only for synapses made onto glutamatergic but not GABAergic neurons (91). Indeed, BDNF has been shown to mediate the retrograde spread of LTP in the retinotecal synapses of the tadpole.
19: Correlated activity modulates pre-synaptic excitability
The same spike-timing induction protocols that have been shown to induce LTP and LTD of the participating synapse has also been found to increase and decrease the intrinsic excitability of the pre- synaptic neuron. Similar to LTP/LTD induction, changes in the presynaptic excitability require activation of NMDARs and postsynaptic influx of Ca2, indicating that transsynaptic retrograde signaling is required. However, the changes in intrinsic excitability can occur in the absence of LTP, suggesting separate downstream mechanisms. The immediate changes of intrinsic excitability in many studies point to the existence of rapid cytoplasmic signaling mechanisms that cause global modulation of ion channels in the neuronal membrane.