Existence and Wandering of Bumps in a Spiking Neural Network Model ∗

We study spatially localized states of a spiking neuronal network populated by a pulse coupled phase oscillator known as the lighthouse model. We show that in the limit of slow synaptic interactions in the continuum limit the dynamics reduce to those of the standard Amari model. For non-slow synaptic connections we are able to go beyond the standard firing rate analysis of localized solutions allowing us to explicitly construct a family of co-existing one-bump solutions, and then track bump width and firing pattern as a function of system parameters. We also present an analysis of the model on a discrete lattice. We show that multiple width bump states can co-exist and uncover a mechanism for bump wandering linked to the speed of synaptic processing. Moreover, beyond a wandering transition point we show that the bump undergoes an effective random walk with a diffusion coefficient that scales exponentially with the rate of synaptic processing and linearly with the lattice spacing.


Introduction.
A goal of theoretical neuroscience is to develop a tractable model of a spiking neuronal network.This must necessarily involve a single cell model, capable of generating spikes of activity (so-called action-potentials), that when connected into a synaptic network can generate the rich repertoire of behavior seen in a real nervous system.For all of the popular conductance-based single neurons models, and also the simpler integrate-and-fire (IF) variety, a full understanding of network dynamics has proved elusive.In essence this is because we have not yet developed an appropriate mathematical framework to understand the neurodynamics of spiking networks.To date progress in this area has been restricted to firing rate neural models [1,2,3,4], which cannot adequately capture known spike-train correlations, or the analysis of phase-locked states and instabilities of homogeneous steady states in spiking IF networks [5,6,7,8,9,10].
The lighthouse model of Haken is a candidate single neuron model that falls between spiking neuron models and firing rate descriptions (see [11] and references therein).We show that in the limit of slow synaptic interactions it reduces to the classic Wilson-Cowan and Amari firing rate models.For fast synaptic interactions, it displays some of the complex properties that have been observed in simulations of spiking neuronal networks.Importantly the lighthouse model is sufficiently simple that it may also be analyzed at the network level even for fast synaptic responses.Hence, a detailed investigation into the network dynamics of the lighthouse model may pave the way to the development of a specific soluble spiking neurodynamics.
With this in mind we turn our attention to spatially localized bumps of persistent activity, which have been linked to working memory (the temporary storage of information within the brain) [12,13,14].In many models of working memory, transient stimuli are encoded by feature-selective persistent neural activity.Such stimuli are imagined to induce the formation of a spatially localized bump of persistent activity which coexists with a stable uniform state (with low firing rate).The most popular mathematical formulations of such models assume long-range inhibition with local recurrent excitation and invoke a population level description in terms of a rate model (see for example [15,16]).Although interesting in their own right and s 6.-*/ed by a number of authors (surveyed in [17]), such models are only useful for describing systems with slow synaptic interactions.
The study of localized states in fully spiking network models with fast synaptic interactions has typically only been possible with the aid of numerical simulation.For example, the work of Laing and Chow [18] on IF networks shows a number of interesting behaviors that would be absent in a firing rate study.These include the observation that i) a bump solution can exist in a spiking network (co-existing with the quiescent state) provided the neurons fire asynchronously within the bump and ii) that a bump can lose stability with increasing rate of synaptic processing to a wandering bump or a traveling wave.Wandering was also observed in a model of working memory that had populations of excitatory and inhibitory spiking neurons [19].
It is precisely these observations that we revisit in this paper within the context of the lighthouse model of a spiking neural network.Not only does the lighthouse model support similar behavior as that seen in the IF network, it is sufficiently simple to admit to further mathematical analysis.In this regard it is a minimal model of a spiking neural network.Although to date only patterns of synchronized activity have been analyzed [11] it is clearly a model that may be studied further in more interesting scenarios, such as the one we now present.Importantly this allows us to uncover the mechanism that governs the transition from a stable to a wandering bump.
In the next section we introduce the lighthouse model and survey some of its known properties.For simplicity we first study the model in the continuum limit, which allows us to more easily show its correspondence with the Amari model in the limit of slow synaptic processing.Next we introduce the definitions of a bump, a stationary bump and an equiperiod bump so that we may carefully formulate appropriate existence conditions for bumps and wandering bumps.We then show how one may analyze spatially localized states of the continuum lighthouse model for non-slow synaptic processing and explicitly construct a family of spiking bump solutions.We follow the continuum description with a treatment of dynamics on a lattice, including bump wandering.Finally, we discuss natural extensions of the work in this paper.
2. The lighthouse model.The lighthouse model was introduced by Haken as a pulse coupled model of a spiking neural network whose behavior could be explicitly analyzed at the network level [20,21].Much is now known about phase-locked solutions and the effect of noise and delays on stability and is summarized in the book by Haken [11].As we will show, the lighthouse model exhibits a transition to wandering like the integrate-and-fire model but is easier to analyze.Here we briefly outline the lighthouse model before presenting our new results on bump solutions.
The lighthouse model is a hybrid between a phase-oscillator and a firing rate model.Indexing a lighthouse neuron with i ∈ Z, the dynamical description of a discrete network is cast as a system of ordinary differential equations for the phase variable with and H(•) is interpreted as a firing rate function, with threshold h.Haken uses the Naka-Rushton sigmoidal function (though many of the analytical results for the lighthouse model are obtained for a linear firing rate function).Throughout this paper we work with the Heaviside step function such that H(x) = 1 for x ≥ 0 and is zero otherwise.The quantity u i is recognized as the input to neuron i, mediated by the synaptic processing of a set of incoming spikes.The spike time T m j marks the mth firing of neuron j, given by the times for which θ j (t) = 1.Each time neuron i fires, a synaptic pulse η(t) (with η(t) = 0 for t < 0) is transmitted to other neurons j = 1, . . ., N with coupling weight w ij .We shall take the synaptic response to be normalized, such that ∞ 0 η(t)dt = 1.When the input to a neuron exceeds the threshold, h, the phase will advance towards threshold at a constant rate which we have arbitrarily set to one.We note that the activity of a given neuron need not be periodic.Depending on the nature of the inputs it could also be quiescent or fire irregularly.
We consider two versions of the lighthouse model.In the first version, which is the model originally proposed by Haken, whenever the input is below threshold or drops below threshold the phase immediately resets to zero.In this case the neuron can only fire if it receives input that remains above threshold for an entire period.In the second version the phase is not reset, except when the neuron fires.We will see that the two versions have different possible bump solutions and dynamics.In particular the instant reset version does not have wandering bumps.
We consider lateral inhibition-type synaptic coupling on a one dimensional lattice.The system has two length scales -the lattice spacing and the coupling footprint.We take the array length to be much longer than either of these two length scales.We can index the N neurons in the array by a distance via x = iδx, where δx is the lattice spacing.The continuum limit is given by N → ∞, δx → 0 with δxN a constant.To make the connection to the continuum limit transparent we rewrite the coupling function as w ij = w([i − j]δx)δx.A continuum version of the lighthouse model (along the whole real line) is obtained under the replacement (θ i , u i , T m i , w((i − j)δx)) → (θ(x, t), u(x, t), T m (x), w(x − y)) x ∈ R, and is governed by the dynamics with Here the function w(x) plays the role of the connectivity kernel, which we shall assume to be homogeneous so that w(x) = w(|x|).For concreteness we will use the explicit functions η(t) = α exp(−αt)H(t) and the "wizard hat" w(x) = A exp(−a|x|) − exp(−|x|).

2.1.
Connection to the Amari model.We first consider the continuum limit for slow synapses and show it is identical to the classical Amari model.Taking the time average of (2.4) gives where Ω(x, t) = M (x, t)/T is the firing rate and M (x, t) = T +t t m δ(s + T m (x))ds is the number of firing events at position x within a window T .
We next recognize that the firing rate is given by For slow synaptic interactions (such that η(t) is approximately constant over the timescale T ), then we have the condition u(x, t) = u(x, t) .For the specific choice η(t) = αe −αt H(t), (2.7) may be written in the form which is recognized as the well known Amari model [22].The equivalence between the lighthouse model and the Amari equation is only strictly true in the limit of α → 0 where u(x) is constant in time.For nonzero α we will see that the dynamics between the two models deviate substantially.
3. Bump solutions.For a threshold h > 0 neurons will not fire spontaneously.However, for a lateral-inhibition-type coupling function, a group of neurons could mutually excite each other to sustain activity.Indeed for this choice of interaction function the Amari model given by (2.8) is known to support spatially localized solutions, commonly referred to as bumps.Moreover, their stability is insensitive to variation in the synaptic rate parameter α.It is therefore of interest to ascertain whether the analog of such solutions can be found in the full spiking lighthouse network, and whether they are also stable to variations in α.As we shall see, the answers to these questions are "yes" and "no" respectively.
Definitions.We are interested in analyzing the existence and dynamics of localized pulses of activity or bumps.A bump is defined as a group of contiguous neurons that fire together.However, given the discontinuous nature of spiking events and that the bump location may change in time it is necessary to make this idea more precise with the following definitions.Definition 1.A bump at time t is the set of all contiguous neurons that fired in the time interval Definition 2. A stationary bump is a bump which persists in a single location for all time i.e. the same set of neurons continue to fire indefinitely.Definition 3.An equiperiod bump is a stationary bump for which the neurons in the bump all fire with the same rate.
On a lattice any of the above states are invariant to shifts by arbitrary lattice spacings.As we will demonstrate, the position or the width of the bump need not be constant in time.
We first consider conditions for the existence of a continuum equiperiod bump in the limit of infinitely slow decaying synapses (α → 0).In this case each neuron in the bump receives constant above threshold input and thus all neurons within the bump will fire with rate one.All neurons outside of the bump receive below threshold input and never fire.We suppose that a bump exists on the domain L = [x 1 , x 2 ].This implies that u(x, t) is above threshold on L and below threshold outside of L and equal to the threshold at x = x 1 and x 2 .From (2.4) where P (t) = m η(t − m) is a periodic function (with period one) and ψ(y) is the location dependent phase.If w(x) is symmetric, we only need to consider one edge.For infinitely slow synapses P (t) = 1.This leads to the Amari existence condition We note that in the Amari limit the bump solution is insensitive to the phases of the neurons.For a standard lateral-inhibition type coupling there are two bump solutions that arise in a saddle node bifurcation [3].Importantly, it is possible to construct a class of bump solutions for the full continuum lighthouse model, and thus take a step beyond the standard Amari style analysis of localized states.
3.1.Continuum bump solutions in the lighthouse model.For the original lighthouse model with instant phase reset whenever activity drops below threshold, an equiperiod bump is expected since neurons within the bump will fire with rate one and those outside of the bump will not fire.For the second version without the phase reset condition, phases may ride on long sub-threshold plateaus.In this case the generic solution is a spatially localized solution with an 'interior' and 'exterior' region.Within the interior, neurons fire with rate one.In the exterior, the firing rate falls off to zero as the bump edge is approached.However, the contribution from the exterior region only weakly perturbs the location of the inner edge, so that the solution in the absence of a phase reset may be regarded as a perturbation of the one with a phase reset.
Here we focus on the class of (symmetric) maximally firing one-bump solutions (with firing rate unity) for the instant reset model parameterized by such that u(x, t) ≥ h for x ∈ [x 1 , x 2 ] for all t, and u(x, t) < h otherwise.Here we take We are free to choose such a coordinate system by translational invariance of the continuum model.In this case u(x, t) is given by (3.1) with ψ(y) = β|y|.Introducing the Fourier transform P (k) = ∞ −∞ dtP (t)e −ikt means that we may write (3.1) as where Note that u(x, t) has period one (inherited from P (t)).It is also continuous if β is nonzero.For slow synapses P (t) may be regarded as a constant so that P (k) = 2πδ(k).In this case (3.4) becomes timeindependent, with u(x, t) = F (0, x).Moreover, the width of the bump is independent of β and is given implicitly by h = F (0, ∆/2).This solution is identical to the one bump solution describing the timeindependent localized state of the standard Amari model [3].In general, however, we proceed with the evaluation of (3.4) using the result that Substitution into (3.4)gives us the following Fourier series representation for u(x, t): To determine the width of a bump in a self-consistent fashion we must enforce the condition that at the boundaries x = x j , the infimum of u (over t) must equal the threshold h.Hence, the existence condition for a bump in the continuum lighthouse model is simply The bump width ∆ is then determined by where g(k) = F (k, ∆/2).Here t * denotes the value of t for which u(∆/2, t) has a minimum.The value of t * is calculated by differentiating (3.7) and then solving In some sense we may regard the second term on the right hand side of (3.9) as a correction to the standard Amari firing rate model description that takes into account the effects of non-slow synaptic processing.
For our choice of temporal and spatial kernels a short calculation gives η(k) = α/(α + ik), and given by (3.12).For u(x) ≥ 0.1, the neuron will fire with rate 1.
In the limit α → 0 the β-independent shape of a bump is given explicitly by where and ∆ satisfies h = ∆ 0 w(y)dy.Figure 3.1 shows an example of the shape of u(x) given by (3.12).This is the classic Amari bump.When u(x) > h, the neuron fires with rate 1.Of the two branches of solutions to ∆ = ∆(h), it is known that the branch with largest ∆ is stable for all α in the Amari model [3,23,24,25].To see the effects of a non-zero choice of α in the lighthouse model we need only to calculate remaining terms from the right hand side of (3.9).
We evaluate (3.9) and (3.10) numerically to determine the pair (∆, t * ) as a function of system parameters.An example of this is shown in Figure 3.2, where for a fixed (non-zero) value of α we plot ∆ = ∆(h) for the synchronous solution (β = 0).As for the standard Amari solution we see two branches of solutions with a saddle-node bifurcation with increasing h.Moreover, taking α as the bifurcation parameter shows that such solutions cannot exist if α is chosen too large, as once again there is a saddle-node bifurcation as seen in the middle panel of Figure 3.2.In the right hand panel of Figure 3.2 we effectively combine the information from the other two panels into an existence plot, such that synchronous solutions are found below the curve shown in the (α, h) plane.In Figure 3.3 we show a plot of how the solution with the largest width varies as a function of β.This theoretical curve is compared with direct numerical simulations of the instant rest lighthouse model and found to be in excellent agreement.
We see in Figure 3.3 that as β is increased there is a general trend towards increasing width with a weak oscillation imposed.We now explain this behavior.The function g(2πn) has a periodic modulation in β of fundamental period β = 2/∆.Hence, from (3.10) t * will oscillate as β is varied.From (3.9) we expect a similar modulation of ∆ = ∆(β).Note that when β = 2/∆ the firing times are asynchronous and the system may be said to be in a so-called splay state.The synchronous state (β = 0) has an infimum at t * = ∆, and hence all other non-synchronous states are such that t * β=0 > t * β =0 , at least avoiding resonances where 2β/∆ = n.Hence, from (3.7) inf t u(x, t * β=0 ) < inf t u(x, t * β =0 ).Since u(x, t) is a decreasing function of x (in the neighborhood of the bump edges where x = ±∆/2), the synchronous solution has the smallest width as a consequence of condition (3.8).As β increases the phases become An example of a spiking bump state is shown in Figure 3.4 with the left and right panels showing simulation results with and without phase reset.The interior and exterior regions are clearly visible in the no phase reset model.Moreover, simulations show that with increasing α bumps can lose stability and begin to wander in exactly the same fashion as seen in earlier work of Laing and Chow on IF networks [18].Examples of this will be presented in Sec.3.4.Direct simulation results are obtained by approximating the continuum model as a finite system sampled at a discrete set of grid points, i.e. a lattice model.We now proceed with a treatment of the lighthouse model dynamics on a lattice.
3.2.Bumps on a lattice: slow synapses.We first derive the existence conditions of a bump on a discrete lattice.Suppose neuron j fires periodically with period one and has a phase ψ j ∈ [0, 1), i.e.T m j = m + ψ j .The periodic synaptic output of neuron j is given by E j (t) = P (t − ψ j ).In the limit of infinitely slow synapses P (t − ψ j ) = 1.Without loss of generality we fix one end of the bump at neuron index 0.The bump is then specified by the set i ∈ {0, . . ., m} where and u i < h elsewhere.The phase does not appear in the existence condition so any configuration of phases is allowed.Thus a family of solutions satisfying condition (3.14) defines an equiperiod bump at a given location.To make this condition more explicit we introduce two more definitions: Definition 4. The edge of a contiguous set of firing neurons is a neuron in the set that has one neighbor in the set and one neighbor not in the set.Definition 5.The next edge of a set of contiguous of firing neurons is a neuron that is not in the set but has one neighbor that is an edge.
Thus the edge neurons i = 0, m must satisfy u 0 ≥ h, u m ≥ h and the next edge neurons i = −1, m + 1 must satisfy u −1 < h, u m+1 < h.Unlike the continuum version, the discrete bump must satisfy two independent conditions, i.e. the edge neuron is above threshold and the next edge neuron is below threshold.If w ij is symmetric, the bump will be symmetric and thus we only need to consider one edge.Following the Amari strategy, we construct two existence functions: ) A bump of width m will exist for any m for which φ ne m < h ≤ φ e m .In the continuum limit, the existence condition is φ ne m = φ e m = h.We now consider conditions on the coupling weight and threshold for which this existence condition can be satisfied.We note that if φ ne m < φ e m for some m then we can choose h to lie between the existence curves.This can be satisfied in a number of ways.One is if w i−1,j − w i,j < 0 for i < j, i.e. the weight function is monotonically decreasing in |i − j|.This can be proved by comparing the terms individually in the sums (3.15) and (3.16).We note that the weight function only needs to decrease monotonically for |i − j| ≤ m + 2. If w ij is positive at i = j and decreases in |i − j| then φ ne m and φ e m will increase until w ij becomes negative whereupon φ ne m and φ e m will begin to decrease.The existence curves will then be concave functions.Hence, if h is initially greater than the maximum of the existence curves, no bump solutions exist.Bump solutions will appear if h is lowered so that it intersects with the existence curves.In the continuum limit, if h is initially above the existence curve and then reduced, two bump states will arise from a saddle-node bifurcation.However, in the discrete case, a set of states can arise simultaneously or in succession.We summarize the conditions for the existence of a bump in Proposition 1. Existence functions for w ij = 2e −0.05|i−j| − e −0.01|i−j| .In this example, multiple bump states are possible for the large bump but not the small bump.
Proposition 1.A symmetric bump or set of bumps can exist for some threshold h > 0 if w ij is a) symmetric, b) maximal and strictly positive at i = j, c) is decreasing in |i − j| (for some large enough finite set around i = j).These are sufficient but not necessary conditions.j=m−k+1 w 0,j < 0 and w 0,0 + m j=m−k+2 w 0,j > 0. Thus k solutions are possible for the small bump set if the self-excitation to a neuron exceeds the excitation arriving from k neurons a bump width away.For the large bump set, multiple solutions are possible if self-excitation dominates the inhibition arriving from k neurons a bump width away.

Stability.
A given equiperiod bump with infinitely slow synaptic processing (α = 0) is degenerate to arbitrary phase arrangements.Hence, any perturbation of the phase will not affect a bump.Additionally, infinitesimally small perturbations to the input to any neuron will also have no effect since generically no neuron in the network has an input that is arbitrarily close to threshold.Thus all firing states in a lattice network are stable to small enough perturbations.Hence, the more relevant question is whether or not the bump states are attractors with a nontrivial basin of attraction.Consider a lateral-inhibition-like coupling function that yields a single large and small bump.In the Amari model, the large bump is stable and the small bump is not.As we have discussed, in the discrete model, both are stable to infinitesimal perturbations.In order to address the nonlinear stability question we need to address the full dynamics.
We examine the ensuing dynamics of threshold crossing perturbations to the edge and next edge neurons.Consider first the small bump.Here, the recurrent excitation between the neurons in the bump is just sufficient to sustain the bump.If the edge neuron is suppressed, the internal neurons will receive subthreshold input and cease firing.The bump will then collapse to the zero state.Conversely, if the next edge is induced to fire it will induce its neighbor and the opposite next edge to fire.Thus the set of firing neurons will spread.These dynamics can be directly deduced from Figure 3.5.If neuron m + 1 begins to fire, the input to neuron −1 will cross threshold and begin to fire.On the other hand if neuron m stops firing, the edge neuron 0 will stop firing as well.Now consider the large bump where the opposite edges inhibit each other.If the next edge neuron fires it suppresses the opposite edge neuron.If the edge neuron stops firing, inhibition is lessened on the opposite next edge and it is induced to begin firing.Thus, a large bump of width m is stable although it could shift location if given a strong enough perturbation.Dynamically, if too many neurons fire they will suppress each other and the bump will shrink.If too few neurons fire, they inhibition will be released and the silent neurons will be induced to fire.The mechanism for this stability is a discretized version of the continuum case first established by Amari [3].These observations will be relevant later when we consider wandering.It is important to note that stability and existence are inherently intertwined.Henceforth, when we refer to a stable neuron we imply nonlinear stability.
3.3.Bumps on a lattice: fast synapses.We now consider the case where the synaptic decay rate α is nonzero.Simulations on a discrete lattice show that a stable bump is possible for small enough α.However, if the synaptic decay rate α is increased sufficiently, a stationary bump will not exist.For the no phase reset version of the model, for large enough α, the location of the bump can shift in a seemingly random way which we call wandering.However, as we will discuss in Section 3.4, the instant reset version does not support wandering for large α.In the instant reset model, a neuron must receive continuous above threshold input to fire, so a finite set of neurons cannot maintain self-sustained firing if the synaptic decay rate is so fast that the sum total of the input to any neuron that exceeds threshold has a time duration less than one.As such, the instant reset model, either has a stationary equiperiod bump solution or the zero solution where no neuron fires.
Hence in this section we examine possible stationary solutions for the no phase reset version of the lighthouse model for nonzero synaptic decay rate.In this version, each time a neuron receives above threshold input its phase advances towards threshold.As shown in the proposition below, if α is extremely large then no self-sustaining bump can exist.However, there can be a range of α for which stationary and wandering bumps can exist.
Proposition 2. For a fixed coupling function and lattice spacing, there exists an α large enough so no self-sustained activity can exist in a finite set of neurons.Hence no bump can exist for large enough α.
We can prove this by considering a set of m neurons that form a self-sustaining set.Whenever a neuron receives suprathreshold input, its phase is advanced at rate one for the duration of time that the input exceeds threshold.Hence, suprathreshold input must total a time of one before the neuron will fire.The maximum rate a neuron can fire in the lighthouse model is one.To sustain activity in a set of neurons, the net output of the neurons in the set must provide sufficient input to maintain firing in all the neurons.The input to a neuron is the weighted sum of synaptic pulses from all neurons in the bump.For any fixed threshold, the synaptic decay rate α can be made large enough so that the suprathreshold duration for the synaptic input due to a given firing event can be made arbitrarily short.When the duration is so short that a neuron requires more than m firing event inputs to fire, then the network cannot sustain itself.
We now investigate the dynamics for α nonzero but not large enough so that no bump can exist.For α nonzero and all neurons firing periodically with rate one, the synaptic input to a neuron is no longer constant in time but is given by (3.17) where P (t) = P (t + 1).For our specific example P (t) = αe −αt /(1 − e −α ), 0 ≤ t < 1.The input function u i (t) now depends on the phases of the firing times of the neurons.As before, in order for a stationary equiperiod bump to exist, the input to the edge neuron must be entirely above threshold and the input to the next edge must be entirely below threshold.We can thus re-express the existence conditions in terms of the following generalized existence functions w −1,j P (t − ψ j ), (3.18) Again we require for an equiperiod bump.
We see for our specific choice of η(t) that as α increases, the maximum of P (t) increases and the minimum decreases.Thus for large enough α the existence conditions will be violated and no equiperiod bump can exist.We can show this more explicitly.When α increases from zero, u i (t) on the edge may not remain above h for an entire period.As an example consider a synchronous bump where all neurons fire together.First we define the sum If all the neurons in the bump are synchronized, we can shift the phase so that u i (t) = W m 0 P (t).On [0, 1), the minimum of P (t) is αe −α /(1 − e −α ) and the maximum is α/(1 − e −α ).The existence condition of an equiperiod bump will not be satisfied when Thus, for each fixed α, there is a critical lattice spacing, below which there is no equiperiod bump.Consider the difference d m = W m 0 − W m −1 .By translational invariance of w ij , d m = w 00 − w 0,m+1 .Recall that w ij = w(|i − j|δx)δx for a fixed function w(x).Consider keeping the bump width ∆ fixed in space while reducing δx.This implies that the number of neurons in the bump m must scale as m = ∆/δx.We can then write d m = (w(0) − w(∆ + δx))δx.As expected, d m scales as δx.Hence, as the lattice spacing decreases, W m 0 approaches W m −1 which implies that φ ne m (α) will approach φ e m (α).When the two existence curves meet, the equiperiod bump will no longer exist.
The crucial fact for the nonexistence of the equiperiod bump is that if the input function u i (t) is not constant then it will have a maximum and a minimum.The minimum of the edge must remain above threshold while the maximum of the next edge must remain below threshold for an equiperiod bump to exist.Figure 3.8 shows an example of u i (t) of the edge and next edge neurons of a bump situated above and below threshold respectively.As the lattice spacing decreases, the maximum of the next edge will approach the minimum of the edge and eventually violate the condition for the existence of an equiperiod bump.Hence, even if the neurons do not fire synchronously, the existence condition for α nonzero will still be violated for α large enough or δx small enough.For any phase arrangement ψ j , u i (t) will always have a time dependence on a finite lattice.The input function u i (t) will only be constant if P (t) is constant and P (t) is constant only if α = 0.
In fact, the input function u(t) cannot be constant even in the continuum.We can see this by considering (3.1).Clearly if ψ(y) is differentiable then u(x, t) is not a constant unless either P (t) or w(x) are constant.If ψ(y) is discontinuous on a countable set in the interval then the integral can be written as a sum of integrals over separate intervals between the discontinuous points.Each piece is not constant in time so the same argument that u(x, t) cannot be constant applies.We summarize our results (for the no phase reset lighthouse model) in the following propositions.Proposition 3.For a fixed lattice spacing δx, there exists a large enough α such that there is no equiperiod bump.Conversely, for a fixed α > 0, there exists a small enough δx so that there is no equiperiod bump.Proposition 4. A continuum equiperiod bump cannot exist for α > 0 if neither w(x) nor P (t) are constants.
Propositions 3 and 4 do not negate the possibility of a nonequiperiod bump.During the time that the input is suprathreshold, the phase of the neuron will advance at rate one.When the accumulated time of the intervals of suprathreshold input is equal to one, the neuron will reach threshold and fire.Thus we could have an α such that a set of interior neurons receive above threshold input while a set of adjacent neurons receive above threshold input intermittently.These neurons will advance towards the firing threshold at a rate slower than one.This could result in a bump solution with an interior region of neurons firing with rate one and an exterior region where the rate decreases to zero as the edge is approached.An example of a nonequiperiod bump is shown in Figure 3.4.
Determining the width of a nonequiperiod bump is difficult because the firing rate in the exterior region must be calculated explicitly from the neuronal dynamics.However, we can present an argument that nonequiperiod bumps can exist for α > 0 for a suitably chosen coupling kernel.We first introduce the following definition.Definition 6.The interior edge of a bump is the neuron nearest to the edge that is firing with period one.It divides the bump into an interior region where the firing rate is one and an exterior region where the firing rate is nonzero but not one.
Suppose we choose w ij so that a symmetric equiperiod bump with all neurons firing in phase on the set {0, . . ., m} exists for α = 0 so φ ne m < h ≤ φ e m .We want to show that there exists an α > 0 such that the original next edge neurons of this bump will fire and become the new edge of a nonequiperiod bump.The original edge becomes an interior edge.Hence we need to show that there exists an α such that 1) inf m+2 j=0 w 1,j P j (t − ψ j ) ≥ h, 2) sup m+2 j=0 w 0,j P j (t − ψ j ) > h, and 3) φ ne m+2 < h.Consider a stable large bump so that w ij is positive for |i − j| > n where n < m and negative elsewhere so neurons in the bump give a positive stimulus to their near neighbors.We also suppose that they give a weaker negative stimulus to the neurons on the opposite side of the bump.We can then choose h such that it is larger than φ e m+2 to satisfy condition 3).We note that for a neuron firing with period T , 1/T − α/2 ≤ P (t) ≤ 1/T + α/2.Thus, we can choose an α such that the original next edge begins to fire at an arbitrarily low rate which satisfies condition 2).Each time this neuron fires it gives a stimulus to each neuron in the network.For a symmetric bump, the two next edges fire synchronously so the net effect is a positive impulse on the original edge.Since the input is already above threshold this added input will not violate condition 1).The new next edge will also receive a positive stimulus.However, for a small enough α it can remain below threshold so that condition 3) is still not violated.Thus we have a nonequiperiod bump with m neurons in the interior and two neurons in the exterior.
3.4.Bump wandering states.It was found in simulations of the IF model that as the synaptic decay rate α was increased, the stationary bump state can destabilize and begin to wander [18].For even larger α traveling waves can arise and finally for fast enough α no sustained activity exists.In simulations of the no phase reset lighthouse model, we find that for increasing α there is a transition to wandering and for high enough α, a bump cannot be self-sustained.However, traveling waves are not observed in the lighthouse model.Figure 3.6 shows an example of wandering in a numerical simulation of the lighthouse model.
We can see how wandering may occur by considering the conditions for firing near the edge of the bump.For increased α, the firing rate of the edge neuron could decrease while the next edge could begin to receive suprathreshold input (for some of the time) and thus begin to fire.The next edge firing would then give inhibitory input to the opposite edge and may cause it to cease firing.In this case the bump could experience a state of frustration where there is no satisfactory equilibrium state to which it can settle.The consequence is that a set of contiguous firing neurons could exist but the set may not be of a fixed width or at a fixed location.The result could be a wandering set of contiguous neurons.
We have shown previously that for large enough α, the edge neurons will decrease their firing rate and next edge neurons will receive above threshold input and begin to fire.Recall that a bump is defined as the set of contiguous neurons that fired within the last window of period one.Given that the edge and next edge neurons fire at a slower rate than the interior neurons, the bump width for time intervals over this window will not be stationary.However, the bulk of the bump could still remain in place.If α is not too large, the neurons in the interior of the bump will receive above threshold input continuously and fire with period one.With this in mind, we make the concept of wandering more precise with the following definition.Definition 7. A bump is wandering if the position of the interior edge is neither time stationary nor periodic.
We argue that a wandering state can exist by first demonstrating that there is a nonstationary attractor state comprised of a contiguous set of firing neurons.As we discussed in Section 3.2.1 there exists an attractor for a set of bump widths.Consider a stable large bump of width m for α = 0 with existence functions of the generic form in Fig. 3.5.For such a bump, contiguous sets of firing neurons broader than m will result in strong inhibition to the edge neurons and the set will diminish.Conversely, for sets smaller than m, but not smaller than the small bump, inhibition will be reduced to the next edge neurons inducing them to fire and the set will expand.For α nonzero, the inputs to each neuron will no longer be constant in time but will remain bounded.However, for nonzero α there will be a range of α for which this attractor property will still hold.
To show that wandering can exist we must first establish the following proposition.
Proposition 5.The firing of the interior edge can be perturbed by the firing of a single exterior neuron and depends sensitively on the firing phase.
We first consider a nonequiperiod bump where the interior has more neurons than the exterior and the input from exterior neurons is positive to the adjacent interior edge but negative to the opposite interior edge.For small enough α, the input to the interior edge is bounded away from the threshold h.As α increases, two things can occur.One is that the infimum of the interior edge input will decrease.The second is that the maximum amplitude of inputs from exterior neurons will increase.When α is large enough such that the input from an exterior neuron is large enough to drop the the interior edge input below threshold for some temporal duration then the interior neuron's firing will be delayed and it will no longer fire with rate one.For a bump with a finite number of neurons, the infima of the interior edge is a finite set.Input from the exterior region will arrive as individual positive or negative exponentially decaying pulses.Hence, there should exist a range of α such that an exterior neuron can slow the firing rate of the interior edge only if it fires in a time window that overlaps with an interior edge infimum.This window can be made arbitrarily small by adjusting α.When the interior edge slows, it no longer has rate one and hence no longer belongs to the interior set.Conversely, a next edge neuron could be receiving input that is above threshold except at a few isolated locations around the infima.When these neurons receive a positive input that overlaps with an infimum, they can fire with rate one and thus become an interior edge.Hence, the location of the interior depends sensitively on the firing phase of the exterior neurons.
This also explains why there is no wandering for the instant reset lighthouse version.Here, when the input to an interior neuron drops below threshold for no matter how brief a time, the phase will be instantly reset to zero and thus the neuron will no longer fire.Additionally, while the next edge neurons receive intermittent above threshold input, their phases do not advance because they are reset each time the input drops below threshold.Hence, they can never fire.Thus infinitesimal perturbations can only shrink a bump so no wandering can occur.
Figure 3.7 shows the sensitivity to initial conditions for a bump that is initially near synchronous in a simulation.The bump was driven above threshold for the first 20 time units.The initial phases were randomized within a small neighborhood of zero phase (θ i (t = 0) < 0.01).In Figure 3.7 a), a stable bump is present.Note that the edge neurons fire at a slightly slower rate than the interior neurons and the next edge fires with a very long period.The initial conditions in b) differs from a) by randomizing the phases in a small neighborhood of the zero phase that is 0.1% larger but a wandering bump arises.Thus there is a neighborhood of initial phases around zero that tend to lead to a stationary bump.Initial conditions outside of this neighborhood lead to wandering.For voltages initialized to random values between 0 and 1, wandering would be generic.The crucial event that induced wandering occurs at time 30 when one of the next edges fires but the other does not.If both fire together then their effects on the interior edges essentially cancel out and the stable bump will persist.Figure 3.8 shows the input to the lower edge and next edge.In a), we can see that the inputs to the neurons just barely cross threshold but the bump can tolerate such excursions.However, in b) one of the crossings disrupts the bump and wandering ensues.Figure 3.9 shows the voltages of the two opposite next edges.In a) we see the voltages reach threshold together and thus fire together.However, in b), one of the next edges reaches threshold first and gives an inhibitory pulse to the opposite next edge delaying its firing.This breaking of symmetry allows wandering to take place.
The implication of Proposition 5 is that time stationary bumps do not exist for sufficiently large α since arbitrarily small rearrangements of the phases could change the input to the neurons and shift the interior edge.However, this does not show that the bumps will wander.It could breathe in and out at a given location or move back and forth periodically.What we need to show is that the bump can shift either left or right and the shift does not depend on history.Given that an interior edge can become  an exterior neuron and vice versa, we see that shifts could occur in either direction.For a stable large bump, the edges on opposite sides of the bump inhibit each other.Thus, if say the left inner edge moves left, inhibition will increase on the right edge and force it to move left as well.If the inner edge moves right, inhibition will be lessened on the right edge and it will move right as well.For the exponential synaptic function, if α > 1, the influence of a neuronal firing decays quickly beyond one period.Thus, past shifts should have little or no affect on future shifts.Hence, after each period, the bump is as likely to move left or right.Thus, wandering is equivalent to a random walk on a lattice.This implies that the position of the inner edge of a bump x will obey (x(t) − x(t − τ )) 2 = Dτ , with D ∝ δx 2 /T s where δx is the lattice spacing, and T s is the average time for a perturbation of a firing neuron to cause a right or leftward shift.The graph of (x(t) − x(t − τ )) 2 versus τ in Figure 3.10 for a direct numerical simulation of the lighthouse model verifies that that wandering is a random walk (i.e.diffusion process).
Figure 3.11 shows a plot of the diffusion constant for different δx.We see that it increases linearly in δx.We can see why this should be true by examining the four key quantities that govern wandering: 1) the amplitude of the input due to the firing of one neuron, 2) the distance to threshold for the interior edge and next interior edge neurons, 3) the average shift size of the bump, and 4) the average time step per shift.The amplitude of the input to location x from a single neuron at y is given by η(t)w(x − y)δx.The distances to threshold are given by d ne T = φ ne m (α) − h and d e T = φ e m (α) − h.For small δx, given the smoothness of u(x, t) in x, d ne T and d e T both should scale as δx.As δx decreases, the amplitude of an individual input from a neuron will decrease but so too will the distance to threshold.Hence, as the lattice spacing decreases, the threshold α for initiating wandering will not change appreciably.However, the minimum shift the interior edge can take also scales as δx.Thus, at the lattice length scale, wandering of the interior edge will take place regardless of the spacing.However, on the global length scale of the bump the wandering step size will decrease with δx.Finally, each step is potentially induced by the firing of a neuron within the bump.Therefore, it is reasonable to expect that the step time T s will scale with δx since the number of neurons in the bump scales as δx −1 .Hence, the result is that wandering decreases to zero as δx approaches zero.Figure 3.12 shows the dependence of the diffusion coefficient on α.Interestingly, the dependence seems to be D ∝ e kα , for some constant k.Thus D(α, δx) ∝ e kα δx.

Discussion.
In this paper we have re-visited the lighthouse model within the context of localized reverberatory activity that can underly the formation of attractor bump states, both stationary and non-stationary (wandering).In particular we have shown how this model can be viewed as a natural extension of the Amari model to include the effects of non-slow synaptic processing.Although, the model is clearly a highly simplified and abstract model of a neuron, it does possess network behavior that is reminiscent of more biological neuron models.
One prediction of our work, supported by direct numerical simulations, is the possibility that bump states with differing widths can coexist.This simple prediction would not be forthcoming in a purely firing rate model (that has no knowledge of underlying spiking patterns), and would seem to be a hard quantitative prediction to make from the analysis of integrate-and-fire networks.Moreover, its simplicity lends itself naturally to both continuum and lattice studies.Indeed by analyzing both we have been able to clarify the role of discreteness in contributing to emergent bump dynamics.In particular we have shown that multiple width bumps can coexist on a discrete lattice and that wandering bumps can coexist with stationary bump states.Moreover, by tracking the dynamics of bump edges we have seen a form of competition giving rise to an equal probability of a bump moving left or right, resulting in an effective random walk of the bump.Our studies suggest a diffusion coefficient that scales with the exponential of the rate of synaptic processing and linearly with the lattice spacing.Interestingly, our numerical simulations of the IF model (data not shown) also seems to exhibit similar scaling for the diffusion coefficient on α and δx.Although this result warrants further investigation, it does give further evidence that the lighthouse model can generate dynamics consistent with that of other popular spiking neural networks.
A number of natural extensions of the work presented here suggest themselves.One would be to study the dynamics of moving bumps.In contrast to an IF network the lighthouse model does not generate waves in response to a brief localized initial stimulus.To obtain a traveling bump in the first instance one would need to augment the lighthouse model to include a form of spike frequency adaptation, as in the work of Hansel and Sompolinsky [26] and Pinto and Ermentrout [23], or perhaps an asymmetry in the synaptic footprint, as in the work of Zhang et al. [27], or even an adaptive threshold, as in the work of Coombes and Owen [28].The persistence of solutions in the presence of noise is also of interest.We say so mainly because the addition of noise is known to reduce wave speed to turn a traveling bump into a stationary bump, so that in this sense noise can lead to stabilization [29].Another important study would be to treat heterogeneous networks, say with a distribution of natural single neuron frequencies and thresholds, i.e., by considering θi = ω i H(u i − h i ).Even for a globally coupled network this may shed light on how heterogeneity can often lead to the emergence of long time-scale oscillations, as in the network studies of Smolen et al. [30].Moreover, deviation from the lattice is completely equivalent to manipulating the coupling function w.The influence of this type of heterogeneity could certainly destroy bump solutions.Including them in the lighthouse model would open up a whole new avenue of research, that might initially be tackled using techniques developed in [31,32,33].

Fig. 3 . 2 . 5 ∆Fig. 3 . 3 .
Fig. 3.2.Left: Bump width as a function of h for the synchronous solution (β = 0) with α = 1.Middle: Bump width as a function of α for the synchronous solution with h = 0.1.Right: Existence window (below the curve) for the synchronous solution in the (α, h) plane obtained by the numerical continuation of the limit point in the left hand plot.In all plots A = a = 2.

Fig. 3 . 4 .
Fig. 3.4.A direct numerical simulation of the continuum model approximated by a finite system with lattice spacing δx = 0.004 on N = 1000 grid points.The dots in the raster plot indicate the times of firing events.Data is shown for the parameters of Figure 3.3 with β = 2.The left and right panels show results with and without phase reset respectively.
Fig.3.5.Existence functions for w ij = 2e −0.05|i−j| − e −0.01|i−j| .In this example, multiple bump states are possible for the large bump but not the small bump.

Figure 3 .
5 shows an example of the existence functions.We immediately can see that unlike the continuum version, there is a possibility that sets of "small" and "large" bump solutions can exist.Multiple bump solutions are possible if the slope of the existence function is not too steep.More precisely, for any m > 0 and k > 1 satisfying φ e m > φ e m−k and φ e m−k > φ ne m , there are k bump solutions with widths ranging from m − k to m for the small set of bumps.Using (3.15) and (3.16), these conditions become m j=m−k+1 w 0,j > 0 and w 0,0 − m+1 j=m−k+1 w 0,j > 0. For φ e m−k > φ e m and φ e m > φ ne m−k , there are k solutions with widths ranging from m − k to m for the large set of bumps.In terms of the coupling function, the conditions are m