Input Novelty as a Control Metric for Time Varying Linear Systems
This paper introduces a framework for quantitative characterization of the controllability of time-varying linear systems (or networks) in terms of input novelty. The motivation for such an approach comes from the study of biophysical sensory networks in the brain, wherein responsiveness to both energy and salience (or novelty) are presumably critical for mediating behavior and function. Here, we use an inner product to define the angular separation of the current input with respect to the past input history. Then, by constraining input energy, we define a non-convex optimal control problem to obtain the minimally novel input that effects a given state transfer. We provide analytical conditions for existence and uniqueness in continuous-time, as well as an explicit closed-form expression for the solution. In discrete time, we show that a relaxed convex optimization formulation provides the global optimal solution of the original non-convex problem. Finally, we show how the minimum novelty control can be used as a metric to study control properties of large scale recurrent neuronal networks and other complex linear systems. In particular, we highlight unique aspects of a system’s controllability that are captured through the novelty-based metric. The result suggests that a multifaceted approach, combining energy-based analysis with other specific metrics, may be useful for obtaining more complete controllability characterizations.
In its most basic form, the systems-theoretic notion of controllability carries a binary definition: a dynamical system either is, or is not, controllable, with respect to its exogenous inputs. Naturally, such a notion has the deficiency of not grading the ease or difficulty associated with particular control objectives. To obviate this issue, consistent research effort has been directed at the characterization of controllability using systems-theoretic metrics. Roughly, these metrics can be grouped into two categories
The latter, in particular, is a natural paradigm that is directly related to the Kalman rank condition and, specifically, the controllability gramian, used to ascertain the controllability of linear systems . Recently, energy-based controllability metrics have been successfully used in the emerging domain of network science to assess the putative controllability of large-scale networks with linear node and connection dynamics [6, 5]. However, for complex networks in general and for biological neuronal networks in particular, there are certain questions related to the overall system dynamics that are not fully captured through an energy-based metric alone.
We appeal, specifically, to the domain of neural coding and the dynamics of sensory networks in the brain. Consider the simple, prototypical ‘feedforward-type’ layered model of a sensory network shown in Figure 1, wherein sensory neurons are tuned to a high dimensional feature space (i.e., environmental variables from the sensory periphery; say, different molecules corresponding to tastes). Those sensory neurons impinge on a complex, interconnected, recurrent sensory network that performs intermediate transformations on the incoming activity, en route to higher brain areas.
One may put forth a supposition that the controllability of such a sensory network, with respect to the afferent input from the sensory neurons, is critical for facilitating perception and behavior. For instance, a network that requires an enormous excitation in order to effect any state trajectory would ostensibly be quite poor in mediating perception of small changes in the environment.
But as much as energy is important in such mediation, orientation, i.e., the alignment of an input with certain features, and novelty, the difference in orientation of an input from past inputs, may be equally so. Indeed, a weak, but highly novel input may be more easily perceived than an intense, but more familiar, stimulus . The ability to assess the responsiveness of neuronal networks to novelty – at a particular moment in time, relative to past inputs – has immediate implications in the analysis and control of physiological neuronal network dynamics in different behavioral and clinical regimes [10, 11, 12].
In this paper, we seek a graded quantification of controllability (or, if one prefers, reachability) of linear time-varying systems (networks) in terms of input novelty. In particular, we ask how responsive are the state (node) trajectories to inputs that differ in orientation from those that have previously been applied. Figure 2 illustrates this basic notion for a simple two-dimensional linear system with a three-dimensional input. A particular input drives the system from a point in the phase space at to an intermediate point at ; from this point emerge two trajectories, both of which reach a common endpoint; one minimizes input novelty (note the similarity between the input over and that over ), while the other minimizes energy.
We proceed to formalize this notion and develop its utility in control analysis. Our contributions are as follows:
We define the notion of input novelty through the use of an inner product evaluated in the input feature-space
We analytically derive, for continuous, linear time-varying systems, the minimum novelty control that effects a desired state transfer by formulating a non-convex optimization problem. The problem seeks the minimum angular separation, defined in terms of the inner product, required in order to create a desired change in the network trajectory, constrained by a fixed average input energy.
For discrete-time systems, we obtain the minimum novelty control through a provably exact convex relaxation of the original non-convex problem. The relaxed problem can be solved efficiently using well established convex programming solvers.
We use the minimally novel control to construct a metric which can be used to characterize large-scale linear systems or networks. Importantly, we demonstrate that this novelty-based metric provides a distinct characterization of controllability compared to energy-based control metrics. The result suggests that a broader, multifaceted set of metrics may be useful for completely characterizing the controllability of such systems.
The remainder of the paper is organized as follows. In Section II, we describe the mathematical notation used in this paper. In Section III, we introduce the inner product-based input novelty measure for continuous-time, linear time-varying systems and formulate a non-convex optimal control problem that minimizes this novelty under the constraint of a fixed average input energy. We establish the existence and the uniqueness of a global optimal solution of the control problem and derive a closed-form expression for the minimally novel input. In Section IV, we derive analogous results for discrete time linear time-varying systems with the additional result of exact relaxation of the non-convex novelty minimization problem. Finally, in Section V-A, we demonstrate the use of the minimally novel input as a control metric to systematically characterize large scale linear networks. The paper concludes with a summary and discussion of future work.
Ii Mathematical Notation
Most notation is standard and will be introduced as the results are developed. We use lower-case letters to represent scalars, boldface lower-case letters to represent vectors, capital letters to represent matrices. Exceptions are , , , and , which we represent as scalars. We use to denote the space of - dimensional vectors with their elements as real numbers. Similarly, we use and to denote the space of dimensional matrices with real and non-negative real entries, respectively. is the Euclidean norm of the vector . is the transpose of a vector . We use and to represent the value of at continuous time and discrete time respectively. , and are constant -dimensional vectors. is the inverse of a matrix and represents the element of .
Iii Continuous-Time, Linear Dynamical Networks
Iii-a Input Novelty based Controllability Metric
We consider a linear, time-varying system with dynamics of the form
Here represents the state of the system at time , describes the time-varying dynamics, is the input matrix, and is the input to the system. Without loss of generality, we say that (1) describes the time evolution of (time-varying) linear networks in the presence of external inputs.
We consider an input , , with total energy , i.e.
Further, we assume that drives from to subject to the dynamics (1). Here, is a constant and is the average per-time input energy. We introduce the inner product-based measure
to quantify the novelty of a subsequent input , (which drives from to ), relative to . For a fixed input energy, measures the change in orientation (thus novelty) of two consecutive inputs trajectories of equal lengths (in time).
It is readily evident that .
It follows from (3) that the novelty of relative to decreases as increases and is minimum when i.e. when , , for all .
Iii-B Minimum Novelty Problem
From the conceptual formulation introduced above, we can develop a control problem to design the minimally novel input such that a desired change in the state of the system can be achieved under the constraint of fixed energy subject to (1). Specifically, we formulate the following optimal control problem:
Here, the state-transition matrix is given by
with , where is the identity matrix.
It should be noted here that the constraint (6c) is obtained by integrating (1) with respect to over the period of . Immediately, we note that the quadratic equality constraint (6b) makes the optimization problem (6) non-convex. Furthermore, we note that our optimal control problem formulation (6) is different from the classical minimum effort problems [13, 14] where the -norm of control inputs is minimized under the constraints of explicit lower and upper bounds on the inputs.
Iii-C Analytical Results
We derive conditions for the existence of a unique global optimal solution of the non-convex optimization problem (6). Based on this, we provide a closed-form expression for the optimal .
Iii-C1 Existence of a Minimally Novel Input
A solution of the non-convex optimization problem (6) exists if
Here, 111For notational simplicity, dependence on the initial time (here, ) is implicit. is the usual controllability gramian at time  and is defined as
From the above remark, the intuition behind Theorem 1 is seemingly straightforward – for a minimally novel solution to exist, the input energy must at least exceed the minimum energy required to effect the state transfer. However, in establishing the closed-form solution for the minimally novel input, it is useful to complete a formal proof.
In differential form, we can write (11) as
Iii-C2 Uniqueness of the Minimally Novel Input
Thus, a unique optimal control input exists and is given by
The expression (22) is the closed form for the minimally novel input. We now proceed to state an important lemma that establishes the dependence of the minimum novelty control on only the initial, intermediate and terminal states.
Given , and , in (20) is independent of , .
This lemma will form the basis for our use of novelty as a control metric, as highlighted in Section V below.
Iii-C3 Euclidean - Inner Product Transformation
As noted in Remark 3, it is a notable consequence of our cost formulation that the problem can be exactly recast in terms of the Euclidean norm. Specifically, if we consider
as the cost function in (6a), we obtain the optimal solution as
Iii-D Average Novelty Measure and Results
The definition of the inner-product novelty measure (3) requires the inputs and to be of equal time length . To allow for characterization of the input novelty in cases where the time lengths of these inputs are different, we define an average novelty measure in the form
Iv Discrete-Time, Linear Dynamical Networks
Iv-a Input Novelty
We consider linear, time-varying discrete time systems with dynamics of the form
Here represents the state of the system at discrete time for , is the time-varying dynamic matrix, is the input matrix, and is the input to the system. We say that (30) describes the time evolution of discrete time linear networks in the presence of external inputs.
Let us assume an input , , with total energy , i.e.
Here, is a constant scaling factor. We assume that the sequence of inputs can drive from to , subject to the dynamics (30). We introduce the inner product
to measure the novelty of a subsequent input , , relative to , required in order to reach the state . In other words for a fixed input energy, measures the required directional change in inputs (thus novelty) to achieve a given state change in the state of the system. Here, is a constant scaling factor.
Iv-B Minimum Novelty Problem
Similar to the continuous-time case, we formulate the following optimal control problem to design the minimally novel input such that a desired change in the state of the system can be achieved under the constraint of fixed energy subject to the system dynamics (30):
We relax the non-convex problem by replacing the equality constraint (34b) with an inequality constraint. Thus, the relaxed problem becomes convex and by deriving the optimality conditions, we establish that the relaxation is exact, i.e. any solution of the non-convex optimization problem (34) is also the solution of the convex optimization problem.
Iv-C1 Convex Relaxation
We relax the equality constraint (34b) with the following inequality constraint
Iv-C2 Exact Relaxation
The result permits numerically efficient evaluation of the minimally novel input.
V Minimum Novelty as a Control Metric
We now return to the original motivation outlined in the introduction: to systematically characterize the responsiveness of linear networks to novelty. For instance, given systems and , identical in structure but different in dynamics, we may ask which requires more novelty in order to execute the same trajectory. In this sense, we can fashion the minimum novelty solutions analytically obtained above into a control (or, controllability) metric. To do so, we standardize the notions of initial, intermediate and terminal states. The space of terminal states can then be sampled systematically (e.g., along eigenvectors of the Controllability gramian), or via some other Monte-Carlo fashion, in order to gain an aggregate novelty-based metric.
V-A1 Rate-based Recurrent Neuronal Networks
We consider a recurrent network of neurons with linearized firing rate dynamics of the form 
Here, represents the firing rate of the neurons at time , is a diagonal matrix whose diagonal elements are the (positive) time constants of the neurons, defines the linear interaction among neurons in the network, is the input matrix, and is the afferent input. Since is invertible, (40) can be represented in the form of (1) by considering where is the identity matrix.
In this example, we consider a recurrent network of neurons where neurons are excitatory and are inhibitory. We choose the time constants (in milliseconds) of the neurons, i.e. the elements of the diagonal matrix , from a uniform distribution . The connectivity weights (in essence, a time constant for excitation/inhibition from the neuron to ) are drawn from a uniform distribution or , if neuron is excitatory or inhibitory, respectively. We assume that for , i.e. neurons do not directly excite/inhibit themselves. Assuming , we proceed to compute the minimally novel inputs required to make a desired directional change in the network firing rate using (8)-(10), (20)-(22).
To complete the example, we specify ms. The states and are specified to satisfy with , where in this particular case we specify . The average energy of the prior input (see (2)) and the average energy of the computed minimally novel input of (see (4)) are set to . The prior input is specified to be constant over the interval . Figure 3A illustrates the schematic of the example setup.
Figure 4 illustrates the outcome of the example for random realizations of the system. Each red dot on the figure depicts the novelty associated with the solution to (20)-(21), i.e., the minimum novelty. Note, again, that in this example, these inputs are constrained to have unit average energy. Each blue dot corresponds to the minimum energy solution (for the same and ). As a verification of our theoretical development, we note that the minimum energy solution consistently requires an injection of novelty (angular orientation) relative to the prior input and relative to the optimum.
V-B Complex Networks
For our second example, we demonstrate the utility of our novelty-based controllability metric for distinguishing linear complex networks based on topological properties. Moreover, we demonstrate that a novelty-based control metric provides a different description of network controllability than does an energy-based characterization.
Consider networks of excitatory neurons with dynamics of the form (40) where the structure of the weight matrix is defined by the adjacency matrix of either the undirected Barabási-Albert scale-free network (BA)  or the undirected Watts-Strogatz small-world network (WS)  with appropriate parameters. If the element of the network adjacency matrix is , we say that neuron is connected to the neuron .
For both types of networks, we construct the matrices and (see (40)) as follows. Given a degree distribution of the BA network, we construct full rank realizations of the adjacency matrix. It should be noted here that the total number of undirected edges in all realizations is the same. For each realization, we replace the nonzero elements of the adjacency matrix by a number drawn from a uniform distribution , thus yielding the matrix . We choose the time constants (in milliseconds) of the neurons (the elements of the diagonal matrix ) from a uniform distribution . An analagous procedure is followed to specify the WS networks. Since the development of a WS network does not involve the creation of new edges, but rather the ‘rewiring’ of existing connections, while a BA network is created by a growth algorithm, we adjust their respective parameters to obtain approximately the same number of edges in both networks.
Assuming , we proceed to compute the minimum directional change in inputs (i.e. minimally novel inputs) required to make a desired directional change in firing rates of neurons using (8)-(9), (19)-(21).
To complete the example, we again specify ms. The states and are specified to satisfy with , where in this particular case we specify . We set the intermediate state to . As a reminder, the prior input drives the network from to in time and the computed minimally novel input of drives the network from to in time . The average energy of (see (2)) and the average energy of (see (4)) are set to . Figure 3B illustrates the example setup. Note that from Lemma 1, it is sufficient to randomly sample the system parameters without explicitly sampling the space of prior inputs.
Figure 5A shows the minimum novelty associated with the solution of (20)-(21) as a function of the total number of edges in the BA network (blue) and the WS network (red). Here, is defined as the ratio of the total number of undirected edges in the network and the total possible number of undirected edges (i.e. ). At each , we show the mean and mean standard deviation over realizations of the network. As shown in this figure, (i) controlling both types of networks becomes progressively harder in terms of minimum novelty (more novelty is required) as these networks become more dense, and (ii) the WS network requires consistently less novelty to control.
Figure 5B shows the minimum energy required to effect the same state trajectories for the BA network (blue) and the WS network (red). As shown in this figure, (i) controlling both types of networks becomes progressively easier in terms of minimum energy (less energy is required) as these networks become more dense, and (ii) the WS network requires consistently less energy to control.
This example demonstrates how a novelty-based analysis can provide a different and important controllability characterization, complementing the use of minimum energy-based methods. Moreover, both metrics are consistent in distinguishing the BA network from the WS network.
In many natural systems, and especially in biological sensory networks, the responsiveness of the system to a given input is dependent on both the energy of the stimulus and also its salience or novelty relative to background and prior inputs. Conventional control analysis aggregates all past input history into the current state and evaluates controllability in terms of the forward trajectory. Here, we introduce a control analysis to explicitly quantify the extent of novelty, defined in terms of an inner product, required to effect a state transfer in a linear system. The analysis, and corresponding solution to the problem of finding minimally novel inputs, are used to establish a new type of characterization of system controllability. Through two illustrative examples, we show how such an analysis is distinct from energy-based metrics in describing the controllability of large-scale linear systems or networks. The results indicate that the analysis of such systems may benefit from a multidimensional metric in which energy, novelty and other control characteristics are components. In future work, we plan to use such an approach in the specific domain of sensory neuroscience to examine how various time-scales of dynamics, including neuroplasticity, impact the ability of networks to transform afferent input and facilitate higher information processing. Application in other domains, including finance and social network analysis, could similarly be conceived.
S. Ching Holds a Career Award at the Scientific Interface from the Burroughs-Wellcome Fund.
-  G. Hu and E. Davison, “A real radius measure for controllability,” in American Control Conference, 2001. Proceedings of the 2001, vol. 4, 2001, pp. 3144–3148. [Online]. Available: http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=946404
-  G. Hu and E. J. Davison, “Real controllability/stabilizability radius of lti systems,” IEEE Transactions on Automatic Control, vol. 49, no. 2, pp. 254–257, 2004.
-  D. Hinrichsen and A. Pritchard, Real and Complex Stability Radii: A Survey, ser. Institut für Dynamische Systeme, Universität Bremen. Inst. für Dynam. Systeme, 1989. [Online]. Available: http://books.google.com/books?id=EpcCHAAACAAJ
-  G. Yan, J. Ren, Y.-C. Lai, C.-H. Lai, and B. Li, “Controlling complex networks: How much energy is needed?” Phys. Rev. Lett., vol. 108, p. 218703, May 2012. [Online]. Available: http://link.aps.org/doi/10.1103/PhysRevLett.108.218703
-  F. Pasqualetti, S. Zampieri, and F. Bullo, “Controllability metrics, limitations and algorithms for complex networks,” IEEE Transactions on Control of Network Systems, vol. 1, no. 1, pp. 40–52, 2014.
-  F. Pasqualetti and S. Zampieri, “On the controllability of isotropic and anisotropic networks,” 2014. [Online]. Available: http://www.fabiopas.it/papers/FP-SZ-14.pdf
-  F. L. Cortesi, T. H. Summers, and J. Lygeros, “Submodularity of energy related controllability metrics,” ArXiv preprint arXiv:1403.6351, 2014.
-  H. Kwakernaak and R. Sivan, Linear optimal control systems. Wiley-Interscience, 1972.
-  J. Downar, A. P. Crawley, D. J. Mikulis, and K. D. Davis, “A cortical network sensitive to stimulus salience in a neutral behavioral context across multiple sensory modalities.” J Neurophysiol, vol. 87, no. 1, pp. 615–620, Jan 2002.
-  S. Ching and J. T. Ritt, “Control strategies for underactuated neural ensembles driven by optogenetic stimulation.” Front Neural Circuits, vol. 7, p. 54, 2013. [Online]. Available: http://dx.doi.org/10.3389/fncir.2013.00054
-  S. Ching, E. N. Brown, and M. A. Kramer, “Distributed control in a mean-field cortical network model: implications for seizure suppression.” Phys Rev E Stat Nonlin Soft Matter Phys, vol. 86, no. 2 Pt 1, p. 021920, Aug 2012.
-  K. Q. Lepage, S. Ching, and M. A. Kramer, “Inferring evoked brain connectivity through adaptive perturbation.” J Comput Neurosci, vol. 34, no. 2, pp. 303–318, Apr 2013. [Online]. Available: http://dx.doi.org/10.1007/s10827-012-0422-8
-  D. E. Kirk, Optimal Control Theory: An Introduction. Dover, 2004.
-  M. Nagahara, D. E. Quevedo, and D. Nesic, “Maximum hands-off control and optimality,” in 52nd IEEE Conference on Decision and Control, Florence, Italy, December 10-13, 2013, pp. 3825–3830.
-  P. Dayan and L. F. Abbott, Theoretical Neuroscience. The MIT Press, 2001.
-  A.-L. Barabási and R. Albert, “Emergence of scaling in random networks,” Science, vol. 286, pp. 509–512, October 1999.
-  D. J. Watts and S. H. Strogatz, “Collective dynamics of ’small-world’ networks,” Nature, vol. 393, pp. 440–442, 1998.