Input Novelty as a Control Metric for Time Varying Linear Systems
Abstract
This paper introduces a framework for quantitative characterization of the controllability of timevarying linear systems (or networks) in terms of input novelty. The motivation for such an approach comes from the study of biophysical sensory networks in the brain, wherein responsiveness to both energy and salience (or novelty) are presumably critical for mediating behavior and function. Here, we use an inner product to define the angular separation of the current input with respect to the past input history. Then, by constraining input energy, we define a nonconvex optimal control problem to obtain the minimally novel input that effects a given state transfer. We provide analytical conditions for existence and uniqueness in continuoustime, as well as an explicit closedform expression for the solution. In discrete time, we show that a relaxed convex optimization formulation provides the global optimal solution of the original nonconvex problem. Finally, we show how the minimum novelty control can be used as a metric to study control properties of large scale recurrent neuronal networks and other complex linear systems. In particular, we highlight unique aspects of a system’s controllability that are captured through the noveltybased metric. The result suggests that a multifaceted approach, combining energybased analysis with other specific metrics, may be useful for obtaining more complete controllability characterizations.
I Introduction
In its most basic form, the systemstheoretic notion of controllability carries a binary definition: a dynamical system either is, or is not, controllable, with respect to its exogenous inputs. Naturally, such a notion has the deficiency of not grading the ease or difficulty associated with particular control objectives. To obviate this issue, consistent research effort has been directed at the characterization of controllability using systemstheoretic metrics. Roughly, these metrics can be grouped into two categories
The latter, in particular, is a natural paradigm that is directly related to the Kalman rank condition and, specifically, the controllability gramian, used to ascertain the controllability of linear systems [8]. Recently, energybased controllability metrics have been successfully used in the emerging domain of network science to assess the putative controllability of largescale networks with linear node and connection dynamics [6, 5]. However, for complex networks in general and for biological neuronal networks in particular, there are certain questions related to the overall system dynamics that are not fully captured through an energybased metric alone.
We appeal, specifically, to the domain of neural coding and the dynamics of sensory networks in the brain. Consider the simple, prototypical ‘feedforwardtype’ layered model of a sensory network shown in Figure 1, wherein sensory neurons are tuned to a high dimensional feature space (i.e., environmental variables from the sensory periphery; say, different molecules corresponding to tastes). Those sensory neurons impinge on a complex, interconnected, recurrent sensory network that performs intermediate transformations on the incoming activity, en route to higher brain areas.
One may put forth a supposition that the controllability of such a sensory network, with respect to the afferent input from the sensory neurons, is critical for facilitating perception and behavior. For instance, a network that requires an enormous excitation in order to effect any state trajectory would ostensibly be quite poor in mediating perception of small changes in the environment.
But as much as energy is important in such mediation, orientation, i.e., the alignment of an input with certain features, and novelty, the difference in orientation of an input from past inputs, may be equally so. Indeed, a weak, but highly novel input may be more easily perceived than an intense, but more familiar, stimulus [9]. The ability to assess the responsiveness of neuronal networks to novelty – at a particular moment in time, relative to past inputs – has immediate implications in the analysis and control of physiological neuronal network dynamics in different behavioral and clinical regimes [10, 11, 12].
In this paper, we seek a graded quantification of controllability (or, if one prefers, reachability) of linear timevarying systems (networks) in terms of input novelty. In particular, we ask how responsive are the state (node) trajectories to inputs that differ in orientation from those that have previously been applied. Figure 2 illustrates this basic notion for a simple twodimensional linear system with a threedimensional input. A particular input drives the system from a point in the phase space at to an intermediate point at ; from this point emerge two trajectories, both of which reach a common endpoint; one minimizes input novelty (note the similarity between the input over and that over ), while the other minimizes energy.
We proceed to formalize this notion and develop its utility in control analysis. Our contributions are as follows:

We define the notion of input novelty through the use of an inner product evaluated in the input featurespace

We analytically derive, for continuous, linear timevarying systems, the minimum novelty control that effects a desired state transfer by formulating a nonconvex optimization problem. The problem seeks the minimum angular separation, defined in terms of the inner product, required in order to create a desired change in the network trajectory, constrained by a fixed average input energy.

For discretetime systems, we obtain the minimum novelty control through a provably exact convex relaxation of the original nonconvex problem. The relaxed problem can be solved efficiently using well established convex programming solvers.

We use the minimally novel control to construct a metric which can be used to characterize largescale linear systems or networks. Importantly, we demonstrate that this noveltybased metric provides a distinct characterization of controllability compared to energybased control metrics. The result suggests that a broader, multifaceted set of metrics may be useful for completely characterizing the controllability of such systems.
The remainder of the paper is organized as follows. In Section II, we describe the mathematical notation used in this paper. In Section III, we introduce the inner productbased input novelty measure for continuoustime, linear timevarying systems and formulate a nonconvex optimal control problem that minimizes this novelty under the constraint of a fixed average input energy. We establish the existence and the uniqueness of a global optimal solution of the control problem and derive a closedform expression for the minimally novel input. In Section IV, we derive analogous results for discrete time linear timevarying systems with the additional result of exact relaxation of the nonconvex novelty minimization problem. Finally, in Section VA, we demonstrate the use of the minimally novel input as a control metric to systematically characterize large scale linear networks. The paper concludes with a summary and discussion of future work.
Ii Mathematical Notation
Most notation is standard and will be introduced as the results are developed. We use lowercase letters to represent scalars, boldface lowercase letters to represent vectors, capital letters to represent matrices. Exceptions are , , , and , which we represent as scalars. We use to denote the space of  dimensional vectors with their elements as real numbers. Similarly, we use and to denote the space of dimensional matrices with real and nonnegative real entries, respectively. is the Euclidean norm of the vector . is the transpose of a vector . We use and to represent the value of at continuous time and discrete time respectively. , and are constant dimensional vectors. is the inverse of a matrix and represents the element of .
Iii ContinuousTime, Linear Dynamical Networks
Iiia Input Novelty based Controllability Metric
We consider a linear, timevarying system with dynamics of the form
(1) 
Here represents the state of the system at time , describes the timevarying dynamics, is the input matrix, and is the input to the system. Without loss of generality, we say that (1) describes the time evolution of (timevarying) linear networks in the presence of external inputs.
We consider an input , , with total energy , i.e.
(2) 
Further, we assume that drives from to subject to the dynamics (1). Here, is a constant and is the average pertime input energy. We introduce the inner productbased measure
(3) 
where
(4) 
to quantify the novelty of a subsequent input , (which drives from to ), relative to . For a fixed input energy, measures the change in orientation (thus novelty) of two consecutive inputs trajectories of equal lengths (in time).
Remark 1
It is readily evident that .
Remark 2
It follows from (3) that the novelty of relative to decreases as increases and is minimum when i.e. when , , for all .
IiiB Minimum Novelty Problem
From the conceptual formulation introduced above, we can develop a control problem to design the minimally novel input such that a desired change in the state of the system can be achieved under the constraint of fixed energy subject to (1). Specifically, we formulate the following optimal control problem:
(6a)  
s.t.  (6b)  
(6c) 
Here, the statetransition matrix is given by
(7) 
with , where is the identity matrix.
It should be noted here that the constraint (6c) is obtained by integrating (1) with respect to over the period of . Immediately, we note that the quadratic equality constraint (6b) makes the optimization problem (6) nonconvex. Furthermore, we note that our optimal control problem formulation (6) is different from the classical minimum effort problems [13, 14] where the norm of control inputs is minimized under the constraints of explicit lower and upper bounds on the inputs.
IiiC Analytical Results
We derive conditions for the existence of a unique global optimal solution of the nonconvex optimization problem (6). Based on this, we provide a closedform expression for the optimal .
IiiC1 Existence of a Minimally Novel Input
Theorem 1
A solution of the nonconvex optimization problem (6) exists if
(8) 
where
(9a)  
(9b) 
Here, ^{1}^{1}1For notational simplicity, dependence on the initial time (here, ) is implicit. is the usual controllability gramian at time [8] and is defined as
(10) 
Remark 4
From the above remark, the intuition behind Theorem 1 is seemingly straightforward – for a minimally novel solution to exist, the input energy must at least exceed the minimum energy required to effect the state transfer. However, in establishing the closedform solution for the minimally novel input, it is useful to complete a formal proof.
Proof:
Define as
(11) 
Clearly, and from (6b). Thus, we can replace the constraint (6b) by
(12) 
In differential form, we can write (11) as
(13) 
To solve the dynamic optimization problem (6a), (6c), and (12) in continuous time, we write the Hamiltonian as
(14) 
Here, and are the costate variables associated with the dynamics (1) and (13) respectively. We derive the following optimality conditions (i.e. the EulerLagrange equations [13]):
(15a)  
(15b)  
(15c) 
Here, is the initial condition (at ) of (15a). From (15c), (16a) and (16b), we derive the optimal control law as
(17) 
For the existence of a solution, must be a real number. Thus, either or . Now it follows directly from Remark 4 that the total energy must satisfy (8) for the existence of a solution i.e. .
IiiC2 Uniqueness of the Minimally Novel Input
Theorem 2
Proof:
By replacing in (17) with (18), then substituting into (3), we obtain the optimal value of as a function of :
(20) 
Thus, a unique optimal control input exists and is given by
(22) 
The expression (22) is the closed form for the minimally novel input. We now proceed to state an important lemma that establishes the dependence of the minimum novelty control on only the initial, intermediate and terminal states.
Lemma 1
Given , and , in (20) is independent of , .
Proof:
This lemma will form the basis for our use of novelty as a control metric, as highlighted in Section V below.
IiiC3 Euclidean  Inner Product Transformation
As noted in Remark 3, it is a notable consequence of our cost formulation that the problem can be exactly recast in terms of the Euclidean norm. Specifically, if we consider
(24) 
as the cost function in (6a), we obtain the optimal solution as
(25a)  
(25b)  
(25c) 
IiiD Average Novelty Measure and Results
The definition of the innerproduct novelty measure (3) requires the inputs and to be of equal time length . To allow for characterization of the input novelty in cases where the time lengths of these inputs are different, we define an average novelty measure in the form
(26) 
with
(27) 
Here, is the time duration of the prior input which drives the system (1) from to . If we consider (26) as the cost function in (6a), we obtain the optimal as
(28) 
where
(29a)  
(29b) 
Iv DiscreteTime, Linear Dynamical Networks
Iva Input Novelty
We consider linear, timevarying discrete time systems with dynamics of the form
(30) 
Here represents the state of the system at discrete time for , is the timevarying dynamic matrix, is the input matrix, and is the input to the system. We say that (30) describes the time evolution of discrete time linear networks in the presence of external inputs.
Let us assume an input , , with total energy , i.e.
(31) 
Here, is a constant scaling factor. We assume that the sequence of inputs can drive from to , subject to the dynamics (30). We introduce the inner product
(32) 
where
(33) 
to measure the novelty of a subsequent input , , relative to , required in order to reach the state . In other words for a fixed input energy, measures the required directional change in inputs (thus novelty) to achieve a given state change in the state of the system. Here, is a constant scaling factor.
IvB Minimum Novelty Problem
Similar to the continuoustime case, we formulate the following optimal control problem to design the minimally novel input such that a desired change in the state of the system can be achieved under the constraint of fixed energy subject to the system dynamics (30):
(34a)  
s.t.  
(34b)  
(34c) 
IvC Results
We relax the nonconvex problem by replacing the equality constraint (34b) with an inequality constraint. Thus, the relaxed problem becomes convex and by deriving the optimality conditions, we establish that the relaxation is exact, i.e. any solution of the nonconvex optimization problem (34) is also the solution of the convex optimization problem.
IvC1 Convex Relaxation
We relax the equality constraint (34b) with the following inequality constraint
(35) 
IvC2 Exact Relaxation
Lemma 2
Proof:
It is sufficient to show that equality holds for the constraint (35) in the convex optimization problem (34a), (34c), and (35). For this, we write the Lagrangian as
(36) 
Here, and are the Lagrange multipliers associated with the constraints (35) and (34c) respectively. We derive the KarushKuhnTucker (KKT) conditions for optimality as:
(37a)  
(37b)  
(37c)  
(37d)  
(37e) 
The result permits numerically efficient evaluation of the minimally novel input.
V Minimum Novelty as a Control Metric
We now return to the original motivation outlined in the introduction: to systematically characterize the responsiveness of linear networks to novelty. For instance, given systems and , identical in structure but different in dynamics, we may ask which requires more novelty in order to execute the same trajectory. In this sense, we can fashion the minimum novelty solutions analytically obtained above into a control (or, controllability) metric. To do so, we standardize the notions of initial, intermediate and terminal states. The space of terminal states can then be sampled systematically (e.g., along eigenvectors of the Controllability gramian), or via some other MonteCarlo fashion, in order to gain an aggregate noveltybased metric.
Va Examples
VA1 Ratebased Recurrent Neuronal Networks
We consider a recurrent network of neurons with linearized firing rate dynamics of the form [15]
(40) 
Here, represents the firing rate of the neurons at time , is a diagonal matrix whose diagonal elements are the (positive) time constants of the neurons, defines the linear interaction among neurons in the network, is the input matrix, and is the afferent input. Since is invertible, (40) can be represented in the form of (1) by considering where is the identity matrix.
In this example, we consider a recurrent network of neurons where neurons are excitatory and are inhibitory. We choose the time constants (in milliseconds) of the neurons, i.e. the elements of the diagonal matrix , from a uniform distribution . The connectivity weights (in essence, a time constant for excitation/inhibition from the neuron to ) are drawn from a uniform distribution or , if neuron is excitatory or inhibitory, respectively. We assume that for , i.e. neurons do not directly excite/inhibit themselves. Assuming , we proceed to compute the minimally novel inputs required to make a desired directional change in the network firing rate using (8)(10), (20)(22).
To complete the example, we specify ms. The states and are specified to satisfy with , where in this particular case we specify . The average energy of the prior input (see (2)) and the average energy of the computed minimally novel input of (see (4)) are set to . The prior input is specified to be constant over the interval . Figure 3A illustrates the schematic of the example setup.
Figure 4 illustrates the outcome of the example for random realizations of the system. Each red dot on the figure depicts the novelty associated with the solution to (20)(21), i.e., the minimum novelty. Note, again, that in this example, these inputs are constrained to have unit average energy. Each blue dot corresponds to the minimum energy solution (for the same and ). As a verification of our theoretical development, we note that the minimum energy solution consistently requires an injection of novelty (angular orientation) relative to the prior input and relative to the optimum.
VB Complex Networks
For our second example, we demonstrate the utility of our noveltybased controllability metric for distinguishing linear complex networks based on topological properties. Moreover, we demonstrate that a noveltybased control metric provides a different description of network controllability than does an energybased characterization.
Consider networks of excitatory neurons with dynamics of the form (40) where the structure of the weight matrix is defined by the adjacency matrix of either the undirected BarabásiAlbert scalefree network (BA) [16] or the undirected WattsStrogatz smallworld network (WS) [17] with appropriate parameters. If the element of the network adjacency matrix is , we say that neuron is connected to the neuron .
For both types of networks, we construct the matrices and (see (40)) as follows. Given a degree distribution of the BA network, we construct full rank realizations of the adjacency matrix. It should be noted here that the total number of undirected edges in all realizations is the same. For each realization, we replace the nonzero elements of the adjacency matrix by a number drawn from a uniform distribution , thus yielding the matrix . We choose the time constants (in milliseconds) of the neurons (the elements of the diagonal matrix ) from a uniform distribution . An analagous procedure is followed to specify the WS networks. Since the development of a WS network does not involve the creation of new edges, but rather the ‘rewiring’ of existing connections, while a BA network is created by a growth algorithm, we adjust their respective parameters to obtain approximately the same number of edges in both networks.
Assuming , we proceed to compute the minimum directional change in inputs (i.e. minimally novel inputs) required to make a desired directional change in firing rates of neurons using (8)(9), (19)(21).
To complete the example, we again specify ms. The states and are specified to satisfy with , where in this particular case we specify . We set the intermediate state to . As a reminder, the prior input drives the network from to in time and the computed minimally novel input of drives the network from to in time . The average energy of (see (2)) and the average energy of (see (4)) are set to . Figure 3B illustrates the example setup. Note that from Lemma 1, it is sufficient to randomly sample the system parameters without explicitly sampling the space of prior inputs.
Figure 5A shows the minimum novelty associated with the solution of (20)(21) as a function of the total number of edges in the BA network (blue) and the WS network (red). Here, is defined as the ratio of the total number of undirected edges in the network and the total possible number of undirected edges (i.e. ). At each , we show the mean and mean standard deviation over realizations of the network. As shown in this figure, (i) controlling both types of networks becomes progressively harder in terms of minimum novelty (more novelty is required) as these networks become more dense, and (ii) the WS network requires consistently less novelty to control.
Figure 5B shows the minimum energy required to effect the same state trajectories for the BA network (blue) and the WS network (red). As shown in this figure, (i) controlling both types of networks becomes progressively easier in terms of minimum energy (less energy is required) as these networks become more dense, and (ii) the WS network requires consistently less energy to control.
This example demonstrates how a noveltybased analysis can provide a different and important controllability characterization, complementing the use of minimum energybased methods. Moreover, both metrics are consistent in distinguishing the BA network from the WS network.
Vi Conclusions
In many natural systems, and especially in biological sensory networks, the responsiveness of the system to a given input is dependent on both the energy of the stimulus and also its salience or novelty relative to background and prior inputs. Conventional control analysis aggregates all past input history into the current state and evaluates controllability in terms of the forward trajectory. Here, we introduce a control analysis to explicitly quantify the extent of novelty, defined in terms of an inner product, required to effect a state transfer in a linear system. The analysis, and corresponding solution to the problem of finding minimally novel inputs, are used to establish a new type of characterization of system controllability. Through two illustrative examples, we show how such an analysis is distinct from energybased metrics in describing the controllability of largescale linear systems or networks. The results indicate that the analysis of such systems may benefit from a multidimensional metric in which energy, novelty and other control characteristics are components. In future work, we plan to use such an approach in the specific domain of sensory neuroscience to examine how various timescales of dynamics, including neuroplasticity, impact the ability of networks to transform afferent input and facilitate higher information processing. Application in other domains, including finance and social network analysis, could similarly be conceived.
Acknowledgment
S. Ching Holds a Career Award at the Scientific Interface from the BurroughsWellcome Fund.
References
 [1] G. Hu and E. Davison, “A real radius measure for controllability,” in American Control Conference, 2001. Proceedings of the 2001, vol. 4, 2001, pp. 3144–3148. [Online]. Available: http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=946404
 [2] G. Hu and E. J. Davison, “Real controllability/stabilizability radius of lti systems,” IEEE Transactions on Automatic Control, vol. 49, no. 2, pp. 254–257, 2004.
 [3] D. Hinrichsen and A. Pritchard, Real and Complex Stability Radii: A Survey, ser. Institut für Dynamische Systeme, Universität Bremen. Inst. für Dynam. Systeme, 1989. [Online]. Available: http://books.google.com/books?id=EpcCHAAACAAJ
 [4] G. Yan, J. Ren, Y.C. Lai, C.H. Lai, and B. Li, “Controlling complex networks: How much energy is needed?” Phys. Rev. Lett., vol. 108, p. 218703, May 2012. [Online]. Available: http://link.aps.org/doi/10.1103/PhysRevLett.108.218703
 [5] F. Pasqualetti, S. Zampieri, and F. Bullo, “Controllability metrics, limitations and algorithms for complex networks,” IEEE Transactions on Control of Network Systems, vol. 1, no. 1, pp. 40–52, 2014.
 [6] F. Pasqualetti and S. Zampieri, “On the controllability of isotropic and anisotropic networks,” 2014. [Online]. Available: http://www.fabiopas.it/papers/FPSZ14.pdf
 [7] F. L. Cortesi, T. H. Summers, and J. Lygeros, “Submodularity of energy related controllability metrics,” ArXiv preprint arXiv:1403.6351, 2014.
 [8] H. Kwakernaak and R. Sivan, Linear optimal control systems. WileyInterscience, 1972.
 [9] J. Downar, A. P. Crawley, D. J. Mikulis, and K. D. Davis, “A cortical network sensitive to stimulus salience in a neutral behavioral context across multiple sensory modalities.” J Neurophysiol, vol. 87, no. 1, pp. 615–620, Jan 2002.
 [10] S. Ching and J. T. Ritt, “Control strategies for underactuated neural ensembles driven by optogenetic stimulation.” Front Neural Circuits, vol. 7, p. 54, 2013. [Online]. Available: http://dx.doi.org/10.3389/fncir.2013.00054
 [11] S. Ching, E. N. Brown, and M. A. Kramer, “Distributed control in a meanfield cortical network model: implications for seizure suppression.” Phys Rev E Stat Nonlin Soft Matter Phys, vol. 86, no. 2 Pt 1, p. 021920, Aug 2012.
 [12] K. Q. Lepage, S. Ching, and M. A. Kramer, “Inferring evoked brain connectivity through adaptive perturbation.” J Comput Neurosci, vol. 34, no. 2, pp. 303–318, Apr 2013. [Online]. Available: http://dx.doi.org/10.1007/s1082701204228
 [13] D. E. Kirk, Optimal Control Theory: An Introduction. Dover, 2004.
 [14] M. Nagahara, D. E. Quevedo, and D. Nesic, “Maximum handsoff control and optimality,” in 52nd IEEE Conference on Decision and Control, Florence, Italy, December 1013, 2013, pp. 3825–3830.
 [15] P. Dayan and L. F. Abbott, Theoretical Neuroscience. The MIT Press, 2001.
 [16] A.L. Barabási and R. Albert, “Emergence of scaling in random networks,” Science, vol. 286, pp. 509–512, October 1999.
 [17] D. J. Watts and S. H. Strogatz, “Collective dynamics of ’smallworld’ networks,” Nature, vol. 393, pp. 440–442, 1998.