Learning in cognitive systems with autonomous dynamics
The activity patterns of highly developed cognitive systems like the human brain are dominated by autonomous dynamical processes, that is by a self-sustained activity which would be present even in the absence of external sensory stimuli.
During normal operation the continuous influx of external stimuli could therefore be completely unrelated to the patterns generated internally by the autonomous dynamical process. Learning of spurious correlations between external stimuli and autonomously generated internal activity states needs therefore to be avoided.
We study this problem within the paradigm of transient state dynamics for the internal activity, that is for an autonomous activity characterized by a infinite time-series of transiently stable attractor states. We propose that external stimuli will be relevant during the sensitive periods, the transition period between one transient state and the subsequent semi-stable attractor. A diffusive learning signal is generated unsupervised whenever the stimulus influences the internal dynamics qualitatively.
For testing we have presented to the model system stimuli corresponding to the bar-stripes problem and found it capable to perform the required independent-component analysis on its own, all the time being continuously and autonomously active.
It is well known that the brain has a highly developed and complex self-generated dynamical neural activity. We are therefore confronted with a dichotomy when attempting to understand the overall functioning of the brain or when designing an artificial cognitive system: A highly developed cognitive system, such as the brain, is influenced by sensory input but is not driven directly by the input signals. The cognitive system needs however this sensory information vitally for adapting to a changing environment and survival.
In this context we then want to address a two-fold goal:
Can we formulate a meaningful paradigm for the self-sustained internal dynamics of an autonomous cognitive system?
How is the internal activity process influenced by sensory signals, viz which are the principles for the respective learning processes?
With respect to the first problem we note that an increasing flux of experimental results from neurobiology supports the notion of quasi-stationary spontaneous neural activity in the cortex [1, 2, 3, 4, 5]. It is therefore reasonable to investigate possible set-ups for artificial cognitive systems centrally based on the notion of internally generated spontaneous transient states, as we do in the present investigation.
Ii General Outline
We will present the model and the issues being studied in two steps, starting with a general overview, discussing technical details in a second step.
Ii-a Transient-state and competitive dynamics
There are many notions of complex system theory characterizing the long-term behavior of a dynamical system , such as regular vs. chaotic behavior. The term ‘transient-state dynamics’ refers, on the other hand, to the type of activity occurring on intermediate time scales, as illustrated in Fig. 1. A time series of semi-stable activity patterns, also denoted transient attractors, is characterized by two time scales. The typical duration of the activity plateaus and the typical time needed to perform the transition from one semi-stable state to the subsequent one. The transient attractors turns into stable attractors in the limit .
Transient state dynamics is intrinsically competitive in nature. When the current transient attractor turns unstable the subsequent transient state is selected by a competitive process. Transient-state dynamics is a form of ‘multi-winners-take-all’ process, with the winning coalition of dynamical variables suppressing all other competing activities.
Similar processes have been proposed to be relevant for various neural functionalities. Edelman and Tononi [7, 8] argue that ‘critical reentrant events’ constitute transient conscious states in the human brain. These ‘states-of-mind’ are in their view semi-stable global activity states of a continuously changing ensemble of neurons, the ‘dynamic core’. This activity takes place in what Dehaene and Naccache  denote the ‘global workspace’. The global workspace serves, in the view of Baars and Franklin , as an exchange platform for conscious experience and working memory. Crick and Koch  and Koch  have suggested that the global workspace is made-up of ‘essential nodes’, i.e. ensembles of neurons responsible for the explicit representation of particular aspects of visual scenes or other sensory information.
Ii-B Competitive dynamics and sensitive periods
Within the setting of a generalized neural network we utilize a continuous-time formulation with rate-encoding neural activity centers, characterized by normalized activity levels . One can then define, quite generally, via
the respective growth rates . Representative time series of growth rates are illustrated in Fig. 2. When the the respective neural activity increases, approaching rapidly the upper bound, as illustrated in Fig. 1; when it decays to zero. The model is specified [13, 14], by providing the functional dependence of the growth rates with respect to the set of activity-states .
During the transition periods many, if not all, neurons will enter the competition to become a member of the new winning coalition. The competition is especially pronounced whenever most of the growth rates are small in magnitude, with no small subset of growth rates dominating the all the others. Whether this does or does not happen depends on the specifics of the model set-up, in Fig. 2 two cases are illustrated (upper/lower graph in Fig. 2). In the first case the competition for the next winning coalition is restricted to a subset of neurons, in the second case the competition is network-wide.
When most neurons participate in the competition process for a new winning coalition the model will have ‘sensitive periods’ during the transition times and it will able to react to eventual external signals.
Ii-C Sensitive periods and learning
So far we have discussed in general terms the properties of isolated models exhibiting a self-sustained dynamical behavior in terms of a never-ending time series of semi-stable transient states, as illustrated in Fig. 2, using rate-encoding equations specified in previous work [13, 14].
The importance of sensitive periods comes when this model is coupled to a stream of sensory input signals. It is reasonable to assume, that external input signals will contribute to the growth rates via
where the encode the influence of the input signals. Let us furthermore assume that the input signals are suitably normalized, such that
For the transient states the for all sites not forming part of the winning coalition and the input signal will therefore not destroy the transient state. With the normalization given by Eq. (3) the total growth rate will remain negative for all inactive sites. The input signal will however enter into the competition for the next winning coalition during a sensitive period, providing an additional boost for the respective neurons.
This situation is exemplified in Fig. 3, where we present simulation-results for a system containing neurons subject to two sensory inputs . The self-generated time series of winning coalitions is both times redirected by the strongest component of the sensory input, the details depending on whether the input signal partially overlaps with the current winning coalition or not.
We then have a model system in which there are well defined time-windows suitable for the learning of correlations between the input signal and the intrinsic dynamical activity, namely during and shortly after a transition period, that is the sensitive period. A possible concrete implementation for this type of learning algorithm will be given further below.
This set-up would allow the system also to react to an occasional very strong input signal having . Such a strong signal would suppress the current transient state altogether and impose itself. This possibility of rare strong input signals is evidently important for animals and would be, presumably, also helpful for an artificial cognitive system.
Ii-D Diffusive learning signals
Let us return to the central problem inherent to all system reacting to input signals having at the same time a non-trivial intrinsic dynamical activity. Namely when should learning occur, i.e. when should a distinct activity center become more sensitive to a specific input pattern and when should it suppress its sensibility to a sensory signal.
The above developed framework of competitive dynamics allows for a straightforward solution of this central issue: Learning should occur then and only then when the input signal makes a qualitative difference, viz when the input signal deviates the transient-state process. For illustration let us assume that the series of winning coalitions is
where the index indicates that the transition is driven by the internal autonomous dynamics and that the series of winning coalitions take the form
in the presence of a certain sensory signal . Note, that a background of weak or noisy sensory input could be present in the first case, but learning should nevertheless occur only in the second case.
Technically this is achieved by defining a suitable diffusive learning signal111The name ‘diffusive learning signal’  stems from the fact, that many neuromodulators are released in the brain in the intercellular medium and then diffuse physically to the surrounding neurons, influencing the behavior of large neural assemblies. . It is activated whenever any of the input signals changes the sign of the respective growth rates during the sensitive periods,
viz when it makes a qualitative difference. Here is the internal contribution to the growth rate, i.e. the value it would have in the absence of the input signal and the are the growth and decay rates for the diffusive learning signal respectively. The diffusive learning signal is a global signal and a sum over all dynamical variables is therefore implicit on the right-hand side of Eq. (4).
Ii-E The role of attention
The general procedure for the learning of correlation between external signals and intrinsic dynamical state for a cognitive system presented here does not rule out other mechanisms. Here we concentrate on the learning algorithm which occurs automatically, one could say sub-consciously. Active attention focusing, which is well known in the brain to potentially shut off a sensory input pathway or to enhance sensibility to it, may very well work in parallel, to the continuously ongoing mechanism investigated here.
Iii Associative Thought Processes
So far we have described, in general terms, the system we are investigating, having sensitive periods during the transition periods of the continuously ongoing transient-state process, with learning of input signals regulated by a diffusive learning signal.
Our system consists of two components, as illustrated in Fig. 4. For the component generating an infinite time-series of transient state we employ a dense homogeneous associative network (dHAN). The dHAN-model has been studied previously and shown to generated a time series of transient states characterized by high associative overlaps between subsequent winning coalitions [13, 14]. The time series might therefore be interpreted as an associative thought process, it carries with it a dynamical attention field .
Iii-a Input data-stream analysis
The input signal acts via Eq. (2) on the dHAN layer, with the contribution to the growth rate of the dHAN neuron given by
where we have denoted now with the superscripts the dHAN- and input-layer respectively. For subsequent use we also define in Eq. (5) an auxiliary variable which quantifies the influence of inactive input-neurons. The task is now to find a suitable learning algorithm which extracts the relevant information from the input-data stream by mapping distinct input-patterns onto selected winning coalitions of the dHAN layer. This is the setup typical for an independent component analysis .
The multi-winners-take-all dynamics in the dHAN module implies individual neural activities to be close to 0/1 during the transient states and we can therefore define three types of inter-layer links (see Fig. 4):
Links connecting active input neurons with the winning coalition of the dHAN module.
Links connecting inactive input neurons with the winning coalition of the dHAN module.
Links connecting active input neurons with inactive neurons of the dHAN module.
The orthogonal links take their name from the circumstance that the receptive fields of the winning coalition of the target layer need to orthogonalize to all input-patters differing from the present one. Note that it is not the receptive field of individual dHAN-neurons which is relevant, but rather the cumulative receptive field of a given winning coalition.
We can then formulate three simple rules for the respective link-plasticity. Whenever the new winning coalition in the dHAN-layer is activated by the input layer, viz whenever there is a substantial diffusive learning signal, i.e. when exceeds a certain threshold , the following optimization procedures should take place:
The sum over active links should take a large but finite value ,
The sum over orthogonal links should take a small value ,
The sum over inactive links should take a small but non-vanishing value ,
The , and are the target values for the respective optimization processes. In order to implement these three rules we define three corresponding contributions to the link plasticities:
where the inputs and to the target layer are defined by Eq. (5). For the sign-function is valid for and respectively, denotes the Heaviside-step function in Eq. (6). The inter-layer links cease to be modified whenever the total input is optimal, viz when no more ‘mistakes‘ are made .
|0.008 0.001 0.001||0.9 0.3 0.2||0.4 0.2||0.25|
Using these definitions, the link plasticity may be written as
where is an appropriate threshold for the diffusive learning signal and are the desired contributions to the growth rate for active/inactive postsynaptic neurons, and the respective rates and and the critical activity reservoirs defining an active/inactive postsynaptic center respectively. A suitable set of parameters, which has been used throughout this work, is given in Table I.
We note, that a given interlayer-link is in general subject to competitive optimization from the three processes (act/orth/ina). Averaging would occur if the respective learning rates // would be of the same order of magnitude. It is therefore necessary, that
Iii-B Homeostatic normalization
It is desirable that the interlayer connections neither grow unbounded with time (runaway-effect) nor disappear into irrelevance. Suitable normalization procedures are therefore normally included explicitly into the respective neural learning rules and are present implicitly in Eqs. (6) and Eq. (III-A).
The strength of the input-signal is optimized by Eq. (III-A) both for active as well as for inactive -layer neurons, a property referred to as fan-in normalization. Eqs. (6) and (III-A) also regulate the overall strength of inter-layer links emanating from a given -layer neuron, a property called fan-out normalization.
Next we note, that the timescales for the intrinsic autonomous dynamics in the dHAN-layer and for the input signal could in principle differ substantially. Potential interference problems can be avoided when learning is switched/one very fast, viz when activation and when the decay rate are larger, i.e. when is smaller than both the typical time scales of the input and of the self-sustained dynamics.
Iv The Bars Problem
A cognitive system needs to autonomously extract meaningful information about its environment from its sensory input data stream via signal separation and features extraction. The identification of recurrently appearing patterns, i.e. of objects, in the background of fluctuation combinations of distinct and noisy patterns constitutes a core demand in this context. This is the domain of the independent component analysis  and blind source separation , which seeks to find distinct representations of statistically independent input patterns.
In order to test our system made-up by an input-layer coupled to a dHAN layer, as illustrated in Fig. 4, we have selected the bars problem . The bars problem constitutes a standard non-linear reference task for the feature extraction via an independent component analysis for a input layer. Basic patterns are the vertical and horizontal bars. The individual input patterns are made-up of a non-linear superposition of the basic bars, containing with probability any one of them, as shown in Fig. 5.
Iv-a Simulation results
For the simulations we presented to the system about randomly generated input patterns of the type shown in Fig. 5, including a small noise-level of about . The individual patterns lasted with about for the time in between two successive input signals. These time-scales are to be compared with the time-scale of the autonomous dHAN-dynamics illustrated in the Figs. 2 and 3, for which the typical stability-period for a transient state is about . This implies that we presented the system sparse (in time) sensory input. We also note that the number of training patterns in our simulation is exceedingly small, standard neural algorithms use routinely and more training patterns .
The results for the simulations are presented in Figs. 6 and 7. For the geometry of the dHAN network we used a simple 15-site chain containing 14 potential winning coalitions, as illustrated in Fig. 7, namely , , …, .
In Fig. 6 we present the response
of the 14 potential winning coalitions to the 10 basic input patterns , the isolated bars. Here denotes the set of sites of the winning-coalition and its size, here .
The individual potential winning coalitions have acquired in the course of the simulation, via the learning rule Eq. (III-A), distinct susceptibilities to the 10 bars compare Fig. 6. We note that the learning is unsupervised and quite fast. This example-problem is overcomplete, there are more potential winning coalitions than statistically independent basic pattern. Perfect signal separation can therefore not be expected. E.g. the second-last vertical bar in Fig. 7 is not resolved.
Iv-B Receptive fields
The receptive fields
of the 14 potential winning coalitions are given in Fig. 7. The inter-layer synaptic weights can be both positive and negative and the orthogonalization procedure, Eq. (6), results in complex receptive fields.
Physically separated target neurons, viz a ‘single-winner-takes-all’ set-up, are normally used for standard analysis neural algorithms performing an independent component analysis . The winning coalitions of the dHAN-layer are however overlapping and every link targets in general more than one potential winning coalition in the dHAN layer, in our case two for (and one for ).
The unsupervised learning procedure, Eq. (III-A), involves therefore a competition between the active contribution and the and , as given by Eq. (6). The inter-layer synaptic weights are therefore only changed to the point necessary for recognition, but not up to the saturation point, as evident from the data presented in Fig. 7. This behavior is consistent with the ‘learning by mistakes’ paradigm , which states that a cognitive system needs to learn in general only when committing a mistake.
V Conclusions and Future Works
A cognitive system has its own internal dynamics and we studied here the interplay of these self-generated activity states, the time-series of winning coalitions, with the sensory input for the purpose of unsupervised feature extraction. We proposed learning to be autonomously activated during the transition from one winning coalition to the subsequent one.
This general principle may be implemented algorithmically in various fashion. Here we used a generalized neural net (dHAN - dense homogeneous associative net) for the autonomous generation of a time series of associatively connected winning coalitions and controlled the unsupervised extraction of input-features by an autonomously generated diffusive learning signal.
We tested the algorithm for the bars problem and found good and fast learning for the case of sparse temporal input. Preliminary results indicate that the learning algorithm retains functionality under a wide range of conditions. We plan to extend the simulations to various forms of temporal inputs, especially to quasi-continuous input and to natural scene analysis and to study the embedding of the here proposed concept within the framework of a full-fledged and autonomously active cognitive system.
-  M. Abeles et al., Cortical activity flips among quasi-stationary states, PNAS, vol. 92, 1995, pp 8616-8620.
-  D.L. Ringach, States of mind, Nature, vol. 425, 2003, pp 912-913.
-  T. Kenet, D. Bibitchkov, M. Tsodyks, A. Grinvald and A. Arieli, Spontaneously emerging cortical representations of visual attributes, Nature, vol. 425, 2003, pp 954-956.
-  J.S. Damoiseaux, S.A.R.B. Rombouts, F. Barkhof, P. Scheltens, C.J. Stam, S.M. Smith and C.F. Beckmann, Consistent resting-state networks across healthy subjects, PNAS, vol. 103, 2006, pp 13848-13853.
-  C.J. Honey, R. Kötter, M. Breakspear and Olaf Sporns, Network structure of cerebral cortex shapes functional connectivity on multiple time scales, PNAS, vol. 104, 2007, pp 10240-10245.
-  C. Gros, Complex and Adaptive Dynamical Systems, A Primer, Springer; 2007 (in press).
-  G.M. Edelman and G.A. Tononi, A Universe of Consciousness, New York: Basic Books; 2000.
-  G.M. Edelman, Naturalizing consciousness: A theoretical framework, PNAS, vol. 100, 2003, pp 5520-5524.
-  S. Dehaene and L. Naccache, Towards a cognitive neuroscience of consciousness: basic evidence and a workspace framework, Cognition, vol. 79, 2003, pp 1-37.
-  B.J. Baars and S. Franklin, How conscious experience and working memory interact, Trends Cog. Science, vol. 7, 2003, pp 166-172.
-  F.C. Crick and C. Koch, A framework for consciousness, Nature Neurosci., vol. 6, 2003, pp 119-126.
-  C. Koch, The Quest for Consciousness - A Neurobiological Approach, Robert and Company; 2004.
-  C. Gros, Self-Sustained Thought Processes in a Dense Associative Network, in KI 2005, U. Furbach (Ed.), Springer Lecture Notes in Artificial Intelligence, vol. 3698, 2005, 366-379; also available as http://arxiv.org/abs/q-bio.NC/0508032.
-  C. Gros, Neural networks with transient state dynamics, New J. Phys., vol. 9, 2007, 109.
-  A. Hyvärinen, and E. Oja, Independent component analysis: Algorithms and applications, Neural Networks, vol. 13, 2000, 411-430.
-  D.R. Chialvo and P. Bak, Learning from mistakes. Neuroscience, vol. 90, 1999, 1137-1148.
-  S. Choi, A. Cichocki, H.M. Park, and S.Y. Lee, Blind Source Separation and Independent Component Analysis: A Review, Neural Information Processing, vol. 6, 2005, 1-57.
-  P. Földiák, Forming sparse representations by local anti-Hebbian learning, Biological Cybernetics, vol. 64, 1990, 165-170.
-  N. Butko, and J. Triesch, Learning Sensory Representations with Intrinsic Plasticity, Neurocomputing, vol. 70, 2007, 1130-1138.