1 Introduction

# Centralized and Decentralized Global Outer-synchronization of Asymmetric Recurrent Time-varying Neural Network by Data-sampling

Centralized and Decentralized Global Outer-synchronization of Asymmetric Recurrent Time-varying Neural Network by Data-sampling1

[0.2in]

Wenlian Lu2, Ren Zheng3, Tianping Chen4

Abstract

In this paper, we discuss the outer-synchronization of the asymmetrically connected recurrent time-varying neural networks. By both centralized and decentralized discretization data sampling principles, we derive several sufficient conditions based on diverse vector norms that guarantee that any two trajectories from different initial values of the identical neural network system converge together. The lower bounds of the common time intervals between data samples in centralized and decentralized principles are proved to be positive, which guarantees exclusion of Zeno behavior. A numerical example is provided to illustrate the efficiency of the theoretical results.

Keywords: Outer-synchronization, Data sampling, Centralized principle, Decentralized principle, Recurrent neural networks

## 1 Introduction

Recurrently connected neural networks, also known as the Hopfield neural networks, have been extensively studied in past decades and found many applications in different areas. Such applications heavily depend on the dynamical behaviors of the system. Therefore, analysis of the dynamics is a necessary step for practical design of neural networks.

The dynamical behaviors of continuous-time recurrently asymmetrically connected neural networks (CTRACNN) have been studied at the very early stage of neural network research. For example, multistable and oscillatory behaviors were studied by Amari (1971, 1972) and Wilson & Cowan (1972). Chaotic behaviors were studied by Sompolinsky, & Crisanti (1988). Hopfield, & Tank (1984, 1986) studied stability of symmetrically connected networks and showed their practical applicability to optimization problems. It should be noted that Cohen and Grossberg, see Cohen, & Grossberg (1983) gave more rigorous results on the global stability of networks.

The global stability of symmetrically connected networks described by differential equations has now been well established. See Chen (1999); Chen, & Amari (2001); Chen, & Lu (2002); Fang, & Kincaid (1996); Forti, & Marini (1994); Hirsch (1989); Kaszkurewicz, & Bhaya (1994); Kelly (1990); Li, Michel, & Porod (1988); Matsuoka (1992); Yang, & Dillon (1994) and the references therein. More related to the present paper, the previous paper (Liu, Lu, & Chen, 2011) addressed the global self-synchronization of general continuous-time asymmetrically connected recurrent networks and discussed the independent identically distributed switching process on the selecting the time-varying parameters in detail.

However, in applications, discrete iteration is popular to be employed to realize neural network process, rather than continuous-time equations. Generally, synchronization analysis for differential equations cannot be applicable to the discrete-time situation. There are several papers (Jin, Nikiforuk, & Gupta, 1994; Jin, & Gupta, 1999; Wang, 1997) that discussed different types of discrete-time neural networks, where the step sizes were constants. However, in Liu, Chen, & Yuan (2012); Manuel, & Tabuada (2011); Seyboth, Dimarogonas, & Johansson (2013); Wang, & Lemmon (2008), these papers pointed out that the constant time-step size was costly. This motivates us to design adaptive step sizes for synchronization of asymmetric recurrent time-varying neural network.

Moreover, the discretization is related to the concept of sampled-data control. There are a number of papers discussing dynamics of neural networks, using sampled-data control. The papers (Lam, & Leung, 2006; Wu, Shi & Su, 1972; Zhu, & Wnag, 2011) applied the sampled-data control technique towards stabilization of three-layer fully connected feedforward neural networks. In Chandrasekar, Rakkiyappan, Rihan, & Lakshmanan (2014); Jung, Park, & Lee (2014); Lee, Park, Kwon, & Lee (2013); Liu, Yu, Cao, & Chen (2015); Rakkiyappan, Chandrasekar, Park, & Kwon (2014), the authors used sampled-data control strategy for exponential synchronization for the neural networks with Markovian jumping parameters and time varying delays. Rakkiyappan, Sakthivel, Park, & Kwon (2013) discussed state estimation for Markovian jumping fuzzy cellular neural networks with probabilistic time-varying delays with sampled-data.

The purpose of this paper is to give a comprehensive analysis on out-synchronization of the discrete-time recurrently asymmetrically connected time-varying neural networks. We propose two schemes of discretizations, named centralized and decentralized discretization respectively, and present sufficient conditions for the global out-synchronization. The common step size for every neuron in centralized discretization but in decentralized discretization process, the distributed step size for each neuron is used to guarantee that any two trajectories from different initial values converge together.

## 2 Preliminaries and problem formulation

In this section, we provide the models of asymmetric recurrent neural networks with data-sampling, and some notations. The continuous-time version of the recurrent connected neural networks is described by the following differential equations

 dui(t)dt= −γi(t)ui(t)+n∑j=1aij(t)gj(uj(t))+Ii(t), (1)

where , and are piece-wise continuous and bounded, , and satisfies

 0≤gi(x)−gi(y)x−y⩽Gi (2)

for all , where is a constant and .

In the centralized data-sampling strategy, the continuous-time system (1) is rewritten as

 dui(t)dt=−γi(t)ui(tk)+n∑j=1aij(t)gj(uj(tk))+Ii(t) (3)

for . The increasing time sequence ordered as is uniform for all the neuron . Each neuron broadcasts its state to its out-neighbours and receives its in-neighbours’ states information at time .

Comparatively, in the decentralized data-sampling strategy, Eq. (1) is rewritten as the following push-based decentralized system

 dui(t)dt=−γi(t)ui(tik)+n∑j=1aij(t)gj(uj(tjk))+Ii(t) (4)

for . The increasing time sequence order as is distributed for the neuron . Every neuron pushes its state information to its out-neighbours at time when it updates its state. It receives its in-neighbours’ state information at time when its neighbour neuron renews it state.

To begin the discussion, we give the following three norms of and recall the definition of out-synchronization proposed in Wu, Zheng, & Zhou (2009).

###### Definition 1

Let be a positive constant and we can define three generalized norms as follow

1. norm:

2. norm:

3. norm:

where is a vector.

###### Definition 2

Consider any two trajectories and starting from different initial values and of the following system

 dx(t)dt=f(x(t),t). (5)

The system (5) is said to achieve out-synchronization if there exists a controller for the two trajectories and such that

 limt→+∞∥∥u(t)−v(t)∥∥=0.

Other major notations which will be used throughout this paper are summarized in the following definition.

###### Definition 3

Let be a positive constant and then we define

 μ1,j(ξ,t)= γj(t)−Gj(ajj(t))+−Gj∑i≠jξiξj∣∣aij(t)∣∣ μ2,j(ξ,t)= γi(t)−Gi(ajj(t))+ −12∑j≠i[Gj∣∣aij(t)∣∣+Giξjξi∣∣aji(t)∣∣] μ∞,j(ξ,t)= {γi(t)−Gi(ajj(t))+−Gi∑j≠iξjξi∣∣aij(t)∣∣

and

 ν(t)=maxj=1,⋯,n {γj(t)−Gj(ajj(t))−}

where and .

Because of the boundedness of the functions, it can be seen that and are bounded for all . That is, there exist positive constants and such that

 supt∈[0,+∞)ν(t)⩽M, supt∈[0,+∞)μm,j(ξ,t)≤Nm

with .

## 3 Structure-dependent data-sampling principle

In this section, we provide several the structure-based data-sampling rules for the next triggering time point at which the neurons renew their states and the control signals.

### 3.1 Structure-dependent centralized data-sampling

For any neuron , consider two trajectories and of the system (3) starting from different initial values. Denote with . And it holds

 dwi(t)dt=−γi(t)wi(tk)+n∑j=1aij(t)hj(tk) (6)

where for all , and .

The following theorem gives conditions that guarantee the system (3) reaches out-synchronization via - norm.

###### Theorem 1

Let and be constants with and . Suppose that there exist , such that for all and . Set an increasing time-point sequence as

 tk+1=supτ⩾tk{τ:minj=1,⋯,n∫ttkμ1,j(ξ,s)ds≤ϵc, ∀ t∈(tk,τ]} (7)

. Then the system (3) reaches out-synchronization.

Proof.   From the condition , one can see that

 ∫ttkμ1,j(ξ,s)ds≥ϵ0(t−tk), ∀ j

which implies that exists for all and . Thus, one can further see

 ∫ttkν(t)dt≤M(t−tk)≤Mϵc/ϵ0≤1 (8)

and

 ∫ttkμ1,j(ξ,t)dt≤N1(t−tk)≤N1ϵc/ϵ0≤2−ϵc (9)

for all and . Furthermore, we have

 ϵc/ϵ0≥tk+1−tk≥ϵc/N1.

Consider for each , and we have

 n∑i=1ξi∣∣wi(t)∣∣ = n∑i=1ξi∣∣∣wi(tk)+∫ttk˙wi(s)ds∣∣∣ = n∑i=1∣∣∣ξiwi(tk)−∫ttk[γi(s)−aii(s)mi(s)]dsξiwi(tk) +n∑j≠i∫ttk[aij(s)ξiξjmj(tk)ξjwj(tk))]ds∣∣ ∣∣

with

 mj(tk)=⎧⎨⎩hj(tk)wj(tk)wj(tk)≠00wj(tk)=0

which implies for all and , according to (2). Note

 (aii(s))−Gi≤aii(s)mi(tk)≤(aii(s))+Gi.

From (8), one can see

 ∫ttk[γ(s)−aii(s)mi(tk)]ds≤∫ttk[γ(s)−(aii(s))+Gi]ds ≤∫ttkμ(s)ds≤M(tk+1−tk)≤1

 0≤1−∫ttk[γ(s)−aii(s)mi(tk)]ds ≤1−∫ttk[γ(s)−(aii(s))+Gi]ds (10)

Then, it follows

 Missing or unrecognized delimiter for \big +∑i≠jξiξj∫ttk∣∣aij(s)∣∣ds×Gjds}ξj∣∣wj(tk)∣∣ =n∑i=1∣∣∣1−∫ttkμ1,j(ξ,s)ds∣∣∣ξj|wj(tk)| (11)

The last equality holds due to (10). Thus, according to the rule (7) and (9), which implies

 1−ϵc≥1−∫tk+1tkμ1,j(ξ,s)ds≥−1+ϵc, ∀ j

since the equality in (7) occurs at , thus we have

 n∑i=1ξi∣∣wi(tk+1)∣∣⩽(1−ϵc)n∑i=1ξi∣∣wi(tk)∣∣

which implies

 limtk→+∞∥∥w(tk)∥∥1,ξ=0.

In addition, for each , from the rule (7) and the condition , inequality (11) implies that for each , . Hence, it holds

 limt→+∞∥∥w(t)∥∥1,ξ=0.

The out-synchronization of system (3) is proved.

The proofs of the following results are analog to Theorem 3 but via and norm. Their proofs are similar to that of Theorem 3, which can be found in Zheng, Chen, & Lu (2015) and so neglected in the present paper.

###### Proposition 1

Let and be constants with and . Suppose that there exist , such that for all and . Set an increasing time-point sequence as

 tk+1=supτ⩾tk {τ:minj=1,⋯,n∫ttkμ2,j(ξ,t)ds≤ϵc, ∀ t∈(tk,τ]} (12)

. Then the system (3) reaches out-synchronization.

###### Proposition 2

Let and be constants with and . Suppose that there exist , such that for all and . Set an increasing time-point sequence as

 tk+1=maxτ⩾tk{τ :minj=1,⋯,n∫ttkμ∞,j(ξ,t)ds≤ϵc, ∀ t∈(tk,τ]} (13)

. Then the system (3) reaches out-synchronization.

###### Remark 1

From the proof, one can see , which excludes the Zeno behaviours for the rules (7,12,13).

To explain the independence of the results via three norms, we give out the following example. Denote

 L1=⎡⎢ ⎢ ⎢ ⎢⎣γ1(t)−G1a+11(t)−G1ξ2ξ1∣∣a21(t)∣∣γ2(t)−G2a+22(t)−G2ξ1ξ2∣∣a12(t)∣∣⎤⎥ ⎥ ⎥ ⎥⎦ L2=⎡⎢ ⎢ ⎢ ⎢ ⎢⎣γ1(t)−G1a+11(t)−12G1ξ′2ξ′1∣∣a21(t)∣∣−12G2∣∣a12(t)∣∣γ2(t)−G2a+22(t)−12G2ξ′1ξ′2∣∣a12(t)∣∣−12G1∣∣a21(t)∣∣⎤⎥ ⎥ ⎥ ⎥ ⎥⎦.

Let with

 γ(t)=[2.10480.9234]A(t) =[   1.0235   0.2538   0.5014−0.1526]

when and

 γ(t)=[2.10480.9234]A(t) =[−0.3253   0.4384−2.0341−0.1526]

when .

In the first time interval , we have found that when

 [ξ1ξ2]=[0.89020.3562][ξ′1ξ′2]=[0.34790.7727]

it holds

 L1=[0.87860.2901]>0L2=[0.39510.6210]>0

where . In the second time interval , we can find that when

 [ξ1ξ2]=[0.71820.3570]£¬

it follows

 L1=[1.09200.0429]>0.

However, to maintain , we have to solve the following inequalities

 ⎧⎪ ⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪ ⎪⎩γ1(t)−G1a+11(t)−12G1ξ′2ξ′1∣∣a21(t)∣∣−12G2∣∣a12(t)∣∣>0γ2(t)−G2a+22(t)−12G2ξ′1ξ′2∣∣a12(t)∣∣−12G1∣∣a21(t)∣∣>0

that is

 ⎧⎪ ⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪ ⎪⎩1.8860−1.0189ξ′2ξ′1>0−0.0954−0.2188ξ′1ξ′2>0.

One can see that there is no such solution of and .

Hence, the conditions (7), (12) are uncorrelated. By using the similar method, we can obtain that the three conditions are pairwise independent. Therefore, we can assert that the results via three norms are independent.

### 3.2 Structure-dependent push-based decentralized data-sampling

For each neuron , consider two trajectories and of system (4) starting from different initial values. Denote with . It follows

 dwi(t)dt=−γi(t)wi(tik)+n∑j=1aij(t)hj(tjk) (14)

where for all , and .

The following theorem and propositions give conditions that guarantee the convergence of system (14) via three generalized norms (, and ).

###### Theorem 2

Let and be constants with and . Suppose that there exist such that for all and . Set as the triggering time points as

 tjk+1=supτ⩾tjk{τ Missing or unrecognized delimiter for \Big ×∫τtik∣∣aij(s)∣∣ds⩾ϵd, ∀ t∈(tjk,τ]} (15)

for and . Then the system (4) reaches out-synchronization.

Proof.   For each , let . Similar to the arguments up to (11) in the proof of Theorem 1, one can derive the following inequality immediately:

 n∑i=1ξi|wi(t)|≤n∑i=1∣∣∣1−∫ttjkj(t)μ1,j(ξ,s)ds∣∣∣ξj|wj(tjkj(t))| (16)

From the arguments of (9), one can conclude

 1≥1−∫ttjkj(t)μ1,j(ξ,s)ds≥−1+ϵc, ∀ j. (17)

in an analog way.

Let be an increasing sequence such that and , which implies that for each neuron , equality in the rule (2) occurs at least once. Thus, we have

Consider for any neuron at triggering time where , and we have

 n∑i=1ξi∣∣wi(tik+1)∣∣ = n∑i=1sign(wi(tik+1))ξi[wi(tik)+∫tik+1tik˙wi(s)ds] = n∑i=1sign(wi(tik+1))ξiwi(tik)+n∑i=1sign(wi(tik+1))ξi ×∫tik+1tik[−γi(s)wi(tik)+n∑j=1aij(s)hj(wj(tjk))]ds = n∑i=1sign(wi(tik+1))ξiwi(tik) −n∑i=1sign(wi(tik+1))ξiwi(tik)∫tik+1tikγi(s)ds +n∑i=1sign(wi(tik+1))ξihi(wi(tik))∫tik+1tika+ii(s)ds +∑i≠jsign(wi(tik+1))ξin∑j=1hj(wj(tjk))∫tik+1tikaij(s)ds.

By the inequality (2), it holds

 n∑i=1ξi∣∣wi(tik+1)∣∣⩽ n∑j=1{1−∫tjk+1tjk[γj(s)−Gja+jj(s)]ds +Gj∑i≠jξiξj∫tik+1tik∣∣aij(s)∣∣ds}ξj∣∣wj(tjk)∣∣ ⩽ (1−ϵd)n∑j=1ξj∣∣wj(tjk)∣∣

Based on the triggering rule (2), we can obtain

 n∑i=1ξi∣∣wi(tik+1)∣∣⩽(1−ϵd)n∑i=1ξi∣∣wi(tik)∣∣

which means

 limtik→+∞∥∥w(tik)∥∥1=0.

For any time , the state becomes

 n∑i=1ξi∣∣wi(t)∣∣ ⩽ n∑j=1{1−∫ttjk[γj(s)−Gja+jj(s)]ds +Gj∑i≠jξiξj∫ttik∣∣aij(s)∣∣ds}ξj∣∣wj(tjk)∣∣ ⩽ n∑j=1{1−∫ttjk[γj(t)−Gja+jj(t)]dt+Gj∑i≠jξiξj ×[∫ttik∣∣aij(s)∣∣ds+∫τt∣∣aij(s)∣∣ds]}ξj∣∣wj(tjk)∣∣ ⩽ (1−ϵd)n∑i=1ξi∣∣wi(tik)∣∣

where . Thus,

 ∥∥w(tik+1)∥∥1⩽∥∥w(t)∥∥1⩽(1−ϵd)∥∥w(tik)∥∥1

for any and , which implies

 limt→+∞∥∥w(t)∥∥1⩽limtik→+∞∥∥w(tik)∥∥1=0.

The proof for the out-synchronization of system (4) is completed.

###### Proposition 3

Let be a constant and . Set as the time points such that

 tik+1=maxτ⩾tik {τ:∫ttik[γi(s)−Gia+ii(s)−12∑j≠iGj∣∣aij(s)∣∣]ds −12Gi∑j≠iξjξi∫τtjk∣∣aji(t)∣∣dt⩾ϵd, ∀ t∈(tik,τ]} (18)

for and . Then the system (4) reaches out-synchronization.

###### Proposition 4

Let be a constant and . Set