Observability of Nonlinear Complex Networks in the Presence of Symmetries:A Graphical Approach

Observability of Nonlinear Complex Networks in the Presence of Symmetries:
A Graphical Approach

Afroza Shirin, Dionicio F. Rios, and Francesco Sorrentino *We gratefully acknowledge support from the National Science Foundation through NSF grant CMMI- 1400193, NSF grant CRISP- 1541148 and the Office of Naval Research through ONR Award No. N00014-16-1-2637.Department of Mechanical Engineering, The University of New Mexico, Albuquerque, New Mexico, 87131 ashirin@unm.edu
Abstract

Reconstructing the states of the nodes of a dynamical network is a problem of fundamental importance in the study of neuronal and genetic networks. An underlying related problem is that of observability, i.e., identifying the conditions under which such a reconstruction is possible. In this paper we study observability of complex dynamical networks, where, we consider the effects of network symmetries on observability. We present an efficient algorithm that returns a minimal set of necessary sensor nodes for observability in the presence of symmetries.

I Introduction

Observability is the ability to deduce the state variables of a system for which only the outputs, often coinciding with a subset of the states, can be measured. There are different methods that can be used to determine observability, ranging from an algebraic test [2] to a graphical approach (GA) [3]. The last allows to determine a set of output states from which one can infer information about the rest of the states by analyzing the properties of an inference graph. In [1], this method was applied to complex networks, for which the ability to directly sense all of the nodes is typically unavailable. Examples of interest are genetic and neural networks, which could be reconstructed by observing a limited number of output states if these provide observability.

In biological networks, such as genetic or neuronal networks, an important problem is that of reconstructing the network architecture and the time evolution of the connections from existing data. For these applications, a typical limitation is the cost and the possibility of sensing the states of the individual nodes. The network inference problem and its application to the genome [4, 5, 6, 7, 8] has received significative attention over the last decade. References [9, 10] review and compare the more popular existing approaches. An important question that is relevant to both genetic and neuronal networks, is what is the minimal amount of information needed to be able to reconstruct the state of these networks.

In this paper we will be concerned with the conditions for observability of a complex dynamical network, which provides a conditio sine qua non for implementing any type of estimation strategy.

Ii Definitions

Definition 1

Observability of Non-Linear System. Consider a nonlinear system represented by,

(1)

where is the system state, is the system input, and

(2)

is the system output, , . If the initial-state vector can be found from and measured over a finite interval of time, the system in Eq. (1) is said to be observable.

The Lie derivative of h with respect to f is defined to be

(3)

where the function f is of the form:

(4)

Then the Lie derivative is:

(5)

and the result of this derivative operation is a scalar.

Two states and are distinguishable if there exists an input function such that:

(6)

where for . This implies that the system is locally observable about if there exists a neighborhood of such that every x in that neighborhood other than is distinguishable from .

A test for local observability about is that the matrix:

(7)

is of rank . If the system is locally observable about any , the system is said to be observable. Note that testing observability may require that the rank observability test is repeated for potentially infinitely many points . Alternatively, one may recur to using a symbolic method for calculating the rank of (7).

Definition 2

Observability of Linear System. When a system is linear time invariant [13],

(8)

the condition for observability in Eq. (7) reduces to the Kalman rank condition, i.e., the matrix

must have rank . In the case that Eq. (8) represents the dynamics of a dynamical network, the matrix is called the adjacency matrix.

Definition 3

Network Symmetries. We consider the group of network symmetries (or automorphisms of a graph). Each symmetry is represented by a permutation matrix that permutes with the adjacency matrix , i.e., . Given a matrix the set of all the network symmetries can be computed by using available efficient discrete algebra routines [23].

Iii A Graphical Approach to Observability

The algebraic methods introduced in Sec. II to determine observability of a nonlinear system can get quite complicated as , the number of states, increases. In [1] a graphical approach is proposed, which reduces to the analysis of an inference diagram (ID) that describes the interactions between the state variables. This approach presents computational advantages. The inference diagram is a graphical representation of the interactions between state variables: consider two variables and , if variable appears in the dynamical equation for the evolution of variable ( affects ), then there will be a directed edge from state to state in the inference diagram. Note here that the convention in this paper to construct the ID is different from that in Ref. [1], namely if state affects state we draw a directed edge from to in the ID (while in [1] we would have a directed edge from to ).

Fig. 1: Inference diagram and its partition in strongly connected components. The ID is used to identify strongly connected components (SCCs) and root strongly connected components (root SCCs). In the figure (non-root) SCCs are delimited by dashed ovals and root SCCs by solid ovals. In order to achieve observability, at least one node needs to be selected from each root SCC.

The ID contains information that can be used to identify the sensor nodes required to achieve observability. Consider a directed graph , where is the set of nodes and is the set of directed edges. A path from node to node in is an alternating sequence of the type of nodes and edges that belong to and , respectively. The entries of the adjacency matrix are , if and , if . Two nodes and in are path equivalent if there is a path from to and a path from to . Path equivalence partitions into maximal disjoint sets of path equivalent nodes. These sets are called the strongly connected components (SCCs) of the graph [20]. This means that, in a SCC, there exists a path from any node to all other nodes in that SCC. The sensor nodes predicted from the GA analysis are chosen from the root SCCs. A root SCC is a SCC which has no outgoing links, meaning that their information cannot be concluded from any other node in the inference diagram. However, root SCCs do have incoming links from other SCCs. This implies that information of nodes within other SCCs, can be inferred from the root SCCs. In order to achieve observability it is necessary that at least one sensor node (any node) is selected from each root SCC [1]. In the case presented in Fig.1, since there are three root SCCs, three sensor nodes must be selected to achieve observability. Note that if the diagram is undirected and connected, choosing any node in the graph as sensor node would provide observability.

When applied to the ID of a network of coupled dynamical systems, GA may be used to identify a set of sensor nodes [1] to be monitored to yield observability. In what follows, we briefly review the main results from [1]:

R1) The set of sensor nodes identified by the GA are necessary for observing any linear or nonlinear dynamical system.

R2) If the dynamics is linear the maximal matching method introduced in [19] returns the set of nodes that are necessary, but also the minimum set sufficient for observability.

R3) For nonlinear systems, under general assumptions, the sensor set predicted by GA is not only necessary but also sufficient (for more details the reader is directed to the supplementary information of Ref. [1]).

Result R3) is supported in [1] by using a mix of the structural controllability analysis [14] and an extensive numerical analysis. The sufficiency argument is based on the assumptions that () all the nodes are accessible from the sensors which is guaranteed by placing the sensors in the root SCCs and () for each node , i.e., the dynamics of each node is affected by the state of the node itself.

Assumption. Throughout this paper we assume , . Note that for the sake of simplicity, in the figures of this paper we omit to represent self-loops at each node. However, though not graphically shown, those self-connections are present.

We depart from result R3, which applies to general nonlinear networks, and extend it to consider the case where network symmetries are present. Our results are mainly meant to apply to large nonlinear complex networks in the presence of network symmetries. Whether these symmetries are a generic feature or not of complex networks, it is a matter of discussion. While an analysis of model networks seems to indicate that network symmetries may sometimes emerge in small numbers in model networks [21], it has also been reported that they are pervasive in the case of many real networks [22]. For example, in Ref. [22] it was found that the Human B cell genetic interactions network is characterized by approximately symmetries, the C. elegans genetic interaction network is characterized by approximately symmetries, and the network of the Internet at the Autonomous System level by approximately symmetries (!) Hence, the presence of network symmetries appears to be a typical feature of several real networks.

Iv Symmetries and Observability

In this section we will analyze cases in which symmetries occur and their effect on observability. In what follows, we will present several inference diagrams with symmetries. For each individual diagram, we will use GA to determine SCCs as well as root SCCs and we will methodically examine the conditions that must be met for observability.

While we do not attempt to present an exhaustive treatment of the effects of symmetries on the observability of networks, we will present several examples and we will propose and show that: (i) the GA method presented in the previous section fails in the presence of symmetries and (ii) necessary conditions for observability can be restored by selecting appropriate additional sensor nodes. Our main goal in this section is to show that a GA can be conveniently exploited not only to find (root) SCCs, but also to detect symmetries. Given a network, available software is capable to efficiently provide all the network symmetries, e.g. SAGE [23]. Here we propose that SCCs analysis in conjunction with the analysis of symmetries in the ID may allow us to identify the necessary set of sensor nodes to achieve observability.

Fig. 2: Inference diagram representation of four networks displaying symmetries. SCCs and root SCCs are delimited by dashed ovals and solid ovals, respectively. For each network, a minimal set of sensor nodes to achieve observability is determined (the sensor nodes are those pointed by small purple arrows).

In what follows, we will specify the network nonlinear equations (1) and (2) to be,

(9)

where is the scaler output function of node , , and is a symmetric function of its variables.

Definition 4

A symmetry for the set of equations (9) is a permutation between tuples of nodes for which the dynamics is equivariant.

It can be easily shown that a network symmetry (Definition 3) yields an equation symmetry (Definition 4) for Eq. (9). As an example, we consider the following nonlinear dynamic equations for the network in Fig. 2a,

The GA method predicts node 1 is the only sensor node. By monitoring , we check the observability of the nonlinear system using the algebraic method described in Sec. II. The Lie derivatives of are:

For arbitrary x and the constant , the observability matrix,

where, , , , , , . A symbolic rank calculation shows that , i.e, the system is not observable by only sensing node 1. Note that because the entries of the matrix are not independent of each other, the theory of structural controllability and observability [14] does not exactly apply to this case [1].

Now we consider the case that we choose node 1 and 2 as sensor nodes. By monitoring , we check the observability of the system. The Lie derivatives of y are:

For arbitrary x and the constant , the observability matrix,

where, , , , , , , , , , , , . It can be shown using symbolic rank calculation that , i.e, the system is observable by sensing nodes 1 and 2.

From Fig. 2a we see that swapping nodes 2 and 3 does not affect the dynamics of node 1. In Fig. 2b, any permutation of the ordered pair of nodes (2,3), (4,5), (6,7) and (8,9) leaves the dynamics of node 1 unaltered. In Fig. 2c, a permutation of the ordered triplets of the nodes (2,3,4) and (5,6,7) leaves the dynamics of node 1 unaltered. Fig. 2d, shows four independent symmetries occurring in the same diagram, namely, swapping nodes 4 and 5 would leave the dynamics of the rest of the network unaltered, as well as swapping nodes 2 and 3, and 6 and 7. Moreover, another symmetry is generated by the pairs (2,3) and (6,7). Hereafter, we will discuss how the symmetries of the diagrams in Fig. 2 limit their observability and how additional sensor nodes can be selected to enable it. In Fig. 2a, one can see a network composed of three nodes 1,2, and 3, where node 1 forms a root SCC and nodes 2 and 3 form individual SCCs. Moreover, nodes 2 and 3 are symmetric. This means that monitoring node 1 alone is not sufficient to achieve observability, while monitoring node 1 and either node 2 or 3 will provide observability. In Fig. 2b, the symmetric ordered pairs (2,3), (4,5), (6,7), (8,9) form individual SCCs. To achieve observability we need to monitor node 1 along with at least three other nodes and each of these extra nodes need to be selected from three different symmetric pairs. The third case shown in Fig.2c is that of a network in which the ordered triples (2,3,4) and (5,6,7) are symmetric. Similarly to the case presented in Fig. 2b, for observability, we need to monitor node 1 and at least one other node from either ordered triplet. In the forth case, Fig. 2d, there are symmetries both inside the root SCC, as well as in the other two SCCs. A necessary set of sensor nodes for this case will be: either node 4 or 5, either node 2 or 3 and either node 6 or 7. An algorithm to identify the minimal set of necessary sensor nodes for observability in the presence of symmetries will be presented in the Sec. V.

Fig. 3: a) Inference diagram representation of a network with symmetries between two SCCs. b) Inference diagram representation of a network with symmetries between two root SCCs. c) Inference diagram representation of a network with symmetries between the SSCs. d) Inference diagram representation of a network with symmetries between two SCCs.

We continue to analyze cases in which symmetries occur. We see from Fig. 3a, that nodes 2 and 3, are symmetric with respect to any node in the root SCC (if they are symmetric with respect to one they are symmetric with respect to all). In this case, for observability, we need to monitor either one of the nodes in the root SCC along with either node 2 or node 3. The case in Fig. 3a is that of a symmetry between two nodes in two different SCCs. By definition of SCC and root SCC, a node in a SCC and a node in a root SCC cannot be symmetric. However, there can be symmetries between nodes that belong to the same root SCC and also between nodes in different root SCCs (as shown in Fig. 3b). In Fig. 3b, the ordered triplets (1,2,3) and (4,5,6) are symmetric and there exist symmetries within each root SCC, nodes 2 and 3 are symmetric as well as nodes 5 and 6. In this case, by monitoring a node from each root SCC, the symmetries inside each root SCC are resolved and observability is achieved. The same symmetries are present in Fig. 3c, but in this case the triplets (1,2,3) and (4,5,6) contain nodes from both SCCs and root SCCs. However, nodes belonging to non-root (root) SCCs can only be swapped with other nodes belonging to non-root (root) SCCs.

In Fig. 3d, nodes 2 and 4 are symmetric. Hence, as in the previous cases, monitoring only the root SCC is not sufficient for observability. Because of the symmetry generated by nodes 2 and 4, we need to monitor the root SCC node 1 and either node 2 or 4.

V Algorithm to Determine the Minimal Set of Necessary Sensor Nodes for Observability

In the cases presented in Figs. 2 and 3, in which symmetries are present, monitoring one node from each root SCC is not sufficient to obtain observability. In general, additional nodes need to be monitored. Here we focus on the effect of the network symmetries and present an algorithm that, given a ID with symmetries, returns a minimal set of necessary sensor nodes for observability. First we present the algorithm for the case that there are no symmetries. Consider a diagram with root SCCs. Each root SCC can be described by a set as follows,

(10)

where is the number of elements in the set . Then as we need to choose a node from each root SCC, and as SCCs are disjoint sets, the minimal set of necessary sensor nodes for observability , where is any element from the set [1].

In the case of a ID with symmetries, let a symmetry be represented by the following list of -tuples,

(11)

. Here, is the total number of tuples in (note that any permutation of tuples belonging to the same symmetry is equivariant with respect to the network dynamics (9)) and is the number of elements of the tuples in . We call the number of symmetry and the length of symmetry . Without loss of generality, we label the symmetries so that , i.e., in order of nondecreasing length.

Now we introduce the definition of internal and external symmetry. A symmetry is internal to another symmetry , (or, equivalently, is external to ) if there exists a tuple of , say , such that all the entries of the tuples are also entries of , .

In order to achieve observability, we need to include in : (i) at least one element from each set , and (ii) at least one element from different tuples of , , . Given a set of sensor nodes , if condition (ii) is verified for a given , we say that the root SCC is resolved, and if condition (ii) is verified for a given , we say that symmetry is resolved.

We now present an efficient algorithm to obtain a minimal set of necessary sensor nodes for observability, in the presence of symmetries. Our algorithm is based on the observation that if a symmetry does not present internal symmetries, it will be resolved by choosing any elements from different tuples. Otherwise, if a symmetry presents internal symmetries, then resolving the internal symmetries ensures that the external symmetry is resolved as well. As the algorithm proceeds by considering symmetries of nondecreasing length, it first resolves internal symmetries, which automatically also resolves the external ones.

The algorithm consists of the following steps:

  1. Consider symmetries in the order of nondecreasing length. For each symmetry that has not been already resolved, select any elements from different tuples and add them to . Terminate when all symmetries have been resolved.

  2. Consider one by one all the root strongly connected components. For each root strongly connected component that has not been already resolved, select any element and add it to .

As an example, we apply our algorithm to the ID representation in Fig. 4. This network is directed and is composed of 6 non-root SCCs and 1 root SCC. We computed the network symmetries by using the SAGE routine AUTOMORPHISM_GROUP. The symmetries for this network are given by , , , , , and . In this case, and ; moreover, , , and . Note that the symmetries , and are all internal to . According to our algorithm, we resolve the internal symmetries first. To do that, we choose one node from each one of the symmetries , and and add them to . For example, a possible choice is to set . We see that with this , we have automatically resolved the external symmetry . In fact, contains different nodes from different tuples of . We then need to resolve by choosing one node from either the tuple or . Say we choose , then the updated set is . The network has one root SCC. According to step 2 of the algorithm, we need to choose one node from this root SCC, unless one is already included in . Here node 1 is the only option and it has not been already selected. Then, the final set is . Note that this set is minimal, as eliminating any node from would correspond to loss of observability.

Fig. 4: The algorithm described in the text to obtain , the minimal set of nodes for observability, has been applied to the network in the figure.

We now apply our algorithm to two different real networks. First we consider the Zachary’s Karate club which represents friendship relations among 34 members [24, 25]. Since the network is undirected and connected, the whole network coincides with a root SCC (see Fig. 5 ). The symmetries for this network have been evidenced in Fig. 5, where same shapes (ovals, rectangles and circles) have been used to indicate nodes that are symmetric. By applying the algorithm, we find a minimum set of nodes necessary for observability .

Fig. 5: Undirected network for Zachary’s Karate Club [24, 25]. The ovals, circles and rectangles are presenting symmetries.

We now consider the Caenorhabditis Elegans neuronal networks [26] formed of neurons. These can be linked through connections of two different types, either chemical synapses or gap junctions. Here we focus on the network formed of chemical synapses alone, which is directed. This network is not fully connected, it has 42 SCCs, 11 of which are root SCCs. This network also presents some symmetries. A minimal set of necessary sensor nodes for observability of the chemical synapses network can be found by using our algorithm and the results of our computations are presented in Table I.

No. of nodes Name of The Neurons Neuron Label No.
1 IL2DL 1
2 IL2DR 6
3 ASIL 76
4 ASIR 90
5 AINL 120
6 SDQR 179
7 DB05 204
8 AS08 220
9 PVDR 227
10 DVB 253
11 PLNR 260
12 PHCR 274
13 PLML 279
TABLE I: The minimal set of necessary sensor nodes for observability of chemical synapse network of C. elegans(directed network).

Vi Conclusions

In this paper we have presented an efficient algorithm to determine a minimal set of necessary sensor nodes for observability of complex network in the presence of network symmetries. These symmetries have been shown to be a typical property of several real networks [22]. We have shown that observability requires by a proper selection of the sensor nodes, which takes into account the network symmetries.

Our method based on a graphical approach could be used to uncover the conditions required for the observation of neural and genetic networks, for which access to all the network states is limited and knowledge about the connectivity pattern is crucial. For example, this is relevant to the estimation of neuronal [11, 12] and genetic networks [4, 5, 6, 7, 8, 9, 10] and to numerous engineering applications [27, 28, 29, 30, 31].

References

  • [1] Y.-Y. Liu, J.-J. Slotine, and A.-L. Barabási, “Observability of complex systems,” Proceedings of the National Academy of Sciences, vol. 110, no. 7, pp. 2460–2465, 2013.
  • [2] B. N. Datta, Numerical methods for linear control systems: design and analysis.   Academic Press, 2004, vol. 1.
  • [3] S. Diop and M. Fliess, “Nonlinear observability, identifiability, and persistent trajectories,” in Decision and Control, 1991., Proceedings of the 30th IEEE Conference on.   IEEE, 1991, pp. 714–719.
  • [4] A. de la Fuente, P. Brazhnik, and P. Mendes, “Linking the genes: inferring quantitative gene networks from microarray data,” TRENDS in Genetics, vol. 18, no. 8, pp. 395–398, 2002.
  • [5] N. Friedman, “Inferring cellular networks using probabilistic graphical models,” Science Signalling, vol. 303, no. 5659, p. 799, 2004.
  • [6] A. De la Fuente and D. Makhecha, “Unravelling gene networks from noisy under-determined experimental perturbation data,” Systems biology, vol. 153, no. 4, p. 257, 2006.
  • [7] J. Faith, B. Hayete, J. Thaden, I. Mogno, J. Wierzbowski, G. Cottarel, S. Kasif, J. Collins, and T. Gardner, “Large-scale mapping and validation of escherichia coli transcriptional regulation from a compendium of expression profiles,” PLoS biology, vol. 5, no. 1, p. e8, 2007.
  • [8] D. Marbach, C. Mattiussi, and D. Floreano, “Replaying the evolutionary tape: Biomimetic reverse engineering of gene networks,” Annals of the New York Academy of Sciences, vol. 1158, no. 1, pp. 234–245, 2009.
  • [9] R. De Smet and K. Marchal, “Advantages and limitations of current network inference methods,” Nature Reviews Microbiology, vol. 8, no. 10, pp. 717–729, 2010.
  • [10] D. Marbach, R. Prill, T. Schaffter, C. Mattiussi, D. Floreano, and G. Stolovitzky, “Revealing strengths and weaknesses of methods for gene network inference,” Proceedings of the National Academy of Sciences, vol. 107, no. 14, pp. 6286–6291, 2010.
  • [11] J. White, E. Southgate, J. Thomson, S. Brenner, J. White, E. Southgate, J. Thomson, and S. Brenner, “The structure of the nervous system of the nematode caenorhabditis elegans,” Philosophical Transactions of the Royal Society of London. B, Biological Sciences, vol. 314, no. 1165, pp. 1–340, 1986.
  • [12] O. Sporns, G. Tononi, and R. Kötter, “The human connectome: a structural description of the human brain,” PLoS computational biology, vol. 1, no. 4, p. e42, 2005.
  • [13] R. E. Kalman, “Mathematical description of linear dynamical systems,” Journal of the Society for Industrial and Applied Mathematics, Series A: Control, vol. 1, no. 2, pp. 152–192, 1963.
  • [14] C.-T. Lin, “Structural controllability,” IEEE Transactions on Automatic Control, vol. 19, no. 3, pp. 201–208, 1974.
  • [15] K. Murota, Matrices and matroids for systems analysis.   Springer Science & Business Media, 2009, vol. 20.
  • [16] R. Shields and J. Pearson, “Structural controllability of multiinput linear systems,” IEEE Transactions on Automatic control, vol. 21, no. 2, pp. 203–212, 1976.
  • [17] K. Glover and L. Silverman, “Characterization of structural controllability,” IEEE Transactions on Automatic control, vol. 21, no. 4, pp. 534–537, 1976.
  • [18] C. Sueur and G. Dauphin-Tanguy, “Structural controllability/observability of linear systems represented by bond graphs,” Journal of the Franklin Institute, vol. 326, no. 6, pp. 869–883, 1989.
  • [19] Y.-Y. Liu, J.-J. Slotine, and A.-L. Barabási, “Controllability of complex networks,” Nature, vol. 473, no. 7346, pp. 167–173, 2011.
  • [20] E. Nuutila and E. Soisalon-soininen, “On finding the strongly connected components in a directed graph,” Information Processing Letters, vol. 49, 1994.
  • [21] L. M. Pecora, F. Sorrentino, A. M. Hagerstrom, T. E. Murphy, and R. Roy, “Cluster synchronization and isolated desynchronization in complex networks with symmetries,” Nature Communications, vol. 5, 2014.
  • [22] B. D. MacArthur, R. J. Sánchez-García, and J. W. Anderson, “On automorphism groups of networks,” Discrete Applied Mathematics, vol. 156, p. 3525, 2008.
  • [23] W. Stein et al., Sage Mathematics Software (Version 7.5.1), The Sage Development Team, YYYY, http://www.sagemath.org.
  • [24] W. W. Zachary, “An information flow model for conflict and fission in small groups,” Journal of anthropological research, pp. 452–473, 1977.
  • [25] M. Girvan and M. E. Newman, “Community structure in social and biological networks,” Proceedings of the National Academy of Sciences, vol. 99, no. 12, pp. 7821–7826, 2002.
  • [26] L. R. Varshney, B. L. Chen, E. Paniagua, D. H. Hall, and D. B. Chklovskii, “Structural properties of the caenorhabditis elegans neuronal network,” PLoS computational biology, vol. 7, no. 2, p. e1001066, 2011.
  • [27] R. Autariello, R. Dzakpasu, and F. Sorrentino, “Estimating the structure of small dynamical networks from the state time evolution of one node,” Physical Review E, vol. 87, no. 1, p. 012915, 2013.
  • [28] N. Bezzo, F. Sorrentino, and R. Fierro, “Decentralized estimation of topology changes in wireless robotic networks,” Proceedings of the American Control Conference, 2013.
  • [29] I. I. Hussein, F. Sorrentino, and R. Erwin, “Bayesian hybrid estimation of lti networked systems using finite set statistics,” Proceedings of the American Control Conference, 2013.
  • [30] F. Sorrentino and E. Ott, “Adaptive synchronization of dynamics on evolving complex networks,” Physical review letters, vol. 100, no. 11, p. 114101, 2008.
  • [31] ——, “Using synchronization of chaos for adaptive learning of network topology,” Physics Review E., no. 80, p. 056206, 2009.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
""
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
   
Add comment
Cancel
Loading ...
66251
This is a comment super asjknd jkasnjk adsnkj
Upvote
Downvote
""
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters
Submit
Cancel

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test
Test description