Modelling stochastic resonance in humans: the influence of lapse rate

Modelling stochastic resonance in humans: the influence of lapse rate

Jeroen J.A. van Boxtel j.j.a.vanboxtel@gmail.com [ School of Psychological Sciences and Monash Institute of Cognitive and Clinical Neurosciences, Monash University, Clayton 3800 Vic, Australia
July 11, 2019
Abstract

Adding noise to a sensory signal generally decreases human performance. However noise can improve performance too, due to a process called stochastic resonance (SR). This paradoxical effect may be exploited in psychophysical experiments, to provide additional insights into how the sensory system deals with noise. Here, I develop a model for stochastic resonance to study the influence of noise on human perception, in which the biological parameter of ‘lapse rate’ was included. I show that the inclusion of lapse rate allows for the occurrence of stochastic resonance in terms of the performance metric . At the same time, I show that high levels of lapse rate cause stochastic resonance to disappear. It is also shown that noise generated in the brain (i.e., internal noise) may obscure any effect of stochastic resonance in experimental settings. I further relate the model to a standard equivalent noise model, the linear amplifier model, and show that the lapse rate can function to scale the threshold versus noise (TvN) curve, similar to the efficiency parameter in equivalent noise (EN) models. Therefore, lapse rate provides a psychophysical explanation for reduced efficiency in EN paradigms. Furthermore, I note that ignoring lapse rate may lead to an overestimation of internal noise in equivalent noise paradigms. Overall, describing stochastic resonance in terms of signal detection theory, with the inclusion of lapse rate, may provide valuable new insights into how human performance depends on internal and external noise.

pacs:

]http://users.monash.edu/ jeroenv thanks:

I Introduction

The brain is an inherently noisy system; much of the brain’s activity is not driven by external stimulation, or by purposeful internal processes, but by seemingly random activity: noise. Noise poses a fundamental problem for information processing Von Neumann (1956); Shannon (2001) as it increases variability and limits the clarity of a signal. Yet, given the abundance of noise in neural processing, the brain still achieves remarkably stable perception, presumably because the brain adapted to its own noisiness and that of its inputs. Therefore, studying how the brain responds to noise may help reveal the internal workings of the brain.

Noise is often considered to limit optimal performance Von Neumann (1956); Shannon (2001). Indeed the very definition of the performance metric is signal-to-noise ratio, where increases in noise decrease Green and Swets (1974). While noise is expected to degrade performance in general, noise can actually improve performance. For example, noise can push a subthreshold signal — that would normally lead to chance performance in behavioral tasks— above threshold, and thereby lead to above-change level performance, an effect called stochastic resonance (SR) McDonnell and Abbott (2009). Note that SR here refers to any occasion where noise increases performance, and not just cases with periodic input McDonnell and Abbott (2009). SR causes optimal performance (i.e., highest detectability) to be reached at non-zero levels of noise. This has potentially important implications, because it suggests that inducing SR by adding noise can be used to boost performance of humans and machines. Because of this potential beneficial effect of noise, it is useful to study SR in more detail, specifically in relation to human performance.

In humans, the influence of noise on performance or perception is often investigated using paradigms in which external noise is added to the signal, and performance thresholds are measured Lu and Dosher (2008). A common method is called the equivalent noise (EN) paradigm, which is often used to estimate internal (brain-generated) noise Lu and Dosher (2008), and has its origins in engineering to measure noise dependence in electronic amplifiers North (1942). In the EN paradigm, the overall noise is assumed to be a combination of internal noise and externally added noise , whose variances add: . In experimental settings, various amounts of external noise are added to a signal and the effects on detection thresholds are mapped out. When , only internal noise limits performance. At high levels of external noise (), performance is determined by external noise. At intermediate amounts of noise, the threshold-versus-noise (TvN) curve shows an elbow separating the two regimes, which is located at the level where internal and external noise are equivalent; the location of the elbow thus represents the level of internal noise.

Although extensively used, the EN paradigm often only considers 2 parameters of interest (i.e., internal noise, and efficiency). Although several extensions on the basic EN paradigm exist Lu and Dosher (2008) they almost invariably assume that the brain is optimal at setting decisions thresholds. However, human observers do not always set decision thresholds optimally Green and Swets (1974). This sub-optimal behavior allows for SR Gong et al. (2002), which indeed has been reported in human observers Simonotto et al. (1997); Collins et al. (1996); Moss et al. (2004); Ward et al. (2002); Goris et al. (2008). The fact that SR occurs indicates that the effects of noise are more complicated than often modelled in EN approaches.

Stochastic resonance has been extensively investigated from an engineering point of view, but less so in humans. A good description of SR in humans is more complicated because the human sensory system presents some challenges in researching stochastic resonance. First, the noise may originate from one of many processing levels in the brain, from early sensory, to later decisional stages, each potentially having different effects on performance. Second, humans are not machines, and they suffer from attentional lapses (induced by, e.g., decreased arousal or vigilance). The influence of attentional lapses has not been studied in the stochastic resonance literature. To determine the optimal level of external noise to achieve optimal performance (i.e., maximal SR), these two specifically human challenges need to be better understood.

Here, I model the influence of noise in sensory signals and decision criteria. To provide further insight into the usefulness of SR as a measure of the influence of noise on human perception, I will compare this model to the EN paradigm Lu and Dosher (2008); Pelli (1981).

Ii A signal detection model of stochastic resonance

Previous work has shown that signal detection theory can provide a good framework for modeling stochastic resonance in human behaviour. Gong et al. Gong et al. (2002) used the following equations:

(1)
(2)
(3)

where H is the hit rate, and F is the false alarm rate, is a criterion (i.e., threshold), is the signal strength, is the mean noise strength, is the normal distribution with a certain mean ( or ) and variance (). The variable represents the noise in the system. This model produced SR for accuracy (), when .

This model also predicted that in the limit of zero noise, , another measure of (human) performance, increased to infinity. Although this may make sense when is interpreted as a signal-to-noise ratio, it does not when is considered a metric of human performance. As a measure of human performance, should drop to chance level when noise is decreased, just as accuracy does. Therefore, I extended the model presented in Gong et al. (2002), with another human characteristic, namely lapse rate .

(4)
(5)
(6)

where is an unbiased measure of performance, is the lapse rate () , is the inverse of the cumulative normal distribution function. The variable again represents the noise in the system, but for our purposes it is important to realise that this noise can have an external () or internal (brain-generated; ) origin, as is under control of the experimenter, while is an internal property of the system under study, i.e. the human participant.

The lapse rate represents the proportion of guesses that a participant is required to make due to not paying attention to the stimulus. When (i.e. sub-threshold) and is small, H and F approach , and approaches 0, as required for a measure of performance. Importantly, without the inclusion of , both H and F approach 0 as decreases, but F much faster then H, because , leading to increase drastically (see equation 6).

Figure 1: Dependence of on noise (), and signal strength (). Parameters are , and , while the different lines are solutions at different values of (indicated with the boxed numbers).

When , the model can produce SR for , as shown in Fig. 1 in which the dependence of on is shown for various levels of signal strength (indicated by the boxed numbers), with , and . When signal strength is below criterion (i.e., ) — which without SR would lead to chance performance — performance rises above chance (i.e., ) for intermediate levels of . This is a form of stochastic resonance. At small , does not increase to infinity (as was found by Gong et al. (2002)), but instead

(7)

This behaviour is consistent with human performance, where subthreshold signals (i.e., ) lead to chance performance, and supra-threshold performance is limited by attentional lapse rate.

Iii Detection thresholds

In psychophysical experiments, it is common to determine the stimulus strength at which performance reaches some predefined performance threshold at various levels of externally applied noise. Common thresholds in two-choice or 2 alternative forced choice tasks are 75% accuracy, and a of 1.

By solving eq. 6 for , one can calculate the threshold signal strength , and thus TvN curves. Using Mathematica® 10 it was found that

(8a)
(8b)

where erf and are the error function, and inverse error function, respectively. Note that we will use the superscript SR to refer to specifically in equation 8a. When referring to the threshold signal strength more generally, no superscript it used. In this paper, we will assume .

When , this equation reduces to a straight line

(9)

independent of the criterion, and without a possibility to yield SR. Rewriting eq. 9 yields , which is the signal-to-noise ratio (c.f., Gong et al., 2002).

Figure 2: Dependence of detection threshold on noise . All parameters are as for the reference line (and as in Fig. 1), except where otherwise specified. Reference: , , ; higher c: ; higher : ; higher : ; Equivalent Noise: , , . For the ‘higher ’, and ‘equivalent noise’ curves, the x-axis represents .

When taking , SR is possible. Equation 8a is plotted for various parameter combinations in Fig. 2. An (arbitrary) base-line curve (blue) describes a characteristic ‘dipper’ function [or threshold-versus-noise (TvN) function], which has been repeatedly reported in the literature for various masking paradigms Solomon (2009). The curve shows that the detection threshold is lowest at intermediate noise levels , characteristic of SR (highlighted in blue shading). In the following sections, the influence of various factors on the shape of this curve, and the appearance of stochastic resonance are investigated.

iii.1 The influence of

First, the influence of lapse rate is investigated. With an increase in (green line; ) relative to the reference curve, SR decreases in magnitude, and the point of maximal SR moves to higher noise levels. The maximum level of before SR is lost can be calculated by setting eq. 8b to zero. Solving for in the limit of zero noise gives the maximum value of (i.e., ):

(10)

Figure 3 shows the dependence of on . At , a typical value of in psychophysical experiments, . This level of is high compared to typical lapse rates around 0.06 or less Wichmann and Hill (2001), and suggests that is not a limiting factor on the occurrence of SR in many experiments.

Figure 3: The maximum lapse rate, dependent on . The shaded area allows for SR.

A final observation is that the effects of remain visible at large values of , while other factors, discussed later, do not have a mayor effect at large (see Fig. 2).

iii.2 The influence of criterion, and decision noise

When the decision criterion is set optimally (between and ), SR cannot take place. However, c is often set suboptimally Green and Swets (1974). The level at which criterion s set, has a strong influence on the vertical position of the TvN curve. With a higher , the curve moves upward, and maximal SR moves to higher noise levels (red line, Fig. 2). In a biological system, the setting of this decision boundary probably depends on the amount of noise in the decision system (). The parameter would sensibly be set at a level which is high enough to prevent many false positives, but low enough to prevent too many misses. Thus, where is put determines how liberal or conservative the decision stage is. A ‘present’ response could have the requirement that the decision signal is larger than the th percentile of response distribution in the decision stage in the absence of input (i.e., purely noise-driven activity in the decision stage, here assumed to be normally distributed). Assuming an unbiased (mean = 0) but noisy response originating from the decision stage, the criterion should be set at

(11)

Therefore, can be interpreted as reflecting noise in a post-sensory decision stage. If increases, so does c, which results in an upwards shift in the detection threshold , as can be observed in Fig. 2. Conversely, because SR can only occur when , systems with low decision noise (or alternatively, very liberal systems with a low ) will not show SR. The condition for SR is that

(12)

As an aside, in signal detection theory, optimal performance on a task is reached when the criterion is set between mean of the signal and noise distribution (see e.g. Gong et al. (2002); Green and Swets (1974)). In our case this means . Because is smaller than , no SR will occur (see also Gong et al. (2002)). However, because is larger than zero, the optimal decision noise is, counterintuitively, larger than zero:

(13)

This conclusion is consistent with a recent analysis on economic decisions making Tsetsos et al. (2016).

Iv The influence of internal noise, and a comparison to the Equivalent Noise paradigm

In our model, the noise is the total amount of noise in the sensory-perceptual system. One can subdivide into two independent components: (i) noise that is external to the system , and e.g. is present in the stimulus; (ii) noise that is internally-generated , where their variances add. In the equivalent noise paradigm, such as the linear amplifier model Lu and Dosher (2008), this is often rewritten as:

(14)

When inserting this into eq. 8a, one can show the effect of adding a constant amount of on performance thresholds (Fig. 2; purple line). Increases in result in left-ward shift of the TvN curve relative to the reference curve. The increase in simultaneously results in lower thresholds (better performance) at low , while it increases thresholds at high . The former result is due to itself causing SR relative to the condition where . In effect, it is moving the SR dip to the left. With even higher levels of internal noise, the curve is shifted so far to the left that upward arm of the curve at low noise levels, disappears. Even though still increases performance relative to the reference curve in this case, and thus shows SR, it will not be recognised as such because it does not present itself as a dip in the TvN curve. I come back to this in the discussion. Increasing even further will remove SR completely.

iv.1 A comparison to the equivalent noise paradigm

In stead of assuming a fixed criterion , as in the derivations above, the equivalant noise paradigm assumes that the decision criterion is set optimally, i.e, . When this is inserted into eq. 6, and deriving the threshold performance, we obtain:

(15)

Setting (one of the assumptions often made) results in the standard dependence of the detection threshold on internal an external noise

(16)

(when is set to 1). This dependence is plotted in Fig 2 (dashed curve). Due to the fact that , there is no SR possible in this description of human performance.

iv.2 The relationship between and efficiency

Because measured thresholds in psychophysical experiments are often higher than the optimal thresholds, the EN paradigm often includes an “efficiency” parameter when fitting the curve to experimental data:

(17)

where , which scales the curve up (i.e., higher detection thresholds).

When reordering Eq. 15 one can see that provides a scaling factor in our model, similar to in equivalent noise paradigms. This relationship makes intuitive sense, as an increased number of lapses, decreases efficiency:

(18)

where AE is a constant, and related to the calculation efficiency (cf. Pelli (1981)), but here abbreviated as AE (attentional efficiency) to capture the fact that it is determined by attentional lapse rate :

(19)

When equating AE to (see eq. 17) to derive the relationship between and , one finds that

(20)

and

(21)

This relationship is plotted in Fig. 4 for different values of . These results show that lower efficiency can be captured by increased lapse rates.

Overall, the comparison between our model and the equivalent noise paradigm (specifically, the linear amplifier model) suggests that can expressed in terms of , and thus that lapse rate may explain at least part of the suboptimal efficiency (i.e., ) that is often reported in psychophysical experiments.

Figure 4: The relationship between efficiency parameter in equivalent noise paradigms, and lapse rate .

iv.3 The pooling parameter

In some psychophysical experiments, performance thresholds are actually lower than predicted. This would imply an efficiency . Generally, however, the results are interpreted differently, with total noise calculated as:

(22)

where , representing a pooling parameter, which replaces . Equation 22, in essence, describes an important property of the central limit theorem, and quantifies the number of samples that are taken to estimate a mean. For example, how many individual moving dots are combined to estimate a global pattern motion (e.g., Dakin et al., 2005).

Detection thresholds can be obtained from eq. 8a or eq. 9, by replacing with , while setting , :

(23)

Equation 23 and eq. 17, are very closely related as they describe the same relationship when is equal to .

I set , here, because otherwise both (or ) and need to be determined, and as both have the same scaling effect, the system would be underdetermined. However, in experimental settings, unless participants are experienced and extremely motivated, it is unlikely that , and thus a correct value of the pooling parameter can only be obtained if is also determined.

iv.4 Overestimation of internal noise in the EN paradigm

When comparing the EN results in Figure 2 to those of the ‘high ’ condition, one can observe that the suboptimal positioning of the criterion in the latter case caused an expected increased detection thresholds at low external noise values (left side of the plot). An important consequence is that the curves are quite different even though the internal noise is the same. Consequently, fitting the equivalent noise function to experimental data with SR will give incorrect (over)estimates of internal noise.

Figure 5: (a) The dependence of on , when SR data is fit with the EN equation. Internal noise is overestimated, especially at small levels of .

One can approximate the overestimation of internal noise by equating the EN model (eq. 15) and the SR model (eq. 8a) at . Then, while taking the noise in the SR equation as the actual internal noise () and that in the EN equation as the estimated internal noise (), we obtain:

(24)

which is plotted in Figure 5. Estimated internal noise is close to actual internal noise when the actual internal noise is large. However, when the actual internal noise is low, the estimated internal noise is overestimated. When is relatively low, the estimated internal noise is not even monotonically related to the actual internal noise. Therefore, the EN paradigm overestimates low internal noise. The precise amount of overestimation depends on various parameters (e.g., , , ), and on the distribution of external noise values tested in experimental settings, and how much the dip caused by SR influences the EN model fit.

V Stochastic resonance in terms of accuracy

Figure 6: (a) The dependence of on noise . The warm colors are for the current model, the cold colors are for those of Gong et al. (2002). The models are very different at low , but converge at high . (b) The dependence of accuracy [P(correct)] on . The current model is a scaled version of the model by Gong et al. (2002). Parameter were , .

I have focussed so far on the dependence of SR on , because data in terms of accuracy have previously been reported on in a similar model without Gong et al. (2002). A direct comparison between the current model and that of Gong et al. (2002) in terms of and accuracy is made in Fig. 6. As one can see, the largest difference is in terms of , and not in terms of accuracy. In fact, the level of above-chance accuracy in our model is times that of the model where Gong et al. (2002). This fixed relationship means the analyses by Gong et al. (2002), which rested on finding the point where the derivative of accuracy relative to was 0, will remain valid in the current model.

Vi Discussion

I have presented a model that explains SR in a signal detection framework, adding the human characteristic of lapse rate. The inclusion of lapse rate allows for stochastic resonance to be calculated at various levels of the performance metric , in addition to accuracy. The model produces experimentally observed ‘dipper’ functions, when plotting threshold signal strength versus noise (TvN). I compared the current model of SR to the equivalent noise paradigm, and show that the lapse rate may be used to explain the suboptimal ‘efficiency’ that is often found in experimental paradigms, as it scales the TvN curve up. I also argue that fitting data with the equivalent noise approach when SR is present can lead to incorrect estimates of the level of internal noise.

In our model the lapse rate has a strong influence on SR. With a typically employed performance threshold of , the maximum lapse rate is about 0.16 before SR is lost. This is relatively high, and unlikely to be reached in most psychophysical experiments, suggesting that SR could be observed in many experiments. However, lapse rates increase when people perform dual-tasks Buckley et al. (2016), and when they are tired Anderson et al. (2012), and thus even with a relatively lenient performance threshold of , SR may not be observed. When more stringent performance thresholds are employed such as , or , the maximum lapse rate rapidly declines to  0.023, and  0.0013, which are at or below typically-observed lapse rates Wichmann and Hill (2001), resulting in weak or absent SR.

I also show that the position of the decision criterion has a large influence on the signal thresholds. When criterion is increased, performance thresholds are also much higher. I proposed that the criterion level depends on the noise in the decision stage, as well as on how liberal or conservative the decision stage is. SR is only possible in humans that have a noisy decision stage (large decision noise), or those that are quite conservative (i.e., need a strong signal before they say ‘present’).

The current framework also allows one to model influences of internal noise (). When internal noise is small it allows for SR, but when internal noise is large, SR will disappear, which is a potential explanation of why SR is not observed more often then it is. Low amounts of , cause SR in their own right, just as external noise does. In Fig. 2, this can be observed as a decrease in threshold at low external noise (on the left of the plot), compared to the reference curve. In psychophysical settings, however, this would not be interpreted as -induced SR, because in most experiments internal noise is a fixed value that is not manipulated (and thus a reference curve is lacking).

One could potentially design experiments to manipulate internal noise. For example, it has been suggested that inattention increases internal noise Rahnev et al. (2011); Lu and Dosher (1998), which could counterintuitively cause a decrease in detection threshold through SR in inattention conditions. Alternatively, one could use an individual differences approach to investigate whether internal noise causes SR. With this approach one would predict that individuals with moderate levels of internal noise have lower (detection) thresholds than individuals with either low or high levels of internal noise. In this context it is interesting that SR has been proposed as a potential explanation for the increased perceptual functioning of people with autism spectrum disorder (ASD) on some perceptual tasks Simmons et al. (2009). Specifically, it is argued that people with ASD have increased levels of internal noise Dinstein et al. (2012); Simmons et al. (2009); Milne (2011), and thus could outperform typically-developing (TD) individuals ,who have low internal noise, on some tasks due to SR. Incidentally, the TvN curves in Fig. 2, can further explain the general finding that interindividual variation in task performance is larger in the ASD group than in the TD group, because with low noise the TD group would fall on a relatively flat part of the curve, while with intermediate noise, the ASD group would fall in an area of the curve where small changes in internal or external noise can lead to large changes in threshold measurements.

Overall, these speculations suggest that SR may have a more important role in human performance than often realised. Even in the literature, unidentified SR signatures are present in some data (e.g., Dakin et al., 2005; Mareschal et al., 2008; Lu et al., 2006), which were missed because the data were fitted with an equivalent noise approach (which does not allow for SR). Combined with the data that already showed SR Simonotto et al. (1997); Collins et al. (1996); Moss et al. (2004); Ward et al. (2002); Goris et al. (2008), these data suggest that beneficial effects of noise (i.e., SR) may be more common than acknowledged, even in human performance. Our model may help determine which factors play important roles in determining when (and in who) SR occurs.

Acknowledgements.
The author thanks Dr. Dror Cohen for suggestions on a previous version of the manuscript.

References

  • Von Neumann (1956) J. Von Neumann, Automata studies 34, 43 (1956).
  • Shannon (2001) C. E. Shannon, ACM SIGMOBILE Mobile Computing and Communications Review 5, 3 (2001).
  • Green and Swets (1974) D. Green and J. Swets, Huntington, NY: RF Krieger  (1974).
  • McDonnell and Abbott (2009) M. D. McDonnell and D. Abbott, PLoS Comput Biol 5, e1000348 (2009).
  • Lu and Dosher (2008) Z.-L. Lu and B. A. Dosher, Psychol Rev 115, 44 (2008).
  • North (1942) D. North, RCA Review 6, 332 (1942).
  • Gong et al. (2002) Y. Gong, N. Matthews,  and N. Qian, Phys Rev E Stat Nonlin Soft Matter Phys 65, 031904 (2002).
  • Simonotto et al. (1997) E. Simonotto, M. Riani, C. Seife, M. Roberts, J. Twitty,  and F. Moss, Physical review letters 78, 1186 (1997).
  • Collins et al. (1996) J. J. Collins, T. T. Imhoff,  and P. Grigg, Nature 383, 770 (1996).
  • Moss et al. (2004) F. Moss, L. M. Ward,  and W. G. Sannita, Clin Neurophysiol 115, 267 (2004).
  • Ward et al. (2002) L. M. Ward, A. Neiman,  and F. Moss, Biol Cybern 87, 91 (2002).
  • Goris et al. (2008) R. L. T. Goris, J. Wagemans,  and F. A. Wichmann, J Vis 8, 17.1 (2008).
  • Pelli (1981) D. Pelli, Effects of visual noise, Ph.D. thesis, Cambridge University (1981).
  • Solomon (2009) J. A. Solomon, Atten Percept Psychophys 71, 435 (2009).
  • Wichmann and Hill (2001) F. A. Wichmann and N. J. Hill, Percept Psychophys 63, 1293 (2001).
  • Tsetsos et al. (2016) K. Tsetsos, R. Moran, J. Moreland, N. Chater, M. Usher,  and C. Summerfield, Proc Natl Acad Sci U S A 113, 3102 (2016).
  • Dakin et al. (2005) S. C. Dakin, I. Mareschal,  and P. J. Bex, Vision Res 45, 3027 (2005).
  • Buckley et al. (2016) R. J. Buckley, W. S. Helton, C. R. H. Innes, J. C. Dalrymple-Alford,  and R. D. Jones, Conscious Cogn 45, 174 (2016).
  • Anderson et al. (2012) C. Anderson, J. P. Sullivan, E. E. Flynn-Evans, B. E. Cade, C. A. Czeisler,  and S. W. Lockley, Sleep 35, 1137 (2012).
  • Rahnev et al. (2011) D. Rahnev, B. Maniscalco, T. Graves, E. Huang, F. P. de Lange,  and H. Lau, Nat Neurosci 14, 1513 (2011).
  • Lu and Dosher (1998) Z. L. Lu and B. A. Dosher, Vision Res 38, 1183 (1998).
  • Simmons et al. (2009) D. R. Simmons, A. E. Robertson, L. S. McKay, E. Toal, P. McAleer,  and F. E. Pollick, Vision Res 49, 2705 (2009).
  • Dinstein et al. (2012) I. Dinstein, D. J. Heeger, L. Lorenzi, N. J. Minshew, R. Malach,  and M. Behrmann, Neuron 75, 981 (2012).
  • Milne (2011) E. Milne, Front Psychol 2, 51 (2011).
  • Mareschal et al. (2008) I. Mareschal, P. J. Bex,  and S. C. Dakin, Vision Res 48, 1719 (2008).
  • Lu et al. (2006) Z.-L. Lu, W. Chu,  and B. A. Dosher, Vision Res 46, 2315 (2006).
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
""
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
   
Add comment
Cancel
Loading ...
193804
This is a comment super asjknd jkasnjk adsnkj
Upvote
Downvote
""
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters
Submit
Cancel

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test
Test description