Depressed individuals express more distorted thinking on social media.

Depressed individuals express more distorted thinking on social media.

Abstract

Depression is a leading cause of disability worldwide, but is often under-diagnosed and under-treated. One of the tenets of cognitive-behavioral therapy (CBT) is that individuals who are depressed exhibit distorted modes of thinking, so-called cognitive distortions, which can negatively affect their emotions and motivation. Here, we show that individuals with a self-reported diagnosis of depression on social media express higher levels of distorted thinking than a random sample. Some types of distorted thinking were found to be more than twice as prevalent in our depressed cohort, in particular Personalizing and Emotional Reasoning. This effect is specific to the distorted content of the expression and can not be explained by the presence of specific topics, sentiment, or first-person pronouns. Our results point towards the detection, and possibly mitigation, of patterns of online language that are generally deemed depressogenic. They may also provide insight into recent observations that social media usage can have a negative impact on mental health.

  1. Center for Social and Biomedical Complexity, Indiana University

  2. Department of Psychological and Brain Sciences, Indiana University.

1 Introduction

Depression is a leading contributor to the burden of disability worldwide[23, 46], with some evidence that disability attributed to depression is rising, particularly among youth[10, 34]. A key challenge in reducing the prevalence of depression has been that it is often under-recognized[33] as well as under-treated[44]. Cognitive-behavioral therapy (CBT), is the most widely researched psychotherapy for depression. It is equivalent to antidepressant medications in its short-term efficacy and evidences superior outcomes in the long-term[24, 14]. The cognitive theory underlying CBT argues that the ways in which individuals process and interpret information about themselves and their world is directly related to the onset, maintenance, and recurrence of their depression[2, 13]. This model is consistent with information processing accounts of mood regulation[22] and its dynamics[43], as well as basic research that supports the role of cognitive reappraisal and language in emotion regulation[45, 20, 16, 42].

In CBT, therapists work with their clients to identify depressogenic thinking patterns by identifying lexical or verbal markers of rigid, distorted, or overly negative interpretations[32, 37]. For example, statements that include “should” or “must” are often challenged as reflecting overly rigid rules about the world (“I shouldn’t be lazy”, “I must never fail”). This process often entails a series of conversations with the client to uncover and address statements that reflect these so-called Cogntive Distortions (CD).

The idea that language is predictive of depression is supported by data-driven approaches detecting depression from various lexical markers including the use of language to describe negative emotions[39, 41], the use of first-person pronouns[6, 40, 47, 9], and mentions of common symptoms[11]. Machine learning approaches have been shown to successfully predict whether Facebook users suffer from depression[18, 12], identifying the most useful lexical markers to render a prediction. These results, while useful for prediction and the detection of depression, do not offer insights into the cognitive dynamics of the disease pattern, nor its relationship to language, which is crucial in developing treatments and interventions.

Here, we emphloy a theory-driven approach to studying depressive language on Twitter. Rather than attemphting to extract relevant text features from text data, e.g. “sleep”, “health”, or other mental health related features, we define a clinical lexicon of 241 n-grams[27] that a panel of clinical psychologists deemed to form a schema involved in the expression of a particular type of distorted thinking according to CBT theory and practice. For example, “I will never _” would be implicated in the expression of a cognitive distortions such as Catastrophizing or Fortune-telling, whereas “I am a _” would be used to express a Labeling and Mislabeling distortion.

We then compare the longitudinal prevalence of this set of Cognitive Distortion Schemata (CDS) in the language of a large cohort of depressed individuals vs. a random sample on social media (Twitter). Our results indicate significantly higher prevalence of most types of CDS in the Depressed cohort, both at the within-subjects and between-groups level. Particularly CDS in the Personalizing and Emotional Reasoning types occur approximately 2.3 times more frequently in the online language of Depressed users. Our results are robust to changes in our user sample, our choice of CDS n-grams, text sentiment, and the known propensity of Depressed individuals to make self-referential statements.

1.1 Cognitive distortion types and n-gram schemata

Aaron T. Beck introduced the concept of cognitive distortions to characterize the thinking of individuals with depression[3, 4]. Subsequently, other clinicians expanded on his typology of distortions[8], including most recently his daughter, clinical psychologist and CBT expert, Judith Beck[5]. We drew upon these latest lists to identify 12 types of cognitive distortions that may characterize the thinking of individuals who are depressed.

Category Definition Examples

Catastrophizing
Exaggerating the importance of negative events The evening will be a disaster

Dichotomous Reasoning
Thinking that an inherently continuous situation can only fall into two categories No one will ever like me.

Disqualifying the Positive
Unreasonably discounting positive experiences OK but my grade was not that good.

Emotional Reasoning
Thinking that something is true based on how one feels, ignoring the evidence to the contrary My grades are good but it still feels like I will fail.

Fortune-telling
Making predictions, usually negative ones, about the future. Whatever I try I will not be successful

Labeling and Mislabeling
Labeling yourself or others while discounting evidence that could lead to less disastrous conclusions I am a total loser.

Magnification and Minimization
Magnifying negative aspects or minimizing positive aspects My good grades are really not important.

Mental Filtering
Paying too much attention to negative details instead of the whole picture If I only worked harder, I would be more successful.

Mindreading
Believing you know what others are thinking Everyone believes I am a failure.

Overgeneralizing
Making sweeping negative conclusions based on a few examples Nobody ever cares for me.

Personalizing
Believing others are behaving negatively because of oneself, without considering more plausible or external explanations for behavior Everyone thinks I am a loser for calling her.

Should Statements
Having a fixed idea on how you and/or others should behave I have to to do this or I will not make it to the weekend.

Table 1: Common cognitive distortions identified in CBT for depression[5]. We provide examples of how Cognitive Distortion Schemata (CDS) can be embedded in common expressions (bold). Contractions are expanded. Numbers indicate where the example contains more than one CDS.
Cognitive Distortion Types

We defined 241 CDS n-grams in total, each expressing at least 1 type of cognitive distortion (see Appendix Table 7). The schemata in each category were formulated to capture the “minimal semantic building blocks” of expressing distorted thinking for the particular type, avoiding expressions that are specific to a depression-related topics, such as poor sleep or health issues. For example, the 3-gram “I am a” was included as a building block of expressing Labeling and Mislabeling, because it would be a highly likely (and nearly unavoidable) n-gram to express many self-referential (“I”) expressions of labeling (“am a”) (for an example see Table 1). Where possible, higher-order n-grams were chosen to capture as much of the semantic structure of one or more distorted schemata as possible, e.g. the 3-gram “everyone will believe” captures both Overgeneralizing and Mindreading. We did include 1-grams such as “nobody” and “everybody” in spite of their prevalence in common language, since they strongly correspond to the expression of Dichotomous Reasoning. Table 2 shows the number of schemata per category in our CDS set along with the average n-gram size, and a number of relevant grammatical features. The complete set of CD schemata is provided in Table 7 in the Appendix.

We note that a significant sub-set of the CDS do not occur in the Twitter content for both cohorts (see Table 2: ), indicating that parts of our set of CDS are “lexically exhaustive” with respect to capturing the major modes of CD expression in natural language.

Significant Avg. Length Pronouns
CD Category (%) (%)
Catastrophizing 11 10 2 18.2 3.000
Dichotomous Reasoning 23 23 16 69.6 1.347
Disqualifying the Positive 14 13 4 28.6 2.286 7.1
Emotional Reasoning 7 7 6 85.7 2.857 85.7
Fortune-telling 8 8 8 100.0 3.125 87.5
Labeling and Mislabeling 44 44 20 45.5 2.273 36.4
Magnification and Minimization 8 8 4 50.0 2.000
Mental Filtering 14 14 3 21.4 2.786 50.0
Mindreading 72 62 9 12.5 3.167 83.3
Overgeneralizing 21 21 14 66.7 2.762 57.1
Personalizing 14 14 9 64.3 2.429 100.0
Should Statements 5 5 2 40.0 1.400
Total 241 229 97 40.2 2.585 51.0
Table 2: Descriptive statistics of our set of Cognitive Distortion Schemata which are grouped in 12 types (“CD Category”). Column indicates the number of schemata in the specific category. Mindreading is the largest category because it contains many grammatical variations of the same schema, e.g. “I/you/they _ will think”. The column shows the number of n-grams in each category that were actually found in our Twitter data. The and columns show respectively the number and ratio of n-grams in the category for which we found a statistically significantly higher prevalence in the Depressed than the Random Sample cohort. is the average length (n) of each n-gram in the category. Column shows the ratio of CDS in each category that contain a first-person pronoun. indicates the absence of any CDS with the feature.

1.2 Depressed and random sample

We identified a cohort of social media users that had self-reported a clinical diagnosis of depression by posting a variant of the explicit statement “I was diagnosed with depression” (see “Materials and Methods”). To make sure we were only including truly self-referential statements of diagnosis of depression, 3 of the authors manually removed quotes, retweets, jokes, and external references. Note that we exclude all diagnosis statements themselves from our analysis, including all tweets that contain the term “diagnos” and “depress”. We also examine the sensitivity of our results to the propensity of this cohort to make similar self-referential statements (see “Absence of personal pronoun effect.”)

With this final set of adjudicated diagnosis tweets, we harvested the maximum number of tweets allowed by the Twitter API (the most recent 3200) for each individual, resulting in a sample of 1,207 users and their 1,759,644 tweets (ranging from May 2008 to September 2018). We refer to this cohort as “Depressed”, but acknowledge that we have no independent confirmation of their present mental health state. We also established a baseline sample of randomly chosen individuals with a similar distribution of account creation dates as the Depressed cohort to account for changes in user behavior and platform effects. Here too we exclude all tweets that contain the terms “diagnos” and “depress” from subsequent analysis. Our “Random Sample” cohort contains 8,791 individuals and a total 8,498,574 tweets (see “Materials and Methods”).

2 Results

We first compare the within-subject prevalence of the established set of CDS between the Depressed and Random Sample cohorts. For each individual we count how many of their tweets contained any of the 241 CDS and divide it by their total number of tweets, resulting in an individual within-subject CD prevalence (see “Materials and Methods”). The density distribution of individual prevalence values can then be compared between Depressed and Random Sample individuals as shown in Fig. 1. We restrict this analysis to individuals with at least 150 tweets so that we have sufficient data to determine prevalence reliably, but retain all individuals in subsequent between-groups analyses since the latter does not require the calculation of within-subject prevalence values.

Figure 1: Density of within-subject prevalence of tweets containing CDS for individuals with at least 150 tweets shows a greater number of individuals with high CDS levels in the Depressed than in the Random Sample cohort. The results of a two-sample K–S test allows us to reject the null-hypothesis that the two distributions are drawn from the same sample ( (depressed), (random), ).

We observe that the distribution of within-subject CDS prevalence is shifted significantly to the right for the Depressed cohort relative to that of the Random Sample, indicating that individuals in the Depressed cohort express significantly more CDS. Note that % of the Random Sample individuals have no tweets with CDS whereas the Depressed sample has no cases without CDS. Results from a Two-Sample Kolmogorov–Smirnov test () indicate that we can reject the null hypothesis that the two samples are drawn from the same distribution.

Figure 2: Density of bootstrapped between-groups Prevalence Ratios, with median () and 95% Confidence Intervals (CI) between the Depressed and Random Sample cohort (see Table 3: “All CDs”). The 95% CI of the distribution does not include (vertical line) indicating a significantly higher prevalence of all CDS for the Depressed cohort ().

Furthermore, we conduct a between-groups analysis to compare the prevalence of CDS between the Depressed vs. the Random Sample cohort. We do so by calculating the Prevalence of CDS for all tweets from each cohort and calculating the Prevalence Ratio () between the two cohorts (see Materials and Methods “Prevalence Ratio”). A Prevalence Ratio significantly larger than 1 indicates that the presence of CDS in the tweets written by the Depressed cohort is greater than the Random Sample cohort. To assess the sensitivity of our results to changes in our cohort samples, we repeatedly calculate the estimated value over 10,000 random re-samples (with replacement) of both groups, resulting in a distribution of PR values shown in Fig. 2 (see Materials and Methods “Bootstrapping”). Note, Prevalence Ratios express the relative difference between the 2 cohorts, not the absolute difference which is provided in Appendix Table 6.

We observe in Fig. 2 that the median of this distribution of PR values is significantly larger than 1 (and its 95% confidence intervals does not include 1), indicating that we find a statistically significant higher prevalence of CDS in the Depressed cohort () than in the Random Sample, and that this result is robust to random changes in our cohorts.

median 95% CI median 95% CI median 95% CI
All CDS 1.186 [1.157, 1.216] 1.169 [1.140, 1.197] 1.220 [1.160, 1.310]
Personalizing 2.402 [2.242, 2.576] / / 2.412 [1.671, 2.957]
Emotional Reasoning 2.323 [2.049, 2.639] 2.065 [1.702, 2.485] 2.317 [2.012, 3.184]
Overgeneralizing 1.580 [1.501, 1.661] 1.486 [1.409, 1.566] 1.574 [1.369, 1.734]
Mental Filtering 1.468 [1.291, 1.656] 1.346 [1.069, 1.660] 1.470 [1.173, 1.919]
Labeling and Mislabeling 1.328 [1.267, 1.391] 1.204 [1.144, 1.268] 1.319 [1.156, 1.478]
Disqualifying the Positive 1.349 [1.210, 1.498] 1.349 [1.210, 1.498] 1.346 [1.176, 1.555]
Dichotomous Reasoning 1.195 [1.163, 1.226] 1.195 [1.163, 1.226] 1.216 [1.158, 1.303]
Mindreading 1.136 [1.060, 1.230] 1.136 [1.060, 1.229] 1.129 [0.808, 1.274]
Should Statements 1.103 [1.050, 1.153] 1.103 [1.050, 1.153] 1.100 [0.836, 1.409]
Magnification and Minimization 1.075 [1.023, 1.130] 1.075 [1.023, 1.130] 1.078 [1.016, 1.472]
Fortune-telling 0.954 [0.837, 1.075] 0.586 [0.483, 0.698] 0.944 [0.501, 1.288]
Catastrophizing 0.729 [0.554, 0.902] 0.729 [0.554, 0.902] 0.718 [0.654, 1.062]
Table 3: Prevalence Ratio (PR) and 95% CIs of CDS between the Depressed and Random Sample cohort. PR values indicate a significantly higher prevalence in the Depressed sample (marked by ). Values are calculated under 3 distinct conditions, labeled : values for the entire set of CDS, : values for CDS without first-person pronouns (“I”,“me”, “my”, “mine”, and “myself”), and : values with a 95% CI resulting from resampling the CDS, instead of our sample of individuals (see Materials and Methods). Appendix Table 6 provides absolute Prevalence Differences.
Figure 3: Comparison of Cognitive Distortion Schemata (CDS) Prevalence Ratios between the Depressed and Random Sample cohort for each cognitive distortion type. Black dots indicate the boundaries of the 95% CI. The medians are indicated by a green, yellow, or red diamond respectively corresponding to a higher prevalence in the Depressed, no difference, or higher prevalence among the Random Sample respectively. The Depressed cohort showed a significantly higher use of CDS than the Random Sample cohort for all CD types combined (“All CDS”) and most CDS types separately () with the exception of Fortune-telling and Catastrophizing. Please see Table 3 for more details about the Prevalence Ratios.

The between-groups PR values shown in Fig. 2 do not reflect specific distortion types; all CDS are equally and independently matched to all tweets. Total CDS prevalence over all tweets is 21.8% and 18.407% for the Depressed and Random Sample cohort respectively but differs significantly for each CD type (See Appendix Table 5). It is reasonable to expect that the different types of CDS may differ in their prevalence between our cohorts. We therefore repeat the above analysis, with CDS separated by CD type (see Table 2).

As shown in Table 3 and Fig. 3, the prevalence of CDS is significantly higher for nearly all CD types in the tweets of the Depressed cohort than those of the Random Sample with Prevalence Ratio values ranging from to , with the exception of Catastrophizing and Fortune-telling, with the latter not producing a PR significantly different from parity. The CD types Personalizing and Emotional Reasoning have the greatest PR values of and , followed by Overgeneralizing (), Mental Filtering (), Labeling and Mislabeling (), and Disqualifying the positive (). The CD types Mind Reading, Should Statements, and Magnification and Minimization have lower yet significant PR values of . Table 2 “Significant N” shows the number and ratios of schemata for each CD type that have PR values significantly different from parity.

The PR individual CDS n-grams can differ significantly as well. Appendix Fig. 6 shows the contributions of each individual CDS n-gram separately. Table 4 shows the CDS with the individually highest and lowest PR values to illustrate the CDS that are most prevalent in the Depressed and Random Sample cohort respectively. As shown, the highest ranked CDS for the Depressed cohort belong to the Mindreading, Emotional Reasoning, and Personalizing type, whereas the highest ranked CDS for the Random Sample belong to the non-reflexive Mindreading and Fortune-telling type.

PR rank Depressed Random Sample
1 everyone will think we know
2 since it feels they will not
3 I caused he believes
4 will go wrong we believe
5 because of my we will not
6 because I feel she will not
7 all my fault an incompetent
8 a burden that will not
9 because my we will know
10 I am always we do not know
Table 4: Ten CDS with highest individual PR values between the Depressed and Random Sample cohorts to illustrate most prevalent n-grams in online language of either cohort.

2.1 Absence of sentiment effect

Previous research has shown that the language of depressed individuals is less positive (lower text valence) and contains higher levels of self-referential language[35, 21, 1, 7, 39, 26]. To determine the degree to which our results can be explained by text sentiment or self-referential statements instead of distorted thinking, we examine the valence loadings of our collection of tweets and CDS, and reproduce our results with and without CDS containing self-referential statements.

First, we determine the valence values of each CDS n-gram in our set using the VADER sentiment analysis tool[25] which in a recent survey was shown to outperform other available sentiment analysis tools for social media language[38]. VADER is particularly appropriate for this use, since its sentiment ratings take into account grammatical context, such as negation, hedging, and boosting. We find that 75.9% of our CDS have either no sentiment-loaded content or are rated to have zero valence (neutral sentiment scores). The average valence rating of all CDS is on a scale from to . Fig. 4A shows the VADER sentiment distribution of only CDS n-grams with non-zero ratings. Here we observe only a slight negative skew of CDS sentiment for this small minority of CDS n-grams (24.1%).

Figure 4: (A) Density of Vader scores for CDS with non-zero sentiment values (24.1% of 241 schemata). Most CDS carried no valence loading (75.9%). The average rating for the complete set CDS is -0.05 (N=241). (B) Density of VADER valence ratings for all individual tweets posted by the Depressed and Random Sample cohorts, indicating a significant right-hand skew towards positive sentiment for both. The Depressed cohort has more extreme positive and negative sentiment than the Random Sample (blue line vs. orange bars). The results of a two-sample K–S test allows us to reject the null-hypothesis that the two sentiment distributions are drawn from the same distribution ().

Furthermore, as shown in Fig. 4B, the sentiment distributions of all tweets for the Depressed and Random Sample cohorts are both skewed towards positive sentiment (right side of distribution). This matches earlier findings that human language exhibits a so-called Polyanna effect[17], a near-universal phenomenon that skews human language towards positive valence. Surprisingly, we find no indications that the tweets of the Depressed cohort carry more negative valence than those of the Random Sample cohort. To the contrary, VADER sentiment ratings in the range seem to be slightly more prevalent among the tweets of the Depressed cohort (see Fig. 4B), possibly indicating an increased emotionality (higher levels of both negative and positive affect). One particular deviation in the sentiment range of was found to be uniquely associated with the Random Sample cohort using the “Face With Tears of Joy” emoji (VADER sentiment=0.4404) more often than the Depressed cohort. A two-sample K–S test allows us to reject the null-hypothesis that the two distributions are drawn from the same sample (1).

Combined, these findings strongly suggest that the higher prevalence of CDS in the language of the Depressed cohort can neither be attributed to a negative valence skew in the CDS set, nor the sentiment distribution of the tweets produced by either the Depressed and Random Sample cohorts.

2.2 Absence of personal pronoun effect

Research has shown that First-Person Pronouns (FPP) are more prevalent in the language of depressed individuals[39, 40]. Since many CDS contain FPPs (see Table 2 “Pronouns”), our results may to a degree reflect this phenomenon instead of the “distorted” nature of our CDS. To test the sensitivity of our results to the presence of FPPs in our set of CDS, we repeat our analysis entirely without CDS that contain the FPPs “I” (upper-case), “me”, “my”, “mine”, and “myself”. As shown in Table 3: PR, we find that their removal does not significantly alter the observed effect. The respective confidence intervals resulting from our removal of FPP schemata change, but most overlap with those obtained from an analysis that includes the full set of CDS (see Table 3: PR vs Table 3: PR). This demonstrates that our observations are not a product of the presence of first-person pronouns in our set of CDS. Note that we could not determine any values for Personalizing because its CDS all contain first-person pronouns (see Appendix Fig. 5).

2.3 Robustness to CDS changes

To determine the sensitivity of our results to the particular choice of CDS, we re-calculated PR values between the Depressed and Random Sample cohorts, but instead of re-sampling our Depressed and Random Sample cohort, we randomly re-sampled (with replacement) the set of 241 CDS n-gram. The 95% CI of the resulting distribution of PR values then indicates how sensitive our results are to random changes of our CDS set. The results of this analysis are shown in Table 3: PR. We observe slight changes in the dispersion of the resulting distribution of PR values, but the median values and 95% CIs remain largely unchanged. As before, the 95% CIs continue to exclude for all CD types, except Mindreading, Should Statements, Fortune-telling, and Catastrophizing, and we can continue to reject the null-hypothesis that PR values are similar between the Depressed and Random Sample cohort for nearly all CD types. Furthermore, as shown in Table 3, the 95% CIs of PR and PR largely overlap across all CD types indicating our results are robust to random changes of our cohort samples as well as our CDS set.

3 Discussion

In an online sample of individuals, we emphloyed a theory-driven approach to measure linguistic markers that may indicate cognitive vulnerability to depression, according to CBT theory. We defined a set of Cognitive Distortion Schemata (CDS) that we grouped along 12 widely accepted types of distorted thinking and compared their prevalence between two cohorts of Twitter users: one of individuals who self-identified as having received a clinical diagnosis of depression and the other a similar random sample.

As hypothesized, the Depressed cohort use significantly more CDS of distorted thinking in their online language than the Random Sample, particularly schemata associated with Personalizing and Emotional Reasoning. We observed significantly elevated levels of CDS across nearly all CD types, sometimes more than twice as much, but did not find a statistically significant elevated prevalance among the Depressed cohort for two specific types, namely Fortune-telling and Catastrophizing. This may be due to the difficulty of capturing these specific cognitive distortions in the form of a set of 1 to 5-grams as their expression in language can involve an interactive process of conversation and interpretation. Of note, our findings are not explained by the use of first-person pronouns or more negatively loaded language, both of which had been identified in past research as markers of depressed individuals. These results shed a light on how depression may affect public discourse on social media, but also reveals the degree to which depressogenic language is manifested in the colloquial language of social media platforms. This is of social relevance given that these platforms are specifically designed to propagate information through the social ties that connect individuals on a global scale.

An advantage of studying theory-driven differences between the language of depressed and non-depressed individuals, as opposed to a purely data-driven or machine learning approach, is that we can explicitly use the principles underpinning CBT to understand the cognitive and lexical components that may shape depression. Cognitive behavioral therapists have developed a set of strategies to challenge the distorted thinking that is characteristic of depression. Preliminary findings suggest that specific language can be related to specific therapeutic practices and seems to be related to outcomes[19]. These practices, however, have largely been shaped by a clinical understanding and not necessarily informed by objective measures of how patterns of language can determine the path of recovery.

Our results suggest a path for mitigation and intervention, including applications that engage individuals suffering from mood disorders such as major depressive disorder via social media platforms and that challenge particular expressions and types of depressogenic language. Future characterizations of the relations between depressogenic language and mood may aid in the development of automated interventions (e.g., “chatbots”) or suggest promising targets for psychotherapy. Another approach that has shown promise in leveraging social media for the treatment of mental health problems involves “crowdsourcing” the responses to cognitively-distorted content[36].

Several limitations of our theory-driven approach should be considered. First, we rely on self-reported depression diagnoses on social media which have not been independently verified by a clinician. However, the potential inaccuracy of this inclusion criterion would reduce the observed effect sizes (PR values between cohorts) due to the larger heterogeneity of our cohorts. Consequently, our results are likely not an artifact of the accuracy of our inclusion criterion. Second, our lexicon of CDS was composed and approved by a panel of 9 experts who may have been only partially successful in capturing all n-grams used to express distorted ways of thinking. Nevertheless, a significant portion of CDS in our set did not occur in our collections of Twitter content, indicating the scope of our lexicon exceeds that of common online language. On a related note, the use of CDS n-grams implies that we measure distorted thinking by proxy, namely via language, and our observations may be therefore be affected by linguistic and cultural factors. Common idiosyncratic or idiomatic expressions may syntactically represent a distorted form of thinking, but no longer do in practice. For example, an expression such as “literally the worst” may be commonly emphloyed to express dismay, without necessarily involving the speaker experiencing a distorted mode of thinking. Third, both cohorts were sampled from Twitter, a leading social media platform, whose use may be associated with higher levels of psychopathology and reduced well-being[31, 29, 30]. We may thus be observing elevated or biased rates of distorted thinking in both cohorts as a result of platform effects. However, we report relative prevalence numbers with respect to a carefully construed random sample, which likely compensates for this effect. Furthermore, recent analysis indicates that representative samples with respect to psychological phenomena can be obtained from social media content[28]. This is an important discussion in computational social science that will continue to be investigated. Data-driven approaches that analyze natural language in real-time will continue to complement theory-driven work such as ours.

4 Materials and Methods

4.1 Data and sample construction

Using the Twitter Application Program Interface (API) and the IUNI OSoMe[15] (a service which provides searchable access to the Twitter “Gardenhose”, a 10% sample of all daily tweets), we search for tweets that matched both “diagnos*” and “depress*.” The resulting set of tweets are then filtered for matching the expressions “i”, “diagnos*”, “depres*” in that order in a case-insensitive manner allowing insertions to match the greatest variety of diagnosis statements, e.g. a tweet that states “I was in fact just diagnosed with clinical depression” would match. Finally, to ensure we are only including true self-referential statements of a depression diagnosis, a team of 3 experts manually removed quotes, jokes, and external references. For each qualifying diagnosis tweet we retrieve the timeline of the corresponding Twitter user using the Twitter user_timeline API endpoint 2. Subsequently, we remove all non-English tweets (Twitter API machine-detected“lang” field), all retweets, and tweets that contain “diagnos*” or “depress*”, but not a valid diagnosis statement. The resulting Depressed cohort contains 1,207 individuals and 1,759,644 tweets ranging from from May 2008 to September 2018.

To compare CDS prevalence rates of the Depressed cohort to a baseline, we construct a Random Sample cohort of individuals. To do so, we collect a large sample of random tweets in 3 weeks (i.e. September 1-8, 2017, March 1-8, 2018, and September 1-8, 2018) from the IUNI OSOME[15]. We extract all Twitter user identifiers from these tweets (N=588,356), and retain only those that specified their geographical location and were not already included in our Depressed cohort. To equalize platform, interface, and behavioral changes over time, we select a sub-sample of these individuals such that the distribution of their account creation dates matches those of the Depressed cohort, resulting in an initial set of 9,525 random individuals. Finally, we harvested the Twitter timelines of these users and filtered the obtained data in the same way as described for the Depressed cohort. Since some user data was found to be no longer publicly available and others have no tweets left after our filters, our final Random Sample Cohort consists of 8,791 individuals and a total 8,498,574 tweets.

The code and data used in this analysis are freely available at https://github.com/kbathina/CDs_Depressed_Twitter. Upon reasonable request we will provide all Twitter user IDs and tweet IDs to reproduce our results.

4.2 Prevalence Ratios

For each Twitter user in our sample, we retrieved a timeline of their time-ordered most recent tweets, . We also defined a set of n-grams where (see Table 2) with varying number of terms. The elements of set C are intended to represent the lexical building blocks of expressing cognitive distortions (see Table 2 and Appendix Table 7). We introduce a CDS matching function , which maps each individual tweet to either or according to whether a tweet contains one or more of the schemata in set . Note that the range of is binary, thus a tweet that contains more than one CDS still counts as .

The within-subject prevalence of tweets for individual is defined as the ratio of tweets that contain a CDS in over all tweets in their timeline :

Our sample is separated into two cohorts: one of 1,207 Depressed and another of 8,791 Random Sample individuals. We denote the set of all individuals in the depressed cohort and random sample cohort . Hence, the sets of all tweets written by users in the Depressed and Random Sample cohorts are defined as:

and (1)

We can then define the Prevalence () of tweets with CDS for each the Depressed () and Random Sample () cohorts as follows:

(2)

or, informally, the ratio of tweets that contain any CDS over all tweets written by the individuals of that cohort.

Consequently, the Prevalence Ratio () of CDS in set between the two cohorts and , denoted , is defined simply as the ratio of their respective CDS prevalence and in the tweet sets and respectively:

(3)

If the prevalence of CDS in the tweets of the depression cohort are comparable to their prevalence in the tweets of the random sample. However, any value or may indicate a significantly higher prevalence in each respective cohort. Here we use and to signifiy that a PR value is significantly higher or lower than 1 respectively, which we asses by whether its 95% CI includes or not (see Bootstrapping below).

4.3 Bootstrapping estimates

The estimated Prevalence and Prevalence Ratio can vary with the particular composition of either our set (CDS n-grams) or the set of individuals in our Depressed and Random Sample cohorts, respectively and . We verify the reliability of our results by randomly re-sampling either or both and , with replacement. This is repeated number of times, leading to a set of re-sampled CD sets or cohort samples. Each of these number of re-samples of either (1) the set of CDS or (2) or the sets and of all individuals in our Depressed and Random Sample cohorts results in number of corresponding Prevalence or Prevalence Ratio values:

(4)

The distributions of and are then characterized by their median () and their 95% confidence interval (). A 95% confidence interval of a PR that does not contain is held to indicate a significant difference in prevalence between the two cohorts.

5 Acknowledgements

We thank Luis M. Rocha for his feedback on the general methodology and terminology, as well as Drs. Keith Dobson, Rob DeRubeis, Christian Webb, Stephan Hoffman, Nikolaos Kazantzis, Judy Garber, and Robin Jarrett for their feedback on the content of our list of CDS. Johan Bollen thanks NSF grant #SMA/SME1636636, the Indiana University “Grand Challenges - Prepared for Environmental Change” PR-IUB grant, Wageningen University, and the ISI Foundation for their support.

References

Appendix

(%) (S%)
All CDs 21.838 18.407
Dichotomous Reasoning 16.650 13.933
Should statements 3.191 2.896
Magnification and Minimization 1.992 1.851
Labeling and mislabeling 1.199 0.903
Mindreading 1.168 1.026
Personalizing 1.026 0.427
Overgeneralizing 0.752 0.476
Disqualifying the Positive 0.081 0.060
Emotional Reasoning 0.053 0.023
Fortune-telling 0.047 0.050
Mental Filtering 0.024 0.016
Catastrophizing 0.014 0.019
Table 5: Raw CDS Prevalence in tweets from the Depressed and Random Sample cohorts in decreasing rank order.
Figure 5: Prevalence Ratio of CD markers without First-Person Pronouns (FPP) for each type of cognitive distortion between the Depressed and Random Sample cohort. The two black dots represent the 2.5th and 97.5th percentile of the bootstrapped distribution of means. The diamonds indicate the location of the median, with green, yellow, and red diamonds indicating Prevalence Ratio values significantly higher than parity (1) for the Depressed cohort, no significant difference from parity (yellow), and sigfificantly lower prevalence for the Depressed cohort than the Random Sample (red). Catastrophizing and Fortune-telling are significantly more prevalent in the Random Sample population () while the other types of CDS are more significant in the Depressed cohort ().
median 95% CI median 95% CI median 95% CI
All CDs 3.431 [2.912, 3.939] 3.069 [2.568, 3.561] 0.021 [0.013, 0.031]
Dichotomous Reasoning 2.714 [2.293, 3.130] 2.714 [2.294, 3.130] 0.143 [0.072, 0.231]
Personalizing 0.599 [0.537, 0.663] / / 0.041 [0.003, 0.110]
Overgeneralizing 0.276 [0.242, 0.311] 0.177 [0.151, 0.203] 0.013 [0.003, 0.025]
Labeling and Mislabeling 0.296 [0.244, 0.349] 0.146 [0.104, 0.189] 0.007 [0.002, 0.013]
Should Statements 0.297 [0.149, 0.436] 0.297 [0.149, 0.436] 0.060 [-0.050, 0.213]
Mindreading 0.140 [0.063, 0.233] 0.140 [0.063, 0.233] 0.002 [-0.001, 0.007]
Magnification and Minimization 0.139 [0.043, 0.235] 0.139 [0.043, 0.235] 0.018 [0.004, 0.033]
Emotional Reasoning 0.030 [0.025, 0.036] 0.006 [0.004, 0.008] 0.004 [0.001, 0.009]
Disqualifying the Positive 0.021 [0.013, 0.029] 0.021 [0.013, 0.029] 0.001 [0.000, 0.003]
Mental Filtering 0.008 [0.005, 0.010] 0.002 [0.000, 0.004] 0.001 [0.000, 0.001]
Fortune-telling -0.002 [-0.009, 0.003] -0.012 [-0.017, -0.008] -0.000 [-0.002, 0.003]
Catastrophizing -0.005 [-0.010, -0.002] -0.005 [-0.010, -0.002] -0.000 [-0.001, 0.000]
Table 6: Prevalence Difference (PD) percentages for set of CDS , defined as , and its 95% CIs between the Depressed and Random Sample cohort. PD values indicate a significantly higher prevalence in the Depressed sample (marked by ). Values are calculated under 3 distinct conditions, labeled : PD percentage difference values for the entire set of CD markers, : PD percentage difference values for CD markers without first-person pronouns (“I”,“me”, “my”, “mine”, and “myself”), and : PD percentage values with a 95% CIs resulting from re-sampling the CD marker set 10,000 times, instead of the individuals in the Depressed and Random Sample cohorts (see Main: Materials and Methods).
Figure 6: Median and 95% confidence interval of the bootstrapped PR means of CDS n-grams that were significantly more prevalent, i.e. or , in the Depressed or Random Sample cohort. CDS n-grams are colored by the type of CD they belong to.
Category CD Markers

Catastrophizing
will fail, will go wrong, will end, will be impossible, will not happen, will be terrible, will be horrible, will be a catastrophe, will be a disaster, will never end, will not end

Dichotomous Reasoning
only, every, everyone, everybody, everything, everywhere, always, perfect, the best, all, not a single, no one, nobody, nothing, nowhere, never, worthless, the worst, neither, nor, either or, black or white, ever

Disqualifying the Positive
great but, good but, OK but, not that great, not that good, it was not, not all that, fine but, acceptable but, great yet, good yet, OK yet, fine yet, acceptable yet

Emotional Reasoning
but I feel, since I feel, because I feel, but it feels, since it feels, because it feels, still feels

Fortune-telling
I will not, we will not , you will not, they will not, it will not, that will not, he will not, she will not

Labeling and Mislabeling
I am a, he is a, she is a, they are a, it is a, that is a, sucks at, suck at, I never, he never, she never, you never, we never, they never, I am an, he is an, she is an, they are an, it is an, that is an, a burden, a complete, a completely, a huge, a loser, a major, a total, a totally, a weak, an absolute, an utter, a bad, a broken, a damaged, a helpless, a hopeless, an incompetent, a toxic, an ugly, an undesirable, an unlovable, a worthless, a horrible, a terrible

Magnification and Minimization
worst, best, not important, not count, not matter, no matter, the only thing, the one thing

Mental Filtering
I see only, all I see, all I can see, can only think, nothing good, nothing right, completely bad, completely wrong, only the bad, only the worst, if I just, if I only, if it just, if it only

Mindreading
everyone believes, everyone knows, everyone thinks, everyone will believe, everyone will know, everyone will think, nobody believes, nobody knows, nobody thinks, nobody will believe, nobody will know, nobody will think, he believes, he knows, he thinks, he does not believe, he does not know, he does not think, he will believe, he will know, he will think, he will not believe, he will not know, he will not think, she believes, she knows, she thinks, she does not believe, she does not know, she does not think, she will believe, she will know, she will think, she will not believe, she will not know, she will not think, they believe, they know, they think, they do not believe, they do not know, they do not think, they will believe, they will know, they will think, they will not believe, they will not know, they will not think, we believe, we know, we think, we do not believe, we do not know, we do not think, we will believe, we will know, we will think, we will not believe, we will not know, we will not think, you believe, you know, you think, you do not believe, you do not know, you do not think, you will believe, you will know, you will think, you will not believe, you will not know, you will not think

Overgeneralizing
all of the time, all of them, all the time, always happens, always like, happens every time, completely, no one ever, nobody ever, every single one of them, every single one of you, I always, you always, he always, she always, they always, I am always, you are always, he is always, she is always, they are always

Personalizing
all me, all my, because I, because my, because of my, because of me, I am responsible, blame me, I caused, I feel responsible, all my doing, all my fault, my bad, my responsibility

Should Statements
should, ought, must, have to, has to

Table 7: Cognitive Distortion types and corresponding schemata
Figure 7: To prevent the calculation of within-subject prevalence values for individuals with too few data points, we calculate the within-subject prevalence values only for individuals with a given minimum number of tweets (150). To determine the sensitivity of our results to the chosen value of this threshold, we calculate the KS-test statistic of the distributions of within-subject CDS prevalence given each threshold value (x-axis). The blue line shows the KS statistic for the raw prevalence distribution while the orange line shows the KS statistic for the Probability Distribution Function (PDF). We visually determine an inflection point in both graphs at 150 minimum tweets at which point the corresponding p-value for the KS-test stabilizes at and respectively. Consequently we calculate within-subject prevalence values only for individuals in either cohort for which we retrieved at least 150 tweets. We can reject the null-hypothesis that the distributions were drawn from the same population for all threshold values indicating the conclusions we draw from a comparison of the within-subject CDS prevalences between the Depressed and the Random Sample cohort are not sensitive to our choice of threshold. It is also important to note that we do not apply this individual threshold to the calculation of between-group Prevalence Ratios, since this indicator is calculated at the group level pertaining to millions of tweets and not calculated at the level of individual users.

Footnotes

  1. Value is below 32byte floating point precision.
  2. https://developer.twitter.com/en/docs/tweets/timelines/api-reference/get-statuses-user_timeline

References

  1. M. Al-Mosaiwi and T. Johnstone (2018) In an absolute state: elevated use of absolutist words is a marker specific to anxiety, depression, and suicidal ideation. Clinical Psychological Science 6 (4), pp. 529–542. Cited by: §2.1.
  2. A. T. Beck and E. A. Haigh (2014) Advances in cognitive theory and therapy: the generic cognitive model. Annual review of clinical psychology 10, pp. 1–24. Cited by: §1.
  3. A. T. Beck (1963) Thinking and depression: I. Idiosyncratic content and cognitive distortions. Archives of general psychiatry 9 (4), pp. 324–333. Cited by: §1.1.
  4. A. T. Beck (1964) Thinking and depression: II. Theory and therapy. Archives of general psychiatry 10 (6), pp. 561–571. Cited by: §1.1.
  5. J. S. Beck and A. T. Beck (1995) Cognitive therapy: basics and beyond. Guilford press New York. Cited by: §1.1, Table 1.
  6. J. D. Bernard, J. L. Baddeley, B. F. Rodriguez and P. A. Burke (2016) Depression, language, and affect: an examination of the influence of baseline depression and affect induction on language. Journal of Language and Social Psychology 35 (3), pp. 317–326. Cited by: §1.
  7. T. Brockmeyer, J. Zimmermann, D. Kulessa, M. Hautzinger, H. Bents, H. Friederich, W. Herzog and M. Backenstrass (2015) Me, myself, and I: self-referent word use as an indicator of self-focused attention in relation to depression and anxiety. Frontiers in psychology 6, pp. 1564. Cited by: §2.1.
  8. D. Burns (1989) The feeling good handbook. Harper-Collins Publishers. Cited by: §1.1.
  9. F. Cacheda, D. Fernandez, F. J. Novoa and V. Carneiro (2019) Early detection of depression: social network analysis and random forest techniques.. Journal of medical Internet research 21, pp. e12554. Cited by: §1.
  10. A. Case and A. Deaton (2015) Rising morbidity and mortality in midlife among white non-hispanic americans in the 21st century. Proceedings of the National Academy of Sciences 112 (49), pp. 15078–15083. Cited by: §1.
  11. P. A. Cavazos-Rehg, M. J. Krauss, S. Sowles, S. Connolly, C. Rosas, M. Bharadwaj and L. J. Bierut (2016) A content analysis of depression-related tweets. Comput Human Behav 54, pp. 351–357. Cited by: §1.
  12. M. D. Choudhury, M. Gamon, S. Counts and E. Horvitz (2013-07) Predicting depression via social media. In Proceedings of the Seventh International AAAI Conference on Weblogs and Social Media (ICWSM-13), pp. 128–137. Cited by: §1.
  13. D. A. Clark and A. T. Beck (2010) Cognitive theory and therapy of anxiety and depression: convergence with neurobiological findings. Trends in cognitive sciences 14 (9), pp. 418–424. Cited by: §1.
  14. P. Cuijpers, S. D. Hollon, A. van Straten, C. Bockting, M. Berking and G. Andersson (2013) Does cognitive behaviour therapy have an enduring effect that is superior to keeping patients on continuation pharmacotherapy? A meta-analysis. BMJ open 3 (4), pp. e002542. Cited by: §1.
  15. C. A. Davis, G. L. Ciampaglia, L. M. Aiello, K. Chung, M. D. Conover, E. Ferrara, A. Flammini, G. C. Fox, X. Gao and B. Gonçalves (2016) OSoMe: the IUNI observatory on social media. PeerJ Computer Science 2, pp. e87. Cited by: §4.1, §4.1.
  16. R. J. DeRubeis, G. J. Siegle and S. D. Hollon (2008) Cognitive therapy versus medication for depression: treatment outcomes and neural mechanisms. Nature Reviews Neuroscience 9 (10), pp. 788–796. Cited by: §1.
  17. P. S. Dodds, E. M. Clark, S. Desu, M. R. Frank, A. J. Reagan, J. R. Williams, L. Mitchell, K. D. Harris, I. M. Kloumann, J. P. Bagrow, K. Megerdoomian, M. T. McMahon, B. F. Tivnan and C. M. Danforth (2015-02) Human language reveals a universal positivity bias. Proceedings of the National Academy of Sciences 112 (8), pp. 2389–2394. Cited by: §2.1.
  18. J. C. Eichstaedt, R. J. Smith, R. M. Merchant, L. H. Ungar, P. Crutchley, D. Preoţiuc-Pietro, D. A. Asch and H. A. Schwartz (2018) Facebook language predicts depression in medical records. Proceedings of the National Academy of Sciences 115 (44), pp. 11203–11208. External Links: Document, https://www.pnas.org/content/115/44/11203.full.pdf, ISSN 0027-8424, Link Cited by: §1.
  19. M. P. Ewbank, R. Cummins, V. Tablan, S. Bateup, A. Catarino, A. J. Martin and A. D. Blackwell (2019) Quantifying the association between psychotherapy content and clinical outcomes using deep learning. JAMA psychiatry. Cited by: §3.
  20. R. Fan, O. Varol, A. Varamesh, A. Barron, I. A. van de Leemput, M. Scheffer and J. Bollen (2019) The minute-scale dynamics of online emotions reveal the effects of affect labeling. Nature Human Behavior 3, pp. 92–100. Cited by: §1.
  21. L. A. Fast and D. C. Funder (2010) Gender differences in the correlates of self-referent word use: authority, entitlement, and depressive symptoms. Journal of personality 78 (1), pp. 313–338. Cited by: §2.1.
  22. L. C. Foland-Ross and I. H. Gotlib (2012) Cognitive and neural aspects of information processing in major depressive disorder: an integrative perspective. Frontiers in psychology 3, pp. 489. Cited by: §1.
  23. P. E. Greenberg, A. Fournier, T. Sisitsky, C. T. Pike and R. C. Kessler (2015) The economic burden of adults with major depressive disorder in the United States (2005 and 2010). The Journal of clinical psychiatry 76 (2), pp. 155–162. Cited by: §1.
  24. S. G. Hofmann, A. Asnaani, I. J. Vonk, A. T. Sawyer and A. Fang (2012) The efficacy of cognitive behavioral therapy: a review of meta-analyses. Cognitive therapy and research 36 (5), pp. 427–440. Cited by: §1.
  25. C. J. Hutto and E. Gilbert (2014) Vader: a parsimonious rule-based model for sentiment analysis of social media text. In Eighth international AAAI conference on weblogs and social media, Cited by: §2.1.
  26. R. E. Ingram (1990) Self-focused attention in clinical disorders: review and a conceptual model.. Psychological bulletin 107 (2), pp. 156. Cited by: §2.1.
  27. D. Jurafsky and J. H. Martin (2008) Speech and language modeling. Cited by: §1.
  28. K. Kalimeri, M. G. Beiro, A. Bonanomi, A. Rosina and C. Cattuto (2020-01) Traditional versus facebook-based surveys: evaluation of biases in self-reported demographic and psychometric information. Demographic Research 42, pp. 133–148. Cited by: §3.
  29. B. Keles, N. McCrae and A. Grealish (2020) A systematic review: the influence of social media on depression, anxiety and psychological distress in adolescents. International Journal of Adolescence and Youth 25 (1), pp. 79–93. Cited by: §3.
  30. Y. Kelly, A. Zilanawala, C. Booker and A. Sacker (2018) Social media use and adolescent mental health: findings from the uk millennium cohort study. EClinicalScience (The Lancet) 6, pp. 59–68. Cited by: §3.
  31. L. Y. Lin, J. E. Sidani, A. Shensa, A. Radovic, E. Miller, J. B. Colditz, B. L. Hoffman, L. M. Giles and B. A. Primack (2016) Association between social media use and depression among us young adults. Depression and anxiety 33 (4), pp. 323–331. Cited by: §3.
  32. L. Lorenzo-Luaces, R. E. German and R. J. DeRubeis (2015) It’s complicated: the relation between cognitive change procedures, cognitive change, and symptom change in cognitive therapy for depression. Clinical psychology review 41, pp. 3–15. Cited by: §1.
  33. A. J. Mitchell, A. Vaze and S. Rao (2009) Clinical diagnosis of depression in primary care: a meta-analysis. The Lancet 374 (9690), pp. 609–619. Cited by: §1.
  34. R. Mojtabai, M. Olfson and B. Han (2016) National trends in the prevalence and treatment of depression in adolescents and young adults. Pediatrics 138 (6), pp. e20161878. Cited by: §1.
  35. M. L. Molendijk, L. Bamelis, A. A. van Emmerik, A. Arntz, R. Haringsma and P. Spinhoven (2010) Word use of outpatients with a personality disorder and concurrent or previous major depressive disorder. Behaviour Research and Therapy 48 (1), pp. 44–51. Cited by: §2.1.
  36. R. R. Morris, S. M. Schueller and R. W. Picard (2015) Efficacy of a web-based, crowdsourced peer-to-peer cognitive reappraisal platform for depression: randomized controlled trial. Journal of medical Internet research 17 (3), pp. e72. Cited by: §3.
  37. K. Ozdel, I. Taymur, S. O. Guriz, R. G. Tulaci, E. Kuru and M. H. Turkcapar (2014) Measuring cognitive errors using the Cognitive Distortions Scale (CDS): Psychometric properties in clinical and non-clinical samples. PloS one 9 (8). Cited by: §1.
  38. F. N. Ribeiro, M. Araújo, P. Gonçalves, M. A. Gonçalves and F. Benevenuto (2016) Sentibench-a benchmark comparison of state-of-the-practice sentiment analysis methods. EPJ Data Science 5 (1), pp. 23. Cited by: §2.1.
  39. S. Rude, E. Gortner and J. Pennebaker (2004) Language use of depressed and depression-vulnerable college students. Cognition & Emotion 18 (8), pp. 1121–1133. Cited by: §1, §2.1, §2.2.
  40. D. Smirnova, P. Cumming, E. Sloeva, N. Kuvshinova, D. Romanov and G. Nosachev (2018) Language patterns discriminate mild depression from normal sadness and euthymic state. Frontiers in Psychiatry 9, pp. 105. Cited by: §1, §2.2.
  41. A. M. Tackman, D. A. Sbarra, A. L. Carey, M. B. Donnellan, A. B. Horn, N. S. Holtzman, T. S. Edwards, J. W. Pennebaker and M. R. Mehl (2019) Depression, negative emotionality, and self-referential language: a multi-lab, multi-measure, and multi-language-task research synthesis.. Journal of Personality and Social Psychology 116, pp. 817–834. Cited by: §1.
  42. A. S. Troy, F. H. Wilhelm, A. J. Shallcross and I. B. Mauss (2010) Seeing the silver lining: cognitive reappraisal ability moderates the relationship between stress and depressive symptoms.. Emotion 10 (6), pp. 783. Cited by: §1.
  43. I. A. van de Leemput, M. Wichers, A. O. J. Cramer, D. Borsboom, F. Tuerlinckx, P. Kuppens, E. H. van Nes, W. Viechtbauer, E. J. Giltay, S. H. Aggen, C. Derom, N. Jacobs, K. S. Kendler, H. L. J. van der Maas, M. C. Neale, F. Peeters, E. Thiery, P. Zachar and M. Scheffer (2014) Critical slowing down as early warning for the onset and termination of depression. PNAS 111, pp. 87–92. Cited by: §1.
  44. P. S. Wang, M. Lane, M. Olfson, H. A. Pincus, K. B. Wells and R. C. Kessler (2005) Twelve-month use of mental health services in the united states: results from the national comorbidity survey replication. Archives of general psychiatry 62 (6), pp. 629–640. Cited by: §1.
  45. T. L. Webb, E. Miles and P. Sheeran (2012) Dealing with feeling: a meta-analysis of the effectiveness of strategies derived from the process model of emotion regulation.. Psychological bulletin 138 (4), pp. 775. Cited by: §1.
  46. World Health Organization (2017) Depression and other common mental disorders: global health estimates. Geneva: WHO. Cited by: §1.
  47. J. Zimmermann, T. Brockmeyer, M. Hunn, H. Schauenburg and M. Wolf (2017) First-person pronoun use in spoken language as a predictor of future depressive symptoms: preliminary evidence from a clinical sample of depressed patients. Clinical psychology & psychotherapy 24 (2), pp. 384–391. Cited by: §1.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
""
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
   
Add comment
Cancel
Loading ...
407082
This is a comment super asjknd jkasnjk adsnkj
Upvote
Downvote
""
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters
Submit
Cancel

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test
Test description