Predicting Financial Markets: Comparing Survey, News, Twitter and Search Engine Data

Predicting Financial Markets: Comparing Survey, News, Twitter and Search Engine Data

Huina Mao,  Scott Counts,  and Johan Bollen, 
Abstract

Financial market prediction on the basis of online sentiment tracking has drawn a lot of attention recently. However, most results in this emerging domain rely on a unique, particular combination of data sets and sentiment tracking tools. This makes it difficult to disambiguate measurement and instrument effects from factors that are actually involved in the apparent relation between online sentiment and market values. In this paper, we survey a range of online data sets (Twitter feeds, news headlines, and volumes of Google search queries) and sentiment tracking methods (Twitter Investor Sentiment, Negative News Sentiment and Tweet & Google Search volumes of financial terms), and compare their value for financial prediction of market indices such as the Dow Jones Industrial Average, trading volumes, and market volatility (VIX), as well as gold prices. We also compare the predictive power of traditional investor sentiment survey data, i.e. Investor Intelligence and Daily Sentiment Index, against those of the mentioned set of online sentiment indicators. Our results show that traditional surveys of Investor Intelligence are lagging indicators of the financial markets. However, weekly Google Insight Search volumes on financial search queries do have predictive value. An indicator of Twitter Investor Sentiment and the frequency of occurrence of financial terms on Twitter in the previous 1-2 days are also found to be very statistically significant predictors of daily market log return. Survey sentiment indicators are however found not to be statistically significant predictors of financial market values, once we control for all other mood indicators as well as the VIX.

Financial prediction, behavioral finance, sentiment analysis, investor sentiment, Twitter mood, search engine, news media.

I Introduction

The efficient market hypothesis (EMH) asserts that financial market valuations incorporate all existing, new, and even hidden information, since investors act as rational agents who seek to maximize profits. Behavioral finance [13] has challenged this notion by emphasizing the important role of behavioral and emotional factors, including social mood [17], in financial decision-making. As a consequence, measuring investor and social mood has become a key research issue in financial prediction.

Traditionally, public and investor mood are measured by surveys. For example, the Gallup Life Evaluation Index measures the general well-being of the US public on a daily basis by conducting a survey across a representative sample of the US population. Investor mood is likewise assessed by surveys, in which investors or newsletter writers rate their current stance on the market, e.g. Daily Investor Sentiment 111www.trade-futures.com and Investor Intelligence 222http://www.investorsintelligence.com/x/us_advisors_sentiment.html. In spite of their popularity, surveys are, however, resource intensive and thus expensive to conduct, and can be subject to problems related to responder truthfulness [9, 20], individual biases, social biases, and group think.

In recent years, researchers have explored a variety of methods to compute indicators of the public’s sentiment and mood state from large-scale online data. This approach holds considerable promise. First, computational analysis of public sentiment or mood may be more rapid, accurate and cost-effective to conduct than large-scale surveys. Second, there now exists considerable support for the claim that the resulting public mood and sentiment indicators are indeed valid measurements of public sentiment and mood, even to the degree that they have been found to predict a variety of socio-economic phenomena, including presidential elections [23], commercial sales [7, 16], and influenza epidemics [8]. It is of considerable interest to behavioral finance that a respectable and growing amount of literature in this area has shown that computational indicators of public sentiment may also have predictive value with respect to financial market movements [1, 4, 5, 9, 10, 18].

To the best of our knowledge, three distinct classes of online data sources have been investigated for financial prediction. First, news media content has been shown to be an important factor shaping investor sentiment. For instance, Tetlock found that high levels of pessimism in the Wall Street precede lower market returns the following day [21]. This effect has also been observed at the level of individual firms, with high negative sentiment forecasting lower firm earnings [22]. In [19] it was shown that adding textual features of news to a stock prediction system can improve the forecasting accuracy.

Second, web search (query) data has been shown to be related to and even predictive of market fluctuations. Search volumes of stock names reveal investor attention and interest, and high search volumes thus predict higher stock prices in the short-term, and price reversals in the long-term [9]. Also, search volumes of stocks correlate highly with trading volumes of the corresponding stocks, with peaks of search volume anticipating peaks of trading volume by one day or more [5]. Similar phenomena have been found at the weekly level [18].

Third, social media feeds are becoming an important source of data to support the measurement of investor and social mood extraction. In an early study, Internet stock message boards were studied to predict market volatility and trading volumes [1]. In past couple of years, public mood indicators extracted from social networks such as Facebook [14], LiveJournal [11] and Twitter [4] have been used to predict stock market fluctuations.

Together these results are highly suggestive that a variety of web-scale data sources may provide predictive power in financial analytics. However, each of the mentioned investigations uses different types of web data to predict different financial indicators. It is not clear which mood indicators constructed from particular data sources most effectively capture investor mood-related signals and thereby provide the best predictive power.

In this paper, we therefore collect multiple data sources, i.e. surveys, news headlines, search engine data and Twitter feeds, from which we define a variety of sentiment indicators, i.e. Survey Investor Sentiment, Negative News Sentiment, Google search volumes of financial terms, Twitter Investor Sentiment and Tweet volumes of financial terms. Subsequently, we determine the predictive value of these sentiment indicators over a range of financial indicators, i.e. Dow Jones Industrial Average price, trading volumes, market volatility (VIX) and the price of gold.

Ii Data Collection and Sentiment Analysis

In this section we outline our data collection methods, and how we computed investor sentiment indicators from Twitter, news, and search engine data.

Ii-a Survey Data

Surveys are the most direct and common method for collecting investor sentiment. Investor Intelligence (II), published by an investment services company, determines whether opinion in over one hundred independent market newsletters points towards a bullish, bearish or correction market. II has been available at a weekly level dating to 1964. Daily Sentiment Index (DSI) provides daily market sentiment readings on all active US markets daily since 1987, and is one of the most popular short-term market sentiment indices for futures traders. High vs. low DSI values of respectively above 90% or below 10%, suggests that a short-term top or bottom is either developing, or has been achieved.

Ii-B News Media

We chose eight news media outlets to collect our news data from: Wall Street Journal, Bloomberg, Forbes.com, Reuters Business& Finance, BusinessWeek, Financial Times, CNNMoney and CNBC. These are the top news sources for financial traders and investors. In order to track recent and featured news from these sources, we followed their respective Twitter accounts (“wsjusnews”, “wsjbreakingnews”, “wsjmarkets”, “bloombergnews”, “bloombergnow”, “bloomberg”, “forbes”, “BusinessWeek”, “Reuters Business”, “reuters_biz”, “financialtimes”, “FinancialTimes”, “CNNMoney”, “CNBC”). We then extracted and parsed the URLs from these tweets, saving the story headlines as our news corpus. This approach of using headlines is based on previous research that studied stock price reaction to news headlines [6].

Previous research has demonstrated that negative mood seems to be more predictive of financial market values than positive mood [21]. There are two well-accepted financial lexicons for negative word identification. One is the Harvard IV-4 dictionary 333http://www.wjh.harvard.edu/~inquirer/ as used in [21, 22]. The other 444http://www.nd.edu/~mcdonald/Word_Lists.html is developed by Loughran and McDonald in [15], which is shown to better reflect the tone of financial text than the Harvard IV-dictionary. In our paper, we apply the latter financial negative lexicon to our news headlines. We count the total number of words in a news headline and take the ratio of the number of negative sentiment words to the total number of words in the headline. Then, we sum the emotional ratio and divide by the total number of news articles on the same day, yielding our Negative News Sentiment score. Fig. 1 shows the example of top negative financial terms of the news headlines from July 31st to August 9th 2011, when the DJIA dropped while market volatility increased. As a result words such as “downgrade”, “cut”,“crisis” and “losses” frequently occur in news headlines in that period.

Fig. 1: Frequency of negative terms in News headlines from July 31st to August 9th 2011.

Ii-C Search Engine Data

Previous research has shown that search volume itself can be a mood indicator for financial market [5, 9, 10, 18]. In [9], it has been shown that the more people search on economic negative terms such as “recession” and “bankruptcy”, the more pessimistic people feel about the economy. To create a search query-based indicator of financial mood, we took the following steps. First, we downloaded the weekly search volume data for a set of seed queries including “dow jones”, “stock market”, “stock to buy”, “stock”, “bullish”, “bearish”, “financial news” and “wall street” from Google Insights for Search (GIS)555http://www.google.com/insights/search/. GIS is a Google service that provides search volume data from January 2004 to the present. Second, to more fully capture search activity related to the financial markets we expanded these seed keywords with those terms that are top relevant search terms as recommend by GIS. This procedure resulted in a lexicon of about 26 financial search terms for which we again retrieved GIS search frequency indices, resulting in a time series of GIS frequencies for all searches containing those 26 terms as shown in Table I.

DJIA, Dow,Dow Jones, Dow Jones Industrial Average,
bearish, bear market,best stock, bullish, bull market,
finance, finance news, financial news, financial market,
long stock, SP500, stock, stock market, stock decline, stock fall,
stock market crash, stock market news, stock market today,
stock price, stock to buy, wall street, wall street news today


TABLE I: 26 Search Terms

Ii-D Social Media Data

The enormous amount of social media data that has become available in recent years has provided significant research opportunities for social scientists and computer scientists. In fact, Twitter, which is now one of the most popular microblogging services, has been extensively used for real time sentiment tracking and public mood modeling [3, 12]. And its financial predictive power has also been explored. In [2] , it has been shown that Twitter content and sentiment can be used to forecast box-office revenues of movies. In [24], the correlation between emotional tweets and financial market indicators are studied, indicating that the percentage of emotional tweets is significantly negatively correlated with Dow Jones, NASDAQ and S&P500 values, but positively correlated with VIX values. Moreover, in [4] a six-dimensional model of public emotions is derived from Twitter (Calm, Alert, Sure, Vital, Kind and Happy) and found to have significant predictive power with respect to DJIA fluctuations.

In this paper, we use a 15%-30% random sample of all public tweets posted every day from July 2010 to September 2011. From this collection, we define two Twitter-based financial mood indicators: Twitter Investor Sentiment (TIS) and Tweet volumes of financial search terms (TV-FST). These are discussed in greater detail below.

Ii-D1 Twitter Investor Sentiment

We simply define a tweet as bullish if it contains the term “bullish”, and bearish if it contains the “bearish”. On the basis of the number of Bearish and Bullish tweets on a given day, we define the investor sentiment score, Twitter Investor Sentiment () on day , denoted as follows:

(1)

where is the number of bullish tweets on day and is the number of bearish tweets on day .

Ii-D2 Tweet Volumes of Financial Search Terms (TV-FST)

As mentioned in Section II-C, search query volume of stock names and various financial/economic terms has been used in previous research as proxies of public and investor mood. Our proposal is to apply a similar approach to define our Tweet Volumes of Financial Search Terms indicator (TV-FST). We want to compare Tweet volumes and Search volumes of the same search queries. To do so, we use the following procedure for data processing: First, we compute both the weekly Google search volumes (GIS) and daily Tweet volumes of those 26 financial search terms from July 2010 to September 2011. Second, we calculate the weekly mean over the daily volumes of tweets. This step is necessary to compare Twitter (daily) and GIS (weekly) at the same time scale. Third, and finally, we take the average of the separate, weekly time series generated for each individual term, which yields a GIS and Tweet volume time series over 66 weeks, for the combination of all the financial search terms. Fig. 2 shows these two time series.

Fig. 2: Weekly TV-FST vs. GIS.

A correlation analysis over all weekly values of the two time series reveals a statistically significant Pearson correlation coefficient of 0.62 (). To see whether these two indicators signal notable movement in the financial market, we marked the time period from July 23rd to August 20th 2011 in a shaded rectangle as shown in Fig. 2. During this period, the stock market had a huge decline (i.e., the DJIA declined 1864 points between July 22nd and August 19th 2011.) We can see that from June 4th, 2011 (at the first vertical line), TV-FST values started to increase, while 5 weeks later, on July 9th 2011 (at the second vertical line), GIS followed. This suggests that GIS may be less efficient than Twitter in revealing public/investor negative sentiment.

Ii-E Economic and Financial Market Data

We collected daily and weekly Dow Jones Industrial Average, trading volume, Volatility (VIX) from Yahoo! Finance. In addition, we calculate the market log returns of stock prices over a time interval as follows:

(2)

Here . Additionally, we also retrieved the price of gold 666http://www.gold.org/investment/statistics/gold_price_chart/ over the same period of time. Table II summarizes the corresponding time range and daily/weekly scale for all the data we obtained.

Data Daily(mm/dd/yy) Weekly(mm/yy)
DSI (Daily Sentiment Index) 07/01/10 -09/05/11 /
II (Investor Intelligence) / 01/08 - 09/11
TIS (Twitter Investor Sentiment) 07/01/10 – 09/29/11 /

TV-FST (Tweet volumes

of financial search terms)

07/01/10 – 09/29/11 /
NNS(Negative News Sentiment) 07/01/10 – 09/29/11 /
GIS (Google Insight Search) / 01/08 - 09/11
DJIA/VIX/Volume/Gold 07/01/10 – 09/29/11 01/08 - 09/11

TABLE II: Time-range coverage of different data sources.

Iii Search Volume (GIS)-based prediction of financial indicators

Iii-a Search Volume and Financial Indicator Correlations

In this section, we compare the GIS time series (search query volume of 26 financial search terms) with the DJIA price, volume, and the price of gold from January 2008 to September 2011, roughly 196 weeks. This period was punctuated by significant market volatility, as well as significant bear and bull markets, thus allowing us to perform our analysis under a variety of market conditions.

We first compute the pair-wise correlation between our 26 time series of GIS search terms and the financial time series. All time series are transformed to log scale for analysis. The results are summarized in Table III. Due to the space limitations, we only list the correlations of 10 search terms.

Search Query VIX DJIA Volume
DJIA 0.88 -0.76 0.69
Dow Jones 0.84 -0.69 0.68
Dow 0.83 -0.67 0.68
Dow Jones Industrial Average 0.78 -0.77 0.65
Stock market news 0.77 -0.37 0.59
Finance 0.71 -0.50 0.70
Stock market today 0.69 -0.62 0.51
Financial news 0.68 -0.43 0.57
Stock 0.66 -0.38 0.57
SP500 0.65 -0.34 0.49
TABLE III: Pearson correlation coefficients between GIS and VIX, DJIA, Trading Volume.

We find relatively strong correlations in most cases, especially for what seem to be DJIA-relevant search terms such as “DJIA”, “Dow Jones”, etc. The GIS time series has a positive correlation with the VIX and trading volumes, but negative correlations with DJIA, which may indicate that as more people search on financial terms, the market will be more volatile (i.e. high VIX), and trading volumes will be higher, while DJIA prices will move lower.

For further testing, we keep the top search term whose search volume has the highest correlation with the corresponding financial index for each time series. In Fig. 3, we overlaid the resulting time series with the mentioned financial indicators to visually examine the occurrence of any particular trend.

The top panels of Fig. 3 show the actual time series whereas the lower panels show the scatter plot of GIS values vs. financial indicator values in log-log scale. A simple visual inspection of the top panels reveal a clear correlation between GIS search term volumes and the financial indicator time series; peaks in GIS values generally co-occur with those of VIX and Volume values, and in some cases even precede the peaks of the various financial time series (DJIA, Gold). The scatter plots in Fig. 3 show that search volumes exhibit a high positive correlation with VIX and trading volume (, ), and a high negative correlation with DJIA price (). The correlation between gold price and search volumes on “gold” is also satisfactory (). This correlation value may in fact be an underestimation due to non-linear patterns in how the two variables relate. For (gold prices) we do observe a linear pattern of correlation. Below that value there seems to be little to no correlation. This pattern is confirmed by the trend plot at the upper right of Fig. 3: from mid-2010 to the end, at higher gold prices, we indeed observe a strong positive correlation, and in fact two spikes of search volumes appear before the gold price reached its peak in early September 2011.

Fig. 3: Trend analysis and log scale scatter plots of GIS time series vs. financial indicators such as VIX, DJIA closing values, gold price and DJIA trading volume. (Search query terms are inside the brackets).

VIX is a widely used measure of market risk and is often referred to as the “investor fear gauge”. Our results show that search volumes of financial terms reflect VIX fluctuations, implying that search volume for key financial terms may be a computational gauge of “investor fear”.

To evaluate time-lag correlations between search volume and financial time series, we compute their cross-correlation. In order to compare the effectiveness of search volumes with the survey data with respect to how well they predict the financial markets, we also include the Investor Intelligence (II) time series in our analysis.

Consider two series and , the cross correlation at lag is then defined as:

(3)

where and are the sample mean values of the and , respectively. We use the cross-correlation function provided in , an R statistics package. For example, where estimates the correlation between and , it means that we keep still, but move forward or backward in time by a lag of . Where , it means anticipates , and vice versa.

As can be seen in Fig. 4, DJIA values and GIS (search volume) exhibit the highest correlation and particularly so on the right side of the graph where lag values are positive, i.e. , and, in other words, GIS values lead DJIA values. A similar effect can be observed for GIS vs. VIX values, especially where weeks. In contrast, as shown in Fig. 4, the cross correlation between II and VIX seems to work in the opposite direction, indicating that VIX leads changes in II values. The correlation coefficients at both sides seem to be roughly balanced for trading volume. The search query time series for “gold” exhibits the opposite effect of other search query time series: GIS search volumes on “gold” do not lead gold prices. This runs counter to our earlier observation (in Fig. 3) that spikes of “gold” search volumes precede spikes in gold prices, indicating that “gold” GIS may yet have predictive value under certain conditions. We speculate this may be due to a non-linear interaction with absolute gold price levels, but we leave this for future exploration.

Fig. 4: Cross correlation analysis between financial time series and search volume (GIS) time series.

Iii-B Granger Causality Analysis

We further refine the observations discussed above by a Granger causality test, a technique that is widely used to analyze the relations between economic time series. The Granger causality test is a statistical hypothesis test to determine whether a time series is useful in forecasting another time series by attempting to reject the null hypothesis that does not help predict, i.e. Granger-cause, . The alternative hypothesis is that adding does help predict . An F-test is conducted to examine if the null hypothesis can be rejected.

We caution that Granger causality analysis might establish that the lagged value of exhibits a statistically significant correlation with . However, correlation does not prove causation. In other words, Granger causality testing does not establish actual causality, merely a statistical pattern of lagged correlation. This is similar to the observation that cloud cover may precede rain and may thus be used to predict rain, but does not itself actually cause rain.

Table IV presents the results of applying the Granger causality test in two directions, i.e. with positive and negative lags, reflecting the hypothesis that each time series may Granger cause the other.

1 2 3
VIXGIS 0.0051 0.0004 0.0010
GISVIX 0.0025 0.0202 0.0091
VIXII 8.04e-05 3.63e-07 9.98e-08
IIVIX 0.398 0.726 0.849
DJIAGIS 0.207 0.040 0.096
GISDJIA 7.85e-04 1.48e-03 9.31e-04
VolumeGIS 0.409 0.705 0.843
GISVolume 0.020 0.028 0.101
GoldGIS 0.055 0.104 0.082
GISGold 0.139 0.00036 0.0013
(: , : , : )
TABLE IV: Statistical significance (p-values) of Granger causality analysis between search volumes/ II and financial indicators over lags of 1, 2, and 3 weeks.

The values in the first column of Table IV represent the particular hypothesis under consideration. For example, “VIXGIS” represents the null hypothesis that adding VIX does not help predict GIS. As can be seen from the listed -values, this particular null-hypothesis is rejected with a high level of confidence. In the row below, we observe that adding GIS can also help predict VIX. However, the Granger causality between Investor Intelligence (II) and VIX runs in only one direction, i.e. VIXII: adding survey data (II) does not help predict VIX. In addition, the null hypothesis that adding GIS does not help predict DJIA, is strongly rejected at a high level of confidence level. Similarly, we find a very significant -value for GISGold at lag 2 and 3 weeks. GIS of the previous 1 to 2 weeks significantly Granger-cause trading volume.

Iii-C Forecasting Analysis

Can search volumes predict future values of financial indicators? As a further validation, we conduct a 1-step ahead prediction over 20 weeks based on a baseline model, denoted , and an advanced model, denoted . Here represents the particular financial index (i.e. DJIA, trading volumes or VIX) and represents a sentiment indicator. In this section we will focus on GIS in particular.

(4)
(5)

Forecasting accuracy is measured in terms of the Mean Absolute Percentage Error (MAPE) and the direction accuracy. The MAPE is defined as follows:

(6)

where is the predicted value and is the actual value. Direction accuracy is measured simply in terms of whether . In other words, if the difference between today’s and yesterday’s predicted value has the same sign as the difference between today’s vs. yesterday’s observed value, we conclude that the direction of the change was predicted accurately for that day.

Our search volume and financial indicator time series are available from January 2008 to September 2011. There are 196 weeks in total and we use the last 20 weeks, i.e. May 21st 2011 to October 1st 2011, as the predicting period. Each forecast uses only the information available up to the time the forecast is made. The raw data are transformed to log scale before prediction. For VIX and DJIA prediction, the lag is chosen to be 3 weeks. However, according to the Granger test analysis shown in Table IV, the is not significant for lags weeks in the case of GIS vs. trading volume. We therefore chose in Eq. 4 and Eq. 5 for trading volume prediction. Fig. 5 shows the prediction errors for these 20 forecasting weeks. Table V shows the forecasting errors expressed as MAPE and direction accuracy.

Model MAPE Direction
DJIA Model 0 0.253 0.55
Model 1 0.244 0.70
Volume Model 0 0.386 0.55
Model 1 0.366 0.55
VIX Model 0 4.560 0.55
Model 1 4.148 0.65
TABLE V: Forecasting accuracy of using weekly search volumes to predict financial indicators (DJIA, Volume and VIX).

From these results it appears that adding search volumes (1) reduces the MAPE prediction error for VIX, DJIA and trading volumes predictions, and (2) improves the direction accuracy for DJIA and VIX forecasting, but not for trading volumes.

Fig. 5: Prediction Error Plot.

Fig. 5 furthermore shows that during several weeks the baseline model output outperformed the advanced model. This again highlights the difficulty of financial market prediction, even using data that has been shown to have statistically significant Granger causality with the particular financial indicators. We offer the observation that on August 15th 2011 (highlighted with a yellow bar), the prediction error of the advanced model (red) dropped well below that of the baseline model (blue). In that period (August 15th -19th) the weekly VIX reached a high value of 43.05, the DJIA decreased over 450 points, and trading volumes increased significantly compared to the previous week. This is suggestive that search volumes of financial terms may be particularly useful for prediction when the market experiences high degrees of volatility, significant changes in values and high trading volumes.

Iv Twitter, Search Engine, News Media and Survey-based prediction of financial indicators

Iv-a Correlation Analysis

In previous sections we focused on weekly analysis due to data availability. However, our Twitter data and the Daily Sentiment Index (DSI) were recorded daily from July 1st 2010 to September 29th 2011, for a total of 456 days. Given the availability of daily data, in this section our analysis will focus on daily time series, rather than weekly.

Again, Google Insight Search (GIS) does not provide daily volume search data. We therefore do not use GIS search volumes in our daily analyses, and instead use the Tweet volumes of financial search terms (TV-FST), as introduced in Section II-D2.

In total, we examine four daily sentiment indicators, i.e. Twitter Investor Sentiment (TIS), Tweet Volume of Financial Search Terms (TV-FST), Negative News Sentiment (NNS) and Daily Sentiment Index (DSI). Using the same definition as shown in Section II-D2, the TV-FST is calculated as the average of Tweet volumes of all these financial search terms. Table VI displays the Pearson correlation values observed between these sentiment indicators.

TIS NNS TV-FST DSI
TIS 1
NNS -0.237 1
TV-FST -0.304 0.225 1
DSI 0.431 -0.322 -0.202 1

TABLE VI: TIS, NNS, TV-FST, and DSI correlations.

Survey data, DSI (percentage of bullish readings), has a positive correlation with TIS, but negative correlations with the other two sentiment indicators: TV-FST and NNS. TV-FST exhibits a negative correlation with DSI and TIS, but a positive correlation with NNS, which suggests that TV-FST may be a bearish/negative sentiment indicator. All listed correlations are statistically significant with .

After linearly extrapolating financial indicators values missing on weekends (because of markets closing), we analyze the correlation between these sentiment indicators and financial market indexes. The results are shown in Table VII.

DJIA Log return Volume VIX
TIS -0.071 0.267 -0.127 -0.314
NNS 0.147 -0.147 0.039 0.237
TV-FST 0.449 -0.091 0.096 0.183
DSI 0.277 0.181 -0.341 -0.832
( indicates )
TABLE VII: Correlations between sentiment and financial indicators.

We observe that TIS is positively correlated with market log returns (cf. Eq. 2) and negatively correlated with VIX. DSI is positively correlated with DJIA closing values, as well as log return, but negatively correlated with trading volume and VIX. VIX reflects perceived market risk, with higher VIX values potentially indicating greater levels of investor fear. Its negative correlation with DSI and TIS may therefore indicate that the latter correspond to positive sentiment, or a lower perception of risk or fear among investors. Conversely, the positive correlation between VIX vs. NNS and TV-FST may indicate that these are indeed indicators of fear or negative sentiment.

To better view the correlation between the sentiment indicators and financial market, we plot the time series of DJIA and four sentiment indicators, in Fig. 6.

Fig. 6: Time series of DJIA and TIS, DSI, NNS as well as TV-FST.

In Fig. 6, the time series in the top panel shows the daily DJIA closing value from July 1st 2010 to September 29th 2011. The four time series in the lower panel represent TIS, DSI, NNS and TV-FST during the same time period and they are smoothed over the past 30 days. We invert the TIS and DSI to make them consistent with the directionality of the other two negative market indicators (i.e. NNS and TV-FST). As such, “up” means negative sentiment, while “down” indicates positive sentiment.

We marked five time periods in the lower panel of Fig. 6 with rectangle bars to indicate when DJIA prices fell in August and November 2010, and March, June and August 2011. Before DJIA prices fell in August 2010 (indicated by the first rectangle bar), it can be seen that the TIS and NNS graphs moved upwards (i.e. a rise in negative sentiment), while DSI dropped (i.e. positive). Before the second bar (November 2010), we see TIS and TV-FST trending upward. Before the fall in DJIA prices in March 2011 (third bar) we observe a clear and long-term increase of TV-FST, NNS and TIS values. TIS and TV-FST values are trending upwards before the fourth bar that marks June 2011. All four sentiment indicators trend upwards before the last bar that makes August 2011, but the “up” trend of DSI seem to lag the “up” trend of NNS and TV-FST. In conclusion, though there is considerable noise in the daily data, the non-survey sentiment indicators, especially TIS and TV-FST, do show significant increases in negative sentiment that clearly precede periods of falling DJIA prices.

Iv-B Granger Causality Analysis

The prediction of stock market returns is a matter of considerable interest. To determine whether any of our sentiment indicators are useful to predict daily DJIA log returns, we conduct a Granger causality analysis similar to Section III. According to Table VII, the correlation coefficient between TV-FST and log returns is statistically insignificant (). To determine which of our set of search terms are most effective to predict log returns, we conduct a correlation analysis between the search volumes of each financial term individually and log returns. Then, we select the search terms 777DJIA, dow, Dow Jones, Dow Jones Industrial Average, SP500, stock(s) fall(s), stocks decline, financial market. whose search volumes exhibit the most significant correlations with log returns, and take the average of their time series to be the TV-FST. The correlation coefficient between the resulting TV-FST and daily log returns is -0.30 with a -value . Table VIII lists the -values for a number of bi-directional Granger causality tests of log returns vs. our sentiment indicators. We find statistically significant Granger causation in both directions between log returns and TIS, NNS, and TV-FST, with the exceptions of lag = 1, TV-FSTReturn, and lag = 3, 5, ReturnTV-FST. No statistically significant Granger causation was observed between DSI and log returns. These results indicate that sentiment indicators extracted from Twitter (TIS and TV-FST) and News headlines (NNS) are predictive to the DJIA log return, but DSI is not predictive.

1 2 3 4 5
TISReturn <0.001 0.0086 0.035 0.028 0.021
ReturnGIS <0.001 <0.001 <0.001 <0.001 <0.001
NNSReturn 0.017 0.030 0.011 0.005 0.004
ReturnNNS 0.014 0.013 0.031 0.030 0.055
DSIReturn 0.523 0.138 0.203 0.308 0.377
ReturnDSI 0.267 0.174 0.647 0.377 0.059
TV-FSTReturn 0.413 <0.001 <0.001 <0.001 <0.001
ReturnTV-FST 0.0025 0.019 0.151 0.071 0.140
(: , : , : )
TABLE VIII: Statistical significance (p-values) of Granger causality analysis between daily log return and TIS, NNS, DSI and TV-FST.

Iv-C Multiple Regression Analysis

In this section, we conduct a multiple regression for daily log returns obtained according to Eq. 2. The regression inputs are our four sentiment indicators and the past financial values of log return. As an additional control, we include VIX, since it is a well-accepted predictor for market return. The multiple regression model is shown in Eq. 7, where days, and represents the daily log return. In order to maintain a common scale, we normalized all data to standard scores.

(7)

Table IX provides the summary statistics of the multiple regression. Compared with the baseline model, the adjusted improves from 0.092 to 0.200. This means that an additional 11% of the variation in log returns is accounted for by adding these sentiment indicators. After controlling for all other variables, we find that DSI is not a statistically significant predictor. The two sentiment indicators extracted from Twitter, i.e. TIS and TV-FST, are however very significant predictors at a lag of 1 to 2 days. Here, we observe a reversal effect, namely that daily log returns are positively associated with TIS and TV-FST on the previous day, but negatively correlated with those on the lag of 2 days. VIX values at lags of 2 days are highly statistically significant predictors of log return. NNS is also a statistically significant predictor at lags ranging from 1 or 4 days, but with much less lower coefficients, e.g. at a lag we find that the , and the coefficient is -0.087, which means we expect to see a log return decrease of only 0.087 standard deviations for each one standard deviation increase of NNS.

Lag Return TIS NNS DSI VIX TV-FST
Coeff. -value Coeff. -value Coeff. -value Coeff. -value Coeff. -value Coeff. -value
1 0.282 0.004 0.170 0.008 -0.087 0.080 0.389 0.385 0.494 0.182 0.235 0.0007
2 -0.139 0.175 -0.164 0.018 -0.066 0.191 -0.239 0.709 -1.161 0.029 -0.324 3.73e-05
3 0.0006 0.995 0.069 0.316 0.048 0.349 -0.265 0.678 0.641 0.235 0.059 0.486
4 -0.115 0.275 -0.088 0.208 -0.097 0.058 0.730 0.251 0.384 0.479 -0.059 0.490
5 -0.0212 0.837 0.152 0.031 -0.017 0.740 0.177 0.780 0.502 0.351 0.171 0.045
6 0.071 0.472 -0.132 0.062 0.057 0.257 -0.514 0.405 -0.490 0.341 -0.096 0.261
7 -0.117 0.040 0.005 0.935 0.008 0.874 -0.115 0.789 -0.204 0.559 -0.024 0.743
(: , : , : )
Residual standard error: 0.893 on 406 degrees of freedom.
Multiple -squared: 0.2841, Adjusted -squared: 0.200 (baseline model (cf. Equation 4, is daily log return here): Adjusted -squared 0.092).
-statistic: 3.67 on 42 and 406 DF, -value: 5.523e-12
TABLE IX: Summary statistics of multiple regression.

Iv-D Forecasting analysis

To further test the hypothesis that adding sentiment indicators can help predict financial indicators such as the DJIA, trading volumes, and VIX, we conduct a 1-step forecasting test over 30 days, i.e. from August 31st 2011 to September 29th 2011. As with the weekly prediction in Section III-C, the baseline model is based on its own historical financial values (cf. Model 0 in Eq. 4) whereas the advanced model (cf. Model 1 in Eq.5) adds the historical values of the sentiment indicators TIS, NNS, TV-FST and DSI. Here we assume . The forecasting accuracy is measured in terms of the Mean Absolute Percentage Error (MAPE) and the direction accuracy. Results are shown in Table X.

Model MAPE Direction
DJIA Model 0 1.00 0.5
Model 1 0.97 0.63
Volume Model 0 7.24 0.47
Model 1 7.56 0.60
VIX Model 0 4.00 0.6
Model 1 3.88 0.67
TABLE X: Forecasting accuracy of using TIS, NNS, TV-FST and DSI to predict financial indicators (DJIA, trading volume and VIX).

We find improvements in the direction accuracy and MAPE of the forecasting accuracy for DJIA, VIX and volume prediction, with the exception of the MAPE for volume prediction. However, the improvement is not highly significant. The extremely high volatility in the financial markets during our training and testing periods, especially in August and September 2011, may account for this. In addition, we used relatively simple linear models in this paper that may not be suited to model the complex interactions of factors involved in shaping financial market values. Further research will need to focus on the development of more accurate and more advanced linear/non-linear prediction models.

V Conclusion

Behavioral finance challenges the Efficient Market Hypothesis by emphasizing the important role that human emotion, sentiment and mood play in financial decision-making. Thus places the accurate measurement of sentiment and mood at the heart of a discussion over how to best model and predict the behavior of the financial markets. Previous research in this domain has relied mainly on surveys or news analysis to obtain investor sentiment. Research has recently started to leverage very large-scale web data, including search engine and social media data, to assess public as well as investor sentiment. However, most existing work adopts only a single data source (survey, social media or search engine data) as a proxy to public and investor sentiment, and then uses it to computer a particular financial index. To the best of our knowledge, no work has been done to perform a detailed survey of a variety of different classes of mood indicators extracted from a variety of classes of data sources. Studying the relations between different mood indicators and their predictive relationships to different financial indexes is necessary to unravel the causal relations sentiment and mood relate to the financial markets, and thus crucial in improve financial forecasting models. Our paper is a first, preliminary contribution of such a comparison to the rapidly emerging domain of computational behavioral finance.

In this paper, we collect six sentiment indicators from investor sentiment surveys (II and DSI), social media (Twitter), news media services, as well as search engine (Google). Those include DSI bullish percentage, Investor Intelligence (II), Twitter Investor Sentiment (TIS), Tweet volumes of financial search terms (TV-FST), Negative News Sentiment (NNS) and Google search volumes of financial search terms (GIS).

First, in a weekly analysis, we find a significant correlation between weekly GIS of financial terms with DJIA closing values, trading volume, and VIX values. Granger causality tests confirm that GIS is indeed predictive of financial indicators, but surveys of investor sentiment (i.e. II) are not. Weekly forecasting accuracy was improved by adding GIS search volumes of financial terms, notably so when the DJIA was trending downward and the VIX was indicating high volatility, such as in August 2011.

Second, in our daily analysis, all mood indicators exhibited a significant correlation with log returns and VIX values. Controlling for other mood indicators including the VIX, we find that TIS and the TV-FST values of the previous 1-2 days are very statistically significant predictors of daily market returns, while DSI is not. NNS is also found to be a statistically significant predictor, however, compared to the TIS and TV-FST, we find less significant predictability of log return. This finding indicates that the predictive power of Twitter’s two sentiment indicators outperformed survey sentiment as well as news media analysis. Moreover, we found that before the highly downward movement of DJIA in the end of July and August 2011, Tweet volumes of financial terms started to increase several weeks earlier than Google volumes did. This indicates a potential efficiency gain of Twitter over GIS.

Studying the predictive power of the online web data is still in its infancy. The various correlations and limitations of these different data sources, different sentiment measures, and its general prediction applicability to different domains remain unclear. Our work is the first attempt to extract a range of sentiment indicators from several popular data sources (Twitter, search engine and news) and use various sentiment indicators to predict different financial indexes (DJIA, trading volumes, VIX and gold) in both daily and weekly scale. Continued research is needed to deepen our understanding of why and how these different mood indicators relate to and predict socio-economic phenomena such as the financial markets.

Acknowledgements

We would like to express our gratitude to trade-futures and Investor Intelligence for making their sentiment data available to our research program. We also thank the Socionomics Institute for their kind advice on a range of research questions, and their assistance in obtaining research data that was crucial in conducting the research presented in this paper.

References

  • [1] W. Antweiler and M. Frank. Is all that talk just noise ? the information content of internet stock message boards. The Journal of Finance, LIX(3):1259–1294, June 2004.
  • [2] S. Asur and B. Huberman. Predicting the future with social media. http://arxiv.org/abs/1003.5699/, 2010.
  • [3] J. Bollen, H. Mao, and A. Pete. Modeling public mood and emotion: Twitter sentiment and socio-economic phenomena. In ICWSM, 2011.
  • [4] J. Bollen, H. Mao, and X.-J. Zeng. Twitter mood predict the stock market. Journal of Computational Science, 2011.
  • [5] I. Bordino, S. Battiston, G. Caldarelli, M. Cristelli, A. Ukkonen, and I. Weber. Web search queries can predict stock market volumes. http://arxiv.org/abs/1110.4784.
  • [6] W. Chan. Stock price reaction to news and to no-news- drift and reversal after headlines. Journal of Financial Economics, 70:223–236, 2003.
  • [7] H. Choi and H. Varian. Predicting the present with google trends. Technical report, Google Inc, 2009.
  • [8] A. Culotta. Towards detecting influenza epidemics by analyzing twitter messages. In 1st Workshop on Social Media Analytics (SOMA ’10), Washington, DC, USA, 2010.
  • [9] Z. Da, J. Engelberand, and P. Gao. The sum of all fears: investor sentiment and asset prices. http://ssrn.com/abstract=1509162, 2010.
  • [10] Z. Da, J. Engelberand, and P. Gao. In search of attention. Journal of Finance, 66:1461–1499, 2011.
  • [11] E. Gilbert and K. Karahalios. Widespread worry and the stock market. In International AAAI Conference on Weblogs and Social Media, 2010.
  • [12] S. Golder and M. Macy. Diurnal and seasonal mood vary with work, sleep, and daylight across diverse cultures. Science, pages 1814–1815, 2011.
  • [13] D. Kahneman and A. Tversky. Prospect Theory: An Analysis of Decision under Risk. Econometrica, 47(2):263–291, 1979.
  • [14] Y. Karabulut. Can facebook predict stock market activity ? Standford, CA., 2011. National Bureau of Economic Research, Behavioral Finance Meeting.
  • [15] T. Loughran and B. McDonald. When is a liability not a liability? textual analysis, dictionaries, and 10-ks. Journal of Finance, 66(1):67–97, 2011.
  • [16] G. Mishne and N. Glance. Predicting movie sales from blogger sentiment. In AAAI 2006 Spring Symposium on Computational Approaches to Analysing Weblogs, 2006.
  • [17] R. Prechter. The Wave Principle of Human Social Behavior and the New Science of Socionomics,. New Classics Library, Gainesville,GA., 1999.
  • [18] T. Preis, D. Reith, and H. E. Stanley. Complex dynamics of our economic life on different scales: insights from search engine query data. Philos Transact A Math Phys Eng Sci, 368(1933):5707–19, 2010.
  • [19] R. Schumaker and H. Chen. Textual analysis of stock market prediction using breaking financial news: Theazfintext system. ACM Transactions on Information Systems, 27(2), 2009.
  • [20] E. Singer. The use of incentives to reduce nonresponse in household surveys. Survey Nonresponse edited by Robert M. Groves, et al., 2002.
  • [21] P. C. Tetlock. Giving content to investor sentiment: The role of media in the stock market. Journal of Finance, 62(3):1139–1168, 2007.
  • [22] P. C. Tetlock, M. Saar-Tsechansky, and S. Macskassy. More than words: Quantifying language to measure firms’ fundamentals. Journal of Finance, 63:1437–1467, 2008.
  • [23] A. Tumasjan, T. Sprenger, P. Sandner, and I. Welpe. Predicting elections with twitter: What 140 characters reveal about political sentiment. In International AAAI Conference on Weblogs and Social Media, Washington, D.C., 2010.
  • [24] H. F. Xue Zhang and P. Gloor. Predicting stock market indicators through twitter “i hope it is not as bad as i fear”. Social and behavioral sciences, 2010.
{biography}

[]Huina Mao School of Informatics and Computing, Indiana Unviersity-Bloomington, United States. Email: huinmao@indiana.edu

{biography}

[]Scott Counts Microsoft Research, Redmond, Washington, United States.
Email: counts@microsoft.com

{biography}

[]Johan Bollen School of Informatics and Computing, Indiana Unviersity-Bloomington, United States. Email:jbollen@indiana.edu

Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
""
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
   
Add comment
Cancel
Loading ...
77164
This is a comment super asjknd jkasnjk adsnkj
Upvote
Downvote
""
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters
Submit
Cancel

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test
Test description