A First Look at Emoji Usage on GitHub: An Empirical Study

A First Look at Emoji Usage on GitHub: An Empirical Study

Xuan Lu   Yanbin Cao   Zhenpeng Chen   Xuanzhe Liu  Key Laboratory of High Confidence Software Technologies (Peking University), Ministry of Education, PRC
{luxuan, caoyanbin, czp, xzl}@pku.edu.cn
Abstract

Emoji is becoming a ubiquitous language and gaining worldwide popularity in recent years including the field of software engineering (SE). As nonverbal cues, emojis are widely used in user understanding tasks such as sentiment analysis, but few work has been done to study emojis in SE scenarios. This paper presents a large scale empirical study on how GitHub users use emojis in development-related communications. We find that emojis are used by a considerable proportion of GitHub users. In comparison to Internet users, developers show interesting usage characteristics and have their own interpretation of the meanings of emojis. In addition, the usage of emojis reflects a positive and supportive culture of this community. Through a manual annotation task, we find that sentimental usage is a main intention of using emojis in issues, pull requests, and comments, while emojis are mainly used to emphasize important contents in README. These findings not only deepen our understanding about the culture of SE communities, but also provide implications on how to facilitate SE tasks with emojis such as sentiment analysis.

emoji, GitHub, developer, sentiment

I Introduction

Emoji, defined as “a digital image that is added to a message in electronic communication in order to express a particular idea or feeling”,111https://dictionary.cambridge.org/us/dictionary/english/emoji is emerging as a ubiquitous language with its compact visual and live presentation, rich semantics, and understandability, and gaining worldwide popularity in recent years. As nonverbal cues, emojis have been widely adopted on online communication services such as Twitter and Facebook, and various efforts have been made over emojis, in terms of sentiment analysis [1, 2], user profiling [3], personality assessment [4], and even cluture difference analysis [5].

A noticeable trend is that, emojis are not only widely adopted in regular online communication, but also increasingly used in software development practice, ranging from code in programs,222https://www.tjvantoll.com/2016/06/10/emoji-and-coding/333http://www.emojicode.org/ to online forums such as StackOverflow and GitHub, and even to the requirements engineering [6]. Given that emojis are expected to provide a new way to enrich the expression, increase the interaction vitality, and improve the communication efficiency, we are motivated to understand how and why emojis are used in software engineering practice.

In this paper, we conduct an empirical study of emoji usage in the software engineering community by exploring GitHub,444https://github.com the most popular online community where millions of programmers host and manage projects and build software collaboratively, to study the way that they communicate with one another in development activities. More specific, we aim to answer the following research questions.

RQ1: How are emojis used by developers on GitHub? We begin with investigating the characteristics of emoji usage on GitHub. We establish a large-scale data set of communicational posts (i.e., issue, issue comments, pull requests, pull request comments, and README) collected from GitHub, spanning 66 months since January 2012. We make the descriptive analysis of the favored emojis, the density and position of emojis in GitHub posts, and derive some typical patterns. Domain specific usage can be observed, e.g., emojis can be assembled to represent slangs such as “” (i.e., dogfood).

RQ2: Do emojis have domain-specific meanings by developers on GitHub? Suppose that the topics on GitHub can be quite different from daily communication due to the technical nature of the platform, we are interested in there are unique usage specific to the software development activities. We explore the meaning and sentiment of words that can be domain specific [7]. Regarding that emojis are widely used as complements or surrogates of plain texts, we further leverage the state-of-the-art word embedding method for emoji interpretation in the technical context. Interestingly, the semantics of emojis on GitHub can be quite different from that on Twitter. For example, emojis such as and can present domain specific meanings. Results also indicate the tendency to express positive sentiments through emojis on GitHub.

RQ3: What are the intentions of using emojis by developers on GitHub? Given the increasing importance of sentiment analysis in software development and the lack of efficient tools in this field [8, 9], we expect that emojis can provide a new complementary signal to understand developers’ sentiment in practice. We conduct an annotation task and build an eight dimension taxonomy of emoji usage intentions. We find that sentimental usage is a main intention in issues, pull requests, and comments, while emojis are mainly used to emphasize important content in README. In addition, sentimental emojis can also be used for non-sentimental intentions, indicating the significance of intention recognition before further application of emojis in analysis tasks in the SE field.

Findings. By answering the preceding questions, we find that developers use a diversity of emojis in their communications with domain-specific meanings and usage patterns. The use of emojis on Github reflects a positive and supportive culture of this community. Sentimental usage is still a main intention of using emojis on GitHub.

Contributions. To the best of our knowledge, this paper makes the first step of exploring emoji usage on software engineering activities and further understanding the behavior of the software developers with a new signal. Our derived findings and implications can probably help various stakeholders on how to better make use of emojis on GitHub, so as to improve the representation vitality, the communication efficiency, the possible sentiment inference, and to promote better project collaboration, problem solving, and productivity.

The rest of the paper is organized as follows. Section II introduces related work. Section III describes the data used in this study and demonstrates the popularity and distribution of emojis in our data set. Section IV characterizes emoji usage patterns on GitHub. Section V develops an emoji interpretation method based on state-of-the-art embedding technique. Section VI proposes a taxonomy of intentions of using emojis with a manual annotation task. Section VII provides implications. Section VIII discusses threats to validity and future direction. Section IX concludes the paper.

Ii Related Work

Ii-a Emoji Usage Analysis

Increasingly popular, emojis are becoming a ubiquitous language and widely adopted by Internet users in recent years. Various studies have been done to analyze emoji usage across countries [10], across cultures [5], and across demographic groups [3]. These research on emoji usage are mainly conducted in input methods [5, 11, 3], instant messaging apps such as WhatsApp [12] and WeChat [13], and social networks [14, 15]. So far, there is no study to analyze emoji usage and effects in a tech community. In addition, the prevalence of emojis also attracted many researchers to study the intentions and effects of using them. These research demonstrated that besides replacing the content words in text, emojis can also be used to provide emotional or situational information, adjust tones, express irony, engage the audience, decorate texts, etc [16, 17, 18]. However, due to the lack of emoji studies in tech community, we don’t know which role emojis play and what functions emojis have in it. To bridge the knowledge gap, we make the first effort to measure the usage of emojis on Github.

Ii-B Sentiment Analysis in Software Engineering

Understanding sentiments of developers is a research focus in software communities [19, 20, 21, 9, 22, 7, 23, 24, 25]. The sentiments of developers can be a sensor of the contributors’ status and activity, as well as the quality of projects. For example, sentiment analysis can be used to detect the psychological state and job satisfaction of developers [26], which are strongly associated with their productivity and task completion quality. However, current sentiment analysis tools have been demonstrated to have strong limitations on sentiment analysis in the SE field [9]. In fact, to improve the sentiment analysis technique, many researchers in natural language processing community have started to use emojis as weak sentiment labels [27, 28], which is called distant-supervised learning. Such practice with emojis sheds lights on the improvement of SE sentiment analysis tools. To discover the possibility of leveraging emojis in sentiment analysis in SE, we first describe how emojis are used with the data set collected from GitHub in this work.

Iii Data Set

To investigate emoji usage in communications on GitHub, we select five typical types of posts, i.e., issues, issue comments, pull requests, pull request comments,555In general, an issue opens up a discussion thread for bugs, enhancement, questions, etc., while a pull request is a request for the project owner to “pull” the source code change from contributors and merge to the code repository. See https://help.github.com/categories/collaborating-with-issues-and-pull-requests/, retrieved in August 2018. and README.666README helps developers to communicate expectations for their projects, see https://help.github.com/articles/about-readmes/, retrieved in August 2018. To simplify, we refer to the issues, pull requests and their comments as conversational posts for they can be replied by users.

Iii-a Data Collection

Through the GHTorrent project [29], we collect the conversational posts spanning 66 months from January, 2012 to June, 2017. After a data cleaning process to filter duplicates, spams, and expired URLs, the data set covers 3,088,360 projects and 3,952,924 users. We crawled the README files using the official API of GitHub [30]. Due to the rate limits for API requests, we choose only README files in projects with no less than 10 stars.777https://help.github.com/articles/about-stars/, retrieved in August 2018 Table I summaries the data set.

Note that the data of emoji reaction,888https://github.com/blog/2119-add-reactions-to-pull-requests-issues-and-comments, retrieved in August 2018. which is provided by GitHub since March 2016 to respond to conversational posts, are not used in this work. Instead of the six emojis (i.e., , , , , , and ) provided by the reaction function, we collect the emojis that the users spontaneously typed into the free text for they have a greater variety in both the adopted type and usage pattern.

#issue #issue comment #pull request #pull request comment #README
Non-emoji 30,805,343 49,173,170 12,696,474 23,950,062 744,660
Emoji 60,742 307,298 34,576 407,302 21,444
Total 30,866,085 49,480,468 12,731,050 24,357,364 766,104
TABLE I: Summary of Data Set.

Iii-B Emoji Popularity

Before looking at the emojis, we measure their popularity by demonstrating the proportions of posts and users that adopted emojis in the studied period. We include only the conversational posts here because the README posts are not with timestamps.

The fraction of emoji posts remains at nearly zero in the first few years in our data set, increasing slowly, and shows a sharp increase since March, 2016. Statistics show that in June, 2017, 0.58% of issues, 0.75% of pull requests, 1.32% of comments for issues, and 3.23% of comments for pull requests contain emojis. Although the proportion is much lower than the coverage of emojis on nontechnical communities such as Twitter (13.69% as reported in [31]) possibly due to the nature of technical discussions, the proportion keeps increasing over time in the studied period.

Fig. 1: Proportion of emoji users in GitHub conversations grows over the year 2016, with a sharply increase after the release of the emoji reaction feature.

We can find a quite high proportion of users who actively use emojis in their conversations. Figure 1 illustrates the proportion of users who have used at least one emoji in each month, over all users involved in different types of posts in the same month. The proportion of emoji users started from below 1% in the beginning of the year 2016 and increased over time. Users are more likely to use emojis in comments than in the original posts of issues and pull requests. In all four categories of the conversational posts, there was a significant increase of emoji users from March to April of 2016, which coincides with the release of the emoji reaction feature. Among users who commented on pull request, this proportion increased sharply for nearly seven times in April. 10.29% of users who commented on pull requests in June of 2017 used emojis in their comments, a surprisingly larger proportion compared to the proportion of emoji posts. As the emoji reactions are not included, the proportion of emojis users may be underestimated.

Iii-C Emoji Distribution

Emojis are not evenly distributed in posts. For example, 0.20% of the issues in our data set contain at least one emoji. After grouping the issues by the number of comments, we find the proportion of emoji issues is 0.40% for the top 10% issues, significantly higher than 0.20% (). Interestingly, such a doubled proportion can be observed in most months in our data set. This finding implies a positive relation between emoji usage and responses to issues.

Similar findings can be made for the users. In general, 3.66% of users have used at least one emoji in conversational posts. When restricting the scope of users to those who have posted issues, this proportion is significantly higher to be 4.71% (). Further, we rank the users in descending order of number of issues they posted, and find that this proportion is 23.39% for the top 10% of the users, significantly higher than 4.71% (). Such findings indicate the potential correlation between emoji usage and user activeness.

Considering that the use of emojis can be influenced by multiple factors, we leave further research of the possible effect of using emojis in the participation and activeness of users for future work. In this paper we focus only on the posts with emojis.

Iv Usage Characteristics

In this section we address RQ1: How are emojis used by developers on GitHub? by looking at the top emojis and typical usage patterns on GitHub.

Iv-a Top Emojis

In total, there are 1,271 emojis used on GitHub. Ranked by occurrence, the 10 most used emojis are , , , , , , , , , and . To compare with, the 10 most used emojis on Twitter tracked by EmojiTracker999http://emojitracker.com, retrieved on September 3, 2017. are (21), (9), (57), (41), (47), (11), (222), (123), (375), and (152). The number in parentheses shows the ranking of the corresponding emoji on GitHub. Interestingly, only one emoji () is overlapped by the two sets of top 10 emojis. Most of the left 9 popular emojis on Twitter rank pretty low on GitHub. The most popular emoji (face with tears of joy) on Twitter, which was even elected as “Oxford Dictionaries word of 2015”,101010http://blog.oxforddictionaries.com/2015/11/word-of-the-year-2015-emoji is the 21st most used emoji on GitHub.

We conduct a Wilcoxon signed-rank test [32] on rankings of emojis on the two platforms and find significant difference ( for top 10 emojis, for top 50 emojis), indicating domain specific preferences for emojis on this technical platform.

Post Type Top 10 emojis
All
issue
issue comment
pull request
pull request comment
README
TABLE II: The Most Used Emojis.

In the top 10 emojis on GitHub, most emojis can be used to show positivity such as happiness, congratulations, praises, and appreciation, implying a positive atmosphere which can be of significance to communication and project collaboration. Another interesting observation is the popularity of , , and , which can be used as “bullets” for a list of items.

When decomposed to different types of post (see Table II), greater variety and more interesting usage can be found in the top emojis. For example, the ship (), whose popularity ranks no.609 on Twitter, is the ninth popular emoji in pull request comments. To understand the popularity of such emojis, we will interpret the meanings and intentions in the following sections.

Iv-B Emojis in Text

Used as complements or surrogates of plain text, emojis can generally make texts more vivid, expressive, and easy to read. How frequently are emojis used and where are they put in a post? We investigate the density and position of emojis in English texts.

Fig. 2: Emoji density in different posts.

Iv-B1 Density

We define the density of emojis as the number of emojis normalized to the length of post, and present the density distribution with frequency histograms in Fig. 2. Comparing the five distributions, we can find that in README the emoji density is particularly close to zero, possibly due to the relatively long texts. For the four conversational posts, the densities of emojis are mainly distributed between 0.0 and 0.2 with the largest peak near 0.0, indicating that emojis are mostly used with textual words, accounting for a small fraction of length in the post. Interestingly, there are discrete peaks with density of 0.25, 0.33, 0.5, and 1, especially in comments for issue and comments for pull request. We trace back and find massive emoji posts with 3 words (e.g., “hooray for tests ”), 2 words (e.g., “Whoops . Merging!”), 1 word (e.g., “Thanks! ”), and even no word (e.g., “”). Such use of emojis help explain the discrete peaks.

Iv-B2 Position

Emojis tend to come at the end of messages, providing cues about how to understand the words that came before them [33]. To verify this rule for emojis on GitHub, we extract emoji sentences from conversational posts and find most of them end with emojis. In particular, emojis are at the end of 66.77% emoji sentences in issue comments.

Fig. 3: Emoji position in a post.

At the post level, we segment sentences in a post to three position classes [34], i.e., the first sentence, the last sentence, and the left in middle. The distribution of emojis in posts containing no less than three sentences is shown in Fig. 3. It can be observed that emojis are mostly used in the middle of issues (75.34%) and pull requests (53.83%). However, in the two types of comments, emojis are more likely to be used at the end. Considering that users often express attitudes or emotions in comments, they can use emojis at the end of comments to enrich such expression.

Iv-C Appearance Patterns

When emojis appear in posts on GitHub, three typical patterns including using an emoji as a post, repeating the same emoji, and assembling different emojis, can be observed. We study such patterns and illustrate them with examples.

A single emoji as a post. We are interested in the posts with only one emoji but no plain texts, because the understanding of such a post relies on the emoji. We suppose that emojis in such cases can convey relatively clear meanings, otherwise they will be resistance to the communication. Ranking emojis by their possibilities to independently constitute a post, we have (26.33%), (23.76%), (16.67%), (16.13%), (15.23%), (13.24%), (12.50%), and (10.82%) as the top ones. With our interpretation approach in Section V, the and can indicate the “launch” of something such as a new project or a new feature, and the rest ones can show attitudes or emotions. In contrast, emojis such as and are seldom used independently for they need supplementary words to express an complete idea.

Emoji repetition. Emojis are often repetitively used to communicate a particular type of effect such as emphasis [35]. On GitHub, , , , , and are with the highest possibilities to be used repetitively. Interestingly, these emojis tend to convey positive attitudes or sentiments in most cases. For example, the comment “” in an issue commentis to express confirmation and praise to the issue assignee, after which the corresponding commit is merged to master. In contrast, emojis that are the least likely to used in repetition include , , , , and , which often mean warning, failure, disappointment, disagreement, and sadness. Such finding implies a friendly atmosphere on GitHub, which confirms the observation in Section IV-A.

Emoji assembling. In some cases emojis are assembled to express a complete and even complicated meaning in a post. For example, often occurs together with , , , , or . One typical example of such co-occurrences comes from an issue comment saying “”. The writer of this comment is the owner of the project and he merged the corresponding commit after the comment. The two emojis are combined to make up a sentence, which can be inferred to mean “I have reviewed this commit, it’s perfect!” Another example is “Thanks for this! We’re checking it out. ” in an issue comment. The comment writer, a member of the project, added a label to the commented issue and reviewed the committed code after the comment. In this example, the three emojis are combined to retell the meaning of the plain texts, making the expression more vivid.

Understanding assembled emojis may be not as easy to understand single or repeated ones, yet such usage can spice up the communication as the assembling can be quite creative especially when combined with domain knowledge or slangs. For example, in the README of the project Releasor, i.e., “ Releasor is used in ProgressBar.js, git-hours, arr-mutations, and many others”, the dog food indicates that the Releasor tool has been used in the developers’ own projects and demonstrates confidence.111111Eating your own dog food, or dogfooding, is a slang term used to refer to a situation in which an organization uses its own product. This can be a way for an organization to test its products in real-world usage. Hence dogfooding can act as quality control, and eventually a kind of testimonial advertising. See https://en.wikipedia.org/wiki/Eating_your_own_dog_food, retrieved in August, 2018.

Summary. This section characterizes emoji usage on GitHub from the aspects of the top emojis, the density and position of emojis in posts, and typical appearance patterns. We find that the favored emojis on GitHub are quite different from Twitter, indicating domain-specific usage such as technical discussions. In comments for issues and pull requests, emojis are often co-used with few words, and often come at the end of long comments. Additionally, emojis show different potentials to independently constitute a post, to emphasize sentiments by repetition, and to illustrate complicated meanings by assembling. It should be noted that these findings are not rules but usage patterns learned from data.

V Emoji Interpretation

Although emojis are regarded as a ubiquitous language across different countries and user groups [5], we propose that the interpretation of emojis, similar to textual words [7], would have specific meaning in a technical field. In fact, we have obtained some clues for domain-specific usage of emojis. To address this research question (i.e., RQ2: Do emojis have domain-specific meanings by developers on GitHub?), we develop an embedding-based approach to interpret the meanings of emojis and study the sentiment distribution of emojis on GitHub.

V-a Semantic Understanding

Developers on GitHub have been spontaneously proposing standards for using emojis to fit the functions and scenarios in this community. For example, gitemoji121212https://github.com/carloscuesta/gitmoji provides an initiative to standardize and explain the use of emojis on GitHub commit messages. However, how it is defined does not necessarily determine how it is used. An example is that the defined as party popper in Unicode131313http://unicode.org/emoji/charts/full-emoji-list.html represents initial commit in gitemoji, yet this emoji is often used to express congratulations.

We decide to study the interpretations of the emojis by developers on GitHub with the state-of-the-art text representation learning methods. By projecting language tokens into a semantic space, we are able to directly assess the meanings of emojis. We extract all emojis and English words from the markdown141414https://guides.github.com/features/mastering-markdown/ texts, replace each code block with “[code]” and each URL with “[url]”, and filter out punctuation and special characters. We use the NLTK package151515http://www.nltk.org to tokenize and stem the words.

With the processed texts, we use word2vec [36] to train a 300-dimension embedding for each token, including words and emojis. Such a semantic space reflects the developers’ interpretation of the meanings of both words and emojis. Based on cosine similarities between the embedding vectors, we are able to find the closest neighbors of any emoji in the semantic space. These neighbors, either words or emojis, can help us infer the meaning of the target emoji. We rank the closest word tokens to the given emoji in the semantic space. To compare, we also report the closest neighbors of the emoji in a different semantic space,161616http://sempub.taln.upf.edu/tw/emojis/ in which the embeddings of emojis were trained with 10 millions Tweets posted by USA users [14]. This alternative semantic space represents the interpretations of emojis in common Internet communications.

Similar words on GitHub Similar words on Twitter
snuck, elus, insidi, uncov,
nonbreak, bug, untitl, skater,
crept, smell, worrisom, fix,
report, gonzal, buggi, glare,
floater, badminton, undetect,
spectacular, miscellan
luvbug, Modjeska, zoomars, natie,
getstheworm, buzzzzz, superfun,
pangderbear, grindcore, kcm,
localis, arcane, royalbluegroceryhp,
coulby, meow, hunnies, gyp,
kymimim, perdy, djcdannylewis
merg, lgtm, ship, readi,
nit, approv, land, leisur,
ahead, squash, alright, ahoy,
hooray, tum, thanksgiv,
punt, immin, thatch,
hurrah, appeas, sucker
cruise, carnival, indiaday,
disneycruiseline, Bahamas,
norweigian, ship, norweigan,
jaymullins, Norwegian,
jordanknight, sail, luby, maddecent,
ncl, satsko, oob, guana
TABLE III: Similar Words of Emojis Based on word2vec Embeddings.

Following to the word2vec results, in Table III we report similar words of two typical emojis, and , on GitHub. For the emoji , the most similar words on GitHub are obviously different with those on Twitter. For example, the similar words on GitHub includes bug, smell, worrisom, fix, and report. Such word neighbors reflect the domain specific meaning of on GitHub. That is, represents bugs in code on GitHub, but not on platforms such as Twitter.

Similar conclusions can be derived for the emoji . On Twitter, this emoji appears in similar contexts with words like cruise, ship, and sail. However, on GitHub, it has similar meanings with words like merge, lgtm (look good to me), ship, approv, and land. It indicates a status that the code is ready for use (to be “shipped”).

In addition to the closest words to an emoji, we can also discover its closest emojis in the semantic space. For example, given that is referred to as a code bug rather than an animal bug, one may expect that has different emoji neighbors on GitHub than on Twitter. The 10 emojis that have the most similar embeddings with on GitHub are , , , , , , , , , and . In comparison, the 10 closest emojis on Twitter are , , , , , , , , , and . The most interesting differences are the existence of , , and , which point to the meanings of debugging or fixing issues.

V-B Sentiment Distribution

The observation of domain-specific preference for emojis and the implicated positive atmosphere motivate us to look at the sentiments expressed by emojis on this platform. By comparing the sentimental emojis on GitHub and Twitter may help us understand the culture of the developer community.

Given that emojis can have domain-specific meanings rather than their original definitions or interpretations by the public, their sentiments may also change. Instead of directly using the reported sentiments of certain emojis, we carefully extract the sentiment score of each emoji based on the sentiments of its neighboring words.

Specifically, we calculate the sentiment score for each word with the SentiStrength-SE tool [7], which was designed for sentiment analysis in the software engineering domain. Each word has a score in {-100, 0, 100}, corresponding to {negative, neutral, positive}. The sentiment score of a given emoji is the weighted average of the sentiment scores of its 100 nearest neighboring words, weighted by the cosine similarity between the embeddings of the emoji and the word.

Fig. 4: Sentiment distribution of emojis.

After deriving the sentiment scores of emojis used on GitHub, we group the emojis into different score intervals and plot the total frequency of emojis used in GitHub posts against the sentiment scores in Fig. 4. The distribution has a clear tendency towards positive sentiments, which suggests that programmers on GitHub tend to express positive sentiments through emojis, as support or appreciation for each other.

To understand the difference between developers and common Internet users, we also measure the sentiment scores of emojis used on Twitter and plot the same distribution (the frequency of emojis used on Twitter are obtained through Emojitracker). Sentiment scores of neighboring words of emojis on Twitter are calculated with the LIWC package.171717http://liwc.wpengine.com Clearly, the entire distribution of Twitter emoji sentiment shifts to the left ( for the difference of mean), which suggests that emojis are more frequently used to express positive sentiments on GitHub compared to those on Twitter.

Summary. Analysis in this section evidences that emojis can have domain-specific interpretations by developers on GitHub in comparison with the common Internet users. Hence, the technical context should be considered for accurate understanding of emojis before leveraging emojis for further research. In addition, the developer community tend to use more positive emojis, which presents a positive and supportive culture compared to the public.

Vi Intention Understanding

Intentions of emoji usage in daily communication have been studied [16] while the understanding of intentions on a technical platform such as GitHub is lacked. In this section, we address the research question RQ3: What are the intentions of using emojis by developers on GitHub? by proposing a taxonomy of intentions with a manual annotation task.

Vi-a Taxonomy

Based on existing studies about emoji usage intentions, we develop an initial set of intention categories and adapt them to emoji usage on GitHub with a subset of posts. The final taxonomy of intentions is as follows.

  1. Sentimental usage. Using emojis can help express sentiments of users including their feelings, emotions, and attitudes. Considering the final effect of the expression (with the emoji) and the original sentiment of the plain text (without the emoji), the sentimental intention of using emojis can be de-composed to two sub-categories, i.e., sentiment expression and sentiment strengthening. Note that we also considered sentiment weakening and sentiment reversing at first, yet no such examples have been found. We claim this intention category can be extended if necessary.

    1. Emojis conveying sentiment in a non-sentimental textual context have the intention of sentiment expression. For example, in the issue “Even the Guardian has TLS now…. , the attitude of appreciation is expressed through the emoji.

    2. In the case that the plain text has already expressed some sentiment, the emoji often makes the post more expressive. We name this intention as sentiment strengthening. For example, in the issue “(that’s right, it’s blurring the actual content of the table cell! ”, the negative emotion is expressed by the text and strengthened by the emoji.

  2. Statement enriching. Emojis can make expressions, which are not limited to sentimental expressions, more vivid. With this intention, emojis are often used to replace or illustrate contents such as concepts and objects in text. For example, in the issue “This should not happen, I would even go as far as to call it a .”, the emoji enriches this statement by representing the word bug.

  3. Content organization. The pictographic nature of emojis makes them a good choice to assist the organization of contents in a post for readability improvement. When multiple items are listed, emojis such as and can be item bullets as alternatives of symbols like and . Additionally, some emojis such as and can be used in checklists with the semantics they conveyed.

  4. Content emphasis. To avoid being overwhelmed by massive content, the important points can be demonstrated with emojis to attract more eyeballs. Emojis with this intention are not necessarily semantically connected with the text but contribute with their pictographic characteristics. Typical emojis often used with such intention include , , , and .

  5. Atmosphere adjustment. Two main scenarios are included in this intention category. First, emojis can be used to adjust tone, making the messages less serious and more friendly. An example is “Please do not be terrified ” in a pull request.As nonverbal cues, emojis especially facial expressions can be used with this intention. Although facial emojis are often combined with emotion, they do not show obvious emotion but often politeness and kindness in its context when categorized to this intention. Second, when one intends to say something but has no idea of a specific topic, they can use emojis, which can even be not semantically related to the context. Emojis include hearts, gestures, animals, and objects can often be with this intention.

  6. Unintentional usage. Emojis can be unintentionally used in posts when they are not input by the writer. Emojis in pasted contents including codes and logs are categorized to this intention.

  7. Emoji. In some cases, emojis are just emojis, e.g., “ is for upgrading dependencies, from other libraries, where is for things like binaries, from the local project.

Vi-B Annotation

Based on the proposed taxonomy, we manually annotated the intention of using emojis in 2,000 emoji posts. To give a 95% confidence level and a 5% confidence interval, we randomly select 400 posts in English for each of the five post types for annotation. For multiple appearances of emojis in a post, only the first one is annotated except that it is used in combination with others (e.g., “ Work in Progress ”. We discuss the annotation process in detail before reporting the results.

Vi-B1 Discrepancies

Understanding the meaning beyond language is often subjective and has always been challenging, where discrepancies can occur when different individuals interpret the same expression. The two authors who annotate the intentions of using emojis complete their task independently and discuss about the discrepancies until they agree. For example, in the statement “I’ve tried to reinstall the game, but the error keeps happening ” in an issue, one of the two authors classified the intention of to sentiment strengthening while the other believe it is to express an sentiment. After discussion, they agreed that the text “I’ve tried to reinstall the game, but the error keeps happening” is stating a fact without obvious sentiments, but the emoji brings emotion of sadness and disappointment to this expression; hence it is finally categorized to sentiment expression.

Vi-B2 Multiple intentions

In some cases an emoji can not be categorized to only one intention. For emojis showing complicated intentions, we allocate multiple labels to them and select a primary intention for further study. For example, in the sentence “Please review the guidelines for contributing to this repository.” of a pull request, the is effective in drawing attention (i.e., the content emphasis intention) to this warning. Meanwhile, this police car light emoji contains the semantic of emergency that enriches the context of warning, indicating an effect in making the statement more expressive. We label this emoji in this post with a content emphasis intention for its main effect. Another example is “Also,  for data-driven styles!” in an issue where the clapping hands represent a round of applause (i.e., the sentiment expression intention) meanwhile improve the expressiveness of the statement by replacing the corresponding textual words. With a comparison, we adopt sentiment expression as the main intention of in this post.

Vi-B3 Scope of context

To determine the role of emojis in a post, it is not sufficient to study the sentence containing the emoji because the content can be loosely organized, and the emoji can be connected to a sentence far from it. For example, in the first paragraph of an issue, i.e., “Sorry, but I’m unfamiliar with Javascript apps. How do you install the cypher plugin? Step-by-step instructions might be useful for people like me who are not familiar with how this works and too dumb to figure this out. ”, the confounded face on the verge of tears is used to strengthen the sentiment expressed by the word “sorry” in the beginning.

Additionally, due to the orientation of project development and collaboration, the context of emojis on GitHub consists of not only texts but also operational behaviors. A typical scenario is that a single emoji constitutes a post without any textual words such as the pull request comment “”, which demonstrates sparkles. By investigating the original pull request, the previous replies, and behaviors including adding labels and merging a commit around this post, we find that the represents a positive attitude to a commit and categorize its intention to sentiment expression.

Vi-B4 Intention vs. position

The position of the emoji in text can also affect the intention categorization. In specific, when a sentimental emoji appears in the beginning of an expression followed by sentimental texts (e.g., “ LGTM, thanks!”), it can be labelled to either of the two sub-categories of sentimental usage. In this work, we determine to label it as sentiment expression for no sentiment has been conveyed before the appearance of this emoji. As a comparison, the thumbs-up gesture in “LGTM, thanks! ” will be labelled with a sentiment strengthening intention.

Vi-B5 Definition vs. context-based understanding

The embedding-based analysis in Section V has revealed domain-specific semantics of emojis determined by the technical context on GitHub. In the manual annotation process, we confirmed such usage of emojis (such as , , and ) in our sampled posts. However, we also find that such emojis can have other meanings. For example, the rocket can mean launch (“Launching soon ”) as well as acceleration (“It uses the new osmium and its nodejs bindings for  performance.”), which is not domain specific on GitHub. Additionally, the emoji can be used as a symbol regardless of its meaning (e.g., “ gym has moved to the fastlane main repo ”).

Another observation comes from the emoji , which is known as clear button by definition and can find its application to mean “clear”. This emoji can also be interpreted to be “change log” or “change list” in the vision control system. For example, a list of change logs are wrapped by “” and “/” in a pull request, where we annotate its intention as statement enriching.

Such observation indicates ambiguity of emojis on GitHub, and the meaning should be determined by both the contextual information and the intention of using the emoji.

Vi-B6 Sentiment expression vs. atmosphere adjusting

To distinguish the two different intentions in practice is challenging for emojis especially facial emojis. It is not rare that such emojis are used merely to make the expression more friendly instead of showing emotions or attitudes. In the pull request “… Please take a look and check if it’s worth it having it here ”, the smiling face with open mouth and smiling eyes is used to adjust the tone in order not to be serious, just like the facial expression in face-to-face communications. The intention of this is categorized to atmosphere adjusting while in the following example, the pull request “… All covered with new tests ”, it is sentiment expression because this emoji expressed happiness for adding new methods covered with new tests. In a word, the contextual information is leveraged to determine the real intention of such emojis.

Vi-B7 Unusual usage

Emojis can be used in an unusual way, which is reasonable because there are no strict rules and people are still experimenting with emojis. However, unusual usage can introduce obstacles to understand the emoji and even the post. For example, the emoji of thumbs up in the issue “I have tried to implement the ABC algorithm, but am getting problem with ” followed by a line of code is challenging for us to figure out. As the other intention categories can not apply to this case, we suppose the is used to referring to the following code as the mentioned problem where and are more suitable, and to enrich the statement.

Vi-B8 Criteria consistency

Considering that some of the criteria emerge during the annotation process, we looked back to the posts after the process and made necessary adjustments to ensure the consistency of the categorization.

Vi-C Intention Distribution

The intentions of using emojis in the sampled posts are presented in Fig. 5. We next briefly report the distribution of intentions in different posts, respectively.

Fig. 5: Distribution of intentions.

Vi-C1 Issue

In issues, emojis are mostly used unintentionally with a 21.50% proportion. This is because emojis often occur in pasted logs and code in raised issues, which echoes the extremely dense usage of emojis in the middle of issue posts (see Fig. 3). Such finding also evidences that emojis are widely adopted in multiple scenarios in software engineering. Followed intentions are sentiment strengthening (20.00%), sentiment expression (18.50%), atmosphere adjusting (17.00%), and statement enriching (14.00%).

The intention of representing emojis, although is the last but one intention with a 3.75% proportion, is mostly seen in issues as well as README. Such intention often occur in emoji related contexts and in opinion consultations such as “Click on + to add your votes” where is one of the six reactions provided by GitHub.

Vi-C2 Pull request

The four main intentions of using emojis in pull requests are atmosphere adjusting (28.75%), statement enriching (23.50%), sentiment expression (19.75%), and sentiment strengthening (14.00%). In comparison with issues, the unintentional use is significantly reduced, and the emojis are more used to enrich statement and adjust atmosphere. On possible explanation of such difference is the relatively narrow scope of target audience (i.e., the project owner) and explicit requests (i.e., to get the contributed code pulled and merged to the repository) of pull requests.

Vi-C3 Comments

Sentimental usage of emojis is dominant in the two types of comments especially in pull request comments, where more than half of emojis (i.e., 51.75%) are used to express sentiments and 28.25% to strengthen sentiments. This is understandable because opinions are needed to response to raised questions, discussions, plans, and implementations. In the comments of issues, another main intention of using emojis is atmosphere adjusting (29.00%), possibly due to the demand of smoothing communications in various discussion contexts.

Vi-C4 Readme

Instead of sentimental usage, the main intention of using emojis in README is content emphasis, occupying 34.25% of emoji occurrence. Because of the nature to communicate expectations for projects, README files are often with long texts and multiple points. Using emojis can effectively help attract attention to the emphasized points from the massive content. Another observation is that emojis contribute in hierarchical lists or checklists with a probability of 5.50%. These two intentions of using emojis in README are significantly higher than in other posts.

Summary. Knowing why an emoji is used is of significance to understand the real meaning of the emoji and the expression with the emoji. In this section, we proposed a taxonomy of emoji usage intention on GitHub based on our understanding and a manual annotation task conducted with 2,000 posts. The ambiguity of emojis, the diversity of emoji selection, and the complicated context of the emoji can affect the intention determination process and make it challenging. The generated taxonomy as well as the demonstrated usage examples can provide a guidance for intention recognition before leveraging emojis in further analysis.

By completing the annotation task, we obtain the distribution of emoji usage intentions in different posts. We find that emojis are heavily used to express or strengthen sentiments in conversational posts especially in comments, while emojis are mainly used to emphasize the content in README, which often contains a mass of information. We also find that emojis are widely used to smooth communications to develop a positive and friendly atmosphere of the open-source collaboration platform.

Vii Implications

Supplement for sentiment analysis. Understanding the thoughts and sentiments of developers has always been an active direction in software engineering research. However, sentiment analysis in SE tasks is challenging due to the unreliable results provided by existing tools [9, 37, 8, 34]. One difficulty in automated sentiment analysis with lexical approaches such as SentiStrength [38] in the SE domain is the lack of sentimental words in dictionary. In our study, emojis are found to be widely leveraged in GitHub posts to not only strengthen but also express sentiments as substitute of plain text, which implies the necessity to enlarge the dictionary of sentimental words with emojis. In addition, it is also possible to use emojis as weak sentiment labels [27, 28] in distant-supervised learning.

Status sensor for contribution activeness management. The activity of contributors of open source projects can be affected by their emotion and psychological status. For example, contributors are more likely to become inactive when they express strong positive or negative emotions [39]. [1] investigated food-related emotional experiences by analyzing emoji usage on Twitter, and find emoji usage can reflect the status of users such as being accompanied or not. Emojis can be leveraged as a sensor of contributors’ status on open source platforms such as GitHub. In addition, developers can add emojis to their conversations to promote a mild and friendly atmosphere for the platform [40].

Visual design for problem solving and project collaboration. The analysis in Section III-C implies that emoji usage can be related with participation in issues and activeness of developers, possibly due to the eye-catching visual design which makes the issue more readable and less boring. Such initial findings encourage further research of the role of emojis in helping problems to be solved and promoting project collaboration in the open source community. Broadly, adding visual designs into traditionally text-heavy tasks not only adds fun to the work, but may also help engage users in the tasks and even improve the quality of work [41]. On the other side, GitHub and other developer communities like StackOverflow may consider to add more visual features to attract users into discussions, such as animations or GIF images.

In fact, the current low ratio of posts containing emojis indicates a great opportunity for the GitHub community to promote emojis in the conversation, through the designs of recommender systems or specialized interfaces.

Community-specific and identity-based design. Although emojis have evolved into an ubiquitous language, we have seen that a specific user group tend to use emojis and interpret emojis in different ways. The community-specific interpretations of emojis has become the norm of the social group and are strongly tied to the social identity of the community [42]. This indicates the rationale and opportunity of designing community-specific emojis, emojis that advocate for the unique identity and culture of the community.

Viii Discussion

Viii-a Threats to Validity

The quality of data can be determined by the design of GitHub API and the collection strategy of GHTorrent. For example, some links can not be reached because users changed their usernames.181818https://help.github.com/articles/what-happens-when-i-change-my-username/, retrieved in August 2018 The data pre-processing, during which we extract texts from the crawled markdown files, replace code, URLs, and pictures with labels, and filter duplicates and spam posts containing obviously redundant emojis, can also lead to loss of information.

The proposed intention taxonomy is based on our understanding and observation from the selected posts, which can be affected from three factors. First, although we select 400 posts for each post type for statistical significance, we can not guarantee the full coverage of emoji usage types. We claim the extensibility of the taxonomy for occurrence of new uses. Second, the selected posts are all in English, hence the derived distribution of emoji usage intentions can not represent that in non-English posts. Third, although the annotation task was conducted by two of the authors independently followed by a discussion to address discrepancies, mistakes may still occur because the gap between the reader and the writer can always exist. The results can be influenced by subjective opinions of the annotators and their language, education and culture backgrounds.

The findings and the proposed taxonomy can not be directly generalized to the whole field of software engineering, due to the following two reasons. On the one hand, the studied emojis are from free communication texts on GitHub, not including other artifacts such as code and documentation. On the other hand, emojis in communications on other platform can show different characteristics, hence need to be studied with the specific contexts.

Viii-B Future Work

The use of emojis can be affected by multiple factors, including the popularity of the project, the topic and atmosphere of the discussion, the background and personality of the user, the time of posting, etc. We plan to study emoji usage in depth by modeling the factors to uncover such relations. Another direction is to automate the intention categorization by developing machine learning models.

Ix Conclusion

We presented a large-scale empirical study on how developers use emojis in their communications on GitHub. The data set involves free texts including conversations and README from 3.09 million GitHub projects covering 3.95 million users spanning from January 2012 to June 2017. We found that developers show domain specific preference, usage, and understanding of emojis in their communications in comparison to common Internet users. We conducted a manual annotation task with 2,000 posts to propose a taxonomy of emoji usage intention for GitHub users. Results show that in addition to be widely used to smooth communications, emojis are heavily used to help express sentiments in conversations especially in comments and are mainly used as eye-catching symbols in README files. Emojis may be used as sentiment sensors of developers and should be considered in textual analysis in the software engineering field.

References

  • [1] L. Vidal, G. Ares, and S. R. Jaeger, “Use of emoticon and emoji in tweets for food-related emotional expression,” Food Quality and Preference, vol. 49, pp. 119–128, 2016.
  • [2] Y. Chen, J. Yuan, Q. You, and J. Luo, “Twitter sentiment analysis via bi-sense emoji embedding and attention-based lstm,” arXiv preprint arXiv:1807.07961, 2018.
  • [3] Z. Chen, X. Lu, W. Ai, H. Li, Q. Mei, and X. Liu, “Through a gender lens: Learning usage patterns of emojis from large-scale Android users,” in Proceedings of the 27th Web Conference2016, 2018, pp. 763–772.
  • [4] D. Marengo, F. Giannotta, and M. Settanni, “Assessing personality using emoji: An exploratory study,” Personality and Individual Differences, vol. 112, pp. 74–78, 2017.
  • [5] X. Lu, W. Ai, X. Liu, Q. Li, N. Wang, G. Huang, and Q. Mei, “Learning from the ubiquitous language: an empirical analysis of emoji usage of smartphone users,” in Proceedings of 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2016, pp. 770–780.
  • [6] R. Siadati, P. Wernick, and V. Veneziano, “Modelling politics in requirements engineering: Adding emoji to existing notations,” arXiv preprint arXiv:1703.06101, 2017.
  • [7] M. R. Islam and M. F. Zibran, “Leveraging automated sentiment analysis in software engineering,” in Proceedings of the 14th International Conference on Mining Software Repositories, 2017, pp. 203–214.
  • [8] N. Novielli, F. Calefato, and F. Lanubile, “The challenges of sentiment detection in the social programmer ecosystem,” in International Workshop on Social Software Engineering, 2015, pp. 33–40.
  • [9] B. Lin, F. Zampetti, G. Bavota, M. Di Penta, M. Lanza, and R. Oliveto, “Sentiment analysis for software engineering: How far can we go?” in Proceedings of 40th International Conference on Software Engineering, 2018, pp. 94–104.
  • [10] N. Ljubesic and D. Fiser, “A global analysis of emoji usage,” in Proceedings of the 10th Web as Corpus Workshop, 2016, pp. 82–89.
  • [11] W. Ai, X. Lu, X. Liu, N. Wang, G. Huang, and Q. Mei, “Untangling emoji popularity through semantic embeddings,” Proceedings of the 11th International AAAI Conference on Web and Social Media, pp. 2–11, 2017.
  • [12] F. Al Rashdi, “Forms and functions of emojis in whatsapp interaction among omanis,” 2015.
  • [13] R. Zhou, J. Hentschel, and N. Kumar, “Goodbye text, hello emoji: Mobile communication on WeChat in china,” in Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 2017, pp. 748–759.
  • [14] F. Barbieri, F. Ronzano, and H. Saggion, “What does this emoji mean? a vector space skip-gram model for twitter emojis.” in Language Resources and Evaluation Conference, 2016.
  • [15] P. K. Novak, J. Smailovic, B. Sluban, and I. Mozetic, “Sentiment of emojis,” PloS One, vol. 10, no. 12, 2015.
  • [16] T. Hu, H. Guo, H. Sun, T. T. Nguyen, and J. Luo, “Spice up your chat: The intentions and sentiment effects of using emoji,” in Proceedings of the 11th International AAAI Conference on Web and Social Media, 2017, pp. 101–111.
  • [17] H. Cramer, P. de Juan, and J. R. Tetreault, “Sender-intended functions of emojis in US messaging,” in Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services, 2016, pp. 504–509.
  • [18] H. Pohl, C. Domin, and M. Rohs, “Beyond just text: semantic emoji similarity modeling to support expressive communication ,” ACM Transactions on Computer-Human Interaction, vol. 24, no. 1, pp. 6:1–6:42, 2017.
  • [19] V. Sinha, A. Lazar, and B. Sharif, “Analyzing developer sentiment in commit logs,” in Proceedings of the 13th International Conference on Mining Software Repositories, 2016, pp. 520–523.
  • [20] W. N. Robinson, T. Deng, and Z. Qi, “Developer behavior and sentiment from data mining open source repositories,” in System Sciences (HICSS), 2016 49th Hawaii International Conference on, 2016, pp. 3729–3738.
  • [21] D. Pletea, B. Vasilescu, and A. Serebrenik, “Security and emotion: sentiment analysis of security discussions on github,” in Proceedings of the 11th working conference on mining software repositories, 2014, pp. 348–351.
  • [22] R. Jongeling, S. Datta, and A. Serebrenik, “Choosing your weapons: On sentiment analysis tools for software engineering research,” in 2015 IEEE International Conference on Software Maintenance and Evolution, 2015, pp. 531–535.
  • [23] T. Ahmed, A. Bosu, A. Iqbal, and S. Rahimi, “Senticr: a customized sentiment analysis tool for code review interactions,” in Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering, 2017, pp. 106–111.
  • [24] M. R. Islam and M. F. Zibran, “A comparison of dictionary building methods for sentiment analysis in software engineering text,” in 2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, 2017, pp. 478–479.
  • [25] E. Guzman, D. Azócar, and Y. Li, “Sentiment analysis of commit comments in github: an empirical study,” in 11th Working Conference on Mining Software Repositories, 2014, pp. 352–355.
  • [26] A.-I. Rousinopoulos, G. Robles, and J. M. González-Barahona, “Sentiment analysis of free/open source developers: preliminary findings from a case study,” Revista Electronica de Sistemas de Informaçao, vol. 13, no. 2, p. 1, 2014.
  • [27] J. Zhao, L. Dong, J. Wu, and K. Xu, “MoodLens: an emoticon-based sentiment analysis system for Chinese Tweets,” in The 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2012, pp. 1528–1531.
  • [28] B. Felbo, A. Mislove, A. Søgaard, I. Rahwan, and S. Lehmann, “Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm,” in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017, pp. 1615–1625.
  • [29] G. Gousios, “The ghtorrent dataset and tool suite,” in Proceedings of the 10th Working Conference on Mining Software Repositories, 2013, pp. 233–236.
  • [30] “REST API v3.” [Online]. Available: https://developer.github.com/v3/
  • [31] U. Pavalanathan and J. Eisenstein, “Emoticons vs. emojis on twitter: A causal inference approach,” arXiv preprint arXiv:1510.08480, 2015.
  • [32] F. Wilcoxon, “Individual comparisons by ranking methods,” Biometrics bulletin, vol. 1, no. 6, pp. 80–83, 1945.
  • [33] K. Steinmetz, “TIME Exclusive: Here Are Rules of Using Emoji You Didn’t Know You Were Following.” [Online]. Available: http://time.com/2993508/emoji-rules-tweets/
  • [34] N. Na’aman, H. Provenza, and O. Montoya, “Varying Linguistic Purposes of Emoji in (Twitter) Context,” in Proceedings of ACL 2017, Student Research Workshop, 2018, pp. 136–141.
  • [35] R. C. Jackson et al., “The pragmatics of repetition, emphasis and intensification,” Ph.D. dissertation, Salford, 2016.
  • [36] T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” in Advances in neural information processing systems, 2013, pp. 3111–3119.
  • [37] R. Jongeling, P. Sarkar, S. Datta, and A. Serebrenik, “On negative results when using sentiment analysis tools for software engineering research,” Empirical Software Engineering, vol. 22, no. 5, pp. 2543–2584, 2017.
  • [38] M. Thelwall, K. Buckley, and G. Paltoglou, “Sentiment strength detection for the social web,” Journal of the Association for Information Science & Technology, vol. 63, no. 1, pp. 163–173, 2011.
  • [39] D. Garcia, M. S. Zanetti, and F. Schweitzer, “The role of emotions in contributors activity: A case study on the gentoo community,” in Third International Conference on Cloud and Green Computing, 2013, pp. 410–417.
  • [40] E. Hatfield, J. T. Cacioppo, and R. L. Rapson, “Emotional contagion,” Current directions in psychological science, vol. 2, no. 3, pp. 96–100, 1993.
  • [41] A. Sutcliffe, “Designing for user engagement: Aesthetic and attractive user interfaces,” Synthesis lectures on human-centered informatics, vol. 2, no. 1, pp. 1–55, 2009.
  • [42] J. J. Gumperz, Language and social identity.   Cambridge University Press, 1982, vol. 2.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
""
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
   
Add comment
Cancel
Loading ...
326933
This is a comment super asjknd jkasnjk adsnkj
Upvote
Downvote
""
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters
Submit
Cancel

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test
Test description