How are Emotions Evoked?  

Mohammad, S. M., & Turney, P. D. (2010). Emotions evoked by common words and phrases: Using Mechanical Turk to create an emotion lexicon. In Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text (pp. 26–34). Association for Computational Linguistics. Retrieved from https://pdfs.semanticscholar.org/b89f/2a5ef026f326 45348584a57ba23dcdf03d3e.pdf

The authors created a “moderate-sized English emotion lexicon by manual annotation through Amazon’s Mechanical Turk [MT] service” (pp. 26–27) named EmoLex, the largest lexicon to date, to conduct emotion analysis. To develop the lexicon, the authors first identified a list of words and phrases requiring human annotations. The authors created MT Human Intelligence Tasks (HITs) for each set of terms, requesting five different assignments for each HIT. The authors validated the completed assignments using automated scripts, repeating the previous step to re-validate failed assignments. The authors then consolidated the different annotations for a target term “by determining the majority class of emotion intensities” (p. 30), analyzing the data for frequency, agreement, and semantic orientation of words. The results showed that MT is a viable and inexpensive way to create a high-quality emotion lexicon that provides “insights into how prevalent emotion bearing terms are among common unigrams and bigrams” and how “emotions tend to be evoked simultaneously by the same term” (p. 33).  

‍