Analyze Sentiments in Survey Responses

You can use the Categorize Survey Data window to analyze and autocode the sentiment of each response. To do this, start the Analysis > Categorize Survey Data feature and drag a code containing the open-ended answers to a question from the “Code System” window into the dialog box that appears:

After clicking OK, the following interactive working environment appears:

Analyze Sentiments of Survey Responses

With MAXQDA you can perform a sentiment analysis of the responses listed in the Survey Analysis window. This automatically evaluates whether the content is to be assessed as negative, neutral, or positive. The survey responses can then be autocoded according to their sentiments. Sentiment analysis is accessed from the ribbon of the Survey Analysis window (in the top right):

1. 1. Click Analyze Sentiments to start the process.
   2. In the dialog that appears, you can select various options:

Select language – this selection ensures, that MAXQDA will use the appropriate resource for sentiment evaluation.

Ignore hashtags – if this option is enabled, hashtags, such as #bestever, are not taken into account when evaluating sentiment.

Apply stop word list – if this option is enabled, all words from the selected stop word list will be ignored when evaluating the sentiment. Also hashtags that correspond to a stop word are ignored.

1. After clicking OK, the sentiment analysis is performed and four columns are added to the table view:
   • Sentiment: Contains the rating of the sentiment in five levels from "negative" to "positive". Responses that do not contain words with a sentiment value are marked with "No sentiment".
   • (Positive) Words: Number of words that were evaluated as positive.
   • (Negative) Words: Number of words that were evaluated as negative.
   • Difference: Difference between positive and negative words. If the number is negative, the negative words predominate. Emojis and Symbols count as a word.

Autocode survey responses with sentiments

You can save the assignment of the sentiments to the individual responses as codes in MAXQDA’s code system:

1. Call the function Start > Autocode Responses with Sentiment in the menu tab.
2. In the dialog that appears, select the sentiment levels you want to autocode.

After clicking OK, the selected sentiments are created as codes. If they already exist in the code system, the existing codes will be used for coding. The text of the reponses will be autocoded with the selected codes.

How does MAXQDA evaluate the sentiments?

MAXQDA uses a lexicon to evaluate sentiments, which contains a sentiment score for each word in the lexicon. This value is negative for words with negative connotations, close to zero for neutral words, and positive for words with positive connotations.

When analyzing a response, MAXQDA checks each word whether it is contained in the lexicon and assigns the sentiment score to this word (hashtags and stop word list words are ignored if desired). If the word is not in the lexicon, MAXQDA looks up the word in a lemma list. If a lemma is found for the word and this lemma is contained in the sentiment lexicon, the sentiment score of the lemma is used for the word.

In addition, two rules are applied to optimize the evaluation of sentiment:

1. In case of negation, the scores of the following 3 words are reversed, for example, the statement "I was not very happy" is classified as negative.
2. In case of modal verbs, such as "can", "should", etc., the sentiment scores of the following words are reduced.

The mean value is calculated from the sentiment scores of the evaluated words in a response. If the mean value is negative, the response is evaluated as "negative" or "slightly negative", if the mean value is positive, the response is evaluated as "positive" or "slightly positive". If the mean value is equal or close to zero, the response text is classified as "neutral". Responses that don’t contain any word with a sentiment score are classified as "no sentiment".

In the literature on automatic sentiment analysis, values of 60 to 70% agreement compared to a human control assignment are usually reported. It has to be taken into account that human assessments of the same text can also differ. The quality of a sentiment analysis depends on several factors, especially the fit of the lexicon for a given topic. Please also note that a sentiment analysis has little meaning for responses in which little to no evaluations are expressed.

English lexicon

The lexicon for English is based on the resource SentiWordNet 3.0, which is licensed under an Attribution-ShareAlike 4.0 Unported (CC BY-SA 4.0) license (for more info on SentiWordNet, see Baccianella, Esuli & Sebastiani, 2010). The SentiWordNet 3.0 lexicon has been optimized for use in MAXQDA, including

• removal of special characters and duplicates
• addition of typical social media abbreviations (e.g. LOL - laughing out loud), emojis, and superlatives
• manual adjustment of scores for individual words to scores of their synonyms

Bibliography

Baccianella, S.; Esuli, A.; Sebastiani, F. (2010). SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining. Proceedings of LREC. 10.
http://nmis.isti.cnr.it/sebastiani/Publications/LREC10.pdf