UPCOMING WEBINAR: Build it once. Use it again: Create reusable charts with AI Displayr has an awesome array of cutting-edge text analysis features that can make working with text data so much faster. Text data comes in various shapes and sizes, and you may want to perform different analyses on the same data, such as seeing the percentage of people mentioning topics as well as their sentiment. There are also pros and cons to certain features based on whether you want to use previous results to analyze updated data.
This article discusses the various uses of the text analysis features available in Displayr.
Requirements
- All features accept a variable with a Structure > Text (though different analyses may have additional requirements).
- If wanting to analyze multiple text variables alongside each other, you should Combine them into a Text-Multi Structure first.
- If you plan to use any AI-enhanced categorization tools (such as the automated features in our text categorization tool), you must opt-in to Displayr AI. If you don't enable Displayr AI, then Displayr will use its proprietary algorithm to create themes and classify text items.
- Examples shown below are in the data file here.
Method 1 - Identifying items in text responses
Sometimes, instead of grouping responses into broad themes based on their content, you may want to just pull out the keywords or items (i.e., brands) to analyze. Variables like this are similar to spontaneous awareness/unaided awareness questions in surveys, brands purchased, or other items. There are various features and methods that can help with this:
| Data Format | Method | Result |
| Delimited strings of items (or brands) |
First, parse the items into an individual variable for each mention (see how to split a text string see how to split text string Using R or Using JavaScript). Then use the Text-Multi variable set this creates in the Text Categorization tool along with a custom prompt to classify the responses into categories. |
A - Categorized Binary-Multi variable set. |
|
Items (or brands) saved in separate text variables (max-multi format) |
To analyze as one group of items, first Combine variables into a Text-Multi variable set. Then use the Text Categorization tool along with a custom prompt to classify the responses into categories.
Otherwise, you can keep the variables separate, use the Text Categorization tool along with a custom prompt to classify the first variable into categories, and share the mapping and coding when categorizing each remaining variable, see How to Reuse an Existing Categorization (Code Frame) on a Different Variable. |
A - Categorized Binary-Multi variable set for each variable set analyzed. To see all items from each variable in the same table broken out by 1, 2, and 3, for example, you will need to create a Binary-Grid from the variable sets. See How to Combine Separate Questions into a Grid in Displayr. |
|
Open-end text responses
|
Use the Text Categorization tool along with a custom prompt to classify the responses into categories. For a more basic analysis, see How to Extract Keywords and Phrases from Text. This can be used to identify keywords and clean the data (see the article for various settings). Many people use tidied text variables saved from this to create Word Clouds, or, more rarely, use the output to create a Term Document Matrix for advanced custom R code. |
A - Categorized Binary-Multi variable set is created from the Text Categorization tool.
If running a more basic analysis: You can save a Tidied Text variable (see the right column below): You can save Categories into a new variable set to see the proportions mentioning each keyword: |
|
Various formats including: Delimited strings of items Items saved in separate text variables (max-multi format) Open-end text responses Text-Multi variable sets A table with a single column of text |
How to Create a Word Cloud can be used to create a word cloud visualization. There is some attempt at removing connector words by the word cloud, but the Set Up Text for Analysis output is more robust. |
Method 2 - Classifying text responses
Classifying open-ended text responses, also known as coding or categorization in Market Research, has historically been a very long and manual process. Displayr has features with more advanced modeling and AI built in that can streamline this process, drastically reducing the time spent manually classifying responses. There are a few tools to do this:
| Feature | Method | Result |
| Categorization Tool |
Our categorization tool is the most robust text analysis tool.
|
If you plan on updating your text data later, you should use this tool because it retains previously classified responses as-is. |
| Sentiment analysis |
How to Calculate Sentiment Scores for Open-Ended Responses A simple technique of calculating a sentiment score for each response: negative (below 0), neutral (0), and positive (above 0). These scores can be used in stat testing and can be used to create themes of the sentiment for each response. |
A variable "Sentiment scores from ____" is created in the Data Sources tree with the score for each response. This can be crossed by other variables for testing sentiment between groups. |
| Entity extraction |
How To Automatically Classify Unstructured Text Data Into an Entity List The algorithm identifies entities within text responses. Instead of specific items in the list (i.e. New York, Paris, the beach) entities are more generic (i.e. Location). A list of the entities currently supported by default can be found in the technical documentation. However, you can add more entities manually. |
If the text variable is updated, the same algorithm is used to identify items that go into each entity. Thus, previously classified responses will remain and new responses will be automatically classified. |
Types of Categorizations
If using the Categorization Tool described above, you will need to specify what type of categorization you want to create beforehand:
- Only one theme - each text response can only be assigned one theme. That is, these categories are mutually exclusive.
- Multiple themes - each text response can be assigned more than one theme. That is, these are multiple overlapping categories.
You can use the table below to decide how to set up the categorization tool for your data based on the type of table you want to create.
| Data format | Number of themes for each response | Structure of final variable set and table |
| Single Text Variable | Only one theme |
Makes a Nominal variable |
| Single Text Variable or Text-Multi Variable Set | Multiple themes | Makes a Binary-Multi variable set |
| More than 1 Text Variable selected (not in a set) | Only one theme |
Makes a Nominal-Multi variable set |
| More than 1 Text Variable selected (not in a set) | Multiple themes |
Makes a Binary-Grid variable set |
If you have more than one text variable you want to classify using the same themes, but you don't want to show the data in the same table, you should share the categorization code frame. See the sharing section in How to Reuse a Categorization (Code Frame) on a Different Variable.