Finding the Best Text Analysis for Your Data

How to Automatically Classify Lists of Items

Note that the feature attempts to correct misspellings and inconsistent formatting. You can specify the delimiter if needed.

If you plan to update the data later, you should fill in the Required Variants table with all the themes and their variants to keep results consistent before updating the data. You will also need to edit the saved variables per the article to update those instead of recreating them from scratch.

An interactive output of the results from the categorization.

You can also create a variable for the First Category mentioned in the list and a variable set for all Categories mentioned in each observation in the list.

Items saved in separate text variables (max-multi format)

In this case, when you save Categories and First Category, you will get a Binary-Multi (Compact) variable Set for each of the variables. 

To combine these into one variable set similar to above, you must change each to Structure > Nominal-Multi, select all the sets, and use the Binary Variables transformation.

To see all items from each variable in the same table broken out by 1, 2, and 3, for example, you will need to create Binary Variables for each variable set and follow How to Combine Separate Questions into a Grid in Displayr.

Open-end text responses

You can save a Tidied Text variable (see the right column below):

You can save Categories into a new variable set to see the proportions mentioning each keyword:

Various formats including:

Delimited strings of items

Items saved in separate  text variables (max-multi format)

Open-end text responses

Text-Multi variable sets

A table with a single column of text 

How to Classify Text Data

Our categorization tool is the most robust text analysis tool.

• You can use a proprietary algorithm or our enhanced Displayr AI modeling to automatically create themes and classify responses. Using the Create and Classify buttons.
• You can use our Similarity to: search to search for responses similar to responses already assigned to a specific theme.
• You can use our Fuzzy Match search to search responses similar to a given search term (synonyms and misspellings included!)
• You can easily manually add/remove responses to different themes.
• The tool uses a .QCodes file on the backend to store the matching responses for each theme. This can be imported and exported to use across different variables and documents.
• You can also easily classify "Other" text responses from multi-response questions back into the main variable set of options (known as Back Coding Variables).
• You can create a QCodes file based on a text variable and another variable set of themes, import that into the tool, and finish classifying or revising the themes in the tool.
• Text is automatically translated.

1. A variable set with the themes for each response.
2. A .QCodes file (an XML file of the responses and themes) can be exported to reuse in the categorization tool elsewhere.

If you plan on updating your text data later, you should use this tool because it retains previously classified responses as-is.

How To Automatically Classified Unstructured Text Data

Think of this as more of a quick and dirty categorization tool.

This is created via an output on the page. It uses the traditional large language model (LLM) to classify responses (not as intelligent as Displayr AI), and manual categorization and revisions are not supported. However, you can use an existing variable set of themes to train the algorithm to try to classify the remaining text responses into those themes (there's no guarantee that previously classified responses will be classified into the same theme). Note if responses don't fit into a theme they will not be coded at all. QCodes files are also not supported.

1. An output listing themes, their proportions, and examples of responses.
2. You can save a variable set of the Categories or First Category (if using a Binary-Multi set for the existing categorization) for each response.

If the text variable data is updated, the categorization is recreated from scratch, and thus the themes and responses classified into them may change. The variable set that was initially saved will be invalid and must be resaved as well. 

If you plan on updating your text data later on you should use the categorization tool described above because it keeps previously classified responses as-is.

How to Calculate Sentiment Scores for Open-Ended Responses

A simple technique of calculating a sentiment score for each response: negative (below 0), neutral (0), and positive (above 0). These scores can be used in stat testing and can be used to create themes of the sentiment for each response. 

A variable "Sentiment scores from ____" is created in the Data Sources tree with the score for each response. This can be crossed by other variables for testing sentiment between groups.

How To Automatically Classify Unstructured Text Data Into an Entity List

The algorithm identifies entities within text responses. Instead of specific items in the list (i.e. New York, Paris, the beach) entities are more generic (i.e. Location).

A list of the entities currently supported by default can be found in the technical documentation. However, you can add more entities manually.

1. An output listing entities, their proportions, and examples of responses.
2. You can save a variable set of the Categories or First Category mentioned for each response.

If the text variable is updated, the same algorithm is used to identify items that go into each entity. Thus, previously classified responses will remain and new responses will be automatically classified.

Mutually Exclusive makes a Nominal variable

More than 1

Mutually Exclusive makes Nominal-Multi variable set

Multiple Overlapping makes Binary-Grid variable set