UPCOMING WEBINAR: Run Research Your Way With AI Skills Sometimes a result on a table is different from what you were expecting. A process for tracking down the cause of the results is:
- Check the number of cases in the data set
- Review the sample size of the table or visualization
- Review the sample size for each cell in the table
- Review the missing data settings
- Check filters and weights
- Check the structure of the variable sets used to construct the table
- Review the raw data
- Review value attributes
- Review the inputs and other settings in the Properties
- Follow links back to their source
- Review Rules
- Review the statistical testing assumptions
- Search Help
- Contact us
Check the number of cases in the data set
A common cause of results that look wrong is that the data file contains too few or too many cases. This is checked by clicking on the data set in the Data Sources tree and reviewing the Number of cases shown in the Properties.
Review the sample size of the table or visualization
Data issues are often discovered by looking at the sample size of a table or visualization. (If you can't see the sample size, change the visualization back into a table first.) For example, the visualization below has a sample size of 895 (highlighted in yellow at the bottom).
Sometimes tables or visualizations will show a range of sample sizes, indicating that the sample size is different for different results. For example, the table below shows sample sizes varying from 758 to 873. This is caused by different cells in the table having different sample sizes. This is discussed in more detail in the next section.
Review the sample size for each cell in the table or visualization
If you have a visualization, rather than a table, you need first change it back to a table by pressing the Table button in the Properties.
The table above shows row percentages. We can see the sample size for each row by selecting the table and selecting Data > Statistics > Right > Row sample size from the Properties . The resulting table is shown below. The last column shows us that the smallest sample size is for Cancel your subscription, and it is 758.
Note that in the footer, it says 69 are missing. This is calculated as the total sample size (600) less the sample size for any of the rows (531).
In grid tables, we would instead use Statistics > Cells > Sample size, as the result can differ from cell to cell.
Special note should be made of the sample size for any NET or SUMs. These are always calculated based on people who have no missing data for any of the other cells in the row or column of the table. For example, the footer in the table below tells us that the sample size varies from 828 to 869, however, the small sample of 828 is only for the NET, and each of the other categories has, on its own, a substantially larger sample size.
Review the missing data settings
If your table is showing as 100% across all rows:
You should check your missing data settings. This can occur if missing data is excluded from binary structured variable sets (binary - multi, binary - grid). Select the variable set in the Data Sources tree, and from Properties , click Categories. From there, update the Missing Values setting for the value representing not selected, not aware, no, etc. to Include in analyses.
The table will update to show the % of those selected from the total sample size:
Check filters and weights
You can see if a weight or filter has been applied in the footer (if there is a footer), or, by selecting the output and viewing the filters and weights in the Properties panel. In this example, no weight has been applied, but the table has been filtered based on males. If a visualization has been created by a separate table as an input, you need to review the weights and filters of that table, rather than the visualization.
Please see How to Investigate Filters that Show Incorrect or Odd Results and How to Troubleshoot Weights for more instructions on how to figure out what is wrong with your filter/weight variable.
Check the structure of the variable sets used to construct the table
When Displayr imports data, it automatically groups variables into variable sets. Sometimes it groups the variables or chooses a structure that may be different from what you expect. See Variable Sets and Manipulating Data for more information.
Review the raw data
Select a variable set in the Data Sources tree, and select Raw Data from the Data tab in Properties
. If you select Values in the Raw Data panel, you will see any recoding of the data denoted using an arrow from the source value (4 below) to the recoded value (100 below).
Review value attributes
You can see how data has been recoded by selecting any variables and pressing the Values & Labels, Values, or Categories button, depending on the variable's structure, in Properties > Data > Attributes. Please see Variable Sets on how to adjust and confirm value attribute settings.
Additional insight can be obtained by using the Reset and Split from the right-click menu to see what categories are used in NETs and merged rows/columns on the table.
Review the inputs and other settings in the Properties
Anything that is calculated in Displayr - variables, tables, calculations, visualizations - can be clicked on, allowing you to see their data inputs in the Properties. For example, below we can see that the table is a Summary of Customer effort.
Additionally, the result may be caused by other settings in Properties, so have a hunt around. For more complex tables, you may also want to review their Inputs & Outputs
to confirm all of the correct upstream data is being used.
Follow links back to their source
When you hover over an input, a tooltip will appear in Properties. For example, the tooltip obtained when hovering your mouse pointer over Customer effort is shown above. You can click on the arrow in the tooltip, and Displayr will then select this input. This allows you to trace any calculation back to its source.
Review Rules
A Rule is a bit like conditional formatting in Excel, except that it can also change the data and be applied to visualizations. As an example, the table below replaces small values with * symbols. You can verify if a rule has been applied to a table by selecting it and going to Data > Rules in Properties.
For more details on how Rules can be used in Displayr and other caveats, please see How to Use Rules in Displayr.
Review the statistical testing assumptions
The statistical testing options are set in Properties > Appearance > Significance.
To see more detail on how these settings are applied, please see How to Apply Significance Testing in Displayr, and to work through what might be causing incorrect stat testing results, please see How to Investigate Your Statistical Significance Testing.
Search Help
The search feature in Displayr Help can be used to search through our documentation. There, you may find articles addressing specific issues, such as Why don't my percentages add up to 100% and How to Set Value Attributes for a Binary-Multi and Binary-Grid.
Contact us
If the result looks wrong, and you can't troubleshoot it yourself, please contact support@displayr.com, and give us as much information as you can. Please see the last section in What To Do When Displayr's Results Are Different Than Another Program's Results for tips on how to give us the information that we need to track such things down.