UPCOMING WEBINAR: From Thousands of Tables To One Clear Story: using AI in survey analysis While analyzing your data for insights, it's one thing for a value to be higher than another, but it's a much more powerful statement if a value is significantly higher/lower than another. In Displayr, statistical testing is built into summary tables and crosstabs systematically - pulling out these insights for you. Based on your data and testing assumptions, Displayr automatically chooses the best test and shows all the significant results in your table. Testing is always performed on the %, Column %, or Average statistic (depending on the structure of the data), regardless of the statistics shown in the table.
This article describes how to go from a standard table in Displayr showing no significant differences:
To a table with statistical testing applied. The example below shows arrows and font colors to represent significant differences for each cell against its complement:
Requirements
- A document containing a standard built-in Displayr table. The default built-in significance testing system cannot be run on tables that you create in R or pasted tables.
- Understanding of the two types of testing done in Displayr, see our Introduction to Significance Testing article in The Data Story Guide. There are detailed examples of how exception testing and column comparison testing work, as well.
Method
1. Determine the way you want to run significance testing, select your table(s), and choose the appropriate way in the Properties > Appearance > Appearance > Significance.
|
Using Arrows, Font colors, or Arrows and Font colors Run Exception Testing (Complement testing) |
Compare columns Run column comparisons on individual column values, also known as pairwise testing |
| The value in a cell is tested against the value of respondents not in that column, or its exception. | The value in a cell is tested against the individual values in each column. |
2. [OPTIONAL]: Select the Advanced option if you want to review or modify the formatting or assumptions for the significance testing. See Advanced Statistical Testing Assumptions section below for more details.
3. [OPTIONAL]: If you want to use this testing as the default testing across tables, see How to Set the Default Type of Significance Test. Otherwise, these settings will only apply to this and any other tables or visualizations you have selected before changing the Significance dropdown.
Advanced Statistical Testing Assumptions
Displayr allows you to adjust various aspects of significance testing, including significance levels, test types, multiple comparisons correction, and more. To see the full list of options, see Step 3 below. To access these settings:
- Select the table(s) you want to change significance testing assumptions on (you only need to select one if you want to change the default settings on all tables using default testing).
- In Properties
, select Appearance > Appearance > Significance > Advanced.
You will see the Advanced Statistical Testing Assumptions dialog: - Select from one of the four tabs at the top depending on the type of task you want to perform:
Significance Levels:
- Show significance: show higher or lower significance with arrows, font colors (tables only), or using symbols to show significant differences between columns
- Overall significance level: is used when determining which results to show as being statistically significant
- Minimal sample size for testing: where cells have sample sizes of less than the minimal sample size for testing, no significance test is conducted when conducting automated tests of statistical significance between cells (i.e., Cell Comparisons and Column Comparisons).
- Weights and significance: determines how Displayr deals with weights when computing significance. See Design Effects for more details about each setting.
- Extra deff: by default, the deff value (design effect) is set to 1.00
- Significance levels and appearance: symbols used to denote different levels of statistical significance.
Test Type:
- Proportions: non-parametric tests will be done on categorical data
- Means: t-test will be done on numeric data and corrected with Bessel’s correction
- Equal variance in tests when sample size is less than: if the sample size is less than 10, variance is assumed equal.
- Correlations: default is Pearson
- Date/Time/Wave: When using data from Date/Time or nominal or ordinal variables tagged as Treat as wave in statistical tests variables, you can specify whether significance tests compare to the date in the previous period (Compare to previous period), or to the rest of the data (Compare to rest of data).
Exceptions Test:
- Multiple comparison correction: No correction is applied by default on exception tests. A check box is available if instead, you prefer to apply the correction within each span within each row. See How to Apply Multiple Comparison Correction to Statistical Significance Testing for more details.
- Significance symbol: choose the symbol you want to show significance, either Arrow or Triangle. The default is Arrow.
Column Comparisons:
- Multiple Comparison correction: No correction is applied by default on column comparisons. The following corrections are available for post hoc testing: Fisher LSD, Duncan, Newman Keuls (S-N-K), Tukey HSD, False Discovery Rate (FDR), False Discovery Rate (pooled t-test), Bonferroni, Bonferroni (pooled t-test), Dunnett. See How to Apply Multiple Comparison Correction to Statistical Significance Testing for more details.
- Overlaps: The default is to ignore the sample that overlaps between columns when respondents in columns are not mutually exclusive
- No test symbol: - is shown if a test isn’t performed due to settings
- Symbol for non-significant test: nothing is shown if a test comes back insignificant
- ANOVA-Type Test: Select whether ANOVA-type tests are run as part of the testing
- Show redundant tests: Select whether to show significance on one cell (the one with the higher value) or all cells involved in testing.
- Show as groups: Show letters for insignificant columns rather than significant
- Recycle column letters: Each span begins labeling columns at A
- Maximum columns to compare: sets an upper limit for the number of columns within a span (or, if no spans, the table) that will be tested when conducting column comparisons. If your span or table has more than 26 columns, increase this limit accordingly.
Once you've made your changes, click Apply to Selection to apply the changes to the selected tables or visualizations. You can also click Set as Default to change the document's default settings.
Restore to Document Default Settings
To revert to the document's current default settings, click the Restore button
Note: The Restore button will only be enabled when settings for the selected item(s) are different from the document's current default settings (settings saved by Set as Default). The Restore button will be disabled if you do not save any settings for the selected item(s). See: How to Set the Default Type of Significance Test to learn more about changing the default settings.
See Detail from Statistical Testing
If you'd like more information on how these settings are used in the testing on your table, you can investigate the specific statistical test, see intermediary calculations, and get a description of what is being tested by using the Statistical Test (alpha ) feature, see How to See Statistical Testing Detail using a Table.
Rules to Customize Significance Testing
It's possible to use built-in or custom rules to customize significance testing in tables. See Customize How Significance Tests are Shown on Tables for options.
Next
How to Set the Default Type of Significance Test
How to Compare Significant Differences Between Columns
How to Conduct Significance Tests by Comparing to Previous Time Periods
How to Change the Confidence Level at which Significant Differences are Displayed
How to Apply Multiple Comparison Correction to Statistical Significance Testing
How to Override Displayr's Default Statistical Testing Settings