This article describes how to determine why the Sample Size, Column/Row Sample Size, or NET row/column on a table is smaller than expected. The NET can be the overall NET (like in the table below), a custom or span sub-net, or a combined row/column (which is the same as a NET but without also keeping the categories separate). Such as the following table showing a Top 2 Box score across different brands:
Requirements
A table with one of the following that is smaller than expected:
- A combined category (created using Combine on rows/columns of a table)
- A NET (created using Create NET on a table or simply the an overall NET or sub-net on a table)
- A sample size statistic (such as Sample Size, Column Sample Size, or Row Sample Size)
Knowledge of how NETs and Sample Sizes are computed with missing data. NETs and Sample Sizes are calculated, including only respondents with data across all the included categories. Any respondents with Missing data are excluded from the NET and Sample Size. It is obvious when there is missing data on a table, when the footnote lists a range, such as "sample size from 591 to 600". To understand the numbers in the table above, adding the Sample Size statistics or Missing Count to the table (as seen below) can help identify the cause of lower-than-expected numbers.
In the above table, nine respondents didn't provide an answer for Pepsi (have missing data). Thus 591 respondents have provided an answer across all categories resulting in a Sample Size of 591.
The NET on the table is 96% (567/591). The Count of 567 tells us that out of the 591 respondents who provided an answer for all brands, 567 (96%) selected the Top 2 Box score for at least one of the brands. The Column Sample Size is the number of people who selected at least one category in the column (in this instance, the Top 2 Box score), Column Sample Size is the same as the NET of the table at 567.
Increase Sample Size
Method - Include Missing Data
If you believe the missing data is in error and want to include the nine respondents with missing data for Pepsi, you can click on the variable and select Missing Values in the Object Inspector. This will open the Value Attributes menu where you can include Missing data in the Sample size, by selecting Missing Values > Include in analyses similar to below:
However, often this will an unwanted side-effect, whereby the percentages on the table will change to include the larger of the sample sizes. For example, with the data shown above, Pepsi's value of 45.2% is compute as 267 divided by 591, and if its Sample Size is changed to 600, the percentage drops to 44.5%.
Method - Show the Maximum Sample Size
Displayr's approach is based on the principle of conservativism - it minimizes the chance that a research user will rely upon an unreliable finding. Any approach which puts a higher sample size than what is used by Displayr runs the risk that users of the research will assume the data to be more robust than is the case. Given this, there is a custom rule you can use; see How to Show Maximum Column Sample Size in a Table to show the maximum Column Sample Size at the bottom of the table. For tables where this is not applicable, you can modify the JavaScript code of the rule to use Sample Size instead; see How to Customize a Rule.
Method - Revise Weights
If your table is weighted, you should reference Weighted statistics to see the sample sizes used in the figures and testing on the table. If these numbers appear incorrect, review your weight variable's targets and construction to determine if it needs to be revised. Do note that if a respondent is given a 0 weight, they will also be removed from the unweighted n statistics.
Method - Revise Data
If you have investigated the above options, and you still believe your sample size is too small. You can right-click on the variables in question in the Data Sets tree and select View in Data Editor. Then right-click on the header of the raw data, select Copy and paste the data into Excel to manually review it. If the sample sizes in Excel match that of Displayr, you must go back to your data provider to investigate.
Make NET = 100%
Method - Rebase to the NET
Rebasing the table to the NET will force the NET to be 100%; see How to Rebase Questions. After running the automation, Displayr will create a new version of the question in which respondents not included in the NET will have missing data.
Method - Create a NET Filter
1. Go to Anything > Filter > New.
2. Create a filter of all the positive responses in the variable set:
3. You can then apply the filter to this table to show the NET as 100%.
Method - Add "None of these" option
When the categories do not add up to 100%, this creates a new category for cases which are not part of any of the existing categories. This will result in the categories adding up to 100%.
Note that this method requires a Binary - Multi variable or a table constructed from a Binary - Multi variable.
- Select a binary-multi variable or an existing table that was constructed using a binary-multi variable.
- Go to Anything > Data > Variables > Modify > Add Categories > Add None of These.
- The variable or output will automatically update with a "None of these" option added.
See also
How to Turn Off Missing Data Selection for Specific Values
How to Show Maximum Column Sample Size in a Table
How to Rebase Multiple Response Data in Variable(s) to NET
How to Add a None of These Option