How to Investigate a Sample Size or NET that is Too Small

This article describes how to determine why the Sample Size, Column/Row Sample Size, or NET on a table is smaller than expected. Such as the following table showing a Top 2 Box score across different brands:

Requirements

A table with a NET or Sample Size statistic that is smaller than expected.

Knowledge of how NETs and Sample Sizes are computed with missing data. NETs and Sample Sizes are calculated including only common respondents across all the categories being included. To understand the numbers in the table above, adding sample size statistics or Missing Count to the table (as seen below) will help you identify what is causing the small number.

In the above table, 9 respondents have missing data for the Pepsi category, thus there are 591 respondents who have data across all categories making the Sample Size of the table 591. The NET on the table is 96% (567/591) - meaning out of the 591 respondents that answered all categories, 96% of them gave a Top 2 Box score for at least one of the categories. As Column Sample Size is the number of people who selected the category in the column (in this instance it is a Top 2 Box score), Column Sample Size is the same as the NET of the table at 567.

Increase Sample Size

Method - Include Missing Data

If you believe the missing data is in error and want to include those 9 respondents with missing data for Pepsi in the numbers, click on the variable and select Missing Values in the Object Inspector to change the Value Attributes. You will need to include Missing data on all values to include all respondents in the base, by selecting Missing Values > Include in analyses similar to below:

Method - Show the Maximum Sample Size

Displayr's approach is based on the principle of conservativism - it minimizes the chance that a user of the research will rely upon an unreliable finding. Any approach which puts a higher sample size than what is used by Displayr runs the risk that users of the research will believe the data to be more robust than is the case. Given this, there is a custom rule you can use, see How to Show Maximum Column Sample Size in a Table to show the maximum Column Sample Size at the bottom of the table. For tables where this is not applicable, you can modify the JavaScript code of the rule to use Sample Size instead, see How to Customize a Rule.

Method - Revise Weights

If your table is being weighted, you should reference Weighted statistics to see the sample sizes used in the figures and testing on the table. If these numbers appear to be incorrect, review the targets and construction of your weight variable to determine if it needs to be revised. Do note, that if a respondent is given a 0 weight, they will also be removed from the unweighted n statistics as well.

Method - Revise Data

If you have investigated the above options, and you still believe your sample size is too small. You can right click on the variables in question on the Data Sets tab and View in Data Editor. Then right click on the header of the raw data and select Copy to manually review the raw data in Excel. If the sample sizes in Excel match that of Displayr, you will need to go back to your data provider to investigate.

Make NET = 100%

Method - Rebase to the NET

Rebasing the table to the NET will force the NET to be 100%, see How to Rebase Questions. After running the automation, a new version of the question will be created where respondents not included in the NET will have missing data.