How to Investigate a Sample Size or NET that is Too Small

Given how a multi-response variable set is structured, sometimes the NET in the table may not be 100%, or the Sample Size can show as 0 (or some other relatively low number). This article describes how to determine why the Sample Size, Column/Row Sample Size, or NET row/column on a table is smaller than expected. The NET can be the overall NET (as in the table below), a custom or span subnet, or a combined row/column (which is the same as a NET but without keeping the categories separate). Below is an example of a table showing a Top 2 Box score across different brands where the NET is less than 100%, and the Column Sample Size and Sample Size are less than the Total Sample Size of 600:

Table - net too small.png

Requirements

A table with one of the following that is smaller than expected:
- A combined category (created using Combine on rows/columns of a table)
- A NET (created using Create NET on a table or simply an overall NET or sub-net on a table)
- A sample size statistic (such as Sample Size, Column Sample Size, or Row Sample Size)
Knowledge of how NETs and Sample Sizes are computed with missing data. NETs and Sample Sizes are calculated, excluding respondents with missing data across all included categories. Any respondents with Missing data are excluded from the NET and the sample size. It is obvious when there is missing data in a table and the footnote lists a range, such as "sample size from 591 to 600". To understand the numbers in the table above, adding the Sample Size statistics or Missing Count to the table (as seen below) can help identify the cause of lower-than-expected numbers.

Table - net too small 2.png

In the table above, you'll see that the Missing Count for Pepsi is 9. So nine respondents didn't provide an answer for the Pepsi question (and have missing data). Thus, 591 respondents have provided an answer across all categories, resulting in a Sample Size of 591.
The NET on the table is 96% (567/591). The Count of 567 tells us that out of the 591 respondents who provided an answer for all brands (the Sample Size), 567 (96%) selected the Top 2 Box score for at least one of the brands.
In a summary table like the one above, with just one variable set analyzed, the Column Sample Size is the number of people who selected at least one category in the column (in this instance, the Top 2 Box score). Column Sample Size is the same as the NET of the table at 567. For crosstab tables, the Column Sample Size is the number of people who selected the category in the column and have non-missing data in all rows included.

Given the explanation above and what you'd like to show in your table, there are different options for how to resolve this issue and get you what you need:

Increasing Sample Size:
Making NET 100%

Increase Sample Size

Method - Include Missing Data

If you believe the missing data is in error and want to include the nine respondents with missing data for Pepsi, you can click on the variable and select Values & Labels, Values, or Categories, depending on the variable's structure, in Properties > Data > Attributes. This will open the Value Attributes menu, where you can include Missing data in the Sample size by selecting Missing Values > Include in analyses, similar to below:
Missing Values - Top 2 Box.png

However, often this will be an unwanted side-effect, whereby the percentages on the table will change to include the larger of the sample sizes. For example, with the data shown above, Pepsi's value of 45.2% is computed as 267 divided by 591, and if its Sample Size is changed to 600, the percentage drops to 44.5%.

Method - Show the Maximum Sample Size

Displayr's approach is based on the principle of conservativism - it minimizes the chance that a research user will rely upon an unreliable finding. Any approach that puts a higher sample size than what is used by Displayr runs the risk that users of the research will assume the data to be more robust than is the case. Given this, there is a custom rule you can use; see How to Show Maximum Column Sample Size in a Table to show the maximum Column Sample Size at the bottom of the table. For tables where this is not applicable, you can modify the JavaScript code of the rule to use Sample Size instead; see How to Customize a Rule.

Table maximum sample size.png

Method - Revise Weights

If your table is weighted, you should reference Weighted statistics to see the sample sizes used in the figures and testing on the table. If these numbers appear incorrect, review your weight variable's targets and construction to determine if it needs to be revised. Do note that if a respondent is given a 0 weight, they will also be removed from the unweighted n statistics.

Method - Revise Data

If you have investigated the above options, and you still believe your sample size is too small. You can select the variable from the Data Sources and select Raw Data from the Data tab in Properties. Then right-click on the header of the variable in the Raw Data panel, select Copy, and paste the data into Excel to manually review it. If the sample sizes in Excel match those of Displayr, you must go back to your data provider to investigate.

Make NET = 100%

Method - Rebase to the NET

Rebasing the table to the NET will force the NET to be 100%; see How to Rebase Questions. After running the automation, Displayr will create a new version of the question in which respondents not included in the NET will have missing data.
Rebase to net.png

Method - Create a NET Filter

1. In the Data Sources pane, hover over a variable where you want the new filter variable to go, and click + > Filter > New.

2. Create a filter of all the positive responses in the variable set:
Brand attitude filter.png

3. You can then apply the filter to this table to show the NET as 100%.

Method - Add "None of these" option

When the categories do not add up to 100%, this creates a new category for cases which are not part of any of the existing categories. This will result in the categories adding up to 100%.

Note that this method requires a Binary - Multi variable or a table constructed from a Binary - Multi variable.

Select a binary-multi variable or an existing table that was constructed using a binary-multi variable.
From the toolbar, go to Anything > Data > Variables > Modify > Add Categories > Add None of These.
The variable or output will automatically update with a "None of these" option added.