Common Errors in Weights

When creating a weight it is possible for there to be a mismatch between the targets that have been specified and the actual sample sizes in the data set which prevents the weight from being calculated. This page describes common situations when a mismatch occurs and the steps that can be taken to allow the weight to be calculated.

Problems with weights often occur when the samples in the categories used by the weight are small or empty. Errors that are encountered when constructing a weight can indicate that there are problems with the sample, or that the weight scheme is too complicated. The general approach to solving problems with weighting is to simplify the weighting scheme by either reducing the number of questions that are being used, or by consolidating categories within the input questions, see How to Create a New Variable with Merged/Combined Categories. For information on things to consider when weighting your survey see How to Weight a Survey, and when you are ready to construct your weight in Displayr see How to Configure a Weight from Variable(s).

Common errors covered in this article are:

• Empty categories
• Rim weighting does not converge

Empty categories

You may run a survey that does not have any respondents in a particular target weighting group. In this case, it is impossible to create the weight because there is no respondent to apply the target weight to. 

Diagnosing and Solving the Problem

For example, let's say you attempt to create a weight with the following variable. Notice that the first category has zero cases. 

From the Data Sources Tree, select the '+' icon that appears when hovering over a variable and navigate to Weight > New. From there, define the desired percentages. You will see the following error along with a suggestion about how to correct it. In this case, it is to set the targets for the empty categories to 0.  

Version with error:

Version with weight set to 0:

Another solution is to merge the empty categories with other categories that have samples. Merging is done by dragging-and-dropping categories in the table where weight targets are entered, or using a variable with those categories already merged.

Version with empty categories merged with other categories:

This may also arise when you weight respondents within a wave of a tracker, and one wave doesn't have any respondents in a particular target. In this case, you will need to use a method similar to How To Apply Different Weighting Structures To Sub-Samples.

Rim weighting does not converge

When the weighting involves multiple adjustment variables, Displayr uses an algorithm called Rim Weighting or raking to estimate the weight for each respondent so that each of the different sets of targets is achieved. Because each set of targets is independent of the others, the situation can arise where one set of weight targets contradicts one of the other sets of targets by calculating two different sample sizes for the same group of respondents. For example, say you have one respondent who is the only respondent that falls into the "18-20 years old" and "Less than High School" categories, yet the targets for those categories are 10% and 3%, respectively. Because the respondent is the only one in those categories, the targets would mean they would need to be weighted to make up both 10% and 3% of the weighted sample size, which is logically impossible, as each respondent can only have one weight value.

Diagnosing and Solving the Problem

If you have used more than two adjustment variables, then the first step to solving the problem is to identify which pair of questions is in conflict. It is possible that three or more questions can be in conflict with one another, but most issues will pop up between two. The following process will allow you to identify which adjustment variables are contributing to the conflict:

1. Use the '+' symbol from the Data Sources Tree once again and select Weight > New Weight to create your weight. Include as many adjustment variables in the weight as possible without getting an error in the Diagnostics Report at the lower right.
2. Save the variable by clicking the + New Weight at the lower right.
3. For each remaining adjustment variable:
   1. Select the weight variable in the Data Sources tree, click General > Weight Definition > Edit weight in the object inspector .
   2. Complete the weight, ensuring that all targets are entered for all adjustment variables.
   3. Remove the current adjustment variable.
   4. Check the Diagnostics Report at the bottom right. If an error is still generated, then you know that the adjustment variable that you removed is not the source of the conflict in the weight. If an error is not generated, then the adjustment variable that has been removed is contributing to the conflict.
   5. Repeat these steps as needed to identify the source of the conflict.

The final step is to try to identify the reason why the questions that have been identified above are in conflict with the weight targets. There is no general solution, and the problem can be tricky to identify. Some trial-and-error is required. One approach is to:

1. Create a cross-tab between each pair of questions identified above, and show the Count in the Data > Statistics > Cells.
2. Examine the sample sizes in the tables and try to identify those that are particularly small (e.g., n = 1 or n = 2), as these are most likely to present problems.
3. Create your weight again.
4. In the Edit weight window, merge the category with the small sample size with another category, enter a target for the combined category, and then check the Diagnostics Report. If the same error still occurs, then a different combination of categories is required, and the process should be repeated. The choice of category to merge will depend on both what makes the most sense from the point of view and which combination of categories solves the issue.

Next

How to Change Weighting Using a Control Box