This article describes how to use the root likelihood (RLH) to identify respondents that appear to have randomly chosen options when answering a choice-based conjoint questionnaire. It then describes how to remove these respondents from further analyses.
Requirements
A document containing a choice model.
Method
- Fit a Hierarchical Bayes model. See How to do the Statistical Analysis of Choice-Based Conjoint Data. Preferably, leave out the Alternative attribute by unchecking MODEL > Alternative-specific constants.
- Save the RLH as a variable SAVE VARIABLES > RLH (Root Likelihood). Note that if the Hierarchical Bayes model has Questions left out for cross-validation, this step will create multiple RLH variables. Both of these variables may be used in this exercise, but it is recommended that you turn off cross-validation in this process.
- Examine the distribution of the RLH statistic
- Select the variable created in the previous step.
- Create a histogram (Visualization > Distributions > Histogram) and click and drag to draw the visualization.
- Duplicate the Hierarchical Bayes model in step 1. It is usually best to select the entire page in the Pages tree and press Duplicate.
- Set RESPONDENT DATA > Data Source to Simulate choices from priors and Simulated sample size to the sample size. Provided that no priors have been set (as may have occurred if using an efficient or partial profiles design), this will simulate data under the assumption that people are randomly choosing alternatives.
- Plot the distribution of the RLH for the simulated data:
- Insert a calculation
- Enter the code of
hist(choice.model.2$rlh)
- Replace choice.model.2 with the name of the choice model that contains the simulated choices.
- Create a calculation with code of
random.rlh.cutoff = quantile(choice.model.2$rlh, .95)
replacing choice.model.2 with the name of the choice model and .95 with some other percentile if you desire (0.95 will set the cutoff to the value that 95% of simulated respondents are below). - Create a filter variable as follows:
- Hover over a variable in the Data tree, and press + > Custom Code > R Numeric
- Activate the cursor in the R CODE block, click on the RLH variable and type in > random.rlh.cutoff (e.g., if your model is called choice.model, then the code will appear as:
`RLH from choice.model` > random.rlh.cutoff
- Label: Non-Random Choosers
- Name: nonRandom
- Check Usable as a filter
- Duplicate the initial model, and apply the filter.
See Also
How to Read Displayr's Choice Model Output
How to Remove Irrational Respondents from a Choice-Based Conjoint Model
How to Create an Experimental Design for Conjoint Analysis
How to Set Up a Choice-Based Conjoint Analysis in Qualtrics
How to Preview a Choice Model Questionnaire
How to Compare Discrete Choice Models
How to Create a Choice Model Utilities Plot
How to Save Utilities from a Choice Model
How to Save Class Membership from a Choice Model
How to Create a Choice Model Simulator
How to Create a Choice Model Optimizer