Tracking Study Best Practices

Tracking studies, trackers, brand studies, longitudinal studies, or whatever your favorite term is, provide important insights into how consumer preferences change over time, but they require special thought into the best way to conduct and analyze the data. Displayr drastically reduces the time it takes to update and review tracking studies. Incorporating these best practices will ensure you take full advantage of Displayr's robust updating capabilities so you spend less time updating your report and more time crafting your story. This article contains info on:

• Data Files
• Setting Things Up to Update for New Waves
  • Dates
  • Weighting within each wave/date
  • Report and dashboard design
• Updating with New Waves and Fixing Errors
  • Question changes
  • Updating Tables, Visuals, and Other Items on Pages
  • Weights
  • Text Categorization
  • Exporting to PowerPoint
  • Updating Published Dashboards
• Speed
• Archiving

Requirements

Please note these steps require a Displayr license.

Data Files

A well-designed and organized data file is important in any Displayr document, but it will save a lot of time and prevent future headaches when analyzing brand tracking research. You will want to create your report from a single combined dataset that includes all dates/waves that need to be analyzed. In general, the first wave of a project informs how the subsequent waves should be structured. The fewer structural differences in later waves, the easier it will be to merge the data in subsequent waves.

General data file best practices include:

• Never change a variable name or code value from wave to wave. For example, if Sydney is code 3 in wave 1, it should be code 3 in wave 32. Or if your location question is Q1 in the first wave, then that should stay the same, too.
• If you add new codes, such as adding brands, then use new code values. Don’t reuse the old ones!
• Similarly, if you remove a code or question, like removing a brand, then leave the variables or code in your data file, even if it won’t contain any data.

Other best data file setup best practices can be found here: How to Set up a Data File for Tracker Studies and How to Set Up Your SPSS File for Importing into Displayr. Once you've created a combined data file with all waves, you can then update your document to have all of the calculated variables and outputs update with the new data automatically, see How to Update a Data Set in Displayr.

Setting Things Up to Update for New Waves

Dates

You will want some sort of date variable to use in the Columns of tables to see changes in variable sets over time. By default, date/time variables are tested against the previous time period, which is what you likely want to use in a tracking study. See How to Conduct Significance Tests by Comparing to Previous Time Periods for more information. If needed, you can update the settings so that all dates are tested against one another when using stat testing with a date/time variable in the columns.

Other times, you will want to compare specific time periods or waves to one another or show a smaller window of time instead of the full history. To make sure these time periods are shifted when the data is updated, don't use verbatim dates in variables/calculations that you construct. Instead, use one of the following methods where appropriate:

1. Create a new Date/Time variable to use in tables and analyses for the time period you like, using the Aggregation and Duration settings as outlined here: Date/Time Variable Settings. For instance, the snapshot of the setting below will create a variable for the 2 most recent months.
   
2. If significance testing isn't needed, use visualizations to show a specific number of rows/columns. When dates are added to a table, the visualization updates to the most recent information. This is described in Methods 6 and 7 in How to Filter Rows and Columns in Visualizations and Tables Without Code.
3. If you need to filter rather than show dates in the table/visualization, you can create a filter based on the time period needed, per How to Build Tables that Automatically Filter to the Latest Periods and How to Dynamically Filter Data to the Latest Waves.
4. In R and JavaScript code, do not use hard-coded dates, but rather create variables to calculate the dates you need based on the data. See examples #3 and #4 in How to Work with Date Ranges Using R for more details.

If you don't have a Date/Time variable to use in your report, you can use a categorical wave variable instead to do reporting. Special care needs to be taken if wanting to test one wave against the previous wave, see How to Test Against the Previous Period without a Date/Time Variable.

Weighting within each wave/date

You can construct weights to be created for respondents within each wave by using the Recompute weights for dropdown as explained in How to Configure a Weight from Variable(s). You can also create weights with a different configuration within each wave by following the instructions in How to Apply Different Weighting Structures to Sub-Samples.

Report and dashboard design

Many other aspects of your report/dashboard can be automated to update based on new data.

• Controls used for filtering can be created using Items from rather than hard-coded values as described in the Method - manual section of How to Create a Combo Box Filter. You can also create dynamic controls as outlined in How to Create a Combo Box (Drop-Down Control) With a Dynamic List.
• You can set up text on the page to update based on your data. Examples are listed under the Customize and Create Auto-Updating Text Outputs help section, which include:
  • How to Automatically Update Commentary
  • How to Add Color-Changing Messages to Your Page When a Sample Size Threshold is Not Met
  • How to Customize the Sample Size Description Widget Using R Code
  • How to Create a Dynamic Bullet-Point List in Displayr Using R
• You can also set up images to automatically update based on your data. A common request is to add up or down arrows to show differences in data vs the previous wave. You can achieve this by adding annotations to visualizations or, if more complex, by following How to Use R Code to Create Icons.

Updating with New Waves and Fixing Errors

When working on tracking projects, you should maintain a single master data file that stores all of your data. This means all the different waves and any other external data need to be merged into this single file. If you’re collecting this data on the same platform, you may be able to get a raw data file directly from the data collection software rather than merging after the fact. To merge data files in Displayr, please follow How to Combine Data Files by Adding New Records, keeping in mind: 

In cases where you're not sure about the error, it's always safer to separate the variables. 

Once you have your master data file, follow the steps in How to Update a Data Set in Displayr. After the update process is complete, you may be presented with a Data Difference Warning box outlining any variables that are new, missing, or were significantly changed in the updated data set. You should review this list and ensure these differences are as you expect. 

Question changes

There certainly will be changes to questions in a tracker over time, but luckily, most of these changes can be handled within Displayr or by following some of the defensive programming tips in How to Set up a Data File for Tracker Studies. 

New codes in a Nominal variable set

New codes are almost entirely handled by Displayr. If there is an issue Displayr can't handle, you will be notified in the merge tool (if merging in Displayr) or in the Data Difference warning window after updating your Displayr Document with your new updated data set.

Removed codes in a Nominal variable set

If a code is deleted in the questionnaire (e.g., a brand has left the market and is not asked in the questionnaire), and there is a need to continue to report on the brand, then the code should still be included in the exported data file.

Changed coding in multiple response questions

When multiple response questions are changed, such as adding a new brand, this can cause considerable complexity, particularly if the question is then used to construct filters and other questions. In general, the best approach to dealing with this is to anticipate this possibility when setting up the data collection program (see How to Set up a Data File for Tracker Studies).

Scenario 1 - Where there are two different versions of a multiple-response question:

You can combine them in the Data Sources tree by highlighting the variable sets, right-clicking, and pressing Combine > As Variable. Variables that have the same label in both sets are merged, and unmatched variables are added.

Scenario 2 - Where the response options from the previous wave have already updated correctly, but there are new response options (shown as individual variables) that are not associated with the variable set:

You can add the new response option to the variable set in the Data Sources tree by highlighting the variables and variable sets, right-clicking, and pressing Combine > As Variable Set. Variables that have the same label in both sets are merged, and unmatched variables are added.

Scenario 3 - A new brand is added to a multiple response question, and there are Simple or Complicated Filters that rely on the existing variable set:
These filters rely on the structure of the question, and as a result, you will not be allowed to use Combine. Note that this problem does not occur for JavaScript variables. One way to deal with this problem is to Duplicate the variable set before making changes. The Filter can then be modified to refer to the new copy of the question. 

Scenario 4 - A new option/variable is added to a Binary-Multi variable set used in a Banner:

Because Banners create their own variables/variable sets, you will need to remove and re-add your updated Binary-Multi variable set to your Banner. If there is a new response to a Nominal structured question used in a Banner, this is an addition to the code (not an added variable) and will be updated in the Banner automatically. 

Scenario 5 - A new option/brand is added to a variable set used to compute a NET or other calculation:

For NETs, you can create a NET of the current NET with the new row/brand to include it in tables. Other NETs and/or calculations that need to take the new brand into account should be amended as well. 

Updating Tables, Visuals, and Other Items on Pages

The real magic of Displayr happens after you update your data set. Everything that you've created in your report, including tables, visuals, constructed variables, etc., will be updated with the new data. As a part of this process, Displayr may flag items in your Data Sources tree or Pages tree with warnings () or errors (). You should review these and confirm whether the change is expected or if it signals something that needs to be amended in your data or code.

Sometimes you may wish to compare a previous analysis with the analysis using updated data. In this case, you can uncheck the Calculate automatically checkbox in the object inspector. This is available for most visualizations and analyses, as well as any custom Calculation output(s) on the page. You can even use the model from a previous analysis to assign clusters/segments of new data.

Weights

Weights will be automatically updated and created for the new wave of data. Keep in mind, if you need to change how your weight is constructed in later waves or if you're missing a specific quota for a wave, you may need to follow the process for How to Apply Different Weighting Structures to Sub-Samples.

Text Categorization

Categorization variables that you created by using the text categorization tool will automatically categorize responses in the new data that are exact matches to the text used to categorize the old data. To code responses that did not automatically match, you can open the text categorization tool for the categorized variable and use it to categorize the remaining responses, or you can use your previous categorization frame as an input into an Automatic Text Categorization analysis, as in How to Automatically Classify Text Data Using an Existing Categorization to Train a Model.

Exporting to PowerPoint

When you are ready, you can export your Document to a new PowerPoint file or the file from a previous wave to update the outputs. If you've not exported to the PowerPoint file before, you will need to follow the steps on How to Retrospectively Link PowerPoint Tables and Charts to Displayr Outputs, then use How to Update an Existing PowerPoint Document.

Updating Published Dashboards

Similarly, once you're ready, if you've previously published your Displayr document as a dashboard, you'll need to republish the document once the updates are ready. If you've never published the document before, follow these steps: How to Publish a Document as a Web Page (Dashboard). If you've already published the document before, just go to Share > Republish to make the changes live. 

Speed

For tracking studies with many variables or for tracking studies that have lasted a long time, you may begin to experience issues with speed in your document. In this case, you should review How to Optimize Speed of Displayr Documents. As the study evolves and more waves are added, you should consider whether the same analysis and tables are still needed or if you can get rid of old questions or be more selective in what time periods are shown. Many users, with large tracking projects, end up with hundreds or even thousands of temporary variables they use to "check something out". Be sure to delete these! Or make a copy of your dashboard where you can mess around with things like this.

Also, be aware that anytime you update your Data Set or republish your Dashboard, every single calculation in all of the variables, analyses, and visualizations constructed will need to be recalculated. Don't be surprised if it takes longer to reload the Document after this, but subsequent changes to the Document or Dashboard should go quicker.

Archiving

You may want to save a final version of your Document for each wave for auditing purposes. You can do this by downloading the QPack of your Document from Undo > Version History in the toolbar. This can be loaded into a new Document in Displayr or opened in our sister software Q. More specifics are outlined in our How to Manage Documents in Displayr.