As introduced in Automatic Updating, Duplicate & Modify, and Extensibility, this article describes how to refresh your data or replace an existing data set or data source in Displayr with updated data to have all of the outputs in your document update automatically. Keep in mind that the new data set should contain all respondents that you want included in the report, so it may be necessary to Combine Data Files by Adding New Records.
This article is broken into the following sections:
- Update Data Source - use this method if you want to either update the existing data set with a new revised data set or data source.
- Refresh - use this method if you want to manually refresh your data that's connected to a live data source, such as Qualtrics, Forsta, a URL, etc.
- Technical Details
Requirements
- A Displayr document with an imported data file in your Data Sources tree.
Method - Update Data Source
If you want to replace your data set with a new wave of data, for example, you will want to update the data source. Use this option for manually uploaded files (i.e., stored locally).
It's possible to change the import method of the data source, such as replacing a data set imported from your computer with one saved as a URL by selecting URL in the Where is your data? dialog. However, special care should be taken to ensure the new data set is structured the same as the old data set, with the same metadata, in order for Displayr to match and update the variables accordingly.
1. Select the data set you wish to update in the Data Sources tree.
2. From the floating toolbar, click Update , or click Update in the Properties
panel.
3. In the Where is your data screen, select the appropriate option to load your updated file.
You can follow the same steps as when you first imported the data set:
- Data files (.sav, .zsav, .xml, .sss, .mdd, .xls, .xlsx, .csv) stored on your computer or local drive.
- Data files saved in Cloud storage (Displayr Cloud Drive, Google Drive, OneDrive, Dropbox, Box).
- Data files made available as a URL.
Otherwise, for sources below, you will be taken directly to the code/API login interface used to import the data:
- API Integrations (Qualtrics, Forsta (formerly Decipher), SurveyMonkey)
- SQL databases
- Custom R code
Method - Refresh data
If your data is connected to Displayr via a live data source, such as Qualtrics or a URL, you can manually refresh the data set to reflect the most recent data. If you want to refresh the data on a regular basis, such as daily, it's best to set up a Schedule.
- Select the data set you wish to update in the Data Sources tree.
- From the floating toolbar, click Refresh
, or click Refresh in the Properties
panel.
Technical Details
Data Difference Warning
After the data is refreshed or updated, Displayr will compare the new data set with the old version to flag any material changes in the variables that are contained or the structure and values of variables. A Data Difference Warning window will open with these alerts. These alerts include instances in the new file where:
- A variable is missing - you should confirm that this is expected. Variables are matched between files based on the Name of the variable, and if that has changed in the new file, it may show up as a new variable instead.
- A new variable is found - you should confirm these indeed are new variables and not merely variables present in the old file named differently. If so, you should amend the Name in the new data file so it can be mapped correctly and calculations and outputs dependent on that variable can be updated automatically.
- A variable's categories have changed - this includes conflicts in category labels where the same category value has a new category label. See the second example in the picture below where a label changed from 65 or more to 65+.
- A variable's structure has changed - if a variable that was previously numeric in the raw data became text or something else.
- A variable that was calculated has become hard-coded - usually this comes up when you update a data set with a QDat or .sav version that was exported from Displayr. Any calculated variables like custom R variables, JavaScript variables, or filtering/weight tool filters (including combo box filters) will become hardcoded in the export, and will not be recalculated in the report (definitely a problem for combo box/control filters).
- A Filter or Weight is now INVALID - this happens when an input variable to a variable created using the built-in Filtering or Weight tool has changed. This does not affect filters or weights created via R, AI, or JavaScript variables. You will need to open up the Filter/Weight tool for each variable to confirm that the original logic/targets are still applicable, based on how the variable has changed. For example, if you add a brand to your brand list used in a filter, you need to confirm how the new brand should be included in the filtering logic.
Below is an example of the data difference warning that pops up listing information for the Variable Set, individual Variable Name, and History of the change.
You can copy and paste this information in Excel if you'd like to track confirming the changes line by line. If you want to dismiss these warnings from popping up again, press Accept. Otherwise, you can click on Remind Me Later to see this message again. Keep in mind, accepting the warning does not "fix" invalid variables, these must be reviewed and confirmed/amended on a variable-by-variable basis.
Recalculate R-based variables and outputs
Displayr will use the variable names to map the new data file to the data file currently loaded. All steps performed on the previous data set, such as table modifications, relabeling, constructing variables, creating tables, charts, analyses, etc., will be updated with the new data. If there are changes in the new data set, Displayr may not be able to automatically update everything and will alert you to the errors. Also note, if there are any R-based outputs or calculations, they will only automatically update if the Calculate automatically checkbox in Properties is ticked. Otherwise, you will need to click the Calculate button for the output to update with the new data. Any specific IDs (based on the Unique Identifier variable selected) that had manual edits or deletions will be automatically applied to the new data set, so you don't need to worry about losing any manual edits when updating.
Invalid outputs
Any outputs that become invalid due to a data change will be identified in red in the Data Sources and Report tree. If there are discrepancies between the data file used to create the document and the updated data file, it may result in work being lost. For example, if a code frame has been revised, it may “break” any filters created using the variable set with the revised code frame and ruin any merging and renaming of categories in tables using the variable set with the revised code frame. This is by design. That is, while painful, it is better to lose work than to continue working without being aware that the code frame has been revised. Displayr gives warnings when this is about to occur.
Updating pasted/entered data sets
If you have a pasted or manually entered data set, you will only be able to use the Update option. You can also modify the selected cells/range via the Data tab in Properties .
Files stored in the Displayr Cloud Drive
Note: if you are updating or refreshing a data set stored on the Displayr Cloud Drive, you will receive an error if you don't have the appropriate folder permissions or if the file was moved.
Next
How to Import Data into Displayr
How to Combine Data Files by Adding New Records
How to Combine Data Files by Adding New Variables
How to Merge Data Sets with Different Cases/Records via the Displayr Cloud Drive Using R