Create a Separate Data Preparation Document

Sometimes documents are slow because of the size of their Data Set. This could be due to the raw data or the number of new variables created in Displayr. Many times performance can improve by simply saving the raw data file from a new document to a more compressed format (a leaner .sav or .QDat) and updating that file to the document with your report.

However, significant improvements can be gained by using a separate document to perform more of the data preparation outside of the document you build your report in, such as:

Removing unneeded variables and cases.
Creating constructed variables that do not need to be updated in real-time in the report, such as top 2 box scores, text categorizations, segmentation membership, etc. Importantly, there are a couple types of variables you would not use this for:
1. Filter variables that depend on user selections in Combo Boxes or other Controls on the page as they need to be recalculated when the control selection changes.
2. Constructed Banners created by our drag-and-drop tool, as they use a dynamic interface and have extra nets, spans, and other structures integrated into them.
Stacking data.
Merging data, such as real-time API data into a historical data file - this is especially handy for tracking studies.

Any changes to the Data Reduction of a variable set that you have changed on a table directly, such as combining categories, creating NETs, removing categories, reordering rows, etc, will not pass through to the saved file.

The general setup for this workflow can be grouped into two higher-level steps:

Method - Creating the data preparation document
Method - Updating Your Report Document

Requirements

A data set of raw observation data.
If setting any data up to export automatically, you'll need Server time for private dashboards and API hours (view hours) on your account.

Method - Creating the data preparation document

The data preparation document is separate from the document containing the final report as mentioned above. In it you can do data cleaning, configuring, merging, and stacking for your final report. The purpose of the data preparation document is to export a more efficient data file to the Displayr Cloud Drive.

Decide if you would like to start with the raw data and edit from there, or take the Data Set you've created (with constructed variables) and whittle down your report to what you want to include in the data preparation.
- Starting from the raw data - create a New Document from your Documents page, append something like Data Preparation to the report name, and import the raw data used for your report.
- Starting from your current data set - Duplicate the document from your Documents page (via the ellipses button when hovering over the document's name), appending something like Data Preparation to the name.
In the data preparation document, edit the data to add in constructed variables that you'd like hardcoded in the report, hide/delete unneeded variables/cases, and delete any banner or variables that need to be recalculated in real-time in the report (if working from a duplicated document).
Save to your cloud drive for the most efficient workflow. Click Share > Export Data > To Displayr Cloud Drive as Q Data File (.QDat) (the leanest format type as it was created specifically for Q/Displayr) or you can choose To Displayr Cloud Drive as SPSS File (.SAV) if you are using it in the merging or stacking widget later in the workflow.
On the next screen, select an option below then click OK.
- If you'd like to set up automatic exporting, select Now and whenever document is published.
- To export manually whenever you want, select Just now.
Give the exported file a name and click OK.
[OPTIONAL]: Set up the automatic updating of this data set by scheduling the document to be republished on a schedule (it doesn't matter that it's not been published to a dashboard yet). See How to Automatically Republish a Dashboard for how to set up the Scheduler to do this.
[OPTIONAL]: Perform any merging or stacking that you require using the exported file from your Displayr cloud drive. If desired, use the Automatic updating sections in their object inspectors to set up a schedule for exporting the merged/stacked data to the cloud (note this will happen whether or not the input data sets have been changed).

Method - Updating your report document

Now that you've used the data prep document to do the heavy data lifting, you will end up with a new data file that was exported to the Displayr Cloud drive. You will then need to update your final report document with this updated data set from the cloud.

In the document used for the final reporting, select your existing data set in Data Sources.
Press Update in the object inspector to update it with a new version.
Click Displayr Cloud Drive and select the appropriate file from your cloud drive that you created in your data preparation document.
Review any warnings or errors that pop up in the document.
Repeat steps 1-4 for any other data sets you'd like to update with a file from your data preparation document.
OPTIONAL: Once, you've linked the data set in your final report document to the appropriate file on the Displayr Cloud Drive, you can also set up automatic updates for this using a Schedule, if desired; see How to Set Up Schedules in Displayr. Keep in mind that if you are also automatically updating and exporting from your data preparation document, you will need to allow enough time for that to complete before scheduling your final report to update.