Minimize the Size and Distance of Data Being Moved

Whenever Displayr creates an output (e.g., a calculation or a visualization) we can think of the time taken as having three components:

1. The time taken to create the output. For example, if computing an average, the time taken to calculate the average.
2. The amount of data that needs to be used in the calculation. For example, if we are calculating the Average of 1,000 cases of data, then we have 1,000 values. This data all needs to be "moved" to where the calculations are performed.
3. The distance that the data needs to be moved. Broadly speaking, we can think of this as being:
   • Movements involving variables and tables. E.g., when a variable set is dragged onto a page, causing a variable to be created. These movements essentially take 0 seconds and can be ignored.
   • Anything that involves R calculations, including R variables. These require the data to be moved to an R server. This takes a very small amount of time if the size of the data is small, but it's not 0.
   • To and from the Displayr Cloud Drive. This is typically longer than an R calculation.
   • To or from an external source (e.g., a SQL query of your company's database). This is the longest distance, so takes the most time.
   • The distance from Displayr's computer to your browser. This is typically the longest distance.

The rest of this article contains a simple worked example and a more realistic worked example.

Simple worked example

Consider the following calculation that creates 10 million random numbers:

random.numbers = runif(10000000)

And, consider a second calculation that computes the average:

Average(random.numbers)

At the time this article was written, the dependency graph for this was as follows, suggesting a best case of 3.75 + 2.23 = 5.98 seconds to calculate:

If we look at the raw R output, we can see that of the 3.75 seconds taken to generate the random numbers, only 0.36 seconds were required to generate the random numbers (i.e., Time executing code) and the rest of the time was transferring data (0.80 seconds) and "overhead" (2.59 seconds).

An alternative way of structuring the same calculations is to have the first calculation instead just return the sum of the random numbers and the number of random numbers:

random.numbers = runif(10000000)
results = list(numerator = Sum(random.numbers),
                               denominator = length(random.numbers))

We can then compute the result using:

results$numerator / results$denominator

More realistic worked example

The screenshot below is from a simplified version of the campground dashboard. It shows 18 visualizations.

The Cooking visualization showing as 14%, for example, is a number visualization, and its value is derived from two variables:

• A variable measuring satisfaction (Q4d), has (due to some dodgy file preparation) two categories:
  • Very dissatisfied+Somewhat dissatisfied+Neither satisfied nor dissatisfied
  • Satisfied
• A dynamic filter variable, called `Campsight filter`, linked to a combo box.

An efficient way of creating the visualizations is to create one (Visualization > Number > Number in Oval), set Data source to Use an existing R output, and enter the following code, which calculates the percentage of the filtered data that are satisfied:

Average(Q5d[`Campsite Filter` == 1] == "Satisfied")