Caveats and Limitations for Rules

Introduction

This article describes the caveats and limitations involved when applying Rules to tables.

Requirements

A table with a Rule applied.

Method

Performance

Rules are generally applied via looping in JavaScript, and some are designed for applying to a single row or column only. The following should therefore be kept in mind:

• Tables with hundreds of rows and/or columns will take longer to load as the code needs to run through each row and column every time it loads.
• Applying a rule multiple times to a table to perform a specific task on a different row or column (instead of including this within the same loop) will result in inefficient code that could cause performance issues when loading the table.
• Rules that rely on temporary tables being generated within the code may also cause slow-running tables, especially when used on a large table, a large data set or both.

Significance Testing in Rules

Some caution should be applied when customizing a table using Table JavaScript, particularly with regard to significance testing. The p-values that control the significance results that are displayed in a table using arrows, font coloring, font sizes, and column letters are determined before the table JavaScript is applied to the table. As a result, some Table JavaScript functions can have unintended consequences with regards to which cells are shown to be significant, causing cells to be highlighted incorrectly. In these cases, significance should be prevented from being displayed, and this can be done by selecting Properties > Significance > No.

One example is the addition of rows or columns to the table. Since the significance results are computed prior to the addition of these rows or columns, the new cells can have no significance results.

In general, significance results should be turned off when the Table JavaScript involves any function that:

• Changes the number of cells in the table
• Changes the value of a statistic in any of the cells
• Applying filters or weights to some cells and not others

Modifying Significance Tests via Rules

In general, we do not recommend modifying tests. The general experience of our support team is that more often than not, modifications of testing performed by users seems to be likely to reduce the validity of the users' analyses.

Care needs to be undertaken if modifying Displayr's tests using rules. Some particular aspects that need to be addressed are:

• That the test that is to be used is actually better. That is, the automatic tests in Displayr are designed to take the following factors into account, and if over-riding these tests it is important to verify that these issues are still being addressed appropriately:
  • Multiple Comparisons (Post Hoc Testing).
  • Weights.
  • Dependence.
  • Statistical power.
  • Robustness.
  • Edge cases (e.g., variables with zero variance).
  • Numerical precision.
• That users do not inadvertently apply the modifications to the wrong data (e.g., if applying a rule to a specific table, care needs to be taken to avoid inadvertently copying that rule and applying it to other rules).
• Avoiding inconsistencies. For example:
  • If you modify the way that tests are conducted on tables, this will not have any impact on other areas where Displayr conducts testing automatically.
  • Different implications from tests shown on the main section of the table versus those shown in STATISTICS > Right and STATISTICS > Below.

See Also

How to Use Rules in Displayr