How to De-duplicate Raw Data Using R

This article describes how to go from a raw data table...

...to a table that shows only the unique responses:

Requirements

You will need a raw data table with multiple columns created via custom code or Table > Raw Data > Variables from the toolbar. See How to Export Specific Raw Data Variables to Excel for details on creating the latter.

In this example, our table is called cola.raw.data. To perform this on your Data Set, see How to Remove Duplicate Cases From a Data Set.

Method

1. In the Report tree, click + > Calculation > Custom Code, or from the toolbar, select Calculation > Custom Code.

2. Click on the page to place the output.

3. In Properties > General > General, update the Name.

4. Paste the below in the Code panel:

df = unique(cola.raw.data)

Note, when there are multiple columns of data, the unique function will look at the unique combination of all columns.

To instead filter the table on unique categories from the first column, you would use:

df = unique(cola.raw.data[,1])

5. An alternative is to use the duplicated function, which offers more flexibility. The following is the equivalent of the previous code:

x = cola.raw.data
df = x[!duplicated(x[,1]),]

By default, in both situations, the code de-duplicates by displaying the first unique row when there are duplicates. However, by appending the fromLast argument, we can display the last row instead:

df = x[!duplicated(x[,1], fromLast=T),]

Note that using the code below will only result in a TRUE or FALSE based on whether the condition is met.

!duplicated(x[,1])

6. Click Calculate.

Next

How to Remove Duplicate Cases From a Data Set

How to Use R in Displayr

How to Export Specific Raw Data Variables to Excel