How to Import Data from Twitter

This article describes how to import data from Twitter into Displayr using R. You can see more examples of what the R functions can do if you google the twitteR R package. It is recommended to test your modified R code from below first in a custom calculation on the page (Calculation > Custom Code) in case it requires troubleshooting, see How to Troubleshoot R Code in Displayr.

Requirements

A Displayr document.
A Twitter Developer account with the tokens and keys below from your Developer Platform via Apps > Details > Keys and tokens
- a valid consumer API key and consumer API secret
- access token and access secret

Please note this requires the Data Stories module or a Displayr license.

Method - Search Twitter

The following steps create an R Data Set with, the last 100 tweets with a mention of 'Tesla'.

1. In the Data Sources pane, click the Plus (+) button at the top and click on R.

2. Enter a name for the data set under Name.

3. Paste the below R code in the editor box and modify to your example:

#load libraries with R functions needed
library("twitteR")
library("ROAuth")

# Provide Authentication information
consumer_key = 'your_consumer_key'
consumer_secret = 'your_consumer_secret'
access_token = 'your_access_token'
access_secret = 'your_access_secret'

#authenticate the connection
setup_twitter_oauth(consumer_key, consumer_secret, access_token, access_secret)

#Add in Query information
search.string = 'Tesla'
no.of.tweets = 100

#search twitter with query information
tweets <- searchTwitter(search.string, n=no.of.tweets, lang="en")

#return search results as a table of data (data frame)
search.df <- twListToDF(tweets)

4. Click OK.

Method - User Timeline

In this example, we are pulling the last 1000 tweets from the Displayr Twitter timeline.

1. In the Data Sources pane, click the Plus (+) button at the top and click on R.

2. Enter a name for the data set under Name.

3. Paste the below R code in the editor box and modify it to your example:

#load libraries with R functions needed
library("twitteR")
library("ROAuth")

# Provide Authentication information
consumer_key = 'your_consumer_key'
consumer_secret = 'your_consumer_secret'
access_token = 'your_access_token'
access_secret = 'your_access_secret'

#authenticate the connection
setup_twitter_oauth(consumer_key, consumer_secret, access_token, access_secret)

# Query info
twitter.account = "displayrr"
number.results = 1000
tweets.user = userTimeline(twitter.account, 
                           n=number.results, 
                           maxID=NULL, 
                           sinceID=NULL, 
                           includeRts=TRUE)

#turn the query results into a table of data (data frame)
tweets.user = twListToDF(tweets.user)

3. Click OK.

Notes

If you wish to make this call take place daily, you can add the below lines to the top of your R code, see How to Automatically Update Calculations, Variables and Data Sets Using R for more detail:

library(flipTime)
UpdateEvery(1, "days", options = "wakeup")

API Limits

Rate Limits

Twitter imposes API rate limiting on a per-user (per access token) basis. Details regarding Twitter API rate limiting can be found on the Twitter API Rate Limiting page.

R Server Limits

When making API calls from Displayr, the R server has a timeout limit of approximately 2 minutes. Under normal circumstances, the Twitter API will return approximately 1500 records within this timeframe. The number of records that will be returned will vary depending on the type of API call being made (for example, searching Twitter versus querying Tweet data from a specific user). Therefore, it is generally good practice to split larger data sets into multiple API calls so as to work within the R server timeout limits.

How to Create a Word Cloud

How to Show Sentiment in Word Clouds

How to Semi-Automatically Code Text Data

How to Map Text Analysis

Creating Summary Tables and Crosstabs