How to Import Data From X (Formerly Known as Twitter)

This article describes how to import data from X (formerly known as Twitter) into Displayr using R. You can see more examples of what the R functions can do if you google the twitteR R package. It is recommended to test your modified R code from below first in a custom calculation on the page (Calculation > Custom Code) in case it requires troubleshooting, see How to Troubleshoot R Code in Displayr.

Requirements

A Displayr document.
An X Developer account with the tokens and keys below from your Developer Platform via Apps > Details > Keys and tokens
- a valid consumer API key and consumer API secret
- access token and access secret

Method - Search X

The following steps create an R Data Set with, the last 100 posts (formerly known as tweets) with a mention of 'Tesla'.

1. In the Data Sources pane, click the Plus (+) button at the top and click on R.

2. Enter a name for the data set under Name.

3. Paste the below R code in the editor box and modify to your example:

#load libraries with R functions needed
library("twitteR")
library("ROAuth")

# Provide Authentication information
consumer_key = 'your_consumer_key'
consumer_secret = 'your_consumer_secret'
access_token = 'your_access_token'
access_secret = 'your_access_secret'

#authenticate the connection
setup_twitter_oauth(consumer_key, consumer_secret, access_token, access_secret)

#Add in Query information
search.string = 'Tesla'
no.of.tweets = 100

#search twitter with query information
tweets <- searchTwitter(search.string, n=no.of.tweets, lang="en")

#return search results as a table of data (data frame)
search.df <- twListToDF(tweets)

4. Click OK.

Method - User Timeline

In this example, we are pulling the last 1000 posts from the Displayr X timeline.

1. In the Data Sources pane, click the Plus (+) button at the top and click on R.

2. Enter a name for the data set under Name.

3. Paste the below R code in the editor box and modify it to your example:

#load libraries with R functions needed
library("twitteR")
library("ROAuth")

# Provide Authentication information
consumer_key = 'your_consumer_key'
consumer_secret = 'your_consumer_secret'
access_token = 'your_access_token'
access_secret = 'your_access_secret'

#authenticate the connection
setup_twitter_oauth(consumer_key, consumer_secret, access_token, access_secret)

# Query info
twitter.account = "displayrr"
number.results = 1000
tweets.user = userTimeline(twitter.account, 
                           n=number.results, 
                           maxID=NULL, 
                           sinceID=NULL, 
                           includeRts=TRUE)

#turn the query results into a table of data (data frame)
tweets.user = twListToDF(tweets.user)

3. Click OK.

Notes

If you wish to make this call take place daily, see How to Set Up Schedules in Displayr.

API Limits

Rate Limits

X imposes API rate limiting on a per-user (per access token) basis. Details regarding X API rate limiting can be found on the X API Rate Limiting page.

R Server Limits

When making API calls from Displayr, the R server has a timeout limit of approximately 2 minutes. Under normal circumstances, the X API will return approximately 1500 records within this timeframe. The number of records that will be returned will vary depending on the type of API call being made (for example, searching X versus querying post data from a specific user). Therefore, it is generally good practice to split larger data sets into multiple API calls so as to work within the R server timeout limits.

How to Create a Word Cloud

How to Show Sentiment in Word Clouds

How to Semi-Automatically Code Text Data

How to Map Text Analysis

Creating Summary Tables and Crosstabs