02-Getting-Started-with-R.Rmd

# Getting started with R 

## RStudio Interface and Data 

### Download and Install RStudio 

This course is based on the statistical software R. R is easier to use within RStudio, which works on Windows, macOS, and other operating systems.

It is possible to [download a free version of RStudio Desktop](https://rstudio.com/products/rstudio/download/) from the official websites. You might also use a free *online* version of RStudio by registering to the [RStudio Cloud free plan](https://rstudio.cloud). However, the free plan gives you just 15 hours per months.

Now let’s see how to get started with RStudio Desktop. First, [download and install a free version of RStudio Desktop](https://rstudio.com/products/rstudio/download/) and open the software.

### Create a RStudio Project and Import data

When starting a data analysis project with RStudio, we create a new dedicated environment where we will keep all the scripts (files containing the code to perform the analysis), data sets, and outputs of the analysis (such as plots and tables). This dedicated workspace is simply called a *project*.

To create a new project with RStudio, follow these steps:

-   click on *File* (on the top left);
-   then, click on *New Project*;
-   select *New Directory*, and *New Project*;
-   Choose a folder for the project and give a name to your project. You can use the name *Time-Series-Analysis-With-R*.

This will create a new folder for the project in the main folder specified in the previous step. In this folder, you will find a file **.Rproj**, which has the same name as the project you assigned. To work on this project, you just need to open the *.Rproj* file.

```{r echo=FALSE, fig.height=3, fig.width=5}
knitr::include_graphics("images/r-studio-project-creation.png")
```

### Create a Script

Once the project is created, you can open a new **script** and save it.

A script is a file containing code. We can create a first script named *basic-r-syntax*, where you will test the basic code we are going to see. The script will be saved with extension *.r*.

You can open, change, and save the file every time you work on it. Saving your code is important; otherwise, you would have to write the same code every time you work on the project!

```{r echo=FALSE, fig.height=3, fig.width=5}
knitr::include_graphics("images/create-script.png")
```

[Click here to watch "Create and Save Scripts"](https://youtu.be/jF28dJS9xgQ)

[Click here to watch "Update Scripts and Run Code"](https://youtu.be/Mt3giRtUWZI)

### The RStudio User Interface

The interface of RStudio is organized in four main quadrants:

-    The top-left quadrant is the editor. Here you can create or open a script and compose the R commands.
-   The top-right quadrant shows the R workspace, which holds the data and other objects you have created in the current R session. There is also the *Files* tab, where you can navigate files and folders and find, for instance, the data sets you want to upload.
-   On the bottom left is the R Console window, where the code gets executed and the output is produced. You can run the commands, sending the code from the editor to the console, by highlighting it and hitting the *Run* button, or the Ctrl-Enter key combination. It is also possible to type and run commands directly into the console window (in this case, nothing will be saved).
-   The bottom-right quadrant is a window for graphics output. Here you can visualize your plots. There are also tabs for R packages and the R Help facility.


```{r echo=FALSE}
knitr::include_graphics("images/r-environment.png")
```

### Load and Save Data

To load data into R, you can click on the Files window in the top-right quadrant, navigate your files and folders, and once you have found your data set file, you can just click it and follow the semi-automated import procedure.

```{r echo=FALSE}
knitr::include_graphics("images/load-data.png")
```

```{r echo=FALSE}
knitr::include_graphics("images/import-data-1.png")
```

[Click here to watch "Import Data"](https://youtu.be/SRh3N6mSaLA)

Otherwise, you can upload a data set by using a function. For instance, to import a *csv* file, one of the most common formats for data sets, you can use the function **read.csv**. The main argument of this function is the *path* of the file you want to upload. To specify the file path, consider that you are working within a specific environment; that is, your working directory is the folder of the project (you can double-check the *working directory* you are in by running the command **getwd()**). Thus, to indicate the path of the data set you want to upload, you can write a dot followed by a slash **./**, followed by the path of the data set *inside the working directory*. For instance, in the case below, the data set is saved in a folder named *data* inside the working directory. The name of the data set is *tweets_vienna* and its extension is .csv. Therefore, the code to upload the file is as follows:

```{r}
fake_news <- read.csv("./data/fake-news-stories-over-time-20210111144200.csv")
```

To save data, there are a few options. Generally, if you want to save a data set, you can opt for the .csv or the .rds format. The .rds format is only readable by R, while the .csv format is "universal" (you can read it with Excel, for instance).

To save a file as .csv, you can use the function **write.csv**. The main arguments of this function are the name of the object that has to be saved, the path to the folder where the object will be saved, and the name you want to assign to the file.

```{r}
write.csv(fake_news, file = "./data/fake_news.csv")
```

To save a .rds file, the procedure is similar, but the saveRDS function has to be employed. To read an .rds file, the appropriate function is **readRDS**.

```{r}
saveRDS(fake_news, file = "./data/fake_news.rds")

fake_news <- readRDS("./data/fake_news.rds")   # read a .rds file
```

In the code above, you can notice a hash mark sign followed by some text. It is a **comment**. Comments are textual content used to describe the code in order to make it easier to understand and reuse. Comments are written after the **hash mark sign (#)** because the text written after the hash mark sign is ignored by R: you can read the comments, but R does not consider them as code.

### Create new Folders

It is good practice to create, in the main folder of the project, sub-folders dedicated to different *types* of files used in the project, such as a folder *data* for the data sets.

To create a new folder, you can go to the *Files* window in the RStudio interface, click **New Folder**, and give it a name.


```{r echo=FALSE}
knitr::include_graphics("images/new-folder.png")
```

## Basic R

### Objects

An *object* is an R entity composed of a name and a value.

The **arrow (<-)** sign is used to *create objects* and *assign a value* to an object (or to change or "update" its previous value).

Example: create an object with the name *object_consisting_of_a_number* and value equal to 2:

```{r}

object_consisting_of_a_number <- 2
```

Enter the name of the object in the console and run the command: the value assigned to the object will be displayed.

```{r}
object_consisting_of_a_number
```

The object is equal to its value. Therefore, for instance, an object with a numerical value can be used to perform arithmetical operations.

```{r}
object_consisting_of_a_number * 10
```

The value of an object can be transformed:

```{r}
object_consisting_of_a_number <- object_consisting_of_a_number * 10

object_consisting_of_a_number
```

An object can also represent a **function**.

Example: create an object for the *sum* (addition) function:

```{r}
function_sum <- function(x, y){
  result <- x + y
  return(result)
}
```

The function can now be applied to two numerical values:

```{r}
function_sum(5, 2)
```

Actually, we don't need this function, since mathematical functions are already implemented in R.

```{r}
sum(5, 2)
```

```{r}
5 + 7
```

```{r}
2 * 3
```

```{r}
3^2
```

```{r}
sqrt(9)
```

The value of an object can be a number, a function, or a **vector**. Vectors are sequences of values.

```{r}
vector_of_numbers <- c(1,2,3,4,5,6,7,8,9,10) 
```

```{r}
vector_of_numbers
```

A vector of numbers can be the argument of mathematical operations.

```{r}
vector_of_numbers * 2
```

```{r}
vector_of_numbers + 3
```

Other R objects are *matrix*, *list*, and *data.frame.*

A **matrix** is a table composed of rows and columns containing only numerical values.

```{r}
a_matrix <- matrix(data = 1:50, nrow = 10, ncol = 5)

a_matrix
```

A **list** is just a list of other objects. For instance, this list includes a numerical value, a *vector* of numbers, and a matrix.

```{r}
a_list <- list(object_consisting_of_a_number, vector_of_numbers, a_matrix)

a_list
```

A **data.frame** is like a matrix that can contain *numbers* but also other types of data, such as *characters* (a textual type of data) or *factors* (unordered categorical variables, such as gender, or ordered categories, such as low, medium, high).

Data sets are usually stored in *data.frames*. For instance, if you import a *csv* or an *Excel* file in R, the corresponding R object is a *data.frame*.

```{r}
# this is an object (vector) consisting of a series of numerical values
numerical_vector <- c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
numerical_vector
```

```{r}
# this is another object (vector) consisting of a series of categorical values
categorical_vector <- c("Monday", "Tuesday", "Monday", "Tuesday", "Monday", "Wednesday","Thursday", "Wednesday", "Thursday", "Saturday", "Sunday", "Friday", "Saturday", "Sunday")
categorical_vector
```

```{r}
# this is an object consisting of a data.frame, created combining vectors through the function "data.frame"
a_dataframe <- data.frame("first_variable" = numerical_vector,
                          "second_variable" = categorical_vector)
a_dataframe
```

To access a specific column of a data.frame, you can use the name of the data.frame, the dollar symbol **\$**, and the name of the column.

```{r}
a_dataframe$first_variable
```

```{r}
a_dataframe$second_variable
```

It is possible to add columns to a data.frame by writing:

-   the name of the data.frame
-   the dollar sign
-   a name for the new column
-   the *arrow sign \<-*
-   a vector of values to be stored in the new column (it has to have length equal to the other vectors composing the data.frame)

```{r}
a_dataframe$a_new_variable <- c(12, 261, 45, 29, 54, 234, 45, 42, 6, 267, 87, 3, 12, 9)
```

```{r}
a_dataframe
```

It is possible to visualize the first few rows of a data.frame by using the function *head*.

```{r}
head(a_dataframe)
```

```{r echo=FALSE}
knitr::include_graphics("images/data-frame-example.png")
```

### Functions

A **function** is a coded operation that applies to an object (e.g.: a number, a textual feature etc.) to transform it based on specific rules. A function has a name (the name of the function) and some *arguments*. Among the arguments of a function there is always an *object* or a value, for instance a numerical value, which is the content the function is applied to, and other possible arguments (either mandatory or optional).

Functions are operations applied to objects that give a certain output. E.g.: the arithmetical operation "addition" is a function that applies to two or more numbers to give, as its output, their sum. The *arguments* of the "sum" function are the numbers that are added together.

The name of the function is written out of *parentheses*, and the arguments of the function inside the parentheses:

```{r}
sum(5, 3)
```

Arguments of functions can be numbers but also textual features. For instance, the function *paste* creates a string composed of the strings that it takes as arguments.

```{r}
paste("the", "cat", "is", "at", "home")
```

In R you can sometimes find a "nested" syntax, which can be confusing. The best practice is to keep things as simple as possible.

```{r}
# this comment, written after the hash mark, describe what is going on here: two "paste" function nested together have been used (improperly! because they make the code more complicated than necessary) to show how functions can be nested together. It would have been better to use the "paste" function just one time!
paste(paste("the", "cat", "is", "at", "home"), "and", "sleeps", "on", "the", "sofa")
```

To sum up, functions manipulate and transform objects. Data wrangling, data visualization, as well as data analysis, are performed through functions.

### Data Types

Variables can have different R *formats*, such as:

-   **double**: numbers that include decimals (0.1, 5.676, 121.67). This format is appropriate for *continuous variables*;
-   **integer**: such as 1, 2, 3, 10, 400. It is a format suitable to *count data*;
-   **factors**: for categorical variables. Factors can be ordered (e.g.: level of agreement: "high", "medium", "low"), or not (e.g.: hair colors "blond", "dark brown", "brown");
-   **characters**: textual labels;
-   **logicals**: the format of logical values (i.e.: TRUE and FALSE)
-   **dates**: used to represent days;
-   **POSIX**: a class of R format to represent dates and times.

```{r fig.cap="R data formats. Tables from Gaubatz, K. T. (2014). [A Survivor's Guide to R: An Introduction for the Uninitiated and the Unnerved](https://us.sagepub.com/en-us/nam/a-survivors-guide-to-r/book242607). SAGE Publications.", echo=FALSE}
knitr::include_graphics("images/r-data-format.png")
```

It is better to specify the appropriate type of data when importing a data set. In the example below, the data format are specified by using the import process of RStudio.

Notice that the data of type "date" requires users to specify the additional information regarding the format of the dates. Indeed, dates can be written in many different ways, and to read dates in R it is necessary to specify the structure of the date. In the example, dates are in the format Year-Month-Day, which is represented in R as "%Y-%m-%d" (further details will be provided in another section of the book).

[Click here to watch "Import Data and Specify Data Types"](https://youtu.be/2re79pNEy2g)

### Excercise

-   Upload the data set "election news small", using the appropriate data format;
-   Open the script "basic-r-script" and perform the following operations:
    -   Check the first few rows of the data set;
    -   Access the single columns;
    -   Save the data frame with the name "election_news_small_test" in the folder "data" by using the function "write.csv" (to review the procedure go to the section "Load and Save Data" on this book);
    -   Comment the code (the comments have to be written after the hash sign *\#*);
    -   Save the script.

# Basic Data Wrangling with Tidyverse

[Data wrangling](https://en.wikipedia.org/wiki/Data_wrangling) *is the process of transforming and mapping data from one "raw" data form into another format with the intent of making it more appropriate and valuable for a variety of downstream purposes such as analytics. The goal of data wrangling is to assure quality and useful data. Data analysts typically spend the majority of their time in the process of data wrangling compared to the actual analysis of the data.*

[Another definition](https://link.springer.com/referenceworkentry/10.1007%2F978-3-319-63962-8_9-1) is as follows: *Data wrangling is the process of profiling and transforming datasets to ensure they are actionable for a set of analysis tasks. One central goal is to make data usable: to put data in a form that can be parsed and manipulated by analysis tools. Another goal is to ensure that data is responsive to the intended analyses: that the data contain the necessary information, at an acceptable level of description and correctness, to support successful modeling and decision-making.*

How to "manipulate" data sets in R:

-   use basic R functions;
-   employ specific libraries such as **tidyverse**. Tidyverse is an R library composed of functions that allow users to perform basic and advanced data science operations. <https://www.tidyverse.org>.

In R, a **library (or "package")** is a coherent collection of functions, usually created for specific purposes.

To work with the *tidyverse* library, it is necessary to install it first, by using the following command: *install.packages("tidyverse")*.

After having installed tidyverse (or any other library), it is necessary to load it, so as we can work with its functions in the current R session:

```{r warning=FALSE}
# to load a library used the command library(NAME-OF-THE-LIBRARY)
library(tidyverse)
```

Besides using the function *install.packages(NAME-OF-THE-LIBRARY)* by using a line of code, it is also possible to use the RStudio interface.

[Click here to watch "Install and Load Libraries"](https://youtu.be/4C60FIZZTRU)

## The Pipe Operator %\>%

Tidyverse has a peculiar syntax that makes use of the so-called [**pipe**](https://style.tidyverse.org/pipes.html) operator **%\>%**, like in the following example:

```{r paged.print=TRUE}
a_dataframe %>%
  group_by(second_variable) %>%
  summarize(mean = mean(a_new_variable))
```

To manipulate data sets we can rely on the functions included in [**dplyr**](https://dplyr.tidyverse.org): *a grammar of data manipulation, providing a consistent set of verbs that help you solve the most common data manipulation challenges*, such as *mutate*, *rename*, *summarize*.

```{r "Load example data set", paged.print=TRUE}
library(readr)
tweets <- read_csv("data/tweets_covid_small.csv", 
    col_types = cols(created_at = col_datetime(format = "%Y-%m-%d %H:%M:%S"), 
        retweet_count = col_integer()))
```

```{r}
head(tweets)
```

[Click here to watch "Import Data and Specify Data Types (Dates and Times)"](https://youtu.be/BnUV-hC4fSo)

## Mutate

The function **mutate** adds new variables to a data.frame or overwrites existing variables.

```{r paged.print=TRUE}
tweets <- tweets %>%
  mutate(log_retweet_count = log(retweet_count))
  
head(tweets)
```

## Rename

**rename** is a function to change the name of columns (sometimes it can be useful).

```{r paged.print=TRUE}
tweets <- tweets %>%
  # rename (new_name = old_name)
  rename(device = source)

head(tweets)
```

The previous two steps can be performed at the same time, by concatenating the operations through the *pipe %\>%* operator.

```{r paged.print=TRUE}
# load again the data set
library(readr)
tweets <- read_csv("data/tweets_covid_small.csv", 
    col_types = cols(created_at = col_datetime(format = "%Y-%m-%d %H:%M:%S"), 
        retweet_count = col_integer()))

tweets <- tweets %>%
  mutate(log_retweet_count = log(retweet_count+1)) %>%
  rename(device = source)

head(tweets)
```

To check the data format of the variables stored in the data.frame, you can use the command *str()*.

```{r paged.print=TRUE}
str(tweets)
```

Sometimes variables are stored in the data.frame in the wrong format (see the paragraph "data type"), so we may want to convert them to a new format. For this purpose, we can use the function **mutate** along with other functions such as *as.integer*, *as.numeric*, *as.character*, *as.factor*, *as.logical*, *as.Date*, or *as.POSIXct()* depending on the desired data format (it is possible and advisable to upload the data by paying attention to the type of data. If you upload the data in the correct format, you can skip this step).

```{r paged.print=TRUE}
tweets %>%
  mutate(device = as.character(device)) %>%
  head()

```

## Summarize and group_by

To aggregate data and calculate summary values (for instance, the average number of tweets *by day*), you can use the function *group_by* (to aggregate data, for instance by day) and *summarize* to calculate the summary values.

```{r message=FALSE, warning=FALSE, paged.print=TRUE}
tweets_summary <- tweets %>%
  group_by(screen_name) %>%
  summarize(average_retweets = mean(retweet_count))

head(tweets_summary)
```

It is also possible to create more than one summary variables at once.

```{r message=FALSE, warning=FALSE, paged.print=TRUE}
tweets_summary <- tweets %>%
  group_by(screen_name) %>%
  summarize(average_retweets = mean(retweet_count),
            average_log_retweets = mean(log_retweet_count))

head(tweets_summary)
```

### Count occurrences

A useful operation to perform when summarizing data is to count the occurrences of a certain variable. For instance, to count the number of tweets sent by each user, you can use the function **n()** inside the *summarize* function.

```{r message=FALSE, warning=FALSE, paged.print=TRUE}
tweets_summary <- tweets %>%
  group_by(screen_name) %>%
  summarize(average_retweets = mean(retweet_count),
            average_log_retweets = mean(log_retweet_count),
            number_of_tweets = n())

head(tweets_summary)
```

## Arrange

To explore a data set, it can be useful to sort the data (e.g., from the lowest to the highest value of a variable). With *tidyverse*, we can order a data.frame by using the function *arrange*.

To sort the data from the highest to the lowest value (descending order), the *minus* sign (or the *desc* function) has to be added.

```{r paged.print=TRUE}
tweets_summary  %>%
  arrange(-number_of_tweets) %>%
  head()
```

```{r paged.print=TRUE}
tweets_summary  %>%
  arrange(desc(average_retweets)) %>%
  head()
```

Without the minus sign (or the "desc" command), data are sorted from the lowest to the highest value.

```{r paged.print=TRUE}
tweets_summary  %>%
  arrange(number_of_tweets) %>%
  head()
```

## Filter

The function *filter* keeps only the cases (the "rows") we want to focus on. The arguments of this function are the conditions that must be fulfilled to filter the data: a) the name of the column that we want to filter, and b) the values to be kept.

```{r paged.print=TRUE}
tweets %>%
  filter(retweet_count >= 500) %>%
  arrange(-retweet_count)
```

In the examples below, notice the use of a double equal sign *==*, and also of the quotation marks to indicate the modalities of a categorical variable.

```{r paged.print=TRUE}
tweets %>%
  filter(retweet_count == 1988) 
```

```{r paged.print=TRUE}
tweets %>%
  filter(device == "Twitter for Android")
```

It is also possible to use several conditions at the same time.

```{r paged.print=TRUE}
tweets %>%
  filter(device == "Twitter for Android",
         retweet_count > 200) %>%
  arrange(-retweet_count)
```

## Select

**select** is used to keep only some of the columns of the original data.frame. For instance, we can apply the function to keep just the columns *device* and *retweet_count*.

```{r paged.print=TRUE}
tweets %>%
  dplyr::select(device, retweet_count) %>%
  head()
```