Frequently used R tips and tricks (updated regularly)
When using R for data wrangling, visualization, and analysis, I sometimes forget code I wrote previously and end up on the exact same web pages every time I search for it. Here, I will regularly post handy tips that I have had to look up more than a few times, wondering, "Wait, how did that go again?"
Data wrangling
Read and combine all .csv files in a folder at once
csv_list <- fs::dir_ls(path = "../data/",
glob = "*.csv", # .csvを検索
recurse = TRUE # サブディレクトリも検索したい場合はTRUE
)
dat <- map(csv_list, ~ readr::read_csv(.x)) |> bind_rows()Code language: PHP (php)Convert kanji numerals to Arabic numerals
pacman::p_load(zipangu)
dat$x <- dat$x %>% kansuji2arabic()Code language: PHP (php)Convert full-width characters to half-width
pacman::p_load(stringi)
dat$x <- dat$x %>% stri_trans_nfkc()Code language: PHP (php)Pad numbers to a specified number of digits (9 → 09)
dat %>% mutate(x_pad = str_pad(x, 2, pad=0))Replace NAs with 0 in numeric columns
dat <- dat %>%
mutate_if(is.numeric, ~ replace_na(.x, 0))Code language: CSS (css)dat <- dat %>%
mutate(across(where(is.numeric), ~ replace_na(.x, 0))) Code language: CSS (css)When using mutate(across()), note that the function must be placed inside across().
If you write mutate(across(where(is.numeric)), ~ replace_na(.x, 0)),
Error: Problem with `mutate()` input `..2`. x Input `..2` must be a vector, not a `formula` object. ℹ Input `..2` is `~replace_na(.x, 0)`.Code language: JavaScript (javascript)you will get this error.
Select rows without NA
dat %>% filter(!(is.na(x)))Code language: CSS (css)Replace 0 with NA using dplyr::if_else
dat %>% mutate(x = if_else(x == 0, NA_real_, x))Note that dplyr::if_else requires matching data types. If you just use NA,
`false` must be a logical vector, not a double vector.Code language: JavaScript (javascript)it returns an error like this. Use the double-type NA_real_ instead.
Concatenate strings (str_c()) in data containing columns with NAs
To use str_c() including a column that contains NAs, wrap the target column (column Y in this example) in str_replace_na(). Because NA is generated as the string "NA", you then perform a string replacement to remove the ":NA" generated alongside the ":" specified in sep.
dat <- dat %>%
mutate( Z = str_c(X,str_replace_na(Y),sep = ":")) %>%
mutate(across(Z, ~ gsub(":NA","",.x))) Code language: JavaScript (javascript)Batch convert data types
factor_list <- c("x","y","z")
dat <- dat %>% mutate_at(vars(all_of(factor_list)),as.factor)Code language: JavaScript (javascript)Using across(), it looks like this:
factor_list <- c("x","y","z")
dat <- dat %>% mutate(across(all_of(factor_list),as.factor))Code language: JavaScript (javascript)Or explicitly define the function using ~ and .x:
dat <- dat %>% mutate(across(all_of(factor_list), ~ as.factor(.x)))Code language: CSS (css)Batch rename columns
name_vector <- c("apple(変更前)"="1(変更後)",
"banana"="2",
"orange"="3")
dat %>%
mutate(across(label, ~ str_replace_all(.x, name_vector)))Code language: JavaScript (javascript)Rename factor levels
pacman::p_load(forcats)
fct_list <- c("全くそう思う(変更後)" = "1(変更前)","少しそう思う" = "2",
"あまりそう思わない" = "3", "全くそう思わない" = "4")
dat <- dat %>% mutate(x = fct_recode(x, !!!fct_list))Code language: PHP (php)You can also rename factor levels with the method below, but note that it does not change their order. Explicitly mapping them to numbers as shown above is less prone to mistakes.
levels(dat$x) <- c("全くそう思う","少しそう思う","あまりそう思わない","そう思わない")Code language: PHP (php)Collapse and regroup factor levels
dat <- dat %>% mutate(
x2 = fct_recode(x,
"fruits" = "apple",
"fruits" = "banana",
"fruits" = "orange",
"vehicle" = "car",
"vehicle" = "bike"))Code language: JavaScript (javascript)Change the order of factor levels
fct_levels <- c("a","b","c")
dat$x <- dat$x %>% factor(levels = fct_levels) Code language: PHP (php)You can also write this in mutate() using across as follows:
dat <- dat %>% mutate(across(x, ~ factor(.x, levels = fct_levels)))Code language: HTML, XML (xml)Using fct_relevel() from {forcats}, it looks like this. Note, however, that this will issue a warning if you specify a value that does not exist.
pacman::p_load(forcats)
fct_levels <- c("a","b","c")
dat <- dat %>% mutate(across(x, ~ fct_relevel(.x, levels = fct_levels)))Code language: PHP (php)To reverse the order, fct_rev() from {forcats} is useful.
pacman::p_load(forcats)
dat <- dat %>% mutate(across(x, ~ fct_rev(.x)))Code language: CSS (css)Batch rename with dplyr::rename using a character vector
To rename multiple columns at once, rename(!!! named_vector) is convenient. Alternatively, you can use dput() (described below).
named_vector <- c("id" = "x",
"fruits" = "y")
dat <- dat %>% rename(!!! named_vector)Code language: JavaScript (javascript)Get column names as a vector
For example, if you want to rename columns in a batch, running
dat %>% names() %>% dput()gives you
c("ID", "果物", "個数")Code language: JavaScript (javascript)a character vector like this, which you can copy and edit.
# c("ID", "果物", "個数") コピペした後に書き換えて setNames()を利用して一括で列名を変更
dat <- dat %>% setNames(c("ID", "fruits", "n"))Code language: PHP (php)Count occurrences of a specific value (X) in a vector
sum(dat$x == X)Code language: PHP (php)Specify a string separately and pass it to a function
cat <- "x"
dat %>% filter(!is.na(.data[[cat]]))Code language: CSS (css)Writing is.na(cat) causes an error. In this case, use .data[[ column_name ]].
You can use the same technique inside ggplot's aes() (another option is to use aes_string()).
Specify columns by index rather than string in dplyr::filter
df %>% dplyr::filter(.[1] == 1)Code language: PHP (php)However, for reasons explained below, enclosing the index in double brackets like .[[column_index]] is less confusing and preferable.
df %>% dplyr::filter(.[[1]] == 1)Code language: PHP (php)Use partial string matching in dplyr::filter (+ specify by column index)
dat %>% filter(str_detect(.[[1]], "文字"))Code language: JavaScript (javascript)Or
dat %>% filter(grep("文字", .[[1]]))Code language: JavaScript (javascript).[column_index] does not work here, whereas .[[column_index]] does. For that reason, using [[ from the start is less error-prone.
Separate a column containing combined numbers and text
For example, with data like "0100札幌市保健所" (0100 Sapporo City Public Health Center):
dat <- dat %>%
mutate(code = str_extract(code_area,"^(\\d)+")) %>%
mutate(area = str_extract(code_area, "(\\D)+$"))Code language: JavaScript (javascript)- \\d : digit
- \\D : non-digit
- + : sequence of characters
- ^ : start of string
- $ : end of string
are the regular expressions used here. (Reference: List of basic regular expressions)
Calculate age from two Date columns
pacman::p_load(lubridate)
dat <- dat %>% mutate_at(vars(d_birth, d_x), as.Date)
dat <- dat %>% mutate( age = as.duration(d_x - d_birth))
dat <- dat %>% mutate( age = as.double(str_extract(age, "(?<=~).+(?=\s)"))) Code language: PHP (php)(?<=~).+(?=\s) is a regular expression meaning "characters between ~ and a space".
- ?<=~ : preceded by ~
- .+ : sequence of any characters
- ?=\s : followed by a space (\s)
Extract data for a specific period from date data
Extracting data from January 1, 2020 to December 31, 2020 from a date column (date)
dat %>%
filter(date >= as.Date("2020-01-01") & date <= as.Date("2020-12-31"))Code language: JavaScript (javascript)Converting values calculated with {lubridate} as.duration() into days
pacman::p_load(lubridate)
dat <- dat %>%
mutate(day = as.duration(x_2 - x_1 + ddays(1))/dseconds(86400))Code language: PHP (php)Expanding list data into a data.frame
dat_list <- dat_list %>% reduce(rbind)Code language: HTML, XML (xml)Removing duplicate rows
dat %>% distinct(x, y, .keep_all = TRUE)Code language: PHP (php)Checking for duplicate rows
Use table() if you also want to know the number of duplicates.
dat %>%
pull(col_name) %>%
table() %>%
data.frame() %>%
filter(Freq >= 2) %>%
arrange(desc(Freq))Replacing NAs in specific columns
dat <- dat %>%
replace_na(list(x = 0,
y = 0,
z = 0))Code language: HTML, XML (xml)Copying the upper triangle of a matrix to the lower triangle
M <- dat
for(i in 1:nrow(M)) {for(j in 1:i) {M[j,i]=M[i,j]}}Code language: HTML, XML (xml)For example, when calculating ordinal correlation coefficients (such as polychoric or polyserial correlations) with hetcor() in {polycor}, the function returns values only in the lower triangle. To plot a correlation matrix using a package like {corrplot}, handle it as follows.
pacman::p_load(polycor)
ans <- dat %>% hetcor()
M <- ans$tests
for(i in 1:nrow(M)) {for(j in 1:i) {M[j,i]=M[i,j] }}Code language: HTML, XML (xml)Summing multiple selected columns in mutate()
Use rowSums or reduce.
dat %>%
mutate(sum = rowSums(across(c(●, ●))))Or
dat %>%
mutate(sum = reduce(across(c(●, ●)), `+`))Code language: JavaScript (javascript)Running batch calculations across columns while keeping data in wide format
dat |> group_by(group) |> summarise(across(where(is.double), ~ first(.x) – last(.x)))
Getting data from subdirectories
To get file paths from multiple subdirectories as well, set recursive=TRUE inside list.files().
read_xlsx_list <- list.files(
"../materials/folder", # 検索するフォルダパスを指定
"*.xlsx", # .xlsx ファイルを抽出
recursive=TRUE) %>% # サブフォルダ内も検索できるようになる
str_c("../materials/folder/",.) # パスを結合しておけば後でread_excel()しやすいCode language: PHP (php)Visualization
Using theme_set() in {ggplot2}
Define this at the beginning of a file to apply it to all subsequent ggplot plots.
theme_set(
theme_classic() +
theme(
strip.background = element_blank(),
text = element_text(size = 10),
axis.text.x = element_text(angle = 0, vjust = 1, hjust = 1),
axis.ticks.length.x = unit(0.1, "cm"),
legend.position = 'none')
)Code language: JavaScript (javascript)Lately, I set the following, including theme_set(), as the default in my .md files.
---
title: "yyyymmdd_xxxxx"
output: html_document
date: "2024-xx-xx"
author: "xxxxxx"
output:
html_document:
theme: cerulean
toc: yes
toc_depth: 3
toc_float: yes
df_print: "kable"
powerpoint_presentation:
reference_doc: template.pptx
---
```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = FALSE)
knitr::opts_chunk$set(eval = TRUE)
knitr::opts_chunk$set(tidy = FALSE)
knitr::opts_chunk$set(warning = FALSE)
knitr::opts_chunk$set(error = FALSE)
knitr::opts_chunk$set(message = FALSE)
knitr::opts_chunk$set(dpi = 300, fig.width = 10, fig.height = 10*0.618)
```
```{r}
theme_set(
theme_classic() +
theme(
strip.background = element_blank(),
text = element_text(size = 10, family = "HiraginoSans-W3"),
axis.text.x = element_text(angle = 0, vjust = 1, hjust = 1),
axis.ticks.length.x = unit(0.1, "cm"),
legend.position = 'none')
)
```
```{r}
pacman::p_load(tidyverse)
```Code language: PHP (php)Passing column names as strings to aes() in {ggplot2}
dat %>% ggplot(aes_string(x=x, y=y))Use aes_string() instead of aes(). Alternatively, use the method below.
dat %>% ggplot(aes(x=.data[["x"]], y=.data[["y"]]))Code language: JavaScript (javascript)Remembering .data[[“x”]] can be useful when you have trouble passing arguments while writing functions (e.g., with is.na(), arrange(), pull()).
Slightly tilting x-axis labels in {ggplot2}
dat %>% ggplot(aes(x)) +
geom_count() +
theme(axis.text.x = element_text(angle=30, hjust=1))Adjusting legend position in {ggplot2}: Bottom and right-aligned
g <- ggplot() + ...
g + theme(legend.position = "bottom",
legend.justification = c("right"))Code language: HTML, XML (xml)Controlling the width of geom_jitter() by density distribution with {ggforce} geom_sina
Violin plots are easy to read, but geom_sina() plots individual dots in the shape of a violin plot, making the figure visually intuitive.
pacman::p_load(ggforce)
dat %>% ggplot(aes(X,Y)) +
geom_sina(aes(coular = Z))Code language: PHP (php)Adjusting label positions on a 100% stacked bar chart
ddat %>%
group_by(x,y) %>%
summarize(count=n()) %>%
arrange(x,desc(y)) %>%
ddply(x,transform,label_y=cumsum(count)) %>%
ggplot(aes(x = x, y = count, fill = y)) +
geom_col()+
geom_text ( aes ( y = label_y ,
label = count ), colour = "black", vjust = "inward") +
scale_fill_brewer(palette = "Blues",
name="Y",
breaks=c("a", "b", "c"),
labels=c("A", "B", "C") ) +
scale_x_continuous(breaks=c(1,2,3)) +
xlab("X")+
ylab("Count")+
theme_minimal() Code language: JavaScript (javascript)Choosing color schemes with ColorBrewer
Choosing color schemes with "Color Palette Cinema"
Colorblind-safe palettes (safe color)
palette("Okabe-Ito")Code language: JavaScript (javascript)Analysis
Steel-Dwass multiple comparison from tidy data
For the Steel-Dwass nonparametric multiple comparison test, Professor Aoki's article (Gunma University) is a helpful reference, and I wrote a custom function to run Steel-Dwass directly from tidy data.
pacman::p_load(tidyverse)
source("http://aoki2.si.gunma-u.ac.jp/R/src/Steel-Dwass.R", encoding="euc-jp")
steel_dwass <- function(df, data, group){
data_vector <- df %>% pull(data)
group_vector <- df %>% pull(group)
result <- Steel.Dwass(data_vector, group_vector)
result_tibble <- result %>% as_tibble()
result_tibble <- cbind(steel_dwass = rownames(result),result_tibble)
result_tibble <- result_tibble %>%
mutate(star = if_else( p < 0.001, "**", if_else(p < 0.05, "*", "NS"))) %>%
mutate(note = if_else(p < 0.001, "p＜0.001", if_else(p < 0.05, "p＜0.05", "NS"))) %>%
mutate(across(p, ~ round(.x, digits = 3)))
result_tibble
}Code language: PHP (php)Specify the data column first, followed by the group column.
dat %>% steel_dwass("data", "group")Code language: JavaScript (javascript)Speeding up {rstan} execution
rstan_options(auto_write = TRUE)
options(mc.cores = parallel::detectCores())Code language: PHP (php){gtsummary} Personal template (paired tests)
dat |>
tbl_summary(
by = case_control, # 比較したいもの
include = -ID, # 省きたいもの。でも対応ある検定で使いたい。
type = list(all_continuous() ~ "continuous2",
c("count") ~ "continuous2"), # 列名を明示してもOK
statistic = all_continuous() ~ c(
"{mean} ({sd})",
"{median} ({p25}, {p75})",
"{min}, {max}"),
digits = list(where(is.numeric) ~ c(1,1,1,1,1)) # 有効数字
) |>
add_p(
# test = list(
# all_continuous() ~ "t.test"), # t.testを行いたい場合はこちら
group = ID, # 対応ある検定をする際はIDを明示
test = list(
all_continuous() ~ "wilcox.test",
all_categorical() ~ "mcnemar.test"),
pvalue_fun = function(x) style_pvalue(x, digits = 2)) |> # 有効数字
bold_p() |>
bold_labels() |>
add_overall() Code language: PHP (php)Specifying continuous and categorical variables all at once in {gtsummary}
pacman::p_load(gtsummary)
table_list <- c("x_1", "x_2", "y_1", "y_2", "z")
table_label_list <- c("X1", "X2", "Y1", "Y2", "Z")
cnt_list <- c("x_1", "x_2")
ctg_list <- c("y_1", "y_2")
dct_list <- c("z")
table_one <- dat %>%
select(all_of(table_list)) %>%
setNames(all_of(table_label_list)) %>%
tbl_summary(
type = list(cnt_list ~ "continuous",
ctg_list ~ "categorical",
dct_list ~ "dichotomous"))Code language: PHP (php){gtsummary} Writing summaries of continuous variables across multiple lines
pacman::p_load(gtsummary)
table_one <- dat %>%
select("x","y","z") %>%
tbl_summary(
type = all_continuous() ~ "continuous2",
statistic = all_continuous() ~ c(
"{N_nonmiss}",
"{mean} ({sd})",
"{median} ({p25}, {p75})",
"{min}, {max}"),
digits = list(where(is.numeric) ~ c(0,1,1,1,1,1,1,1)),
missing = "no")Code language: PHP (php)Note that you need to set type to “continuous2”. Group the decimal places into c() in order of appearance. Here, N is set to an integer, and the rest to 1 decimal place.
{gtsummary} Displaying row/column names and footnotes in Japanese
theme_gtsummary_language("ja")
Code language: JavaScript (javascript)To revert to the default, run the code below.
reset_gtsummary_theme(){gtsummary} {snakecase} Converting snake_case variable names to camelCase for display
pacman::p_load(gtsummary, snakecase)
table_one <- dat %>%
select("x","y","z") %>%
set_variable_labels(
.labels = to_title_case(names(.))) %>%
tbl_summary()Code language: PHP (php)Sorting columns created with {gtsummary} by frequency
pacman::p_load(gtsummary)
table_one <- dat %>%
select("x","y","z") %>%
tbl_summary(
sort = all_categorical() ~ "frequency")Code language: PHP (php)For specific columns only (such as A and B),
tbl_summary(sort = c("A","B") ~ "frequency")Code language: JavaScript (javascript)use the above.
Transposing a table created with {gtsummary}
pacman::p_load(gtsummary)
table_one <- dat %>%
select("x","y","z") %>%
tbl_summary() %>%
bold_labels() %>%
add_n() %>%
as_tibble() %>%
t() %>%
as.data.frame()Code language: PHP (php)Adding significance stars in {modelsummary}
They will appear if you write "{estimate} ({std.error}){stars}" instead of estimate = "{estimate} ({std.error})stars" as shown in the help documentation.
Grouping with by = across two or more variables in {gtsummary}
In that case, use tbl_strata().
d |>
tbl_strata(
strata = group1,
~.x %>%
tbl_summary(by = group2)){corrplot} Customizing a correlation matrix
pacman::p_load(corrplot)
col_list <- c("x","y","z")
labelname_list <- c("X","Y","Z")
dat_cor <- dat %>%
select(all_of(col_list)) %>%
setNames(all_of(labelname_list)) %>%
na.omit()
p <- dat_cor %>%
cor.mtest(method = "spearman")
corrplot(cor(dat_cor, method = "spearman"),
method="number",
tl.col="black",
tl.cex=1,
tl.srt=45,
diag=FALSE)
corrplot(cor(dat_cor, method = "spearman"),
p.mat = p$p,
sig.level=0,
insig = "p-value",
method="ellipse",
type="lower",
tl.pos = "n",
cl.pos="n",
add=TRUE,
diag=FALSE)Code language: PHP (php){polycor} Correlation matrix between categorical variables (polychoric correlation matrix)
pacman::p_load(polycor)
ans <- dat %>% hetcor()
ans$correlation
M <- ans$tests
for(i in 1:nrow(M)) {for(j in 1:i) {M[j,i]=M[i,j] }}
p$p <- MCode language: PHP (php)Proceed as above, and modify part of the {corrplot} code shown earlier.
corrplot(ans$correlation,
method="number",
tl.col="black",
tl.cex=1,
tl.srt=45,
diag=FALSE)
corrplot(ans$correlation,
p.mat = p$p,
sig.level=0,
insig = "p-value",
method="ellipse",
type="lower",
tl.pos = "n",
cl.pos="n",
add=TRUE,
diag=FALSE)Code language: PHP (php){ggplot} Why you shouldn't use xlim and ylim to set the display range
To specify a range, use coord_cartesian(xlim = c(xmin,xmax)).
g + coord_cartesian(xlim = c(0, 1000))Adding significance bars with {ggsignif} to a plot created with {ggstatsplot}
pacman::p_load(ggsignif, ggstatsplot, wesanderson)
ggstatsplot::ggbetweenstats(
data = dat,
x = x,
y = y,
pairwise.comparisons = FALSE,
k = 0,
xlab = "x",
ylab = "y",
ggstatsplot.layer = TRUE,
package = "wesanderson",
palette = "GrandBudapest1"
) +
geom_signif(comparisons = list(c("a", "b")),
map_signif_level = c("<strong>* p＜0.001"=0.001, "</strong> p＜0.01"=0.01, "* p＜0.05"=0.05)) Code language: PHP (php)Communication and miscellaneous
Using pacman::p_load instead of library()
if (!require("pacman")) install.packages("pacman")
pacman::p_load(tidyverse, magrittr, package_n)Code language: PHP (php){tomwenseleers/export} {eoffice} Exporting figures and tables to PPT format
pacman::p_load(officer,rvg,openxlsx,ggplot2,flextable,xtable,rgl,stargazer,tikzDevice,xml2,broom,devtools)
pacman::p_load_gh("tomwenseleers/export")
graph2ppt(append=TRUE, upscale=TRUE) Code language: PHP (php)If you get an error, try the installation method at the link first, and then run graph2ppt().
install.packages("officer")
install.packages("rvg")
install.packages("openxlsx")
install.packages("ggplot2")
install.packages("flextable")
install.packages("xtable")
install.packages("rgl")
install.packages("stargazer")
install.packages("tikzDevice")
install.packages("xml2")
install.packages("broom")
install.packages("devtools")
devtools::install_github("tomwenseleers/export")Code language: PHP (php)If {export} doesn't work, use {eoffice}.
pacman::p_load(eoffice)
topptx(file="filename.pptx", left = 0, top = 0, width = 8, height = 6, append = TRUE)Code language: PHP (php)Reducing image file size in ggsave()
ggsave("figure1.tiff",
width = ...,
height = ...,
dpi=300,
compression = "lzw")Code language: JavaScript (javascript)How to fix transparent backgrounds in plots saved with ggsave()
You can specify the background color with bg, as in ggsave(…, bg = “white“).
ggsave("figure.png", g, bg = "white")Code language: JavaScript (javascript)
Conversely, if you want to make the background transparent, specify bg = “transparent”.
ggsave("figure.png", g, bg = "transparent")Code language: JavaScript (javascript)Creating necessary folders after launching a project file
Enter the following in the Terminal tab next to the Console:
mkdir {data,thesis,analysis,function,materials}Code language: JavaScript (javascript)Changing Rmarkdown's default path to a path relative to the project file
knitr::opts_knit$set(root.dir = rprojroot::find_rstudio_root_file())Code language: PHP (php)Common initial settings in Rmarkdown
Here is an example of the initial setup I often use, along with the default path setting above. Output figures and tables are set to 300 dpi by default.
---
title: "…"
output:
html_document:
theme: cerulean
toc: yes
toc_depth: 3
toc_float: yes
df_print: "kable"
---Code language: JavaScript (javascript){r setup, include=FALSE}
knitr::opts_knit$set(root.dir = rprojroot::find_rstudio_root_file())
knitr::opts_chunk$set(echo = FALSE)
knitr::opts_chunk$set(eval = TRUE)
knitr::opts_chunk$set(tidy = FALSE)
knitr::opts_chunk$set(warning = FALSE)
knitr::opts_chunk$set(error = FALSE)
knitr::opts_chunk$set(message = FALSE)
knitr::opts_chunk$set(dpi = 300)Code language: PHP (php){r}
if (!require("pacman")) install.packages("pacman")
pacman::p_load(tidyverse,readxl,readr,magrittr,...)Code language: PHP (php)Linking in-text citations to the bibliography with hyperlinks in Rmarkdown
By default, in-text references appear as superscript numbers like 1 that correspond to the numbered list at the end of the document, without hyperlinks. Specifying the “link-citations: true” option in the YAML header before knitting links them with hyperlinks.
link-citations: trueCode language: JavaScript (javascript)