Effect sizes for Bayesian models

Obtaining indices of effect size for Bayesian model is currently an issue, as no test statistics are present to help us compute such indices. The two alternatives (both available in effectsize) is 1) to compute standardized parameters or 2) obtain effect sizes by test-statistic approximation. This second method is described here.

Bayesian Regression Coefficients to Correlation via Test-Statistic Approximation

Overview

library(ppcor)

df <- iris[, 1:4]  # Remove the Species factor

ppcor::pcor(df)$estimate[2:4, 1]  # Select the rows of interest
>  Sepal.Width Petal.Length  Petal.Width 
>         0.63         0.72        -0.34

The goal is to retrieve coefficients similar to the above (partial) correlations for this multiple regression model: Sepal.Length ~ Sepal.Width + Petal.Length + Petal.Width. This can easily be achieved for frequentist models by converting the t statistic into a correlation:

library(effectsize)
library(parameters)

model <- lm(Sepal.Length ~ Sepal.Width + Petal.Length + Petal.Width, data = df) 

parameters <- model_parameters(model)[2:4,]
convert_t_to_r(parameters$t, parameters$df_residual)
> [1]  0.63  0.72 -0.34

Note that these are not equivalent to standardized parameters from multiple regressions (from which coefficients can be higher than 1).

standardize_parameters(model)$Std_Coefficient[2:4]
> [1]  0.34  1.51 -0.51

Let’s start by fitting the Bayesian regression:

library(rstanarm)

model <- stan_glm(Sepal.Length ~ Sepal.Width + Petal.Length + Petal.Width, data = df) 

The effectsize package provides the posteriors_to_r() function, which attempts to estimate the posterior of the (partial) correlation coefficient by approximating the t statistic (deviding the coefficient by the SD of the posterior) and using frequentist degrees of freedom. It is an hybrid method that needs to be validated. However, the results appear as very close:

library(bayestestR)

r <- convert_posteriors_to_r(model)
bayestestR::describe_posterior(r)$Median[2:4]

Applicability

Does it work in all cases?

Logistic Models

Let’s start with the frequentist model:

model <- glm(vs ~ cyl + disp + drat, data = mtcars, family = "binomial")

parameters <- model_parameters(model)
parameters$r <- convert_z_to_r(parameters$z, n = insight::n_obs(model))
parameters

However, as logistic models return log-odds, these can be directly converted to r:

parameters$r_from_odds <- convert_odds_to_r(parameters$Coefficient, log = TRUE)
parameters

And now the Bayesian:

model <- stan_glm(vs ~ cyl + disp + drat,, data = mtcars, family = "binomial") 
parameters <- model_parameters(model)
r <- convert_posteriors_to_r(model)
parameters$r <- bayestestR::describe_posterior(r)$Median
parameters
parameters <- model_parameters(model)
r <- convert_posteriors_to_r(model)
parameters$r <- bayestestR::describe_posterior(r)$Median
parameters$r_from_odds <- convert_odds_to_r(parameters$Median, log = TRUE)
parameters

References