05-ThreeWayInteraction.Rmd

# Power for Three-way Interactions
```{r include=FALSE}
load("data/threeway_data.RData")
```
There are almost no software solutions that allow researchers to perform power anaysis for more complex designs. Through simulation, it is relatively straightforward to examine the power for designs with multiple factors with many levels. 

Let's start with a 2x2x2 between-subjects design. We collect 50 participants in each between participant condition (so 400 participants in total - $50(n)\times2(levels)\times2(levels)\times2(levels)= 400$). 

```{r start_threewayinteraction}
# With 2x2x2 designs, 
# the names for paired comparisons can become very long. 
# So here I abbreviate terms: 
#   Size, Color, and Cognitive Load, have values:
# b = big, s = small, g = green, 
# r = red, pres = present, abs = absent.  
labelnames <- c("Size", "b", "s", "Color", "g", "r", 
                "Load", "pres", "abs") #
design_result <- ANOVA_design(design = "2b*2b*2b", 
                              #sample size per group 
                              n = 50, 
                              #pattern of means
                              mu = c(2, 2, 6, 1, 6, 6, 1, 8), 
                              sd = 10, #standard deviation
                              labelnames = labelnames) 
```
```{r eval=FALSE}
simulation_result <- ANOVA_power(design_result, 
                                 alpha_level = alpha_level, 
                                 nsims = nsims,
                                 verbose = FALSE)
```

```{r echo=FALSE}
knitr::kable(simulation_result_5.1$main_results,
             caption = "Simulated ANOVA Result")%>%
  kable_styling(latex_options = "hold_position")
```


```{r}
exact_result <- ANOVA_exact(design_result,
                            alpha_level = alpha_level,
                            verbose = FALSE)
```


```{r echo=FALSE}
knitr::kable(exact_result$main_results,
             caption = "Exact ANOVA Result")%>%
  kable_styling(latex_options = "hold_position")
```



```{r}
#Analytical power calculation
power_analytic <- power_threeway_between(design_result)
power_analytic$power_A
power_analytic$power_B
power_analytic$power_C
power_analytic$power_AB
power_analytic$power_AC
power_analytic$power_BC
power_analytic$power_ABC
power_analytic$eta_p_2_A
power_analytic$eta_p_2_B
power_analytic$eta_p_2_C
power_analytic$eta_p_2_AB
power_analytic$eta_p_2_AC
power_analytic$eta_p_2_BC
power_analytic$eta_p_2_ABC
```

We can also confirm the power analysis in g\*power [@faul2007g]. g\*power allows you to compute the power for a three-way interaction - if you know the Cohen's *f* value to enter. Cohen's *f* is calculated based on the means for the interaction, the sum of squares of the effect, and the sum of squares of the errors. This is quite a challenge by hand, but we can simulate the results, or use the analytical solution we programmed to get Cohen's *f* for the pattern of means that we specified. 

```{r}
# The power for the AC interaction (Size x Load) is 0.873535. 
power_analytic$power_AC
# We can enter the Cohen's f for this interaction. 
power_analytic$Cohen_f_AC
# We can double check the calculated lambda
power_analytic$lambda_AC
# We can double check the critical F value
power_analytic$F_critical_AC
```

![](screenshots/gpower_8.png)

A three-way ANOVA builds on the same principles as a one-way ANOVA. We look at whether the differences between groups are large, compared to the standard deviation. For the main effects we simply have 2 groups of 200 participants, and 2 means. If the population standard deviations are identical across groups, this is not in any way different from a one-way ANOVA. Indeed, we can show this by simulating a one-way ANOVA, where instead of 8 conditions, we have two conditions, and we average over the 4 groups of the other two factors. For example, for the main effect of size above can be computed analytically. There might be a small difference in the degrees of freedom of the two tests, or it is just random variation (And it will disappear when the number of iterations in the simulation, `nsim`, is increased). 

```{r}
string <- "2b"
n <- 200
mu <- c(mean(c(2, 2, 6, 1)), mean(c(6, 6, 1, 8)))
sd <- 10
labelnames <- c("Size", "big", "small")
design_result <- ANOVA_design(design = string,
                   n = n, 
                   mu = mu, 
                   sd = sd, 
                   labelnames = labelnames)
```
```{r eval=FALSE}
simulation_result <- ANOVA_power(design_result, 
                                 alpha_level = alpha_level, 
                                 nsims = nsims,
                                 verbose = FALSE)
```

```{r echo=FALSE}
knitr::kable(simulation_result_5.3$main_results,
             caption = "Simulated ANOVA Result")%>%
  kable_styling(latex_options = "hold_position")
```


```{r}
exact_result <- ANOVA_exact(design_result,
                            alpha_level = alpha_level,
                            verbose = FALSE)
```


```{r echo=FALSE}
knitr::kable(exact_result$main_results,
             caption = "Exact ANOVA Result")%>%
  kable_styling(latex_options = "hold_position")
```



```{r}
# Power based on analytical solution
power_oneway_between(design_result)$power 
```

Similarly, we can create a 2 factor design where we average over the third factor, and recreate the power analysis for the Two-Way interaction. For example, we can group over the Cognitive Load condition, and look at the Size by Color Interaction:

```{r}
string <- "2b*2b"
n <- 100
mu <- c(mean(c(1, 1)), mean(c(6, 1)), mean(c(6, 6)), mean(c(1, 6)))
sd <- 10
labelnames <- c("Size", "big", "small", "Color", "green", "red")
design_result <- ANOVA_design(design = string,
                   n = n, 
                   mu = mu, 
                   sd = sd, 
                   labelnames = labelnames)
```
```{r eval=FALSE}
simulation_result <- ANOVA_power(design_result, 
                                 alpha_level = alpha_level, 
                                 nsims = nsims,
                                 verbose = FALSE)
```

```{r echo=FALSE}
knitr::kable(simulation_result_5.5$main_results,
             caption = "Simulated ANOVA Result")%>%
  kable_styling(latex_options = "hold_position")
```


```{r}
exact_result <- ANOVA_exact(design_result,
                            alpha_level = alpha_level,
                            verbose = FALSE)
```


```{r echo=FALSE}
knitr::kable(exact_result$main_results,
             caption = "Exact ANOVA Result")%>%
  kable_styling(latex_options = "hold_position")
```



```{r}
# Power based on analytical solution
power_res <- power_twoway_between(design_result) 
power_res$power_A
power_res$power_B
power_res$power_AB
```



```{r}
string <- "2b*2b*2b"
n <- 50
mu <- c(5, 3, 2, 6, 1, 4, 3, 1) 
sd <- 10
r <- 0.0
labelnames <- c("Size", "big", "small", 
                "Color", "green", "red", 
                "CognitiveLoad", "present", "absent") 
design_result <- ANOVA_design(design = string,
                   n = n, 
                   mu = mu, 
                   sd = sd, 
                   labelnames = labelnames)
```
```{r eval=FALSE}
simulation_result <- ANOVA_power(design_result, 
                                 alpha_level = alpha_level, 
                                 nsims = nsims,
                                 verbose = FALSE)
```

```{r echo=FALSE}
knitr::kable(simulation_result_5.7$main_results,
             caption = "Simulated ANOVA Result")%>%
  kable_styling(latex_options = "hold_position")
```

```{r}
exact_result <- ANOVA_exact(design_result,
                            alpha_level = alpha_level,
                            verbose = FALSE)
```

```{r echo=FALSE}
knitr::kable(exact_result$main_results,
             caption = "Exact ANOVA Result")%>%
  kable_styling(latex_options = "hold_position")
```

```{r}
#Analytical power calculation
power_analytic <- power_threeway_between(design_result)
power_analytic$power_A
power_analytic$power_B
power_analytic$power_C
power_analytic$power_AB
power_analytic$power_AC
power_analytic$power_BC
power_analytic$power_ABC
power_analytic$eta_p_2_A
power_analytic$eta_p_2_B
power_analytic$eta_p_2_C
power_analytic$eta_p_2_AB
power_analytic$eta_p_2_AC
power_analytic$eta_p_2_BC
power_analytic$eta_p_2_ABC
```


The power for interactions depends on Cohen's *f*, the alpha level, the sample size, and the degrees of freedom. 

```{r}
# With 2x2x2 designs, 
# the names for paired comparisons can become very long. 
# So here the sample size abbreviate terms
# Size, Color, and Cognitive Load, have values:
# b = big, s = small, g = green, 
# r = red, pres = present, abs = absent.  
labelnames <- c("Size", "b", "s", "x", "Color", "g", "r", 
                "Load", "pres", "abs") #
design_result <- ANOVA_design(design = "3b*2b*2b", 
                              n = 15, 
                              mu = c(20, 0, 0, 0, 0, 
                                     0, 0, 0, 0, 0, 0, 20), 
                              
                              sd = 20, 
                              labelnames = labelnames) 

# Power based on exact simulations
exact_result <- ANOVA_exact(design_result,
                            verbose = FALSE)
```

```{r echo = FALSE}
knitr::kable(exact_result$main_results, 
             caption = "Exact ANOVA Result")%>%
  kable_styling(latex_options = "hold_position")
```

```{r}
#Analytical power calculation
power_analytic <- power_threeway_between(design_result)
power_analytic$power_A
power_analytic$power_B
power_analytic$power_C
power_analytic$power_AB
power_analytic$power_AC
power_analytic$power_BC
power_analytic$power_ABC
power_analytic$eta_p_2_A
power_analytic$Cohen_f_A
```

We see that a pattern of means of 0, 0, 0, 0, 0, 0, 0, 20 for a 2x2x2 interaction equals a Cohen's *f* of 0.25.  

```{r}
labelnames <- c("Size", "b", "s", "Color", "g", "r")
design_result <- ANOVA_design(design = "2b*2b", 
                              n = 10, 
                              mu = c(0, 0, 0, 10), 
                              sd = 10, 
                              labelnames = labelnames) 

# Power based on exact simulations
exact_result <- ANOVA_exact(design_result,
                            verbose = FALSE)
```

```{r echo = FALSE}
knitr::kable(exact_result$main_results, 
             caption = "Exact ANOVA Result")%>%
  kable_styling(latex_options = "hold_position")
```

```{r}
#Analytical power calculation
power_analytic <- power_twoway_between(design_result)
power_analytic$power_A
power_analytic$eta_p_2_A
power_analytic$Cohen_f_A
```

Cohen's *f* is twice as large for a 2x2 design with the same mean value in one of four cells. In a 2 factor between design. 

```{r}
labelnames <- c("Size", "b", "s")
design_result <- ANOVA_design(design = "2b", 
                              n = 10,  
                              mu = c(0, 5), 
                              sd = 10, 
                              labelnames = labelnames) 

# Power based on exact simulations
exact_result <- ANOVA_exact(design_result,
                            verbose = FALSE)
```

```{r echo = FALSE}
knitr::kable(exact_result$main_results, 
             caption = "Exact ANOVA Result")%>%
  kable_styling(latex_options = "hold_position")
```


```{r }
#Analytical power calculation
power_analytic <- power_oneway_between(design_result)
power_analytic$power
power_analytic$eta_p_2
power_analytic$Cohen_f
```