Improve error message in `add_difference()` when test does not contain a difference estimate

[tombmiller]

I noticed that add_difference() with "paired.wilcox.test" and "mcnemar.test" returns some interesting error messages. Both tests work with add_p(), but add_difference() seems to not work.

Here is a reproducible example using the paired test example from the FAQs that shows the error messages I got:

library(gtsummary)

trial_paired <-
  trial |> 
  select(trt, marker, response) |> 
  mutate(.by = trt, id = dplyr::row_number()) |>
  tidyr::drop_na() |> 
  dplyr::filter(.by = id, dplyr::n() == 2)

trial_paired |> 
  tbl_summary(
    by = trt, 
    include = marker) |> 
  add_difference(
    test = list(marker ~ "paired.wilcox.test"),
    group = id
  )
#> Error in `add_difference()`:
#> ! Could not process `test` argument for variable "marker".
#> ℹ See `?gtsummary::tests()` for details on constructing custom tests. See error
#>   message below:
#> ✖ object 'paired.wilcox.test' not found

trial_paired |> 
  tbl_summary(
    by = trt, 
    include = response) |> 
  add_difference(
    test = list(response ~ "mcnemar.test"),
    group = id
  )
#> The following errors were returned during `add_difference()`:
#> ✖ For variable `response` (`trt`) and "estimate", "std.error", "parameter",
#>   "statistic", "conf.low", "conf.high", and "p.value" statistics: unused
#>   arguments (data = list(c("Drug A", "Drug B", "Drug A", "Drug A", "Drug A",
#>   "Drug B", "Drug A", "Drug A", "Drug A", "Drug B", "Drug B", "Drug B", "Drug
#>   B", "Drug B", "Drug B", "Drug B", "Drug A", "Drug B", "Drug A", "Drug A",
#>   "Drug A", "Drug A", "Drug A", "Drug A", "Drug B", "Drug B", "Drug A", "Drug
#>   B", "Drug B", "Drug A", "Drug B", "Drug A", "Drug B", "Drug A", "Drug A",
#>   "Drug B", "Drug B", "Drug B", "Drug A", "Drug B", "Drug A", "Drug B", "Drug
#>   A", "Drug B", "Drug B", "Drug B", "Drug B", "Drug B", "Drug B", "Drug B",
#>   "Drug B", "Drug B", "Drug A", "Drug B", "Drug B", "Drug A", "Drug A", "Drug
#>   A", "Drug B", "Drug B", "Drug B", "Drug B", "Drug A", "Drug A", "Drug A",
#>   "Drug B", "Drug A", "Drug A", "Drug A", "Drug B", "Drug B", "Drug A", "Drug
#>   A", "Drug B", "Drug A", "Drug A", "Drug A", "Drug A", "Drug A", "Drug B",
#>   "Drug B", "Drug B", "Drug A", "Drug B", "Drug B", "Drug B", "Drug B", "Drug
#>   A", "Drug A", "Drug B", "Drug B", "Drug A", "Drug A", "Drug B", "Drug B",
#>   "Drug A", "Drug A", "Drug B", "Drug A", "Drug A", "Drug A", "Drug B", "Drug
#>   A", "Drug A", "Drug B", "Drug A", "Drug B", "Drug B", "Drug A", "Drug B",
#>   "Drug A", "Drug B", "Drug B", "Drug B", "Drug B", "Drug A", "Drug B", "Drug
#>   A", "Drug A", "Drug A", "Drug B", "Drug A", "Drug B", "Drug A", "Drug B",
#>   "Drug B", "Drug A", "Drug B", "Drug A", "Drug B", "Drug A", "Drug A", "Drug
#>   A", "Drug B", "Drug A", "Drug B", "Drug A", "Drug B", "Drug B", "Drug B",
#>   "Drug B", "Drug A", "Drug B", "Drug B", "Drug B", "Drug A", "Drug A", "Drug
#>   A", "Drug A", "Drug B", "Drug B", "Drug B", "Drug A", "Drug A", "Drug A",
#>   "Drug B", "Drug B", "Drug B", "Drug A", "Drug A", "Drug A", "Drug A", "Drug
#>   A", "Drug A", "Drug A", "Drug A"), c(0.16, 1.107, 0.277, 2.067, 2.767, 0.613,
#>   0.354, 1.739, 0.144, 0.205, 0.513, 0.06, 0.831, 0.258, 0.128, 0.445, 2.083,
#>   0.157, 0.066, 0.325, 0.266, 0.719, 1.713, 0.096, 0.105, 0.043, 0.981, 1.156,
#>   0.105, 0.175, 0.309, 1.869, 2.008, 1.894, 0.16, 0.108, 0.222, 0.803, 0.177,
#>   1.479, 0.161, 0.737, 0.124, 0.092, 0.385, 0.21, 0.475, 1.628, 0.583, 0.702,
#>   1.206, 2.213, 1.406, 0.101, 0.013, 2.032, 1.046, 0.408, 2.636, 2.447, 1.041,
#>   0.531, 0.924, 0.733, 2.157, 0.333, 1.527, 2.238, 0.153, 0.305, 0.131, 0.386,
#>   0.229, 0.615, 1.976, 1.941, 0.22, 3.874, 0.982, 1.68, 1.091, 0.169, 0.511,
#>   2.141, 0.599, 0.389, 0.005, 0.075, 1.491, 0.358, 1.709, 1.354, 2.522, 0.387,
#>   0.592, 0.243, 1.207, 0.29, 0.718, 0.589, 0.003, 3.249, 0.039, 1.804, 0.238,
#>   2.702, 1.441, 0.27, 0.062, 2.19, 0.976, 3.062, 0.124, 0.045, 1.892, 0.711,
#>   1.079, 1.061, 0.239, 0.361, 1.133, 1.225, 1.418, 3.751, 3.02, 0.086, 0.772,
#>   1.882, 2.725, 0.352, 0.895, 0.141, 2.288, 1.658, 1.255, 1.306, 0.081, 0.667,
#>   0.046, 0.662, 1.985, 1.55, 0.015, 0.056, 0.51, 0.929, 2.345, 0.816, 0.022,
#>   0.16, 0.547, 3.642, 0.092, 2.124, 0.862, 0.182, 1.075, 0.021, 1.129, 0.61,
#>   0.717, 0.205, 0.946, 0.136, 0.439, 1.148), c(0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0,
#>   1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0,
#>   0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 1, 0, 0,
#>   0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0,
#>   0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0,
#>   0, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0,
#>   1, 0, 0, 1, 0, 1, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0),
#>   c(1, 1, 2, 3, 4, 2, 5, 6, 7, 3, 4, 5, 6, 7, 8, 9, 8, 10, 9, 10, 11, 12, 13,
#>   14, 11, 12, 15, 13, 14, 16, 15, 17, 16, 18, 19, 17, 18, 19, 22, 22, 23, 23,
#>   24, 24, 25, 26, 27, 28, 29, 31, 32, 33, 25, 34, 35, 26, 27, 28, 36, 39, 40,
#>   41, 29, 31, 32, 42, 33, 34, 35, 43, 44, 36, 39, 45, 40, 41, 42, 43, 44, 46,
#>   47, 48, 45, 49, 50, 52, 53, 46, 47, 54, 55, 48, 49, 57, 58, 50, 52, 59, 53,
#>   54, 55, 62, 57, 58, 63, 59, 64, 65, 62, 66, 63, 67, 68, 69, 70, 64, 71, 65,
#>   66, 67, 73, 68, 74, 69, 75, 76, 70, 77, 71, 78, 73, 74, 75, 80, 76, 81, 77,
#>   82, 83, 84, 85, 78, 87, 88, 89, 80, 81, 82, 83, 91, 92, 93, 84, 85, 87, 96,
#>   97, 98, 88, 89, 91, 92, 93, 96, 97, 98)), variable = "response", by = "trt",
#>   group = "id", type = "dichotomous", test.args = NULL, adj.vars =
#>   character(0), conf.level = 0.95, tbl = list( list("response", NA,
#>   "dichotomous", "label", "Tumor Response", "Tumor Response", "21 (25%)", "28
#>   (34%)"), list(list(c("variable", "var_type", "row_type", "var_label",
#>   "label", "stat_1", "stat_2"), c(TRUE, TRUE, TRUE, TRUE, FALSE, FALSE, FALSE),
#>   c("center", "center", "center", "center", "left", "center", "center"),
#>   c("gt::md", "gt::md", "gt::md", "gt::md", "gt::md", "gt::md", "gt::md"),
#>   c("variable", "var_type", "row_type", "var_label", "**Characteristic**",
#>   "**Drug A** \nN = 83", "**Drug B** \nN = 83" ), c(NA, NA, NA, NA, NA, "Drug
#>   A", "Drug B"), c(166, 166, 166, 166, 166, 166, 166), c(NA, NA, NA, NA, NA,
#>   83, 83), c(NA, NA, NA, NA, NA, 0.5, 0.5)), list(integer(0), character(0),
#>   character(0), character(0), logical(0)), list(c("stat_1", "stat_2"), c("n
#>   (%)", "n (%)"), c("gt::md", "gt::md"), c(TRUE, TRUE), c(FALSE, FALSE)),
#>   list(character(0), list(), character(0), character(0), logical(0),
#>   logical(0)), list(character(0), character(0), integer(0), character(0),
#>   logical(0), logical(0)), list(character(0), character(0), character(0)),
#>   list(integer(0), character(0), character(0), logical(0)), list(character(0),
#>   list(), character(0), logical(0)), list(c("label", "label"), list(TRUE,
#>   ~.data$row_type %in% c("level", "missing")), c(0, 4)), list(character(0),
#>   list(), character(0)), list(character(0), list(), list()), list(character(0),
#>   list(), character(0)), list(character(0), list(), list())),
#>   list(tbl_summary(data = trial_paired, by = trt, include = response)),
#>   list(list(c("trt", "trt", "trt", "trt", "trt", "trt", NA, NA, NA, NA, "trt",
#>   "trt", "trt", "trt", "trt", "trt", "trt", "trt", "trt", "trt", NA, NA, NA,
#>   NA, NA, NA, NA), list("Drug A", "Drug A", "Drug A", "Drug B", "Drug B", "Drug
#>   B", NULL, NULL, NULL, NULL, "Drug A", "Drug A", "Drug A", "Drug A", "Drug A",
#>   "Drug B", "Drug B", "Drug B", "Drug B", "Drug B", NULL, NULL, NULL, NULL,
#>   NULL, NULL, NULL), c("response", "response", "response", "response",
#>   "response", "response", "response", "response", "trt", "trt", "response",
#>   "response", "response", "response", "response", "response", "response",
#>   "response", "response", "response", "..ard_total_n..", "trt", "trt", "trt",
#>   "trt", "trt", "trt"), list(1, 1, 1, 1, 1, 1, NULL, NULL, NULL, NULL, NULL,
#>   NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, "Drug A", "Drug
#>   A", "Drug A", "Drug B", "Drug B", "Drug B"), c("dichotomous", "dichotomous",
#>   "dichotomous", "dichotomous", "dichotomous", "dichotomous", "attributes",
#>   "attributes", "attributes", "attributes", "missing", "missing", "missing",
#>   "missing", "missing", "missing", "missing", "missing", "missing", "missing",
#>   "total_n", "categorical", "categorical", "categorical", "categorical",
#>   "categorical", "categorical"), c("n", "N", "p", "n", "N", "p", "label",
#>   "class", "label", "class", "N_obs", "N_miss", "N_nonmiss", "p_miss",
#>   "p_nonmiss", "N_obs", "N_miss", "N_nonmiss", "p_miss", "p_nonmiss", "N", "n",
#>   "N", "p", "n", "N", "p"), c("n", "N", "%", "n", "N", "%", "Variable Label",
#>   "Variable Class", "Variable Label", "Variable Class", "No. obs.", "N
#>   Missing", "N Non-missing", "% Missing", "% Non-missing", "No. obs.", "N
#>   Missing", "N Non-missing", "% Missing", "% Non-missing", "N", "n", "N", "%",
#>   "n", "N", "%"), list(21, 83, 0.253012048192771, 28, 83, 0.337349397590361,
#>   "Tumor Response", "integer", "Chemotherapy Treatment", "character", 83, 0,
#>   83, 0, 1, 83, 0, 83, 0, 1, 166, 83, 166, 0.5, 83, 166, 0.5), list(function
#>   (x) style_number(x, digits = digits, big.mark = big.mark, decimal.mark =
#>   decimal.mark, scale = scale, prefix = prefix, suffix = suffix, na = na, ...),
#>   function (x) style_number(x, digits = digits, big.mark = big.mark,
#>   decimal.mark = decimal.mark, scale = scale, prefix = prefix, suffix = suffix,
#>   na = na, ...), function (x) style_percent(x, prefix = prefix, suffix =
#>   suffix, digits = digits, big.mark = big.mark, decimal.mark = decimal.mark, na
#>   = na, ...), function (x) style_number(x, digits = digits, big.mark =
#>   big.mark, decimal.mark = decimal.mark, scale = scale, pref

Characteristic	Drug A N = 831	Drug B N = 831	Difference	95% CI	p-value
Tumor Response	21 (25%)	28 (34%)
Abbreviation: CI = Confidence Interval
1 n (%)

^{Created on 2025-08-26 with reprex v2.1.1}

[ddsjoberg]

Hi @tombmiller , thanks for the post.

Certainly, these messages can be improved upon, oof! 😆

The underlying issue is that the paired Wilcox test doesn't return a difference estimate, so the add_difference() function doesn't have anything to return.

library(gtsummary)

trial_paired <-
  trial |> 
  dplyr::select(trt, marker, response) |> 
  dplyr::mutate(.by = trt, id = dplyr::row_number()) |>
  tidyr::drop_na() |> 
  dplyr::filter(.by = id, dplyr::n() == 2)

trial_paired |> 
  tidyr::pivot_wider(
    id_cols = c(id),
    names_from = trt,
    values_from = marker
  ) |> 
  with(wilcox.test(x = `Drug A`, y = `Drug B`)) |> 
  broom::tidy()
#> # A tibble: 1 × 4
#>   statistic p.value method                                           alternative
#>       <dbl>   <dbl> <chr>                                            <chr>      
#> 1     3880.   0.161 Wilcoxon rank sum test with continuity correcti… two.sided

^{Created on 2025-08-26 with reprex v2.1.1}

[tombmiller]

Ah, I see, that makes total sense. Thanks very much for your response!

(And, thank you for all your work on this incredible package. I use it practically every day and it has made my life a million times easier.)