Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

estimate_contrasts() ignores transform = "" input #210

Open
goesjon opened this issue Aug 30, 2022 · 20 comments
Open

estimate_contrasts() ignores transform = "" input #210

goesjon opened this issue Aug 30, 2022 · 20 comments

Comments

@goesjon
Copy link

goesjon commented Aug 30, 2022

Hello all,

I have an issue that maybe you can help me with. Whenever I have a model with log-transformed response e.g.,

model <- lm(log(Sepal.Width) ~ Species, data = iris)

the estimate_contrasts() function ignores the transform input:

estimate_contrasts(model, transform = "none")
estimate_contrasts(model, transform = "response")

the estimate_means() function on the other hand considers the transform input as desired:

estimate_means(model, transform = "none")
estimate_means(model, transform = "response")

Am I doing something wrong or did I misunderstood the transform input? How do I get the contrasts of the back-transformed response? Of course I could just manually subtract the estimated means but that is a lot of work when I have a lot of factor levels.

I should have everything up to date:
R version 4.2.1 (2022-06-23 ucrt) -- "Funny-Looking Kid"
RStudio 2022.07.1 Build 554
modelbased 0.8.5

Thank you very much.

Best regards

@DominiqueMakowski
Copy link
Member

Thanks for reporting, let me give a look asap

@IndrajeetPatil
Copy link
Member

Possibly related to (fixed by?) #204

@DominiqueMakowski
Copy link
Member

Possibly related to easystats/insight#584

@goesjon
Copy link
Author

goesjon commented Sep 21, 2022

Possibly related to (fixed by?) #204

The problem still persists (not fixed)

@bwiernik
Copy link
Contributor

We really should solve these issues by transitioning the function to use marginaleffects instead of emmeans

@DominiqueMakowski @vincentarelbundock

@vincentarelbundock
Copy link
Contributor

One thing to keep in mind is that marginaleffects doesn't do anything to variables which are transformed in the formula. In the example above, make predictions on the response scale would make predictions on the logged response scale. In that respect, the package follows the practice of all predict() methods. I'm not sure if this is an obstacle...

I'm more than happy to help out by answering questions, and by trying to replicate emmeans/modelbased code. However, I don't want to promise too much help with actual coding, since I feel the need to scale back my open source contributions a little bit for quality of life / mental health reasons.

@IndrajeetPatil
Copy link
Member

I feel the need to scale back my open source contributions a little bit for quality of life / mental health reasons

I feel ya. I also want to do this, but nothing short of burning my laptop seems to work.

@strengejacke
Copy link
Member

Not sure what the expected output would be, but I get different results, suggesting that it works now?

library(easystats)
#> # Attaching packages: easystats 0.5.2.8
#> ✔ bayestestR  0.13.0     ✔ correlation 0.8.2.4 
#> ✔ datawizard  0.6.0.1    ✔ effectsize  0.7.9.1 
#> ✔ insight     0.18.4.3   ✔ modelbased  0.8.5   
#> ✔ performance 0.9.2.4    ✔ parameters  0.18.2.9
#> ✔ report      0.5.5.1    ✔ see         0.7.3.1
model <- lm(log(Sepal.Width) ~ Species, data = iris)

estimate_contrasts(model, transform = "none")
#> No variable was specified for contrast estimation. Selecting `contrast = "Species"`.
#> Marginal Contrasts Analysis
#> 
#> Level1     |     Level2 | Difference |         95% CI |   SE | t(147) |      p
#> ------------------------------------------------------------------------------
#> setosa     | versicolor |       0.21 | [ 0.16,  0.27] | 0.02 |   9.47 | < .001
#> setosa     |  virginica |       0.14 | [ 0.09,  0.20] | 0.02 |   6.28 | < .001
#> versicolor |  virginica |      -0.07 | [-0.13, -0.02] | 0.02 |  -3.19 | 0.002 
#> 
#> Marginal contrasts estimated at Species
#> p-value adjustment method: Holm (1979)
estimate_contrasts(model, transform = "response")
#> No variable was specified for contrast estimation. Selecting `contrast = "Species"`.
#> Marginal Contrasts Analysis
#> 
#> Level1     |     Level2 | Ratio |       95% CI |   SE | t(147) |      p
#> -----------------------------------------------------------------------
#> setosa     | versicolor |  1.24 | [1.17, 1.31] | 0.03 |   9.47 | < .001
#> setosa     |  virginica |  1.15 | [1.09, 1.22] | 0.03 |   6.28 | < .001
#> versicolor |  virginica |  0.93 | [0.88, 0.98] | 0.02 |  -3.19 | 0.002 
#> 
#> Marginal contrasts estimated at Species
#> p-value adjustment method: Holm (1979)

Created on 2022-09-22 with reprex v2.0.2

@goesjon
Copy link
Author

goesjon commented Sep 22, 2022

@strengejacke thank you very much for your comment. Your result is indeed the desired result, however, I am not able to reproduce it (see code below). I noticed, that some of your easystats package versions are different from mine. I updated with remotes::install_github("easystats/easystats") prior to running the code below, I am not able to get to your versions. In any case, the modelbased package is up-to-date (same version to yours), I do not understand why my output is different to yours.

library(easystats)
#> # Attaching packages: easystats 0.5.2.8
#> ✔ bayestestR  0.13.0    ✔ correlation 0.8.2  
#> ✔ datawizard  0.6.0     ✔ effectsize  0.7.0.5
#> ✔ insight     0.18.4    ✔ modelbased  0.8.5  
#> ✔ performance 0.9.2     ✔ parameters  0.18.2 
#> ✔ report      0.5.5     ✔ see         0.7.3
model <- lm(log(Sepal.Width) ~ Species, data = iris)
estimate_contrasts(model, transform = "none")
#> No variable was specified for contrast estimation. Selecting `contrast = "Species"`.
#> Marginal Contrasts Analysis
#> 
#> Level1     |     Level2 | Difference |         95% CI |   SE | t(147) |      p
#> ------------------------------------------------------------------------------
#> setosa     | versicolor |       0.21 | [ 0.16,  0.27] | 0.02 |   9.47 | < .001
#> setosa     |  virginica |       0.14 | [ 0.09,  0.20] | 0.02 |   6.28 | < .001
#> versicolor |  virginica |      -0.07 | [-0.13, -0.02] | 0.02 |  -3.19 | 0.005 
#> 
#> Marginal contrasts estimated at Species
#> p-value adjustment method: Holm (1979)
estimate_contrasts(model, transform = "response")
#> No variable was specified for contrast estimation. Selecting `contrast = "Species"`.
#> Marginal Contrasts Analysis
#> 
#> Level1     |     Level2 | Difference |         95% CI |   SE | t(147) |      p
#> ------------------------------------------------------------------------------
#> setosa     | versicolor |       0.21 | [ 0.16,  0.27] | 0.02 |   9.47 | < .001
#> setosa     |  virginica |       0.14 | [ 0.09,  0.20] | 0.02 |   6.28 | < .001
#> versicolor |  virginica |      -0.07 | [-0.13, -0.02] | 0.02 |  -3.19 | 0.005 
#> 
#> Marginal contrasts estimated at Species
#> p-value adjustment method: Holm (1979)
Created on 2022-09-22 with reprex v2.0.2.9000

@IndrajeetPatil
Copy link
Member

Run easystats::install_latest(), refresh, and try again. It should work.

@goesjon
Copy link
Author

goesjon commented Sep 22, 2022

@IndrajeetPatil thank you very much for your comment. With the suggested code I was able to update my easystats packages. I suggest to add this code to the easystats homepage in the Installation section.

Unfortunately, also with the updated packages I am unable to reproduce the desired result:

library(easystats)
#> # Attaching packages: easystats 0.5.2.8
#> ✔ bayestestR  0.13.0     ✔ correlation 0.8.2.4 
#> ✔ datawizard  0.6.0.1    ✔ effectsize  0.7.9.1 
#> ✔ insight     0.18.4.3   ✔ modelbased  0.8.5   
#> ✔ performance 0.9.2.4    ✔ parameters  0.18.2.9
#> ✔ report      0.5.5.1    ✔ see         0.7.3.1
model <- lm(log(Sepal.Width) ~ Species, data = iris)
estimate_contrasts(model, transform = "none")
#> No variable was specified for contrast estimation. Selecting `contrast = "Species"`.
#> Marginal Contrasts Analysis
#> 
#> Level1     |     Level2 | Difference |         95% CI |   SE | t(147) |      p
#> ------------------------------------------------------------------------------
#> setosa     | versicolor |       0.21 | [ 0.16,  0.27] | 0.02 |   9.47 | < .001
#> setosa     |  virginica |       0.14 | [ 0.09,  0.20] | 0.02 |   6.28 | < .001
#> versicolor |  virginica |      -0.07 | [-0.13, -0.02] | 0.02 |  -3.19 | 0.005 
#> 
#> Marginal contrasts estimated at Species
#> p-value adjustment method: Holm (1979)
estimate_contrasts(model, transform = "response")
#> No variable was specified for contrast estimation. Selecting `contrast = "Species"`.
#> Marginal Contrasts Analysis
#> 
#> Level1     |     Level2 | Difference |         95% CI |   SE | t(147) |      p
#> ------------------------------------------------------------------------------
#> setosa     | versicolor |       0.21 | [ 0.16,  0.27] | 0.02 |   9.47 | < .001
#> setosa     |  virginica |       0.14 | [ 0.09,  0.20] | 0.02 |   6.28 | < .001
#> versicolor |  virginica |      -0.07 | [-0.13, -0.02] | 0.02 |  -3.19 | 0.005 
#> 
#> Marginal contrasts estimated at Species
#> p-value adjustment method: Holm (1979)
Created on 2022-09-22 with reprex v2.0.2.9000

@DominiqueMakowski
Copy link
Member

I'm going to try now to add marginaleffects as a backend for estimate_contrasts / estimate_means 🤞

@goesjon
Copy link
Author

goesjon commented Sep 28, 2022

Hello everyone,

thank you very much so far for your comments and help. I did a complet re-installation of R, RTools and RStudio following this guideline:

https://stackoverflow.com/questions/55204017/how-to-uninstall-r-and-rstudio-with-all-packages-settings-and-everything-else

I also installed every package again, including easystats (including the easystats::install_latest() command).

I did NOT get the result of @strengejacke . I seriously do not understand. @strengejacke are you using Linux or Mac, because I am using Windows 10? Are you using a strange "for developers only" version of easystats or any other package, which I am not able to get?

I am really desperate at this point ...

How about the others in this issue? Do you also get @strengejacke 's result? Am I the only one who does not get the desired result?

@DominiqueMakowski
Copy link
Member

DominiqueMakowski commented Sep 28, 2022

This is what I get: (you can see my versions of packages installed) - what version of emmeans are you using (mine is emmeans_1.7.4-1)?

library(easystats)
#> # Attaching packages: easystats 0.4.3 (red = needs update)
#> ✖ insight     0.18.3.2   ✖ datawizard  0.6.0.1 
#> ✔ bayestestR  0.13.0     ✔ performance 0.9.2.4 
#> ✔ parameters  0.18.2.7   ✔ effectsize  0.7.9.1 
#> ✔ modelbased  0.8.5      ✔ correlation 0.8.2.4 
#> ✔ see         0.7.3      ✔ report      0.5.5.1 
#> 
#> Restart the R-Session and update packages in red with 'easystats::easystats_update()'.
model <- lm(log(Sepal.Width) ~ Species, data = iris)

estimate_contrasts(model, transform = "none")
#> No variable was specified for contrast estimation. Selecting `contrast = "Species"`.
#> Marginal Contrasts Analysis
#> 
#> Level1     |     Level2 | Difference |         95% CI |   SE | t(147) |      p
#> ------------------------------------------------------------------------------
#> setosa     | versicolor |       0.21 | [ 0.16,  0.27] | 0.02 |   9.47 | < .001
#> setosa     |  virginica |       0.14 | [ 0.09,  0.20] | 0.02 |   6.28 | < .001
#> versicolor |  virginica |      -0.07 | [-0.13, -0.02] | 0.02 |  -3.19 | 0.002 
#> 
#> Marginal contrasts estimated at Species
#> p-value adjustment method: Holm (1979)
estimate_contrasts(model, transform = "response")
#> No variable was specified for contrast estimation. Selecting `contrast = "Species"`.
#> Marginal Contrasts Analysis
#> 
#> Level1     |     Level2 | Ratio |       95% CI |   SE | t(147) |      p
#> -----------------------------------------------------------------------
#> setosa     | versicolor |  1.24 | [1.17, 1.31] | 0.03 |   9.47 | < .001
#> setosa     |  virginica |  1.15 | [1.09, 1.22] | 0.03 |   6.28 | < .001
#> versicolor |  virginica |  0.93 | [0.88, 0.98] | 0.02 |  -3.19 | 0.002 
#> 
#> Marginal contrasts estimated at Species
#> p-value adjustment method: Holm (1979)

Created on 2022-09-28 by the reprex package (v2.0.1)

@goesjon
Copy link
Author

goesjon commented Sep 28, 2022

The result (see below) looks exactly like in my previous comments. The result indicates, that estimate_contrasts() still ignores the transform input.

I am using a fresh install of everything (as mentioned before) with the following versions:

  • R version 4.2.1 (2022-06-23 ucrt) -- "Funny-Looking Kid"
  • RStudio 2022.07.2+576 "Spotted Wakerobin" Release (e7373ef832b49b2a9b88162cfe7eac5f22c40b34, 2022-09-06) for Windows
  • Rtools 4.2
  • emmeans 1.8.1-1
  • Windows 10 20H2
  • for the easystats versions see below

What is going wrong, I seriously do not get it?????

library(easystats)
#> # Attaching packages: easystats 0.5.2.8
#> ✔ bayestestR  0.13.0       ✔ correlation 0.8.2.4   
#> ✔ datawizard  0.6.1.1      ✔ effectsize  0.7.9.1999
#> ✔ insight     0.18.4.5     ✔ modelbased  0.8.5     
#> ✔ performance 0.9.2.4      ✔ parameters  0.18.2.9  
#> ✔ report      0.5.5.1      ✔ see         0.7.3.1
model <- lm(log(Sepal.Width) ~ Species, data = iris)
estimate_contrasts(model, transform = "none")
#> No variable was specified for contrast estimation. Selecting `contrast = "Species"`.
#> Marginal Contrasts Analysis
#> 
#> Level1     |     Level2 | Difference |         95% CI |   SE | t(147) |      p
#> ------------------------------------------------------------------------------
#> setosa     | versicolor |       0.21 | [ 0.16,  0.27] | 0.02 |   9.47 | < .001
#> setosa     |  virginica |       0.14 | [ 0.09,  0.20] | 0.02 |   6.28 | < .001
#> versicolor |  virginica |      -0.07 | [-0.13, -0.02] | 0.02 |  -3.19 | 0.005 
#> 
#> Marginal contrasts estimated at Species
#> p-value adjustment method: Holm (1979)
estimate_contrasts(model, transform = "response")
#> No variable was specified for contrast estimation. Selecting `contrast = "Species"`.
#> Marginal Contrasts Analysis
#> 
#> Level1     |     Level2 | Difference |         95% CI |   SE | t(147) |      p
#> ------------------------------------------------------------------------------
#> setosa     | versicolor |       0.21 | [ 0.16,  0.27] | 0.02 |   9.47 | < .001
#> setosa     |  virginica |       0.14 | [ 0.09,  0.20] | 0.02 |   6.28 | < .001
#> versicolor |  virginica |      -0.07 | [-0.13, -0.02] | 0.02 |  -3.19 | 0.005 
#> 
#> Marginal contrasts estimated at Species
#> p-value adjustment method: Holm (1979)
Created on 2022-09-28 with reprex v2.0.2

@strengejacke
Copy link
Member

This now includes my session info:

library(easystats)
#> # Attaching packages: easystats 0.5.2.2
#> ✔ insight     0.18.4.5     ✔ datawizard  0.6.1.1   
#> ✔ bayestestR  0.13.0       ✔ performance 0.9.2.4   
#> ✔ parameters  0.18.2.9     ✔ effectsize  0.7.9.1999
#> ✔ modelbased  0.8.5        ✔ correlation 0.8.2.4   
#> ✔ see         0.7.3.1      ✔ report      0.5.5.1
model <- lm(log(Sepal.Width) ~ Species, data = iris)

estimate_contrasts(model, transform = "none")
#> No variable was specified for contrast estimation. Selecting `contrast = "Species"`.
#> Marginal Contrasts Analysis
#> 
#> Level1     |     Level2 | Difference |         95% CI |   SE | t(147) |      p
#> ------------------------------------------------------------------------------
#> setosa     | versicolor |       0.21 | [ 0.16,  0.27] | 0.02 |   9.47 | < .001
#> setosa     |  virginica |       0.14 | [ 0.09,  0.20] | 0.02 |   6.28 | < .001
#> versicolor |  virginica |      -0.07 | [-0.13, -0.02] | 0.02 |  -3.19 | 0.002 
#> 
#> Marginal contrasts estimated at Species
#> p-value adjustment method: Holm (1979)

estimate_contrasts(model, transform = "response")
#> No variable was specified for contrast estimation. Selecting `contrast = "Species"`.
#> Marginal Contrasts Analysis
#> 
#> Level1     |     Level2 | Ratio |       95% CI |   SE | t(147) |      p
#> -----------------------------------------------------------------------
#> setosa     | versicolor |  1.24 | [1.17, 1.31] | 0.03 |   9.47 | < .001
#> setosa     |  virginica |  1.15 | [1.09, 1.22] | 0.03 |   6.28 | < .001
#> versicolor |  virginica |  0.93 | [0.88, 0.98] | 0.02 |  -3.19 | 0.002 
#> 
#> Marginal contrasts estimated at Species
#> p-value adjustment method: Holm (1979)
Session Info
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.2.1 (2022-06-23 ucrt)
#>  os       Windows 10 x64 (build 22000)
#>  system   x86_64, mingw32
#>  ui       RTerm
#>  language (EN)
#>  collate  German_Germany.utf8
#>  ctype    German_Germany.utf8
#>  tz       Europe/Berlin
#>  date     2022-09-28
#>  pandoc   2.18 @ C:/Users/mail/AppData/Local/Pandoc/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package      * version    date (UTC) lib source
#>  assertthat     0.2.1      2019-03-21 [1] CRAN (R 4.2.0)
#>  bayestestR   * 0.13.0     2022-09-18 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  cli            3.4.1      2022-09-23 [1] CRAN (R 4.2.1)
#>  coda           0.19-4     2020-09-30 [1] CRAN (R 4.2.0)
#>  codetools      0.2-18     2020-11-04 [2] CRAN (R 4.2.1)
#>  colorspace     2.0-3      2022-02-21 [1] CRAN (R 4.2.0)
#>  correlation  * 0.8.2.4    2022-09-13 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  datawizard   * 0.6.1.1    2022-09-25 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  DBI            1.1.3      2022-06-18 [1] CRAN (R 4.2.0)
#>  digest         0.6.29     2021-12-01 [1] CRAN (R 4.2.0)
#>  dplyr          1.0.10     2022-09-01 [1] CRAN (R 4.2.1)
#>  easystats    * 0.5.2.2    2022-08-31 [1] local
#>  effectsize   * 0.7.9.1999 2022-09-28 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  emmeans        1.8.1-1    2022-09-08 [1] CRAN (R 4.2.1)
#>  estimability   1.4.1      2022-08-05 [1] CRAN (R 4.2.1)
#>  evaluate       0.16       2022-08-09 [1] CRAN (R 4.2.1)
#>  fansi          1.0.3      2022-03-24 [1] CRAN (R 4.2.0)
#>  fastmap        1.1.0      2021-01-25 [1] CRAN (R 4.2.0)
#>  fs             1.5.2      2021-12-08 [1] CRAN (R 4.2.0)
#>  generics       0.1.3      2022-07-05 [1] CRAN (R 4.2.1)
#>  ggplot2        3.3.6      2022-05-03 [1] CRAN (R 4.2.0)
#>  glue           1.6.2      2022-02-24 [1] CRAN (R 4.2.0)
#>  gtable         0.3.1      2022-09-01 [1] CRAN (R 4.2.1)
#>  highr          0.9        2021-04-16 [1] CRAN (R 4.2.0)
#>  htmltools      0.5.3      2022-07-18 [1] CRAN (R 4.2.1)
#>  insight      * 0.18.4.5   2022-09-27 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  knitr          1.40       2022-08-24 [1] CRAN (R 4.2.1)
#>  lattice        0.20-45    2021-09-22 [1] CRAN (R 4.2.0)
#>  lifecycle      1.0.2      2022-09-09 [1] CRAN (R 4.2.1)
#>  magrittr       2.0.3      2022-03-30 [1] CRAN (R 4.2.0)
#>  MASS           7.3-58.1   2022-08-03 [1] CRAN (R 4.2.1)
#>  Matrix         1.5-1      2022-09-13 [1] CRAN (R 4.2.1)
#>  modelbased   * 0.8.5      2022-09-26 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  multcomp       1.4-20     2022-08-07 [1] CRAN (R 4.2.1)
#>  munsell        0.5.0      2018-06-12 [1] CRAN (R 4.2.0)
#>  mvtnorm        1.1-3      2021-10-08 [1] CRAN (R 4.2.0)
#>  parameters   * 0.18.2.9   2022-09-27 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  performance  * 0.9.2.4    2022-09-28 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  pillar         1.8.1      2022-08-19 [1] CRAN (R 4.2.1)
#>  pkgconfig      2.0.3      2019-09-22 [1] CRAN (R 4.2.0)
#>  purrr          0.3.4      2020-04-17 [1] CRAN (R 4.2.0)
#>  R.cache        0.16.0     2022-07-21 [1] CRAN (R 4.2.1)
#>  R.methodsS3    1.8.2      2022-06-13 [1] CRAN (R 4.2.0)
#>  R.oo           1.25.0     2022-06-12 [1] CRAN (R 4.2.0)
#>  R.utils        2.12.0     2022-06-28 [1] CRAN (R 4.2.0)
#>  R6             2.5.1      2021-08-19 [1] CRAN (R 4.2.1)
#>  report       * 0.5.5.1    2022-09-12 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  reprex         2.0.2      2022-08-17 [1] CRAN (R 4.2.1)
#>  rlang          1.0.6      2022-09-24 [1] CRAN (R 4.2.1)
#>  rmarkdown      2.16       2022-08-24 [1] CRAN (R 4.2.1)
#>  rstudioapi     0.14       2022-08-22 [1] CRAN (R 4.2.1)
#>  sandwich       3.0-2      2022-06-15 [1] CRAN (R 4.2.0)
#>  scales         1.2.1      2022-08-20 [1] CRAN (R 4.2.1)
#>  see          * 0.7.3.1    2022-09-27 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  sessioninfo    1.2.2      2021-12-06 [1] CRAN (R 4.2.0)
#>  stringi        1.7.8      2022-07-11 [1] CRAN (R 4.2.1)
#>  stringr        1.4.1      2022-08-20 [1] CRAN (R 4.2.1)
#>  styler         1.7.0      2022-03-13 [1] CRAN (R 4.2.0)
#>  survival       3.4-0      2022-08-09 [1] CRAN (R 4.2.1)
#>  TH.data        1.1-1      2022-04-26 [1] CRAN (R 4.2.0)
#>  tibble         3.1.8      2022-07-22 [1] CRAN (R 4.2.1)
#>  tidyselect     1.1.2      2022-02-21 [1] CRAN (R 4.2.0)
#>  utf8           1.2.2      2021-07-24 [1] CRAN (R 4.2.0)
#>  vctrs          0.4.1      2022-04-13 [1] CRAN (R 4.2.0)
#>  withr          2.5.0      2022-03-03 [1] CRAN (R 4.2.0)
#>  xfun           0.33       2022-09-12 [1] CRAN (R 4.2.1)
#>  xtable         1.8-4      2019-04-21 [1] CRAN (R 4.2.0)
#>  yaml           2.3.5      2022-02-21 [1] CRAN (R 4.2.0)
#>  zoo            1.8-11     2022-09-17 [1] CRAN (R 4.2.1)
#> 
#>  [1] C:/Users/mail/AppData/Local/R/win-library/4.2
#>  [2] C:/Program Files/R/R-4.2.1/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

@goesjon
Copy link
Author

goesjon commented Sep 28, 2022

@strengejacke thank you very much for your comment. I did the same with my system, see result below.

What surprises me is your easystats package version 0.5.2.2. I was unable to get this exact version, the CRAN version is 0.5.2 (which is the one I tried out below) and the r-universe version is 0.5.2.8 which is the same as the GitHub version. In addition, in your session info it says "local" as source of the easystats package. Do you maybe have a modified version of easystats, which I am unable to access? Could you do me the favor and update your system with install.packages("remotes") remotes::install_github("easystats/easystats") easystats::install_suggested() and see, if you still get the same result?

library(easystats)
#> # Attaching packages: easystats 0.5.2
#> ✔ insight     0.18.4.5     ✔ datawizard  0.6.1.1   
#> ✔ bayestestR  0.13.0       ✔ performance 0.9.2.4   
#> ✔ parameters  0.18.2.9     ✔ effectsize  0.7.9.1999
#> ✔ modelbased  0.8.5        ✔ correlation 0.8.2.4   
#> ✔ see         0.7.3.1      ✔ report      0.5.5.1
model <- lm(log(Sepal.Width) ~ Species, data = iris)
estimate_contrasts(model, transform = "none")
#> No variable was specified for contrast estimation. Selecting `contrast = "Species"`.
#> Marginal Contrasts Analysis
#> 
#> Level1     |     Level2 | Difference |         95% CI |   SE | t(147) |      p
#> ------------------------------------------------------------------------------
#> setosa     | versicolor |       0.21 | [ 0.16,  0.27] | 0.02 |   9.47 | < .001
#> setosa     |  virginica |       0.14 | [ 0.09,  0.20] | 0.02 |   6.28 | < .001
#> versicolor |  virginica |      -0.07 | [-0.13, -0.02] | 0.02 |  -3.19 | 0.005 
#> 
#> Marginal contrasts estimated at Species
#> p-value adjustment method: Holm (1979)
estimate_contrasts(model, transform = "response")
#> No variable was specified for contrast estimation. Selecting `contrast = "Species"`.
#> Marginal Contrasts Analysis
#> 
#> Level1     |     Level2 | Difference |         95% CI |   SE | t(147) |      p
#> ------------------------------------------------------------------------------
#> setosa     | versicolor |       0.21 | [ 0.16,  0.27] | 0.02 |   9.47 | < .001
#> setosa     |  virginica |       0.14 | [ 0.09,  0.20] | 0.02 |   6.28 | < .001
#> versicolor |  virginica |      -0.07 | [-0.13, -0.02] | 0.02 |  -3.19 | 0.005 
#> 
#> Marginal contrasts estimated at Species
#> p-value adjustment method: Holm (1979)
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.2.1 (2022-06-23 ucrt)
#>  os       Windows 10 x64 (build 19042)
#>  system   x86_64, mingw32
#>  ui       RTerm
#>  language (EN)
#>  collate  German_Germany.utf8
#>  ctype    German_Germany.utf8
#>  tz       Europe/Berlin
#>  date     2022-09-28
#>  pandoc   2.19.2 @ C:/Program Files/RStudio/bin/quarto/bin/tools/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package      * version    date (UTC) lib source
#>  bayestestR   * 0.13.0     2022-09-18 [1] CRAN (R 4.2.1)
#>  cli            3.4.1      2022-09-23 [1] CRAN (R 4.2.1)
#>  coda           0.19-4     2020-09-30 [1] CRAN (R 4.2.1)
#>  codetools      0.2-18     2020-11-04 [1] CRAN (R 4.2.0)
#>  colorspace     2.0-3      2022-02-21 [1] CRAN (R 4.2.1)
#>  correlation  * 0.8.2.4    2022-09-13 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  datawizard   * 0.6.1.1    2022-09-28 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  digest         0.6.29     2021-12-01 [1] CRAN (R 4.2.1)
#>  dplyr          1.0.10     2022-09-01 [1] CRAN (R 4.2.1)
#>  easystats    * 0.5.2      2022-08-30 [1] CRAN (R 4.2.1)
#>  effectsize   * 0.7.9.1999 2022-09-28 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  emmeans        1.8.1-1    2022-09-08 [1] CRAN (R 4.2.1)
#>  estimability   1.4.1      2022-08-05 [1] CRAN (R 4.2.1)
#>  evaluate       0.16       2022-08-09 [1] CRAN (R 4.2.1)
#>  fansi          1.0.3      2022-03-24 [1] CRAN (R 4.2.1)
#>  fastmap        1.1.0      2021-01-25 [1] CRAN (R 4.2.1)
#>  fs             1.5.2      2021-12-08 [1] CRAN (R 4.2.1)
#>  generics       0.1.3      2022-07-05 [1] CRAN (R 4.2.1)
#>  ggplot2        3.3.6      2022-05-03 [1] CRAN (R 4.2.1)
#>  glue           1.6.2      2022-02-24 [1] CRAN (R 4.2.1)
#>  gtable         0.3.1      2022-09-01 [1] CRAN (R 4.2.1)
#>  highr          0.9        2021-04-16 [1] CRAN (R 4.2.1)
#>  htmltools      0.5.3      2022-07-18 [1] CRAN (R 4.2.1)
#>  insight      * 0.18.4.5   2022-09-27 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  knitr          1.40       2022-08-24 [1] CRAN (R 4.2.1)
#>  lattice        0.20-45    2021-09-22 [2] CRAN (R 4.2.1)
#>  lifecycle      1.0.2      2022-09-09 [1] CRAN (R 4.2.1)
#>  magrittr       2.0.3      2022-03-30 [1] CRAN (R 4.2.1)
#>  MASS           7.3-58.1   2022-08-03 [1] CRAN (R 4.2.1)
#>  Matrix         1.5-1      2022-09-13 [1] CRAN (R 4.2.1)
#>  modelbased   * 0.8.5      2022-08-18 [1] CRAN (R 4.2.1)
#>  multcomp       1.4-20     2022-08-07 [1] CRAN (R 4.2.1)
#>  munsell        0.5.0      2018-06-12 [1] CRAN (R 4.2.1)
#>  mvtnorm        1.1-3      2021-10-08 [1] CRAN (R 4.2.0)
#>  parameters   * 0.18.2.9   2022-09-27 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  performance  * 0.9.2.4    2022-09-27 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  pillar         1.8.1      2022-08-19 [1] CRAN (R 4.2.1)
#>  pkgconfig      2.0.3      2019-09-22 [1] CRAN (R 4.2.1)
#>  purrr          0.3.4      2020-04-17 [1] CRAN (R 4.2.1)
#>  R.cache        0.16.0     2022-07-21 [1] CRAN (R 4.2.1)
#>  R.methodsS3    1.8.2      2022-06-13 [1] CRAN (R 4.2.0)
#>  R.oo           1.25.0     2022-06-12 [1] CRAN (R 4.2.0)
#>  R.utils        2.12.0     2022-06-28 [1] CRAN (R 4.2.1)
#>  R6             2.5.1      2021-08-19 [1] CRAN (R 4.2.1)
#>  report       * 0.5.5.1    2022-09-12 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  reprex         2.0.2      2022-08-17 [1] CRAN (R 4.2.1)
#>  rlang          1.0.6      2022-09-24 [1] CRAN (R 4.2.1)
#>  rmarkdown      2.16       2022-08-24 [1] CRAN (R 4.2.1)
#>  rstudioapi     0.14       2022-08-22 [1] CRAN (R 4.2.1)
#>  sandwich       3.0-2      2022-06-15 [1] CRAN (R 4.2.1)
#>  scales         1.2.1      2022-08-20 [1] CRAN (R 4.2.1)
#>  see          * 0.7.3.1    2022-09-27 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  sessioninfo    1.2.2      2021-12-06 [1] CRAN (R 4.2.1)
#>  stringi        1.7.8      2022-07-11 [1] CRAN (R 4.2.1)
#>  stringr        1.4.1      2022-08-20 [1] CRAN (R 4.2.1)
#>  styler         1.7.0      2022-03-13 [1] CRAN (R 4.2.1)
#>  survival       3.4-0      2022-08-09 [1] CRAN (R 4.2.1)
#>  TH.data        1.1-1      2022-04-26 [1] CRAN (R 4.2.1)
#>  tibble         3.1.8      2022-07-22 [1] CRAN (R 4.2.1)
#>  tidyselect     1.1.2      2022-02-21 [1] CRAN (R 4.2.1)
#>  utf8           1.2.2      2021-07-24 [1] CRAN (R 4.2.1)
#>  vctrs          0.4.1      2022-04-13 [1] CRAN (R 4.2.1)
#>  withr          2.5.0      2022-03-03 [1] CRAN (R 4.2.1)
#>  xfun           0.33       2022-09-12 [1] CRAN (R 4.2.1)
#>  xtable         1.8-4      2019-04-21 [1] CRAN (R 4.2.1)
#>  yaml           2.3.5      2022-02-21 [1] CRAN (R 4.2.1)
#>  zoo            1.8-11     2022-09-17 [1] CRAN (R 4.2.1)
#> 
#>  [1] C:/Users/*USERNAME*/AppData/Local/R/win-library/4.2
#>  [2] C:/Program Files/R/R-4.2.1/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

Created on 2022-09-28 with reprex v2.0.2

@strengejacke
Copy link
Member

I doubt it's due to easystats since this package is on a higher level. Can you try easystats::install_latest(force = TRUE)? Maybe we forgot to increase a minor version and your easystats packages are not the latest dev-versions?

@goesjon
Copy link
Author

goesjon commented Sep 28, 2022

@strengejacke thank you very much, now it is working (see result below)!!! easystats::install_latest() was not enough! Only your suggested easystats::install_lates(force = TRUE) was successful. Maybe it is a good idea to add this to the easystats homepage in the "Installation" section?

library(easystats)
#> # Attaching packages: easystats 0.5.2.8
#> ✔ bayestestR  0.13.0       ✔ correlation 0.8.2.4   
#> ✔ datawizard  0.6.1.1      ✔ effectsize  0.7.9.1999
#> ✔ insight     0.18.4.5     ✔ modelbased  0.8.5     
#> ✔ performance 0.9.2.4      ✔ parameters  0.18.2.9  
#> ✔ report      0.5.5.1      ✔ see         0.7.3.1
model <- lm(log(Sepal.Width) ~ Species, data = iris)
estimate_contrasts(model, transform = "none")
#> No variable was specified for contrast estimation. Selecting `contrast = "Species"`.
#> Marginal Contrasts Analysis
#> 
#> Level1     |     Level2 | Difference |         95% CI |   SE | t(147) |      p
#> ------------------------------------------------------------------------------
#> setosa     | versicolor |       0.21 | [ 0.16,  0.27] | 0.02 |   9.47 | < .001
#> setosa     |  virginica |       0.14 | [ 0.09,  0.20] | 0.02 |   6.28 | < .001
#> versicolor |  virginica |      -0.07 | [-0.13, -0.02] | 0.02 |  -3.19 | 0.002 
#> 
#> Marginal contrasts estimated at Species
#> p-value adjustment method: Holm (1979)
estimate_contrasts(model, transform = "response")
#> No variable was specified for contrast estimation. Selecting `contrast = "Species"`.
#> Marginal Contrasts Analysis
#> 
#> Level1     |     Level2 | Ratio |       95% CI |   SE | t(147) |      p
#> -----------------------------------------------------------------------
#> setosa     | versicolor |  1.24 | [1.17, 1.31] | 0.03 |   9.47 | < .001
#> setosa     |  virginica |  1.15 | [1.09, 1.22] | 0.03 |   6.28 | < .001
#> versicolor |  virginica |  0.93 | [0.88, 0.98] | 0.02 |  -3.19 | 0.002 
#> 
#> Marginal contrasts estimated at Species
#> p-value adjustment method: Holm (1979)
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.2.1 (2022-06-23 ucrt)
#>  os       Windows 10 x64 (build 19042)
#>  system   x86_64, mingw32
#>  ui       RTerm
#>  language (EN)
#>  collate  German_Germany.utf8
#>  ctype    German_Germany.utf8
#>  tz       Europe/Berlin
#>  date     2022-09-28
#>  pandoc   2.19.2 @ C:/Program Files/RStudio/bin/quarto/bin/tools/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package      * version    date (UTC) lib source
#>  bayestestR   * 0.13.0     2022-09-18 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  cli            3.4.1      2022-09-23 [1] CRAN (R 4.2.1)
#>  coda           0.19-4     2020-09-30 [1] CRAN (R 4.2.1)
#>  codetools      0.2-18     2020-11-04 [1] CRAN (R 4.2.0)
#>  colorspace     2.0-3      2022-02-21 [1] CRAN (R 4.2.1)
#>  correlation  * 0.8.2.4    2022-09-13 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  datawizard   * 0.6.1.1    2022-09-28 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  digest         0.6.29     2021-12-01 [1] CRAN (R 4.2.1)
#>  dplyr          1.0.10     2022-09-01 [1] CRAN (R 4.2.1)
#>  easystats    * 0.5.2.8    2022-09-23 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  effectsize   * 0.7.9.1999 2022-09-28 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  emmeans        1.8.1-1    2022-09-08 [1] CRAN (R 4.2.1)
#>  estimability   1.4.1      2022-08-05 [1] CRAN (R 4.2.1)
#>  evaluate       0.16       2022-08-09 [1] CRAN (R 4.2.1)
#>  fansi          1.0.3      2022-03-24 [1] CRAN (R 4.2.1)
#>  fastmap        1.1.0      2021-01-25 [1] CRAN (R 4.2.1)
#>  fs             1.5.2      2021-12-08 [1] CRAN (R 4.2.1)
#>  generics       0.1.3      2022-07-05 [1] CRAN (R 4.2.1)
#>  ggplot2        3.3.6      2022-05-03 [1] CRAN (R 4.2.1)
#>  glue           1.6.2      2022-02-24 [1] CRAN (R 4.2.1)
#>  gtable         0.3.1      2022-09-01 [1] CRAN (R 4.2.1)
#>  highr          0.9        2021-04-16 [1] CRAN (R 4.2.1)
#>  htmltools      0.5.3      2022-07-18 [1] CRAN (R 4.2.1)
#>  insight      * 0.18.4.5   2022-09-27 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  knitr          1.40       2022-08-24 [1] CRAN (R 4.2.1)
#>  lattice        0.20-45    2021-09-22 [2] CRAN (R 4.2.1)
#>  lifecycle      1.0.2      2022-09-09 [1] CRAN (R 4.2.1)
#>  magrittr       2.0.3      2022-03-30 [1] CRAN (R 4.2.1)
#>  MASS           7.3-58.1   2022-08-03 [1] CRAN (R 4.2.1)
#>  Matrix         1.5-1      2022-09-13 [1] CRAN (R 4.2.1)
#>  modelbased   * 0.8.5      2022-09-26 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  multcomp       1.4-20     2022-08-07 [1] CRAN (R 4.2.1)
#>  munsell        0.5.0      2018-06-12 [1] CRAN (R 4.2.1)
#>  mvtnorm        1.1-3      2021-10-08 [1] CRAN (R 4.2.0)
#>  parameters   * 0.18.2.9   2022-09-27 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  performance  * 0.9.2.4    2022-09-28 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  pillar         1.8.1      2022-08-19 [1] CRAN (R 4.2.1)
#>  pkgconfig      2.0.3      2019-09-22 [1] CRAN (R 4.2.1)
#>  purrr          0.3.4      2020-04-17 [1] CRAN (R 4.2.1)
#>  R.cache        0.16.0     2022-07-21 [1] CRAN (R 4.2.1)
#>  R.methodsS3    1.8.2      2022-06-13 [1] CRAN (R 4.2.0)
#>  R.oo           1.25.0     2022-06-12 [1] CRAN (R 4.2.0)
#>  R.utils        2.12.0     2022-06-28 [1] CRAN (R 4.2.1)
#>  R6             2.5.1      2021-08-19 [1] CRAN (R 4.2.1)
#>  report       * 0.5.5.1    2022-09-12 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  reprex         2.0.2      2022-08-17 [1] CRAN (R 4.2.1)
#>  rlang          1.0.6      2022-09-24 [1] CRAN (R 4.2.1)
#>  rmarkdown      2.16       2022-08-24 [1] CRAN (R 4.2.1)
#>  rstudioapi     0.14       2022-08-22 [1] CRAN (R 4.2.1)
#>  sandwich       3.0-2      2022-06-15 [1] CRAN (R 4.2.1)
#>  scales         1.2.1      2022-08-20 [1] CRAN (R 4.2.1)
#>  see          * 0.7.3.1    2022-09-27 [1] https://easystats.r-universe.dev (R 4.2.1)
#>  sessioninfo    1.2.2      2021-12-06 [1] CRAN (R 4.2.1)
#>  stringi        1.7.8      2022-07-11 [1] CRAN (R 4.2.1)
#>  stringr        1.4.1      2022-08-20 [1] CRAN (R 4.2.1)
#>  styler         1.7.0      2022-03-13 [1] CRAN (R 4.2.1)
#>  survival       3.4-0      2022-08-09 [1] CRAN (R 4.2.1)
#>  TH.data        1.1-1      2022-04-26 [1] CRAN (R 4.2.1)
#>  tibble         3.1.8      2022-07-22 [1] CRAN (R 4.2.1)
#>  tidyselect     1.1.2      2022-02-21 [1] CRAN (R 4.2.1)
#>  utf8           1.2.2      2021-07-24 [1] CRAN (R 4.2.1)
#>  vctrs          0.4.1      2022-04-13 [1] CRAN (R 4.2.1)
#>  withr          2.5.0      2022-03-03 [1] CRAN (R 4.2.1)
#>  xfun           0.33       2022-09-12 [1] CRAN (R 4.2.1)
#>  xtable         1.8-4      2019-04-21 [1] CRAN (R 4.2.1)
#>  yaml           2.3.5      2022-02-21 [1] CRAN (R 4.2.1)
#>  zoo            1.8-11     2022-09-17 [1] CRAN (R 4.2.1)
#> 
#>  [1] C:/Users/gwn/AppData/Local/R/win-library/4.2
#>  [2] C:/Program Files/R/R-4.2.1/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

Created on 2022-09-28 with reprex v2.0.2

@goesjon
Copy link
Author

goesjon commented Sep 29, 2022

Hallo everybody,

thank you very much for your help so far!

I just realized that transform = "response" does not give me the differences between the back-transformed values but rather the ratios.

In the example above we have:

library(easystats)
#> # Attaching packages: easystats 0.5.2.8
#> ✔ bayestestR  0.13.0       ✔ correlation 0.8.2.4   
#> ✔ datawizard  0.6.1.1      ✔ effectsize  0.7.9.1999
#> ✔ insight     0.18.4.5     ✔ modelbased  0.8.5     
#> ✔ performance 0.9.2.4      ✔ parameters  0.18.2.9  
#> ✔ report      0.5.5.1      ✔ see         0.7.3.1
model <- lm(log(Sepal.Width) ~ Species, data = iris)
estimate_means(model, transform = "response")
#> We selected `at = c("Species")`.
#> Estimated Marginal Means
#> 
#> Species    | Mean |   SE |       95% CI
#> ---------------------------------------
#> setosa     | 3.41 | 0.05 | [3.30, 3.52]
#> versicolor | 2.75 | 0.04 | [2.67, 2.84]
#> virginica  | 2.96 | 0.05 | [2.87, 3.05]
#> 
#> Marginal means estimated at Species

Created on 2022-09-29 with reprex v2.0.2

I would like to know the following differences and if these differences are stat. sig.:
setosa - versicolor: 3.41 - 2.75 = 0.66
setosa - virginica: 3.41 - 2.96 = 0.45
versicolor - virginica: 2.75 - 2.96 = -0.21

Unfortunately estimate_contrasts() does not give me this information:

estimate_contrasts(model, transform = "none")
#> No variable was specified for contrast estimation. Selecting `contrast = "Species"`.
#> Marginal Contrasts Analysis
#> 
#> Level1     |     Level2 | Difference |         95% CI |   SE | t(147) |      p
#> ------------------------------------------------------------------------------
#> setosa     | versicolor |       0.21 | [ 0.16,  0.27] | 0.02 |   9.47 | < .001
#> setosa     |  virginica |       0.14 | [ 0.09,  0.20] | 0.02 |   6.28 | < .001
#> versicolor |  virginica |      -0.07 | [-0.13, -0.02] | 0.02 |  -3.19 | 0.002 
#> 
#> Marginal contrasts estimated at Species
#> p-value adjustment method: Holm (1979)

Created on 2022-09-29 with reprex v2.0.2

So, with transform = "none" I get:
setosa - versicolor: log(3.41) - log(2.75) = 0.21
setosa - virginica: log(3.41) - log(2.96) = 0.14
versicolor - virginica: log(2.75) - log(2.96) = -0.07

estimate_contrasts(model, transform = "response")
#> No variable was specified for contrast estimation. Selecting `contrast = "Species"`.
#> Marginal Contrasts Analysis
#> 
#> Level1     |     Level2 | Ratio |       95% CI |   SE | t(147) |      p
#> -----------------------------------------------------------------------
#> setosa     | versicolor |  1.24 | [1.17, 1.31] | 0.03 |   9.47 | < .001
#> setosa     |  virginica |  1.15 | [1.09, 1.22] | 0.03 |   6.28 | < .001
#> versicolor |  virginica |  0.93 | [0.88, 0.98] | 0.02 |  -3.19 | 0.002 
#> 
#> Marginal contrasts estimated at Species
#> p-value adjustment method: Holm (1979)

Created on 2022-09-29 with reprex v2.0.2

So, with transform = "response" I get:
setosa / versicolor: 3.41 / 2.75 = 1.24
setosa / virginica: 3.41 / 2.96 = 1.15
versicolor / virginica: 2.75 / 2.96 = 0.93
It uses the back-transformed values, but calculates the ratios rather than the desired differences. Basically it gives me exponential values of the differences before:
exp(0.21) = 1.24
exp(0.14) = 1.15
exp(-0.07) = 0.93

I would like to not always have to calculate the differences of the back-transformed values by hand with estimate_means(), but rather get this information directly from estimate_contrasts(). Is this possible?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants