Docs around R/test-helpers.R vs tests/testthat/setup.R

[maelle]

In https://testthat.r-lib.org/articles/custom-expectation.html it is written "Since these functions will need to be loaded when your package is loaded for testing, it is recommended that expect_ functions be defined in test-helpers.R in your packages R/ directory"

• why not in setup.R?

• depending on the answer to the previous question, should the recommendation to use the filename "test-helpers.R" be added to https://testthat.r-lib.org/reference/test_dir.html#special-files:?

[maelle]

For reference https://community.rstudio.com/t/why-are-helper-functions-discouraged-in-testthat/85253

[maelle]

I see

testthat/vignettes/test-fixtures.Rmd

Line 264 in fbbd667

To solve this problem we create a test fixture, which we place in `R/test-helpers.R` so that's it's available for both testing and interactive experimentation:

, maybe it could be repeated in the section about special files even if R/test-helpers is not really special

[hadley]

Because we're moving away from setup.R, and you really want these helpers to be available during interactive exploration with load_all().

It's not a special file — you can put it in any file in R/.

[maelle]

But there are still things you'd put in setup.R, right?

• setup-ish things in setup (e.g. loading a http testing package 🙂, setting an environment variable)
• functions that you use throughout your tests in R/ so that they are loaded when loading the package.

[maelle]

Right I see https://testthat.r-lib.org/articles/test-fixtures.html#package

It is now clearer to me, thanks. I wonder whether a summary of where to put helper code (based on when it should be run and available) might be useful somewhere, but if not then this issue is useless. 🙂

[maelle]

From the community thread, it seems one "point" that helper files in tests/testthat/helper- have is to be clearly distinct from package code. What would you answer to this objection? 🤔

[hadley]

Why do you want test and code to be distinct? They have to be intertwined.

[maelle]

Maybe the docs could state that since that's a difference between the previous recommendation of tests/testthat/helper- and the current recommendation? (I have to say I have no opinion on this)

[maelle]

If your helper code is in R you can no longer skip its loading using the corresponding argument of pkgload::load_all() but I imagine it's because it wasn't used a lot?

[gaborcsardi]

Why do you want test and code to be distinct? They have to be intertwined.

A couple of possible reasons:

• setup / helper code is not needed for the end users, so it is just unnecessary bloat in the final (binary) package.
• package code and test code have different standards on CRAN, e.g. test code can refer to a non-existent package, package code cannot. Etc.
• helper-* code does not run at package installation time. E.g. I can use it to start up a test web server, every time you test or load_all() the package.

For the packages I have encountered, setup/helper code is small, and the other issues are not very hard to work around, so I am not saying that they are strong reasons.

[maelle]

I've just thought of another reason, not important at all: to globally analyze what people put in their testthat helpers (à la https://rud.is/b/2018/04/08/dissecting-r-package-utility-belts/), it's easier if such helpers live in tests/testthat/helper- 😁

[hadley]

To be clear, helper files aren't going away, they're just no longer recommended.

@gaborcsardi if you have a lot of setup/helper code I'd argue that it's even more important that it live in R/ so you can test it 😉

[maelle]

@hadley do you actually know of any package testing setup/helper code?

[hadley]

Nope 😄

[maelle]

I've just thought of an advantage of defining helpers under R/: if you use two test directories (see point 19 in https://cran.r-project.org/doc/manuals/R-exts.html#Checking-packages), then both of them have access to the helpers.

[hadley]

I think the conclusion to this thread is that I don't need to do anything; please let me know if you think otherwise.

[maelle]

This became inspiration for a blog post with a summary table https://blog.r-hub.io/2020/11/18/testthat-utility-belt/

[sebffischer]

Also locating them in R/ makes usage of the R language server more enjoyable!