crew worker does not actually heed reset_packages = FALSE, reset_globals = FALSE etc
#143
Comments
|
I think crew <- crew::crew_controller_local(
name = "user",
workers = 1,
local_log_directory = "crew_logs",
reset_globals = FALSE
)
crew$start()
crew$push(
name = "add_the_data",
command = the_data,
globals = list(the_data = "test")
)
crew$wait()
a <- crew$pop()
a$result[[1]]
#> [1] "test"
a$error
#> [1] NA
crew$push(
name = "verify_the_data",
the_data
)
crew$wait()
b <- crew$pop()
b$result[[1]]
#> [1] "test"
b$error
#> [1] NA
crew$push(
name = "verify_the_data_again",
the_data
)
crew$wait()
c <- crew$pop()
c$result[[1]]
#> [1] "test"
c$error
#> [1] NACreated on 2024-01-26 with reprex v2.1.0 And with crew <- crew::crew_controller_local(
name = "user",
workers = 1,
local_log_directory = "crew_logs",
reset_globals = TRUE
)
crew$start()
crew$push(
name = "add_the_data",
command = the_data,
globals = list(the_data = "test")
)
crew$wait()
a <- crew$pop()
a$result[[1]]
#> [1] "test"
a$error
#> [1] NA
crew$push(
name = "verify_the_data",
the_data
)
crew$wait()
b <- crew$pop()
b$result[[1]]
#> [1] NA
b$error
#> [1] "object 'the_data' not found"
crew$push(
name = "verify_the_data_again",
the_data
)
crew$wait()
c <- crew$pop()
c$result[[1]]
#> [1] NA
c$error
#> [1] "object 'the_data' not found"Created on 2024-01-26 with reprex v2.1.0 |
Neither |
Hi @wlandau,
The reprex demonstrates how it is indeed resetting the value in the package though, possibly by reloading the package? The code written to the package in the reprex above creates an environment when the In the first task, we pass a variable into the crew task which is then assigned by that function to the environment internal to the package. In your reprex, you pass the variable to each subsequent If Does the issue make more sense now? |
As written originally in #143 (comment), your test with packages also uses |
@wlandau Ahh, the worker is closing out 10s after every task and that's why the packages refresh when loaded each time. Got it, thank you! |
Hi
crewdevs,I think I've encountered a bug based on what the documentation suggests for
reset_packages&reset_globalsarguments tocrew_controller_local. Reprexes belowPrework
crewpackage itself and not a user error, known limitation, or issue from another package thatcrewdepends on. For example, if you get errors runningtar_make_clustermq(), try isolating the problem in a reproducible example that runsclustermqand notcrew. And for miscellaneous troubleshooting, please post to discussions instead of issues.Description
Every task allocated to the worker resets the packages and globals, there is nothing that persists between tasks pushed to the worker.
Reproducible example for globals
Expected result
the_datapersists as a global on the worker between subsequent tasks allocated to the worker.However, though it will occasionally persist on one subsequent task
verify_the_data, by the final task:verify_the_data_againThe result will beNAbecause the global has disappeared.The same is true for packages.
Reprex for packages
Expected result
The variable assigned to the value in the internal environment of the package persists between tasks at least once, but it returns
NULLby the second task.Diagnostic information
Diagnostic information
GithubSHA: f31eea1 Latest
The text was updated successfully, but these errors were encountered: