When you call <jobqueue>$run(expr), the calling
environment is not visible to expr. Therefore, all
variables needed by expr must be explicitly provided to
$run(vars = list()) or previously set in the
jobqueue(globals = list()) constructor.
The expr parameter can be given in two ways.
expr = { 42 }expr = quote({ 42 })The second form is helpful if your expression needs to be passed
around your own code before being handed off to
<jobqueue>$run(). Other call-generating functions can
be used instead of quote(), such as call() or
bquote().
Global variables can be set when the jobqueue is
created.
globals <- list(MY_DATA = mtcars)
jq <- jobqueue(globals = globals)
expr <- quote(colnames(MY_DATA))
jq$run(expr)$result[1:6]
#> [1] "mpg" "cyl" "disp" "hp" "drat" "wt"Additional variables for a job can be defined with
$run(vars = list()).
The remote environment inherits from R’s baseenv(), so
you can call base functions as usual.
To use a function from a package, call it with ::
notation.
To use a custom function, pass it like a variable.
But remember, like expr, functions do not have access to
the local environment.
x <- 5
f <- function () x * 2
f()
#> [1] 10
jq$run({ f() }, vars = list(f=f))$result
#> <simpleError in f(): object 'x' not found>Functions from packages are the exception - they retain access to their namespace when passed as a variable.
x <- list(x = 42)
f <- jsonlite::toJSON
jq$run({ f(x) }, vars = list(f=f, x=x))$result
#> {"x":[42]}If you’re using a lot of functions from a package, you can set
jobqueue(packages = '<packageName>'), then call that
package’s functions without the :: notation.
If you’re a package developer, you can set
jobqueue(namespace = '<yourPackageName>') to give the
remote environment access to all functions and variables (both exported
and non-exported) defined in your package’s namespace. Note however that
your package must be installed for changes to take effect in the remote
environment. I.e. devtools::load_all() won’t cut it; an
R CMD INSTALL is needed (via
devtools::install(), pak::install_local(),
etc).
library(jobqueue)
jq <- jobqueue(
globals = list(A = 1),
init = { B <- 12 },
packages = 'jsonlite' )
job <- jq$run(
'vars' = list(x = 37),
'expr' = { toJSON(c(A, B, x)) } )
job$result
#> [1,12,37]Here we defined two global variables: A and
B. We also attached the ‘jsonlite’ R package to the search
path. When expr is evaluated, it uses toJSON,
A, and B from the remote environment, and
x from vars.