Parallel pomp computations under Windows

Author

Aaron A. King and Edward L. Ionides

For your convenience, the R codes for this document are provided in a script which you can download, edit, and run.

Some Windows users have reported trouble with the parallel codes used in several of the Lessons. Specifically, pomp codes that require compilation of C snippets trigger errors when executed in parallel on certain Windows machines.

Calls that induce compilation include:

  1. Every call to a pomp function in which a basic model component is specified using a C snippet. Recall that the basic model components include rmeasure, dmeasure, rprocess, dprocess, skeleton, rprior, dprior, partrans, rinit, and dinit. Whenever one of these arguments is furnished a Csnippet, the snippet is compiled.
  2. Every call to a pomp function in which a parameter transformation is specified by giving the names of parameters to log-transform, logit-transform, or log-barycentric transform. Even though the user does not supply a Csnippet in such cases, each such call results in a Csnippet being written internally, and then compiled.

In addition, the use of temporary directories to hold the C files and dynamically loadable libraries created from them that underlie the C snippet facility causes problems on certain Windows machines. We hypothesize that this is due to Windows securities features.

The workaround is to execute serially all codes that require compilation; that is, to execute them outside of all parallel code blocks. Also, one can dictate to pomp that all compilation should be done in the current working directory. The easiest way to do this is to give a global option indicating to which directory the C files should be written.

For example, the following command sets a global option, with the effect that every pomp compilation operation will be performed in the directory tmp, located just under the current working directory.

options(pomp_cdir="./tmp")

As an illustration, we will consider two code chunks. Before we do so, we load some packages, set up a parallel environment, and build the SIR model we have seen in the Lessons. The latter is stored in an object named measSIR.

library(tidyverse)
library(pomp)
library(doFuture)

source("https://kingaa.github.io/sbied/pfilter/model.R")

Now compare the following two code chunks. The first produces an error on many Windows machines:

plan(multisession)
foreach (
  i=1:4,
  .combine=c,
  .options.future=list(seed=TRUE)
) %dofuture% {
  measSIR |>
    mif2(
      Np=1000, Nmif=5,
      cooling.fraction.50=0.5,
      rw.sd=rw_sd(Beta=0.2, rho=0.2, eta=ivp(0.2)),
### Compilation is triggered here, by the call to `parameter_trans`.
      partrans=parameter_trans(log="Beta",logit=c("rho","eta")),
      paramnames=c("Beta","rho","eta")
    )
} -> mifs_local

The next one does not:

measSIR |>
  pomp(
### Compilation is triggered here, outside the parallel block.
    partrans=parameter_trans(log="Beta",logit=c("rho","eta")),
    paramnames=c("Beta","rho","eta")
  ) -> measSIR2

plan(multisession)
foreach (
  i=1:4,
  .combine=c,
  .options.future=list(seed=TRUE)
) %dofuture% {
### No compilation is triggered inside the parallel code block.
  measSIR2 |>
    mif2(
      Np=1000, Nmif=5,
      cooling.fraction.50=0.5,
      rw.sd=rw_sd(Beta=0.2, rho=0.2, eta=ivp(0.2))
    )
} -> mifs_local

The only difference between the code chunks is that, in the first, compilation is needed inside the parallel code block; In the second, the compilation is done before the parallelization, via the call to pomp.