Future RStan/StanHeaders 2.31+ Issues

[andrjohns]

Opening this issue to flag early any difficulties that are going to be encountered in the (eventual) transition from 2.26 -> 2.31+

• Incompatibility between rstan 2.26 & StanHeaders 2.31 due to a significant refactor of indexing functions in stan

Will update as/if more are encountered

[andrjohns]

@hsbadr What would you think of bundling the stanc.js in StanHeaders rather than rstan? Then it would be much harder to encounter mismatches between the generated c++ and the available headers during the upgrade process

[hsbadr]

What would you think of bundling the stanc.js in StanHeaders rather than rstan?

I think that this is handled by the dependency on the newer version of StanHeaders. rstan >= 2.31 requires preinstalled StanHeaders >= 2.31. Also, stanc is linked to rstan::stanc not the Stan headers. We could check the versions or checksums.

The future issue I see now is due to merging stan-dev/math#2583 while RcppEigen on CRAN hasn't been updated to Eigen 3.4 yet (see RcppCore/RcppEigen#103). That's why I've freezon the experimental branch rstan for now.

[andrjohns]

But the issue is with the rstan 2.26 and StanHeaders 2.31 combination - since the code generated by the 2.26 stanc is not compatible with the 2.31 headers

[andrjohns]

Also, the rstan 2.31 & StanHeaders 2.31 combination does not appear to be compatible with any packages. Installing a package with Stan models results in the following error:

In file included from stanExports_stanmarg.cc:5:
In file included from ./stanExports_stanmarg.h:23:
In file included from /Users/andrew/Library/R/arm64/4.2/library/rstan/include/rstan/rstaninc.hpp:4:
/Users/andrew/Library/R/arm64/4.2/library/rstan/include/rstan/stan_fit.hpp:930:9: error: field type 'model_stanmarg_namespace::model_stanmarg' is an abstract class
  Model model_;

[hsbadr]

But the issue is with the rstan 2.26 and StanHeaders 2.31 combination - since the code generated by the 2.26 stanc is not compatible with the 2.31 headers

Ummm. This makes sense.

Also, the rstan 2.31 & StanHeaders 2.31 combination does not appear to be compatible with any packages.

I didn't run into any issues with earlier versions. So, we need to update stan_fit & Module after stan-dev/stan#3139.

[hsbadr]

Also, the rstan 2.31 & StanHeaders 2.31 combination does not appear to be compatible with any packages.

Looks like we need to update stanc3 JS after stan-dev/stan#3139.

[andrjohns]

Looks like we need to update stanc3 JS after stan-dev/stan#3139.

Good call! I just tested with the latest nightly stanc and all worked without issue.

[hsbadr]

Good call! I just tested with the latest nightly stanc and all worked without issue.

Yes, I've shipped it in the experimental version. This will get automatically fixed when stanc3 v2.32 is released. Note that RcppEigen v3.4 is required now.

[andrjohns]

When you've got a minute, would you mind backporting the stanc location change to the 2.26 branch and doing another release? Once it's updated in the repo I can run the 2.26 + 2.31 revdeps

[hsbadr]

v2.26.18 has been released.

[andrjohns]

Brilliant, thanks!

[andrjohns]

Looks like expose_stan_functions() is broken (for both the 2.26+2.31 & 2.31+2.31 combinations):

funmod <- "
functions {
  real rtn_input(real x) {
    return x;
  }
}
"

rstan::expose_stan_functions(rstan::stanc(model_code = funmod))
#> Error in compiled$functions: $ operator is invalid for atomic vectors

This is causing a test failure for the prophet package

[andrjohns]

The 2.26 + 2.31 revdep checks only identified six failures in addition to those from #1034:

package	version	status	Merged	On CRAN
EcoEnsemble	1.0.1	No public repo, emailed maintainer		✓
prophet	1.0	expose_stan_functions failure		Resolved by #1050
ctsem	3.7.2	PR Opened	✓	✓
RoBTT	1.0.1	PR Opened
tmbstan	1.0.4	PR Opened	✓	✓
TriDimRegression	1.0.1	Stanc3 Issue		Resolved

Looks like it might be a fairly painless transition from 2.26 -> 2.31 after all!

[hsbadr]

Looks like expose_stan_functions() is broken

Confirmed! I think the generated C++ code fails to compile. We may need to fix it in expose_stan_functions_hacks(). I don't have time to look into it right now, but if you create a PR, I can review it ASAP.

Looks like it might be a fairly painless transition from 2.26 -> 2.31 after all!

Yes, once StanHeaders v2.26 is on CRAN, we could probably release rsran v2.32 followed by StanHeaders v2.32.