+
+
+
+
+csurvey: Implementing Order Constraints in Survey Data Analysis
+ + + +Recent work in survey domain estimation has shown that incorporating a priori assumptions about orderings of population domain means reduces the variance of the estimators. The new R package csurvey allows users to implement order constraints using a design specified in the well-known survey package. The order constraints not only give estimates that satisfy a priori assumptions, but they also provide smaller confidence intervals with good coverage, and allow for design-based estimation in small-sample or empty domains. A test for constant versus increasing domain means is available in the package, with generalizations to other one-sided tests. A cone information criterion may be used to provide evidence that the order constraints are valid. Examples with well-known survey data sets show the utility of the methods. This package is now available from the Comprehensive R Archive Network at http://CRAN.R-project.org/package=csurvey.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+1 Introduction
+We assume that a finite population is partitioned into a number of domains, and the goal is to estimate the population domain means for the study variable and provide valid inference, such as confidence intervals and hypothesis tests.
+Order constraints are a common type of a priori knowledge in survey data analysis. For example, we might know that salary increases with job rank within job type and location, or average cholesterol in a population increases with age category, or test scores decrease as poverty increases, or that the amount of pollution decreases with distance from the source. There might be a “block ordering” of salaries by location, where the locations are partitioned into blocks where salaries for all locations in one block (such as major metropolitan areas) are assumed to be higher, on average, than salaries in another block (such as rural areas), without imposing orderings within the blocks.
+The order constraints may be imposed on domain mean estimates, or systematic component estimates if the response variable is not Gaussian but belongs to an exponential family of distributions. For example, if we are estimating the proportion of people with diabetes in a population, the study variable might be binary (whether or not the subject has diabetes), and perhaps we want to assume that diabetes prevalence increases with age, within ethnicity and socio-economic categories. The csurvey package extends the survey package in that it allows the user to impose linear inequality constraints on the domain means. If the inequality constraints are valid, this leads to improved inference, as well as the ability to estimate means for empty or small-sample domains without additional assumptions. The csurvey package provides constrained estimates of means or proportions, estimated variances of the estimates, and confidence intervals where the upper and lower bounds of the intervals also satisfy the order constraints.
+The csurvey package implements one-sided tests. Suppose the study variable is salary, and interest is in whether the subject’s salary is affected by the level of education of the subject’s father, controlling for the subject’s education level and field. The null hypothesis is that there is no difference in salary by the level of father’s education. The one-sided test with alternative hypothesis that the salary increases with father’s education has a higher power than the two-sided test with the larger alternative “salary is not the same across the levels of father’s education.”
+Finally, the csurvey package includes graphical functions for visualizing the order-constrained estimator and comparing it to the unconstrained estimator. Confidence bands can also be displayed to illustrate estimation uncertainty.
+This package relies on coneproj and survey for its core computations and handling of complex sampling designs. The domain grid is constructed using the data.table package. Additional functionality, such as data filtering, variable transformation, and visualization of model results, leverages functions from igraph , dplyr , and ggplot2 . When simulating the mixture covariance matrix and the sampling distribution of the one-sided test statistic, selected functions from MASS and Matrix are employed.
+2 Research incorporated in csurvey
+The csurvey package provides estimation and inference on population domain variables with order constraints, using recently developed methodology. considered a complete ordering on a sequence of domain means. They applied the pooled adjacent violators algorithm for domain mean estimation, and derived asymptotic confidence intervals that have smaller width without sacrificing coverage, compared to the estimators that do not consider the ordering. developed methodology for partial orderings and more general constraints on domains. These include block orderings and orderings on domains arranged in grids by multiple variables of interest. refined these methods by proposing variance estimators based on a mixture of covariance matrices, and showed that the mixture covariance estimator improves coverage of confidence intervals while retaining smaller interval lengths. showed how to use the order constraints to provide conservative design-based inference in domains with small sample sizes, or even in empty domains, without additional model assumptions. developed a test for constant versus increasing domain means, and extended it to one-sided tests for more general orderings. proposed a cone information criterion (CIC) for survey data as a diagnostic method to measure possible departures from the assumed ordering. This criterion is similar to Akaike’s information criterion (AIC) and the Bayesian information criterion (BIC) , and can be used for model selection. For example, we can use CIC to choose between an unconstrained estimator and an order constrained estimator. The one with a smaller CIC value will be chosen.
+All of the above papers include extensive simulations showing that the methods substantially improve inference when the order restrictions are valid. These methods are easy for users to implement in csurvey, with commands to impose common orderings.
+3 How to use csurvey
+Statisticians and practitioners working with survey data are familiar with the R package survey (see ) for analysis of complex survey data. Commands in the package allow users to specify the survey design with which their data were collected, then obtain estimation and inference for population variables of interest. The new csurvey package extends the utility of survey by allowing users to implement order constraints on domains.
+The csurvey package relies on the functions in the survey package such as svydesign and svrepdesign, which allow users to specify the survey design. This function produces an object that contains information about the sampling design, allowing the user to specify strata, sampling weights, etc. This object is used in statistical functions in the csurvey package in the same manner as for the survey package. In addition, the mixture covariance matrix in is constructed from an initial estimate of covariance obtained from survey.
Consider a finite population with labels \(U = \{1,\ldots,N\}\), partitioned into domains \(U_d\), \(d=1,\ldots,D\), where \(U_d\) has \(N_d\) elements. For a study variable \(y\), suppose interest is in estimating the population domain means +\[ +\bar{y}_{U_d} = \frac{\sum_{k\in U_d} y_k}{N_d} +\] +for each \(d\), and providing inference such as confidence intervals for each \(\bar{y}_{U_d}\). +Given a survey design, a sample \(s\subset U\) is chosen, and the unconstrained estimator of the population domain means is a weighted average of the sample observations in each domain. This estimator \(\tilde{\mathbf{y}}_s=(\tilde{y}_{s_1},\ldots, \tilde{y}_{s_D})\) in is provided by the survey package.
+The desired orderings are imposed as linear inequality constraints on the domain means, in the form of an \(m\times D\) constraint matrix \(\mathbf{A}\). The csurvey package will find the constrained estimator \(\tilde{\boldsymbol{\theta}}\) by minimizing +\[ +\min_{\theta}(\tilde{\mathbf{y}}_s - \boldsymbol{\theta})^{\top}\mathbf{W}_s(\tilde{\mathbf{y}}_s-\boldsymbol{\theta}) \quad \mbox{such that} \hspace{2mm} \mathbf{A}\boldsymbol{\theta}\geq \mathbf{0} +\] +where the weights \(\mathbf{W}_s\) are provided by the survey design. (See for details.) For a simple example of a constraint matrix, consider five domains with a simple ordering, where we assume \(\bar{y}_{U_1}\leq \bar{y}_{U_2}\leq \bar{y}_{U_3}\leq \bar{y}_{U_4}\leq \bar{y}_{U_5}\). Perhaps these are average salaries over five job levels. Then the constraint matrix is +\[ \mathbf{A} = \left(\begin{array}{ccccc} -1 & 1 & 0 & 0 & 0 \\0 & -1 & 1 & 0 & 0 \\0 & 0 & -1 & 1 & 0 \\0 &0 &0 & -1 & 1 \end{array}\right). +\] +For simple orderings on \(D\) domains, the constraint matrix is \((D-1) \times D\). +For a block ordering example, suppose we again have five domains, and we know that each of the population means in the first two domains must be smaller than each of the means of the last three domains. The constraint matrix is +\[ \mathbf{A} = \left(\begin{array}{ccccc} -1 & 0 & 1 & 0 & 0 \\ -1 & 0 & 0 & 1& 0 \\ -1 & 0 & 0 & 0 & 1\\ +0 & -1 & 1 & 0 & 0 \\0 & -1 & 0 & 1& 0 \\ 0 & -1 & 0 & 0 & 1\\ +\end{array}\right). +\] +The number of rows for a block ordering with two blocks is \(D_1 \times D_2\), where \(D_1\) is the number of domains in the first block and \(D_2\) is the number in the second block. The package also allows users to specify grids of domains with various order constraints along the dimensions of the grid. The constraint matrices are automatically generated for some standard types of constraints.
+The csurvey package allows users to specify a simple ordering with the symbolic function. For example, suppose is the survey variable of interest (say, cholesterol level), and the variable takes values \(1\) through \(D\) (age groups for example). Suppose we assume that average cholesterol level is increasing with age in this population, and that the design is specified by the object . Then
+ans <- csvy(y ~ incr(x), design = ds)creates an object containing the estimated population means and confidence intervals. The function is used similarly, to specify decreasing means.
+Next, suppose takes values \(1\) through \(D_1\) and takes values \(1\) through \(D_2\) (say, age group and ethnicity), and we wish to estimate the population means over the \(D_1\times D_2\) grid of values of and If we assume that the population domain means are ordered in but there is no ordering in , then the command
+ans <- csvy(y ~ incr(x1) * x2, design = ds)will provide domain mean estimates where the means are non-decreasing in age, within each ethnicity. Note that we don’t allow “+” when defining a formula in csvy() because we consider all combinations \(D_1\times D_2\) given by and .
For an example of a block ordering with three blocks, the command
+ans <- csvy(y ~ block.Ord(x, order = c(1,1,1,2,2,2,3,3,3)), design = ds)specifies that the variable takes values \(1\) through \(9\), and the domains with values 1, 2, and 3 each have population means not greater than each of the population means in the domains with values 4, 5, and 6. The domains with values 4, 5, and 6 each have population means not greater than each of the population means in the domains with values 7, 8, and 9. More examples of implementation of constraints will be given below.
+Implementing order constraints leads to “pooling” of domains where the order constraints are binding. This naturally leads to smaller confidence intervals as the averaging is over a larger number of observations. The mixture covariance estimator for \(\tilde{\boldsymbol{\theta}}\) that was derived in is provided by csurvey. This covariance estimator is constructed by recognizing that for different samples, different constraints are binding, so that different sets of domains are “pooled” to obtain the constrained estimator. The covariance estimator then is a mixture of pooled covariance matrix estimators, with the mixture distribution approximated via simulations. Using this mixture rather than the covariance matrix for the observed pooling provides confidence intervals with coverage that is closer to the target, while retaining the shorter lengths. The method introduced in further pools information across domains to provide upper and lower confidence interval bounds that also satisfy the constraints, effectively reducing the confidence interval length for domains with small sample sizes, and allowing for estimation and inference in empty domains.
+The test \(H_0:\mathbf{A}\bar{\mathbf{y}}_U=0\) versus the one-sided \(H_1:\mathbf{A}\bar{\mathbf{y}}_U\geq0\) in csurvey has improved power over the \(F\)-test implemented using the command using the object from in the survey package. That \(F\)-test uses the two-sided alternative \(H_2:\mathbf{A}\bar{\mathbf{y}}_U\neq0\). For example, suppose we have measures of amounts of pollutants in samples of water from small lakes in a certain region. We also have measurements of distances from sources, such as factories or waste dumps, that are suspected of contributing to the pollution. The test of the null hypothesis that the amount of pollution does not depend on the distance will have greater power if the alternative is one-sided, that is, that the pollution amount is, on average, larger for smaller distances.
+In the following sections, we only show the main part of code used to generate the results. Supplementary or non-essential code is available in the accompanying R script submitted with this manuscript.
+4 NHANES Example with monotonic domain means
+The National Health and Nutrition Examination Survey (NHANES) combines in-person interviews and physical examinations to produce a comprehensive data set from a probability sample of residents of the U.S. The data are made available to the public at this CDC NHANES website. The subset used in this example is derived from the 2009–2010 NHANES cycle and is provided in the csurvey package as the data set. We consider the task of estimating the average total cholesterol level (mg/dL), originally recorded as LBXTC in the raw NHANES data, by age (in years), corresponding to the original variable RIDAGEYR. Focusing on young adults aged 21 to 45, we analyze a sample of \(n = 1,933\) participants and specify the survey design using the associated weights and strata available in the data.
+
+
+Then, to get the proposed constrained domain mean estimate, we use the function in the csurvey package. In this function, is a symbolic function used to define that the population domain means of chol are increasing with respect to the predictor age.
+
+
+
+ans <- csvy(chol ~ incr(age), design = dstrat, n.mix = 100)The n.mix parameter controls the number of simulations used to estimate the mixture covariance matrix, with a default of n.mix = 100. To speed up computation, users can set n.mix to be a smaller number, e.g., 10.
We can extract from the CIC value for the constrained estimator as
+
+
+
+
+cat("CIC (constrained):", ans$CIC, "\n")CIC (constrained): 32.99313 and the CIC value for the unconstrained estimator as
+
+
+
+
+cat("CIC (unconstrained):", ans$CIC.un, "\n")CIC (unconstrained): 51.65159 We see that for this example, the constrained estimator has a smaller CIC value and this implies it is a better fit.
+If we want to construct a contrast of domain means and get its standard error, we can use the function, which is inherited from the survey package. Note that in the survey package, it is impossible to get a contrast estimate when there is any empty domain. In the csurvey package, we inherit this feature. For example, suppose we want to compare the average cholesterol for the first thirteen age groups to the last twelve age groups, we can code it as:
+ +The function produces both the constrained fit and the corresponding unconstrained fit using methods from the survey package. A visual comparison between the two fits can be easily obtained by applying the method to the resulting object by specifying the argument type = "both". For illustration, Figure 1 displays the estimated domain means along with 95% confidence intervals, generated by the following code:
plot(ans, type = "both")
+
+
+
+
+
+
+Figure 1: Estimates of average cholesterol level for 25 ages, with 95% confidence intervals, for a stratified sample in the R dataset nhdat2, \(n=1933\).
+
The function can be used to extract the confidence interval for domain mean. When the response is not Gaussian, then produces the confidence interval for the average systematic component over domains, while produces the confidence interval for the domain mean.
+For this data set, the sample sizes for each of the twenty-five ages range from 54 to 99, so that none of the domains is “small.” To demonstrate in the case of small domains, we next provide domain mean estimates and confidence intervals for \(400\) domains, arranged in a grid with 25 ages, four waist-size categories, and four income categories. We divide the waist measurement by height to make a relative girth, and split the observations into four groups with variable name wcat, which is a 4-level ordinal categorical variable representing waist-to-height ratio categories, computed from BMXWAIST (waist circumference in cm) and BMXHT (height in cm) in the body measures file BMX_F.XPT from NHANES. We have income information for the subjects, in terms of a multiple of the federal poverty level. Our first income category includes those with income that is 75% or less than the poverty line. The second category is .75 through 1.38 times the poverty level (1.38 determines Medicaid eligibility), the third category goes from above 1.38 to 3.5, and finally above 3.5 times the poverty level is the fourth category. These indicators are contained in the variable icat (categorized income). We create icat by the INDFMPIR variable from the file DEMO_F.XPT from NHANES.
The domain sample sizes average only 4.8 observations, and there are 16 empty domains. The sample size for each domain can be checked by ans$nd. To estimate the population means we assume average cholesterol level is increasing in both age and waist size, but no ordering is imposed on income categories:
To extract estimates and confidence intervals for specific domains defined by the model, the user can use the function as follows:
+
+
+
+
+domains <- data.frame(age = c(24, 35), wcat = c(2, 4), icat = c(2, 3))
+pans <- predict(ans, newdata = domains, se.fit = TRUE)
+cat("Predicted values, confidence intervals and standard errors for specified domains:\n")Predicted values, confidence intervals and standard errors for specified domains:
+
+print (pans)$fit
+[1] 162.9599 202.0061
+
+$lwr
+[1] 148.6083 183.6379
+
+$upp
+[1] 177.8976 219.7764
+
+$se.fit
+[1] 7.471908 9.219167Figure 2 displays the domain mean estimates along with 95% confidence intervals, highlighting in red the two domains specified in the function. The code to create this plot is as below. The control argument is used to let the user adjust the aesthetics of the plot.
+
+
+
+
+
++Figure 2: Constrained estimates of population domain means for 400 domains in a 25x4x4 grid. The increasing population domain estimates for the 25 ages are shown within the waist size and income categories. The blue bands indicate 95% confidence intervals for the population domain means, with two specific domains, namely, (age, waist, income) = (24, 2, 2) and (35, 4, 3) marked in red. Empty domains are marked with a red ‘x’ sign. +
+ctl <- list(x1lab = "waist", x2lab = "income", subtitle.size = 8)
+plot(ans, x1 = "wcat", x2 = "icat", control = ctl, domains = domains)The user can visualize the unconstrained fit by specifying in the function. The corresponding output is presented in Figure 3.
+plot(ans, x1 = "wcat", x2 = "icat", control = ctl, type="unconstrained")
+
+
+
+
+
++Figure 3: Unconstrained estimates of population domain means for 400 domains in a 25x4x4 grid. The population domain estimates for the 25 ages are shown within the waist size and income categories. The green bands indicate 95% confidence intervals for the population domain means. Empty domains are marked with a red ‘x’ sign. +
+Without the order constraints, the sample sizes are too small to provide valid estimates and confidence intervals, unless further model assumptions are used, as in some small area estimation methods. With order constraints, design-based estimation and inference are possible for substantially smaller domain sample sizes, compared to the unconstrained design-based estimation.
+5 Constrained domain means with a block ordering
+We consider the 2019 National Survey of College Graduates (NSCG), conducted by the U.S. Census Bureau and sponsored by the National Center for Science and Engineering Statistics (NCSES) within the National Science Foundation. The NSCG provides data on the characteristics of the nation’s college graduates, with a focus on those in the science and engineering workforce. The datasets and documentation are available to the public on the National Survey of College Graduates (NSCF) website. Replicate weights are available separately from NCSES upon request. Because the size of subsets of the NSCG survey used in this paper exceeds the size limit allowed for an R package stored on CRAN, the subsets are not included in the csurvey package. Instead, we provide the link to access the subsets nscg19.rda and nscg19_2.rda used in this section at this website.
The study variable of interest is annual salary (denoted as SALARY in the dataset), which exhibits substantial right-skewness in its raw form. To reduce the influence of outliers and improve model stability, we restricted the analysis to observations with annual salaries between $30,000 and $400,000. A logarithmic transformation was applied to the salary variable to address skewness. Four predictors of salary were considered:
+-
+
Field of study (
field, denoted byNDGMEMGin the raw dataset): This nominal variable defines the field of study for the highest degree. There are five levels: (1) Computer and mathematical sciences; (2) Biological, agricultural and environmental life sciences; (3) Physical and related sciences; (4) Social and related sciences; (5) Engineering. Block ordering constraint: given the other predictors, the average annual salary for each of the fields (2) and (4) is less than for the STEM fields (1), (3) and (5).
+Grouped year of award of highest degree (
hd_year_grouped, denoted byHDAY5): This ordinal variable has five levels: (1) 1995 to 1999; (2) 2000 to 2004; (3) 2005 to 2009; (4) 2010 to 2014; (5) 2015 or later. Isotonic constraint: given the other predictors, the average annual salary decreases with the year of award of highest degree; i.e., the more experience respondents have, on average, the higher the annual salary.
+Highest degree type (
hd_type, denoted by DGRDG): The three levels are: (1) Bachelor’s; (2) Master’s; (3) Doctorate and Professional. Isotonic constraint: given the other predictors, the average annual salary increases with respect to the highest degree type.
+Region code for employer (
region, denoted by EMRG): This nominal variable defines the regions in which the respondents worked within the U.S. Nine levels are: (1) New England; (2) Middle Atlantic; (3) East North Central; (4) West North Central; (5) South Atlantic; (6) East South Central; (7) West South Central; (8) Mountain; (9) Pacific and US Territories. There is no constraint for this predictor.
+
This data set contains \(n=30,368\) observations in a four-dimensional grid of 675 domains, where the sample size in the domains ranges from one to 491. Here, we specify the shape and order constraints in a similar fashion as we did in previous examples. The symbolic routine is used to impose a block ordering on field and the order is specified in the argument. The specifies a survey design with no clusters. The command creates a survey design with replicate weights, where the columns named as “RW0001”, “RW0002”,,“RW0320” are the 320 NSCG replicate weights and denotes the sampling weight. The variance is computed as the sum of squared deviation of the replicates from the mean. The general formula for computing a variance estimate using replicate weights follows:
+\[v_{REP}(\hat{\theta})=\sum_{r=1}^{R}c_r(\hat{\theta}_r-\hat{\theta})^2\]
+where the estimate \(\hat{\theta}\) is computed based on the final full sample survey weights and the calculation of each replicate estimate \(\hat{\theta}_r\) is according to the \(r\)th set of replicate weights (\(r=1,\cdots, R\)). The replication adjustment factor \(c_r\) is a multiplier for the \(r\)th replicate of squared difference. The is an overall multiplier and the denotes a vector of replicate-specific multipliers, which are the values of \(c_r\) in above formula.
load("./nscg19.rda")
+rds <- svrepdesign(data = nscg, repweights = dplyr::select(nscg, "RW0001":"RW0320"),
+ weights = ~w, combined.weights = TRUE, mse = TRUE, type = "other", scale = 1, rscale = 0.05)Estimates of domain means for the 225 domains for which the highest degree is a Bachelor’s are shown in Figure 4, as surfaces connecting the estimates over the grid of field indicators and year of award of highest degree. The block-ordering constraints can be seen in the surfaces, where the fields labeled 2 and 4 have lower means than those labeled 1, 3, and 5. The surfaces are also constrained to be decreasing in year of award of highest degree.
+To improve computational efficiency, the simulations used to estimate the mixture covariance matrix and the sampling distribution of the test statistic can be parallelized. This can be enabled by setting the following R option
+options(csurvey.multicore = TRUE)The model is fitted using the following code
+ans <- csvy(logSalary~decr(hd_year_grouped)*incr(hd_type)*block.Ord(field,
+ order = c(2, 1, 2, 1, 2))*region, design = rds)The function is used to create Figure 4. It will create a three-dimensional estimated surface plot when there are at least two predictors. In this example, we use to generate a 3D perspective plot of the estimated average log-transformed salary with respect to field and year of award of highest degree. Among the three hd_type categories, the Bachelor’s degree group has the highest number of observations. The function visualizes the 225 domains corresponding to the most frequently observed level of this fourth predictor. Plot aesthetics are customized via the control argument:
ctl <- list(categ = "region", categnm = c("New England", "Middle Atlantic", "East North Central",
+ "West North Central", "South Atlantic", "East South Central", "West South Central", "Mountain",
+ "Pacific and US Territories"), NCOL = 3, th = 60, xlab = "years since degree",
+ ylab = "field of study", zlab = "log(salary)")
+
+plotpersp(ans, x1="hd_year_grouped", x2="field", control = ctl)
+
+
+
+
+
+
+
+
++Figure 4: Estimates of average log(salary) by field of study and year of degree, for observations where highest degree is a Bachelor’s, for each of the nine regions. +
+Estimates and confidence intervals for the 75 domain means associated with three of the regions are shown in Figure 5. The constrained estimates are shown as blue dots, and the unconstrained means are depicted as grey dots. The confidence intervals are indicated with blue bands for the constrained estimates and grey bands for the unconstrained estimates.
+For the Northeast Region, the average length of the constrained confidence intervals is .353, while the average length for the unconstrained confidence intervals is .477. In the domain for those in the math and computer science field, with PhDs obtained in 2000-2004, the unconstrained log-salary estimate for this domain is much below the corresponding constrained estimate, because the latter is forced to be at least that of lower degrees, and that of newer PhDs. If the constraints hold in the population, then the unconstrained confidence interval is unlikely to capture the population value. As seen in the NHANES example in Section 4, the unconstrained estimators are unreliable when the sample domain size is small. The Pacific region has the largest sample sizes, ranging from 12 to 435 observations. With this larger sample size, most of the unconstrained estimates already satisfy the constraints, but the average length for the constrained estimator is .301, while the average length for the unconstrained estimator is .338, showing that the mixture covariance matrix leads to more precision in the confidence intervals. Also shown is the region with the smallest sample sizes: the East South Central region. Here the average length for the unconstrained confidence intervals is .488 while the average length for the constrained confidence intervals is .444.
+
+
+
+
+
+
++Figure 5: Estimates of average log(salary) for the 75 domains in each of three regions. The blue dots represent the constrained domain mean estimates, while the grey dots represent the unconstrained domain mean estimates. The blue band is the 95% confidence interval for the domains, using the constraints; the grey band is the 95% unconstrained domain mean confidence interval. +
+6 One-sided testing
+For this example we again use the NCGS data set. But for this example, we use some new variables and make some variable groupings. First, instead of using the grouped year of award of highest degree (hd_year_grouped), we use the actual calendar year when the highest degree was awarded (hd_year, denoted as HDACYR in the raw data set). We conduct the one-sided test for six pairs of consecutive calendar years. Besides, we choose father’s education level (daded, denoted as EDDAD) as the main predictor, and the interest is in determining whether salaries are higher for people whose father has a higher education. In the raw data set, EDDAD has seven categories, we group them into five categories for this example: 1 = no high school degree, 2 = high school degree but no college degree, 3 = bachelor’s degree, 4 = master’s degree, and 5 = PhD or professional degree. We further group region (EMRG) as: 1 = Northeast, 2 = North Central, 3 = Southeast, 4 = West, 5 = Pacific and Territories, and group field (NDGMEMG) as: 1 = Computer and Mathematical Sciences, Physical and Related Sciences, Engineering, which can be considered as core STEM fields, 2 = Biological, Agricultural, and Environmental Life Sciences, Social and Related Sciences, which can be considered as life and social sciences, 3 = Science and Engineering-Related Fields, 4 = Non-Science and Engineering Fields, which is Non-STEM. Finally, people’s highest degree type (denoted as DGRDG in the raw data) is used to limit the study sample to people whose highest degree is a Master’s degree. The sample size is \(n=25,177\).
It seems reasonable that people whose parents are more educated are more educated themselves, but if we control for the person’s level of education, as well as field, region, and year of degree, will the salary still be increasing with level of father’s education? For this, the null hypothesis is that within the region, field, and year of degree, the salary is constant over levels of father’s education. The one-sided alternative is that salaries increase with father’s education level.
+The constrained and unconstrained fits to the data under the alternative hypotheses are shown in Figure 6 for five regions and four fields, for degree years 2016-2017. The dots connected by solid blue lines represent the fit with the one-sided alternative, i.e., constrained to be increasing in father’s education. The dots connected by solid red lines represent the fit with the two-sided alternative, i.e., with unconstrained domain means.
+
+
+
+
+
+
++Figure 6: Estimates of average log(salary) by father’s education level, for each of five regions and four fields, for subjects whose degree was attained in 2016-2017. The solid blue lines connect the estimates where the average salary is constrained to be increasing in father’s education, and the solid red lines connect unconstrained estimates of average salary. +
+The \(p\)-values for the test of the null hypotheses are given in Table 1, where n/a is shown for the two-sided \(p\)-value in the case of an empty domain. The test is conducted for six pairs of consecutive calendar years with the sample sizes to be 2,129, 2,795, 3,069, 2,895, 2,423, and 1,368 respectively. The one-sided test consistently has a small \(p\)-value, indicating that the father’s education level is positively associated with salary earned, even after controlling for region, degree type, and field.
+
+
+
+
+| +one + | ++two + | ++one + | ++two + | ++one + | ++two + | ++one + | ++two + | ++one + | ++two + | ++one + | ++two + | +
|---|---|---|---|---|---|---|---|---|---|---|---|
| +.008 + | ++n/a + | ++<.001 + | ++.018 + | ++<.001 + | ++<.001 + | ++<.001 + | ++n/a + | ++.003 + | ++.417 + | ++<.001 + | ++n/a + | +
The \(p\)-value of the one-sided test is included in the object . For example, to check the \(p\)-value for the fit for the year 2008 to 2009, we can fit the model and print out its summary table as:
+
+
+
+
+load("./nscg19_2.rda")
+data <- nscg2 |>
+ dplyr::filter(hd_year %in% c(2008, 2009))
+
+rds <- svrepdesign(data = data, repweights = dplyr::select(data, "RW0001":"RW0320"), weights = ~w,
+ combined.weights = TRUE, mse = TRUE, type = "other",
+ scale = 1, rscale = 0.05)
+
+set.seed(1)
+ans <- csvy(logSalary ~ incr(daded) * field * region, design = rds, test = TRUE)
+
+
+
+summary(ans)Call:
+csvy(formula = logSalary ~ incr(daded) * field * region, design = rds,
+ test = TRUE)
+
+Null deviance: 460.6182 on 97 degrees of freedom
+Residual deviance: 391.3722 on 51 degrees of freedom
+
+Approximate significance of constrained fit:
+ edf mixture.of.Beta p.value
+incr(daded):field:region 47 0.5536 0.0068 **
+---
+Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
+CIC (constrained estimator): 0.0275
+CIC (unconstrained estimator): 0.0354 7 Binary outcome
+Finally, we use another subset of the NHANES 2009–2010 data to demonstrate how our method applies when the outcome is binary. This subset is included in the csurvey package as nhdat. The construction of variables, sampling weights, and strata in this subset closely follows the approach described in Section 4. It contains \(n = 1,680\) observations with complete records on total cholesterol, age, height, and waist circumference for adults aged 21–40. The binary outcome indicates whether an individual has high total cholesterol, coded as 1 if total cholesterol exceeds 200 mg/dL, and 0 otherwise. We estimate the population proportion with high cholesterol by age, waist, and gender (1 = male, 2 = female). The waist variable, denoted as wcat, is a 4-level categorized ordinal variable representing waist-to-height ratios.
It is reasonable to assume that, on average, the proportion of individuals with high cholesterol increases with both age and waist. The model is specified using the following code:
+
+
+
+The CIC of the constrained estimator is smaller than that of the unconstrained estimator, and the one-sided hypothesis test has a \(p\)-value close to zero.
+
+
+
+
+summary(ans)Call:
+csvy(formula = chol ~ incr(age) * incr(wcat) * gender, design = dstrat,
+ family = binomial(link = "logit"), test = TRUE)
+
+Null deviance: 2054.947 on 159 degrees of freedom
+Residual deviance: 1906.762 on 126 degrees of freedom
+
+Approximate significance of constrained fit:
+ edf mixture.of.Beta p.value
+incr(age):incr(wcat):gender 34 0.5174 < 0.00000000000000022
+
+incr(age):incr(wcat):gender ***
+---
+Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
+CIC (constrained estimator): 0.3199
+CIC (unconstrained estimator): 1.2778 The combination of age, waist, and gender gives 160 domains. This implies that the average sample size for each domain is only around 10. Due to the small sample sizes, the unconstrained estimator shows unlikely jumps as age increases within each waist category. On the other hand, the constrained estimator is more stable and tends to have smaller confidence intervals compared with the unconstrained Hájek estimator.
+
+
+
+
+
+
+
+
+
++Figure 7: Estimates of probability of high cholesterol level for each combination of age, waist and gender. The blue dots represent the constrained domain mean estimates, while the green dots represent the unconstrained domain mean estimates. The blue band is the 95% confidence interval for the domains, using the constraints; the green band is the 95% unconstrained domain mean confidence interval. +
+8 Discussion
+While model-based small area estimators - such as those implemented in the sae package and the emdi package - are powerful tools for borrowing strength across domains, they rely on parametric assumptions that may be violated in practice. Design-based methods remain essential for official statistical agencies, as they provide transparent and model-free inference that is directly tied to the survey design. Estimation and inference for population domain means with survey data can be substantially improved if constraints based on natural orderings are implemented. The csurvey package (version 1.15) allows users to specify orderings on grids of domains, and obtain estimates of and confidence intervals for population domain means. This package also implements the design-based small area estimation method, which allows inference for population domain means for which the sample domain is empty, and further is used to improve estimates for domains with small sample size. The one-sided testing procedure available in csurvey has higher power than the standard two-sided test, and further can be applied in grids with some empty domains. Confidence intervals for domain means have better coverage rate and smaller interval width than what is produced by unconstrained estimation. Finally, the package provides functions to allow the user to easily visualize the data and the fits. The utility of the package has been demonstrated with well-known survey data sets.
+Acknowledgment: This work was partially funded by NSF MMS-1533804.
+8.1 Supplementary materials
+Supplementary materials are available in addition to this article. It can be downloaded at +RJ-2025-032.zip
+8.2 CRAN packages used
+csurvey, survey, coneproj, data.table, igraph, dplyr, ggplot2, MASS, Matrix, sae, emdi
+8.3 CRAN Task Views implied by cited packages
+ChemPhys, Databases, Distributions, Econometrics, Environmetrics, Finance, GraphicalModels, HighPerformanceComputing, MixedModels, ModelDeployment, NetworkAnalysis, NumericalMathematics, OfficialStatistics, Optimization, Phylogenetics, Psychometrics, Robust, Spatial, Survival, TeachingStatistics, TimeSeries, WebTechnologies
+ + +
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/_articles/RJ-2025-032/RJ-2025-032.pdf b/_articles/RJ-2025-032/RJ-2025-032.pdf
new file mode 100644
index 0000000000..e39b8dc472
Binary files /dev/null and b/_articles/RJ-2025-032/RJ-2025-032.pdf differ
diff --git a/_articles/RJ-2025-032/RJ-2025-032.tex b/_articles/RJ-2025-032/RJ-2025-032.tex
new file mode 100644
index 0000000000..472fc53c28
--- /dev/null
+++ b/_articles/RJ-2025-032/RJ-2025-032.tex
@@ -0,0 +1,449 @@
+% !TeX root = RJwrapper.tex
+\title{csurvey: Implementing Order Constraints in Survey Data Analysis}
+
+
+\author{by Xiyue Liao and Mary C. Meyer}
+
+\maketitle
+
+\abstract{%
+Recent work in survey domain estimation has shown that incorporating a priori assumptions about orderings of population domain means reduces the variance of the estimators. The new R package csurvey allows users to implement order constraints using a design specified in the well-known survey package. The order constraints not only give estimates that satisfy a priori assumptions, but they also provide smaller confidence intervals with good coverage, and allow for design-based estimation in small-sample or empty domains. A test for constant versus increasing domain means is available in the package, with generalizations to other one-sided tests. A cone information criterion may be used to provide evidence that the order constraints are valid. Examples with well-known survey data sets show the utility of the methods. This package is now available from the Comprehensive R Archive Network at \url{http://CRAN.R-project.org/package=csurvey}.
+}
+
+\section{Introduction}\label{introduction}
+
+We assume that a finite population is partitioned into a number of domains, and the goal is to estimate the population domain means for the study variable and provide valid inference, such as confidence intervals and hypothesis tests.
+
+Order constraints are a common type of \emph{a priori} knowledge in survey data analysis. For example, we might know that salary increases with job rank within job type and location, or average cholesterol in a population increases with age category, or test scores decrease as poverty increases, or that the amount of pollution decreases with distance from the source. There might be a ``block ordering'' of salaries by location, where the locations are partitioned into blocks where salaries for all locations in one block (such as major metropolitan areas) are assumed to be higher, on average, than salaries in another block (such as rural areas), without imposing orderings within the blocks.
+
+The order constraints may be imposed on domain mean estimates, or systematic component estimates if the response variable is not Gaussian but belongs to an exponential family of distributions. For example, if we are estimating the proportion of people with diabetes in a population, the study variable might be binary (whether or not the subject has diabetes), and perhaps we want to assume that diabetes prevalence increases with age, within ethnicity and socio-economic categories. The \CRANpkg{csurvey} package extends the \CRANpkg{survey} \citep{survey} package in that it allows the user to impose linear inequality constraints on the domain means. If the inequality constraints are valid, this leads to improved inference, as well as the ability to estimate means for empty or small-sample domains without additional assumptions. The \CRANpkg{csurvey} package provides constrained estimates of means or proportions, estimated variances of the estimates, and confidence intervals where the upper and lower bounds of the intervals also satisfy the order constraints.
+
+The \CRANpkg{csurvey} package implements one-sided tests. Suppose the study variable is salary, and interest is in whether the subject's salary is affected by the level of education of the subject's father, controlling for the subject's education level and field. The null hypothesis is that there is no difference in salary by the level of father's education. The one-sided test with alternative hypothesis that the salary increases with father's education has a higher power than the two-sided test with the larger alternative ``salary is not the same across the levels of father's education.''
+
+Finally, the \CRANpkg{csurvey} package includes graphical functions for visualizing the order-constrained estimator and comparing it to the unconstrained estimator. Confidence bands can also be displayed to illustrate estimation uncertainty.
+
+This package relies on \CRANpkg{coneproj} \citep{coneproj} and \CRANpkg{survey} for its core computations and handling of complex sampling designs. The domain grid is constructed using the \CRANpkg{data.table} \citep{dtable} package. Additional functionality, such as data filtering, variable transformation, and visualization of model results, leverages functions from \CRANpkg{igraph} \citep{igraph}, \CRANpkg{dplyr} \citep{dplyr}, and \CRANpkg{ggplot2} \citep{ggplot2}. When simulating the mixture covariance matrix and the sampling distribution of the one-sided test statistic, selected functions from \CRANpkg{MASS} \citep{mass} and \CRANpkg{Matrix} \citep{matrix} are employed.
+
+\section{Research incorporated in csurvey}\label{research-incorporated-in-csurvey}
+
+The \CRANpkg{csurvey} package provides estimation and inference on population domain variables with order constraints, using recently developed methodology. \citet{wu16} considered a complete ordering on a sequence of domain means. They applied the pooled adjacent violators algorithm \citet{brunk58} for domain mean estimation, and derived asymptotic confidence intervals that have smaller width without sacrificing coverage, compared to the estimators that do not consider the ordering. \citet{oliva20} developed methodology for partial orderings and more general constraints on domains. These include block orderings and orderings on domains arranged in grids by multiple variables of interest. \citet{xu20} refined these methods by proposing variance estimators based on a mixture of covariance matrices, and showed that the mixture covariance estimator improves coverage of confidence intervals while retaining smaller interval lengths. \citet{liao23} showed how to use the order constraints to provide conservative design-based inference in domains with small sample sizes, or even in empty domains, without additional model assumptions. \citet{xu23} developed a test for constant versus increasing domain means, and extended it to one-sided tests for more general orderings. \citet{oliva19} proposed a cone information criterion (CIC) for survey data as a diagnostic method to measure possible departures from the assumed ordering. This criterion is similar to Akaike's information criterion (AIC) \citep{aic73} and the Bayesian information criterion (BIC) \citep{bic78}, and can be used for model selection. For example, we can use CIC to choose between an unconstrained estimator and an order constrained estimator. The one with a smaller CIC value will be chosen.
+
+All of the above papers include extensive simulations showing that the methods substantially improve inference when the order restrictions are valid. These methods are easy for users to implement in \CRANpkg{csurvey}, with commands to impose common orderings.
+
+\section{How to use csurvey}\label{how-to-use-csurvey}
+
+Statisticians and practitioners working with survey data are familiar with the R package \CRANpkg{survey} (see \citet{survey}) for analysis of complex survey data. Commands in the package allow users to specify the survey design with which their data were collected, then obtain estimation and inference for population variables of interest. The new \CRANpkg{csurvey} package extends the utility of \CRANpkg{survey} by allowing users to implement order constraints on domains.
+
+The \CRANpkg{csurvey} package relies on the functions in the \CRANpkg{survey} package such as \texttt{svydesign} and \texttt{svrepdesign}, which allow users to specify the survey design. This function produces an object that contains information about the sampling design, allowing the user to specify strata, sampling weights, etc. This object is used in statistical functions in the \CRANpkg{csurvey} package in the same manner as for the \CRANpkg{survey} package. In addition, the mixture covariance matrix in \cite{xu20} is constructed from an initial estimate of covariance obtained from \CRANpkg{survey}.
+
+Consider a finite population with labels \(U = \{1,\ldots,N\}\), partitioned into domains \(U_d\), \(d=1,\ldots,D\), where \(U_d\) has \(N_d\) elements. For a study variable \(y\), suppose interest is in estimating the population domain means
+\[
+\bar{y}_{U_d} = \frac{\sum_{k\in U_d} y_k}{N_d}
+\]
+for each \(d\), and providing inference such as confidence intervals for each \(\bar{y}_{U_d}\).
+Given a survey design, a sample \(s\subset U\) is chosen, and the unconstrained estimator of the population domain means is a weighted average of the sample observations in each domain. This estimator \(\tilde{\mathbf{y}}_s=(\tilde{y}_{s_1},\ldots, \tilde{y}_{s_D})\) in \cite{hajek71} is provided by the \CRANpkg{survey} package.
+
+The desired orderings are imposed as linear inequality constraints on the domain means, in the form of an \(m\times D\) constraint matrix \(\mathbf{A}\). The \CRANpkg{csurvey} package will find the constrained estimator \(\tilde{\boldsymbol{\theta}}\) by minimizing
+\[
+\min_{\theta}(\tilde{\mathbf{y}}_s - \boldsymbol{\theta})^{\top}\mathbf{W}_s(\tilde{\mathbf{y}}_s-\boldsymbol{\theta}) \quad \mbox{such that} \hspace{2mm} \mathbf{A}\boldsymbol{\theta}\geq \mathbf{0}
+\]
+where the weights \(\mathbf{W}_s\) are provided by the survey design. (See \cite{oliva20} for details.) For a simple example of a constraint matrix, consider five domains with a simple ordering, where we assume \(\bar{y}_{U_1}\leq \bar{y}_{U_2}\leq \bar{y}_{U_3}\leq \bar{y}_{U_4}\leq \bar{y}_{U_5}\). Perhaps these are average salaries over five job levels. Then the constraint matrix is
+\[ \mathbf{A} = \left(\begin{array}{ccccc} -1 & 1 & 0 & 0 & 0 \\0 & -1 & 1 & 0 & 0 \\0 & 0 & -1 & 1 & 0 \\0 &0 &0 & -1 & 1 \end{array}\right).
+\]
+For simple orderings on \(D\) domains, the constraint matrix is \((D-1) \times D\).
+For a block ordering example, suppose we again have five domains, and we know that each of the population means in the first two domains must be smaller than each of the means of the last three domains. The constraint matrix is
+\[ \mathbf{A} = \left(\begin{array}{ccccc} -1 & 0 & 1 & 0 & 0 \\ -1 & 0 & 0 & 1& 0 \\ -1 & 0 & 0 & 0 & 1\\
+0 & -1 & 1 & 0 & 0 \\0 & -1 & 0 & 1& 0 \\ 0 & -1 & 0 & 0 & 1\\
+\end{array}\right).
+\]
+The number of rows for a block ordering with two blocks is \(D_1 \times D_2\), where \(D_1\) is the number of domains in the first block and \(D_2\) is the number in the second block. The package also allows users to specify grids of domains with various order constraints along the dimensions of the grid. The constraint matrices are automatically generated for some standard types of constraints.
+
+The \CRANpkg{csurvey} package allows users to specify a simple ordering with the symbolic \code{incr} function. For example, suppose \code{y} is the survey variable of interest (say, cholesterol level), and the variable \code{x} takes values \(1\) through \(D\) (age groups for example). Suppose we assume that average cholesterol level is increasing with age in this population, and that the design is specified by the object \code{ds}. Then
+
+\begin{verbatim}
+ans <- csvy(y ~ incr(x), design = ds)
+\end{verbatim}
+
+creates an object containing the estimated population means and confidence intervals. The \code{decr} function is used similarly, to specify decreasing means.
+
+Next, suppose \code{x1} takes values \(1\) through \(D_1\) and \code{x2} takes values \(1\) through \(D_2\) (say, age group and ethnicity), and we wish to estimate the population means over the \(D_1\times D_2\) grid of values of \code{x1} and \code{x2.} If we assume that the population domain means are ordered in \code{x1} but there is no ordering in \code{x2}, then the command
+
+\begin{verbatim}
+ans <- csvy(y ~ incr(x1) * x2, design = ds)
+\end{verbatim}
+
+will provide domain mean estimates where the means are non-decreasing in age, within each ethnicity. Note that we don't allow ``+'' when defining a formula in \texttt{csvy()} because we consider all combinations \(D_1\times D_2\) given by \code{x1} and \code{x2}.
+
+For an example of a block ordering with three blocks, the command
+
+\begin{verbatim}
+ans <- csvy(y ~ block.Ord(x, order = c(1,1,1,2,2,2,3,3,3)), design = ds)
+\end{verbatim}
+
+specifies that the variable \code{x} takes values \(1\) through \(9\), and the domains with values 1, 2, and 3 each have population means not greater than each of the population means in the domains with \code{x} values 4, 5, and 6. The domains with \code{x} values 4, 5, and 6 each have population means not greater than each of the population means in the domains with \code{x} values 7, 8, and 9. More examples of implementation of constraints will be given below.
+
+Implementing order constraints leads to ``pooling'' of domains where the order constraints are binding. This naturally leads to smaller confidence intervals as the averaging is over a larger number of observations. The mixture covariance estimator for \(\tilde{\boldsymbol{\theta}}\) that was derived in \cite{xu20} is provided by \CRANpkg{csurvey}. This covariance estimator is constructed by recognizing that for different samples, different constraints are binding, so that different sets of domains are ``pooled'' to obtain the constrained estimator. The covariance estimator then is a mixture of pooled covariance matrix estimators, with the mixture distribution approximated via simulations. Using this mixture rather than the covariance matrix for the observed pooling provides confidence intervals with coverage that is closer to the target, while retaining the shorter lengths. The method introduced in \cite{liao23} further pools information across domains to provide upper and lower confidence interval bounds that also satisfy the constraints, effectively reducing the confidence interval length for domains with small sample sizes, and allowing for estimation and inference in empty domains.
+
+The test \(H_0:\mathbf{A}\bar{\mathbf{y}}_U=0\) versus the one-sided \(H_1:\mathbf{A}\bar{\mathbf{y}}_U\geq0\) in \CRANpkg{csurvey} has improved power over the \(F\)-test implemented using the \code{anova} command using the object from \code{svyglm} in the \CRANpkg{survey} package. That \(F\)-test uses the two-sided alternative \(H_2:\mathbf{A}\bar{\mathbf{y}}_U\neq0\). For example, suppose we have measures of amounts of pollutants in samples of water from small lakes in a certain region. We also have measurements of distances from sources, such as factories or waste dumps, that are suspected of contributing to the pollution. The test of the null hypothesis that the amount of pollution does not depend on the distance will have greater power if the alternative is one-sided, that is, that the pollution amount is, on average, larger for smaller distances.
+
+In the following sections, we only show the main part of code used to generate the results. Supplementary or non-essential code is available in the accompanying R script submitted with this manuscript.
+
+\section{NHANES Example with monotonic domain means}\label{sec-monotonic}
+
+The National Health and Nutrition Examination Survey (NHANES) combines in-person interviews and physical examinations to produce a comprehensive data set from a probability sample of residents of the U.S. The data are made available to the public at this \href{https://wwwn.cdc.gov/nchs/nhanes/}{CDC NHANES website}. The subset used in this example is derived from the 2009--2010 NHANES cycle and is provided in the \CRANpkg{csurvey} package as the \code{nhdat2} data set. We consider the task of estimating the average total cholesterol level (mg/dL), originally recorded as \texttt{LBXTC} in the raw NHANES data, by age (in years), corresponding to the original variable \texttt{RIDAGEYR}. Focusing on young adults aged 21 to 45, we analyze a sample of \(n = 1,933\) participants and specify the survey design using the associated weights and strata available in the data.
+
+\begin{verbatim}
+library(csurvey)
+data(nhdat2, package = "csurvey")
+dstrat <- svydesign(ids = ~id, strata = ~str, data = nhdat2, weight = ~wt)
+\end{verbatim}
+
+Then, to get the proposed constrained domain mean estimate, we use the \code{csvy} function in the \CRANpkg{csurvey} package. In this function, \code{incr} is a symbolic function used to define that the population domain means of \texttt{chol} are increasing with respect to the predictor \texttt{age}.
+
+\begin{verbatim}
+ans <- csvy(chol ~ incr(age), design = dstrat, n.mix = 100)
+\end{verbatim}
+
+The \texttt{n.mix} parameter controls the number of simulations used to estimate the mixture covariance matrix, with a default of \texttt{n.mix\ =\ 100}. To speed up computation, users can set \texttt{n.mix} to be a smaller number, e.g., 10.
+
+We can extract from \code{ans} the CIC value for the constrained estimator as
+
+\begin{verbatim}
+cat("CIC (constrained):", ans$CIC, "\n")
+\end{verbatim}
+
+\begin{verbatim}
+#> CIC (constrained): 32.99313
+\end{verbatim}
+
+and the CIC value for the unconstrained estimator as
+
+\begin{verbatim}
+cat("CIC (unconstrained):", ans$CIC.un, "\n")
+\end{verbatim}
+
+\begin{verbatim}
+#> CIC (unconstrained): 51.65159
+\end{verbatim}
+
+We see that for this example, the constrained estimator has a smaller CIC value and this implies it is a better fit.
+
+If we want to construct a contrast of domain means and get its standard error, we can use the \code{svycontrast} function, which is inherited from the \CRANpkg{survey} package. Note that in the \CRANpkg{survey} package, it is impossible to get a contrast estimate when there is any empty domain. In the \CRANpkg{csurvey} package, we inherit this feature. For example, suppose we want to compare the average cholesterol for the first thirteen age groups to the last twelve age groups, we can code it as:
+
+\begin{verbatim}
+cat(svycontrast(ans, list(avg = c(rep(-1, 13)/13, rep(1, 12)/12))), "\n")
+\end{verbatim}
+
+\begin{verbatim}
+#> 19.44752
+\end{verbatim}
+
+The \code{csvy} function produces both the constrained fit and the corresponding unconstrained fit using methods from the \CRANpkg{survey} package. A visual comparison between the two fits can be easily obtained by applying the \code{plot} method to the resulting \code{csvy} object by specifying the argument \texttt{type\ =\ "both"}. For illustration, Figure \ref{fig:nh1big} displays the estimated domain means along with 95\% confidence intervals, generated by the following code:
+
+\begin{verbatim}
+plot(ans, type = "both")
+\end{verbatim}
+
+\begin{figure}
+
+{\centering \includegraphics[width=1\linewidth]{figures/nhanes1}
+
+}
+
+\caption{Estimates of average cholesterol level for 25 ages, with 95\% confidence intervals, for a stratified sample in the R dataset `nhdat2`, $n=1933$.}\label{fig:nh1big}
+\end{figure}
+
+The \code{confint} function can be used to extract the confidence interval for domain mean. When the response is not Gaussian, then \code{type="link"} produces the confidence interval for the average systematic component over domains, while \code{type="response"} produces the confidence interval for the domain mean.
+
+For this data set, the sample sizes for each of the twenty-five ages range from 54 to 99, so that none of the domains is ``small.'' To demonstrate \code{csvy} in the case of small domains, we next provide domain mean estimates and confidence intervals for \(400\) domains, arranged in a grid with 25 ages, four waist-size categories, and four income categories. We divide the waist measurement by height to make a relative girth, and split the observations into four groups with variable name \texttt{wcat}, which is a 4-level ordinal categorical variable representing waist-to-height ratio categories, computed from \texttt{BMXWAIST} (waist circumference in cm) and \texttt{BMXHT} (height in cm) in the body measures file \texttt{BMX\_F.XPT} from NHANES. We have income information for the subjects, in terms of a multiple of the federal poverty level. Our first income category includes those with income that is 75\% or less than the poverty line. The second category is .75 through 1.38 times the poverty level (1.38 determines Medicaid eligibility), the third category goes from above 1.38 to 3.5, and finally above 3.5 times the poverty level is the fourth category. These indicators are contained in the variable \texttt{icat} (categorized income). We create \texttt{icat} by the \texttt{INDFMPIR} variable from the file \texttt{DEMO\_F.XPT} from NHANES.
+
+The domain sample sizes average only 4.8 observations, and there are 16 empty domains. The sample size for each domain can be checked by \texttt{ans\$nd}. To estimate the population means we assume average cholesterol level is increasing in both age and waist size, but no ordering is imposed on income categories:
+
+\begin{verbatim}
+set.seed(1)
+ans <- csvy(chol ~ incr(age)*incr(wcat)*icat, design = dstrat)
+\end{verbatim}
+
+To extract estimates and confidence intervals for specific domains defined by the model, the user can use the \code{predict} function as follows:
+
+\begin{verbatim}
+domains <- data.frame(age = c(24, 35), wcat = c(2, 4), icat = c(2, 3))
+pans <- predict(ans, newdata = domains, se.fit = TRUE)
+cat("Predicted values, confidence intervals and standard errors for specified domains:\n")
+\end{verbatim}
+
+\begin{verbatim}
+#> Predicted values, confidence intervals and standard errors for specified domains:
+\end{verbatim}
+
+\begin{verbatim}
+print (pans)
+\end{verbatim}
+
+\begin{verbatim}
+#> $fit
+#> [1] 162.9599 202.0061
+#>
+#> $lwr
+#> [1] 148.6083 183.6379
+#>
+#> $upp
+#> [1] 177.8976 219.7764
+#>
+#> $se.fit
+#> [1] 7.471908 9.219167
+\end{verbatim}
+
+Figure \ref{fig:nh2} displays the domain mean estimates along with 95\% confidence intervals, highlighting in red the two domains specified in the \code{predict} function. The code to create this plot is as below. The \texttt{control} argument is used to let the user adjust the aesthetics of the plot.
+
+\begin{figure}
+
+{\centering \includegraphics[width=1\linewidth]{figures/nhanes_grid3}
+
+}
+
+\caption{Constrained estimates of population domain means for 400 domains in a 25x4x4 grid. The increasing population domain estimates for the 25 ages are shown within the waist size and income categories. The blue bands indicate 95\% confidence intervals for the population domain means, with two specific domains, namely, (age, waist, income) = (24, 2, 2) and (35, 4, 3) marked in red. Empty domains are marked with a red 'x' sign.}\label{fig:nh2}
+\end{figure}
+
+\begin{verbatim}
+ctl <- list(x1lab = "waist", x2lab = "income", subtitle.size = 8)
+plot(ans, x1 = "wcat", x2 = "icat", control = ctl, domains = domains)
+\end{verbatim}
+
+The user can visualize the unconstrained fit by specifying \code{type = "unconstrained"} in the \code{plot} function. The corresponding output is presented in Figure \ref{fig:nh2un}.
+
+\begin{verbatim}
+plot(ans, x1 = "wcat", x2 = "icat", control = ctl, type="unconstrained")
+\end{verbatim}
+
+\begin{figure}
+
+{\centering \includegraphics[width=1\linewidth]{figures/nhanes_grid3_un}
+
+}
+
+\caption{Unconstrained estimates of population domain means for 400 domains in a 25x4x4 grid. The population domain estimates for the 25 ages are shown within the waist size and income categories. The green bands indicate 95\% confidence intervals for the population domain means. Empty domains are marked with a red 'x' sign.}\label{fig:nh2un}
+\end{figure}
+
+Without the order constraints, the sample sizes are too small to provide valid estimates and confidence intervals, unless further model assumptions are used, as in some small area estimation methods. With order constraints, design-based estimation and inference are possible for substantially smaller domain sample sizes, compared to the unconstrained design-based estimation.
+
+\section{Constrained domain means with a block ordering}\label{constrained-domain-means-with-a-block-ordering}
+
+We consider the 2019 National Survey of College Graduates (NSCG), conducted by the U.S. Census Bureau and sponsored by the National Center for Science and Engineering Statistics (NCSES) within the National Science Foundation. The NSCG provides data on the characteristics of the nation's college graduates, with a focus on those in the science and engineering workforce. The datasets and documentation are available to the public on the \href{https://www.nsf.gov/statistics/srvygrads}{National Survey of College Graduates (NSCF) website}. Replicate weights are available separately from NCSES upon request. Because the size of subsets of the NSCG survey used in this paper exceeds the size limit allowed for an R package stored on CRAN, the subsets are not included in the \CRANpkg{csurvey} package. Instead, we provide the link to access the subsets \texttt{nscg19.rda} and \texttt{nscg19\_2.rda} used in this section at this \href{https://github.com/xliaosdsu/csurvey-data}{website}.
+
+The study variable of interest is annual salary (denoted as SALARY in the dataset), which exhibits substantial right-skewness in its raw form. To reduce the influence of outliers and improve model stability, we restricted the analysis to observations with annual salaries between \$30,000 and \$400,000. A logarithmic transformation was applied to the salary variable to address skewness. Four predictors of salary were considered:
+
+\begin{itemize}
+\item
+ Field of study (\texttt{field}, denoted by \texttt{NDGMEMG} in the raw dataset): This nominal variable defines the field of study for the highest degree. There are five levels: (1) Computer and mathematical sciences; (2) Biological, agricultural and environmental life sciences; (3) Physical and related sciences; (4) Social and related sciences; (5) Engineering. \emph{Block ordering constraint}: given the other predictors, the average annual salary for each of the fields (2) and (4) is less than for the STEM fields (1), (3) and (5).
+\item
+ Grouped year of award of highest degree (\texttt{hd\_year\_grouped}, denoted by \texttt{HDAY5}): This ordinal variable has five levels: (1) 1995 to 1999; (2) 2000 to 2004; (3) 2005 to 2009; (4) 2010 to 2014; (5) 2015 or later. \emph{Isotonic constraint}: given the other predictors, the average annual salary decreases with the year of award of highest degree; i.e., the more experience respondents have, on average, the higher the annual salary.
+\item
+ Highest degree type (\texttt{hd\_type}, denoted by DGRDG): The three levels are: (1) Bachelor's; (2) Master's; (3) Doctorate and Professional. \emph{Isotonic constraint}: given the other predictors, the average annual salary increases with respect to the highest degree type.
+\item
+ Region code for employer (\texttt{region}, denoted by EMRG): This nominal variable defines the regions in which the respondents worked within the U.S. Nine levels are: (1) New England; (2) Middle Atlantic; (3) East North Central; (4) West North Central; (5) South Atlantic; (6) East South Central; (7) West South Central; (8) Mountain; (9) Pacific and US Territories. There is no constraint for this predictor.
+\end{itemize}
+
+This data set contains \(n=30,368\) observations in a four-dimensional grid of 675 domains, where the sample size in the domains ranges from one to 491. Here, we specify the shape and order constraints in a similar fashion as we did in previous examples. The symbolic routine \code{block.Ord} is used to impose a block ordering on \texttt{field} and the order is specified in the \code{order} argument. The \code{svydesign} specifies a survey design with no clusters. The command \code{svrepdesign} creates a survey design with replicate weights, where the columns named as ``RW0001'', ``RW0002'',\ldots,``RW0320'' are the 320 NSCG replicate weights and \code{weights( = \textasciitilde w)} denotes the sampling weight. The variance is computed as the sum of squared deviation of the replicates from the mean. The general formula for computing a variance estimate using replicate weights follows:
+\[v_{REP}(\hat{\theta})=\sum_{r=1}^{R}c_r(\hat{\theta}_r-\hat{\theta})^2\]
+where the estimate \(\hat{\theta}\) is computed based on the final full sample survey weights and the calculation of each replicate estimate \(\hat{\theta}_r\) is according to the \(r\)th set of replicate weights (\(r=1,\cdots, R\)). The replication adjustment factor \(c_r\) is a multiplier for the \(r\)th replicate of squared difference. The \code{scale( = 1)} is an overall multiplier and the \code{rscale( = 0.05)} denotes a vector of replicate-specific multipliers, which are the values of \(c_r\) in above formula.
+
+\begin{verbatim}
+load("./nscg19.rda")
+rds <- svrepdesign(data = nscg, repweights = dplyr::select(nscg, "RW0001":"RW0320"),
+ weights = ~w, combined.weights = TRUE, mse = TRUE, type = "other", scale = 1, rscale = 0.05)
+\end{verbatim}
+
+Estimates of domain means for the 225 domains for which the highest degree is a Bachelor's are shown in Figure \ref{fig:surface9}, as surfaces connecting the estimates over the grid of field indicators and year of award of highest degree. The block-ordering constraints can be seen in the surfaces, where the fields labeled 2 and 4 have lower means than those labeled 1, 3, and 5. The surfaces are also constrained to be decreasing in year of award of highest degree.
+
+To improve computational efficiency, the simulations used to estimate the mixture covariance matrix and the sampling distribution of the test statistic can be parallelized. This can be enabled by setting the following R option
+
+\begin{verbatim}
+options(csurvey.multicore = TRUE)
+\end{verbatim}
+
+The model is fitted using the following code
+
+\begin{verbatim}
+ans <- csvy(logSalary~decr(hd_year_grouped)*incr(hd_type)*block.Ord(field,
+ order = c(2, 1, 2, 1, 2))*region, design = rds)
+\end{verbatim}
+
+The \code{plotpersp} function is used to create Figure \ref{fig:surface9}. It will create a three-dimensional estimated surface plot when there are at least two predictors. In this example, we use \code{plotpersp} to generate a 3D perspective plot of the estimated average log-transformed salary with respect to field and year of award of highest degree. Among the three \texttt{hd\_type} categories, the Bachelor's degree group has the highest number of observations. The \code{plotpersp} function visualizes the 225 domains corresponding to the most frequently observed level of this fourth predictor. Plot aesthetics are customized via the \texttt{control} argument:
+
+\begin{verbatim}
+ctl <- list(categ = "region", categnm = c("New England", "Middle Atlantic", "East North Central",
+ "West North Central", "South Atlantic", "East South Central", "West South Central", "Mountain",
+ "Pacific and US Territories"), NCOL = 3, th = 60, xlab = "years since degree",
+ ylab = "field of study", zlab = "log(salary)")
+
+plotpersp(ans, x1="hd_year_grouped", x2="field", control = ctl)
+\end{verbatim}
+
+\begin{figure}
+
+{\centering \includegraphics[width=0.6\linewidth]{figures/new_surfaces9}
+
+}
+
+\caption{Estimates of average log(salary) by field of study and year of degree, for observations where highest degree is a Bachelor's, for each of the nine regions.}\label{fig:surface9}
+\end{figure}
+
+Estimates and confidence intervals for the 75 domain means associated with three of the regions are shown in Figure \ref{fig:NEreg}. The constrained estimates are shown as blue dots, and the unconstrained means are depicted as grey dots. The confidence intervals are indicated with blue bands for the constrained estimates and grey bands for the unconstrained estimates.
+
+For the Northeast Region, the average length of the constrained confidence intervals is .353, while the average length for the unconstrained confidence intervals is .477. In the domain for those in the math and computer science field, with PhDs obtained in 2000-2004, the unconstrained log-salary estimate for this domain is much below the corresponding constrained estimate, because the latter is forced to be at least that of lower degrees, and that of newer PhDs. If the constraints hold in the population, then the unconstrained confidence interval is unlikely to capture the population value. As seen in the NHANES example in Section \ref{sec-monotonic}, the unconstrained estimators are unreliable when the sample domain size is small. The Pacific region has the largest sample sizes, ranging from 12 to 435 observations. With this larger sample size, most of the unconstrained estimates already satisfy the constraints, but the average length for the constrained estimator is .301, while the average length for the unconstrained estimator is .338, showing that the mixture covariance matrix leads to more precision in the confidence intervals. Also shown is the region with the smallest sample sizes: the East South Central region. Here the average length for the unconstrained confidence intervals is .488 while the average length for the constrained confidence intervals is .444.
+
+\begin{figure}
+
+{\centering \includegraphics[width=1\linewidth]{figures/newplot4}
+
+}
+
+\caption{Estimates of average log(salary) for the 75 domains in each of three regions. The blue dots represent the constrained domain mean estimates, while the grey dots represent the unconstrained domain mean estimates. The blue band is the 95\% confidence interval for the domains, using the constraints; the grey band is the 95\% unconstrained domain mean confidence interval.}\label{fig:NEreg}
+\end{figure}
+
+\section{One-sided testing}\label{one-sided-testing}
+
+For this example we again use the NCGS data set. But for this example, we use some new variables and make some variable groupings. First, instead of using the grouped year of award of highest degree (\texttt{hd\_year\_grouped}), we use the actual calendar year when the highest degree was awarded (\texttt{hd\_year}, denoted as \texttt{HDACYR} in the raw data set). We conduct the one-sided test for six pairs of consecutive calendar years. Besides, we choose father's education level (\texttt{daded}, denoted as \texttt{EDDAD}) as the main predictor, and the interest is in determining whether salaries are higher for people whose father has a higher education. In the raw data set, \texttt{EDDAD} has seven categories, we group them into five categories for this example: 1 = no high school degree, 2 = high school degree but no college degree, 3 = bachelor's degree, 4 = master's degree, and 5 = PhD or professional degree. We further group \texttt{region} (\texttt{EMRG}) as: 1 = Northeast, 2 = North Central, 3 = Southeast, 4 = West, 5 = Pacific and Territories, and group \texttt{field} (\texttt{NDGMEMG}) as: 1 = Computer and Mathematical Sciences, Physical and Related Sciences, Engineering, which can be considered as core STEM fields, 2 = Biological, Agricultural, and Environmental Life Sciences, Social and Related Sciences, which can be considered as life and social sciences, 3 = Science and Engineering-Related Fields, 4 = Non-Science and Engineering Fields, which is Non-STEM. Finally, people's highest degree type (denoted as \texttt{DGRDG} in the raw data) is used to limit the study sample to people whose highest degree is a Master's degree. The sample size is \(n=25,177\).
+
+It seems reasonable that people whose parents are more educated are more educated themselves, but if we control for the person's level of education, as well as field, region, and year of degree, will the salary still be increasing with level of father's education? For this, the null hypothesis is that within the region, field, and year of degree, the salary is constant over levels of father's education. The one-sided alternative is that salaries increase with father's education level.
+
+The constrained and unconstrained fits to the data under the alternative hypotheses are shown in Figure \ref{fig:test} for five regions and four fields, for degree years 2016-2017. The dots connected by solid blue lines represent the fit with the one-sided alternative, i.e., constrained to be increasing in father's education. The dots connected by solid red lines represent the fit with the two-sided alternative, i.e., with unconstrained domain means.
+
+\begin{figure}
+
+{\centering \includegraphics[width=1\linewidth]{figures/daded}
+
+}
+
+\caption{Estimates of average log(salary) by father's education level, for each of five regions and four fields, for subjects whose degree was attained in 2016-2017. The solid blue lines connect the estimates where the average salary is constrained to be increasing in father's education, and the solid red lines connect unconstrained estimates of average salary.}\label{fig:test}
+\end{figure}
+
+The \(p\)-values for the test of the null hypotheses are given in Table \ref{tab:comppv}, where n/a is shown for the two-sided \(p\)-value in the case of an empty domain. The test is conducted for six pairs of consecutive calendar years with the sample sizes to be 2,129, 2,795, 3,069, 2,895, 2,423, and 1,368 respectively. The one-sided test consistently has a small \(p\)-value, indicating that the father's education level is positively associated with salary earned, even after controlling for region, degree type, and field.
+
+\begin{table}[!h]
+\centering
+\caption{\label{tab:comppv}One-sided and two-sided $p$-values for the test of the null hypothesis that salary is constant in father's education level. The two-sided test results in n/a when the grid has at least one empty domain.}
+\centering
+\begin{tabular}[t]{llllllllllll}
+\toprule
+\multicolumn{2}{c}{2008-09} & \multicolumn{2}{c}{2010-11} & \multicolumn{2}{c}{2012-13} & \multicolumn{2}{c}{2014-15} & \multicolumn{2}{c}{2016-17} & \multicolumn{2}{c}{2018-19} \\
+\cmidrule(l{3pt}r{3pt}){1-2} \cmidrule(l{3pt}r{3pt}){3-4} \cmidrule(l{3pt}r{3pt}){5-6} \cmidrule(l{3pt}r{3pt}){7-8} \cmidrule(l{3pt}r{3pt}){9-10} \cmidrule(l{3pt}r{3pt}){11-12}
+one & two & one & two & one & two & one & two & one & two & one & two\\
+\midrule
+.008 & n/a & <.001 & .018 & <.001 & <.001 & <.001 & n/a & .003 & .417 & <.001 & n/a\\
+\bottomrule
+\end{tabular}
+\end{table}
+
+The \(p\)-value of the one-sided test is included in the object \code{ans}. For example, to check the \(p\)-value for the fit for the year 2008 to 2009, we can fit the model and print out its summary table as:
+
+\begin{verbatim}
+load("./nscg19_2.rda")
+data <- nscg2 |>
+ dplyr::filter(hd_year %in% c(2008, 2009))
+
+rds <- svrepdesign(data = data, repweights = dplyr::select(data, "RW0001":"RW0320"), weights = ~w,
+ combined.weights = TRUE, mse = TRUE, type = "other",
+ scale = 1, rscale = 0.05)
+
+set.seed(1)
+ans <- csvy(logSalary ~ incr(daded) * field * region, design = rds, test = TRUE)
+\end{verbatim}
+
+\begin{verbatim}
+summary(ans)
+\end{verbatim}
+
+\begin{verbatim}
+#> Call:
+#> csvy(formula = logSalary ~ incr(daded) * field * region, design = rds,
+#> test = TRUE)
+#>
+#> Null deviance: 460.6182 on 97 degrees of freedom
+#> Residual deviance: 391.3722 on 51 degrees of freedom
+#>
+#> Approximate significance of constrained fit:
+#> edf mixture.of.Beta p.value
+#> incr(daded):field:region 47 0.5536 0.0068 **
+#> ---
+#> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
+#> CIC (constrained estimator): 0.0275
+#> CIC (unconstrained estimator): 0.0354
+\end{verbatim}
+
+\section{Binary outcome}\label{binary-outcome}
+
+Finally, we use another subset of the NHANES 2009--2010 data to demonstrate how our method applies when the outcome is binary. This subset is included in the \CRANpkg{csurvey} package as \texttt{nhdat}. The construction of variables, sampling weights, and strata in this subset closely follows the approach described in Section \ref{sec-monotonic}. It contains \(n = 1,680\) observations with complete records on total cholesterol, age, height, and waist circumference for adults aged 21--40. The binary outcome indicates whether an individual has high total cholesterol, coded as 1 if total cholesterol exceeds 200 mg/dL, and 0 otherwise. We estimate the population proportion with high cholesterol by age, waist, and gender (1 = male, 2 = female). The waist variable, denoted as \texttt{wcat}, is a 4-level categorized ordinal variable representing waist-to-height ratios.
+
+It is reasonable to assume that, on average, the proportion of individuals with high cholesterol increases with both age and waist. The model is specified using the following code:
+
+\begin{verbatim}
+data(nhdat, package = "csurvey")
+dstrat <- svydesign(ids = ~ id, strata = ~ str, data = nhdat, weight = ~ wt)
+set.seed(1)
+ans <- csvy(chol ~ incr(age) * incr(wcat) * gender, design = dstrat,
+ family = binomial(link = "logit"), test = TRUE)
+\end{verbatim}
+
+The CIC of the constrained estimator is smaller than that of the unconstrained estimator, and the one-sided hypothesis test has a \(p\)-value close to zero.
+
+\begin{verbatim}
+summary(ans)
+\end{verbatim}
+
+\begin{verbatim}
+#> Call:
+#> csvy(formula = chol ~ incr(age) * incr(wcat) * gender, design = dstrat,
+#> family = binomial(link = "logit"), test = TRUE)
+#>
+#> Null deviance: 2054.947 on 159 degrees of freedom
+#> Residual deviance: 1906.762 on 126 degrees of freedom
+#>
+#> Approximate significance of constrained fit:
+#> edf mixture.of.Beta p.value
+#> incr(age):incr(wcat):gender 34 0.5174 < 2.2e-16 ***
+#> ---
+#> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
+#> CIC (constrained estimator): 0.3199
+#> CIC (unconstrained estimator): 1.2778
+\end{verbatim}
+
+The combination of age, waist, and gender gives 160 domains. This implies that the average sample size for each domain is only around 10. Due to the small sample sizes, the unconstrained estimator shows unlikely jumps as age increases within each waist category. On the other hand, the constrained estimator is more stable and tends to have smaller confidence intervals compared with the unconstrained Hájek estimator.
+
+\begin{figure}
+
+{\centering \includegraphics[width=1\linewidth]{figures/nhanes_bin}
+
+}
+
+\caption{Estimates of probability of high cholesterol level for each combination of age, waist and gender. The blue dots represent the constrained domain mean estimates, while the green dots represent the unconstrained domain mean estimates. The blue band is the 95\% confidence interval for the domains, using the constraints; the green band is the 95\% unconstrained domain mean confidence interval.}\label{fig:nhanesbin}
+\end{figure}
+
+\section{Discussion}\label{discussion}
+
+While model-based small area estimators - such as those implemented in the \CRANpkg{sae} package \citep{sae2015} and the \CRANpkg{emdi} package \citep{emdi2019} - are powerful tools for borrowing strength across domains, they rely on parametric assumptions that may be violated in practice. Design-based methods remain essential for official statistical agencies, as they provide transparent and model-free inference that is directly tied to the survey design. Estimation and inference for population domain means with survey data can be substantially improved if constraints based on natural orderings are implemented. The \CRANpkg{csurvey} package (version 1.15) \citep{csurvey2025} allows users to specify orderings on grids of domains, and obtain estimates of and confidence intervals for population domain means. This package also implements the design-based small area estimation method, which allows inference for population domain means for which the sample domain is empty, and further is used to improve estimates for domains with small sample size. The one-sided testing procedure available in \CRANpkg{csurvey} has higher power than the standard two-sided test, and further can be applied in grids with some empty domains. Confidence intervals for domain means have better coverage rate and smaller interval width than what is produced by unconstrained estimation. Finally, the package provides functions to allow the user to easily visualize the data and the fits. The utility of the package has been demonstrated with well-known survey data sets.
+
+\textbf{Acknowledgment:} This work was partially funded by NSF MMS-1533804.
+
+\bibliography{article.bib}
+
+\address{%
+Xiyue Liao\\
+San Diego State University\\%
+San Diego State University\\ Department of Mathematics and Statistics\\ San Diego, California 92182, United States of America\\
+%
+%
+\textit{ORCiD: \href{https://orcid.org/0000-0002-4508-9219}{0000-0002-4508-9219}}\\%
+\href{mailto:xliao@sdsu.edu}{\nolinkurl{xliao@sdsu.edu}}%
+}
+
+\address{%
+Mary C. Meyer\\
+Colorado State University\\%
+Colorado State University\\ Department of Statistics\\ Fort Collins, Colorado 80523, United States of America\\
+%
+%
+%
+\href{mailto:meyer@stat.colostate.edu}{\nolinkurl{meyer@stat.colostate.edu}}%
+}
diff --git a/_articles/RJ-2025-032/RJ-2025-032.zip b/_articles/RJ-2025-032/RJ-2025-032.zip
new file mode 100644
index 0000000000..72e3ec0c39
Binary files /dev/null and b/_articles/RJ-2025-032/RJ-2025-032.zip differ
diff --git a/_articles/RJ-2025-032/RJournal.sty b/_articles/RJ-2025-032/RJournal.sty
new file mode 100644
index 0000000000..351990be38
--- /dev/null
+++ b/_articles/RJ-2025-032/RJournal.sty
@@ -0,0 +1,358 @@
+% Package `RJournal' to use with LaTeX2e
+% Copyright (C) 2010 by the R Foundation
+% Copyright (C) 2013 by the R Journal
+%
+% Originally written by Kurt Hornik and Friedrich Leisch with subsequent
+% edits by the editorial board
+%
+% CAUTION:
+% Do not modify this style file. Any changes to this file will be reset when your
+% article is submitted.
+% If you must modify the style or add LaTeX packages to the article, these
+% should be specified in RJwrapper.tex
+
+\NeedsTeXFormat{LaTeX2e}[1995/12/01]
+\ProvidesPackage{RJournal}[2025/10/05 v0.17 RJournal package]
+
+\RequirePackage{tikz}
+
+% Overall page layout, fonts etc -----------------------------------------------
+
+% Issues of of \emph{The R Journal} are created from the standard \LaTeX{}
+% document class \pkg{report}.
+
+\RequirePackage{geometry}
+\geometry{a4paper,
+ textwidth=14cm, top=1cm, bottom=1cm,
+ includehead,includefoot,centering,
+ footskip=1.5cm}
+\raggedbottom
+\sloppy
+\clubpenalty = 10000
+\widowpenalty = 10000
+\brokenpenalty = 10000
+\usepackage{microtype}
+
+
+\RequirePackage{fancyhdr}
+\fancyhead{}
+\fancyheadoffset{2cm}
+\fancyhead[L]{\textsc{\RJ@sectionhead}}
+\fancyhead[R]{\thepage}
+\fancyfoot{}
+\fancyfoot[L]{The R Journal Vol. \RJ@volume/\RJ@number, \RJ@month~\RJ@year}
+\fancyfoot[R]{ISSN 2073-4859}
+\pagestyle{fancy}
+
+% We use the following fonts (all with T1 encoding):
+%
+% rm & palatino
+% tt & inconsolata
+% sf & helvetica
+% math & palatino
+
+\RequirePackage{microtype}
+
+\RequirePackage[scaled=0.92]{helvet}
+\RequirePackage{palatino,mathpazo}
+\RequirePackage[scaled=1.02]{inconsolata}
+\RequirePackage[T1]{fontenc}
+
+\RequirePackage[hyphens]{url}
+\RequirePackage[pagebackref]{hyperref}
+\renewcommand{\backref}[1]{[p#1]}
+
+% Dark blue colour for all links
+\RequirePackage{color}
+\definecolor{link}{rgb}{0.45,0.51,0.67}
+\hypersetup{
+ colorlinks,%
+ citecolor=link,%
+ filecolor=link,%
+ linkcolor=link,%
+ urlcolor=link
+}
+
+% Give the text a little room to breath
+\setlength{\parskip}{3pt}
+\RequirePackage{setspace}
+\setstretch{1.05}
+
+% Issue and article metadata ---------------------------------------------------
+
+% Basic front matter information about the issue: volume, number, and
+% date.
+
+\newcommand{\volume}[1]{\def\RJ@volume{#1}}
+\newcommand{\volnumber}[1]{\def\RJ@number{#1}}
+\renewcommand{\month}[1]{\def\RJ@month{#1}}
+\renewcommand{\year}[1]{\def\RJ@year{#1}}
+
+
+% Individual articles correspond to
+% chapters, and are contained in |article| environments. This makes it
+% easy to have figures counted within articles and hence hyperlinked
+% correctly.
+
+% An article has an author, a title, and optionally a subtitle. We use
+% the obvious commands for specifying these. Articles will be put in certain
+% journal sections, named by \sectionhead.
+
+\newcommand {\sectionhead} [1]{\def\RJ@sectionhead{#1}}
+\renewcommand{\author} [1]{\def\RJ@author{#1}}
+\renewcommand{\title} [1]{\def\RJ@title{#1}}
+\newcommand {\subtitle} [1]{\def\RJ@subtitle{#1}}
+
+% Control appearance of titles: make slightly smaller than usual, and
+% suppress section numbering. See http://tex.stackexchange.com/questions/69749
+% for why we don't use \setcounter{secnumdepth}{-1}
+
+\usepackage[medium]{titlesec}
+\usepackage{titletoc}
+\titleformat{\section} {\normalfont\large\bfseries}{\arabic{section}}{1em}{}
+\titleformat{\subsection}{\normalfont\normalsize\bfseries}{\arabic{section}.\arabic{subsection}}{0.5em}{}
+\titlecontents{chapter} [0em]{}{}{}{\titlerule*[1em]{.}\contentspage}
+
+% Article layout ---------------------------------------------------------------
+
+% Environment |article| clears the article header information at its beginning.
+% We use |\FloatBarrier| from the placeins package to keep floats within
+% the article.
+\RequirePackage{placeins}
+\newenvironment{article}{\author{}\title{}\subtitle{}\FloatBarrier}{\FloatBarrier}
+
+% Refereed articles should have an abstract, so we redefine |\abstract| to
+% give the desired style
+
+\renewcommand{\abstract}[1]{\noindent\textbf{Abstract} #1}
+\renewenvironment{abstract}{\noindent\textbf{Abstract}~}{}
+
+% The real work is done by a redefined version of |\maketitle|. Note
+% that even though we do not want chapters (articles) numbered, we
+% need to increment the chapter counter, so that figures get correct
+% labelling.
+
+\renewcommand{\maketitle}{%
+\noindent
+ \chapter{\RJ@title}\refstepcounter{chapter}
+ \ifx\empty\RJ@subtitle
+ \else
+ \noindent\textbf{\RJ@subtitle}
+ \par\nobreak\addvspace{\baselineskip}
+ \fi
+ \ifx\empty\RJ@author
+ \else
+ \noindent\textit{\RJ@author}
+ \par\nobreak\addvspace{\baselineskip}
+ \fi
+ \@afterindentfalse\@nobreaktrue\@afterheading
+}
+
+% Now for some ugly redefinitions. We do not want articles to start a
+% new page. (Actually, we do, but this is handled via explicit
+% \newpage
+%
+% The name@of@eq is a hack to get hyperlinks to equations to work
+% within each article, even though there may be multiple eq.(1)
+% \begin{macrocode}
+\renewcommand\chapter{\secdef\RJ@chapter\@schapter}
+\providecommand{\nohyphens}{%
+ \hyphenpenalty=10000\exhyphenpenalty=10000\relax}
+\newcommand{\RJ@chapter}{%
+ \edef\name@of@eq{equation.\@arabic{\c@chapter}}%
+ \renewcommand{\@seccntformat}[1]{}%
+ \@startsection{chapter}{0}{0mm}{%
+ -2\baselineskip \@plus -\baselineskip \@minus -.2ex}{\p@}{%
+ \phantomsection\normalfont\huge\bfseries\raggedright}}
+
+% Book reviews should appear as sections in the text and in the pdf bookmarks,
+% however we wish them to appear as chapters in the TOC. Thus we define an
+% alternative to |\maketitle| for reviews.
+\newcommand{\review}[1]{
+ \pdfbookmark[1]{#1}{#1}
+ \section*{#1}
+ \addtocontents{toc}{\protect\contentsline{chapter}{#1}{\thepage}{#1.1}}
+}
+
+% We want bibliographies as starred sections within articles.
+%
+\RequirePackage[sectionbib,round]{natbib}
+\bibliographystyle{abbrvnat}
+\renewcommand{\bibsection}{\section*{References}}
+
+% Equations, figures and tables are counted within articles, but we do
+% not show the article number. For equations it becomes a bit messy to avoid
+% having hyperref getting it wrong.
+
+% \numberwithin{equation}{chapter}
+\renewcommand{\theequation}{\@arabic\c@equation}
+\renewcommand{\thefigure}{\@arabic\c@figure}
+\renewcommand{\thetable}{\@arabic\c@table}
+
+% Issue layout -----------------------------------------------------------------
+
+% Need to provide our own version of |\tableofcontents|. We use the
+% tikz package to get the rounded rectangle. Notice that |\section*|
+% is really the same as |\chapter*|.
+\renewcommand{\contentsname}{Contents}
+\renewcommand\tableofcontents{%
+ \vspace{1cm}
+ \section*{\contentsname}
+ { \@starttoc{toc} }
+}
+
+\renewcommand{\titlepage}{%
+ \thispagestyle{empty}
+ \hypersetup{
+ pdftitle={The R Journal Volume \RJ@volume/\RJ@number, \RJ@month \RJ@year},%
+ pdfauthor={R Foundation for Statistical Computing},%
+ }
+ \noindent
+ \begin{center}
+ \fontsize{50pt}{50pt}\selectfont
+ The \raisebox{-8pt}{\includegraphics[height=77pt]{Rlogo-5}}\hspace{10pt}
+ Journal
+
+ \end{center}
+ {\large \hfill Volume \RJ@volume/\RJ@number, \RJ@month{} \RJ@year \quad}
+
+ \rule{\textwidth}{1pt}
+ \begin{center}
+ {\Large A peer-reviewed, open-access publication of the \\
+ R Foundation for Statistical Computing}
+ \end{center}
+
+ % And finally, put in the TOC box. Note the way |tocdepth| is adjusted
+ % before and after producing the TOC: thus, we can ensure that only
+ % articles show up in the printed TOC, but that in the PDF version,
+ % bookmarks are created for sections and subsections as well (provided
+ % that the non-starred forms are used).
+ \setcounter{tocdepth}{0}
+ \tableofcontents
+ \setcounter{tocdepth}{2}
+ \clearpage
+}
+
+% Text formatting --------------------------------------------------------------
+
+\newcommand{\R}{R}
+\newcommand{\address}[1]{\addvspace{\baselineskip}\noindent\emph{#1}}
+\newcommand{\email}[1]{\href{mailto:#1}{\normalfont\texttt{#1}}}
+
+% Simple font selection is not good enough. For example, |\texttt{--}|
+% gives `\texttt{--}', i.e., an endash in typewriter font. Hence, we
+% need to turn off ligatures, which currently only happens for commands
+% |\code| and |\samp| and the ones derived from them. Hyphenation is
+% another issue; it should really be turned off inside |\samp|. And
+% most importantly, \LaTeX{} special characters are a nightmare. E.g.,
+% one needs |\~{}| to produce a tilde in a file name marked by |\file|.
+% Perhaps a few years ago, most users would have agreed that this may be
+% unfortunate but should not be changed to ensure consistency. But with
+% the advent of the WWW and the need for getting `|~|' and `|#|' into
+% URLs, commands which only treat the escape and grouping characters
+% specially have gained acceptance
+
+\DeclareRobustCommand\code{\bgroup\@noligs\@codex}
+\def\@codex#1{\texorpdfstring%
+{{\normalfont\ttfamily\hyphenchar\font=-1 #1}}%
+{#1}\egroup}
+\newcommand{\kbd}[1]{{\normalfont\texttt{#1}}}
+\newcommand{\key}[1]{{\normalfont\texttt{\uppercase{#1}}}}
+\DeclareRobustCommand\samp{`\bgroup\@noligs\@sampx}
+\def\@sampx#1{{\normalfont\texttt{#1}}\egroup'}
+\newcommand{\var}[1]{{\normalfont\textsl{#1}}}
+\let\env=\code
+\newcommand{\file}[1]{{`\normalfont\textsf{#1}'}}
+\let\command=\code
+\let\option=\samp
+\newcommand{\dfn}[1]{{\normalfont\textsl{#1}}}
+% \acronym is effectively disabled since not used consistently
+\newcommand{\acronym}[1]{#1}
+\newcommand{\strong}[1]{\texorpdfstring%
+{{\normalfont\fontseries{b}\selectfont #1}}%
+{#1}}
+\let\pkg=\strong
+\newcommand{\CRANpkg}[1]{\href{https://CRAN.R-project.org/package=#1}{\pkg{#1}}}%
+\let\cpkg=\CRANpkg
+\newcommand{\ctv}[1]{\href{https://CRAN.R-project.org/view=#1}{\emph{#1}}}
+\newcommand{\BIOpkg}[1]{\href{https://www.bioconductor.org/packages/release/bioc/html/#1.html}{\pkg{#1}}}
+
+% Example environments ---------------------------------------------------------
+\RequirePackage{fancyvrb}
+\RequirePackage{alltt}
+
+\DefineVerbatimEnvironment{example}{Verbatim}{}
+\renewenvironment{example*}{\begin{alltt}}{\end{alltt}}
+
+% Support for output from Sweave, and generic session style code
+% These used to have fontshape=sl for Sinput/Scode/Sin, but pslatex
+% won't use a condensed font in that case.
+
+% Update (2015-05-28 by DS): remove fontsize=\small to match example environment
+
+\DefineVerbatimEnvironment{Sinput}{Verbatim}{}
+\DefineVerbatimEnvironment{Soutput}{Verbatim}{}
+\DefineVerbatimEnvironment{Scode}{Verbatim}{}
+\DefineVerbatimEnvironment{Sin}{Verbatim}{}
+\DefineVerbatimEnvironment{Sout}{Verbatim}{}
+\newenvironment{Schunk}{}{}
+
+% Mathematics ------------------------------------------------------------------
+
+% The implementation of |\operatorname| is similar to the mechanism
+% \LaTeXe{} uses for functions like sin and cos, and simpler than the
+% one of \AmSLaTeX{}. We use |\providecommand| for the definition in
+% order to keep the one of the \pkg{amstex} if this package has
+% already been loaded.
+% \begin{macrocode}
+\providecommand{\operatorname}[1]{%
+ \mathop{\operator@font#1}\nolimits}
+\RequirePackage{amsfonts}
+
+\renewcommand{\P}{%
+ \mathop{\operator@font I\hspace{-1.5pt}P\hspace{.13pt}}}
+\newcommand{\E}{%
+ \mathop{\operator@font I\hspace{-1.5pt}E\hspace{.13pt}}}
+\newcommand{\VAR}{\operatorname{var}}
+\newcommand{\COV}{\operatorname{cov}}
+\newcommand{\COR}{\operatorname{cor}}
+
+% Figures ----------------------------------------------------------------------
+
+% For use with pandoc > 3.2.1
+\newsavebox\pandoc@box
+\newcommand*\pandocbounded[1]{% scales image to fit in text height/width
+ \sbox\pandoc@box{#1}%
+ \Gscale@div\@tempa{\textheight}{\dimexpr\ht\pandoc@box+\dp\pandoc@box\relax}%
+ \Gscale@div\@tempb{\linewidth}{\wd\pandoc@box}%
+ \ifdim\@tempb\p@<\@tempa\p@\let\@tempa\@tempb\fi% select the smaller of both
+ \ifdim\@tempa\p@<\p@\scalebox{\@tempa}{\usebox\pandoc@box}%
+ \else\usebox{\pandoc@box}%
+ \fi%
+}
+
+\RequirePackage[font=small,labelfont=bf]{caption}
+
+% Wide environments for figures and tables -------------------------------------
+\RequirePackage{environ}
+
+% An easy way to make a figure span the full width of the page
+\NewEnviron{widefigure}[1][]{
+\begin{figure}[#1]
+\advance\leftskip-2cm
+\begin{minipage}{\dimexpr\textwidth+4cm\relax}%
+ \captionsetup{margin=2cm}
+ \BODY
+\end{minipage}%
+\end{figure}
+}
+
+\NewEnviron{widetable}[1][]{
+\begin{table}[#1]
+\advance\leftskip-2cm
+\begin{minipage}{\dimexpr\textwidth+4cm\relax}%
+ \captionsetup{margin=2cm}
+ \BODY
+\end{minipage}%
+\end{table}
+}
diff --git a/_articles/RJ-2025-032/RJwrapper.tex b/_articles/RJ-2025-032/RJwrapper.tex
new file mode 100644
index 0000000000..5fb543f497
--- /dev/null
+++ b/_articles/RJ-2025-032/RJwrapper.tex
@@ -0,0 +1,72 @@
+\documentclass[a4paper]{report}
+\usepackage[utf8]{inputenc}
+\usepackage[T1]{fontenc}
+\usepackage{RJournal}
+\usepackage{amsmath,amssymb,array}
+\usepackage{booktabs}
+
+
+% tightlist command for lists without linebreak
+\providecommand{\tightlist}{%
+ \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}
+
+\usepackage{longtable}
+
+% Always define CSL refs as bib entries are contained in separate doc
+% Pandoc citation processing
+%From Pandoc 3.1.8
+% definitions for citeproc citations
+\NewDocumentCommand\citeproctext{}{}
+\NewDocumentCommand\citeproc{mm}{%
+ \begingroup\def\citeproctext{#2}\cite{#1}\endgroup}
+\makeatletter
+ % allow citations to break across lines
+ \let\@cite@ofmt\@firstofone
+ % avoid brackets around text for \cite:
+ \def\@biblabel#1{}
+ \def\@cite#1#2{{#1\if@tempswa , #2\fi}}
+\makeatother
+\newlength{\cslhangindent}
+\setlength{\cslhangindent}{1.5em}
+\newlength{\csllabelwidth}
+\setlength{\csllabelwidth}{3em}
+\newenvironment{CSLReferences}[2] % #1 hanging-indent, #2 entry-spacing
+ {\begin{list}{}{%
+ \setlength{\itemindent}{0pt}
+ \setlength{\leftmargin}{0pt}
+ \setlength{\parsep}{0pt}
+ % turn on hanging indent if param 1 is 1
+ \ifodd #1
+ \setlength{\leftmargin}{\cslhangindent}
+ \setlength{\itemindent}{-1\cslhangindent}
+ \fi
+ % set entry spacing
+ \setlength{\itemsep}{#2\baselineskip}}}
+ {\end{list}}
+\usepackage{calc}
+\newcommand{\CSLBlock}[1]{#1\hfill\break}
+\newcommand{\CSLLeftMargin}[1]{\parbox[t]{\csllabelwidth}{#1}}
+\newcommand{\CSLRightInline}[1]{\parbox[t]{\linewidth - \csllabelwidth}{#1}\break}
+\newcommand{\CSLIndent}[1]{\hspace{\cslhangindent}#1}
+
+
+% Any extra LaTeX you need in the preamble
+
+
+\begin{document}
+
+
+%% do not edit, for illustration only
+\sectionhead{Contributed research article}
+\volume{17}
+\volnumber{4}
+\year{2025}
+\month{December}
+\setcounter{page}{4}
+
+\begin{article}
+ \input{RJ-2025-032}
+\end{article}
+
+
+\end{document}
diff --git a/_articles/RJ-2025-032/article.bib b/_articles/RJ-2025-032/article.bib
new file mode 100644
index 0000000000..920284a000
--- /dev/null
+++ b/_articles/RJ-2025-032/article.bib
@@ -0,0 +1,523 @@
+@manual{csurvey2025,
+ title = {csurvey: Constrained Regression for Survey Data},
+ author = {Xiyue Liao},
+ year = {2025},
+ note = {R package version 1.15},
+ url = {https://github.com/xliaosdsu/csurvey},
+ doi = {10.32614/CRAN.package.csurvey}
+}
+@article{sae2015,
+ author = {Isabel Molina and Yolanda Marhuenda},
+ title = {{sae}: An {R} Package for Small Area Estimation},
+ journal = {The R Journal},
+ year = {2015},
+ volume = {7},
+ number = {1},
+ pages = {81--98},
+ month = {jun},
+ url = {https://journal.r-project.org/archive/2015/RJ-2015-007/RJ-2015-007.pdf}
+}
+@article{emdi2019,
+ title = {The {R} Package {emdi} for Estimating and Mapping Regionally Disaggregated Indicators},
+ author = {Ann-Kristin Kreutzmann and S\"oren Pannier and Natalia Rojas-Perilla and Timo Schmid and Matthias Templ and Nikos Tzavidis},
+ journal = {Journal of Statistical Software},
+ year = {2019},
+ volume = {91},
+ number = {7},
+ pages = {1--33},
+ doi = {10.18637/jss.v091.i07}
+}
+@manual{dtable,
+ title = {data.table: Extension of `data.frame`},
+ author = {Tyson Barrett and Matt Dowle and Arun Srinivasan and Jan Gorecki and Michael Chirico and Toby Hocking and Benjamin Schwendinger and Ivan Krylov},
+ year = {2025},
+ note = {R package version 1.17.0},
+ url = {https://CRAN.R-project.org/package=data.table},
+ doi = {10.32614/CRAN.package.data.table}
+}
+@manual{igraph,
+ title = {{igraph}: Network Analysis and Visualization in {R}},
+ author = {Cs{\'a}rdi, G{\'a}bor and Nepusz, Tam{\'a}s and Traag, Vincent and Horv{\'a}t, Szabolcs and Fabio Zanini and Daniel Noom and Kirill Müller},
+ year = {2025},
+ note = {R package version 2.1.4},
+ doi = {10.5281/zenodo.7682609},
+ url = {https://CRAN.R-project.org/package=igraph}
+}
+@manual{dplyr,
+ title = {dplyr: A Grammar of Data Manipulation},
+ author = {Hadley Wickham and Romain François and Lionel Henry and Kirill Müller and Davis Vaughan},
+ year = {2023},
+ note = {R package version 1.1.4},
+ url = {https://CRAN.R-project.org/package=dplyr},
+ doi = {10.32614/CRAN.package.dplyr}
+}
+@book{ggplot2,
+ author = {Hadley Wickham},
+ title = {ggplot2: Elegant Graphics for Data Analysis},
+ publisher = {Springer-Verlag New York},
+ year = {2016},
+ isbn = {978-3-319-24277-4},
+ url = {https://ggplot2.tidyverse.org}
+}
+@book{mass,
+ title = {Modern Applied Statistics with {S}},
+ author = {W. N. Venables and B. D. Ripley},
+ publisher = {Springer},
+ edition = {Fourth},
+ address = {New York},
+ year = {2002},
+ note = {ISBN 0-387-95457-0},
+ url = {https://www.stats.ox.ac.uk/pub/MASS4/}
+}
+@manual{matrix,
+ title = {Matrix: Sparse and Dense Matrix Classes and Methods},
+ author = {Douglas Bates and Martin Maechler and Mikael Jagan},
+ year = {2025},
+ note = {R package version 1.7-3},
+ url = {https://CRAN.R-project.org/package=Matrix},
+ doi = {10.32614/CRAN.package.Matrix}
+}
+@article{bic78,
+ title = {Estimating the Dimension of a Model},
+ author = {Gideon Schwarz},
+ year = {1978},
+ journal = {Annals of Statistics},
+ volume = {6},
+ number = {2},
+ pages = {461-464},
+ doi = {10.1214/aos/1176344136}
+}
+@inproceedings{aic73,
+ author = {Akaike, Hirotugu},
+ booktitle = {Second International Symposium on Information Theory},
+ editor = {Petrov, B. N. and Csaki, F.},
+ pages = {267--281},
+ publisher = {{Akad{\'e}miai Kiad{\'o}}},
+ title = {Information theory as an extension of the maximum likelihood principle},
+ year = {1973}
+}
+@article{liao23,
+ author = {Xiyue Liao and Mary C. Meyer and Xiaoming Xu},
+ journal = {Survey Methodology},
+ number = {2},
+ pages = {303-321},
+ title = {Design-Based Estimation of Small and Empty Domains in Survey Data Analysis using Order Constraints},
+ url = {http://www.statcan.gc.ca/pub/12-001-x/2024002/article/00010-eng.pdf},
+ volume = {50},
+ year = {2024}
+}
+@article{survey,
+ author = {Thomas Lumley},
+ doi = {10.18637/jss.v009.i08},
+ journal = {Journal of Statistical Software},
+ number = {8},
+ pages = {1-19},
+ title = {Analysis of Complex Survey Samples},
+ volume = {9},
+ year = {2004}
+}
+@article{daniel14,
+ author = {Daniel Oberski},
+ doi = {10.18637/jss.v057.i01},
+ journal = {Journal of Statistical Software},
+ number = {1},
+ pages = {1--27},
+ title = {lavaan.survey: An {R} Package for Complex Survey Analysis of Structural Equation Models},
+ volume = {57},
+ year = {2014}
+}
+@article{andreas13,
+ author = {Andreas Alfons and Matthias Templ},
+ doi = {10.18637/jss.v054.i15},
+ journal = {Journal of Statistical Software},
+ number = {15},
+ pages = {1--25},
+ title = {Estimation of Social Exclusion Indicators from Complex Surveys: The {R} Package laeken},
+ volume = {54},
+ year = {2013}
+}
+@article{Imai05,
+ author = {Imai, Kosuke and Van Dyk, David A.},
+ doi = {10.18637/jss.v014.i03},
+ journal = {Journal of Statistical Software},
+ number = {3},
+ pages = {1--32},
+ title = {{MNP}: {R} Package for Fitting the Multinomial Probit Model},
+ volume = {14},
+ year = {2005}
+}
+@article{xu23,
+ author = {Xiaoming Xu and Mary C. Meyer},
+ journal = {Survey Methodology},
+ number = {1},
+ pages = {117-138},
+ title = {One-Sided Testing of Population Domain Means in Surveys},
+ url = {https://www150.statcan.gc.ca/n1/en/pub/12-001-x/2023001/article/00001-eng.pdf},
+ volume = {49},
+ year = {2023}
+}
+@article{boistard12,
+ author = {Helene Boistard and Hendrik P. Lopuhaa},
+ journal = {Electronic Journal of Statistics},
+ pages = {1967-1983},
+ title = {Approximation of rejective sampling inclusion probabilities and application to high order correlations},
+ volume = {6},
+ year = {2012}
+}
+@article{xu20,
+ author = {Xiaoming Xu and Mary C. Meyer and Jean D. Opsomer},
+ doi = {10.1016/j.jspi.2021.02.004},
+ journal = {Journal of Statistical Planning and Inference},
+ pages = {47-71},
+ title = {Improved Variance Estimation for Inequality-Constrained Domain Mean Estimators using Survey Data},
+ volume = {215},
+ year = {2021}
+}
+@article{opsomer2005,
+ author = {Opsomer, J.D. and Miller, C.P.},
+ journal = {Journal of Nonparametric Statistics},
+ number = {5},
+ pages = {593--611},
+ title = {Selecting the amount of smoothing in nonparametric regression estimation for complex surveys},
+ volume = {17},
+ year = {2005}
+}
+@article{breidt17,
+ author = {F. J. Breidt and J. D. Opsomer},
+ journal = {Statistical Science},
+ number = {2},
+ pages = {190-205},
+ title = {Model-Assisted Survey Estimation with Modern Prediction Techniques},
+ volume = {32},
+ year = {2017}
+}
+@article{oliva20,
+ author = {Cristian Oliva-Avil\'es AND Mary C. Meyer AND Jean D. Opsomer},
+ journal = {Survey Methology},
+ number = {2},
+ pages = {145-180},
+ title = {Estimation and Inference of Domain Means Subject to Qualitative Constraints},
+ url = {https://www150.statcan.gc.ca/n1/pub/12-001-x/2020002/article/00002-eng.htm.},
+ volume = {46},
+ year = {2020}
+}
+@incollection{akaike73,
+ address = {Budapest},
+ author = {Akaike, H.},
+ booktitle = {Second International Symposium on Information Theory},
+ editor = {Petrov, B. N. and Csaki, F.},
+ pages = {267-281},
+ publisher = {Akademiai Kiado},
+ title = {Information theory as an extension of the maximum likelihood principle},
+ year = {1973}
+}
+@manual{bcgam,
+ author = {Cristian Oliva-Aviles and Mary C. Meyer},
+ note = {{R} package version 1.0, URL \url{https://CRAN.R-project.org/package=bcgam}},
+ title = {bcgam: {Bayesian} Constrained Generalized Additive Model},
+ year = {2018}
+}
+@article{brunk55,
+ author = {Brunk, H. D.},
+ journal = {The Annals of Mathematical Statistics},
+ pages = {1026-1053},
+ title = {Maximum Likelihood Estimates of Monotone Parameters},
+ volume = {28},
+ year = {1955}
+}
+@article{brunk58,
+ author = {Brunk, H. D.},
+ doi = {10.1214/aoms/1177728420},
+ journal = {The Annals of Mathematical Statistics},
+ number = {2},
+ pages = {437-454},
+ title = {Maximum Likelihood Estimates of Monotone Parameters},
+ volume = {29},
+ year = {1958}
+}
+@manual{cgam,
+ author = {Mary C. Meyer and Xiyue Liao},
+ note = {{R} package version 1.9, URL \url{https://CRAN.R-project.org/package=cgam}},
+ title = {cgam: Constrained Generalized Additive Model},
+ year = {2017}
+}
+@article{liao19,
+ author = {Liao, X. and Meyer, M. C.},
+ doi = {10.18637/jss.v089.i05},
+ journal = {Journal of Statistical Software},
+ number = {5},
+ pages = {1--24},
+ title = {cgam: An {R} Package for the Constrained Generalized Additive Model},
+ volume = {89},
+ year = {2019}
+}
+@incollection{duncan61,
+ author = {Duncan, Otis Dudley},
+ booktitle = {Occupations and Social Status},
+ editor = {{Reiss, A. J. Jr.}},
+ publisher = {New York: Free Press [Table VI-1]},
+ title = {A socioeconomic index for all occupations},
+ year = {1961}
+}
+@article{fayherriot79,
+ author = {Fay, R. E. and Herriot, R. A.},
+ journal = {Journal of the American Statistical Association},
+ pages = {761-766},
+ title = {Estimates of income for small places: an application of {J}ames-{S}tein procedures to census data},
+ volume = {52},
+ year = {1979}
+}
+@book{fenchel53,
+ address = {Princeton, New Jersey},
+ author = {Fenchel, W.},
+ publisher = {{Mimeographed notes by D. W. Blackett, Princeton Univ. Press}},
+ title = {Convex cones, sets, and functions},
+ year = {1953}
+}
+@article{fuller99,
+ author = {Fuller, W. A.},
+ journal = {Journal of the Agricultural, Biological and Environmental Statistics},
+ pages = {331-345},
+ title = {Environmental surveys over time},
+ volume = {4},
+ year = {1999}
+}
+@incollection{hajek71,
+ address = {Toronto},
+ author = {H\'ajek, J.},
+ booktitle = {Foundations of Statistical Inference},
+ editor = {V. P. Godambe and D. A. Sprott},
+ pages = {236},
+ publisher = {Holt, Rinehart and Winston},
+ title = {Comment on a paper by {D. Basu}},
+ year = {1971}
+}
+@article{horvitz52,
+ author = {Horvitz, D. G. and Thompson, D. J.},
+ journal = {Journal of the American Statistical Association},
+ pages = {663-685},
+ title = {A generalization of sampling without replacement from a finite universe},
+ volume = {47},
+ year = {1952}
+}
+@article{kott2001,
+ author = {Kott, P. S.},
+ journal = {Journal of Official Statistics},
+ pages = {521-526},
+ title = {The delete-a-group jackknife},
+ volume = {17},
+ year = {2001}
+}
+@article{coneproj,
+ author = {Xiyue Liao and Mary C. Meyer},
+ doi = {10.18637/jss.v061.i12},
+ journal = {Journal of Statistical Software},
+ number = {12},
+ pages = {1--22},
+ title = {coneproj: An {R} Package for the Primal or Dual Cone Projections with Routines for Constrained Regression},
+ volume = {61},
+ year = {2014}
+}
+@article{meyer08,
+ author = {Meyer, M. C.},
+ journal = {Annals of Applied Statistics},
+ number = {3},
+ pages = {1013-1033},
+ title = {Inference using shape-restricted regression splines},
+ volume = {2},
+ year = {2008}
+}
+@article{meyer11,
+ author = {Meyer, M. C. and Hackstadt, A. J. and Hoeting, J. A.},
+ journal = {Journal of Nonparametric Statistics},
+ number = {4},
+ pages = {867-884},
+ title = {{Bayesian} estimation and inference for generalised partial linear models using shape-restricted splines},
+ volume = {23},
+ year = {2011}
+}
+@article{meyer13,
+ author = {Meyer, M. C.},
+ journal = {Journal of Nonparametric Statistics},
+ pages = {715-730},
+ title = {Semi-parametric additive constrained regression},
+ volume = {25},
+ year = {2013}
+}
+@article{meyer13b,
+ author = {Meyer, M. C.},
+ journal = {Communications in Statistics},
+ pages = {1126-1139},
+ title = {A simple new algorithm for quadratic programming with applications in statistics},
+ volume = {42},
+ year = {2013}
+}
+@misc{nimblesoft18,
+ author = {{NIMBLE Development Team}},
+ title = {NIMBLE: An {R} Package for Programming with {BUGS} models, Version 0.6-9},
+ url = {http://r-nimble.org},
+ year = {2018}
+}
+@article{opsomer08,
+ author = {Opsomer, J. D. and Claeskens, G. and Ranalli, M. G. and Kauemann, G. and Breidt, F. J.},
+ journal = {Journal of the Royal Statistical Society, series B},
+ pages = {265-286},
+ title = {Nonparametric small area estimation using penalized spline regression},
+ volume = {70},
+ year = {2008}
+}
+@article{opsomer16,
+ author = {Opsomer, J. D. and Breidt, F. J. and White, M. and Li, Y.},
+ journal = {Journal of Survey Statistical Methodology},
+ number = {1},
+ pages = {43-70},
+ title = {Succesive difference replication variance estimation in Two-Phase sampling},
+ volume = {4},
+ year = {2016}
+}
+@manual{Rteam,
+ address = {Vienna, Austria},
+ author = {{R Core Team}},
+ organization = {R Foundation for Statistical Computing},
+ title = {{R}: A Language and Environment for Statistical Computing},
+ url = {https://www.R-project.org},
+ year = {2018}
+}
+@article{ramsay88,
+ author = {Ramsay, J. O.},
+ journal = {Statistical Science},
+ pages = {425-461},
+ title = {Monotone regression splines in action},
+ volume = {3},
+ year = {1988}
+}
+@book{rao03,
+ address = {Hoboken, New Jersey},
+ author = {Rao, J. N. K.},
+ publisher = {Wiley},
+ title = {Small Area Estimation},
+ year = {2003}
+}
+@article{rao08,
+ author = {Rao, J. N. K.},
+ journal = {Rivista Internazionale di Scienze Sociali},
+ pages = {387-406},
+ title = {Some methods for small area estimation},
+ volume = {4},
+ year = {2008}
+}
+@book{rao15,
+ address = {Hoboken, New Jersey},
+ author = {Rao, J. N. K. and Molina, Isabel},
+ edition = {2nd},
+ publisher = {Wiley},
+ title = {Small Area Estimation},
+ year = {2015}
+}
+@book{robertson88,
+ address = {New York},
+ author = {Robertson, T. and Wright, F. T. and Dykstra, R. L.},
+ publisher = {John Wiley \& Sons},
+ title = {Order Restricted Statistical Inference},
+ year = {1988}
+}
+@book{rockafellar70,
+ address = {New Jersey},
+ author = {Rockafellar, R. T.},
+ publisher = {Princeton University Press},
+ title = {Convex Analysis},
+ year = {1970}
+}
+@article{schwarz78,
+ author = {Schwarz, G.},
+ journal = {Annals of Statistics},
+ pages = {461-464},
+ title = {Estimating the dimension of a model},
+ volume = {6},
+ year = {1978}
+}
+@book{silvapulle05,
+ address = {Hoboken, New Jersey},
+ author = {Silvapulle, M. J. and Sen, P.},
+ publisher = {Wiley},
+ title = {Constrained Statistical Inference},
+ year = {2005}
+}
+@article{vaneeden56,
+ author = {VanEeden, C.},
+ journal = {Indigationes Mathematicae},
+ pages = {444-455},
+ title = {Maximum likelihood estimation of ordered probabilities},
+ volume = {18},
+ year = {1956}
+}
+@incollection{wollan86,
+ address = {New York},
+ author = {Wollan, P. C. and Dykstra, R. L.},
+ booktitle = {Advances in Order Restricted Statistical Inference},
+ editor = {Dykstra, R. L. and Robertson, T. and Wright, F. T.},
+ pages = {279-295},
+ publisher = {Springer-Verlag},
+ title = {Conditional Tests with an order restriction as a null hypothesis},
+ year = {1986}
+}
+@article{wu16,
+ author = {Wu, J. and Meyer, M. C. and Opsomer, J. D.},
+ doi = {10.1002/cjs.11301},
+ journal = {Canadian Journal of Statistics},
+ number = {4},
+ pages = {431-444},
+ title = {Survey Estimation of Domain Means that Respect Natural Orderings},
+ volume = {44},
+ year = {2016}
+}
+@article{oliva19,
+ author = {Cristian Oliva-Avil\'es AND Mary C. Meyer AND Jean D. Opsomer},
+ doi = {10.1002/cjs.11496},
+ journal = {Canadian Journal of Statistics},
+ number = {2},
+ pages = {315-331},
+ title = {Checking Validity of Monotone Domain Mean Estimators},
+ volume = {47},
+ year = {2019}
+}
+@article{breidt2000,
+ author = {Breidt, F. and Opsomer, J.},
+ journal = {The Annals of Statistics},
+ number = {4},
+ pages = {1026-1053},
+ title = {Local Polynomial Regression Estimators in Survey Sampling},
+ volume = {28},
+ year = {2000}
+}
+@article{meyer99,
+ author = {Meyer, M. C.},
+ journal = {Journal of Statistical Planning and Inference},
+ pages = {13-31},
+ title = {An extension of the mixed primal-dual bases algorithm to the case of more constraints than dimensions},
+ volume = {81},
+ year = {1999}
+}
+@book{fuller96,
+ address = {New York},
+ author = {Fuller, W.A.},
+ edition = {2nd},
+ publisher = {Wiley},
+ title = {Introduction to Statistical Time Series},
+ year = {1996}
+}
+@book{sarndal92,
+ address = {New York},
+ author = {S\"arndal, C.-E. and Swensson, B. and Wretman, J.},
+ publisher = {Springer},
+ title = {Model Assisted Survey Sampling},
+ year = {1992}
+}
+@article{zhang13,
+ author = {Zhang, X. and Onufrak, S. and Holt, J. B. and Croft, J. B.},
+ journal = {Preventing Chronic Disease},
+ pages = {120252},
+ title = {A multilevel approach to estimating small area childhood obesity prevalence at the census block-group level.},
+ volume = {10},
+ year = {2013}
+}
diff --git a/_articles/RJ-2025-032/figures/daded.png b/_articles/RJ-2025-032/figures/daded.png
new file mode 100644
index 0000000000..4fcea37902
Binary files /dev/null and b/_articles/RJ-2025-032/figures/daded.png differ
diff --git a/_articles/RJ-2025-032/figures/new_surfaces9.pdf b/_articles/RJ-2025-032/figures/new_surfaces9.pdf
new file mode 100644
index 0000000000..4c8d534117
Binary files /dev/null and b/_articles/RJ-2025-032/figures/new_surfaces9.pdf differ
diff --git a/_articles/RJ-2025-032/figures/new_surfaces9.png b/_articles/RJ-2025-032/figures/new_surfaces9.png
new file mode 100644
index 0000000000..e6ad913364
Binary files /dev/null and b/_articles/RJ-2025-032/figures/new_surfaces9.png differ
diff --git a/_articles/RJ-2025-032/figures/newplot4.png b/_articles/RJ-2025-032/figures/newplot4.png
new file mode 100644
index 0000000000..1160b25931
Binary files /dev/null and b/_articles/RJ-2025-032/figures/newplot4.png differ
diff --git a/_articles/RJ-2025-032/figures/nhanes1.png b/_articles/RJ-2025-032/figures/nhanes1.png
new file mode 100644
index 0000000000..5e5017d338
Binary files /dev/null and b/_articles/RJ-2025-032/figures/nhanes1.png differ
diff --git a/_articles/RJ-2025-032/figures/nhanes_bin.png b/_articles/RJ-2025-032/figures/nhanes_bin.png
new file mode 100644
index 0000000000..94b9a02644
Binary files /dev/null and b/_articles/RJ-2025-032/figures/nhanes_bin.png differ
diff --git a/_articles/RJ-2025-032/figures/nhanes_grid3.png b/_articles/RJ-2025-032/figures/nhanes_grid3.png
new file mode 100644
index 0000000000..16ef62d650
Binary files /dev/null and b/_articles/RJ-2025-032/figures/nhanes_grid3.png differ
diff --git a/_articles/RJ-2025-032/figures/nhanes_grid3_un.png b/_articles/RJ-2025-032/figures/nhanes_grid3_un.png
new file mode 100644
index 0000000000..f307736895
Binary files /dev/null and b/_articles/RJ-2025-032/figures/nhanes_grid3_un.png differ
diff --git a/_articles/RJ-2025-032/nscg19.rda b/_articles/RJ-2025-032/nscg19.rda
new file mode 100644
index 0000000000..032a438f25
Binary files /dev/null and b/_articles/RJ-2025-032/nscg19.rda differ
diff --git a/_articles/RJ-2025-032/nscg19_2.rda b/_articles/RJ-2025-032/nscg19_2.rda
new file mode 100644
index 0000000000..f7cc3012bd
Binary files /dev/null and b/_articles/RJ-2025-032/nscg19_2.rda differ
diff --git a/_articles/RJ-2025-032/rcode/article.R b/_articles/RJ-2025-032/rcode/article.R
new file mode 100644
index 0000000000..7d533a87d3
--- /dev/null
+++ b/_articles/RJ-2025-032/rcode/article.R
@@ -0,0 +1,243 @@
+library(coneproj)
+library(csurvey)
+library(survey)
+library(data.table)
+library(tidyverse)
+library(MASS)
+library(rlang)
+library(zeallot)
+library(patchwork)
+library(knitr)
+library(tinytex)
+#options(csurvey.multicore = TRUE)
+
+#------------------------------------------------
+#NHANES Example with monotonic domain means
+#------------------------------------------------
+data(nhdat2, package = "csurvey")
+#use ?nhdat2 to see details of this data set
+#specify the design:
+dstrat <- svydesign(ids = ~id, strata = ~str, data = nhdat2, weight = ~wt)
+#fit the model
+set.seed(1)
+ans <- csvy(chol ~ incr(age), design = dstrat, n.mix = 100)
+
+cat("CIC (constrained):", ans$CIC, "\n")
+cat("CIC (unconstrained):", ans$CIC.un, "\n")
+cat(svycontrast(ans, list(avg = c(rep(-1, 13)/13, rep(1, 12)/12))), "\n")
+
+#Figure 1
+#see ?plot_csvy_control to find out how to adjust aesthetics
+ctl = list(unconstrained_color = "grey80")
+png("nhanes1.png")
+plot(ans, type = 'both', control = ctl)
+dev.off()
+
+#Figure 2
+data(nhdat2, package = 'csurvey')
+#specify the design:
+dstrat <- svydesign(ids = ~id, strata = ~str, data = nhdat2, weight = ~wt)
+#use parallel computing:
+#options(csurvey.multicore=TRUE)
+set.seed(1)
+ans <- csvy(chol ~ incr(age)*incr(wcat)*icat, design = dstrat)
+
+domains <- data.frame(age = c(24, 35), wcat = c(2, 4), icat = c(2, 3))
+pans <- predict(ans, newdata = domains, se.fit = TRUE)
+cat("Predicted values, confidence intervals and standard errors for specified domains:\n")
+print (pans)
+
+ctl <- list(x1lab = 'waist', x2lab = 'income', subtitle.size = 8)
+
+png('nhanes_grid3.png')
+plot(ans, x1 = "wcat", x2 = "icat", control = ctl, domains = domains)
+dev.off()
+
+#Figure 3
+png('nhanes_grid3_un.png')
+plot(ans, x1 = "wcat", x2 = "icat", control = ctl, type="unconstrained")
+dev.off()
+
+#----------------------------------------------------------------------
+#Example: Constrained domain means with a block ordering
+#----------------------------------------------------------------------
+load('../nscg19.rda')
+rds <- svrepdesign(data = nscg, repweights = dplyr::select(nscg, "RW0001":"RW0320"),
+ weights = ~w, combined.weights = TRUE, mse = TRUE, type = "other", scale = 1, rscale = 0.05)
+
+#options(csurvey.multicore = TRUE)
+ans <- csvy(logSalary~decr(hd_year_grouped)*incr(hd_type)*block.Ord(field, order = c(2, 1, 2, 1, 2))*region, design = rds)
+
+#Figure 4
+ctl <- list(categ = "region", categnm = c("New England", "Middle Atlantic", "East North Central", "West North Central",
+ "South Atlantic", "East South Central", "West South Central", "Mountain", "Pacific and US Territories"),
+ NCOL = 3, th = 60, xlab = "years since degree", ylab = "field of study", zlab = "log(salary)")
+
+png('new_surfaces9.png')
+plotpersp(ans, x1="hd_year_grouped", x2="field", control = ctl)
+dev.off()
+
+#Figure 5
+#use regions: 3, 9, 6
+#use the patchwork package to stack three plots into one
+ctl1 <- list(x1lab = " ",x2lab = "Field of study", x3lab = "Year of award of highest degree",
+ x1_labels = FALSE, x2_labels = FALSE, x3_labels = c(1995, 2000, 2005, 2010, 2015),
+ x4_labels = c("New England", "Middle Atlantic", "East North Central", "West North Central", "South Atlantic", "East South Central", "West South Central",
+ "Mountain", "Pacific and US Territories"), ynm = "log(Salary)", ci = TRUE, legend = FALSE, ylab=FALSE, unconstrained_color = 'grey80', x4_vals = 3, subtitle.size = 10)
+
+p1 <- plot(ans, x1 = 'hd_type', x2 = 'field', x3 = 'hd_year_grouped',
+ control = ctl1, type = 'both')
+
+ctl2 <- list(x1lab = " ", x2lab = " ", x3lab = " ", x1_labels = FALSE, x2_labels = FALSE, x3_labels = FALSE,
+ x4_labels = c("New England", "Middle Atlantic", "East North Central", "West North Central", "South Atlantic", "East South Central", "West South Central",
+ "Mountain", "Pacific and US Territories"), ynm = 'log(Salary)', ci = TRUE, legend = FALSE, ylab = TRUE, unconstrained_color = 'grey80', x4_vals = 9)
+
+p2 <- plot(ans, x1 = 'hd_type', x2 = 'field', x3 = 'hd_year_grouped',
+ control = ctl2, type = 'both')
+
+ctl3 <- list(x1lab = 'Highest degree type', x2lab = " ", x3lab = " ",
+ x1_labels = c('BS', 'MS', 'PHD'), x2_labels = FALSE, x3_labels = FALSE,
+ x4_labels = c("New England", "Middle Atlantic", "East North Central", "West North Central", "South Atlantic", "East South Central", "West South Central",
+ "Mountain", "Pacific and US Territories"), ynm = 'log(Salary)', ci = TRUE, legend = FALSE, ylab = FALSE, x1size = 2.5,
+ unconstrained_color = 'grey80', x4_vals = 6, angle = 0, hjust = .5)
+
+p3 <- plot(ans, x1 = 'hd_type', x2 = 'field', x3 = 'hd_year_grouped', control = ctl3, type = 'both')
+
+#use the patchwork package to stack three plots into one
+library(patchwork)
+
+png('newplot4.png')
+(p1 / p2 / p3)
+dev.off()
+
+#--------------------------
+#Example:One-sided testing
+#--------------------------
+load('../nscg19_2.rda')
+
+#code to generate Table 1
+pvals.table = data.frame(csvy_onesided = numeric(6), svyglm_twosided = numeric(6))
+rownames(pvals.table) = c('2008-09', '2010-11', '2012-13', '2014-15', '2016-17', '2018-19')
+
+#we need to set a random seed because the p-value for csvy is simulated
+set.seed(1)
+#svyglm may throw an error message when there is an empty domain, and we use tryCatch to address this issue
+#we include the failing example for svyglm on purpose because we want to show that the proposed method csvy will work in some cases when svyglm fails
+for(i in 1:6){
+ print (i)
+ years <- c(2008, 2009) + 2 * (i - 1)
+ #select the data for specific years
+ data <- nscg2 |> dplyr::filter(hd_year %in% years)
+ rds <- svrepdesign(data = data, repweights = dplyr::select(data, "RW0001":"RW0320"), weights = ~w,
+ combined.weights = TRUE, mse = TRUE, type = "other", scale = 1, rscale = 0.05)
+
+ ans <- csvy(logSalary ~ incr(daded) * field * region, design = rds, test = TRUE)
+ #p-value for the constrained estimate
+ pvals.table[i, 1] = ans$pval
+
+ #when there is an empty domain, svyglm may not work and will throw an error message, we use tryCatch for the
+ #objects fitted by svyglm in this for loop
+ ansu1 <- tryCatch({svyglm(formula = logSalary ~ factor(field) * factor(daded) * factor(region), design = rds, data = data)}, error = function(e) {e})
+ ansu2 <- tryCatch({svyglm(formula = logSalary ~ factor(field) * factor(region), design = rds, data = data)}, error = function(e) {e})
+ #if ansu1 will be an error, then the anova function will not work and throw an error message, we use tryCatch for the
+ #object fitted by anova in this for loop
+ punc <- tryCatch({as.numeric(anova(ansu1, ansu2)[6])}, error = function(e) {e})
+ if (inherits(punc, "error")) {
+ #p-value for the unconstrained estimate
+ pvals.table[i, 2] = NA
+ } else {
+ pvals.table[i, 2] = punc
+ }
+}
+
+comppv = data.frame(year = c(rep("2008-09", 2), rep("2010-11", 2), rep("2012-13", 2),
+ rep("2014-15", 2), rep("2016-17", 2), rep("2018-19", 2)),
+ test = rep(c("one", "two"), 6), pvals = as.vector(t(round(pvals.table, 3))))
+
+#print out results in Table 1
+#cat("Table 1:", "\n")
+comppv = kable(t(comppv))
+print (comppv)
+
+#compile the Latex file, which saves the result, into a pdf file using the pdflatex function
+tinytex::pdflatex("../tables/comppv.tex")
+
+
+#Figure 6
+data <- nscg2 |> dplyr::filter(hd_year %in% c(2016, 2017))
+#specify a survey design with replicate weights
+rds <- svrepdesign(data = data, repweights = dplyr::select(data, "RW0001":"RW0320"), weights = ~w,
+ combined.weights = TRUE, mse = TRUE, type = "other", scale = 1, rscale = 0.05)
+#fit the model
+ans <- csvy(logSalary ~ incr(daded) * field * region, design = rds)
+
+#create the plot
+ctl <- list(x1lab = 'Field', x2lab = 'Region', subtitle.size=10, x1size=3, x2size=3, x3lab = "father's education",
+ ynm = "log(salary)", unconstrained_color = "red", ci = FALSE,
+ x2_labels = c('Northeast', 'North Central', 'Southeast', 'West', 'Pacific & Territories'))
+
+png('daded.png')
+plot(ans, x1='field', x2='region', control = ctl, type = 'both')
+dev.off()
+
+#show the one-sided test result for 2008-2009
+load("../nscg19_2.rda")
+data <- nscg2 |> dplyr::filter(hd_year %in% c(2008, 2009))
+rds <- svrepdesign(data = data, repweights = dplyr::select(data, "RW0001":"RW0320"), weights = ~w,
+ combined.weights = TRUE, mse = TRUE, type = "other",
+ scale = 1, rscale = 0.05)
+set.seed(1)
+ans <- csvy(logSalary ~ incr(daded) * field * region, design = rds, test = TRUE)
+summary(ans)
+
+#------------------------
+#Example: Binary outcome
+#------------------------
+#use ?nhdat to check details of this data set
+data(nhdat, package = 'csurvey')
+#specify the design
+dstrat <- svydesign(ids = ~ id, strata = ~ str, data = nhdat, weight = ~ wt)
+
+#fit the model
+set.seed(1)
+ans <- csvy(chol ~ incr(age) * incr(wcat) * gender, design = dstrat, family = binomial(link = "logit"), test = TRUE)
+
+#show the one-sided test result for 2008-2009
+summary(ans)
+
+#Figure 7
+#see ?plot_csvy_control to get more information about how to adjust aesthetics
+ctl <- list(x1lab = 'waist', x2lab = 'gender', ynm = 'high cholesterol level', x2_labels = c('male', 'female'), ci = TRUE, subtitle.size = 8)
+png('nhanes_bin.png')
+plot(ans, x1='wcat', x2='gender', type="both", control = ctl)
+dev.off()
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/_articles/RJ-2025-032/tables/comppv.pdf b/_articles/RJ-2025-032/tables/comppv.pdf
new file mode 100644
index 0000000000..502a6a0222
Binary files /dev/null and b/_articles/RJ-2025-032/tables/comppv.pdf differ
diff --git a/_articles/RJ-2025-032/tables/comppv.tex b/_articles/RJ-2025-032/tables/comppv.tex
new file mode 100644
index 0000000000..e223f84cb3
--- /dev/null
+++ b/_articles/RJ-2025-032/tables/comppv.tex
@@ -0,0 +1,20 @@
+\documentclass{article}
+\usepackage{booktabs} % Ensures proper table formatting
+\begin{document}
+
+\begin{table}
+\begin{center}
+\begin{tabular}{cccccccccccc}
+\multicolumn{2}{c}{2008-09} &\multicolumn{2}{c}{2010-11} &\multicolumn{2}{c}{2012-13} &\multicolumn{2}{c}{2014-15} &\multicolumn{2}{c}{2016-17} &\multicolumn{2}{c}{2018-19}\\
+ one & two & one & two & one & two & one & two & one & two & one & two \\
+ \hline
+ .008 & n/a & $<$.001 & .018 & $<$.001 & $<$.001 & $<$.001 & n/a & .003 & .417 & $<$.001 & n/a
+ \\
+ \hline
+ \end{tabular}
+\caption{One-sided and two-sided $p$-values for the test of the null hypothesis that salary is constant in father's education level. The two-sided test results in n/a when the grid has at least one empty domain. }
+\label{comppv}
+\end{center}
+\end{table}
+
+\end{document}
\ No newline at end of file
diff --git a/_articles/RJ-2025-033/RJ-2025-033.Rmd b/_articles/RJ-2025-033/RJ-2025-033.Rmd
new file mode 100644
index 0000000000..bc5bf85f00
--- /dev/null
+++ b/_articles/RJ-2025-033/RJ-2025-033.Rmd
@@ -0,0 +1,1174 @@
+---
+title: 'LHD: An All-encompassing R Package for Constructing Optimal Latin Hypercube
+ Designs'
+abstract: |
+ Optimal Latin hypercube designs (LHDs), including maximin distance
+ LHDs, maximum projection LHDs and orthogonal LHDs, are widely used in
+ computer experiments. It is challenging to construct such designs with
+ flexible sizes, especially for large ones, for two main reasons. One
+ reason is that theoretical results, such as algebraic constructions
+ ensuring the maximin distance property or orthogonality, are only
+ available for certain design sizes. For design sizes where theoretical
+ results are unavailable, search algorithms can generate designs.
+ However, their numerical performance is not guaranteed to be optimal.
+ Another reason is that when design sizes increase, the number of
+ permutations grows exponentially. Constructing optimal LHDs is a
+ discrete optimization process, and enumeration is nearly impossible
+ for large or moderate design sizes. Various search algorithms and
+ algebraic constructions have been proposed to identify optimal LHDs,
+ each having its own pros and cons. We develop the R package LHD which
+ implements various search algorithms and algebraic constructions. We
+ embedded different optimality criteria into each of the search
+ algorithms, and they are capable of constructing different types of
+ optimal LHDs even though they were originally invented to construct
+ maximin distance LHDs only. Another input argument that controls
+ maximum CPU time is added to each of the search algorithms to let
+ users flexibly allocate their computational resources. We demonstrate
+ functionalities of the package by using various examples, and we
+ provide guidance for experimenters on finding suitable optimal
+ designs. The LHD package is easy to use for practitioners and possibly
+ serves as a benchmark for future developments in LHD.
+author:
+- name: Hongzhi Wang
+ affiliation: |
+ [wanghongzhi.ut@gmail.com](wanghongzhi.ut@gmail.com){.uri}
+- name: Qian Xiao
+ affiliation: |-
+ Department of Statistics, School of Mathematical Sciences, Shanghai
+ Jiao Tong University
+ address:
+ - 800 Dongchuan Road, Minhang, Shanghai, 200240
+ - China
+ - |
+ [qian.xiao@sjtu.edu.cn](qian.xiao@sjtu.edu.cn){.uri}
+- name: Abhyuday Mandal
+ affiliation: Department of Statistics, University of Georgia
+ address:
+ - 310 Herty Drive, Athens, GA 30602
+ - USA
+ - |
+ [amandal@stat.uga.edu](amandal@stat.uga.edu){.uri}
+date: '2026-01-05'
+date_received: '2023-11-20'
+journal:
+ firstpage: 20
+ lastpage: 36
+volume: 17
+issue: 4
+slug: RJ-2025-033
+packages:
+ cran: []
+ bioc: []
+preview: preview.png
+bibliography: wang-xiao-mandal.bib
+CTV: ~
+legacy_pdf: yes
+legacy_converted: yes
+output:
+ rjtools::rjournal_web_article:
+ self_contained: yes
+ toc: no
+ mathjax: https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-chtml-full.js
+ md_extension: -tex_math_single_backslash
+draft: no
+
+---
+
+
+::::::::::::: article
+## Introduction
+
+Computer experiments are widely used in scientific research and
+industrial production, where complex computer codes, commonly
+high-fidelity simulators, generate data instead of real physical systems
+[@sacks1989designs; @fang2005design]. The outputs from computer
+experiments are deterministic (that is, free of random errors), and
+therefore replications are not needed
+[@butler2001optimal; @joseph2008orthogonal; @ba2015optimal]. Latin
+hypercube designs (LHDs, [@mckay1979comparison]) may be the most popular
+type of experimental designs for computer experiments
+[@fang2005design; @xiao2018construction], which avoid replications on
+every dimension and have uniform one-dimensional projections. According
+to practical needs, there are various types of optimal LHDs, including
+space-filling LHDs, maximum projection LHDs, and orthogonal LHDs. There
+is a rich literature on the construction of such designs, but it is
+still very challenging to find good ones for moderate to large design
+sizes
+[@ye1998orthogonal; @fang2005design; @joseph2015maximum; @xiao2018construction].
+One key reason is that theoretical results, such as algebraic
+constructions which guarantee the maximin distance property or
+orthogonality, are only established for specific design sizes. These
+constructions provide theoretical guarantees on the design quality but
+are limited in their applicability. For design sizes where such
+theoretical guarantees do not exist, search algorithms can generate
+designs. However, the performance of search-based designs depends on the
+algorithm employed, the search space explored, and the computational
+resources allocated, meaning they cannot be guaranteed to be optimal.
+Constructing optimal LHDs is a discrete optimization process, where
+enumerating all possible solutions guarantees the optimal design for a
+given size. However, this approach becomes computationally infeasible as
+the number of permutations grows exponentially with increasing design
+sizes, making it another key reason that adds to the challenge.
+
+An LHD with $n$ runs and $k$ factors is an $n \times k$ matrix with each
+column being a random permutation of numbers: $1, \ldots, n$. Throughout
+this paper, $n$ denotes the run size and $k$ denotes the factor size. A
+space-filling LHD has its sampled region as scattered as possible,
+minimizing the unsampled region, thus accounting for the uniformity of
+all dimensions. Different criteria were proposed to measure designs'
+space-filling properties, including the maximin and minimax distance
+criteria [@johnson1990minimax; @morris1995exploratory], the discrepancy
+criteria
+[@hickernell1998generalized; @fang2002centered; @fang2005design] and the
+entropy criterion [@shewry1987maximum]. Since there are as many as
+$(n!)^{k}$ candidate LHDs for a given design size, it is nearly
+impossible to find the space-filling one by enumeration when $n$ and $k$
+are moderate or large. In the current literature, both the search
+algorithms
+[@morris1995exploratory; @leary2003optimal; @joseph2008orthogonal; @ba2015optimal; @kenny2000algorithmic; @jin2005efficient; @liefvendahl2006study; @grosso2009finding; @chen2013optimizing]
+and algebraic constructions
+[@zhou2015space; @xiao2017construction; @wang2018optimal] are used to
+construct space-filling LHDs.
+
+Space-filling designs often focus on the full-dimensional space. To
+further improve the space-filling properties of all possible sub-spaces,
+[@joseph2015maximum] proposed to use the maximum projection designs.
+Considering from two to $k-1$ dimensional sub-spaces, maximum projection
+LHDs (MaxPro LHDs) are generally more space-filling compared to the
+classic maximin distance LHDs. The construction of MaxPro LHDs is also
+challenging, especially for large ones, and [@joseph2015maximum]
+proposed a simulated annealing (SA) based algorithm. In the `LHD`
+package, we incorporated the MaxPro criterion with other different
+algorithms such as the particle swarm optimization (PSO) and genetic
+algorithm (GA) framework, leading to many better MaxPro LHDs; see
+Section 3 for examples.
+
+Unlike space-filling LHDs that minimize the similarities among rows,
+orthogonal LHDs (OLHDs) are another popular type of optimal design which
+consider similarities among columns. For example, OLHDs have zero
+column-wise correlations. Algebraic constructions are available for
+certain design sizes
+[@ye1998orthogonal; @cioppa2007efficient; @steinberg2006construction; @sun2010construction; @sun2009construction; @yang2012construction; @georgiou2014some; @butler2001optimal; @tang1993orthogonal; @lin2009construction],
+but there are many design sizes where theoretical results are not
+available. In the `LHD` package, we implemented the average absolute
+correlation criterion and the maximum absolute correlation criterion
+[@georgiou2009orthogonal] with SA, PSO, and GA to identify both OLHDs
+and nearly orthogonal LHDs (NOLHDs) for almost all design sizes.
+
+This paper introduces the R package `LHD` available on the Comprehensive
+R Archive Network
+(References
+ +Reuse
+Text and figures are licensed under Creative Commons Attribution CC BY 4.0. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from ...".
+Citation
+For attribution, please cite this work as
+Liao & Meyer, "csurvey: Implementing Order Constraints in Survey Data Analysis", The R Journal, 2026+
BibTeX citation
+@article{RJ-2025-032,
+ author = {Liao, Xiyue and Meyer, Mary C.},
+ title = {csurvey: Implementing Order Constraints in Survey Data Analysis},
+ journal = {The R Journal},
+ year = {2026},
+ note = {https://doi.org/10.32614/RJ-2025-032},
+ doi = {10.32614/RJ-2025-032},
+ volume = {17},
+ issue = {4},
+ issn = {2073-4859},
+ pages = {4-19}
+}
+
+
+
+
+
+LHD: An All-encompassing R Package for Constructing Optimal Latin Hypercube Designs
+ + + +Optimal Latin hypercube designs (LHDs), including maximin distance +LHDs, maximum projection LHDs and orthogonal LHDs, are widely used in +computer experiments. It is challenging to construct such designs with +flexible sizes, especially for large ones, for two main reasons. One +reason is that theoretical results, such as algebraic constructions +ensuring the maximin distance property or orthogonality, are only +available for certain design sizes. For design sizes where theoretical +results are unavailable, search algorithms can generate designs. +However, their numerical performance is not guaranteed to be optimal. +Another reason is that when design sizes increase, the number of +permutations grows exponentially. Constructing optimal LHDs is a +discrete optimization process, and enumeration is nearly impossible +for large or moderate design sizes. Various search algorithms and +algebraic constructions have been proposed to identify optimal LHDs, +each having its own pros and cons. We develop the R package LHD which +implements various search algorithms and algebraic constructions. We +embedded different optimality criteria into each of the search +algorithms, and they are capable of constructing different types of +optimal LHDs even though they were originally invented to construct +maximin distance LHDs only. Another input argument that controls +maximum CPU time is added to each of the search algorithms to let +users flexibly allocate their computational resources. We demonstrate +functionalities of the package by using various examples, and we +provide guidance for experimenters on finding suitable optimal +designs. The LHD package is easy to use for practitioners and possibly +serves as a benchmark for future developments in LHD.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+1 Introduction
+Computer experiments are widely used in scientific research and +industrial production, where complex computer codes, commonly +high-fidelity simulators, generate data instead of real physical systems +(Sacks et al. 1989; Fang et al. 2005). The outputs from computer +experiments are deterministic (that is, free of random errors), and +therefore replications are not needed +(Butler 2001; Joseph and Hung 2008; Ba et al. 2015). Latin +hypercube designs (LHDs, (McKay et al. 1979)) may be the most popular +type of experimental designs for computer experiments +(Fang et al. 2005; Xiao and Xu 2018), which avoid replications on +every dimension and have uniform one-dimensional projections. According +to practical needs, there are various types of optimal LHDs, including +space-filling LHDs, maximum projection LHDs, and orthogonal LHDs. There +is a rich literature on the construction of such designs, but it is +still very challenging to find good ones for moderate to large design +sizes +(Ye 1998; Fang et al. 2005; Joseph et al. 2015; Xiao and Xu 2018). +One key reason is that theoretical results, such as algebraic +constructions which guarantee the maximin distance property or +orthogonality, are only established for specific design sizes. These +constructions provide theoretical guarantees on the design quality but +are limited in their applicability. For design sizes where such +theoretical guarantees do not exist, search algorithms can generate +designs. However, the performance of search-based designs depends on the +algorithm employed, the search space explored, and the computational +resources allocated, meaning they cannot be guaranteed to be optimal. +Constructing optimal LHDs is a discrete optimization process, where +enumerating all possible solutions guarantees the optimal design for a +given size. However, this approach becomes computationally infeasible as +the number of permutations grows exponentially with increasing design +sizes, making it another key reason that adds to the challenge.
+An LHD with \(n\) runs and \(k\) factors is an \(n \times k\) matrix with each +column being a random permutation of numbers: \(1, \ldots, n\). Throughout +this paper, \(n\) denotes the run size and \(k\) denotes the factor size. A +space-filling LHD has its sampled region as scattered as possible, +minimizing the unsampled region, thus accounting for the uniformity of +all dimensions. Different criteria were proposed to measure designs’ +space-filling properties, including the maximin and minimax distance +criteria (Johnson et al. 1990; Morris and Mitchell 1995), the discrepancy +criteria +(Hickernell 1998; Fang et al. 2002, 2005) and the +entropy criterion (Shewry and Wynn 1987). Since there are as many as +\((n!)^{k}\) candidate LHDs for a given design size, it is nearly +impossible to find the space-filling one by enumeration when \(n\) and \(k\) +are moderate or large. In the current literature, both the search +algorithms +(Morris and Mitchell 1995; Ye et al. 2000; Leary et al. 2003; Jin et al. 2005; Liefvendahl and Stocki 2006; Joseph and Hung 2008; Grosso et al. 2009; Chen et al. 2013; Ba et al. 2015) +and algebraic constructions +(Zhou and Xu 2015; Xiao and Xu 2017; Wang et al. 2018) are used to +construct space-filling LHDs.
+Space-filling designs often focus on the full-dimensional space. To
+further improve the space-filling properties of all possible sub-spaces,
+(Joseph et al. 2015) proposed to use the maximum projection designs.
+Considering from two to \(k-1\) dimensional sub-spaces, maximum projection
+LHDs (MaxPro LHDs) are generally more space-filling compared to the
+classic maximin distance LHDs. The construction of MaxPro LHDs is also
+challenging, especially for large ones, and (Joseph et al. 2015)
+proposed a simulated annealing (SA) based algorithm. In the LHD
+package, we incorporated the MaxPro criterion with other different
+algorithms such as the particle swarm optimization (PSO) and genetic
+algorithm (GA) framework, leading to many better MaxPro LHDs; see
+Section 3 for examples.
Unlike space-filling LHDs that minimize the similarities among rows,
+orthogonal LHDs (OLHDs) are another popular type of optimal design which
+consider similarities among columns. For example, OLHDs have zero
+column-wise correlations. Algebraic constructions are available for
+certain design sizes
+(Tang 1993; Ye 1998; Butler 2001; Steinberg and Lin 2006; Cioppa and Lucas 2007; Lin et al. 2009; Sun et al. 2009, 2010; Yang and Liu 2012; Georgiou and Efthimiou 2014),
+but there are many design sizes where theoretical results are not
+available. In the LHD package, we implemented the average absolute
+correlation criterion and the maximum absolute correlation criterion
+(Georgiou 2009) with SA, PSO, and GA to identify both OLHDs
+and nearly orthogonal LHDs (NOLHDs) for almost all design sizes.
This paper introduces the R package LHD available on the Comprehensive
+R Archive Network
+(https://cran.r-project.org/web/packages/LHD/index.html), which
+implements some currently popular search algorithms and algebraic
+constructions for constructing maximin distance LHDs, Maxpro LHDs, OLHDs
+and NOLHDs. We embedded different optimality criteria including the
+maximin distance criterion, the MaxPro criterion, the average absolute
+correlation criterion, and the maximum absolute correlation criterion in
+each of the search algorithms which were originally invented to
+construct maximin distance LHDs only
+(Morris and Mitchell 1995; Leary et al. 2003; Liefvendahl and Stocki 2006; Joseph and Hung 2008; Chen et al. 2013),
+and each of them is capable of constructing different types of optimal
+LHDs through the package. To let users flexibly allocate their
+computational resources, we also embedded an input argument that limits
+the maximum CPU time for each of the algorithms, where users can easily
+define how and when they want the algorithms to stop. An algorithm can
+stop in one of two ways: either when the user-defined maximum number of
+iterations is reached or when the user-defined maximum CPU time is
+exceeded. For example, users can either allow the algorithm to run for a
+specified number of iterations without restricting the maximum CPU time
+or set a maximum CPU time limit to stop the algorithm regardless of the
+number of iterations completed. After an algorithm is completed or
+stopped, the number of iterations completed along with the average CPU
+time per iteration will be presented to users for their information. The
+R package LHD is an integrated tool for users with little or no
+background in design theory, and they can easily find optimal LHDs with
+desired sizes. Many new designs that are better than the existing ones
+are discovered; see Section 3.
The remainder of the paper is organized as follows. Section 2
+illustrates different optimality criteria for LHDs. Section 3
+demonstrates some popular search algorithms and their implementation
+details in the LHD package along with examples. Section 4 discusses
+some useful algebraic constructions as well as examples of how to
+implement them via the developed package. Section 5 reviews other R packages for Latin hypercubes and provides a comparative discussion. Section 6 concludes with a
+summary.
2 Optimality Criteria for LHDs
+Various criteria are proposed to measure designs’ space-filling
+properties
+(Johnson et al. 1990; Hickernell 1998; Fang et al. 2002). In
+this paper, we focus on the currently popular maximin distance criterion
+(Johnson et al. 1990), which seeks to scatter design points over
+experimental domains so that the minimum distances between points are
+maximized. Let \(\textbf{X}\) denote an LHD matrix throughout this paper.
+Define the \(L_q\)-distance between two runs \(x_i\) and \(x_j\) of
+\(\textbf{X}\) as
+\(d_q(x_i, x_j) = \left\{ \sum_{m=1}^{k} \vert x_{im}-x_{jm}\vert ^q \right\}^{1/q}\)
+where \(q\) is an integer. Define the \(L_q\) distance of the design
+\(\textbf{X}\) as
+\(d_q(\textbf{X}) = \text{min} \{d_q(x_i, x_j), 1 \leq i<j \leq n \}\).
+In this paper, we consider \(q=1\) and \(q=2\), i.e. the Manhattan (\(L_1\))
+and Euclidean (\(L_2\)) distances. A design \(\textbf{X}\) is called a
+maximin \(L_q\) distance design if it has the unique largest
+\(d_q(\textbf{X})\) value among all designs of the same size. When more
+than one design has the same largest \(d_q(\textbf{X})\), the maximin
+distance design sequentially maximizes the next minimum inter-site
+distances. To evaluate the maximin distance criterion in a more
+convenient way, (Morris and Mitchell 1995) and (Jin et al. 2005)
+proposed to minimize a scalar value:
+\[\label{E2}
+ \phi_{p}= \bigg\{\sum_{i=1}^{n-1}\sum_{j=i+1}^{n}d_q(x_i, x_j)^{-p} \bigg\} ^{1/p}, \tag{1}\]
+where \(p\) is a tuning parameter. This \(\phi_{p}\) criterion in
+Equation (1) is asymptotically equivalent to the Maximin
+distance criterion as \(p \to \infty\). In practice, \(p=15\) often suffices
+(Morris and Mitchell 1995). In the LHD package, the function phi_p()
+implements this criterion.
Maximin distance LHDs focus on the space-filling properties in the
+full-dimensional space, but their space-filling properties in the
+subspaces are not guaranteed. (Joseph et al. 2015) proposed the maximum
+projection criterion that considers designs’ space-filling properties in
+all possible dimensional spaces. An LHD \(\textbf{X}\) is called a maximum
+projection LHD (MaxPro LHD) if it minimizes the maximum projection
+criterion such that
+\[\label{E3}
+ \mathop{\mathrm{min}}\limits_{\textbf{X}} \psi (\textbf{X}) = \Bigg\{ \frac{1}{{n \choose 2}} \sum_{i=1}^{n-1} \sum_{j=i+1}^{n} \frac{1}{\Pi_{l=1}^{k}(x_{il}-x_{jl})^2} \Bigg\}^{1/k}. \tag{2}\]
+From Equation (2), we can see that any two design points should
+be apart from each other in any projection to minimize the value of
+\(\psi (\textbf{X})\). Thus, the maximum projection LHDs consider the
+space-filling properties in all possible subspaces. Note that this
+criterion was originally defined using design points scaled to the unit
+hypercube \([0,1]^{k}\) in (Joseph et al. 2015), whereas our design points
+are represented as integer levels. A simple transformation can be
+applied to revert the scaling. For example, the transformation
+\(\textbf{X}_{Scaled}*n-0.5\), can be applied, meaning that each element
+of every design point in scaled unit hypercube is multiplied by its run
+size \(n\) and then adding \(0.5\). The illustrative example at the end of
+Section 3 applies this transformation to ensure a fair comparison of
+performance. In the LHD package, the function MaxProCriterion()
+implements this criterion.
Orthogonal and nearly orthogonal designs that aim to minimize the
+correlations between factors are widely used in experiments
+(Steinberg and Lin 2006; Georgiou 2009; Sun and Tang 2017).
+Two major correlation-based criteria to measure designs’ orthogonality
+are the average absolute correlation criterion and the maximum absolute
+correlation criterion (Georgiou 2009), denoted as ave\((|q|)\)
+and max\(|q|\), respectively:
+\[\label{E4}
+ \mathop{\mathrm{ave}}\limits(|q|) = \frac{2 \sum_{i=1}^{k-1} \sum_{j=i+1}^{k}|q_{ij}|}{k(k-1)} \text{ and } \mathop{\mathrm{max}}\limits|q| = \mathop{\mathrm{max}}\limits_{i,j} |q_{ij}|, \tag{3}\]
+where \(q_{ij}\) is the correlation between the \(i\)th and \(j\)th columns of
+the design matrix \(\textbf{X}\). Orthogonal designs have ave\((|q|)=0\) and
+max\(|q|=0\), which may not exist for all design sizes. Designs with a
+smaller ave\((|q|)\) or max\(|q|\) are generally preferred in practice. In
+the LHD package, functions AvgAbsCor() and MaxAbsCor() implement
+the criteria ave\((|q|)\) and max\(|q|\), respectively.
Illustrating Examples for the Introduced Optimality Criteria
+This subsection demonstrates some examples of how to use the optimality
+criteria introduced above from the developed LHD package. To generate
+a random LHD matrix, the function rLHD can be used. For example,
> X = rLHD(n = 5, k = 3); X #This generates a 5 by 3 random LHD, denoted as X
+ [,1] [,2] [,3]
+[1,] 2 1 4
+[2,] 4 3 3
+[3,] 3 2 2
+[4,] 1 4 5
+[5,] 5 5 1The input arguments for the function rLHD are the run-size n and the
+factor size k. Continuing with the above randomly generated LHD X, we
+evaluate it with respect to different optimality criteria. For example,
> phi_p(X) #The maximin L1-distance criterion.
+[1] 0.3336608
+> phi_p(X, p = 10, q = 2) #The maximin L2-distance criterion.
+[1] 0.5797347
+> MaxProCriterion(X) #The maximum projection criterion.
+[1] 0.5375482
+> AvgAbsCor(X) #The average absolute correlation criterion.
+[1] 0.5333333
+> MaxAbsCor(X) #The maximum absolute correlation criterion.
+[1] 0.9The input arguments of the function phi_p are an LHD matrix X, p
+and q, where p and q come directly from the equation (1).
+Note that the default settings within function phi_p are \(p=15\) and
+\(q=1\) (the Manhattan distance) and user can change the settings. For
+functions MaxProCriterion, AvgAbsCor, and MaxAbsCor, there is only
+one input argument, which is an LHD matrix X.
3 Search Algorithms for Optimal LHDs with Flexible Sizes
+Simulated Annealing Based Algorithms
+Simulated annealing (SA, (Kirkpatrick et al. 1983)) is a
+probabilistic optimization algorithm, whose name comes from the
+phenomenon of the annealing process in metallurgy.
+(Morris and Mitchell 1995) proposed a modified SA that randomly exchanges
+the elements in LHD to seek potential improvements. If such an exchange
+leads to a better LHD under a given optimality criterion, the exchange
+is maintained. Otherwise, it is kept with a probability of
+\(\hbox{exp}[-(\Phi(\textbf{X}_{new})-\Phi(\textbf{X}))/T]\), where \(\Phi\)
+is a given optimality criterion, \(\textbf{X}\) is the original LHD,
+\(\textbf{X}_{new}\) is the LHD after the exchange and \(T\) is the current
+temperature. In this article, we focus on minimizing the optimality
+criteria outlined in Section 2, meaning only minimization optimization
+problems are considered. Such probability guarantees that the exchange
+that leads to a slightly worse LHD has a higher chance of being kept
+than the exchange that leads to a significantly worse LHD, because an
+exchange which leads to a slightly worse LHD has a lower value of
+\(\Phi(\textbf{X}_{new})-\Phi(\textbf{X})\). Such an exchange procedure
+will be implemented iteratively to improve LHD. When there are no
+improvements after certain attempts, the current temperature \(T\) is
+annealed. Note that a large value of
+\(\Phi(\textbf{X}_{new})-\Phi(\textbf{X})\) (exchange leading to a
+significantly worse LHD) is more likely to remain during the early phase
+of the search process when \(T\) is relatively high, and it is less likely
+to stay later when \(T\) decreases (annealed). The best LHD is identified
+after the algorithm converges or the budget constraint is reached. In
+the LHD package, the function SA() implements this algorithm:
SA(n, k, N = 10, T0 = 10, rate = 0.1, Tmin = 1, Imax = 5, OC = "phi_p",
+p = 15, q = 1, maxtime = 5)Table 1 provides an
+overview of all the input arguments in SA(). n and k are the
+desired run size and factor size. T0 is an initial temperature, rate
+is the temperature decreasing rate, and Tmin is the minimum
+temperature. If the current temperature is smaller than Tmin, the
+current loop in the algorithm will stop and the current number of
+iterations will increase by one. There are two stopping criteria for the
+entire function: when the current number of iterations reaches the
+maximum (denoted as N in the function) or when the cumulative CPU time
+reaches the maximum (denoted as maxtime in the function),
+respectively. Either of those will trigger the stop of the function,
+whichever is earlier. For input argument OC (optimality criterion),
+“phi_p" returns maximin distance LHDs,”MaxProCriterion" returns
+MaxPro LHDs, and “AvgAbsCor" or”MaxAbsCor" returns orthogonal LHDs.
+
+
+
+
+
+| Argument | +Description | +
|---|---|
n |
+A positive integer that defines the number of rows (or run size) of output LHD. | +
k |
+A positive integer that defines the number of columns (or factor size) of output LHD. | +
N |
+A positive integer that defines the maximum number of iterations in the algorithm. | +
| + | A large value of N will result in a high CPU time, and it is recommended to be no |
+
| + | greater than 500. The default is set to be 10. | +
T0 |
+A positive number that defines the initial temperature. The default is set to be 10, | +
| + | which means the temperature anneals from 10 in the algorithm. | +
rate |
+A positive percentage that defines the temperature decrease rate, and it should be | +
| + | in (0,1). For example, rate=0.25 means the temperature decreases by 25% each time. | +
| + | The default is set to be 10%. | +
Tmin |
+A positive number that defines the minimum temperature allowed. When current | +
| + | temperature becomes smaller or equal to Tmin, the stopping criterion for current |
+
| + | loop is met. The default is set to be 1. | +
Imax |
+A positive integer that defines the maximum perturbations the algorithm will try | +
| + | without improvements before temperature is reduced. The default is set to be 5. | +
| + | For CPU time consideration, Imax is recommended to be no greater than 5. |
+
OC |
+An optimality criterion. The default setting is “phi_p", and it could be one of | +
| + | the following: “phi_p",”AvgAbsCor", “MaxAbsCor",”MaxProCriterion". | +
p |
+A positive integer, which is one parameter in the \(\phi_{p}\) formula, and p is preferred |
+
| + | to be large. The default is set to be 15. | +
q |
+A positive integer, which is one parameter in the \(\phi_{p}\) formula, and q could be |
+
| + | either 1 or 2. If q is 1, the Manhattan (rectangular) distance will be calculated. |
+
| + | If q is 2, the Euclidean distance will be calculated. |
+
maxtime |
+A positive number, which indicates the expected maximum CPU time, and it is | +
| + | measured by minutes. For example, maxtime=3.5 indicates the CPU time will |
+
| + | be no greater than three and a half minutes. The default is set to be 5. | +
(Leary et al. 2003) modified the SA algorithm in
+(Morris and Mitchell 1995) to search for optimal orthogonal array-based
+LHDs (OALHDs). (Tang 1993) showed that OALHDs tend to have
+better space-filling properties than random LHDs. The SA in
+(Leary et al. 2003) starts with a random OALHD rather than a random LHD.
+The remaining steps are the same as the SA in (Morris and Mitchell 1995).
+Note that the existence of OALHDs is determined by the existence of the
+corresponding initial OAs. In the LHD package, the function OASA()
+implements the modified SA algorithm.:
OASA(OA, N = 10, T0 = 10, rate = 0.1, Tmin = 1, Imax = 5, OC = "phi_p",
+p = 15, q = 1, maxtime = 5),where all the input arguments are the same as in SA except that OA
+must be an orthogonal array.
(Joseph and Hung 2008) proposed another modified SA to identify the
+orthogonal-maximin LHDs, which considers both the orthogonality and the
+maximin distance criteria. The algorithm starts with generating a random
+LHD and then chooses the column that has the largest average pairwise
+correlations with all other columns. Next, the algorithm will select the
+row which has the largest total row-wise distance with all other rows.
+Then, the element at the selected row and column will be exchanged with
+a random element from the same column. The remaining steps are the same
+as the SA in (Morris and Mitchell 1995). In the LHD package, the
+function SA2008() implements this algorithm:
SA2008(n, k, N = 10, T0 = 10, rate = 0.1, Tmin = 1, Imax = 5, OC = "phi_p",
+p = 15, q = 1, maxtime = 5),where all the input arguments are the same as in SA.
Particle Swarm Optimization Algorithms
+Particle swarm optimization (PSO, (Kennedy and Eberhart 1995)) is a
+metaheuristic optimization algorithm inspired by the social behaviors of
+animals. Recent research (Chen et al. 2013) adapted the classic PSO
+algorithm and proposed LaPSO to identify maximin distance LHDs. Since
+this is a discrete optimization task, LaPSO redefines the steps in which
+each particle updates its velocity and position in the general PSO
+framework. In the LHD package, the function LaPSO() implements this
+algorithm:
LaPSO(n, k, m = 10, N = 10, SameNumP = 0, SameNumG = n/4, p0 = 1/(k - 1),
+OC = "phi_p", p = 15, q = 1, maxtime = 5)Table 2 provides an
+overview of all the input arguments in LaPSO(), where n, k, N,
+OC, p, q, and maxtime are exactly the same as the input
+arguments in the function SA(). m is the number of particles, which
+represents candidate solutions in the PSO framework. SameNumP and
+SameNumG are two tuning parameters that denote how many exchanges
+would be performed to reduce the Hamming distance towards the personal
+best and the global best. p0 is the tuning parameter that denotes the
+probability of a random swap for two elements in the current column of
+the current particle to prevent the algorithm from being stuck at the
+local optimum. In (Chen et al. 2013), they provided the following
+suggestions: SameNumP is approximately \(n/2\) when SameNumG is \(0\),
+SameNumG is approximately \(n/4\) when SameNumP is \(0\), and p0
+should be between \(1/(k-1)\) and \(2/(k-1)\). The stopping criterion of the
+function is the same as that of the function SA.
+
+
+
+
+
+| Argument | +Description | +
|---|---|
n |
+A positive integer that defines the number of rows (or run size) of output LHD. | +
k |
+A positive integer that defines the number of columns (or factor size) of output LHD. | +
m |
+A positive integer that defines the number of particles, where each particle is a | +
| + | candidate solution. A large value of N will result in a high CPU time, and it is |
+
| + | recommended to be no greater than 100. The default is set to be 10. | +
N |
+A positive integer that defines the maximum number of iterations in the algorithm. | +
| + | A large value of N will result in a high CPU time, and it is recommended to be no |
+
| + | greater than 500. The default is set to be 10. | +
SameNumP |
+A non-negative integer that defines how many elements in current column of | +
| + | current particle should be the same as corresponding Personal Best. SameNumP | +
| + | can be 0, 1, 2, …, n, where 0 means to skip the element exchange, which is the | +
| + | default setting. | +
SameNumG |
+A non-negative integer that defines how many elements in current column of | +
| + | current particle should be the same as corresponding Global Best. SameNumG can | +
| + | be 0, 1, 2, …, n, where 0 means to skip the element exchange. The default setting | +
| + | is n/4. Note that SameNumP and SameNumG cannot be 0 at the same time. | +
p0 |
+A probability of exchanging two randomly selected elements in current column of | +
| + | current particle LHD. The default is set to be 1/(k - 1). | +
OC |
+An optimality criterion. The default setting is “phi_p", and it could be one of | +
| + | the following: “phi_p",”AvgAbsCor", “MaxAbsCor",”MaxProCriterion". | +
p |
+A positive integer, which is one parameter in the \(\phi_{p}\) formula, and p is preferred |
+
| + | to be large. The default is set to be 15. | +
q |
+A positive integer, which is one parameter in the \(\phi_{p}\) formula, and q could be |
+
| + | either 1 or 2. If q is 1, the Manhattan (rectangular) distance will be calculated. |
+
| + | If q is 2, the Euclidean distance will be calculated. |
+
maxtime |
+A positive number, which indicates the expected maximum CPU time, and it is | +
| + | measured by minutes. For example, maxtime=3.5 indicates the CPU time will |
+
| + | be no greater than three and a half minutes. The default is set to be 5. | +
Genetic Algorithms
+The genetic algorithm (GA) is a nature-inspired metaheuristic
+optimization algorithm that mimics Charles Darwin’s idea of natural
+selection (Goldberg 1989; Holland et al. 1992).
+(Liefvendahl and Stocki 2006) proposed a version of GA for identifying maximin
+distance LHDs. They implement the column exchange technique to solve the
+discrete optimization task. In the LHD package, the function GA()
+implements this algorithm:
GA(n, k, m = 10, N = 10, pmut = 1/(k - 1), OC = "phi_p", p = 15, q = 1,
+maxtime = 5)Table 3 provides an
+overview of all the input arguments in GA(), where n, k, N,
+OC, p, q, and maxtime are exactly the same as the input
+arguments in the function SA(). m is the population size, which
+represents how many candidate solutions in each iteration, and must be
+an even number. pmut is the tuning parameter that controls how likely
+the mutation would happen. When mutation occurs, two randomly selected
+elements will be exchanged in the current column of the current LHD.
+pmut serves the same purpose as p0 in LaPSO(), which prevents the
+algorithm from getting stuck at the local optimum, and it is recommended
+to be \(1/(k-1)\). The stopping criterion of the function is the same as
+that of the function SA.
+
+
+
+
+
+| Argument | +Description | +
|---|---|
n |
+A positive integer that defines the number of rows (or run size) of output LHD. | +
k |
+A positive integer that defines the number of columns (or factor size) of output LHD. | +
m |
+A positive even integer, which stands for the population size and it must be an even | +
| + | number. The default is set to be 10. A large value of m will result in a high CPU time, | +
| + | and it is recommended to be no greater than 100. | +
N |
+A positive integer that defines the maximum number of iterations in the algorithm. | +
| + | A large value of N will result in a high CPU time, and it is recommended to be no |
+
| + | greater than 500. The default is set to be 10. | +
pmut |
+A probability for mutation. When the mutation happens, two randomly selected | +
| + | elements in current column of current LHD will be exchanged. The default is | +
| + | set to be 1/(k - 1). | +
OC |
+An optimality criterion. The default setting is “phi_p", and it could be one of | +
| + | the following: “phi_p",”AvgAbsCor", “MaxAbsCor",”MaxProCriterion". | +
p |
+A positive integer, which is one parameter in the \(\phi_{p}\) formula, and p is preferred |
+
| + | to be large. The default is set to be 15. | +
q |
+A positive integer, which is one parameter in the \(\phi_{p}\) formula, and q could be |
+
| + | either 1 or 2. If q is 1, the Manhattan (rectangular) distance will be calculated. |
+
| + | If q is 2, the Euclidean distance will be calculated. |
+
maxtime |
+A positive number, which indicates the expected maximum CPU time, and it is | +
| + | measured by minutes. For example, maxtime=3.5 indicates the CPU time will |
+
| + | be no greater than three and a half minutes. The default is set to be 5. | +
Illustrating Examples for the Implemented Search Algorithms
+This subsection demonstrates some examples on how to use the search
+algorithms in the developed LHD package. In
+Table 4, we summarize
+the R functions of the algorithms discussed in the previous subsections,
+which can be used to identify different types of optimal LHDs. Users who
+seek fast solutions can use the default settings of the input arguments
+after specifying the design sizes. See the following examples.
+
+
+
+
+
+| Function | +Description | +
|---|---|
| SA | +Returns an LHD via the simulated annealing algorithm (Morris and Mitchell 1995). | +
| OASA | +Returns an LHD via the orthogonal-array-based simulated annealing algorithm | +
| + | (Leary et al. 2003), where an OA of the required design size must exist. | +
| SA2008 | +Returns an LHD via the simulated annealing algorithm with the multi-objective | +
| + | optimization approach (Joseph and Hung 2008). | +
| LaPSO | +Returns an LHD via the particle swarm optimization (Chen et al. 2013). | +
| GA | +Returns an LHD via the genetic algorithm (Liefvendahl and Stocki 2006). | +
#Generate a 5 by 3 maximin distance LHD by the SA function.
+> try.SA = SA(n = 5, k = 3); try.SA
+ [,1] [,2] [,3]
+[1,] 2 2 1
+[2,] 5 3 2
+[3,] 4 5 5
+[4,] 3 1 4
+[5,] 1 4 3
+> phi_p(try.SA) #\phi_p is smaller than that of a random LHD (0.3336608).
+[1] 0.2169567
+
+#Similarly, generations of 5 by 3 maximin distance LHD by the SA2008, LaPSO and GA functions.
+> try.SA2008 = SA2008(n = 5, k = 3)
+> try.LaPSO = LaPSO(n = 5, k = 3)
+> try.GA = GA(n = 5, k = 3)
+
+#Generate an OA(9,2,3,2), an orthogonal array with 9 runs, 2 factors, 3 levels, and 2 strength.
+> OA = matrix(c(rep(1:3, each = 3), rep(1:3, times = 3)),
++ ncol = 2, nrow = 9, byrow = FALSE)
+#Generates a maximin distance LHD with the same design size as the input OA
+#by the orthogonal-array-based simulated annealing algorithm.
+> try.OASA = OASA(OA)
+> OA; try.OASA
+ [,1] [,2] [,1] [,2]
+[1,] 1 1 [1,] 1 2
+[2,] 1 2 [2,] 2 6
+[3,] 1 3 [3,] 3 9
+[4,] 2 1 [4,] 4 3
+[5,] 2 2 [5,] 6 5
+[6,] 2 3 [6,] 5 7
+[7,] 3 1 [7,] 7 1
+[8,] 3 2 [8,] 9 4
+[9,] 3 3; [9,] 8 8Note that the default optimality criterion embedded in all search
+algorithms is “phi_p" (that is, the maximin distance criterion),
+leading to the maximin \(L_2\)-distance LHDs. For other optimality
+criteria, users should change the setting of the input argument OC
+(with options”phi_p", “MaxProCriterion",”MaxAbsCor" and
+“MaxProCriterion"). The following examples illustrate some details of
+different argument settings.
#Below try.SA is a 5 by 3 maximin distance LHD generated by the SA with 30 iterations (N = 30).
+#The temperature starts at 10 (T0 = 10) and decreases 10% (rate = 0.1) each time.
+#The minimium temperature allowed is 1 (Tmin = 1) and the maximum perturbations that
+#the algorithm will try without improvements is 5 (Imax = 5). The optimality criterion
+#used is maximin distance criterion (OC = "phi_p") with p = 15 and q = 1, and the
+#maximum CPU time is 5 minutes (maxtime = 5).
+> try.SA = SA(n = 5, k = 3, N = 30, T0 = 10, rate = 0.1, Tmin = 1, Imax = 5, OC = "phi_p",
++ p = 15, q = 1, maxtime = 5); try.SA
+ [,1] [,2] [,3]
+[1,] 1 3 4
+[2,] 2 5 2
+[3,] 5 4 3
+[4,] 4 1 5
+[5,] 3 2 1
+> phi_p(try.SA)
+[1] 0.2169567
+
+#Below try.SA2008 is a 5 by 3 maximin distance LHD generated by SA with
+#the multi-objective optimization approach. The input arguments are interpreted
+#the same as the design try.SA above.
+> try.SA2008 = SA2008(n = 5, k = 3, N = 30, T0 = 10, rate = 0.1, Tmin = 1, Imax = 5,
++ OC = "phi_p", p = 15, q = 1, maxtime = 5)
+
+#Below try.OASA is a 9 by 2 maximin distance LHD generated by the
+#orthogonal-array-based simulated annealing algorithm with the input
+#OA (defined previously), and the rest input arguments are interpreted the
+#same as the design try.SA above.
+> try.OASA = OASA(OA, N = 30, T0 = 10, rate = 0.1, Tmin = 1, Imax = 5,
++ OC = "phi_p", p = 15, q = 1, maxtime = 5)
+
+#Below try.LaPSO is a 5 by 3 maximum projection LHD generated by the particle swarm
+#optimization algorithm with 20 particles (m = 20) and 30 iterations (N = 30).
+#Zero (or two) elements in any column of the current particle should be the same as
+#the elements of corresponding column from personal best (or global best), because
+#of SameNumP = 0 (or SameNumG = 2).
+#The probability of exchanging two randomly selected elements is 0.5 (p0 = 0.5).
+#The optimality criterion is maximum projection criterion (OC = "MaxProCriterion").
+#The maximum CPU time is 5 minutes (maxtime = 5).
+> try.LaPSO = LaPSO(n = 5, k = 3, m = 20, N = 30, SameNumP = 0, SameNumG = 2,
++ p0 = 0.5, OC = "MaxProCriterion", maxtime = 5); try.LaPSO
+ [,1] [,2] [,3]
+[1,] 4 5 4
+[2,] 3 1 3
+[3,] 5 2 1
+[4,] 2 3 5
+[5,] 1 4 2
+#Recall the value is 0.5375482 from the random LHD in Section 2.
+> MaxProCriterion(try.LaPSO)
+[1] 0.3561056
+
+#Below try.GA is a 5 by 3 OLHD generated by the genetic algorithm with the
+#population size 20 (m = 20), number of iterations 30 (N = 30), mutation
+#probability 0.5 (pmut = 0.5), maximum absolute correlation criterion
+#(OC = "MaxAbsCor"), and maximum CPU time 5 minutes (maxtime = 5).
+> try.GA = GA(n = 5, k = 3, m = 20, N = 30, pmut = 0.5, OC = "MaxAbsCor",
++ maxtime = 5); try.GA
+ [,1] [,2] [,3]
+[1,] 2 1 2
+[2,] 4 4 5
+[3,] 3 5 1
+[4,] 5 2 3
+[5,] 1 3 4
+#Recall the value is 0.9 from the random LHD in Section 2.
+> MaxAbsCor(try.GA)
+[1] 0.1 #The maximum absolute correlation between columns is 0.1Next, we discuss some details of the implementation. In SA based
+algorithms (SA, SA2008, and OASA), the number of iterations N is
+recommended to be no greater than 500 for computing time considerations.
+The input rate determines the percentage of the decrease in current
+temperature (for example, \(0.1\) means a decrease of \(10\%\) each time). A
+high rate would make the temperature rapidly drop, which leads to a fast
+stop of the algorithm. It is recommended to set rate from \(0.1\) to
+\(0.15\). Imax indicates the maximum perturbations that the algorithm
+will attempt without improvements before the temperature reduces, and it
+is recommended to be no greater than 5 for computing time
+considerations. OC chooses the optimality criterion, and the "phi_p"
+criterion in (1) is set as default. OC has other options,
+including "MaxProCriterion", "AvgAbsCor" and "MaxAbsCor". Our
+algorithms support both the \(L_1\) and \(L_2\) distances.
For every algorithm, we incorporate a progress bar to visualize the
+computing time used. After an algorithm is completed, information on
+“average CPU time per iteration" and”numbers of iterations completed"
+will be presented. Users can set the limit for the CPU time used for
+each algorithm using the argument maxtime, according to their
+practical needs.
We also provide some illustrative code to demonstrate that the designs
+found in the LHD package are better than the existing ones, and the
+code in the following can be easily modified to construct other design
+sizes or other LHD types. Out of 100 trials, the code below shows the GA
+in the LHD package constructed better MaxPro LHDs 99 times compared to
+the algorithm in the MaxPro package, when 500 iterations are set for
+both algorithms. We did not compare the CPU time between these two
+packages since one is written in the R environment and the other one is
+written in the C++ environment, but with the same number of iterations,
+the GA in the LHD package almost always constructs better MaxPro LHDs.
#Make sure both packages are properly installed before load them
+> library(LHD)
+> library(MaxPro)
+
+> count = 0 #Define a variable for counting purpose
+
+> k = 5 #Factor size 5
+> n = 10*k #Run size = 10*factor size
+
+#Setting 500 iterations for both algorithms, below loop counts
+#how many times the GA from LHD package outperforms the algorithm
+#from MaxPro package out of 100 times
+> for (i in 1:100) {
+
+ LHD = LHD::GA(n = n, k = k, m = 100, N = 500)
+ MaxPro = MaxPro::MaxProLHD(n = n, p = k, total_iter = 500)$Design
+
+ #MaxPro * n + 0.5 applied the transformation mentioned in Section 2
+ #to revert the scaling.
+ Result.LHD = LHD::MaxProCriterion(LHD)
+ Result.MaxPro = LHD::MaxProCriterion(MaxPro * n + 0.5)
+
+ if (Result.LHD < Result.MaxPro) {count = count + 1}
+
+}
+
+> count
+[1] 994 Algebraic Constructions for Optimal LHDs with Certain Sizes
+There are algebraic constructions available for certain design sizes,
+and theoretical results are developed to guarantee the efficiency of
+such designs. Algebraic constructions almost do not require any
+searching, which are especially attractive for large designs. In this
+section, we present algebraic constructions that are available in the
+LHD package for maximin distance LHDs and orthogonal LHDs.
Algebraic Constructions for Maximin Distance LHDs
+(Wang et al. 2018) proposed to generate maximin distance LHDs via good
+lattice point (GLP) sets (Zhou and Xu 2015) and Williams transformation
+(Williams 1949). In practice, their method can lead to
+space-filling designs with relatively flexible sizes, where the run size
+\(n\) is flexible but the factor size \(k\) must be no greater than the
+number of positive integers that are co-prime to \(n\). They proved that
+the resulting designs of sizes \(n \times (n-1)\) (with \(n\) being any odd
+prime) and \(n \times n\) (with \(2n+1\) or \(n+1\) being odd prime) are
+optimal under the maximin \(L_1\) distance criterion. This construction
+method by (Wang et al. 2018) is very attractive for constructing large
+maximin distance LHDs. In the LHD package, function FastMmLHD()
+implements this method:
FastMmLHD(n, k, method = "manhattan", t1 = 10),where n and k are the desired run size and factor size. method is
+a distance measure method which can be one of the following:
+“euclidean",”maximum", “manhattan",”canberra", “binary" or”minkowski". Any unambiguous substring can be given. t1 is a tuning
+parameter, which determines how many repeats will be implemented to
+search for the optimal design. The default is set to be 10.
(Tang 1993) proposed to construct orthogonal array-based LHDs
+(OALHDs) from existing orthogonal arrays (OAs), and
+(Tang 1993) showed that the OALHDs can have better
+space-filling properties than the general ones. In the LHD package,
+function OA2LHD() implements this method:
OA2LHD(OA),where OA is an orthogonal array matrix. Users only need to input an OA
+and the function will return an OALHD with the same design size as the
+input OA.
Algebraic Constructions for Orthogonal LHDs
+Orthogonal LHDs (OLHDs) have zero pairwise correlation between any two
+columns, which are widely used by practitioners. There is a rich
+literature on the constructions of OLHDs with various design sizes, but
+they are often too hard for practitioners to replicate in practice. The
+LHD package implements some currently popular methods
+(Tang 1993; Ye 1998; Butler 2001; Cioppa and Lucas 2007; Lin et al. 2009; Sun et al. 2010)
+for practitioners and the functions are easy to use.
(Ye 1998) proposed a construction for OLHDs with run sizes
+\(n=2^m+1\) and factor sizes \(k=2m-2\) where \(m\) is any integer bigger than
+2. In the LHD package, function OLHD.Y1998() implements this
+algebraic construction:
OLHD.Y1998(m),where input argument m is the \(m\) in the construction of
+(Ye 1998). (Cioppa and Lucas 2007) extended
+(Ye 1998)’s method to construct OLHDs with run size \(n=2^m+1\)
+and factor size \(k=m+ {m-1 \choose 2}\), where \(m\) is any integer bigger
+than 2. In the LHD package, function OLHD.C2007() implements this
+algebraic construction with input argument m remaining the same:
OLHD.C2007(m)(Sun et al. 2010) extended their earlier work
+(Sun et al. 2009) to construct OLHDs with \(n=r2^{c+1}+1\) or
+\(n=r2^{c+1}\) and \(k=2^c\), where \(r\) and \(c\) are positive integers. In
+the LHD package, function OLHD.S2010() implements this algebraic
+construction:
OLHD.S2010(C, r, type = "odd"),where input arguments C and r are \(c\) and \(r\) in the construction.
+When input argument type is "odd", the output design size would be
+\(n=r2^{c+1}+1\) by \(k=2^c\). When input argument type is "even", the
+output design size would be \(n=r2^{c+1}\) by \(k=2^c\).
(Lin et al. 2009) constructed OLHDs or NOLHDs with \(n^2\) runs and
+\(2fp\) factors by coupling OLHD(\(n\), \(p\)) or NOLHD(\(n\), \(p\)) with an
+OA(\(n^2,2f,n,2\)). For example, an OLHD(11, 7), coupled with an
+OA(121,12,11,2), would yield an OLHD(121, 84). The design size of output
+OLHD or NOLHD highly depends on the existence of the OAs. In the LHD
+package, function OLHD.L2009() implements this algebraic construction:
OLHD.L2009(OLHD, OA),where input arguments OLHD and OA are the OLHD and OA to be coupled,
+and their design sizes need to be aligned with the designated pattern of
+the construction.
(Butler 2001) proposed a method to construct OLHDs with the run
+size \(n\) being odd primes and factor size \(k\) being less than or equal
+to \(n-1\) via the Williams transformation (Williams 1949). In
+the LHD package, function OLHD.B2001() implements this algebraic
+construction with input arguments n and k exactly matching those in
+construction:
OLHD.B2001(n, k)Illustrating Examples for the Implemented Algebraic Constructions
+In Table 5, we summarize
+the algebraic constructions implemented by the developed LHD package,
+where FastMmLHD and OA2LHD are for maximin distance LHDs and
+OLHD.Y1998, OLHD.C2007, OLHD.S2010, OLHD.L2009 and OLHD.B2001
+are for orthogonal LHDs. The following examples will illustrate how to
+use them.
+
+
+
+
+
+| Function | +Description | +
|---|---|
| FastMmLHD | +Returns a maximin distance LHD matrix (Wang et al. 2018). | +
| OA2LHD | +Expands an orthogonal array to an LHD (Tang 1993). | +
| OLHD.Y1998 | +Returns a \(2^m+1\) by \(2m-2\) orthogonal LHD matrix (Ye 1998) | +
| + | where \(m\) is an integer and \(m \geq 2\). | +
| OLHD.C2007 | +Returns a \(2^m+1\) by \(m+{m-1 \choose 2}\) orthogonal LHD matrix | +
| + | (Cioppa and Lucas 2007) where \(m\) is an integer and \(m \geq 2\). | +
| OLHD.S2010 | +Returns a \(r2^{c+1}+1\) or \(r2^{c+1}\) by \(2^c\) orthogonal LHD matrix | +
| + | (Sun et al. 2010) where \(r\) and \(c\) are positive integers. | +
| OLHD.L2009 | +Couples an \(n\) by \(p\) orthogonal LHD with a \(n^2\) by \(2f\) strength \(2\) and | +
| + | level \(n\) orthogonal array to generate a \(n^2\) by \(2fp\) orthogonal LHD | +
| + | (Lin et al. 2009). | +
| OLHD.B2001 | +Returns an orthogonal LHD (Butler 2001) with the run size \(n\) | +
| + | being odd primes and factor size \(k\) being less than or equal to \(n-1\) . | +
#FastMmLHD(8, 8) generates an optimal 8 by 8 maximin L_1 distance LHD.
+>try.FastMm = FastMmLHD(n = 8, k = 8); try.FastMm
+ [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8]
+[1,] 0 1 2 3 4 5 6 7
+[2,] 1 3 5 7 6 4 2 0
+[3,] 2 5 7 4 1 0 3 6
+[4,] 3 7 4 0 2 6 5 1
+[5,] 4 6 1 2 7 3 0 5
+[6,] 5 4 0 6 3 1 7 2
+[7,] 6 2 3 5 0 7 1 4
+[8,] 7 0 6 1 5 2 4 3
+
+#OA2LHD(OA) expands an input OA to an LHD of the same run size.
+>try.OA2LHD = OA2LHD(OA)
+>OA; try.OA2LHD
+ [,1] [,2] [,1] [,2]
+[1,] 1 1 [1,] 1 2
+[2,] 1 2 [2,] 2 4
+[3,] 1 3 [3,] 3 9
+[4,] 2 1 [4,] 4 3
+[5,] 2 2 [5,] 5 5
+[6,] 2 3 [6,] 6 7
+[7,] 3 1 [7,] 9 1
+[8,] 3 2 [8,] 8 6
+[9,] 3 3; [9,] 7 8#OLHD.Y1998(m = 3) generates a 9 by 4 orthogonal LHD.
+#Note that 2^m+1 = 9 and 2*m-2 = 4.
+> try.Y1998 = OLHD.Y1998(m = 3); try.Y1998
+ [,1] [,2] [,3] [,4]
+[1,] 4 -3 -2 1
+[2,] 3 4 -1 -2
+[3,] 1 -2 3 -4
+[4,] 2 1 4 3
+[5,] 0 0 0 0
+[6,] -4 3 2 -1
+[7,] -3 -4 1 2
+[8,] -1 2 -3 4
+[9,] -2 -1 -4 -3
+> MaxAbsCor(try.Y1998) #column-wise correlations are 0.
+[1] 0
+
+#OLHD.C2007(m = 4) generates a 17 by 7 orthogonal LHD.
+#Note that 2^m+1 = 17 and $4+{4-1 \choose 2}$ = 7.
+> try.C2007 = OLHD.C2007(m = 4); dim(try.C2007)
+[1] 17 7
+> MaxAbsCor(try.C2007) #column-wise correlations are 0
+[1] 0
+
+#OLHD.S2010(C = 3, r = 3, type = "odd") generates a 49 by 8 orthogonal LHD.
+#Note that 3*2^4+1 = 49 and 2^3 = 8.
+> dim(OLHD.S2010(C = 3, r = 3, type = "odd"))
+[1] 49 8
+> MaxAbsCor(OLHD.S2010(C = 3, r = 3, type = "odd")) #column-wise correlations are 0
+[1] 0
+
+#OLHD.S2010(C = 3, r = 3, type = "even") generates a 48 by 8 orthogonal LHD.
+#Note that 3*2^4 = 48 and 2^3 = 8.
+> dim(OLHD.S2010(C = 3, r = 3, type = "even"))
+[1] 48 8
+> MaxAbsCor(OLHD.S2010(C = 3, r = 3, type = "even")) #column-wise correlations are 0
+[1] 0
+
+#Create a 5 by 2 OLHD.
+> OLHD = OLHD.C2007(m = 2)
+
+#Create an OA(25, 6, 5, 2).
+> OA = matrix(c(2,2,2,2,2,1,2,1,5,4,3,5,3,2,1,5,4,5,1,5,4,3,2,5,4,1,3,5,2,3,
+1,2,3,4,5,2,1,3,5,2,4,3,1,1,1,1,1,1,4,3,2,1,5,5,5,5,5,5,5,1,4,4,4,4,4,1,
+3,1,4,2,5,4,3,3,3,3,3,1,3,5,2,4,1,3,3,4,5,1,2,2,5,4,3,2,1,5,2,3,4,5,1,2,
+2,5,3,1,4,4,1,4,2,5,3,4,4,2,5,3,1,4,2,4,1,3,5,3,5,3,1,4,2,4,5,2,4,1,3,3,
+5,1,2,3,4,2,4,5,1,2,3,2), ncol = 6, nrow = 25, byrow = TRUE)
+
+#OLHD.L2009(OLHD, OA) generates a 25 by 12 orthogonal LHD.
+#Note that n = 5 so n^2 = 25. p = 2 and f = 3 so 2fp = 12.
+> dim(OLHD.L2009(OLHD, OA))
+[1] 25 12
+> MaxAbsCor(OLHD.L2009(OLHD, OA)) #column-wise correlations are 0.
+[1] 0
+
+#OLHD.B2001(n = 11, k = 5) generates a 11 by 5 orthogonal LHD.
+> dim(OLHD.B2001(n = 11, k = 5))
+[1] 11 55 Other R Packages for Latin Hypercube and Comparative Discussion
+Several R packages have been developed to facilitate Latin hypercube
+samples and design constructions for computer experiments. Among these,
+the lhs package (Carnell 2024) is widely recognized for its utility. It
+provides functions for generating both random and optimized Latin
+hypercube samples (but not designs), and its methods are particularly
+useful for simulation studies where space-filling properties are desired
+but design optimality is not the primary focus. The SLHD package
+(Ba 2015) was originally developed for generating sliced LHDs
+(Ba et al. 2015), while practitioners can set the number of slices to
+one to use the package for generating maximin LHDs. The MaxPro package
+(Ba and Joseph 2018) focuses on constructing designs that maximize projection
+properties. One of its functions, MaxProLHD, generates MaxPro LHDs
+using a simulated annealing algorithm (Joseph et al. 2015).
While we acknowledge the contributions of other relevant R packages, we
+emphasize the distinguishing features of our developed package. The
+LHD package embeds multiple optimality criteria, enabling the
+construction of various types of optimal LHDs. In contrast, lhs and
+SLHD primarily focus on space-filling Latin hypercube samples and
+designs, while MaxPro primarily focuses on maximum projection LHDs.
+The LHD package implements various search algorithms and algebraic
+constructions, whereas the other three packages do not implement
+algebraic constructions, and both SLHD and MaxPro only implement one
+algorithm to construct LHDs. The primary application of LHD is in the
+design of computer experiments, whereas lhs is mainly used for
+sampling and simulation studies. Therefore, LHD emphasizes design
+optimality, while lhs emphasizes the space-filling properties of
+samples.
6 Conclusion and Recommendation
+LHD package implements popular search algorithms, including the SA
+(Morris and Mitchell 1995), OASA (Leary et al. 2003), SA2008
+(Joseph and Hung 2008), LaPSO (Chen et al. 2013) and GA
+(Liefvendahl and Stocki 2006), along with some widely used algebraic
+constructions
+(Tang 1993; Ye 1998; Butler 2001; Cioppa and Lucas 2007; Lin et al. 2009; Sun et al. 2010; Wang et al. 2018),
+for constructing three types of commonly used optimal LHDs: the maximin
+distance LHDs, the maximum projection LHDs and the (nearly) orthogonal
+LHDs. We aim to provide guidance and an easy-to-use tool for
+practitioners to find appropriate experimental designs. Algebraic
+constructions are preferred when available, especially for large
+designs. Search algorithms are used to generate optimal LHDs with
+flexible sizes.
Among very few R libraries particularly for LHDs, LHD is comprehensive
+and self-contained as it not only has search algorithms and algebraic
+constructions, but also has other useful functions for LHD research and
+development such as calculating different optimality criteria,
+generating random LHDs, exchanging two random elements in a matrix, and
+calculating intersite distance between matrix rows. The help manual in
+the package documentation contains further details and illustrative
+examples for users who want to explore more of the functions in the
+package.
Acknowledgments
+This research was partially supported by the National Science Foundation +(NSF) grant DMS-2311186 and the National Key R&D Program of China +2024YFA1016200. The authors appreciate the reviewers’ constructive +comments and suggestions.
+7 Supplementary materials
+Supplementary materials are available in addition to this article. It can be downloaded at +RJ-2025-033.zip
+8 Note
+This article is converted from a Legacy LaTeX article using the +texor package. +The pdf version is the official version. To report a problem with the html, +refer to CONTRIBUTE on the R Journal homepage.
+
+
+
+
+
+S. Ba. SLHD: Maximin-distance (sliced) Latin hypercube designs. 2015. URL https://CRAN.R-project.org/package=SLHD. R package version 2.1-1.
+
+
+S. Ba and V. R. Joseph. MaxPro: Maximum projection designs. 2018. URL https://CRAN.R-project.org/package=MaxPro. R package version 4.1-2.
+
+
+S. Ba, W. R. Myers and W. A. Brenneman. Optimal sliced Latin hypercube designs. Technometrics, 57(4): 479–487, 2015.
+
+
+N. A. Butler. Optimal and orthogonal Latin hypercube designs for computer experiments. Biometrika, 88(3): 847–857, 2001.
+
+
+R. Carnell. Lhs: Latin hypercube samples. 2024. URL https://CRAN.R-project.org/package=lhs. R package version 1.2.0.
+
+
+R.-B. Chen, D.-N. Hsieh, Y. Hung and W. Wang. Optimizing Latin hypercube designs by particle swarm. Statistics and computing, 23(5): 663–676, 2013.
+
+
+T. M. Cioppa and T. W. Lucas. Efficient nearly orthogonal and space-filling Latin hypercubes. Technometrics, 49(1): 45–55, 2007.
+
+
+K.-T. Fang, R. Li and A. Sudjianto. Design and modeling for computer experiments. CRC press, 2005.
+
+
+K.-T. Fang, C.-X. Ma and P. Winker. Centered \({L}_{2}\)-discrepancy of random sampling and Latin hypercube design, and construction of uniform designs. Mathematics of Computation, 71(237): 275–296, 2002.
+
+
+S. D. Georgiou. Orthogonal Latin hypercube designs from generalized orthogonal designs. Journal of Statistical Planning and Inference, 139(4): 1530–1540, 2009.
+
+
+S. D. Georgiou and I. Efthimiou. Some classes of orthogonal Latin hypercube designs. Statistica Sinica, 24(1): 101–120, 2014.
+
+
+D. E. Goldberg. Genetic algorithms in search. Optimization, and MachineLearning, 1989.
+
+
+A. Grosso, A. Jamali and M. Locatelli. Finding maximin Latin hypercube designs by iterated local search heuristics. European Journal of Operational Research, 197(2): 541–547, 2009.
+
+
+F. Hickernell. A generalized discrepancy and quadrature error bound. Mathematics of computation, 67(221): 299–322, 1998.
+
+
+J. H. Holland et al. Adaptation in natural and artificial systems: An introductory analysis with applications to biology, control, and artificial intelligence. MIT press, 1992.
+
+
+R. Jin, W. Chen and A. Sudjianto. An efficient algorithm for constructing optimal design of computer experiments. Journal of statistical planning and inference, 134(1): 268–287, 2005.
+
+
+M. E. Johnson, L. M. Moore and D. Ylvisaker. Minimax and maximin distance designs. Journal of statistical planning and inference, 26(2): 131–148, 1990.
+
+
+V. R. Joseph, E. Gul and S. Ba. Maximum projection designs for computer experiments. Biometrika, 102(2): 371–380, 2015.
+
+
+V. R. Joseph and Y. Hung. Orthogonal-maximin Latin hypercube designs. Statistica Sinica, 171–186, 2008.
+
+
+J. Kennedy and R. Eberhart. Particle swarm optimization. In Proceedings of ICNN’95-international conference on neural networks, pages. 1942–1948 1995. IEEE.
+
+
+S. Kirkpatrick, C. D. Gelatt and M. P. Vecchi. Optimization by simulated annealing. science, 220(4598): 671–680, 1983.
+
+
+S. Leary, A. Bhaskar and A. Keane. Optimal orthogonal-array-based Latin hypercubes. Journal of Applied Statistics, 30(5): 585–598, 2003.
+
+
+M. Liefvendahl and R. Stocki. A study on algorithms for optimization of Latin hypercubes. Journal of statistical planning and inference, 136(9): 3231–3247, 2006.
+
+
+C. D. Lin, R. Mukerjee and B. Tang. Construction of orthogonal and nearly orthogonal Latin hypercubes. Biometrika, 96(1): 243–247, 2009.
+
+
+M. D. McKay, R. J. Beckman and W. J. Conover. Comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics, 21(2): 239–245, 1979.
+
+
+M. D. Morris and T. J. Mitchell. Exploratory designs for computational experiments. Journal of statistical planning and inference, 43(3): 381–402, 1995.
+
+
+J. Sacks, S. B. Schiller and W. J. Welch. Designs for computer experiments. Technometrics, 31(1): 41–47, 1989.
+
+
+M. C. Shewry and H. P. Wynn. Maximum entropy sampling. Journal of applied statistics, 14(2): 165–170, 1987.
+
+
+D. M. Steinberg and D. K. J. Lin. A construction method for orthogonal Latin hypercube designs. Biometrika, 93(2): 279–288, 2006.
+
+
+F. Sun, M.-Q. Liu and D. K. J. Lin. Construction of orthogonal Latin hypercube designs. Biometrika, 96(4): 971–974, 2009.
+
+
+F. Sun, M.-Q. Liu and D. K. J. Lin. Construction of orthogonal Latin hypercube designs with flexible run sizes. Journal of Statistical Planning and Inference, 140(11): 3236–3242, 2010.
+
+
+F. Sun and B. Tang. A general rotation method for orthogonal Latin hypercubes. Biometrika, 104(2): 465–472, 2017.
+
+
+B. Tang. Orthogonal array-based Latin hypercubes. Journal of the American statistical association, 88(424): 1392–1397, 1993.
+
+
+L. Wang, Q. Xiao and H. Xu. Optimal maximin \({L}_{1}\)-distance latin hypercube designs based on good lattice point designs. The Annals of Statistics, 46(6B): 3741–3766, 2018.
+
+
+E. Williams. Experimental designs balanced for the estimation of residual effects of treatments. Australian Journal of Chemistry, 2(2): 149–168, 1949.
+
+
+Q. Xiao and H. Xu. Construction of maximin distance designs via level permutation and expansion. Statistica Sinica, 28(3): 1395–1414, 2018.
+
+
+Q. Xiao and H. Xu. Construction of maximin distance Latin squares and related Latin hypercube designs. Biometrika, 104(2): 455–464, 2017.
+
+
+J. Yang and M.-Q. Liu. Construction of orthogonal and nearly orthogonal Latin hypercube designs from orthogonal designs. Statistica Sinica, 433–442, 2012.
+
+
+K. Q. Ye. Orthogonal column Latin hypercubes and their application in computer experiments. Journal of the American Statistical Association, 93(444): 1430–1439, 1998.
+
+
+K. Q. Ye, W. Li and A. Sudjianto. Algorithmic construction of optimal symmetric Latin hypercube designs. Journal of statistical planning and inference, 90(1): 145–159, 2000.
+
+
+Y. Zhou and H. Xu. Space-filling properties of good lattice point sets. Biometrika, 102(4): 959–966, 2015.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/_articles/RJ-2025-033/RJ-2025-033.pdf b/_articles/RJ-2025-033/RJ-2025-033.pdf
new file mode 100644
index 0000000000..040cc6554f
Binary files /dev/null and b/_articles/RJ-2025-033/RJ-2025-033.pdf differ
diff --git a/_articles/RJ-2025-033/RJ-2025-033.zip b/_articles/RJ-2025-033/RJ-2025-033.zip
new file mode 100644
index 0000000000..a6fd9d186d
Binary files /dev/null and b/_articles/RJ-2025-033/RJ-2025-033.zip differ
diff --git a/_articles/RJ-2025-033/RJournal.sty b/_articles/RJ-2025-033/RJournal.sty
new file mode 100644
index 0000000000..351990be38
--- /dev/null
+++ b/_articles/RJ-2025-033/RJournal.sty
@@ -0,0 +1,358 @@
+% Package `RJournal' to use with LaTeX2e
+% Copyright (C) 2010 by the R Foundation
+% Copyright (C) 2013 by the R Journal
+%
+% Originally written by Kurt Hornik and Friedrich Leisch with subsequent
+% edits by the editorial board
+%
+% CAUTION:
+% Do not modify this style file. Any changes to this file will be reset when your
+% article is submitted.
+% If you must modify the style or add LaTeX packages to the article, these
+% should be specified in RJwrapper.tex
+
+\NeedsTeXFormat{LaTeX2e}[1995/12/01]
+\ProvidesPackage{RJournal}[2025/10/05 v0.17 RJournal package]
+
+\RequirePackage{tikz}
+
+% Overall page layout, fonts etc -----------------------------------------------
+
+% Issues of of \emph{The R Journal} are created from the standard \LaTeX{}
+% document class \pkg{report}.
+
+\RequirePackage{geometry}
+\geometry{a4paper,
+ textwidth=14cm, top=1cm, bottom=1cm,
+ includehead,includefoot,centering,
+ footskip=1.5cm}
+\raggedbottom
+\sloppy
+\clubpenalty = 10000
+\widowpenalty = 10000
+\brokenpenalty = 10000
+\usepackage{microtype}
+
+
+\RequirePackage{fancyhdr}
+\fancyhead{}
+\fancyheadoffset{2cm}
+\fancyhead[L]{\textsc{\RJ@sectionhead}}
+\fancyhead[R]{\thepage}
+\fancyfoot{}
+\fancyfoot[L]{The R Journal Vol. \RJ@volume/\RJ@number, \RJ@month~\RJ@year}
+\fancyfoot[R]{ISSN 2073-4859}
+\pagestyle{fancy}
+
+% We use the following fonts (all with T1 encoding):
+%
+% rm & palatino
+% tt & inconsolata
+% sf & helvetica
+% math & palatino
+
+\RequirePackage{microtype}
+
+\RequirePackage[scaled=0.92]{helvet}
+\RequirePackage{palatino,mathpazo}
+\RequirePackage[scaled=1.02]{inconsolata}
+\RequirePackage[T1]{fontenc}
+
+\RequirePackage[hyphens]{url}
+\RequirePackage[pagebackref]{hyperref}
+\renewcommand{\backref}[1]{[p#1]}
+
+% Dark blue colour for all links
+\RequirePackage{color}
+\definecolor{link}{rgb}{0.45,0.51,0.67}
+\hypersetup{
+ colorlinks,%
+ citecolor=link,%
+ filecolor=link,%
+ linkcolor=link,%
+ urlcolor=link
+}
+
+% Give the text a little room to breath
+\setlength{\parskip}{3pt}
+\RequirePackage{setspace}
+\setstretch{1.05}
+
+% Issue and article metadata ---------------------------------------------------
+
+% Basic front matter information about the issue: volume, number, and
+% date.
+
+\newcommand{\volume}[1]{\def\RJ@volume{#1}}
+\newcommand{\volnumber}[1]{\def\RJ@number{#1}}
+\renewcommand{\month}[1]{\def\RJ@month{#1}}
+\renewcommand{\year}[1]{\def\RJ@year{#1}}
+
+
+% Individual articles correspond to
+% chapters, and are contained in |article| environments. This makes it
+% easy to have figures counted within articles and hence hyperlinked
+% correctly.
+
+% An article has an author, a title, and optionally a subtitle. We use
+% the obvious commands for specifying these. Articles will be put in certain
+% journal sections, named by \sectionhead.
+
+\newcommand {\sectionhead} [1]{\def\RJ@sectionhead{#1}}
+\renewcommand{\author} [1]{\def\RJ@author{#1}}
+\renewcommand{\title} [1]{\def\RJ@title{#1}}
+\newcommand {\subtitle} [1]{\def\RJ@subtitle{#1}}
+
+% Control appearance of titles: make slightly smaller than usual, and
+% suppress section numbering. See http://tex.stackexchange.com/questions/69749
+% for why we don't use \setcounter{secnumdepth}{-1}
+
+\usepackage[medium]{titlesec}
+\usepackage{titletoc}
+\titleformat{\section} {\normalfont\large\bfseries}{\arabic{section}}{1em}{}
+\titleformat{\subsection}{\normalfont\normalsize\bfseries}{\arabic{section}.\arabic{subsection}}{0.5em}{}
+\titlecontents{chapter} [0em]{}{}{}{\titlerule*[1em]{.}\contentspage}
+
+% Article layout ---------------------------------------------------------------
+
+% Environment |article| clears the article header information at its beginning.
+% We use |\FloatBarrier| from the placeins package to keep floats within
+% the article.
+\RequirePackage{placeins}
+\newenvironment{article}{\author{}\title{}\subtitle{}\FloatBarrier}{\FloatBarrier}
+
+% Refereed articles should have an abstract, so we redefine |\abstract| to
+% give the desired style
+
+\renewcommand{\abstract}[1]{\noindent\textbf{Abstract} #1}
+\renewenvironment{abstract}{\noindent\textbf{Abstract}~}{}
+
+% The real work is done by a redefined version of |\maketitle|. Note
+% that even though we do not want chapters (articles) numbered, we
+% need to increment the chapter counter, so that figures get correct
+% labelling.
+
+\renewcommand{\maketitle}{%
+\noindent
+ \chapter{\RJ@title}\refstepcounter{chapter}
+ \ifx\empty\RJ@subtitle
+ \else
+ \noindent\textbf{\RJ@subtitle}
+ \par\nobreak\addvspace{\baselineskip}
+ \fi
+ \ifx\empty\RJ@author
+ \else
+ \noindent\textit{\RJ@author}
+ \par\nobreak\addvspace{\baselineskip}
+ \fi
+ \@afterindentfalse\@nobreaktrue\@afterheading
+}
+
+% Now for some ugly redefinitions. We do not want articles to start a
+% new page. (Actually, we do, but this is handled via explicit
+% \newpage
+%
+% The name@of@eq is a hack to get hyperlinks to equations to work
+% within each article, even though there may be multiple eq.(1)
+% \begin{macrocode}
+\renewcommand\chapter{\secdef\RJ@chapter\@schapter}
+\providecommand{\nohyphens}{%
+ \hyphenpenalty=10000\exhyphenpenalty=10000\relax}
+\newcommand{\RJ@chapter}{%
+ \edef\name@of@eq{equation.\@arabic{\c@chapter}}%
+ \renewcommand{\@seccntformat}[1]{}%
+ \@startsection{chapter}{0}{0mm}{%
+ -2\baselineskip \@plus -\baselineskip \@minus -.2ex}{\p@}{%
+ \phantomsection\normalfont\huge\bfseries\raggedright}}
+
+% Book reviews should appear as sections in the text and in the pdf bookmarks,
+% however we wish them to appear as chapters in the TOC. Thus we define an
+% alternative to |\maketitle| for reviews.
+\newcommand{\review}[1]{
+ \pdfbookmark[1]{#1}{#1}
+ \section*{#1}
+ \addtocontents{toc}{\protect\contentsline{chapter}{#1}{\thepage}{#1.1}}
+}
+
+% We want bibliographies as starred sections within articles.
+%
+\RequirePackage[sectionbib,round]{natbib}
+\bibliographystyle{abbrvnat}
+\renewcommand{\bibsection}{\section*{References}}
+
+% Equations, figures and tables are counted within articles, but we do
+% not show the article number. For equations it becomes a bit messy to avoid
+% having hyperref getting it wrong.
+
+% \numberwithin{equation}{chapter}
+\renewcommand{\theequation}{\@arabic\c@equation}
+\renewcommand{\thefigure}{\@arabic\c@figure}
+\renewcommand{\thetable}{\@arabic\c@table}
+
+% Issue layout -----------------------------------------------------------------
+
+% Need to provide our own version of |\tableofcontents|. We use the
+% tikz package to get the rounded rectangle. Notice that |\section*|
+% is really the same as |\chapter*|.
+\renewcommand{\contentsname}{Contents}
+\renewcommand\tableofcontents{%
+ \vspace{1cm}
+ \section*{\contentsname}
+ { \@starttoc{toc} }
+}
+
+\renewcommand{\titlepage}{%
+ \thispagestyle{empty}
+ \hypersetup{
+ pdftitle={The R Journal Volume \RJ@volume/\RJ@number, \RJ@month \RJ@year},%
+ pdfauthor={R Foundation for Statistical Computing},%
+ }
+ \noindent
+ \begin{center}
+ \fontsize{50pt}{50pt}\selectfont
+ The \raisebox{-8pt}{\includegraphics[height=77pt]{Rlogo-5}}\hspace{10pt}
+ Journal
+
+ \end{center}
+ {\large \hfill Volume \RJ@volume/\RJ@number, \RJ@month{} \RJ@year \quad}
+
+ \rule{\textwidth}{1pt}
+ \begin{center}
+ {\Large A peer-reviewed, open-access publication of the \\
+ R Foundation for Statistical Computing}
+ \end{center}
+
+ % And finally, put in the TOC box. Note the way |tocdepth| is adjusted
+ % before and after producing the TOC: thus, we can ensure that only
+ % articles show up in the printed TOC, but that in the PDF version,
+ % bookmarks are created for sections and subsections as well (provided
+ % that the non-starred forms are used).
+ \setcounter{tocdepth}{0}
+ \tableofcontents
+ \setcounter{tocdepth}{2}
+ \clearpage
+}
+
+% Text formatting --------------------------------------------------------------
+
+\newcommand{\R}{R}
+\newcommand{\address}[1]{\addvspace{\baselineskip}\noindent\emph{#1}}
+\newcommand{\email}[1]{\href{mailto:#1}{\normalfont\texttt{#1}}}
+
+% Simple font selection is not good enough. For example, |\texttt{--}|
+% gives `\texttt{--}', i.e., an endash in typewriter font. Hence, we
+% need to turn off ligatures, which currently only happens for commands
+% |\code| and |\samp| and the ones derived from them. Hyphenation is
+% another issue; it should really be turned off inside |\samp|. And
+% most importantly, \LaTeX{} special characters are a nightmare. E.g.,
+% one needs |\~{}| to produce a tilde in a file name marked by |\file|.
+% Perhaps a few years ago, most users would have agreed that this may be
+% unfortunate but should not be changed to ensure consistency. But with
+% the advent of the WWW and the need for getting `|~|' and `|#|' into
+% URLs, commands which only treat the escape and grouping characters
+% specially have gained acceptance
+
+\DeclareRobustCommand\code{\bgroup\@noligs\@codex}
+\def\@codex#1{\texorpdfstring%
+{{\normalfont\ttfamily\hyphenchar\font=-1 #1}}%
+{#1}\egroup}
+\newcommand{\kbd}[1]{{\normalfont\texttt{#1}}}
+\newcommand{\key}[1]{{\normalfont\texttt{\uppercase{#1}}}}
+\DeclareRobustCommand\samp{`\bgroup\@noligs\@sampx}
+\def\@sampx#1{{\normalfont\texttt{#1}}\egroup'}
+\newcommand{\var}[1]{{\normalfont\textsl{#1}}}
+\let\env=\code
+\newcommand{\file}[1]{{`\normalfont\textsf{#1}'}}
+\let\command=\code
+\let\option=\samp
+\newcommand{\dfn}[1]{{\normalfont\textsl{#1}}}
+% \acronym is effectively disabled since not used consistently
+\newcommand{\acronym}[1]{#1}
+\newcommand{\strong}[1]{\texorpdfstring%
+{{\normalfont\fontseries{b}\selectfont #1}}%
+{#1}}
+\let\pkg=\strong
+\newcommand{\CRANpkg}[1]{\href{https://CRAN.R-project.org/package=#1}{\pkg{#1}}}%
+\let\cpkg=\CRANpkg
+\newcommand{\ctv}[1]{\href{https://CRAN.R-project.org/view=#1}{\emph{#1}}}
+\newcommand{\BIOpkg}[1]{\href{https://www.bioconductor.org/packages/release/bioc/html/#1.html}{\pkg{#1}}}
+
+% Example environments ---------------------------------------------------------
+\RequirePackage{fancyvrb}
+\RequirePackage{alltt}
+
+\DefineVerbatimEnvironment{example}{Verbatim}{}
+\renewenvironment{example*}{\begin{alltt}}{\end{alltt}}
+
+% Support for output from Sweave, and generic session style code
+% These used to have fontshape=sl for Sinput/Scode/Sin, but pslatex
+% won't use a condensed font in that case.
+
+% Update (2015-05-28 by DS): remove fontsize=\small to match example environment
+
+\DefineVerbatimEnvironment{Sinput}{Verbatim}{}
+\DefineVerbatimEnvironment{Soutput}{Verbatim}{}
+\DefineVerbatimEnvironment{Scode}{Verbatim}{}
+\DefineVerbatimEnvironment{Sin}{Verbatim}{}
+\DefineVerbatimEnvironment{Sout}{Verbatim}{}
+\newenvironment{Schunk}{}{}
+
+% Mathematics ------------------------------------------------------------------
+
+% The implementation of |\operatorname| is similar to the mechanism
+% \LaTeXe{} uses for functions like sin and cos, and simpler than the
+% one of \AmSLaTeX{}. We use |\providecommand| for the definition in
+% order to keep the one of the \pkg{amstex} if this package has
+% already been loaded.
+% \begin{macrocode}
+\providecommand{\operatorname}[1]{%
+ \mathop{\operator@font#1}\nolimits}
+\RequirePackage{amsfonts}
+
+\renewcommand{\P}{%
+ \mathop{\operator@font I\hspace{-1.5pt}P\hspace{.13pt}}}
+\newcommand{\E}{%
+ \mathop{\operator@font I\hspace{-1.5pt}E\hspace{.13pt}}}
+\newcommand{\VAR}{\operatorname{var}}
+\newcommand{\COV}{\operatorname{cov}}
+\newcommand{\COR}{\operatorname{cor}}
+
+% Figures ----------------------------------------------------------------------
+
+% For use with pandoc > 3.2.1
+\newsavebox\pandoc@box
+\newcommand*\pandocbounded[1]{% scales image to fit in text height/width
+ \sbox\pandoc@box{#1}%
+ \Gscale@div\@tempa{\textheight}{\dimexpr\ht\pandoc@box+\dp\pandoc@box\relax}%
+ \Gscale@div\@tempb{\linewidth}{\wd\pandoc@box}%
+ \ifdim\@tempb\p@<\@tempa\p@\let\@tempa\@tempb\fi% select the smaller of both
+ \ifdim\@tempa\p@<\p@\scalebox{\@tempa}{\usebox\pandoc@box}%
+ \else\usebox{\pandoc@box}%
+ \fi%
+}
+
+\RequirePackage[font=small,labelfont=bf]{caption}
+
+% Wide environments for figures and tables -------------------------------------
+\RequirePackage{environ}
+
+% An easy way to make a figure span the full width of the page
+\NewEnviron{widefigure}[1][]{
+\begin{figure}[#1]
+\advance\leftskip-2cm
+\begin{minipage}{\dimexpr\textwidth+4cm\relax}%
+ \captionsetup{margin=2cm}
+ \BODY
+\end{minipage}%
+\end{figure}
+}
+
+\NewEnviron{widetable}[1][]{
+\begin{table}[#1]
+\advance\leftskip-2cm
+\begin{minipage}{\dimexpr\textwidth+4cm\relax}%
+ \captionsetup{margin=2cm}
+ \BODY
+\end{minipage}%
+\end{table}
+}
diff --git a/_articles/RJ-2025-033/RJwrapper.tex b/_articles/RJ-2025-033/RJwrapper.tex
new file mode 100644
index 0000000000..0b69174306
--- /dev/null
+++ b/_articles/RJ-2025-033/RJwrapper.tex
@@ -0,0 +1,36 @@
+\documentclass[a4paper]{report}
+\usepackage[utf8]{inputenc}
+\usepackage[T1]{fontenc}
+\usepackage{RJournal}
+\usepackage{amsmath,amssymb,array}
+\usepackage{booktabs}
+
+\usepackage{multirow}
+\usepackage[]{algorithm, algorithmic}
+
+
+\newcommand{\argmax}{\mathop{\mathrm{argmax}}\limits}
+\newcommand{\argmin}{\mathop{\mathrm{argmin}}\limits}
+\newcommand{\minF}{\mathop{\mathrm{min}}\limits}
+\newcommand{\maxF}{\mathop{\mathrm{max}}\limits}
+\newcommand{\aveF}{\mathop{\mathrm{ave}}\limits}
+
+\newcommand{\RR}[1]{\texttt{#1}}
+
+
+\begin{document}
+
+%% do not edit, for illustration only
+\sectionhead{Contributed research article}
+\volume{17}
+\volnumber{4}
+\year{2025}
+\month{December}
+\setcounter{page}{20}
+
+%% replace RJtemplate with your article
+\begin{article}
+ \input{wang-xiao-mandal.tex}
+\end{article}
+
+\end{document}
diff --git a/_articles/RJ-2025-033/wang-xiao-mandal.R b/_articles/RJ-2025-033/wang-xiao-mandal.R
new file mode 100644
index 0000000000..c28b51b1a0
--- /dev/null
+++ b/_articles/RJ-2025-033/wang-xiao-mandal.R
@@ -0,0 +1,166 @@
+library(LHD)
+X = rLHD(n = 5, k = 3); X #This generates a 5 by 3 random LHD, denoted as X
+
+phi_p(X) #The maximin L1-distance criterion.
+phi_p(X, p = 10, q = 2) #The maximin L2-distance criterion.
+MaxProCriterion(X) #The maximum projection criterion.
+AvgAbsCor(X) #The average absolute correlation criterion.
+MaxAbsCor(X) #The maximum absolute correlation criterion.
+
+
+#Generate a 5 by 3 maximin distance LHD by the SA function.
+try.SA = SA(n = 5, k = 3); try.SA
+
+phi_p(try.SA) #\phi_p is smaller than that of a random LHD (0.3336608).
+
+
+#Similarly, generations of 5 by 3 maximin distance LHD by the SA2008, LaPSO and GA functions.
+try.SA2008 = SA2008(n = 5, k = 3)
+try.LaPSO = LaPSO(n = 5, k = 3)
+try.GA = GA(n = 5, k = 3)
+
+#Generate an OA(9,2,3,2), an orthogonal array with 9 runs, 2 factors, 3 levels, and 2 strength.
+OA = matrix(c(rep(1:3, each = 3), rep(1:3, times = 3)),
+ ncol = 2, nrow = 9, byrow = FALSE)
+#Generates a maximin distance LHD with the same design size as the input OA
+#by the orthogonal-array-based simulated annealing algorithm.
+try.OASA = OASA(OA)
+OA; try.OASA
+
+#Below try.SA is a 5 by 3 maximin distance LHD generated by the SA with 30 iterations (N = 30).
+#The temperature starts at 10 (T0 = 10) and decreases 10% (rate = 0.1) each time.
+#The minimium temperature allowed is 1 (Tmin = 1) and the maximum perturbations that
+#the algorithm will try without improvements is 5 (Imax = 5). The optimality criterion
+#used is maximin distance criterion (OC = "phi_p") with p = 15 and q = 1, and the
+#maximum CPU time is 5 minutes (maxtime = 5).
+try.SA = SA(n = 5, k = 3, N = 30, T0 = 10, rate = 0.1, Tmin = 1, Imax = 5, OC = "phi_p",
+ p = 15, q = 1, maxtime = 5); try.SA
+
+phi_p(try.SA)
+
+
+#Below try.SA2008 is a 5 by 3 maximin distance LHD generated by SA with
+#the multi-objective optimization approach. The input arguments are interpreted
+#the same as the design try.SA above.
+try.SA2008=SA2008(n=5,k=3,N=30,T0=10,rate=0.1,Tmin=1,Imax=5,
+ OC="phi_p",p=15,q=1,maxtime=5)
+
+#Below try.OASA is a 9 by 2 maximin distance LHD generated by the
+#orthogonal-array-based simulated annealing algorithm with the input
+#OA (defined previously), and the rest input arguments are interpreted the
+#same as the design try.SA above.
+try.OASA = OASA(OA, N = 30, T0 = 10, rate = 0.1, Tmin = 1, Imax = 5,
+ OC = "phi_p", p = 15, q = 1, maxtime = 5)
+
+#Below try.LaPSO is a 5 by 3 maximum projection LHD generated by the particle swarm
+#optimization algorithm with 20 particles (m = 20) and 30 iterations (N = 30).
+#Zero (or two) elements in any column of the current particle should be the same as
+#the elements of corresponding column from personal best (or global best), because
+#of SameNumP = 0 (or SameNumG = 2).
+#The probability of exchanging two randomly selected elements is 0.5 (p0 = 0.5).
+#The optimality criterion is maximum projection criterion (OC = "MaxProCriterion").
+#The maximum CPU time is 5 minutes (maxtime = 5).
+try.LaPSO = LaPSO(n = 5, k = 3, m = 20, N = 30, SameNumP = 0, SameNumG = 2,
+ p0 = 0.5, OC = "MaxProCriterion", maxtime = 5); try.LaPSO
+
+#Recall the value is 0.5375482 from the random LHD in Section 2.
+MaxProCriterion(try.LaPSO)
+
+
+#Below try.GA is a 5 by 3 OLHD generated by the genetic algorithm with the
+#population size 20 (m = 20), number of iterations 30 (N = 30), mutation
+#probability 0.5 (pmut = 0.5), maximum absolute correlation criterion
+#(OC = "MaxAbsCor"), and maximum CPU time 5 minutes (maxtime = 5).
+try.GA = GA(n = 5, k = 3, m = 20, N = 30, pmut = 0.5, OC = "MaxAbsCor",
+ maxtime = 5); try.GA
+
+#Recall the value is 0.9 from the random LHD in Section 2.
+MaxAbsCor(try.GA)
+
+#Make sure both packages are properly installed before load them
+library(LHD)
+library(MaxPro)
+
+count = 0 #Define a variable for counting purpose
+
+k = 5 #Factor size 5
+n = 10*k #Run size = 10*factor size
+
+#Setting 500 iterations for both algorithms, below loop counts
+#how many times the GA from LHD package outperforms the algorithm
+#from MaxPro package out of 100 times
+
+for (i in 1:100) {
+
+ LHD = LHD::GA(n = n, k = k, m = 100, N = 500)
+ MaxPro = MaxPro::MaxProLHD(n = n, p = k, total_iter = 500)$Design
+
+ #MaxPro * n + 0.5 applied the transformation mentioned in Section 2
+ #to revert the scaling.
+ Result.LHD = LHD::MaxProCriterion(LHD)
+ Result.MaxPro = LHD::MaxProCriterion(MaxPro * n + 0.5)
+
+ if (Result.LHD < Result.MaxPro) {count = count + 1}
+
+}
+
+count
+
+#FastMmLHD(8, 8) generates an optimal 8 by 8 maximin L_1 distance LHD.
+try.FastMm = FastMmLHD(n = 8, k = 8); try.FastMm
+
+#OA2LHD(OA) expands an input OA to an LHD of the same run size.
+try.OA2LHD = OA2LHD(OA)
+OA; try.OA2LHD
+
+#OLHD.Y1998(m = 3) generates a 9 by 4 orthogonal LHD.
+#Note that 2^m+1 = 9 and 2*m-2 = 4.
+try.Y1998 = OLHD.Y1998(m = 3); try.Y1998
+
+MaxAbsCor(try.Y1998) #column-wise correlations are 0.
+
+#OLHD.C2007(m = 4) generates a 17 by 7 orthogonal LHD.
+#Note that 2^m+1 = 17 and $4+{4-1 \choose 2}$ = 7.
+try.C2007 = OLHD.C2007(m = 4); dim(try.C2007)
+
+MaxAbsCor(try.C2007) #column-wise correlations are 0
+
+
+#OLHD.S2010(C = 3, r = 3, type = "odd") generates a 49 by 8 orthogonal LHD.
+#Note that 3*2^4+1 = 49 and 2^3 = 8.
+dim(OLHD.S2010(C = 3, r = 3, type = "odd"))
+
+MaxAbsCor(OLHD.S2010(C = 3, r = 3, type = "odd")) #column-wise correlations are 0
+
+
+#OLHD.S2010(C = 3, r = 3, type = "even") generates a 48 by 8 orthogonal LHD.
+#Note that 3*2^4 = 48 and 2^3 = 8.
+dim(OLHD.S2010(C = 3, r = 3, type = "even"))
+
+MaxAbsCor(OLHD.S2010(C = 3, r = 3, type = "even")) #column-wise correlations are 0
+
+
+#Create a 5 by 2 OLHD.
+OLHD = OLHD.C2007(m = 2)
+
+#Create an OA(25, 6, 5, 2).
+OA = matrix(c(2,2,2,2,2,1,2,1,5,4,3,5,3,2,1,5,4,5,1,5,4,3,2,5,4,1,3,5,2,3,
+ 1,2,3,4,5,2,1,3,5,2,4,3,1,1,1,1,1,1,4,3,2,1,5,5,5,5,5,5,5,1,4,4,4,4,4,1,
+ 3,1,4,2,5,4,3,3,3,3,3,1,3,5,2,4,1,3,3,4,5,1,2,2,5,4,3,2,1,5,2,3,4,5,1,2,
+ 2,5,3,1,4,4,1,4,2,5,3,4,4,2,5,3,1,4,2,4,1,3,5,3,5,3,1,4,2,4,5,2,4,1,3,3,
+ 5,1,2,3,4,2,4,5,1,2,3,2), ncol = 6, nrow = 25, byrow = TRUE)
+
+#OLHD.L2009(OLHD, OA) generates a 25 by 12 orthogonal LHD.
+#Note that n = 5 so n^2 = 25. p = 2 and f = 3 so 2fp = 12.
+dim(OLHD.L2009(OLHD, OA))
+
+MaxAbsCor(OLHD.L2009(OLHD, OA)) #column-wise correlations are 0.
+
+
+#OLHD.B2001(n = 11, k = 5) generates a 11 by 5 orthogonal LHD.
+dim(OLHD.B2001(n = 11, k = 5))
+
+
+
+
+
diff --git a/_articles/RJ-2025-033/wang-xiao-mandal.bib b/_articles/RJ-2025-033/wang-xiao-mandal.bib
new file mode 100644
index 0000000000..6fe166255b
--- /dev/null
+++ b/_articles/RJ-2025-033/wang-xiao-mandal.bib
@@ -0,0 +1,797 @@
+@article{ye1998orthogonal,
+ title={Orthogonal column {L}atin hypercubes and their application in computer experiments},
+ author={Ye, K. Qian},
+ journal={Journal of the American Statistical Association},
+ volume={93},
+ number={444},
+ pages={1430--1439},
+ year={1998},
+ publisher={Taylor \& Francis}
+}
+@article{cioppa2007efficient,
+ title={Efficient nearly orthogonal and space-filling {L}atin hypercubes},
+ author={Cioppa, Thomas M and Lucas, Thomas W},
+ journal={Technometrics},
+ volume={49},
+ number={1},
+ pages={45--55},
+ year={2007},
+ publisher={Taylor \& Francis}
+}
+@article{tang1993orthogonal,
+ title={Orthogonal array-based {L}atin hypercubes},
+ author={Tang, Boxin},
+ journal={Journal of the American statistical association},
+ volume={88},
+ number={424},
+ pages={1392--1397},
+ year={1993},
+ publisher={Taylor \& Francis}
+}
+@article{lin2009construction,
+ title={Construction of orthogonal and nearly orthogonal {L}atin hypercubes},
+ author={Lin, C. Devon and Mukerjee, Rahul and Tang, Boxin},
+ journal={Biometrika},
+ volume={96},
+ number={1},
+ pages={243--247},
+ year={2009},
+ publisher={Oxford University Press}
+}
+@article{sun2010construction,
+ title={Construction of orthogonal {L}atin hypercube designs with flexible run sizes},
+ author={Sun, Fasheng and Liu, Min-Qian and Lin, Dennis K. J.},
+ journal={Journal of Statistical Planning and Inference},
+ volume={140},
+ number={11},
+ pages={3236--3242},
+ year={2010},
+ publisher={Elsevier}
+}
+@article{sun2009construction,
+ title={Construction of orthogonal {L}atin hypercube designs},
+ author={Sun, Fasheng and Liu, Min-Qian and Lin, Dennis K. J.},
+ journal={Biometrika},
+ volume={96},
+ number={4},
+ pages={971--974},
+ year={2009},
+ publisher={Oxford University Press}
+}
+@article{williams1949experimental,
+ title={Experimental designs balanced for the estimation of residual effects of treatments},
+ author={Williams, EJ},
+ journal={Australian Journal of Chemistry},
+ volume={2},
+ number={2},
+ pages={149--168},
+ year={1949},
+ publisher={CSIRO}
+}
+@article{butler2001optimal,
+ title={Optimal and orthogonal {L}atin hypercube designs for computer experiments},
+ author={Butler, Neil A},
+ journal={Biometrika},
+ volume={88},
+ number={3},
+ pages={847--857},
+ year={2001},
+ publisher={Oxford University Press}
+}
+@article{johnson1990minimax,
+ title={Minimax and maximin distance designs},
+ author={Johnson, Mark E and Moore, Leslie M and Ylvisaker, Donald},
+ journal={Journal of statistical planning and inference},
+ volume={26},
+ number={2},
+ pages={131--148},
+ year={1990},
+ publisher={Elsevier}
+}
+@article{morris1995exploratory,
+ title={Exploratory designs for computational experiments},
+ author={Morris, Max D and Mitchell, Toby J},
+ journal={Journal of statistical planning and inference},
+ volume={43},
+ number={3},
+ pages={381--402},
+ year={1995},
+ publisher={Elsevier}
+}
+@article{hickernell1998generalized,
+ title={A generalized discrepancy and quadrature error bound},
+ author={Hickernell, Fred},
+ journal={Mathematics of computation},
+ volume={67},
+ number={221},
+ pages={299--322},
+ year={1998}
+}
+@article{joseph2008orthogonal,
+ title={Orthogonal-maximin {L}atin hypercube designs},
+ author={Joseph, V Roshan and Hung, Ying},
+ journal={Statistica Sinica},
+ pages={171--186},
+ year={2008},
+ publisher={JSTOR}
+}
+@article{jin2005efficient,
+ title={An efficient algorithm for constructing optimal design of computer experiments},
+ author={Jin, Ruichen and Chen, Wei and Sudjianto, Agus},
+ journal={Journal of statistical planning and inference},
+ volume={134},
+ number={1},
+ pages={268--287},
+ year={2005},
+ publisher={Elsevier}
+}
+@article{wang2018optimal,
+ title={Optimal maximin ${L}_{1}$-distance Latin hypercube designs based on good lattice point designs},
+ author={Wang, Lin and Xiao, Qian and Xu, Hongquan},
+ journal={The Annals of Statistics},
+ volume={46},
+ number={6B},
+ pages={3741--3766},
+ year={2018},
+ publisher={Institute of Mathematical Statistics}
+}
+@article{xiao2018construction,
+ title={Construction of maximin distance designs via level permutation and expansion},
+ author={Xiao, Qian and Xu, Hongquan},
+ journal={Statistica Sinica},
+ volume={28},
+ number={3},
+ pages={1395--1414},
+ year={2018},
+ publisher={JSTOR}
+}
+@article{xiao2017construction,
+ title={Construction of maximin distance {L}atin squares and related {L}atin hypercube designs},
+ author={Xiao, Qian and Xu, Hongquan},
+ journal={Biometrika},
+ volume={104},
+ number={2},
+ pages={455--464},
+ year={2017},
+ publisher={Oxford University Press}
+}
+@article{fang2002centered,
+ title={Centered ${L}_{2}$-discrepancy of random sampling and {L}atin hypercube design, and construction of uniform designs},
+ author={Fang, Kai-Tai and Ma, Chang-Xing and Winker, Peter},
+ journal={Mathematics of Computation},
+ volume={71},
+ number={237},
+ pages={275--296},
+ year={2002}
+}
+@inproceedings{korobov1959approximate,
+ title={The approximate computation of multiple integrals},
+ author={Korobov, AN},
+ booktitle={Dokl. Akad. Nauk SSSR},
+ volume={124},
+ pages={1207--1210},
+ year={1959}
+}
+@article{zhou2015space,
+ title={Space-filling properties of good lattice point sets},
+ author={Zhou, Yongdao and Xu, Hongquan},
+ journal={Biometrika},
+ volume={102},
+ number={4},
+ pages={959--966},
+ year={2015},
+ publisher={Oxford University Press}
+}
+@article{leary2003optimal,
+ title={Optimal orthogonal-array-based {L}atin hypercubes},
+ author={Leary, Stephen and Bhaskar, Atul and Keane, Andy},
+ journal={Journal of Applied Statistics},
+ volume={30},
+ number={5},
+ pages={585--598},
+ year={2003},
+ publisher={Taylor \& Francis}
+}
+@article{ba2015optimal,
+ title={Optimal sliced {L}atin hypercube designs},
+ author={Ba, Shan and Myers, William R and Brenneman, William A},
+ journal={Technometrics},
+ volume={57},
+ number={4},
+ pages={479--487},
+ year={2015},
+ publisher={Taylor \& Francis}
+}
+@article{qian2012sliced,
+ title={Sliced {L}atin hypercube designs},
+ author={Qian, Peter ZG},
+ journal={Journal of the American Statistical Association},
+ volume={107},
+ number={497},
+ pages={393--399},
+ year={2012},
+ publisher={Taylor \& Francis Group}
+}
+@article{chen2013optimizing,
+ title={Optimizing {L}atin hypercube designs by particle swarm},
+ author={Chen, Ray-Bing and Hsieh, Dai-Ni and Hung, Ying and Wang, Weichung},
+ journal={Statistics and computing},
+ volume={23},
+ number={5},
+ pages={663--676},
+ year={2013},
+ publisher={Springer}
+}
+@article{liefvendahl2006study,
+ title={A study on algorithms for optimization of {L}atin hypercubes},
+ author={Liefvendahl, Mattias and Stocki, Rafa{\l}},
+ journal={Journal of statistical planning and inference},
+ volume={136},
+ number={9},
+ pages={3231--3247},
+ year={2006},
+ publisher={Elsevier}
+}
+@article{holland1975efficient,
+ title={An efficient genetic algorithm for the traveling salesman problem},
+ author={Holland, JH},
+ journal={European Journal of Operational Research},
+ volume={145},
+ pages={606--617},
+ year={1975}
+}
+@article{joseph2015maximum,
+ title={Maximum projection designs for computer experiments},
+ author={Joseph, V Roshan and Gul, Evren and Ba, Shan},
+ journal={Biometrika},
+ volume={102},
+ number={2},
+ pages={371--380},
+ year={2015},
+ publisher={Oxford University Press}
+}
+@Manual{team2013r,
+ title = {{R}: A Language and Environment for Statistical Computing},
+ author = {{R Core Team}},
+ organization = {R Foundation for Statistical Computing},
+ address = {Vienna, Austria},
+ year = {2013},
+ url = {http://www.R-project.org/}
+}
+@Manual{MaxPro,
+ title = {MaxPro: Maximum Projection Designs},
+ author = {Shan Ba and V. Roshan Joseph},
+ year = {2018},
+ note = {R package version 4.1-2},
+ url = {https://CRAN.R-project.org/package=MaxPro}
+}
+@Manual{LHD,
+ title = {LHD: Latin Hypercube Designs (LHDs)},
+ author = {Hongzhi Wang and Qian Xiao and Abhyuday Mandal},
+ year = {2025},
+ note = {R package version 1.4.1},
+ url = {https://CRAN.R-project.org/package=LHD}
+}
+@article{sacks1989designs,
+ title={Designs for computer experiments},
+ author={Sacks, Jerome and Schiller, Susannah B and Welch, William J},
+ journal={Technometrics},
+ volume={31},
+ number={1},
+ pages={41--47},
+ year={1989},
+ publisher={Taylor \& Francis Group}
+}
+@book{santner2003design,
+ title={The design and analysis of computer experiments},
+ author={Santner, Thomas J and Williams, Brian J and Notz, William and Williams, Brain J},
+ volume={1},
+ year={2003},
+ publisher={Springer}
+}
+@book{fang2005design,
+ title={Design and modeling for computer experiments},
+ author={Fang, Kai-Tai and Li, Runze and Sudjianto, Agus},
+ year={2005},
+ publisher={CRC press}
+}
+@article{mckay1979comparison,
+ title={Comparison of three methods for selecting values of input variables in the analysis of output from a computer code},
+ author={McKay, Michael D and Beckman, Richard J and Conover, William J},
+ journal={Technometrics},
+ volume={21},
+ number={2},
+ pages={239--245},
+ year={1979},
+ publisher={Taylor \& Francis}
+}
+@article{kenny2000algorithmic,
+ title={Algorithmic construction of optimal symmetric {L}atin hypercube designs},
+ author={Ye, K. Qian and Li, William and Sudjianto, Agus},
+ journal={Journal of statistical planning and inference},
+ volume={90},
+ number={1},
+ pages={145--159},
+ year={2000},
+ publisher={Elsevier}
+}
+@article{grosso2009finding,
+ title={Finding maximin {L}atin hypercube designs by iterated local search heuristics},
+ author={Grosso, Andrea and Jamali, ARMJU and Locatelli, Marco},
+ journal={European Journal of Operational Research},
+ volume={197},
+ number={2},
+ pages={541--547},
+ year={2009},
+ publisher={Elsevier}
+}
+@article{viana2010algorithm,
+ title={An algorithm for fast optimal {L}atin hypercube design of experiments},
+ author={Viana, Felipe AC and Venter, Gerhard and Balabanov, Vladimir},
+ journal={International journal for numerical methods in engineering},
+ volume={82},
+ number={2},
+ pages={135--156},
+ year={2010},
+ publisher={Wiley Online Library}
+}
+@article{van2007maximin,
+ title={Maximin {L}atin hypercube designs in two dimensions},
+ author={Van Dam, Edwin R and Husslage, Bart and Den Hertog, Dick and Melissen, Hans},
+ journal={Operations Research},
+ volume={55},
+ number={1},
+ pages={158--169},
+ year={2007},
+ publisher={INFORMS}
+}
+@article{van2009bounds,
+ title={Bounds for maximin {L}atin hypercube designs},
+ author={van Dam, Edwin R and Rennen, Gijs and Husslage, Bart},
+ journal={Operations Research},
+ volume={57},
+ number={3},
+ pages={595--608},
+ year={2009},
+ publisher={INFORMS}
+}
+@article{steinberg2006construction,
+ title={A construction method for orthogonal {L}atin hypercube designs},
+ author={Steinberg, David M and Lin, Dennis K. J.},
+ journal={Biometrika},
+ volume={93},
+ number={2},
+ pages={279--288},
+ year={2006},
+ publisher={Oxford University Press}
+}
+@article{yang2012construction,
+ title={Construction of orthogonal and nearly orthogonal {L}atin hypercube designs from orthogonal designs},
+ author={Yang, Jinyu and Liu, Min-Qian},
+ journal={Statistica Sinica},
+ pages={433--442},
+ year={2012},
+ publisher={JSTOR}
+}
+@article{georgiou2014some,
+ title={Some classes of orthogonal {L}atin hypercube designs},
+ author={Georgiou, Stelios D and Efthimiou, Ifigenia},
+ journal={Statistica Sinica},
+ volume={24},
+ number={1},
+ pages={101--120},
+ year={2014},
+ publisher={JSTOR}
+}
+@article{sun2017general,
+ title={A general rotation method for orthogonal {L}atin hypercubes},
+ author={Sun, Fasheng and Tang, Boxin},
+ journal={Biometrika},
+ volume={104},
+ number={2},
+ pages={465--472},
+ year={2017},
+ publisher={Oxford University Press}
+}
+@article{georgiou2009orthogonal,
+ title={Orthogonal {L}atin hypercube designs from generalized orthogonal designs},
+ author={Georgiou, Stelios D},
+ journal={Journal of Statistical Planning and Inference},
+ volume={139},
+ number={4},
+ pages={1530--1540},
+ year={2009},
+ publisher={Elsevier}
+}
+@inproceedings{kennedy1995particle,
+ title={Particle swarm optimization},
+ author={Kennedy, James and Eberhart, Russell},
+ booktitle={Proceedings of {ICNN}'95-International Conference on Neural Networks},
+ volume={4},
+ pages={1942--1948},
+ year={1995},
+ organization={IEEE}
+}
+@book{holland1992adaptation,
+ title={Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence},
+ author={Holland, John Henry and others},
+ year={1992},
+ publisher={MIT press}
+}
+@article{goldberg1989genetic,
+ title={Genetic algorithms in search},
+ author={Goldberg, David E},
+ journal={Optimization, and MachineLearning},
+ year={1989},
+ publisher={Addison Wesley Publishing Co. Inc.}
+}
+@article{chen2015minimax,
+ title={Minimax optimal designs via particle swarm optimization methods},
+ author={Chen, Ray-Bing and Chang, Shin-Perng and Wang, Weichung and Tung, Heng-Chih and Wong, Weng Kee},
+ journal={Statistics and Computing},
+ volume={25},
+ number={5},
+ pages={975--988},
+ year={2015},
+ publisher={Springer}
+}
+@article{wong2015modified,
+ title={A modified particle swarm optimization technique for finding optimal designs for mixture models},
+ author={Wong, Weng Kee and Chen, Ray-Bing and Huang, Chien-Chih and Wang, Weichung},
+ journal={PLoS One},
+ volume={10},
+ number={6},
+ pages={e0124720},
+ year={2015},
+ publisher={Public Library of Science San Francisco, CA USA}
+}
+@article{kirkpatrick1983optimization,
+ title={Optimization by simulated annealing},
+ author={Kirkpatrick, Scott and Gelatt, C Daniel and Vecchi, Mario P},
+ journal={science},
+ volume={220},
+ number={4598},
+ pages={671--680},
+ year={1983},
+ publisher={American association for the advancement of science}
+}
+@article{loeppky2009choosing,
+ title={Choosing the sample size of a computer experiment: A practical guide},
+ author={Loeppky, Jason L and Sacks, Jerome and Welch, William J},
+ journal={Technometrics},
+ volume={51},
+ number={4},
+ pages={366--376},
+ year={2009},
+ publisher={Taylor \& Francis}
+}
+@article{chapman1994arctic,
+ title={Arctic sea ice variability: Model sensitivities and a multidecadal simulation},
+ author={Chapman, William L and Welch, William J and Bowman, Kenneth P and Sacks, Jerome and Walsh, John E},
+ journal={Journal of Geophysical Research: Oceans},
+ volume={99},
+ number={C1},
+ pages={919--935},
+ year={1994},
+ publisher={Wiley Online Library}
+}
+@article{jones1998efficient,
+ title={Efficient global optimization of expensive black-box functions},
+ author={Jones, Donald R and Schonlau, Matthias and Welch, William J},
+ journal={Journal of Global optimization},
+ volume={13},
+ number={4},
+ pages={455--492},
+ year={1998},
+ publisher={Springer}
+}
+@article{deng2015design,
+ title={Design for computer experiments with qualitative and quantitative factors},
+ author={Deng, Xinwei and Hung, Ying and Lin, C Devon},
+ journal={Statistica Sinica},
+ pages={1567--1581},
+ year={2015},
+ publisher={JSTOR}
+}
+@article{lin2019design,
+ title={Design of order-of-addition experiments},
+ author={Peng, Jiayu and Mukerjee, Rahul and Lin, Dennis K. J.},
+ journal={Biometrika},
+ volume={106},
+ number={3},
+ pages={683--694},
+ year={2019},
+ publisher={Oxford University Press}
+}
+@article{lin1993new,
+ title={A new class of supersaturated designs},
+ author={Lin, Dennis K. J.},
+ journal={Technometrics},
+ volume={35},
+ number={1},
+ pages={28--31},
+ year={1993},
+ publisher={Taylor \& Francis}
+}
+@book{dean1999design,
+ title={Design and analysis of experiments},
+ author={Dean, Angela and Voss, Daniel and Dragulji{\'c}, Danel and others},
+ year={2017},
+ publisher={Springer International Publishing}
+}
+@article{stander1992cooperative,
+ title={Cooperative hunting in lions: the role of the individual},
+ author={Stander, Philip E},
+ journal={Behavioral ecology and sociobiology},
+ volume={29},
+ number={6},
+ pages={445--454},
+ year={1992},
+ publisher={Springer}
+}
+@article{whitacre2011recent,
+ title={Recent trends indicate rapid growth of nature-inspired optimization in academia and industry},
+ author={Whitacre, James M},
+ journal={Computing},
+ volume={93},
+ number={2-4},
+ pages={121--133},
+ year={2011},
+ publisher={Springer}
+}
+@misc{wang2021musings,
+ title={Musings about Constructions of Efficient Latin Hypercube Designs with Flexible Run-sizes},
+ author={Hongzhi Wang and Qian Xiao and Abhyuday Mandal},
+ year={2021},
+ eprint={2010.09154},
+ archivePrefix={arXiv},
+ primaryClass={stat.ME}
+}
+@article{lin2015using,
+ title={Using genetic algorithms to design experiments: a review},
+ author={Lin, C. Devon and Anderson-Cook, Christine M and Hamada, Michael S and Moore, Leslie M and Sitter, Randy R},
+ journal={Quality and Reliability Engineering International},
+ volume={31},
+ number={2},
+ pages={155--167},
+ year={2015},
+ publisher={Wiley Online Library}
+}
+@book{boyd2004convex,
+ title={Convex optimization},
+ author={Boyd, Stephen P and Vandenberghe, Lieven},
+ year={2004},
+ publisher={Cambridge university press}
+}
+@article{byrd1987trust,
+ title={A trust region algorithm for nonlinearly constrained optimization},
+ author={Byrd, Richard H and Schnabel, Robert B and Shultz, Gerald A},
+ journal={SIAM Journal on Numerical Analysis},
+ volume={24},
+ number={5},
+ pages={1152--1170},
+ year={1987},
+ publisher={SIAM}
+}
+@incollection{beni1993swarm,
+ title={Swarm intelligence in cellular robotic systems},
+ author={Beni, Gerardo and Wang, Jing},
+ booktitle={Robots and biological systems: towards a new bionics?},
+ pages={703--712},
+ year={1993},
+ publisher={Springer}
+}
+@article{dorigo2006ant,
+ title={Ant colony optimization},
+ author={Dorigo, Marco and Birattari, Mauro and Stutzle, Thomas},
+ journal={IEEE computational intelligence magazine},
+ volume={1},
+ number={4},
+ pages={28--39},
+ year={2006},
+ publisher={IEEE}
+}
+@incollection{yang2010new,
+ title={A new metaheuristic bat-inspired algorithm},
+ author={Yang, Xin-She},
+ booktitle={Nature inspired cooperative strategies for optimization (NICSO 2010)},
+ pages={65--74},
+ year={2010},
+ publisher={Springer}
+}
+@inproceedings{basturk2006artificial,
+ title={An artificial bee colony ({ABC}) algorithm for numeric function optimization},
+ author={Basturk, Bahriye},
+ booktitle={IEEE Swarm Intelligence Symposium, Indianapolis, IN, {USA}, 2006},
+ year={2006}
+}
+@inproceedings{yang2009cuckoo,
+ title={Cuckoo search via L{\'e}vy flights},
+ author={Yang, Xin-She and Deb, Suash},
+ booktitle={2009 World congress on nature \& biologically inspired computing ({NaBIC})},
+ pages={210--214},
+ year={2009},
+ organization={Ieee}
+}
+@inproceedings{yang2009firefly,
+ title={Firefly algorithms for multimodal optimization},
+ author={Yang, Xin-She},
+ booktitle={International symposium on stochastic algorithms},
+ pages={169--178},
+ year={2009},
+ organization={Springer}
+}
+@article{mirjalili2014grey,
+ title={Grey wolf optimizer},
+ author={Mirjalili, Seyedali and Mirjalili, Seyed Mohammad and Lewis, Andrew},
+ journal={Advances in engineering software},
+ volume={69},
+ pages={46--61},
+ year={2014},
+ publisher={Elsevier}
+}
+@article{garcia2008jumping,
+ title={Jumping frogs optimization: a new swarm method for discrete optimization},
+ author={Garcia, F Javier Martinez and P{\'e}rez, Jos{\'e} A Moreno},
+ journal={Documentos de Trabajo del DEIOC},
+ volume={3},
+ year={2008}
+}
+@book{yang2020nature,
+ title={Nature-inspired optimization algorithms},
+ author={Yang, Xin-She},
+ year={2020},
+ publisher={Academic Press}
+}
+@incollection{kennedy2006swarm,
+ title={Swarm intelligence},
+ author={Kennedy, James},
+ booktitle={Handbook of nature-inspired and innovative computing},
+ pages={187--219},
+ year={2006},
+ publisher={Springer}
+}
+@incollection{lourencco2019iterated,
+ title={Iterated local search: Framework and applications},
+ author={Louren{\c{c}}o, Helena Ramalhinho and Martin, Olivier C and St{\"u}tzle, Thomas},
+ booktitle={Handbook of metaheuristics},
+ pages={129--168},
+ year={2019},
+ publisher={Springer}
+}
+@article{hansen2010variable,
+ title={Variable neighbourhood search: methods and applications},
+ author={Hansen, Pierre and Mladenovi{\'c}, Nenad and P{\'e}rez, Jos{\'e} A Moreno},
+ journal={Annals of Operations Research},
+ volume={175},
+ number={1},
+ pages={367--407},
+ year={2010},
+ publisher={Springer}
+}
+@article{stokes2020using,
+ title={Using Differential Evolution to design optimal experiments},
+ author={Stokes, Zack and Mandal, Abhyuday and Wong, Weng Kee},
+ journal={Chemometrics and Intelligent Laboratory Systems},
+ volume={199},
+ pages={103955},
+ year={2020},
+ publisher={Elsevier}
+}
+@book{antony2014design,
+ title={Design of experiments for engineers and scientists},
+ author={Antony, Jiju},
+ year={2014},
+ publisher={Elsevier}
+}
+@article{storn1997differential,
+ title={Differential evolution--a simple and efficient heuristic for global optimization over continuous spaces},
+ author={Storn, Rainer and Price, Kenneth},
+ journal={Journal of global optimization},
+ volume={11},
+ number={4},
+ pages={341--359},
+ year={1997},
+ publisher={Springer}
+}
+@book{price2006differential,
+ title={Differential evolution: a practical approach to global optimization},
+ author={Price, Kenneth and Storn, Rainer M and Lampinen, Jouni A},
+ year={2006},
+ publisher={Springer Science \& Business Media}
+}
+@article{emich1900explosive,
+ title={{\"U}ber explosive Gasgemenge},
+ author={Emich, F},
+ journal={Monatshefte f{\"u}r Chemie und verwandte Teile anderer Wissenschaften},
+ volume={21},
+ number={10},
+ pages={1061--1078},
+ year={1900},
+ publisher={Springer}
+}
+@article{lin2019order,
+ title={Order-of-addition experiments: A review and some new thoughts},
+ author={Lin, Dennis KJ and Peng, Jiayu},
+ journal={Quality Engineering},
+ volume={31},
+ number={1},
+ pages={49--59},
+ year={2019},
+ publisher={Taylor \& Francis}
+}
+@book{gramacy2020surrogates,
+ title = {Surrogates: {G}aussian Process Modeling, Design and Optimization for the Applied Sciences},
+ author = {Robert B. Gramacy},
+ publisher = {Chapman Hall/CRC},
+ address = {Boca Raton, Florida},
+ note = {\url{http://bobby.gramacy.com/surrogates/}},
+ year = {2020}
+}
+@inproceedings{van1995design,
+ title={Design of experiments where the order of addition is important},
+ author={Van Nostrand, RC},
+ booktitle={{ASA} proceedings of the Section on Physical and Engineering Sciences},
+ pages={155--160},
+ year={1995},
+ organization={American Statistical Association Alexandria, Virginia}
+}
+@article{voelkel2019design,
+ title={The design of order-of-addition experiments},
+ author={Voelkel, Joseph G},
+ journal={Journal of Quality Technology},
+ volume={51},
+ number={3},
+ pages={230--241},
+ year={2019},
+ publisher={Taylor \& Francis}
+}
+@book{fedorov2013theory,
+ title={Theory of optimal experiments},
+ author={Fedorov, Valerii Vadimovich},
+ year={2013},
+ publisher={Elsevier}
+}
+@article{shewry1987maximum,
+ title={Maximum entropy sampling},
+ author={Shewry, Michael C and Wynn, Henry P},
+ journal={Journal of applied statistics},
+ volume={14},
+ number={2},
+ pages={165--170},
+ year={1987},
+ publisher={Taylor \& Francis}
+}
+@article{viana2016tutorial,
+ title={A tutorial on {Latin} hypercube design of experiments},
+ author={Viana, Felipe AC},
+ journal={Quality and reliability engineering international},
+ volume={32},
+ number={5},
+ pages={1975--1985},
+ year={2016},
+ publisher={Wiley Online Library}
+}
+@article{harari2018computer,
+ title={Computer experiments: Prediction accuracy, sample size and model complexity revisited},
+ author={Harari, Ofir and Bingham, Derek and Dean, Angela and Higdon, Dave},
+ journal={Statistica Sinica},
+ pages={899--919},
+ year={2018},
+ publisher={JSTOR}
+}
+@Manual{lhs,
+ title = {lhs: {Latin} Hypercube Samples},
+ author = {Rob Carnell},
+ year = {2024},
+ note = {R package version 1.2.0},
+ url = {https://CRAN.R-project.org/package=lhs}
+}
+@Manual{SLHD,
+ title = {SLHD: Maximin-Distance (Sliced) {Latin} Hypercube Designs},
+ author = {Shan Ba},
+ year = {2015},
+ note = {R package version 2.1-1},
+ url = {https://CRAN.R-project.org/package=SLHD}
+}
+
diff --git a/_articles/RJ-2025-033/wang-xiao-mandal.tex b/_articles/RJ-2025-033/wang-xiao-mandal.tex
new file mode 100644
index 0000000000..35a16f56a6
--- /dev/null
+++ b/_articles/RJ-2025-033/wang-xiao-mandal.tex
@@ -0,0 +1,650 @@
+% !TeX root = RJwrapper.tex
+
+\title{LHD: An All-encompassing R Package for Constructing Optimal Latin Hypercube Designs}
+\author{by Hongzhi Wang, Qian Xiao and Abhyuday Mandal}
+
+\maketitle
+
+\begin{abstract}
+Optimal Latin hypercube designs (LHDs), including maximin distance LHDs, maximum projection LHDs and orthogonal LHDs, are widely used in computer experiments. It is challenging to construct such designs with flexible sizes, especially for large ones, for two main reasons.
+One reason is that theoretical results, such as algebraic constructions ensuring the maximin distance property or orthogonality, are only available for certain design sizes. For design sizes where theoretical results are unavailable, search algorithms can generate designs. However, their numerical performance is not guaranteed to be optimal. Another reason is that when design sizes increase, the number of permutations grows exponentially. Constructing optimal LHDs is a discrete optimization process, and enumeration is nearly impossible for large or moderate design sizes. Various search algorithms and algebraic constructions have been proposed to identify optimal LHDs, each having its own pros and cons. We develop the R package LHD which implements various search algorithms and algebraic constructions. We embedded different optimality criteria into each of the search algorithms, and they are capable of constructing different types of optimal LHDs even though they were originally invented to construct maximin distance LHDs only. Another input argument that controls maximum CPU time is added to each of the search algorithms to let users flexibly allocate their computational resources. We demonstrate functionalities of the package by using various examples, and we provide guidance for experimenters on finding suitable optimal designs. The LHD package is easy to use for practitioners and possibly serves as a benchmark for future developments in LHD.
+\end{abstract}
+
+\section{Introduction}
+
+Computer experiments are widely used in scientific research and industrial production, where
+complex computer codes, commonly high-fidelity simulators, generate data instead of real physical systems \citep{sacks1989designs, fang2005design}. The outputs from computer experiments are deterministic (that is, free of random errors), and therefore replications are not needed \citep{butler2001optimal, joseph2008orthogonal, ba2015optimal}. Latin hypercube designs (LHDs, \cite{mckay1979comparison}) may be the most popular type of experimental designs for computer experiments \citep{fang2005design, xiao2018construction}, which avoid replications on every dimension and have uniform one-dimensional projections. According to practical needs, there are various types of optimal LHDs, including space-filling LHDs, maximum projection LHDs, and orthogonal LHDs. There is a rich literature on the construction of such designs, but it is still very challenging to find good ones for moderate to large design sizes \citep{ye1998orthogonal, fang2005design, joseph2015maximum, xiao2018construction}. One key reason is that theoretical results, such as algebraic constructions which guarantee the maximin distance property or orthogonality, are only established for specific design sizes. These constructions provide theoretical guarantees on the design quality but are limited in their applicability. For design sizes where such theoretical guarantees do not exist, search algorithms can generate designs. However, the performance of search-based designs depends on the algorithm employed, the search space explored, and the computational resources allocated, meaning they cannot be guaranteed to be optimal. Constructing optimal LHDs is a discrete optimization process, where enumerating all possible solutions guarantees the optimal design for a given size. However, this approach becomes computationally infeasible as the number of permutations grows exponentially with increasing design sizes, making it another key reason that adds to the challenge.
+
+An LHD with $n$ runs and $k$ factors is an $n \times k$ matrix with each column being a random permutation of numbers: $1, \ldots, n$. Throughout this paper, $n$ denotes the run size and $k$ denotes the factor size. A space-filling LHD has its sampled region as scattered as possible, minimizing the unsampled region, thus accounting for the uniformity of all dimensions. Different criteria were proposed to measure designs' space-filling properties, including the maximin and minimax distance criteria \citep{johnson1990minimax, morris1995exploratory}, the discrepancy criteria \citep{hickernell1998generalized, fang2002centered, fang2005design} and the entropy criterion \citep{shewry1987maximum}. Since there are as many as $(n!)^{k}$ candidate LHDs for a given design size, it is nearly impossible to find the space-filling one by enumeration when $n$ and $k$ are moderate or large. In the current literature, both the search algorithms \citep{morris1995exploratory, leary2003optimal, joseph2008orthogonal, ba2015optimal, kenny2000algorithmic, jin2005efficient, liefvendahl2006study, grosso2009finding, chen2013optimizing} and algebraic constructions \citep{zhou2015space, xiao2017construction, wang2018optimal} are used to construct space-filling LHDs.
+
+Space-filling designs often focus on the full-dimensional space. To further improve the space-filling properties of all possible sub-spaces, \cite{joseph2015maximum} proposed to use the maximum projection designs. Considering from two to $k-1$ dimensional sub-spaces, maximum projection LHDs (MaxPro LHDs) are generally more space-filling compared to the classic maximin distance LHDs. The construction of MaxPro LHDs is also challenging, especially for large ones, and \cite{joseph2015maximum} proposed a simulated annealing (SA) based algorithm. In the \RR{LHD} package, we incorporated the MaxPro criterion with other different algorithms such as the particle swarm optimization (PSO) and genetic algorithm (GA) framework, leading to many better MaxPro LHDs; see Section 3 for examples.
+
+Unlike space-filling LHDs that minimize the similarities among rows, orthogonal LHDs (OLHDs) are another popular type of optimal design which consider similarities among columns. For example, OLHDs have zero column-wise correlations. Algebraic constructions are available for certain design sizes \citep{ye1998orthogonal, cioppa2007efficient, steinberg2006construction, sun2010construction, sun2009construction, yang2012construction, georgiou2014some, butler2001optimal, tang1993orthogonal, lin2009construction}, but there are many design sizes where theoretical results are not available. In the \RR{LHD} package, we implemented the average absolute correlation criterion and the maximum absolute correlation criterion \citep{georgiou2009orthogonal} with SA, PSO, and GA to identify both OLHDs and nearly orthogonal LHDs (NOLHDs) for almost all design sizes.
+
+This paper introduces the R package \RR{LHD} available on the Comprehensive R Archive Network (\url{https://cran.r-project.org/web/packages/LHD/index.html}), which implements some currently popular search algorithms and algebraic constructions for constructing maximin distance LHDs, Maxpro LHDs, OLHDs and NOLHDs. We embedded different optimality criteria including the maximin distance criterion, the MaxPro criterion, the average absolute correlation criterion, and the maximum absolute correlation criterion in each of the search algorithms which were originally invented to construct maximin distance LHDs only \citep{morris1995exploratory, leary2003optimal, joseph2008orthogonal, liefvendahl2006study, chen2013optimizing}, and each of them is capable of constructing different types of optimal LHDs through the package. To let users flexibly allocate their computational resources, we also embedded an input argument that limits the maximum CPU time for each of the algorithms, where users can easily define how and when they want the algorithms to stop. An algorithm can stop in one of two ways: either when the user-defined maximum number of iterations is reached or when the user-defined maximum CPU time is exceeded. For example, users can either allow the algorithm to run for a specified number of iterations without restricting the maximum CPU time or set a maximum CPU time limit to stop the algorithm regardless of the number of iterations completed. After an algorithm is completed or stopped, the number of iterations completed along with the average CPU time per iteration will be presented to users for their information. The R package \RR{LHD} is an integrated tool for users with little or no background in design theory, and they can easily find optimal LHDs with desired sizes. Many new designs that are better than the existing ones are discovered; see Section 3.
+
+The remainder of the paper is organized as follows. Section 2 illustrates different optimality criteria for LHDs. Section 3 demonstrates some popular search algorithms and their implementation details in the \RR{LHD} package along with examples. Section 4 discusses some useful algebraic constructions as well as examples of how to implement them via the developed package. Section 5 reviews other R packages for Latin hypercubes and provides a comparative discussion. Section 6 concludes with a summary.
+
+\section{Optimality Criteria for LHDs}
+\label{OC}
+
+Various criteria are proposed to measure designs' space-filling properties \citep{johnson1990minimax, hickernell1998generalized, fang2002centered}. In this paper, we focus on the currently popular maximin distance criterion \citep{johnson1990minimax}, which seeks to scatter design points over experimental domains so that the minimum distances between points are maximized. Let $\textbf{X}$ denote an LHD matrix throughout this paper. Define the $L_q$-distance between two runs $x_i$ and $x_j$ of $\textbf{X}$ as $d_q(x_i, x_j) = \left\{ \sum_{m=1}^{k} \vert x_{im}-x_{jm}\vert ^q \right\}^{1/q}$ where $q$ is an integer. Define the $L_q$ distance of the design $\textbf{X}$ as $d_q(\textbf{X}) = \text{min} \{d_q(x_i, x_j), 1 \leq iReferences
+ +Reuse
+Text and figures are licensed under Creative Commons Attribution CC BY 4.0. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from ...".
+Citation
+For attribution, please cite this work as
+Wang, et al., "LHD: An All-encompassing R Package for Constructing Optimal Latin Hypercube Designs", The R Journal, 2026+
BibTeX citation
+@article{RJ-2025-033,
+ author = {Wang, Hongzhi and Xiao, Qian and Mandal, Abhyuday},
+ title = {LHD: An All-encompassing R Package for Constructing Optimal Latin Hypercube Designs},
+ journal = {The R Journal},
+ year = {2026},
+ note = {https://doi.org/10.32614/RJ-2025-033},
+ doi = {10.32614/RJ-2025-033},
+ volume = {17},
+ issue = {4},
+ issn = {2073-4859},
+ pages = {20-36}
+}
+
+
+
+
+
+longevity: An R Package for Modelling Excess Lifetimes
+ + + +The longevity R package provides a maximum likelihood estimation routine for modelling of survival data that are subject to non-informative censoring or truncation. It includes a selection of 12 parametric models of varying complexity, with a focus on tools for extreme value analysis and more specifically univariate peaks over threshold modelling. The package provides utilities for univariate threshold selection, parametric and nonparametric maximum likelihood estimation, goodness of fit diagnostics and model comparison tools. These different methods are illustrated using individual Dutch records and aggregated Japanese human lifetime data.
+
+
+
+
+1 Introduction
+Many data sets collected by demographers for the analysis of human longevity have unusual features and for which limited software implementations exist. The longevity package was initially built for dealing with human longevity records and data from the International Database on Longevity (IDL), which provides age at death of supercentenarians, i.e., people who died above age 110. Data for the statistical analysis of (human) longevity can take the form of aggregated counts per age at death, or most commonly life trajectory of individuals with both birth and death dates. Such lifetimes are often interval truncated (only age at death of individuals dying between two calendar dates are recorded) or left truncated and right censored (when data of individuals still alive at the end of the collection period are also included). Another frequent format is death counts, aggregated per age band. Censoring and truncation are typically of administrative nature and thus non-informative about death.
+Supercentenarians are extremely rare and records are sparse. The most popular parametric models used by practitioners are justified by asymptotic arguments and have their roots in extreme value theory. Univariate extreme value distributions are well implemented in software and Belzile et al. (2023b) provides a recent review of existing implementations. While there are many standard R packages for the analysis of univariate extremes using likelihood-based inference, such as evd (Stephenson 2002), mev and extRemes (Gilleland and Katz 2016), only the evgam package includes functionalities to fit threshold exceedance models with censoring, as showcased in Youngman (2022) with rounded rainfall measurements. Support for survival data for extreme value models is in general wholly lacking, which motivated the development of longevity.
+The longevity package also includes several parametric models commonly used in demography. Many existing packages that focus on tools for modelling mortality rates, typically through life tables, are listed in the CRAN Task View ActuarialScience in the Life insurance section. They do not however allow for truncation or more general survival mechanisms, as the aggregated data used are typically complete except for potential right censoring for the oldest age group. The survival package (Therneau and Grambsch 2000) includes utilities for accelerated failure time models for 10 parametric models. The MortCast package can be used to estimate age-specific mortality rates using the Kannisto and Lee–Carter approaches, among others (Ševčíková et al. 2016). The demography package provides forecasting methods for death rates based on constructed life tables using the Lee–Carter or ARIMA models. MortalityLaws includes utilities to download data from the Human Mortality Database (HMD), and fit a total of 27 parametric models for life table data (death counts and population at risk per age group) using Poisson, binomial or alternative loss functions. The vitality package fits the family of vitality models (Anderson 2000; Li and Anderson 2009) via maximum likelihood based on empirical survival data. The fitdistrplus (Delignette-Muller and Dutang 2015) allows for generic parametric distributions to be fitted to interval censored data via maximum likelihood, with various S3 methods for model assessment. The package also allows user-specified models, thereby permitting custom definitions for truncated distributions whose truncation bounds are passed as fixed vector parameters. Parameter uncertainty can be obtained via additional functions using nonparametric bootstrap. Many parametric distributions also appear in the VGAM package (Yee and Wild 1996; Yee 2015), which allows for vector generalized linear modelling. The longevity package is less general and offers support only for selected parametric distributions, but contrary to the aforementioned packages allows for truncation and general patterns. One strength of longevity is that it also includes model comparison tools that account for non-regular asymptotics and goodness of fit diagnostics.
+Nonparametric methods are popular tools for the analysis of survival data with large samples, owing to their limited set of assumptions. They also serve for the validation of parametric models. Without explanatory variable, a closed-form estimator of the nonparametric maximum likelihood estimator of the survival function can be derived in particular instances, including the product limit estimator (Kaplan and Meier 1958) for the case of random or non-informative right censoring and an extension allowing for left truncation (Tsai et al. 1987). In general, the nonparametric maximum likelihood estimator of the survival function needs to be computed using an expectation-maximization (EM) algorithm (Turnbull 1976). Nonparametric estimators only assign probability mass on observed failure times and intervals and so cannot be used for extrapolation beyond the range of the data, limiting its utility in extreme value analysis.
+The CRAN Task View on Survival Analysis lists various implementations of nonparametric maximum likelihood estimators of the survival or hazard functions: survival implements the Kaplan–Meier and Nelson–Aalen estimators. Many packages focus on the case of interval censoring (Groeneboom and Wellner 1992 3.2), including prodlim; Anderson-Bergman (2017a) reviews the performance of the implementations in icenReg (Anderson-Bergman 2017b) and Icens. The latter uses the incidence matrix as input data. Routines for doubly censored data are provided by dblcens. The interval (Fay and Shaw 2010) implements Turnbull’s EM algorithm for interval censored data. The case of left truncated and right censored data is handled by survival, and tranSurv provides transformation models with the potential to account for truncation dependent on survival times. For interval truncated data, dedicated algorithms that use gradient-based steps (Efron and Petrosian 1999) or inverse probability weighting (Shen 2010) exist and can be more efficient than the EM algorithm of Turnbull (1976). Many of these are implemented in the DTDA package. The longevity package includes a C\(^{++}\) implementation of the corrected Turnbull’s algorithm (Turnbull 1976), returning the nonparametric maximum likelihood estimator for arbitrary censoring and truncation patterns, as opposed to the specific existing aforementioned implementations already available which focus on specific subcases. With a small number of observations, it is also relatively straightforward to maximize the log likelihood for the concave program subject to linear constraints using constrained optimization algorithms; longevity relies for this on Rsolnp, which uses augmented Lagrangian methods and sequential quadratic programming (Ye 1987).
+1.1 Motivating examples
+To showcase the functionality of the package and particularity for the modelling of threshold exceedances, we consider Dutch and Japanese life lengths. The dutch database contains the age at death (in days) of Dutch people who died above age 92 between 1986 and 2015; these data were obtained from Statistics Netherlands and analyzed in Einmahl et al. (2019) and Belzile et al. (2022). Records are interval truncated, as people are included in the database only if they died during the collection period. In addition, there are 226 interval censored and interval truncated records for which only the month and year of birth and death are known, as opposed to exact dates.
The second database we consider is drawn from Maier et al. (2021). The data frame japanese2 consists of counts of Japanese above age 100 by age band and are stratified by both birth cohort and sex. To illustrate the format of the data, counts for female Japanese are reproduced in Table 1. The data were constructed using the extinct cohort method and are interval censored between \(\texttt{age}\) and \(\texttt{age} + 1\) and right truncated at the age reached by the oldest individuals of their birth cohort in 2020. The count variable lists the number of instances in the contingency table, and serves as a weight for likelihood contributions.
+
+
+
+| age | +1874-1878 | +1879-1883 | +1884-1888 | +1889-1893 | +1894-1898 | +1899-1900 | +
|---|---|---|---|---|---|---|
| 100 | +1648 | +2513 | +4413 | +8079 | +16036 | +9858 | +
| 101 | +975 | +1596 | +2921 | +5376 | +11047 | +7091 | +
| 102 | +597 | +987 | +1864 | +3446 | +7487 | +4869 | +
| 103 | +345 | +597 | +1153 | +2230 | +5014 | +3293 | +
| 104 | +191 | +351 | +662 | +1403 | +3242 | +2133 | +
| 105 | +121 | +197 | +381 | +855 | +2084 | +1357 | +
| 106 | +64 | +122 | +210 | +495 | +1284 | +836 | +
| 107 | +34 | +74 | +120 | +274 | +774 | +521 | +
| 108 | +16 | +41 | +66 | +152 | +433 | +297 | +
| 109 | +12 | +30 | +39 | +83 | +252 | +167 | +
| 110 | +6 | +17 | +21 | +49 | +130 | +92 | +
| 111 | +4 | +10 | +15 | +26 | +69 | +47 | +
| 112 | +3 | +3 | +11 | +15 | +29 | +22 | +
| 113 | +2 | +2 | +8 | +5 | +15 | +9 | +
| 114 | +1 | +2 | +4 | +3 | +7 | +2 | +
| 115 | +0 | +1 | +1 | +0 | +3 | +2 | +
| 116 | +0 | +1 | +1 | +0 | +1 | +1 | +
| 117 | +0 | +0 | +0 | +0 | +1 | +1 | +
2 Package functionalities
+The longevity package uses the S3 object oriented system and provides a series of functions with common arguments. The syntax used by the longevity package purposely mimics that of the survival package (Therneau and Grambsch 2000), except that it does not specify models using a formula. Users must provide vectors for the time or age (or bounds in case of intervals) via arguments time and time2, as well as lower and upper truncation bounds (ltrunc and rtrunc) if applicable. The integer vector event is used to indicate the type of event, where following survival 0 indicates right-censoring, 1 observed events, 2 left-censoring and 3 interval censoring. Together, these five vectors characterize the data and the survival mechanisms at play. Depending on the sampling scheme, not all arguments are required or relevant and they need not be of the same length, but are common to most functions. Users can also specify a named list args to pass arguments: as illustrated below, this is convenient to avoid specifying repeatedly the common arguments in each function call. Default values are overridden by elements in args, with the exception of those that are passed by the user directly in the call. Relative to survival, functions have additional arguments ltrunc and rtrunc for left and right truncation limits, as these are also possibly matrices for the case of double interval truncation (Belzile et al. 2022), since both censoring and truncation can be present simultaneously.
We can manipulate the data set to build the time vectors and truncation bounds for the Dutch data. We re-scale observations to years for interpretability and keep only records above age 98 for simplicity. We split the data to handle the observed age at death first: these are treated as observed (uncensored) whenever time and time2 coincide. When exact dates are not available, we compute the range of possible age at which individuals may have died, given their birth and death years and months. The truncation bounds for each individual can be obtained by subtracting from the endpoints of the sampling frame the birth dates, with left and right truncation bounds
+\[\begin{align*}
+\texttt{ltrunc}=\min\{92 \text{ years}, 1986.01.01 - \texttt{bdate}\}, \qquad \texttt{rtrunc} = 2015.12.31- \texttt{bdate}.
+\end{align*}\]
+Table 2 shows a sample of five individuals, two of whom are interval-censored, and the corresponding vectors of arguments along with two covariates, gender (gender) and birth year (byear).
+
+
+
+
+
+
+| time | +time2 | +ltrunc | +rtrunc | +event | +gender | +byear | +
|---|---|---|---|---|---|---|
| 104.67 | +105.74 | +80.00 | +111.00 | +3 | +female | +1905 | +
| 103.50 | +104.58 | +78.00 | +109.00 | +3 | +female | +1907 | +
| 100.28 | +100.28 | +92.01 | +104.50 | +1 | +female | +1911 | +
| 100.46 | +100.46 | +92.01 | +102.00 | +1 | +male | +1913 | +
| 100.51 | +100.51 | +92.01 | +104.35 | +1 | +female | +1911 | +
We can proceed similarly for the Japanese data. Ages of centenarians are rounded down to the nearest year, so all observations are interval censored within one-year intervals. Assuming that the ages at death are independent and identically distributed with distribution function \(F(\cdot; \boldsymbol{\theta})\), the log likelihood for exceedances \(y_i = \texttt{age}_i - u\) above age \(u\) is +\[\begin{align*} +\ell(\boldsymbol{\theta}) = \sum_{i: \texttt{age}_i > u}n_i \left[\log \{F(y_i+1; \boldsymbol{\theta}) - F(y_i; \boldsymbol{\theta})\} - \log F(r_i - u; \boldsymbol{\theta})\right] +\end{align*}\] +where \(n_i\) is the count of the number of individuals in cell \(i\) and \(r_i > \texttt{age}_i+1\) is the right truncation limit for that cell, i.e., the maximum age that could have been achieved for that birth cohort by the end of the data collection period.
+
+
+
+
+data(japanese2, package = "longevity")
+# Keep only non-empty cells
+japanese2 <- japanese2[japanese2$count > 0, ]
+# Define arguments that are recycled
+japanese2$rtrunc <- 2020 -
+ as.integer(substr(japanese2$bcohort, 1, 4))
+# The line above extracts the earliest year of the birth cohort
+# Create a list with all arguments common to package functions
+args_japan <- with(japanese2,
+ list(
+ time = age, # lower censoring bound
+ time2 = age + 1L, # upper censoring bound
+ event = 3, # define interval censoring
+ type = "interval2",
+ rtrunc = rtrunc, # right truncation limit
+ weights = count)) # counts as weights2.1 Parametric models and maximum likelihood estimation
+Various models are implemented in longevity: their hazard functions are reported in Table 3. Two of those models, labelled perks and beard are logistic-type hazard functions proposed in Perks (1932) that have been used by Beard (1963), and popularized in work of Kannisto and Thatcher; we use the parametrization of Richards (2012), from which we also adopt the nomenclature. Users can compare the models with those available in MortalityLaws; see ?availableLaws for the list of hazard functions and parametrizations.
+
+
+
+
+
+
+| +model + | ++hazard function + | ++constraints + | +
|---|---|---|
+exp
+ |
++\(\sigma^{-1}\) + | ++\(\sigma > 0\) + | +
+gomp
+ |
++\(\sigma^{-1}\exp(\beta t/\sigma)\) + | ++\(\sigma > 0, \beta \ge 0\) + | +
+gp
+ |
++\((\sigma + \xi t)_{+}^{-1}\) + | ++\(\sigma > 0, \xi \in \mathbb{R}\) + | +
+weibull
+ |
++\(\sigma^{-\alpha} \alpha t^{\alpha-1}\) + | ++\(\sigma > 0, \alpha > 0\) + | +
+extgp
+ |
++\(\beta\sigma^{-1}\exp(\beta t/\sigma)[\beta+\xi\{\exp(\beta t/\sigma) -1\}]^{-1}\) + | ++\(\sigma > 0, \beta \ge 0, \xi \in \mathbb{R}\) + | +
+extweibull
+ |
++\(\alpha\sigma^{-\alpha}t^{\alpha-1}\{1+\xi(t/\sigma)^{\alpha}\}_{+}\) + | ++\(\sigma > 0, \alpha > 0, \xi \in \mathbb{R}\) + | +
+perks
+ |
++\(\alpha\exp(\nu x)/\{1+\alpha\exp(\nu x)\}\) + | ++\(\nu \ge 0, \alpha >0\) + | +
+beard
+ |
++\(\alpha\exp(\nu x)/\{1+\alpha\beta\exp(\nu x)\}\) + | ++\(\nu \ge 0, \alpha >0, \beta \ge 0\) + | +
+gompmake
+ |
++\(\lambda + \sigma^{-1}\exp(\beta t/\sigma)\) + | ++\(\lambda \ge 0, \sigma > 0, \beta \ge 0\) + | +
+perksmake
+ |
++\(\lambda + \alpha\exp(\nu x)/\{1+\alpha\exp(\nu x)\}\) + | ++\(\lambda \ge 0, \nu \ge 0, \alpha > 0\) + | +
+beardmake
+ |
++\(\lambda + \alpha\exp(\nu x)/\{1+\alpha\beta\exp(\nu x)\}\) + | ++\(\lambda \ge 0, \nu \ge 0, \alpha > 0, \beta \ge 0\) + | +
Many of the models are nested and Figure 1 shows the logical relation between the various families. The function fit_elife allows users to fit all of the parametric models of Table 3: the print method returns a summary of the sampling mechanism, the number of observations, the maximum log likelihood and parameter estimates with standard errors. Depending on the data, some models may be overparametrized and parameters need not be numerically identifiable. To palliate such issues, the optimization routine, which uses Rsolnp, can try multiple starting values or fit various sub-models to ensure that the parameter values returned are indeed the maximum likelihood estimates. If one tries to compare nested models and the fit of the simpler model is better than the alternative, the anova function will return an error message.
The fit_elife function handles arbitrary censoring patterns over single intervals, along with single interval truncation and interval censoring. To accommodate the sampling scheme of the International Database on Longevity (IDL), an option also allows for double interval truncation (Belzile et al. 2022), whereby observations are included only if the person dies between time intervals, potentially overlapping, which defines the observation window over which dead individuals are recorded.
+
+
+
+thresh <- 108
+model0 <- fit_elife(arguments = args_japan,
+ thresh = thresh,
+ family = "exp")
+(model1 <- fit_elife(arguments = args_japan,
+ thresh = thresh,
+ family = "gomp"))#> Model: Gompertz distribution.
+#> Sampling: interval censored, right truncated
+#> Log-likelihood: -3599.037
+#>
+#> Threshold: 108
+#> Number of exceedances: 2489
+#>
+#> Estimates
+#> scale shape
+#> 1.6855 0.0991
+#>
+#> Standard Errors
+#> scale shape
+#> 0.0523 0.0273
+#>
+#> Optimization Information
+#> Convergence: TRUE2.2 Model comparisons
+Goodness of fit of nested models can be compared using likelihood ratio tests via the anova method. Most of the interrelations between models yield non-regular model comparisons since, to recover the simpler model, one must often fix parameters to values that lie on the boundary of the parameter space. For example, if we compare a Gompertz model with the exponential, the limiting null distribution is a mixture of a point mass at zero and a \(\chi^2_1\) variable, both with probability half (Chernoff 1954). Many authors (e.g., Camarda 2022) fail to recognize this fact. The case becomes more complicated with more than one boundary constraint: for example, the deviance statistic comparing the Beard–Makeham and the Gompertz model, which constrains two parameters on the boundary of the parameter space, has a null distribution which is a mixture of \(\chi^2_2/4 + \chi^2_1/2 + \chi^2_0/4\) (Self and Liang 1987).
Nonidentifiability impacts testing: for example, if the rate parameter of the Perks–Makeham model (perksmake) \(\nu \to 0\), the limiting hazard, \(\lambda + \exp(\alpha)/\{1+\exp(\alpha)\}\), is constant (exponential model), but neither \(\alpha\) nor \(\lambda\) is identifiable. The usual asymptotics for the likelihood ratio test break down as the information matrix is singular (Rotnitzky et al. 2000). As such, all three families that include a Makeham component cannot be directly compared to the exponential in longevity and the call to anova returns an error message.
Users can also access information criteria, AIC and BIC. The correction factors implemented depend on the number of parameters of the distribution, but do not account for singular fit, non-identifiable parameters or singular models for which the usual corrections \(2p\) and \(\ln(n)p\) are inadequate (Watanabe 2010).
+
+
+
+
+
++Figure 1: Relationship between parametric models showing nested relations. Dashed arrows represent restrictions that lead to nonregular asymptotic null distribution for comparison of nested models. Comparisons between models with Makeham components and exponential are not permitted by the software because of nonidentifiability issues. +
+To showcase how hypothesis testing is performed, we consider a simple example with two nested models. We test whether the exponential model is an adequate simplification of the Gompertz model for exceedances above 108 years — an irregular testing problem since \(\beta=0\) is a restriction on the boundary of the parameter space. The drop in log likelihood is quite large, indicating the exponential model is not an adequate simplification of the Gompertz fit. This is also what is suggested by the Bayesian information criterion, which is much lower for the Gompertz model than for the exponential.
+
+
+
+
+
+
+
+
+| + | npar | +Deviance | +Df | +Chisq | +Pr(>Chisq) | +
|---|---|---|---|---|---|
| gomp | +2 | +7198.07 | ++ | + | + |
| exp | +1 | +7213.25 | +1 | +15.17 | +0 | +
#> exponential Gompertz
+#> 7221.065 7213.7132.3 Simulation-based inference
+Given the poor finite sample properties of the aforementioned tests, it may be preferable to rely on a parametric bootstrap rather than on the asymptotic distribution of the test statistic (Belzile et al. 2022) for model comparison. +Simulation-based inference requires capabilities for drawing new data sets whose features match those of the original one. For example, the International Database on Longevity (IDL) (Jdanov et al. 2021) features data that are interval truncated above 110 years, but doubly interval truncated since the sampling period for semisupercentenarians (who died age 105 to 110) and supercentenarians (who died above 110) are not always the same (Belzile et al. 2022). The 2018 Istat semisupercentenarians database analyzed by Barbi et al. (2018) on Italians includes left truncated right censored records.
+To mimic the postulated data generating mechanism while accounting for the sampling scheme, we could use the observed birth dates, or simulate new birth dates (possibly through a kernel estimator of the empirical distribution of birth dates) while keeping the sampling frame with the first and last date of data collection to define the truncation interval. In other settings, one could obtain the nonparametric maximum likelihood estimator of the distribution of the upper truncation bound (Shen 2010) using an inverse probability weighted estimator, which for fixed data collection windows is equivalent to setting the birth date.
+The samp_elife function includes multiple type2 arguments to handle these. For interval truncated data (type2="ltrt"), it uses the inversion method (Section 2 of Devroye (1986)): for \(F\) an absolutely continuous distribution function and \(F^{-1}\) the corresponding quantile function, a random variable distributed according to \(F\) truncated on \([a,b]\) is generated as \(X \sim F^{-1}[F(a) + U\{F(b)-F(a)\}]\)
+where \(U \sim \mathsf{U}(0,1)\) is standard uniform.
The function samp_elife also has an argument upper which serves for both right truncation, and right censoring. For the latter, any record simulated that exceeds upper is capped at that upper bound and declared partially observed. This is useful for simulating administrative censoring, whereby the birth date and the upper bound of the collection window fully determine whether an observation is right censored or not. An illustrative example is provided in the next section.
+
+
+The anova method call uses the asymptotic null distribution for comparison of nested parametric distributions \(\mathcal{F}_0 \subseteq \mathcal{F}_1\). We could use the bootstrap to see how good this approximation to the null distribution is. To mimic as closely as possible the data generating mechanism, which is custom in most scenarios, we condition on the sampling frame and the number of individuals in each birth cohort. The number dying at each age is random, but the right truncation limits will be the same for anyone in that cohort. We simulate excess lifetimes, then interval censor observations by keeping only the corresponding age bracket. Under the null hypothesis, the data are drawn from \(\widehat{F}_0 \in \mathcal{F}_0\) and we generate observations from this right truncated distribution using the samp_elife utility, which also supports double interval truncation and left truncation right censoring. This must be done within a for loop since we have count attached to each upper bound, but the function is vectorized should we use a single vector containing all of the right truncation limits.
The bootstrap \(p\)-value for comparing models \(M_0 \subset M_1\) would be obtained by repeating the following steps \(B\) times and calculating the rank of the observed test statistic among alternatives:
+-
+
- Simulate new birth dates \(d_i\) \((i=1, \ldots, n)\) (e.g., drawing from a smoothed empirical distribution of birth dates); the latest possible birth date is one which ensures the person reached at least the threshold by the end of the period. +
- Subtract the endpoints of the sampling period, say \(c_1\) and \(c_2\) to get the minimum (maximum) age at death, \(c_1 - d_i\) (respectively \(c_2 - d_i\)) days, which define the truncation bounds. +
- Use the function
samp_elifeto simulate new observations from a parametric interval truncated distribution from the null model \(M_0\)
+ - Use the optimization procedure in
fit_elifeto fit the model with both \(M_1\) and \(M_2\) and calculate the deviance, and from them the likelihood ratio statistic.
+
The algorithm is implemented below for comparing the Gompertz and the exponential model. Since the procedure is computationally intensive, users must trade off between the precision of the bootstrap \(p\)-value estimate and the number of replications, \(B\).
+
+
+
+
+set.seed(2022)
+# Count of unique right truncation limit
+db_rtrunc <- aggregate(count ~ rtrunc,
+ FUN = "sum",
+ data = japanese2,
+ subset = age >= thresh)
+B <- 1000L # Number of bootstrap replications
+boot_anova <- numeric(length = B)
+boot_gof <- numeric(length = B)
+for(b in seq_len(B - 1L)){
+ boot_samp <- # Generate bootstrap sample
+ do.call(rbind, #merge data frames
+ apply(db_rtrunc, 1, function(x){ # for each rtrunc and count
+ count <- table( #tabulate count
+ floor( #round down
+ samp_elife( # sample right truncated exponential
+ n = x["count"],
+ scale = model0$par,
+ family = "exp", #null model
+ upper = x["rtrunc"] - thresh,
+ type2 = "ltrt")))
+ data.frame( # return data frame
+ count = as.integer(count),
+ rtrunc = as.numeric(x["rtrunc"]) - thresh,
+ eage = as.integer(names(count)))
+ }))
+ boot_mod0 <- # Fit null model to bootstrap sample
+ with(boot_samp,
+ fit_elife(time = eage,
+ time2 = eage + 1L,
+ rtrunc = rtrunc,
+ type = "interval",
+ event = 3,
+ family = "exp",
+ weights = count))
+ boot_mod1 <- # Fit alternative model to bootstrap sample
+ with(boot_samp,
+ fit_elife(time = eage,
+ time2 = eage + 1L,
+ rtrunc = rtrunc,
+ type = "interval",
+ event = 3,
+ family = "gomp",
+ weights = count))
+ boot_anova[b] <- deviance(boot_mod0) -
+ deviance(boot_mod1)
+}
+# Add original statistic
+boot_anova[B] <- deviance(model1) - deviance(model0)
+# Bootstrap p-value
+(pval <- rank(boot_anova)[B] / B)#> [1] 0.001The asymptotic approximation is of similar magnitude as the bootstrap \(p\)-value. Both suggest that the more complex Gompertz model provides a significantly better fit.
+2.4 Extreme value analysis
+Extreme value theory suggests that, in many instances, the limiting conditional distribution of exceedances of a random variable \(Y\) with distribution function \(F\) is generalized Pareto, meaning +\[\begin{align} +\lim_{u \to x^*}\Pr(Y-u > y \mid Y > u)= \begin{cases} +\left(1+\xi y/\sigma\right)_{+}^{-1/\xi}, & \xi \neq 0;\\ +\exp(-y/\sigma), & \xi = 0; +\end{cases} +\tag{1} +\end{align}\] +with \(x_{+} = \max\{x, 0\}\) and \(x^*=\sup\{x: F(x) < 1\}\). This justifies the use of Equation (1) for the survival function of threshold exceedances when dealing with rare events. The model has two parameters: a scale \(\sigma\) and a shape \(\xi\) which determines the behavior of the upper tail. Negative shape parameters correspond to bounded upper tails and a finite right endpoint for the support.
+Study of population dynamics and mortality generally requires knowledge of the total population from which observations are drawn to derive rates. By contrast, the peaks over threshold method, by which one models the \(k\) largest observations of a sample, is a conditional analysis (e.g., given survival until a certain age), and is therefore free of denominator specification since we only model exceedances above a high threshold \(u\). For modelling purposes, we need to pick a threshold \(u\) that is smaller than the upper endpoint \(x^*\) in order to have sufficient number of observations to estimate parameters. The threshold selection problem is a classical instance of bias-variance trade-off: the parameter estimators are possibly biased if the threshold is too low because the generalized Pareto approximation is not good enough, whereas choosing a larger threshold to ensure we are closer to the asymptotic regime leads to reduced sample size and increased parameter uncertainty.
+To aid threshold selection, users commonly resort to parameter stability plots. These are common visual diagnostics consisting of a plot of estimates of the shape parameter \(\widehat{\xi}\) (with confidence or credible intervals) based on sample exceedances over a range of thresholds \(u_1, \ldots, u_K\). If the data were drawn from a generalized Pareto distribution, the conditional distribution above higher threshold \(v > u\) is also generalized Pareto with the same shape: this threshold stability property is the basis for extrapolation beyond the range of observed records. Indeed, if the change in estimation of \(\xi\) is nearly constant, this provides reassurance that the approximation can be used for extrapolation. The only difference with survival data, relative to the classical setting, is that the likelihood must account for censoring and truncation. Note that, when we use threshold exceedances with a nonzero threshold (argument thresh), it however isn’t possible to unambiguously determine whether left censored observations are still exceedances: such cases yield errors in the functions.
Theory on penultimate extremes suggests that, for finite levels and general distribution function \(F\) for which (1) holds, the shape parameter varies as a function of the threshold \(u\), behaving like the derivative of the reciprocal hazard \(r(x) = \{1-F(x)\}/f(x)\). We can thus model the shape as piece-wise constant by fitting a piece-wise generalized Pareto model due to Northrop and Coleman (2014) and adapted in Belzile et al. (2022) for survival data. The latter can be viewed as a mixture of generalized Pareto over \(K\) disjoint intervals with continuity constraints to ensure a smooth hazard, which reduces to the generalized Pareto if we force the \(K+1\) shape parameters to be equal. We can use a likelihood ratio test to compare the model, or a score test if the latter is too computationally intensive, and plot the \(p\)-values for each of the \(K\) thresholds, corresponding to the null hypotheses \(\mathrm{H}_k: \xi_k = \cdots = \xi_{K}\) (\(k=1, \ldots, K-1\)). As the model quickly becomes overparametrized, optimization is difficult and the score test may be a safer option as it only requires estimation of the null model of a single generalized Pareto over the whole range.
+To illustrate these diagnostic tools, Figure 2 shows a threshold stability plot, which features a small increase in the shape parameters as the threshold increases, corresponding to a stabilization or even a slight decrease of the hazard at higher ages. We can envision a threshold of 108 years as being reasonable: the Northrop–Coleman diagnostic plot suggests lower thresholds are compatible with a constant shape above 100. Additional goodness-of-fit diagnostics are necessary to determine if the generalized Pareto model fits well.
+
+
+
+
+par(mfrow = c(1, 2), mar = c(4, 4, 1, 1))
+# Threshold sequence
+u <- 100:110
+# Threshold stability plot
+tstab(arguments = args_japan,
+ family = "gp",
+ method = "profile",
+ which.plot = "shape",
+ thresh = u)
+# Northrop-Coleman diagnostic based on score tests
+nu <- length(u) - 1L
+nc_score <- nc_test(arguments = c(args_japan, list(thresh = u)))
+score_plot <- plot(nc_score)
+graphics.off()
+
+
+
++Figure 2: Threshold diagnostic tools: parameter stability plots for the generalized Pareto model (left) and Northrop–Coleman \(p\)-value path (right) for the Japanese centenarian dataset. Both suggest that a threshold as low as 100 may be suitable for peaks-over-threshold analysis. +
+Each plot in the package can be produced using base R or using ggplot2 (Wickham 2016), which implements the grammar of graphics. To keep the list of package dependencies lean and adhere to the tinyverse principle, the latter can be obtained by using the argument plot.type with the generic S3 method plot, or via autoplot, provided the ggplot2 package is already installed.
2.5 Graphical goodness of fit diagnostics
+Determining whether a parametric model fits well to survival data is no easy task due to the difficulty in specifying the null distribution of many goodness of fit statistics, such as the Cramér–von Mises statistic, which differ for survival data. As such, the longevity package relies mostly on visual diagnostic tools. Waller and Turnbull (1992) discusses how classical visual graphics can be adapted in the presence of censoring. Most notably, only observed failure times are displayed on the \(y\)-axis against their empirical plotting position on the \(x\)-axis. Contrary to the independent and identically distributed case, the uniform plotting positions \(F_n(y_i)\) are based on the nonparametric maximum likelihood estimator discussed in Section 3.
+The situation is more complicated with truncated data (Belzile et al. 2022), since the data are not identically distributed: indeed, the distribution function of observation \(Y_i\) truncated on the interval \([a_i, b_i]\) is \(F_i(y_i) = \{F(y_i)-F(a_i)\}/\{F(b_i) - F(a_i)\}\), so the data arise from different distributions even if these share common parameters. One way out of this conundrum is using the probability integral transform and the quantile transform to map observations to the uniform scale and back onto the data scale. Taking \(\widetilde{F}(y_i) = F_n(y_i)=\mathrm{rank}(y_i)/(n+1)\) to denote the empirical distribution function estimator, a probability-probability plot would show \(x_i = \widetilde{F}_{i}(y_i)\) against \(y_i = F_i(y_i)\), leading to approximately uniform samples if the parametric distribution \(F\) is suitable. Another option is to standardize the observation, taking the collection \(\widetilde{y}_i=F^{-1}\{F_i(y_i)\}\) of rescaled exceedances and comparing them to the usual plotting positions \(x_{(i)} =\{i/(n+1)\}\). The drawback of the latter approach is that the quantities displayed on the \(y\)-axis are not raw observations and the ranking of the empirical quantiles may change, a somewhat counter-intuitive feature. However, this means that the sample \(\{F_i(y_i)\}\) should be uniform under the null hypothesis, and this allows one to use methods from Säilynoja et al. (2022) to obtain point-wise and simultaneous confidence intervals.
+longevity offers users the choice between three types of quantile-quantile plots: regular (Q-Q, "qq"), Tukey’s mean detrended Q-Q plots ("tmd") and exponential Q-Q plots ("exp"). Other options on uniform scale are probability-probability (P-P, "pp") plots and empirically rescaled plots (ERP, "erp") (Waller and Turnbull 1992), designed to ease interpretation with censored observations by rescaling axes. We illustrate the graphical tools with the dutch data above age 105 in Figure 3. The fit is correct above 110, but there is a notable dip due to excess mortality around 109.
+
+
+
+fit_dutch <- fit_elife(
+ arguments = dutch_data,
+ event = 3,
+ type = "interval2",
+ family = "gp",
+ thresh = 105,
+ export = TRUE)
+par(mfrow = c(1, 2))
+plot(fit_dutch,
+ which.plot = c("pp","qq"))
+
+
+
++Figure 3: Probability-probability and quantile-quantile plots for generalized Pareto model fitted above age 105 years to Dutch data. The plots indicate broadly good agreement with the observation, except for some individuals who died age 109 for which too many have deaths close to their birthdates. +
+
+
+
+
+
+
+Censored observations are used to compute the plotting positions, but are not displayed. As such, we cannot use graphical goodness of fit diagnostics for the Japanese interval censored data. An alternative, given that data are tabulated in a contingency table, is to use a chi-squared test for independence, conditioning on the number of individuals per birth cohort. The expected number in each cell (birth cohort and age band) can be obtained by computing the conditional probability of falling in that age band. The asymptotic null distribution should be \(\chi^2\) with \((k-1)(p-1)\) degrees of freedom, where \(k\) is the number of age bands and \(p\) the number of birth cohorts. In finite samples, the expected count for large excess lifetimes are very low so one can expect the \(\chi^2\) approximation to be poor. To mitigate this, we can pool observations and resort to simulation to approximate the null distribution of the test statistic. The bootstrap \(p\)-value for the exponential model above 108 years, pooling observations with excess lifetime of 5 years and above, is 0.872, indicating no evidence that the model is inadequate, but the test here may have low power.
+2.6 Stratification
+Demographers may suspect differences between individuals of different sex, from different countries or geographic areas, or by birth cohort. All of these are instances of categorical covariates. One possibility is to incorporate these covariates with suitable link function through parameters, but we consider instead stratification. We can split the data by levels of covariate (with factors) into sub-data and compare the goodness of fit of the \(K\) models relative to that which pools all observations. The test_elife function performs likelihood ratio tests for the comparisons. The test statistic is \(-2\{\ell(\widehat{\boldsymbol{\theta}}_0) - \ell(\widehat{\boldsymbol{\theta}})\}\), where \(\widehat{\boldsymbol{\theta}}_0\) is the maximum likelihood estimator under the null model with common parameters, and \(\widehat{\boldsymbol{\theta}}\) is the unrestricted maximum likelihood estimator for the alternative model with the same distribution, but which allows for stratum-specific parameters. We illustrate this with a generalized Pareto model for the excess lifetime. The null hypothesis is \(\mathrm{H}_0: \sigma_{\texttt{f}} = \sigma_{\texttt{m}}, \xi_{\texttt{f}}=\xi_{\texttt{m}}\) against the alternative that at least one equality doesn’t hold and so the hazard and endpoints are different.
+
+
+
+print(
+ test_elife(
+ arguments = args_japan,
+ thresh = 110,
+ family = "gp",
+ covariate = japanese2$gender)
+)#> Model: generalized Pareto distribution.
+#> Threshold: 110
+#> Number of exceedances per covariate level:
+#> female male
+#> 642 61
+#>
+#> Likelihood ratio statistic: 0.364
+#> Null distribution: chi-square (2)
+#> Asymptotic p-value: 0.833In the present example, there is no evidence against any difference in lifetime distribution between male and female; this is unsurprising given the large imbalance between counts for each covariate level, with far fewer males than females.
+2.7 Extrapolation
+If the maximum likelihood estimator of the shape \(\xi\) for the generalized Pareto model is negative, then the distribution has a finite upper endpoint; otherwise, the latter is infinite. With \(\xi < 0\), we can look at the profile log likelihood for the endpoint \(\eta = -\sigma/\xi\), using the function prof_gp_endpt, to draw the curve and obtain confidence intervals. The argument psi is used to give a grid of values over which to compute the profile log likelihood. The bounds of the \((1-\alpha)\) confidence intervals are obtained by fitting a cubic smoothing spline for \(y=\eta\) as a function of the shifted profile curve \(x = 2\{\ell_p(\eta)-\ell_p(\widehat{\eta})\}\) on both sides of the maximum likelihood estimator and predicting the value of \(y\) when \(x = -\chi^2_1(1-\alpha)\). This technique works well unless the profile is nearly flat or the bounds lie beyond the range of values of psi provided; the user may wish to change them if they are too far. If \(\widehat{\xi} \approx 0\), then the upper bound of the confidence interval may be infinite and the profile log likelihood may never reach the cutoff value of the asymptotic \(\chi^2_1\) distribution.
The profile log likelihood curve for the endpoint, shifted vertically so that its value is zero at the maximum likelihood estimator, highlights the marked asymmetry of the distribution of \(\eta\), shown in 4, with the horizontal dashed lines showing the limits for the 95% profile likelihood confidence intervals. These suggest that the endpoint, or a potential finite lifespan, could lie very much beyond observed records. The routine used to calculate the upper bound computes the cutoff value by fitting a smoothing spline with the role of the \(y\) and \(x\) axes reversed and by predicting the value of \(\eta\) at \(y=0\). In this example, the upper confidence limit is extrapolated from the model: more accurate measures can be obtained by specifying a longer and finer sequence of values of psi such that the profile log likelihood drops below the \(\chi^2_1\) quantile cutoff.
+
+
+
+# Create grid of threshold values
+thresholds <- 105:110
+# Grid of values at which to evaluate profile
+psi <- seq(120, 200, length.out = 101)
+# Calculate the profile for the endpoint
+# of the generalized Pareto at each threshold
+endpt_tstab <- do.call(
+ endpoint.tstab,
+ args = c(
+ args_japan,
+ list(psi = psi,
+ thresh = thresholds,
+ plot = FALSE)))
+# Compute corresponding confidence intervals
+profile <- endpoint.profile(
+ arguments = c(args_japan, list(thresh = 110, psi = psi)))
+# Plot point estimates and confidence intervals
+g1 <- autoplot(endpt_tstab, plot = FALSE, ylab = "lifespan (in years)")
+# Plot the profile curve with cutoffs for conf. int. for 110
+g2 <- autoplot(profile, plot = FALSE)
+patchwork::wrap_plots(g1, g2)
+
+
+
++Figure 4: Maximum likelihood estimates with 95% confidence intervals as a function of threshold (left) and profile likelihood for exceedances above 110 years (right) for Japanese centenarian data. As the threshold increases, the number of exceedances decreases and the intervals for the upper bound become wider. At 110, the right endpoint of the interval would go until infinity. +
+Depending on the model, the conclusions about the risk of mortality change drastically: the Gompertz model implies an ever increasing hazard, but no finite endpoint for the distribution of exceedances. The exponential model implies a constant hazard and no endpoint. By contrast, the generalized Pareto can accommodate both finite and infinite endpoints. The marked asymmetry of the distribution of lifespan defined by the generalized Pareto shows that inference obtained using symmetric confidence intervals (i.e., Wald-based) is likely very misleading: the drop in fit from having a zero or positive shape parameter \(\xi\) is seemingly smaller than the cutoffs for a 95% confidence interval, suggesting that while the best point estimate is around 128 years, the upper bound is so large (and extrapolated) that everything is possible. The model however also suggests a very high probability of dying in any given year, regardless of whether the hazard is constant, decreasing or increasing.
+2.8 Hazard
+The parameters of the models are seldom of interest in themselves: rather, we may be interested in a summary such as the hazard function. At present, longevity does not allow general linear modelling of model parameters or time-varying covariates, but other software implementations can tackle this task. For example, casebase (Bhatnagar et al. 2022) fits flexible hazard models using logistic or multinomial regression with potential inclusion of penalties for the parameters associated to covariates and splines effects. Another alternative is flexsurv (Jackson 2016), which offers 10 parametric models and allows for user-specified models. +The bshazard package (Rebora et al. 2014) provides nonparametric smoothing via \(B\)-splines, whereas muhaz handles kernel-based hazard for right censored data; both could be used for validation of the parametric models in the case of right censoring. The rstpm2 package (Liu et al. 2018) handles generalized modelling for censoring with the Royston–Parmar model built from natural cubic splines (Royston and Parmar 2002). Contrasting with all of the aforementioned approaches, we focus on parametric models: this is partly because there are few observations for the user case we consider and covariates, except perhaps for gender and birth year, are not available.
+The hazard changes over time; the only notable exception is the exponential hazard, which is constant. longevity includes utilities for computing the hazard function from a fitted model object and computing point-wise confidence intervals using symmetric Wald intervals or the profile likelihood.
+Specifically, the hazard_elife function calculates the hazard \(h(t; \boldsymbol{\theta})\) point-wise at times \(t=\)x; Wald-based confidence intervals are obtained using the delta-method, whereas profile likelihood intervals are obtained by reparametrizing the model in terms of \(h(t)\) for each time \(t\). More naturally perhaps, we can consider a Bayesian analysis of the Japanese excess lifetime above 108 years. Using the likelihood and encoded log posterior provided in logpost_elife, we obtained independent samples from the posterior of the generalized Pareto parameters \((\sigma, \xi)\) with maximal data information prior using the rust package. Each parameter combination was then fed into helife and the hazard evaluated over a range of values. Figure 5 shows the posterior samples and functional boxplots (Sun and Genton 2011) of the hazard curves, obtained using the fda package. The risk of dying increases with age, but comes with substantial uncertainty, as evidenced by the increasing width of the boxes and interquartile range.
+
+
+
+
+
++Figure 5: Left: scatterplot of 1000 independent posterior samples from generalized Pareto model with maximum data information prior; the contour curves give the percentiles of credible intervals, and show approximate normality of the posterior. Right: functional boxplots for the corresponding hazard curves, with increasing width at higher ages. +
+2.9 Nonparametric maximum likelihood estimation
+The nonparametric maximum likelihood estimator is unique only up to equivalence classes. The data for individual \(i\) consists of the tuple \(\{L_i, R_i, V_i, U_i\}\), where the censoring interval is \([L_i, R_i]\) and the truncation interval is \([V_i, U_i\}\), with \(0 \leq V_i \leq L_i \leq R_i \leq U_i \leq \infty\). Turnbull (1976) shows how one can build disjoint intervals \(C = \bigsqcup_{j=1}^m [a_j, b_j]\) where \(a_j \in \mathcal{L} = \{L_1, \ldots, L_n\}\) and \(b_j \in \mathcal{R} = \{R_1, \ldots, R_n\}\) satisfy \(a_1 \leq b_1 < \cdots < a_m \leq b_m\) and the intervals \([a_j, b_j]\) contain no other members of \(L\) or \(R\) except at the endpoints. This last condition notably ensures that the intervals created include all observed failure times as singleton sets in the absence of truncation. Other authors (Lindsey and Ryan 1998) have taken interval censored data as semi-open intervals \((L_i, R_i]\), a convention we adopt here for numerical reasons. For interval censored and truncated data, Frydman (1994) shows that this construction must be amended by taking instead \(a_j \in \mathcal{L} \cup \{U_1, \ldots, U_n\}\) and \(b_j \in \mathcal{R} \cup \{V_1, \ldots, V_n\}\).
+We assign probability \(p_j = F(b_j^{+}) - F(a_j^{-}) \ge 0\) to each of the resulting \(m\) intervals under the constraint \(\sum_{j=1}^m p_j = 1\) and \(p_j \ge 0\) \((j=1, \ldots, m)\). The nonparametric maximum likelihood estimator of the distribution function \(F\) is then +\[\begin{align*} +\widehat{F}(t) = \begin{cases} 0, & t < a_1;\\ +\widehat{p}_1 + \cdots + \widehat{p}_j, & b_j < t < a_{j+1} \quad (1 \leq j \leq m-1);\\ +1, & t > b_m; +\end{cases} +\end{align*}\] +and is undefined for \(t \in [a_j, b_j]\) \((j=1, \ldots, m)\).
+
+
+
+
+
+
++Figure 6: Illustration of the truncation (pale grey) and censoring intervals (dark grey) equivalence classes based on Turnbull’s algorithm. Observations must fall within equivalence classes defined by the former. +
+The procedure of Turnbull can be encoded using \(m \times n\) matrices. For censoring, we build \(\mathbf{A}\) whose \((i,j)\)th entry \(\alpha_{ij}=1\) if \([a_j, b_j] \subseteq A_i\) and zero otherwise. Since the intervals forming the set \(C\) are disjoint and in increasing order, a more storage efficient manner of keeping track of the intervals is to find the smallest integer \(j\) such that \(L_i \leq a_j\) and the largest \(R_i \ge b_j\) \((j=1, \ldots, m)\) for each observation. The same idea applies for the truncation sets \(B_i = (V_i, U_i)\) and matrix \(\mathbf{B}\) with \((i,j)\) element \(\beta_{ij}\).
+The log likelihood function is +\[\begin{align*} +\ell(\boldsymbol{p}) = \sum_{i=1}^n w_i \log \left( \sum_{j=1}^m \alpha_{ij}p_j\right) - w_i\log \left( \sum_{j=1}^m \beta_{ij}p_j\right) +\end{align*}\]
+The numerical implementation of the EM is in principle forward: first identify the equivalence classes \(C\), next calculate the entries of \(A_i\) and \(B_i\) (or the vectors of ranges) and finally run the EM algorithm. In the second step, we need to account for potential ties in the presence of (interval) censoring and treat the intervals as open on the left for censored data. For concreteness, consider the toy example \(\boldsymbol{T} =(1,1,2)\) and \(\boldsymbol{\delta} = (0,1,1)\), where \(\delta_i = 1\) if the observation is a failure time and \(\delta_i=0\) in case of right censoring. +The left and right censoring bounds are \(\mathcal{L} = \{1, 2\}\) and \(\mathcal{R} = \{1, 2, \infty\}\) with \(A_1 = (1, \infty)\), \(A_2 = \{1\}\) and \(A_3 = \{2\}\) and \(C_1=\{1\}, C_2=\{2\}\). If we were to treat instead \(A_1\) as a semi-closed interval \([1, \infty)\), direct maximization of the log likelihood in eq. 2.2 of Turnbull (1976) would give probability half to each observed failure time. By contrast, the Kaplan–Meier estimator, under the convention that right censored observations at time \(t\) were at risk up to and until \(t\), assigns probability 1/3 to the first failure. To retrieve this solution with Turnbull’s EM estimator, we need the convention that \(C_1 \notin A_1\), but this requires comparing the bound with itself. The numerical tolerance in the implementation is taken to be the square root of the machine epsilon for doubles.
+The maximum likelihood estimator (MLE) need not be unique, and the EM algorithm is only guaranteed a local maximum. For interval censored data, Gentleman and Geyer (1994) consider using the Karush–Kuhn–Tucker conditions to determine whether the probability in some intervals is exactly zero and whether the returned value is indeed the MLE.
+Due to data scarcity, statistical inference for human lifespan is best conducted using parametric models supported by asymptotic theory, reserving nonparametric estimators to assess goodness of fit. The empirical cumulative hazard for Japanese is very close to linear from early ages, suggesting that the hazard may not be very far from exponential even if more complex models are likely to be favored given the large sample size.
+The function np_elife returns a list with Turnbull’s intervals \([a_j, b_j]\) and the probability weights assigned to each, provided the latter are positive. It also contains an object of class stepfun with a weighting argument that defines a cumulative distribution function.
+
We can use the nonparametric maximum likelihood estimator of the distribution function to assess a fitted parametric model by comparing the density with a binned distribution, or the cumulative distribution function. The distribution function is defined over equivalence classes, which may be isolated observations or intervals. The data interval over which there is non-zero probability of events is broken down into equispaced bins and the probability of failing in the latter estimated nonparametrically based on the distribution function. Alternatively, users can also provide a set of breaks, or the number of bins.
+
+
+
+ecdf <- np_elife(arguments = args_japan, thresh = 108)
+# Summary statistics, accounting for censoring
+round(summary(ecdf), digits = 2)#> Min. 1st Qu. Median Mean 3rd Qu. Max.
+#> 109.00 110.00 111.00 110.01 111.00 118.00
+
+# Plots of fitted parametric model and nonparametric CDF
+model_gp <- fit_elife(
+ arguments = args_japan,
+ thresh = 108,
+ family = "gp",
+ export = TRUE)
+# ggplot2 plots, wrapped to display side by side
+patchwork::wrap_plots(
+ autoplot(
+ model_gp, # fitted model
+ plot = FALSE, # return list of ggplots
+ which.plot = c("dens", "cdf"),
+ breaks = seq(0L, 8L, by = 1L) # set bins for histogram
+ )
+)
+
+
+
++Figure 7: Nonparametric maximum likelihood estimate of the density (bar plot, left) and distribution function (staircase function, right), with superimposed generalized Pareto fit for excess lifetimes above 108 years. Except for the discreteness inherent to the nonparametric estimates, the two representations broadly agree at year marks. +
+3 Conclusion
+This paper describes the salient features of longevity, explaining the theoretical underpinning of the methods and the design considerations followed when writing the package. While longevity was conceived for modelling lifetimes, the package could be used for applications outside of demography. Survival data in extreme value theory is infrequent yet hardly absent. For example, rainfall observations can be viewed as rounded due to the limited instrumental precision of rain gauges and treated as interval-censored. Some historical records, which are often lower bounds on the real magnitude of natural catastrophes, can be added as right-censored observations in a peaks-over-threshold analysis. In insurance, losses incurred by the company due to liability claims may be right-censored if they exceed the policy cap and are covered by a reinsurance company, or rounded (Belzile and Nešlehová 2025). In climate science, attribution studies often focus on data just after a record-breaking event, and the stopping rule leads to truncation (Miralles and Davison 2023), which biases results if ignored.
+The package has features that are interesting in their own right, including adapted quantile-quantile and other visual goodness of fit diagnostics for model validation. The testing procedures correctly handle tests for restrictions lying on the boundary of the parameter space. Parametric bootstrap procedures for such tests are not straightforward to implement given the heavy reliance on the data generating mechanism and the diversity of possible scenarios: this paper however shows how the utilities of the package can be coupled to ease such estimation and the code in the supplementary material illustrates how this can be extended for goodness of fit testing.
+Acknowledgements
+The author thanks four anonymous reviewers for valuable feedback. This research was supported financially by the Natural Sciences and Engineering Research Council of Canada (NSERC) via Discovery Grant RGPIN-2022-05001.
+3.1 Supplementary materials
+Supplementary materials are available in addition to this article. It can be downloaded at +RJ-2025-034.zip
+3.2 CRAN packages used
+longevity, evd, mev, extRemes, evgam, survival, MortCast, demography, MortalityLaws, vitality, fitdistrplus, VGAM, prodlim, icenReg, dblcens, interval, tranSurv, DTDA, Rsolnp, ggplot2, casebase, flexsurv, bshazard, muhaz, rstpm2, rust, fda
+3.3 CRAN Task Views implied by cited packages
+ActuarialScience, ChemPhys, ClinicalTrials, Distributions, Econometrics, Environmetrics, ExtremeValue, FunctionalData, MissingData, NetworkAnalysis, Optimization, Phylogenetics, Psychometrics, Spatial, Survival, TeachingStatistics
+3.4 Bioconductor packages used
+ +
+
+
+
+
+J. J. Anderson. A vitality-based model relating stressors and environmental properties to organism survival. Ecological Monographs, 70(3): 445–470, 2000. URL https://doi.org/10.1890/0012-9615(2000)070[0445:AVBMRS]2.0.CO;2.
+
+
+C. Anderson-Bergman. An efficient implementation of the EMICM algorithm for the interval censored NPMLE. Journal of Computational and Graphical Statistics, 26(2): 463–467, 2017a. URL https://doi.org/10.1080/10618600.2016.1208616.
+
+
+C. Anderson-Bergman. icenReg: Regression models for interval censored data in R. Journal of Statistical Software, 81(12): 1–23, 2017b. URL https://doi.org/10.18637/jss.v081.i12.
+
+
+E. Barbi, F. Lagona, M. Marsili, J. W. Vaupel and K. W. Wachter. The plateau of human mortality: Demography of longevity pioneers. Science, 360(6396): 1459–1461, 2018. URL https://doi.org/10.1126/science.aat3119.
+
+
+R. E. Beard. A theory of mortality based on actuarial, biological and medical considerations. In Proceedings of the international population conference, new york, 1963. London: International Union for the Scientific Study of Population.
+
+
+L. R. Belzile et al. mev: Modelling extreme values. 2023a. URL https://CRAN.R-project.org/package=mev. R package version 1.15.
+
+
+L. R. Belzile, A. C. Davison, J. Gampe, H. Rootzén and D. Zholud. Is there a cap on longevity? A statistical review. Annual Review of Statistics and its Application, 9: 22–45, 2022. URL https://doi.org/10.1146/annurev-statistics-040120-025426.
+
+
+L. R. Belzile, C. Dutang, P. J. Northrop and T. Opitz. A modeler’s guide to extreme value software. Extremes, 26: 595–638, 2023b. URL https://doi.org/10.1007/s10687-023-00475-9.
+
+
+L. R. Belzile and J. G. Nešlehová. Statistics of extremes for incomplete data, with application to lifetime and liability claim modeling. In Handbook on statistics of extremes, Eds M. de Carvalho, R. Huser, P. Naveau and B. J. Reich pages. in press 2025. CRC Press.
+
+
+S. R. Bhatnagar, M. Turgeon, J. Islam, J. A. Hanley and O. Saarela. casebase: An alternative framework for survival analysis and comparison of event rates. The R Journal, 14: 59–79, 2022. URL https://doi.org/10.32614/RJ-2022-052.
+
+
+C. G. Camarda. The curse of the plateau. Measuring confidence in human mortality estimates at extreme ages. Theoretical Population Biology, 144: 24–36, 2022. URL https://doi.org/10.1016/j.tpb.2022.01.002.
+
+
+H. Chernoff. On the distribution of the likelihood ratio. The Annals of Mathematical Statistics, 25(3): 573–578, 1954. URL https://doi.org/10.1214/aoms/1177728725.
+
+
+S. H. (Steven) Chiou and J. Qian. tranSurv: Transformation model based estimation of survival and regression under dependent truncation and independent censoring. 2021. URL https://CRAN.R-project.org/package=tranSurv. R package version 1.2.2.
+
+
+M. L. Delignette-Muller and C. Dutang. fitdistrplus: An R package for fitting distributions. Journal of Statistical Software, 64(4): 1–34, 2015. DOI 10.18637/jss.v064.i04.
+
+
+L. Devroye. Non-uniform random variate generation. New York: Springer, 1986. URL http://www.nrbook.com/devroye/.
+
+
+B. Efron and V. Petrosian. Nonparametric methods for doubly truncated data. Journal of the American Statistical Association, 94(447): 824–834, 1999. URL https://doi.org/10.1080/01621459.1999.10474187.
+
+
+J. J. Einmahl, J. H. J. Einmahl and L. de Haan. Limits to human life span through extreme value theory. Journal of the American Statistical Association, 114(527): 1075–1080, 2019. URL https://doi.org/10.1080/01621459.2018.1537912.
+
+
+M. P. Fay and P. A. Shaw. Exact and asymptotic weighted logrank tests for interval censored data: The interval R package. Journal of Statistical Software, 36(2): 1–34, 2010. URL https://doi.org/10.18637/jss.v036.i02.
+
+
+H. Frydman. A note on nonparametric estimation of the distribution function from interval-censored and truncated observations. Journal of the Royal Statistical Society. Series B (Methodological), 56(1): 71–74, 1994. URL https://doi.org/10.1111/j.2517-6161.1994.tb01960.x.
+
+
+R. Gentleman and C. J. Geyer. Maximum likelihood for interval censored data: Consistency and computation. Biometrika, 81(3): 618–623, 1994. URL https://doi.org/10.1093/biomet/81.3.618.
+
+
+R. Gentleman and A. Vandal. Icens: NPMLE for censored and truncated data. 2024. URL https://CRAN.R-project.org/package=DTDA. R package version 1.76.0.
+
+
+T. A. Gerds. prodlim: Product-limit estimation for censored event history analysis. 2024. URL https://CRAN.R-project.org/package=prodlim. R package version 2024.06.25.
+
+
+A. Ghalanos and S. Theussl. Rsolnp: General non-linear optimization using augmented Lagrange multiplier method. 2015. URL https://CRAN.R-project.org/package=Rsolnp. R package version 1.16.
+
+
+E. Gilleland and R. W. Katz. extRemes 2.0: An extreme value analysis package in R. Journal of Statistical Software, 72(8): 1–39, 2016. DOI 10.18637/jss.v072.i08.
+
+
+P. Groeneboom and J. A. Wellner. Information bounds and nonparametric maximum likelihood estimation. Basel, Switzerland, 1992. URL https://doi.org/10.1007/978-3-0348-8621-5.
+
+
+G. Grolemund and H. Wickham. Dates and times made easy with lubridate. Journal of Statistical Software, 40(3): 1–25, 2011. URL https://www.jstatsoft.org/v40/i03/.
+
+
+K. Hess and R. Gentleman. muhaz: Hazard function estimation in survival analysis. 2021. URL https://CRAN.R-project.org/package=muhaz. R package version 1.2.6.4.
+
+
+R. Hyndman. demography: Forecasting mortality, fertility, migration and population data. 2023. URL https://pkg.robjhyndman.com/demography/. R package version 2.0.
+
+
+C. Jackson. flexsurv: A platform for parametric survival modeling in R. Journal of Statistical Software, 70(8): 1–33, 2016. URL https://www.jstatsoft.org/index.php/jss/article/view/v070i08.
+
+
+D. A. Jdanov, V. M. Shkolnikov and S. Gellers-Barkmann. The international database on longevity: Data resource profile. In Exceptional lifespans, Eds H. Maier, B. Jeune and J. W. Vaupel pages. 22–24 2021. Cham, Switzerland: Springer. URL https://doi.org/10.1007/978-3-030-49970-9_2.
+
+
+E. L. Kaplan and P. Meier. Nonparametric estimation from incomplete observations. Journal of the American Statistical Association, 53: 457–481, 1958. URL https://doi.org/10.1080/01621459.1958.10501452.
+
+
+T. Li and J. J. Anderson. The vitality model: A way to understand population survival and demographic heterogeneity. Theoretical Population Biology, 76(2): 118–131, 2009. URL https://doi.org/10.1016/j.tpb.2009.05.004.
+
+
+J. C. Lindsey and L. M. Ryan. Methods for interval-censored data. Statistics in Medicine, 17(2): 219–238, 1998. URL https://doi.org/10.1002/(SICI)1097-0258(19980130)17:2<219::AID-SIM735>3.0.CO;2-O.
+
+
+X.-R. Liu, Y. Pawitan and M. Clements. Parametric and penalized generalized survival models. Statistical Methods in Medical Research, 27(5): 1531–1546, 2018. URL https://doi.org/10.1177/0962280216664760.
+
+
+H. Maier, B. Jeune and J. W. Vaupel, eds. Exceptional lifespans. Cham, Switzerland: Springer, 2021. URL https://doi.org/10.1007/978-3-030-49970-9.
+
+
+O. Miralles and A. C. Davison. Timing and spatial selection bias in rapid extreme event attribution. Weather and Climate Extremes, 41: 100584, 2023. URL https://doi.org/10.1016/j.wace.2023.100584.
+
+
+C. Moreira, J. de Uña-Álvarez and R. Crujeiras. DTDA: Doubly truncated data analysis. 2022. URL https://bioconductor.org/packages/Icens/. R package version 3.0.1.
+
+
+P. J. Northrop. rust: Ratio-of-uniforms simulation with transformation. 2023. URL https://paulnorthrop.github.io/rust/. R package version 1.4.2, https://github.com/paulnorthrop/rust.
+
+
+P. J. Northrop and C. L. Coleman. Improved diagnostic plots for extreme value analyses. Extremes, 17: 289–303, 2014. URL https://doi.org/10.1007/s10687-014-0183-z.
+
+
+M. D. Pascariu. MortalityLaws: Parametric mortality models, life tables and HMD. 2024. URL https://CRAN.R-project.org/package=MortalityLaws. R package version 2.1.0.
+
+
+G. Passolt, J. J. Anderson, T. Li, D. H. Salinger and D. J. Sharrow. vitality: Fitting routines for the vitality family of mortality models. 2018. URL https://CRAN.R-project.org/package=vitality. R package version 1.3.
+
+
+W. Perks. On some experiments in the graduation of mortality statistics. Journal of the Institute of Actuaries, 63(1): 12–57, 1932. DOI 10.1017/s0020268100046680.
+
+
+J. Ramsay. fda: Functional data analysis. 2024. URL http://www.functionaldata.org. R package version 6.1.8.
+
+
+P. Rebora, A. Salim and M. Reilly. bshazard: A flexible tool for nonparametric smoothing of the hazard function. The R Journal, 6(2): 114–122, 2014. URL https://doi.org/10.32614/RJ-2014-028.
+
+
+S. J. Richards. A handbook of parametric survival models for actuarial use. Scandinavian Actuarial Journal, 2012(4): 233–257, 2012. URL https://doi.org/10.1080/03461238.2010.506688.
+
+
+A. Rotnitzky, D. R. Cox, M. Bottai and J. Robins. Likelihood-based inference with singular information matrix. Bernoulli, 6(2): 243–284, 2000. URL https://doi.org/10.2307/3318576.
+
+
+P. Royston and M. K. B. Parmar. Flexible parametric proportional-hazards and proportional-odds models for censored survival data, with application to prognostic modelling and estimation of treatment effects. Statistics in Medicine, 21(15): 2175–2197, 2002. URL https://doi.org/10.1002/sim.1203.
+
+
+T. Säilynoja, P.-C. Bürkner and A. Vehtari. Graphical test for discrete uniformity and its applications in goodness of fit evaluation and multiple sample comparison. Statistics and Computing, 32(2): 32, 2022. URL https://doi.org/10.1007/s11222-022-10090-6.
+
+
+S. G. Self and K.-Y. Liang. Asymptotic properties of maximum likelihood estimators and likelihood ratio tests under nonstandard conditions. Journal of the American Statistical Association, 82(398): 605–610, 1987. URL https://doi.org/10.1080/01621459.1987.10478472.
+
+
+H. Ševčíková, N. Li, V. Kantorová, P. Gerland and A. E. Raftery. Age-specific mortality and fertility rates for probabilistic population projections. In Dynamic demographic analysis, Ed R. Schoen pages. 285–310 2016. Cham, Switzerland: Springer. ISBN 978-3-319-26603-9. URL https://doi.org/10.1007/978-3-319-26603-9_15.
+
+
+H. Ševčı́ková, N. Li and P. Gerland. MortCast: Estimation and projection of age-specific mortality rates. 2022. URL https://CRAN.R-project.org/package=MortCast. R package version 2.7-0.
+
+
+P.-S. Shen. Nonparametric analysis of doubly truncated data. Annals of the Institute of Statistical Mathematics, 62(5): 835–853, 2010. URL https://doi.org/10.1007/s10463-008-0192-2.
+
+
+A. G. Stephenson. evd: Extreme value distributions. R News, 2(2): 2002. URL https://cran.r-project.org/doc/Rnews/Rnews_2002-2.pdf.
+
+
+Y. Sun and M. G. Genton. Functional boxplots. Journal of Computational and Graphical Statistics, 20(2): 316–334, 2011. DOI 10.1198/jcgs.2011.09224.
+
+
+T. M. Therneau. A package for survival analysis in R. 2022. URL https://CRAN.R-project.org/package=survival. R package version 3.3-1.
+
+
+T. M. Therneau and P. M. Grambsch. Modeling survival data: Extending the Cox model. New York: Springer, 2000. URL https://doi.org/10.1007/978-1-4757-3294-8.
+
+
+W.-Y. Tsai, N. P. Jewell and M.-C. Wang. A note on the product-limit estimator under right censoring and left truncation. Biometrika, 74(4): 883–886, 1987. URL https://doi.org/10.1093/biomet/74.4.883.
+
+
+B. W. Turnbull. The empirical distribution function with arbitrarily grouped, censored and truncated data. Journal of the Royal Statistical Society, Series B, 38: 290–295, 1976. URL https://doi.org/10.1111/j.2517-6161.1976.tb01597.x.
+
+
+L. A. Waller and B. W. Turnbull. Probability plotting with censored data. American Statistician, 46(1): 5–12, 1992. URL https://doi.org/10.1080/00031305.1992.10475837.
+
+
+S. Watanabe. Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. Journal of Machine Learning Research, 11: 3571–3594, 2010. URL https://dl.acm.org/doi/10.5555/1756006.1953045.
+
+
+H. Wickham. ggplot2: Elegant graphics for data analysis. Springer-Verlag New York, 2016. URL https://ggplot2.tidyverse.org.
+
+
+H. Wickham, M. Averick, J. Bryan, W. Chang, L. D. McGowan, R. François, G. Grolemund, A. Hayes, L. Henry, J. Hester, et al. Welcome to the tidyverse. Journal of Open Source Software, 4(43): 1686, 2019. URL https://doi.org/10.21105/joss.01686.
+
+
+H. Wickham, R. François, L. Henry, K. Müller and D. Vaughan. dplyr: A grammar of data manipulation. 2023. URL https://CRAN.R-project.org/package=dplyr. R package version 1.1.4.
+
+
+Y. Ye. Interior algorithms for linear, quadratic, and linearly constrained non-linear programming. 1987.
+
+
+T. W. Yee. Vector generalized linear and additive models: With an implementation in R. New York, NY: Springer, 2015. URL https://doi.org/10.1007/978-1-4939-2818-7.
+
+
+T. W. Yee and C. J. Wild. Vector generalized additive models. Journal of the Royal Statistical Society: Series B (Methodological), 58(3): 481–493, 1996. URL https://doi.org/10.1111/j.2517-6161.1996.tb02095.x.
+
+
+B. D. Youngman. evgam: An R package for generalized additive extreme value models. Journal of Statistical Software, 103(3): 1–26, 2022. URL https://www.jstatsoft.org/index.php/jss/article/view/v103i03.
+
+
+M. Zhou, L. Lee, K. Chen and Y. Yang. dblcens: Compute the NPMLE of distribution function from doubly censored data, plus the empirical likelihood ratio for \(F(T)\). 2023. URL https://CRAN.R-project.org/package=dblcens. R package version 1.1.9.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/_articles/RJ-2025-034/RJ-2025-034.pdf b/_articles/RJ-2025-034/RJ-2025-034.pdf
new file mode 100644
index 0000000000..7ba35a40a4
Binary files /dev/null and b/_articles/RJ-2025-034/RJ-2025-034.pdf differ
diff --git a/_articles/RJ-2025-034/RJ-2025-034.tex b/_articles/RJ-2025-034/RJ-2025-034.tex
new file mode 100644
index 0000000000..597231e36e
--- /dev/null
+++ b/_articles/RJ-2025-034/RJ-2025-034.tex
@@ -0,0 +1,550 @@
+% !TeX root = RJwrapper.tex
+\title{longevity: An R Package for Modelling Excess Lifetimes}
+
+
+\author{by Léo R. Belzile}
+
+\maketitle
+
+\abstract{%
+The longevity R package provides a maximum likelihood estimation routine for modelling of survival data that are subject to non-informative censoring or truncation. It includes a selection of 12 parametric models of varying complexity, with a focus on tools for extreme value analysis and more specifically univariate peaks over threshold modelling. The package provides utilities for univariate threshold selection, parametric and nonparametric maximum likelihood estimation, goodness of fit diagnostics and model comparison tools. These different methods are illustrated using individual Dutch records and aggregated Japanese human lifetime data.
+}
+
+\section{Introduction}\label{introduction}
+
+Many data sets collected by demographers for the analysis of human longevity have unusual features and for which limited software implementations exist. The \CRANpkg{longevity} package was initially built for dealing with human longevity records and data from the International Database on Longevity (IDL), which provides age at death of supercentenarians, i.e., people who died above age 110. Data for the statistical analysis of (human) longevity can take the form of aggregated counts per age at death, or most commonly life trajectory of individuals with both birth and death dates. Such lifetimes are often interval truncated (only age at death of individuals dying between two calendar dates are recorded) or left truncated and right censored (when data of individuals still alive at the end of the collection period are also included). Another frequent format is death counts, aggregated per age band. Censoring and truncation are typically of administrative nature and thus non-informative about death.
+
+Supercentenarians are extremely rare and records are sparse. The most popular parametric models used by practitioners are justified by asymptotic arguments and have their roots in extreme value theory. Univariate extreme value distributions are well implemented in software and \citet{Belzile.Dutang.Northrop.Opitz:2022} provides a recent review of existing implementations. While there are many standard R packages for the analysis of univariate extremes using likelihood-based inference, such as \CRANpkg{evd} \citep{evd}, \CRANpkg{mev} and \CRANpkg{extRemes} \citep{extRemes}, only the \CRANpkg{evgam} package includes functionalities to fit threshold exceedance models with censoring, as showcased in \citet{evgam} with rounded rainfall measurements. Support for survival data for extreme value models is in general wholly lacking, which motivated the development of \CRANpkg{longevity}.
+
+The \CRANpkg{longevity} package also includes several parametric models commonly used in demography. Many existing packages that focus on tools for modelling mortality rates, typically through life tables, are listed in the CRAN Task View \ctv{ActuarialScience} in the Life insurance section. They do not however allow for truncation or more general survival mechanisms, as the aggregated data used are typically complete except for potential right censoring for the oldest age group. The \CRANpkg{survival} package \citep{survival-book} includes utilities for accelerated failure time models for 10 parametric models. The \CRANpkg{MortCast} package can be used to estimate age-specific mortality rates using the Kannisto and Lee--Carter approaches, among others \citep{Sevcikova:2016}. The \CRANpkg{demography} package provides forecasting methods for death rates based on constructed life tables using the Lee--Carter or ARIMA models. \CRANpkg{MortalityLaws} includes utilities to download data from the Human Mortality Database (HMD), and fit a total of 27 parametric models for life table data (death counts and population at risk per age group) using Poisson, binomial or alternative loss functions. The \CRANpkg{vitality} package fits the family of vitality models \citep{Li.Anderson:2009, Anderson:2000} via maximum likelihood based on empirical survival data. The \CRANpkg{fitdistrplus} \citep{fitdistrplus-package} allows for generic parametric distributions to be fitted to interval censored data via maximum likelihood, with various S3 methods for model assessment. The package also allows user-specified models, thereby permitting custom definitions for truncated distributions whose truncation bounds are passed as fixed vector parameters. Parameter uncertainty can be obtained via additional functions using nonparametric bootstrap. Many parametric distributions also appear in the \CRANpkg{VGAM} package \citep{Yee.Wild:1996, VGAMbook}, which allows for vector generalized linear modelling. The \CRANpkg{longevity} package is less general and offers support only for selected parametric distributions, but contrary to the aforementioned packages allows for truncation and general patterns. One strength of \CRANpkg{longevity} is that it also includes model comparison tools that account for non-regular asymptotics and goodness of fit diagnostics.
+
+Nonparametric methods are popular tools for the analysis of survival data with large samples, owing to their limited set of assumptions. They also serve for the validation of parametric models. Without explanatory variable, a closed-form estimator of the nonparametric maximum likelihood estimator of the survival function can be derived in particular instances, including the product limit estimator \citep{Kaplan.Meier:1958} for the case of random or non-informative right censoring and an extension allowing for left truncation \citep{Tsai.Jewell.Wang:1987}. In general, the nonparametric maximum likelihood estimator of the survival function needs to be computed using an expectation-maximization (EM) algorithm \citep{Turnbull:1976}. Nonparametric estimators only assign probability mass on observed failure times and intervals and so cannot be used for extrapolation beyond the range of the data, limiting its utility in extreme value analysis.
+
+The CRAN Task View on \ctv{Survival} Analysis lists various implementations of nonparametric maximum likelihood estimators of the survival or hazard functions: \CRANpkg{survival} implements the Kaplan--Meier and Nelson--Aalen estimators. Many packages focus on the case of interval censoring \citep[\S 3.2]{Groeneboom.Wellner:1992}, including \CRANpkg{prodlim}; \citet{Anderson-Bergman:2017} reviews the performance of the implementations in \CRANpkg{icenReg} \citep{icensReg} and \BIOpkg{Icens}. The latter uses the incidence matrix as input data. Routines for doubly censored data are provided by \CRANpkg{dblcens}. The \CRANpkg{interval} \citep{interval} implements Turnbull's EM algorithm for interval censored data. The case of left truncated and right censored data is handled by \CRANpkg{survival}, and \CRANpkg{tranSurv} provides transformation models with the potential to account for truncation dependent on survival times. For interval truncated data, dedicated algorithms that use gradient-based steps \citep{Efron.Petrosian:1999} or inverse probability weighting \citep{Shen:2010} exist and can be more efficient than the EM algorithm of \citet{Turnbull:1976}. Many of these are implemented in the \CRANpkg{DTDA} package. The \CRANpkg{longevity} package includes a C\(^{++}\) implementation of the corrected Turnbull's algorithm \citep{Turnbull:1976}, returning the nonparametric maximum likelihood estimator for arbitrary censoring and truncation patterns, as opposed to the specific existing aforementioned implementations already available which focus on specific subcases. With a small number of observations, it is also relatively straightforward to maximize the log likelihood for the concave program subject to linear constraints using constrained optimization algorithms; \CRANpkg{longevity} relies for this on \CRANpkg{Rsolnp}, which uses augmented Lagrangian methods and sequential quadratic programming \citep{Ye:1987}.
+
+\subsection{Motivating examples}\label{motivating-examples}
+
+To showcase the functionality of the package and particularity for the modelling of threshold exceedances, we consider Dutch and Japanese life lengths. The \texttt{dutch} database contains the age at death (in days) of Dutch people who died above age 92 between 1986 and 2015; these data were obtained from Statistics Netherlands and analyzed in \citet{Einmahl:2019} and \citet{ARSIA:2022}. Records are interval truncated, as people are included in the database only if they died during the collection period. In addition, there are 226 interval censored and interval truncated records for which only the month and year of birth and death are known, as opposed to exact dates.
+
+The second database we consider is drawn from \citet{ExceptionalLifespans}. The data frame \texttt{japanese2} consists of counts of Japanese above age 100 by age band and are stratified by both birth cohort and sex. To illustrate the format of the data, counts for female Japanese are reproduced in Table \ref{tab:tbl-japanese-women}. The data were constructed using the extinct cohort method and are interval censored between \(\texttt{age}\) and \(\texttt{age} + 1\) and right truncated at the age reached by the oldest individuals of their birth cohort in 2020. The \texttt{count} variable lists the number of instances in the contingency table, and serves as a weight for likelihood contributions.
+
+\begin{table}
+
+\caption{\label{tab:tbl-japanese-women}Death count by birth cohort and age band for female Japanese.}
+\centering
+\begin{tabular}[t]{rrrrrrr}
+\toprule
+age & 1874-1878 & 1879-1883 & 1884-1888 & 1889-1893 & 1894-1898 & 1899-1900\\
+\midrule
+100 & 1648 & 2513 & 4413 & 8079 & 16036 & 9858\\
+101 & 975 & 1596 & 2921 & 5376 & 11047 & 7091\\
+102 & 597 & 987 & 1864 & 3446 & 7487 & 4869\\
+103 & 345 & 597 & 1153 & 2230 & 5014 & 3293\\
+104 & 191 & 351 & 662 & 1403 & 3242 & 2133\\
+105 & 121 & 197 & 381 & 855 & 2084 & 1357\\
+106 & 64 & 122 & 210 & 495 & 1284 & 836\\
+107 & 34 & 74 & 120 & 274 & 774 & 521\\
+108 & 16 & 41 & 66 & 152 & 433 & 297\\
+109 & 12 & 30 & 39 & 83 & 252 & 167\\
+110 & 6 & 17 & 21 & 49 & 130 & 92\\
+111 & 4 & 10 & 15 & 26 & 69 & 47\\
+112 & 3 & 3 & 11 & 15 & 29 & 22\\
+113 & 2 & 2 & 8 & 5 & 15 & 9\\
+114 & 1 & 2 & 4 & 3 & 7 & 2\\
+115 & 0 & 1 & 1 & 0 & 3 & 2\\
+116 & 0 & 1 & 1 & 0 & 1 & 1\\
+117 & 0 & 0 & 0 & 0 & 1 & 1\\
+\bottomrule
+\end{tabular}
+\end{table}
+
+\section{Package functionalities}\label{package-functionalities}
+
+The \CRANpkg{longevity} package uses the S3 object oriented system and provides a series of functions with common arguments. The syntax used by the \CRANpkg{longevity} package purposely mimics that of the \CRANpkg{survival} package \citep{survival-book}, except that it does not specify models using a formula. Users must provide vectors for the time or age (or bounds in case of intervals) via arguments \texttt{time} and \texttt{time2}, as well as lower and upper truncation bounds (\texttt{ltrunc} and \texttt{rtrunc}) if applicable. The integer vector \texttt{event} is used to indicate the type of event, where following \CRANpkg{survival} \texttt{0} indicates right-censoring, \texttt{1} observed events, \texttt{2} left-censoring and \texttt{3} interval censoring. Together, these five vectors characterize the data and the survival mechanisms at play. Depending on the sampling scheme, not all arguments are required or relevant and they need not be of the same length, but are common to most functions. Users can also specify a named list \texttt{args} to pass arguments: as illustrated below, this is convenient to avoid specifying repeatedly the common arguments in each function call. Default values are overridden by elements in \texttt{args}, with the exception of those that are passed by the user directly in the call. Relative to \CRANpkg{survival}, functions have additional arguments \texttt{ltrunc} and \texttt{rtrunc} for left and right truncation limits, as these are also possibly matrices for the case of double interval truncation \citep{ARSIA:2022}, since both censoring and truncation can be present simultaneously.
+
+We can manipulate the data set to build the time vectors and truncation bounds for the Dutch data. We re-scale observations to years for interpretability and keep only records above age 98 for simplicity. We split the data to handle the observed age at death first: these are treated as observed (uncensored) whenever \texttt{time} and \texttt{time2} coincide. When exact dates are not available, we compute the range of possible age at which individuals may have died, given their birth and death years and months. The truncation bounds for each individual can be obtained by subtracting from the endpoints of the sampling frame the birth dates, with left and right truncation bounds
+\begin{align*}
+\texttt{ltrunc}=\min\{92 \text{ years}, 1986.01.01 - \texttt{bdate}\}, \qquad \texttt{rtrunc} = 2015.12.31- \texttt{bdate}.
+\end{align*}
+Table \ref{tab:tbl-dutch-preview} shows a sample of five individuals, two of whom are interval-censored, and the corresponding vectors of arguments along with two covariates, gender (\texttt{gender}) and birth year (\texttt{byear}).
+
+\begin{table}
+
+\caption{\label{tab:tbl-dutch-preview}Sample of five Dutch records, formatted so that the inputs match the function arguments used by the package. Columns give the age in years at death (or plausible interval), lower and upper truncation bounds giving minimum and maximum age for inclusion, an integer indicating the type of censoring, gender and birth year.}
+\centering
+\begin{tabular}[t]{rrrrrlr}
+\toprule
+time & time2 & ltrunc & rtrunc & event & gender & byear\\
+\midrule
+104.67 & 105.74 & 80.00 & 111.00 & 3 & female & 1905\\
+103.50 & 104.58 & 78.00 & 109.00 & 3 & female & 1907\\
+100.28 & 100.28 & 92.01 & 104.50 & 1 & female & 1911\\
+100.46 & 100.46 & 92.01 & 102.00 & 1 & male & 1913\\
+100.51 & 100.51 & 92.01 & 104.35 & 1 & female & 1911\\
+\bottomrule
+\end{tabular}
+\end{table}
+
+We can proceed similarly for the Japanese data. Ages of centenarians are rounded down to the nearest year, so all observations are interval censored within one-year intervals. Assuming that the ages at death are independent and identically distributed with distribution function \(F(\cdot; \boldsymbol{\theta})\), the log likelihood for exceedances \(y_i = \texttt{age}_i - u\) above age \(u\) is
+\begin{align*}
+\ell(\boldsymbol{\theta}) = \sum_{i: \texttt{age}_i > u}n_i \left[\log \{F(y_i+1; \boldsymbol{\theta}) - F(y_i; \boldsymbol{\theta})\} - \log F(r_i - u; \boldsymbol{\theta})\right]
+\end{align*}
+where \(n_i\) is the count of the number of individuals in cell \(i\) and \(r_i > \texttt{age}_i+1\) is the right truncation limit for that cell, i.e., the maximum age that could have been achieved for that birth cohort by the end of the data collection period.
+
+\begin{verbatim}
+data(japanese2, package = "longevity")
+# Keep only non-empty cells
+japanese2 <- japanese2[japanese2$count > 0, ]
+# Define arguments that are recycled
+japanese2$rtrunc <- 2020 -
+ as.integer(substr(japanese2$bcohort, 1, 4))
+# The line above extracts the earliest year of the birth cohort
+# Create a list with all arguments common to package functions
+args_japan <- with(japanese2,
+ list(
+ time = age, # lower censoring bound
+ time2 = age + 1L, # upper censoring bound
+ event = 3, # define interval censoring
+ type = "interval2",
+ rtrunc = rtrunc, # right truncation limit
+ weights = count)) # counts as weights
+\end{verbatim}
+
+\subsection{Parametric models and maximum likelihood estimation}\label{parametric-models-and-maximum-likelihood-estimation}
+
+Various models are implemented in \CRANpkg{longevity}: their hazard functions are reported in Table \ref{tab:tbl-parametric-models}. Two of those models, labelled \texttt{perks} and \texttt{beard} are logistic-type hazard functions proposed in \citet{Perks:1932} that have been used by \citet{Beard:1963}, and popularized in work of Kannisto and Thatcher; we use the parametrization of \citet{Richards:2012}, from which we also adopt the nomenclature. Users can compare the models with those available in \CRANpkg{MortalityLaws}; see \texttt{?availableLaws} for the list of hazard functions and parametrizations.
+
+\begin{table}
+\centering
+\caption{\label{tab:tbl-parametric-models}List of parametric models for excess lifetime supported by the package, with parametrization and hazard functions.}
+\centering
+\begin{tabular}[t]{lll}
+\toprule
+model & hazard function & constraints\\
+\midrule
+\texttt{exp} & \(\sigma^{-1}\) & \(\sigma > 0\)\\
+\texttt{gomp} & \(\sigma^{-1}\exp(\beta t/\sigma)\) & \(\sigma > 0, \beta \ge 0\)\\
+\texttt{gp} & \((\sigma + \xi t)_{+}^{-1}\) & \(\sigma > 0, \xi \in \mathbb{R}\)\\
+\texttt{weibull} & \(\sigma^{-\alpha} \alpha t^{\alpha-1}\) & \(\sigma > 0, \alpha > 0\)\\
+\texttt{extgp} & \(\beta\sigma^{-1}\exp(\beta t/\sigma)[\beta+\xi\{\exp(\beta t/\sigma) -1\}]^{-1}\) & \(\sigma > 0, \beta \ge 0, \xi \in \mathbb{R}\)\\
+\texttt{extweibull} & \(\alpha\sigma^{-\alpha}t^{\alpha-1}\{1+\xi(t/\sigma)^{\alpha}\}_{+}\) & \(\sigma > 0, \alpha > 0, \xi \in \mathbb{R}\)\\
+\texttt{perks} & \(\alpha\exp(\nu x)/\{1+\alpha\exp(\nu x)\}\) & \(\nu \ge 0, \alpha >0\)\\
+\texttt{beard} & \(\alpha\exp(\nu x)/\{1+\alpha\beta\exp(\nu x)\}\) & \(\nu \ge 0, \alpha >0, \beta \ge 0\)\\
+\texttt{gompmake} & \(\lambda + \sigma^{-1}\exp(\beta t/\sigma)\) & \(\lambda \ge 0, \sigma > 0, \beta \ge 0\)\\
+\texttt{perksmake} & \(\lambda + \alpha\exp(\nu x)/\{1+\alpha\exp(\nu x)\}\) & \(\lambda \ge 0, \nu \ge 0, \alpha > 0\)\\
+\texttt{beardmake} & \(\lambda + \alpha\exp(\nu x)/\{1+\alpha\beta\exp(\nu x)\}\) & \( \lambda \ge 0, \nu \ge 0, \alpha > 0, \beta \ge 0\)\\
+\bottomrule
+\end{tabular}
+\end{table}
+
+Many of the models are nested and Figure \ref{fig:fig-nesting} shows the logical relation between the various families. The function \texttt{fit\_elife} allows users to fit all of the parametric models of Table \ref{tab:tbl-parametric-models}: the \texttt{print} method returns a summary of the sampling mechanism, the number of observations, the maximum log likelihood and parameter estimates with standard errors. Depending on the data, some models may be overparametrized and parameters need not be numerically identifiable. To palliate such issues, the optimization routine, which uses \CRANpkg{Rsolnp}, can try multiple starting values or fit various sub-models to ensure that the parameter values returned are indeed the maximum likelihood estimates. If one tries to compare nested models and the fit of the simpler model is better than the alternative, the \texttt{anova} function will return an error message.
+
+The \texttt{fit\_elife} function handles arbitrary censoring patterns over single intervals, along with single interval truncation and interval censoring. To accommodate the sampling scheme of the International Database on Longevity (IDL), an option also allows for double interval truncation \citep{ARSIA:2022}, whereby observations are included only if the person dies between time intervals, potentially overlapping, which defines the observation window over which dead individuals are recorded.
+
+\begin{verbatim}
+thresh <- 108
+model0 <- fit_elife(arguments = args_japan,
+ thresh = thresh,
+ family = "exp")
+(model1 <- fit_elife(arguments = args_japan,
+ thresh = thresh,
+ family = "gomp"))
+#> Model: Gompertz distribution.
+#> Sampling: interval censored, right truncated
+#> Log-likelihood: -3599.037
+#>
+#> Threshold: 108
+#> Number of exceedances: 2489
+#>
+#> Estimates
+#> scale shape
+#> 1.6855 0.0991
+#>
+#> Standard Errors
+#> scale shape
+#> 0.0523 0.0273
+#>
+#> Optimization Information
+#> Convergence: TRUE
+\end{verbatim}
+
+\subsection{Model comparisons}\label{model-comparisons}
+
+Goodness of fit of nested models can be compared using likelihood ratio tests via the \texttt{anova} method. Most of the interrelations between models yield non-regular model comparisons since, to recover the simpler model, one must often fix parameters to values that lie on the boundary of the parameter space. For example, if we compare a Gompertz model with the exponential, the limiting null distribution is a mixture of a point mass at zero and a \(\chi^2_1\) variable, both with probability half \citep{Chernoff:1954}. Many authors \citep[e.g.,][]{Camarda:2022} fail to recognize this fact. The case becomes more complicated with more than one boundary constraint: for example, the deviance statistic comparing the Beard--Makeham and the Gompertz model, which constrains two parameters on the boundary of the parameter space, has a null distribution which is a mixture of \(\chi^2_2/4 + \chi^2_1/2 + \chi^2_0/4\) \citep{Self.Liang:1987}.
+
+Nonidentifiability impacts testing: for example, if the rate parameter of the Perks--Makeham model (\texttt{perksmake}) \(\nu \to 0\), the limiting hazard, \(\lambda + \exp(\alpha)/\{1+\exp(\alpha)\}\), is constant (exponential model), but neither \(\alpha\) nor \(\lambda\) is identifiable. The usual asymptotics for the likelihood ratio test break down as the information matrix is singular \citep{Rotnitzky:2000}. As such, all three families that include a Makeham component cannot be directly compared to the exponential in \CRANpkg{longevity} and the call to \texttt{anova} returns an error message.
+
+Users can also access information criteria, \texttt{AIC} and \texttt{BIC}. The correction factors implemented depend on the number of parameters of the distribution, but do not account for singular fit, non-identifiable parameters or singular models for which the usual corrections \(2p\) and \(\ln(n)p\) are inadequate \citep{Watanabe:2010}.
+
+\begin{figure}
+
+{\centering \includegraphics[width=1\linewidth,alt={Graph with parametric model names, and arrows indicating the relationship between these. Dashed arrows indicate non-regular comparisons between nested models, and the expression indicates which parameter to fix to obtain the submodel.}]{fig/nesting_graph}
+
+}
+
+\caption{Relationship between parametric models showing nested relations. Dashed arrows represent restrictions that lead to nonregular asymptotic null distribution for comparison of nested models. Comparisons between models with Makeham components and exponential are not permitted by the software because of nonidentifiability issues.}\label{fig:fig-nesting}
+\end{figure}
+
+To showcase how hypothesis testing is performed, we consider a simple example with two nested models. We test whether the exponential model is an adequate simplification of the Gompertz model for exceedances above 108 years --- an irregular testing problem since \(\beta=0\) is a restriction on the boundary of the parameter space. The drop in log likelihood is quite large, indicating the exponential model is not an adequate simplification of the Gompertz fit. This is also what is suggested by the Bayesian information criterion, which is much lower for the Gompertz model than for the exponential.
+
+\begin{verbatim}
+# Model comparison
+anova(model1, model0)
+# Information criteria
+c("exponential" = BIC(model0), "Gompertz" = BIC(model1))
+\end{verbatim}
+
+\begin{tabular}{lrrrrr}
+\toprule
+ & npar & Deviance & Df & Chisq & Pr(>Chisq)\\
+\midrule
+gomp & 2 & 7198.07 & & & \\
+exp & 1 & 7213.25 & 1 & 15.17 & 0\\
+\bottomrule
+\end{tabular}
+
+\begin{verbatim}
+#> exponential Gompertz
+#> 7221.065 7213.713
+\end{verbatim}
+
+\subsection{Simulation-based inference}\label{simulation-based-inference}
+
+Given the poor finite sample properties of the aforementioned tests, it may be preferable to rely on a parametric bootstrap rather than on the asymptotic distribution of the test statistic \citep{ARSIA:2022} for model comparison.
+Simulation-based inference requires capabilities for drawing new data sets whose features match those of the original one. For example, the \href{supercentenarians.org}{International Database on Longevity} (IDL) \citep{IDL:2021} features data that are interval truncated above 110 years, but doubly interval truncated since the sampling period for semisupercentenarians (who died age 105 to 110) and supercentenarians (who died above 110) are not always the same \citep{ARSIA:2022}. The 2018 Istat semisupercentenarians database analyzed by \citet{Barbi:2018} on Italians includes left truncated right censored records.
+
+To mimic the postulated data generating mechanism while accounting for the sampling scheme, we could use the observed birth dates, or simulate new birth dates (possibly through a kernel estimator of the empirical distribution of birth dates) while keeping the sampling frame with the first and last date of data collection to define the truncation interval. In other settings, one could obtain the nonparametric maximum likelihood estimator of the distribution of the upper truncation bound \citep{Shen:2010} using an inverse probability weighted estimator, which for fixed data collection windows is equivalent to setting the birth date.
+
+The \texttt{samp\_elife} function includes multiple \texttt{type2} arguments to handle these. For interval truncated data (\texttt{type2="ltrt"}), it uses the inversion method (Section 2 of \citet{Devroye:1986}): for \(F\) an absolutely continuous distribution function and \(F^{-1}\) the corresponding quantile function, a random variable distributed according to \(F\) truncated on \([a,b]\) is generated as \(X \sim F^{-1}[F(a) + U\{F(b)-F(a)\}]\)
+where \(U \sim \mathsf{U}(0,1)\) is standard uniform.
+
+The function \texttt{samp\_elife} also has an argument \texttt{upper} which serves for both right truncation, and right censoring. For the latter, any record simulated that exceeds \texttt{upper} is capped at that upper bound and declared partially observed. This is useful for simulating administrative censoring, whereby the birth date and the upper bound of the collection window fully determine whether an observation is right censored or not. An illustrative example is provided in the next section.
+
+The \texttt{anova} method call uses the asymptotic null distribution for comparison of nested parametric distributions \(\mathcal{F}_0 \subseteq \mathcal{F}_1\). We could use the bootstrap to see how good this approximation to the null distribution is. To mimic as closely as possible the data generating mechanism, which is custom in most scenarios, we condition on the sampling frame and the number of individuals in each birth cohort. The number dying at each age is random, but the right truncation limits will be the same for anyone in that cohort. We simulate excess lifetimes, then interval censor observations by keeping only the corresponding age bracket. Under the null hypothesis, the data are drawn from \(\widehat{F}_0 \in \mathcal{F}_0\) and we generate observations from this right truncated distribution using the \texttt{samp\_elife} utility, which also supports double interval truncation and left truncation right censoring. This must be done within a for loop since we have count attached to each upper bound, but the function is vectorized should we use a single vector containing all of the right truncation limits.
+
+The bootstrap \(p\)-value for comparing models \(M_0 \subset M_1\) would be obtained by repeating the following steps \(B\) times and calculating the rank of the observed test statistic among alternatives:
+
+\begin{enumerate}
+\def\labelenumi{\arabic{enumi}.}
+\tightlist
+\item
+ Simulate new birth dates \(d_i\) \((i=1, \ldots, n)\) (e.g., drawing from a smoothed empirical distribution of birth dates); the latest possible birth date is one which ensures the person reached at least the threshold by the end of the period.
+\item
+ Subtract the endpoints of the sampling period, say \(c_1\) and \(c_2\) to get the minimum (maximum) age at death, \(c_1 - d_i\) (respectively \(c_2 - d_i\)) days, which define the truncation bounds.
+\item
+ Use the function \texttt{samp\_elife} to simulate new observations from a parametric interval truncated distribution from the null model \(M_0\)
+\item
+ Use the optimization procedure in \texttt{fit\_elife} to fit the model with both \(M_1\) and \(M_2\) and calculate the deviance, and from them the likelihood ratio statistic.
+\end{enumerate}
+
+The algorithm is implemented below for comparing the Gompertz and the exponential model. Since the procedure is computationally intensive, users must trade off between the precision of the bootstrap \(p\)-value estimate and the number of replications, \(B\).
+
+\begin{verbatim}
+set.seed(2022)
+# Count of unique right truncation limit
+db_rtrunc <- aggregate(count ~ rtrunc,
+ FUN = "sum",
+ data = japanese2,
+ subset = age >= thresh)
+B <- 1000L # Number of bootstrap replications
+boot_anova <- numeric(length = B)
+boot_gof <- numeric(length = B)
+for(b in seq_len(B - 1L)){
+ boot_samp <- # Generate bootstrap sample
+ do.call(rbind, #merge data frames
+ apply(db_rtrunc, 1, function(x){ # for each rtrunc and count
+ count <- table( #tabulate count
+ floor( #round down
+ samp_elife( # sample right truncated exponential
+ n = x["count"],
+ scale = model0$par,
+ family = "exp", #null model
+ upper = x["rtrunc"] - thresh,
+ type2 = "ltrt")))
+ data.frame( # return data frame
+ count = as.integer(count),
+ rtrunc = as.numeric(x["rtrunc"]) - thresh,
+ eage = as.integer(names(count)))
+ }))
+ boot_mod0 <- # Fit null model to bootstrap sample
+ with(boot_samp,
+ fit_elife(time = eage,
+ time2 = eage + 1L,
+ rtrunc = rtrunc,
+ type = "interval",
+ event = 3,
+ family = "exp",
+ weights = count))
+ boot_mod1 <- # Fit alternative model to bootstrap sample
+ with(boot_samp,
+ fit_elife(time = eage,
+ time2 = eage + 1L,
+ rtrunc = rtrunc,
+ type = "interval",
+ event = 3,
+ family = "gomp",
+ weights = count))
+ boot_anova[b] <- deviance(boot_mod0) -
+ deviance(boot_mod1)
+}
+# Add original statistic
+boot_anova[B] <- deviance(model1) - deviance(model0)
+# Bootstrap p-value
+(pval <- rank(boot_anova)[B] / B)
+#> [1] 0.001
+\end{verbatim}
+
+The asymptotic approximation is of similar magnitude as the bootstrap \(p\)-value. Both suggest that the more complex Gompertz model provides a significantly better fit.
+
+\subsection{Extreme value analysis}\label{extreme-value-analysis}
+
+Extreme value theory suggests that, in many instances, the limiting conditional distribution of exceedances of a random variable \(Y\) with distribution function \(F\) is generalized Pareto, meaning
+\begin{align}
+\lim_{u \to x^*}\Pr(Y-u > y \mid Y > u)= \begin{cases}
+\left(1+\xi y/\sigma\right)_{+}^{-1/\xi}, & \xi \neq 0;\\
+\exp(-y/\sigma), & \xi = 0;
+\end{cases}
+\label{eq:gpd}
+\end{align}
+with \(x_{+} = \max\{x, 0\}\) and \(x^*=\sup\{x: F(x) < 1\}\). This justifies the use of Equation \eqref{eq:gpd} for the survival function of threshold exceedances when dealing with rare events. The model has two parameters: a scale \(\sigma\) and a shape \(\xi\) which determines the behavior of the upper tail. Negative shape parameters correspond to bounded upper tails and a finite right endpoint for the support.
+
+Study of population dynamics and mortality generally requires knowledge of the total population from which observations are drawn to derive rates. By contrast, the peaks over threshold method, by which one models the \(k\) largest observations of a sample, is a conditional analysis (e.g., given survival until a certain age), and is therefore free of denominator specification since we only model exceedances above a high threshold \(u\). For modelling purposes, we need to pick a threshold \(u\) that is smaller than the upper endpoint \(x^*\) in order to have sufficient number of observations to estimate parameters. The threshold selection problem is a classical instance of bias-variance trade-off: the parameter estimators are possibly biased if the threshold is too low because the generalized Pareto approximation is not good enough, whereas choosing a larger threshold to ensure we are closer to the asymptotic regime leads to reduced sample size and increased parameter uncertainty.
+
+To aid threshold selection, users commonly resort to parameter stability plots. These are common visual diagnostics consisting of a plot of estimates of the shape parameter \(\widehat{\xi}\) (with confidence or credible intervals) based on sample exceedances over a range of thresholds \(u_1, \ldots, u_K\). If the data were drawn from a generalized Pareto distribution, the conditional distribution above higher threshold \(v > u\) is also generalized Pareto with the same shape: this threshold stability property is the basis for extrapolation beyond the range of observed records. Indeed, if the change in estimation of \(\xi\) is nearly constant, this provides reassurance that the approximation can be used for extrapolation. The only difference with survival data, relative to the classical setting, is that the likelihood must account for censoring and truncation. Note that, when we use threshold exceedances with a nonzero threshold (argument \texttt{thresh}), it however isn't possible to unambiguously determine whether left censored observations are still exceedances: such cases yield errors in the functions.
+
+Theory on penultimate extremes suggests that, for finite levels and general distribution function \(F\) for which \eqref{eq:gpd} holds, the shape parameter varies as a function of the threshold \(u\), behaving like the derivative of the reciprocal hazard \(r(x) = \{1-F(x)\}/f(x)\). We can thus model the shape as piece-wise constant by fitting a piece-wise generalized Pareto model due to \citet{Northrop.Coleman:2014} and adapted in \citet{ARSIA:2022} for survival data. The latter can be viewed as a mixture of generalized Pareto over \(K\) disjoint intervals with continuity constraints to ensure a smooth hazard, which reduces to the generalized Pareto if we force the \(K+1\) shape parameters to be equal. We can use a likelihood ratio test to compare the model, or a score test if the latter is too computationally intensive, and plot the \(p\)-values for each of the \(K\) thresholds, corresponding to the null hypotheses \(\mathrm{H}_k: \xi_k = \cdots = \xi_{K}\) (\(k=1, \ldots, K-1\)). As the model quickly becomes overparametrized, optimization is difficult and the score test may be a safer option as it only requires estimation of the null model of a single generalized Pareto over the whole range.
+
+To illustrate these diagnostic tools, Figure \ref{fig:fig-parameterstab} shows a threshold stability plot, which features a small increase in the shape parameters as the threshold increases, corresponding to a stabilization or even a slight decrease of the hazard at higher ages. We can envision a threshold of 108 years as being reasonable: the Northrop--Coleman diagnostic plot suggests lower thresholds are compatible with a constant shape above 100. Additional goodness-of-fit diagnostics are necessary to determine if the generalized Pareto model fits well.
+
+\begin{verbatim}
+par(mfrow = c(1, 2), mar = c(4, 4, 1, 1))
+# Threshold sequence
+u <- 100:110
+# Threshold stability plot
+tstab(arguments = args_japan,
+ family = "gp",
+ method = "profile",
+ which.plot = "shape",
+ thresh = u)
+# Northrop-Coleman diagnostic based on score tests
+nu <- length(u) - 1L
+nc_score <- nc_test(arguments = c(args_japan, list(thresh = u)))
+score_plot <- plot(nc_score)
+graphics.off()
+\end{verbatim}
+
+\begin{figure}
+
+{\centering \includegraphics[width=1\linewidth,alt={Threshold stability plots. The left panel shows shape parameter estimates with 95\% confidence intervals as a function of the threshold value from 100 until 111 years. The right panel shows p-values from a score test for nested models as a function of the same thresholds.}]{RJ-2025-034_files/figure-latex/fig-parameterstab-1}
+
+}
+
+\caption{Threshold diagnostic tools: parameter stability plots for the generalized Pareto model (left) and Northrop--Coleman \(p\)-value path (right) for the Japanese centenarian dataset. Both suggest that a threshold as low as 100 may be suitable for peaks-over-threshold analysis.}\label{fig:fig-parameterstab}
+\end{figure}
+
+Each plot in the package can be produced using base R or using \CRANpkg{ggplot2} \citep{ggplot2}, which implements the grammar of graphics. To keep the list of package dependencies lean and adhere to the \href{https://cran.r-project.org/web/packages/pacs/vignettes/tinyverse.html}{\texttt{tinyverse} principle}, the latter can be obtained by using the argument \texttt{plot.type} with the generic S3 method \texttt{plot}, or via \texttt{autoplot}, provided the \CRANpkg{ggplot2} package is already installed.
+
+\subsection{Graphical goodness of fit diagnostics}\label{graphical-goodness-of-fit-diagnostics}
+
+Determining whether a parametric model fits well to survival data is no easy task due to the difficulty in specifying the null distribution of many goodness of fit statistics, such as the Cramér--von Mises statistic, which differ for survival data. As such, the \CRANpkg{longevity} package relies mostly on visual diagnostic tools. \citet{Waller.Turnbull:1992} discusses how classical visual graphics can be adapted in the presence of censoring. Most notably, only observed failure times are displayed on the \(y\)-axis against their empirical plotting position on the \(x\)-axis. Contrary to the independent and identically distributed case, the uniform plotting positions \(F_n(y_i)\) are based on the nonparametric maximum likelihood estimator discussed in Section 3.
+
+The situation is more complicated with truncated data \citep{ARSIA:2022}, since the data are not identically distributed: indeed, the distribution function of observation \(Y_i\) truncated on the interval \([a_i, b_i]\) is \(F_i(y_i) = \{F(y_i)-F(a_i)\}/\{F(b_i) - F(a_i)\}\), so the data arise from different distributions even if these share common parameters. One way out of this conundrum is using the probability integral transform and the quantile transform to map observations to the uniform scale and back onto the data scale. Taking \(\widetilde{F}(y_i) = F_n(y_i)=\mathrm{rank}(y_i)/(n+1)\) to denote the empirical distribution function estimator, a probability-probability plot would show \(x_i = \widetilde{F}_{i}(y_i)\) against \(y_i = F_i(y_i)\), leading to approximately uniform samples if the parametric distribution \(F\) is suitable. Another option is to standardize the observation, taking the collection \(\widetilde{y}_i=F^{-1}\{F_i(y_i)\}\) of rescaled exceedances and comparing them to the usual plotting positions \(x_{(i)} =\{i/(n+1)\}\). The drawback of the latter approach is that the quantities displayed on the \(y\)-axis are not raw observations and the ranking of the empirical quantiles may change, a somewhat counter-intuitive feature. However, this means that the sample \(\{F_i(y_i)\}\) should be uniform under the null hypothesis, and this allows one to use methods from \citet{Sailynoja.Burkner.Vehtari:2021} to obtain point-wise and simultaneous confidence intervals.
+
+\CRANpkg{longevity} offers users the choice between three types of quantile-quantile plots: regular (Q-Q, \texttt{"qq"}), Tukey's mean detrended Q-Q plots (\texttt{"tmd"}) and exponential Q-Q plots (\texttt{"exp"}). Other options on uniform scale are probability-probability (P-P, \texttt{"pp"}) plots and empirically rescaled plots (ERP, \texttt{"erp"}) \citep{Waller.Turnbull:1992}, designed to ease interpretation with censored observations by rescaling axes. We illustrate the graphical tools with the \texttt{dutch} data above age 105 in Figure \ref{fig:fig-qqplots}. The fit is correct above 110, but there is a notable dip due to excess mortality around 109.
+
+\begin{verbatim}
+fit_dutch <- fit_elife(
+ arguments = dutch_data,
+ event = 3,
+ type = "interval2",
+ family = "gp",
+ thresh = 105,
+ export = TRUE)
+par(mfrow = c(1, 2))
+plot(fit_dutch,
+ which.plot = c("pp","qq"))
+\end{verbatim}
+
+\begin{figure}
+
+{\centering \includegraphics[width=1\linewidth]{RJ-2025-034_files/figure-latex/fig-qqplots-1}
+
+}
+
+\caption{Probability-probability and quantile-quantile plots for generalized Pareto model fitted above age 105 years to Dutch data. The plots indicate broadly good agreement with the observation, except for some individuals who died age 109 for which too many have deaths close to their birthdates.}\label{fig:fig-qqplots}
+\end{figure}
+
+Censored observations are used to compute the plotting positions, but are not displayed. As such, we cannot use graphical goodness of fit diagnostics for the Japanese interval censored data. An alternative, given that data are tabulated in a contingency table, is to use a chi-squared test for independence, conditioning on the number of individuals per birth cohort. The expected number in each cell (birth cohort and age band) can be obtained by computing the conditional probability of falling in that age band. The asymptotic null distribution should be \(\chi^2\) with \((k-1)(p-1)\) degrees of freedom, where \(k\) is the number of age bands and \(p\) the number of birth cohorts. In finite samples, the expected count for large excess lifetimes are very low so one can expect the \(\chi^2\) approximation to be poor. To mitigate this, we can pool observations and resort to simulation to approximate the null distribution of the test statistic. The bootstrap \(p\)-value for the exponential model above 108 years, pooling observations with excess lifetime of 5 years and above, is 0.872, indicating no evidence that the model is inadequate, but the test here may have low power.
+
+\subsection{Stratification}\label{stratification}
+
+Demographers may suspect differences between individuals of different sex, from different countries or geographic areas, or by birth cohort. All of these are instances of categorical covariates. One possibility is to incorporate these covariates with suitable link function through parameters, but we consider instead stratification. We can split the data by levels of \texttt{covariate} (with factors) into sub-data and compare the goodness of fit of the \(K\) models relative to that which pools all observations. The \texttt{test\_elife} function performs likelihood ratio tests for the comparisons. The test statistic is \(-2\{\ell(\widehat{\boldsymbol{\theta}}_0) - \ell(\widehat{\boldsymbol{\theta}})\}\), where \(\widehat{\boldsymbol{\theta}}_0\) is the maximum likelihood estimator under the null model with common parameters, and \(\widehat{\boldsymbol{\theta}}\) is the unrestricted maximum likelihood estimator for the alternative model with the same distribution, but which allows for stratum-specific parameters. We illustrate this with a generalized Pareto model for the excess lifetime. The null hypothesis is \(\mathrm{H}_0: \sigma_{\texttt{f}} = \sigma_{\texttt{m}}, \xi_{\texttt{f}}=\xi_{\texttt{m}}\) against the alternative that at least one equality doesn't hold and so the hazard and endpoints are different.
+
+\begin{verbatim}
+print(
+ test_elife(
+ arguments = args_japan,
+ thresh = 110,
+ family = "gp",
+ covariate = japanese2$gender)
+)
+#> Model: generalized Pareto distribution.
+#> Threshold: 110
+#> Number of exceedances per covariate level:
+#> female male
+#> 642 61
+#>
+#> Likelihood ratio statistic: 0.364
+#> Null distribution: chi-square (2)
+#> Asymptotic p-value: 0.833
+\end{verbatim}
+
+In the present example, there is no evidence against any difference in lifetime distribution between male and female; this is unsurprising given the large imbalance between counts for each covariate level, with far fewer males than females.
+
+\subsection{Extrapolation}\label{extrapolation}
+
+If the maximum likelihood estimator of the shape \(\xi\) for the generalized Pareto model is negative, then the distribution has a finite upper endpoint; otherwise, the latter is infinite. With \(\xi < 0\), we can look at the profile log likelihood for the endpoint \(\eta = -\sigma/\xi\), using the function \texttt{prof\_gp\_endpt}, to draw the curve and obtain confidence intervals. The argument \texttt{psi} is used to give a grid of values over which to compute the profile log likelihood. The bounds of the \((1-\alpha)\) confidence intervals are obtained by fitting a cubic smoothing spline for \(y=\eta\) as a function of the shifted profile curve \(x = 2\{\ell_p(\eta)-\ell_p(\widehat{\eta})\}\) on both sides of the maximum likelihood estimator and predicting the value of \(y\) when \(x = -\chi^2_1(1-\alpha)\). This technique works well unless the profile is nearly flat or the bounds lie beyond the range of values of \texttt{psi} provided; the user may wish to change them if they are too far. If \(\widehat{\xi} \approx 0\), then the upper bound of the confidence interval may be infinite and the profile log likelihood may never reach the cutoff value of the asymptotic \(\chi^2_1\) distribution.
+
+The profile log likelihood curve for the endpoint, shifted vertically so that its value is zero at the maximum likelihood estimator, highlights the marked asymmetry of the distribution of \(\eta\), shown in \ref{fig:fig-endpoint-confint}, with the horizontal dashed lines showing the limits for the 95\% profile likelihood confidence intervals. These suggest that the endpoint, or a potential finite lifespan, could lie very much beyond observed records. The routine used to calculate the upper bound computes the cutoff value by fitting a smoothing spline with the role of the \(y\) and \(x\) axes reversed and by predicting the value of \(\eta\) at \(y=0\). In this example, the upper confidence limit is extrapolated from the model: more accurate measures can be obtained by specifying a longer and finer sequence of values of \texttt{psi} such that the profile log likelihood drops below the \(\chi^2_1\) quantile cutoff.
+
+\begin{verbatim}
+# Create grid of threshold values
+thresholds <- 105:110
+# Grid of values at which to evaluate profile
+psi <- seq(120, 200, length.out = 101)
+# Calculate the profile for the endpoint
+# of the generalized Pareto at each threshold
+endpt_tstab <- do.call(
+ endpoint.tstab,
+ args = c(
+ args_japan,
+ list(psi = psi,
+ thresh = thresholds,
+ plot = FALSE)))
+# Compute corresponding confidence intervals
+profile <- endpoint.profile(
+ arguments = c(args_japan, list(thresh = 110, psi = psi)))
+# Plot point estimates and confidence intervals
+g1 <- autoplot(endpt_tstab, plot = FALSE, ylab = "lifespan (in years)")
+# Plot the profile curve with cutoffs for conf. int. for 110
+g2 <- autoplot(profile, plot = FALSE)
+patchwork::wrap_plots(g1, g2)
+\end{verbatim}
+
+\begin{figure}
+
+{\centering \includegraphics[width=1\linewidth]{RJ-2025-034_files/figure-latex/fig-endpoint-confint-1}
+
+}
+
+\caption{Maximum likelihood estimates with 95\% confidence intervals as a function of threshold (left) and profile likelihood for exceedances above 110 years (right) for Japanese centenarian data. As the threshold increases, the number of exceedances decreases and the intervals for the upper bound become wider. At 110, the right endpoint of the interval would go until infinity.}\label{fig:fig-endpoint-confint}
+\end{figure}
+
+Depending on the model, the conclusions about the risk of mortality change drastically: the Gompertz model implies an ever increasing hazard, but no finite endpoint for the distribution of exceedances. The exponential model implies a constant hazard and no endpoint. By contrast, the generalized Pareto can accommodate both finite and infinite endpoints. The marked asymmetry of the distribution of lifespan defined by the generalized Pareto shows that inference obtained using symmetric confidence intervals (i.e., Wald-based) is likely very misleading: the drop in fit from having a zero or positive shape parameter \(\xi\) is seemingly smaller than the cutoffs for a 95\% confidence interval, suggesting that while the best point estimate is around 128 years, the upper bound is so large (and extrapolated) that everything is possible. The model however also suggests a very high probability of dying in any given year, regardless of whether the hazard is constant, decreasing or increasing.
+
+\subsection{Hazard}\label{hazard}
+
+The parameters of the models are seldom of interest in themselves: rather, we may be interested in a summary such as the hazard function. At present, \CRANpkg{longevity} does not allow general linear modelling of model parameters or time-varying covariates, but other software implementations can tackle this task. For example, \CRANpkg{casebase} \citep{casebase} fits flexible hazard models using logistic or multinomial regression with potential inclusion of penalties for the parameters associated to covariates and splines effects. Another alternative is \CRANpkg{flexsurv} \citep{flexsurv}, which offers 10 parametric models and allows for user-specified models.
+The \CRANpkg{bshazard} package \citep{bshazard} provides nonparametric smoothing via \(B\)-splines, whereas \CRANpkg{muhaz} handles kernel-based hazard for right censored data; both could be used for validation of the parametric models in the case of right censoring. The \CRANpkg{rstpm2} package \citep{rstpm2} handles generalized modelling for censoring with the Royston--Parmar model built from natural cubic splines \citep{Royston.Parmar:2002}. Contrasting with all of the aforementioned approaches, we focus on parametric models: this is partly because there are few observations for the user case we consider and covariates, except perhaps for gender and birth year, are not available.
+
+The hazard changes over time; the only notable exception is the exponential hazard, which is constant. \CRANpkg{longevity} includes utilities for computing the hazard function from a fitted model object and computing point-wise confidence intervals using symmetric Wald intervals or the profile likelihood.
+Specifically, the \texttt{hazard\_elife} function calculates the hazard \(h(t; \boldsymbol{\theta})\) point-wise at times \(t=\)\texttt{x}; Wald-based confidence intervals are obtained using the delta-method, whereas profile likelihood intervals are obtained by reparametrizing the model in terms of \(h(t)\) for each time \(t\). More naturally perhaps, we can consider a Bayesian analysis of the Japanese excess lifetime above 108 years. Using the likelihood and encoded log posterior provided in \texttt{logpost\_elife}, we obtained independent samples from the posterior of the generalized Pareto parameters \((\sigma, \xi)\) with maximal data information prior using the \CRANpkg{rust} package. Each parameter combination was then fed into \texttt{helife} and the hazard evaluated over a range of values. Figure \ref{fig:fig-hazard} shows the posterior samples and functional boxplots \citep{Sun.Genton:2011} of the hazard curves, obtained using the \CRANpkg{fda} package. The risk of dying increases with age, but comes with substantial uncertainty, as evidenced by the increasing width of the boxes and interquartile range.
+
+\begin{figure}
+\includegraphics[width=1\linewidth]{RJ-2025-034_files/figure-latex/fig-hazard-1} \caption{Left: scatterplot of 1000 independent posterior samples from generalized Pareto model with maximum data information prior; the contour curves give the percentiles of credible intervals, and show approximate normality of the posterior. Right: functional boxplots for the corresponding hazard curves, with increasing width at higher ages.}\label{fig:fig-hazard}
+\end{figure}
+
+\subsection{Nonparametric maximum likelihood estimation}\label{sec-nonparametric}
+
+The nonparametric maximum likelihood estimator is unique only up to equivalence classes. The data for individual \(i\) consists of the tuple \(\{L_i, R_i, V_i, U_i\}\), where the censoring interval is \([L_i, R_i]\) and the truncation interval is \([V_i, U_i\}\), with \(0 \leq V_i \leq L_i \leq R_i \leq U_i \leq \infty\). \citet{Turnbull:1976} shows how one can build disjoint intervals \(C = \bigsqcup_{j=1}^m [a_j, b_j]\) where \(a_j \in \mathcal{L} = \{L_1, \ldots, L_n\}\) and \(b_j \in \mathcal{R} = \{R_1, \ldots, R_n\}\) satisfy \(a_1 \leq b_1 < \cdots < a_m \leq b_m\) and the intervals \([a_j, b_j]\) contain no other members of \(L\) or \(R\) except at the endpoints. This last condition notably ensures that the intervals created include all observed failure times as singleton sets in the absence of truncation. Other authors \citep{Lindsey.Ryan:1998} have taken interval censored data as semi-open intervals \((L_i, R_i]\), a convention we adopt here for numerical reasons. For interval censored and truncated data, \citet{Frydman:1994} shows that this construction must be amended by taking instead \(a_j \in \mathcal{L} \cup \{U_1, \ldots, U_n\}\) and \(b_j \in \mathcal{R} \cup \{V_1, \ldots, V_n\}\).
+
+We assign probability \(p_j = F(b_j^{+}) - F(a_j^{-}) \ge 0\) to each of the resulting \(m\) intervals under the constraint \(\sum_{j=1}^m p_j = 1\) and \(p_j \ge 0\) \((j=1, \ldots, m)\). The nonparametric maximum likelihood estimator of the distribution function \(F\) is then
+\begin{align*}
+\widehat{F}(t) = \begin{cases} 0, & t < a_1;\\
+\widehat{p}_1 + \cdots + \widehat{p}_j, & b_j < t < a_{j+1} \quad (1 \leq j \leq m-1);\\
+1, & t > b_m;
+\end{cases}
+\end{align*}
+and is undefined for \(t \in [a_j, b_j]\) \((j=1, \ldots, m)\).
+
+\begin{figure}
+
+{\centering \includegraphics[width=1\linewidth]{RJ-2025-034_files/figure-latex/fig-turnbull-1}
+
+}
+
+\caption{Illustration of the truncation (pale grey) and censoring intervals (dark grey) equivalence classes based on Turnbull's algorithm. Observations must fall within equivalence classes defined by the former.}\label{fig:fig-turnbull}
+\end{figure}
+
+The procedure of Turnbull can be encoded using \(m \times n\) matrices. For censoring, we build \(\mathbf{A}\) whose \((i,j)\)th entry \(\alpha_{ij}=1\) if \([a_j, b_j] \subseteq A_i\) and zero otherwise. Since the intervals forming the set \(C\) are disjoint and in increasing order, a more storage efficient manner of keeping track of the intervals is to find the smallest integer \(j\) such that \(L_i \leq a_j\) and the largest \(R_i \ge b_j\) \((j=1, \ldots, m)\) for each observation. The same idea applies for the truncation sets \(B_i = (V_i, U_i)\) and matrix \(\mathbf{B}\) with \((i,j)\) element \(\beta_{ij}\).
+
+The log likelihood function is
+\begin{align*}
+\ell(\boldsymbol{p}) = \sum_{i=1}^n w_i \log \left( \sum_{j=1}^m \alpha_{ij}p_j\right) - w_i\log \left( \sum_{j=1}^m \beta_{ij}p_j\right)
+\end{align*}
+
+The numerical implementation of the EM is in principle forward: first identify the equivalence classes \(C\), next calculate the entries of \(A_i\) and \(B_i\) (or the vectors of ranges) and finally run the EM algorithm. In the second step, we need to account for potential ties in the presence of (interval) censoring and treat the intervals as open on the left for censored data. For concreteness, consider the toy example \(\boldsymbol{T} =(1,1,2)\) and \(\boldsymbol{\delta} = (0,1,1)\), where \(\delta_i = 1\) if the observation is a failure time and \(\delta_i=0\) in case of right censoring.
+The left and right censoring bounds are \(\mathcal{L} = \{1, 2\}\) and \(\mathcal{R} = \{1, 2, \infty\}\) with \(A_1 = (1, \infty)\), \(A_2 = \{1\}\) and \(A_3 = \{2\}\) and \(C_1=\{1\}, C_2=\{2\}\). If we were to treat instead \(A_1\) as a semi-closed interval \([1, \infty)\), direct maximization of the log likelihood in eq. 2.2 of \citet{Turnbull:1976} would give probability half to each observed failure time. By contrast, the Kaplan--Meier estimator, under the convention that right censored observations at time \(t\) were at risk up to and until \(t\), assigns probability 1/3 to the first failure. To retrieve this solution with Turnbull's EM estimator, we need the convention that \(C_1 \notin A_1\), but this requires comparing the bound with itself. The numerical tolerance in the implementation is taken to be the square root of the machine epsilon for doubles.
+
+The maximum likelihood estimator (MLE) need not be unique, and the EM algorithm is only guaranteed a local maximum. For interval censored data, \citet{Gentleman.Geyer:1994} consider using the Karush--Kuhn--Tucker conditions to determine whether the probability in some intervals is exactly zero and whether the returned value is indeed the MLE.
+
+Due to data scarcity, statistical inference for human lifespan is best conducted using parametric models supported by asymptotic theory, reserving nonparametric estimators to assess goodness of fit. The empirical cumulative hazard for Japanese is very close to linear from early ages, suggesting that the hazard may not be very far from exponential even if more complex models are likely to be favored given the large sample size.
+
+The function \texttt{np\_elife} returns a list with Turnbull's intervals \([a_j, b_j]\) and the probability weights assigned to each, provided the latter are positive. It also contains an object of class \texttt{stepfun} with a weighting argument that defines a cumulative distribution function.
+
+We can use the nonparametric maximum likelihood estimator of the distribution function to assess a fitted parametric model by comparing the density with a binned distribution, or the cumulative distribution function. The distribution function is defined over equivalence classes, which may be isolated observations or intervals. The data interval over which there is non-zero probability of events is broken down into equispaced bins and the probability of failing in the latter estimated nonparametrically based on the distribution function. Alternatively, users can also provide a set of \texttt{breaks}, or the number of bins.
+
+\begin{verbatim}
+ecdf <- np_elife(arguments = args_japan, thresh = 108)
+# Summary statistics, accounting for censoring
+round(summary(ecdf), digits = 2)
+#> Min. 1st Qu. Median Mean 3rd Qu. Max.
+#> 109.00 110.00 111.00 110.01 111.00 118.00
+# Plots of fitted parametric model and nonparametric CDF
+model_gp <- fit_elife(
+ arguments = args_japan,
+ thresh = 108,
+ family = "gp",
+ export = TRUE)
+# ggplot2 plots, wrapped to display side by side
+patchwork::wrap_plots(
+ autoplot(
+ model_gp, # fitted model
+ plot = FALSE, # return list of ggplots
+ which.plot = c("dens", "cdf"),
+ breaks = seq(0L, 8L, by = 1L) # set bins for histogram
+ )
+)
+\end{verbatim}
+
+\begin{figure}
+
+{\centering \includegraphics[width=1\linewidth]{RJ-2025-034_files/figure-latex/fig-ecdf-1}
+
+}
+
+\caption{Nonparametric maximum likelihood estimate of the density (bar plot, left) and distribution function (staircase function, right), with superimposed generalized Pareto fit for excess lifetimes above 108 years. Except for the discreteness inherent to the nonparametric estimates, the two representations broadly agree at year marks.}\label{fig:fig-ecdf}
+\end{figure}
+
+\section{Conclusion}\label{conclusion}
+
+This paper describes the salient features of \CRANpkg{longevity}, explaining the theoretical underpinning of the methods and the design considerations followed when writing the package. While \CRANpkg{longevity} was conceived for modelling lifetimes, the package could be used for applications outside of demography. Survival data in extreme value theory is infrequent yet hardly absent. For example, rainfall observations can be viewed as rounded due to the limited instrumental precision of rain gauges and treated as interval-censored. Some historical records, which are often lower bounds on the real magnitude of natural catastrophes, can be added as right-censored observations in a peaks-over-threshold analysis. In insurance, losses incurred by the company due to liability claims may be right-censored if they exceed the policy cap and are covered by a reinsurance company, or rounded \citep{Belzile.Neslehova:2025}. In climate science, attribution studies often focus on data just after a record-breaking event, and the stopping rule leads to truncation \citep{Miralles.Davison:2023}, which biases results if ignored.
+
+The package has features that are interesting in their own right, including adapted quantile-quantile and other visual goodness of fit diagnostics for model validation. The testing procedures correctly handle tests for restrictions lying on the boundary of the parameter space. Parametric bootstrap procedures for such tests are not straightforward to implement given the heavy reliance on the data generating mechanism and the diversity of possible scenarios: this paper however shows how the utilities of the package can be coupled to ease such estimation and the code in the supplementary material illustrates how this can be extended for goodness of fit testing.
+
+\section*{Acknowledgements}\label{acknowledgements}
+\addcontentsline{toc}{section}{Acknowledgements}
+
+The author thanks four anonymous reviewers for valuable feedback. This research was supported financially by the Natural Sciences and Engineering Research Council of Canada (NSERC) via Discovery Grant RGPIN-2022-05001.
+
+\bibliography{longevity.bib}
+
+\address{%
+Léo R. Belzile\\
+Department of Decision Sciences, HEC Montréal\\%
+3000, chemin de la Côte-Sainte-Catherine\\ Montréal (Québec), Canada\\ H3T 2A7\\
+%
+\url{https://lbelzile.bitbucket.io}\\%
+\textit{ORCiD: \href{https://orcid.org/0000-0002-9135-014X}{0000-0002-9135-014X}}\\%
+\href{mailto:leo.belzile@hec.ca}{\nolinkurl{leo.belzile@hec.ca}}%
+}
diff --git a/_articles/RJ-2025-034/RJ-2025-034.zip b/_articles/RJ-2025-034/RJ-2025-034.zip
new file mode 100644
index 0000000000..ca3580d058
Binary files /dev/null and b/_articles/RJ-2025-034/RJ-2025-034.zip differ
diff --git a/_articles/RJ-2025-034/RJ-2025-034_files/figure-html5/fig-ecdf-1.png b/_articles/RJ-2025-034/RJ-2025-034_files/figure-html5/fig-ecdf-1.png
new file mode 100644
index 0000000000..44f70cdfca
Binary files /dev/null and b/_articles/RJ-2025-034/RJ-2025-034_files/figure-html5/fig-ecdf-1.png differ
diff --git a/_articles/RJ-2025-034/RJ-2025-034_files/figure-html5/fig-endpoint-confint-1.png b/_articles/RJ-2025-034/RJ-2025-034_files/figure-html5/fig-endpoint-confint-1.png
new file mode 100644
index 0000000000..47b44cba86
Binary files /dev/null and b/_articles/RJ-2025-034/RJ-2025-034_files/figure-html5/fig-endpoint-confint-1.png differ
diff --git a/_articles/RJ-2025-034/RJ-2025-034_files/figure-html5/fig-hazard-1.png b/_articles/RJ-2025-034/RJ-2025-034_files/figure-html5/fig-hazard-1.png
new file mode 100644
index 0000000000..d5bc33c9ac
Binary files /dev/null and b/_articles/RJ-2025-034/RJ-2025-034_files/figure-html5/fig-hazard-1.png differ
diff --git a/_articles/RJ-2025-034/RJ-2025-034_files/figure-html5/fig-parameterstab-1.png b/_articles/RJ-2025-034/RJ-2025-034_files/figure-html5/fig-parameterstab-1.png
new file mode 100644
index 0000000000..16d522bec1
Binary files /dev/null and b/_articles/RJ-2025-034/RJ-2025-034_files/figure-html5/fig-parameterstab-1.png differ
diff --git a/_articles/RJ-2025-034/RJ-2025-034_files/figure-html5/fig-qqplots-1.png b/_articles/RJ-2025-034/RJ-2025-034_files/figure-html5/fig-qqplots-1.png
new file mode 100644
index 0000000000..3b7e729359
Binary files /dev/null and b/_articles/RJ-2025-034/RJ-2025-034_files/figure-html5/fig-qqplots-1.png differ
diff --git a/_articles/RJ-2025-034/RJ-2025-034_files/figure-html5/fig-turnbull-1.png b/_articles/RJ-2025-034/RJ-2025-034_files/figure-html5/fig-turnbull-1.png
new file mode 100644
index 0000000000..99292b1711
Binary files /dev/null and b/_articles/RJ-2025-034/RJ-2025-034_files/figure-html5/fig-turnbull-1.png differ
diff --git a/_articles/RJ-2025-034/RJ-2025-034_files/figure-latex/fig-ecdf-1.pdf b/_articles/RJ-2025-034/RJ-2025-034_files/figure-latex/fig-ecdf-1.pdf
new file mode 100644
index 0000000000..d4361874de
Binary files /dev/null and b/_articles/RJ-2025-034/RJ-2025-034_files/figure-latex/fig-ecdf-1.pdf differ
diff --git a/_articles/RJ-2025-034/RJ-2025-034_files/figure-latex/fig-endpoint-confint-1.pdf b/_articles/RJ-2025-034/RJ-2025-034_files/figure-latex/fig-endpoint-confint-1.pdf
new file mode 100644
index 0000000000..b77014293a
Binary files /dev/null and b/_articles/RJ-2025-034/RJ-2025-034_files/figure-latex/fig-endpoint-confint-1.pdf differ
diff --git a/_articles/RJ-2025-034/RJ-2025-034_files/figure-latex/fig-hazard-1.pdf b/_articles/RJ-2025-034/RJ-2025-034_files/figure-latex/fig-hazard-1.pdf
new file mode 100644
index 0000000000..da46f6a2e8
Binary files /dev/null and b/_articles/RJ-2025-034/RJ-2025-034_files/figure-latex/fig-hazard-1.pdf differ
diff --git a/_articles/RJ-2025-034/RJ-2025-034_files/figure-latex/fig-parameterstab-1.pdf b/_articles/RJ-2025-034/RJ-2025-034_files/figure-latex/fig-parameterstab-1.pdf
new file mode 100644
index 0000000000..b87cffac23
Binary files /dev/null and b/_articles/RJ-2025-034/RJ-2025-034_files/figure-latex/fig-parameterstab-1.pdf differ
diff --git a/_articles/RJ-2025-034/RJ-2025-034_files/figure-latex/fig-qqplots-1.pdf b/_articles/RJ-2025-034/RJ-2025-034_files/figure-latex/fig-qqplots-1.pdf
new file mode 100644
index 0000000000..991420134d
Binary files /dev/null and b/_articles/RJ-2025-034/RJ-2025-034_files/figure-latex/fig-qqplots-1.pdf differ
diff --git a/_articles/RJ-2025-034/RJ-2025-034_files/figure-latex/fig-turnbull-1.pdf b/_articles/RJ-2025-034/RJ-2025-034_files/figure-latex/fig-turnbull-1.pdf
new file mode 100644
index 0000000000..0b7aa34d5a
Binary files /dev/null and b/_articles/RJ-2025-034/RJ-2025-034_files/figure-latex/fig-turnbull-1.pdf differ
diff --git a/_articles/RJ-2025-034/RJournal.sty b/_articles/RJ-2025-034/RJournal.sty
new file mode 100644
index 0000000000..351990be38
--- /dev/null
+++ b/_articles/RJ-2025-034/RJournal.sty
@@ -0,0 +1,358 @@
+% Package `RJournal' to use with LaTeX2e
+% Copyright (C) 2010 by the R Foundation
+% Copyright (C) 2013 by the R Journal
+%
+% Originally written by Kurt Hornik and Friedrich Leisch with subsequent
+% edits by the editorial board
+%
+% CAUTION:
+% Do not modify this style file. Any changes to this file will be reset when your
+% article is submitted.
+% If you must modify the style or add LaTeX packages to the article, these
+% should be specified in RJwrapper.tex
+
+\NeedsTeXFormat{LaTeX2e}[1995/12/01]
+\ProvidesPackage{RJournal}[2025/10/05 v0.17 RJournal package]
+
+\RequirePackage{tikz}
+
+% Overall page layout, fonts etc -----------------------------------------------
+
+% Issues of of \emph{The R Journal} are created from the standard \LaTeX{}
+% document class \pkg{report}.
+
+\RequirePackage{geometry}
+\geometry{a4paper,
+ textwidth=14cm, top=1cm, bottom=1cm,
+ includehead,includefoot,centering,
+ footskip=1.5cm}
+\raggedbottom
+\sloppy
+\clubpenalty = 10000
+\widowpenalty = 10000
+\brokenpenalty = 10000
+\usepackage{microtype}
+
+
+\RequirePackage{fancyhdr}
+\fancyhead{}
+\fancyheadoffset{2cm}
+\fancyhead[L]{\textsc{\RJ@sectionhead}}
+\fancyhead[R]{\thepage}
+\fancyfoot{}
+\fancyfoot[L]{The R Journal Vol. \RJ@volume/\RJ@number, \RJ@month~\RJ@year}
+\fancyfoot[R]{ISSN 2073-4859}
+\pagestyle{fancy}
+
+% We use the following fonts (all with T1 encoding):
+%
+% rm & palatino
+% tt & inconsolata
+% sf & helvetica
+% math & palatino
+
+\RequirePackage{microtype}
+
+\RequirePackage[scaled=0.92]{helvet}
+\RequirePackage{palatino,mathpazo}
+\RequirePackage[scaled=1.02]{inconsolata}
+\RequirePackage[T1]{fontenc}
+
+\RequirePackage[hyphens]{url}
+\RequirePackage[pagebackref]{hyperref}
+\renewcommand{\backref}[1]{[p#1]}
+
+% Dark blue colour for all links
+\RequirePackage{color}
+\definecolor{link}{rgb}{0.45,0.51,0.67}
+\hypersetup{
+ colorlinks,%
+ citecolor=link,%
+ filecolor=link,%
+ linkcolor=link,%
+ urlcolor=link
+}
+
+% Give the text a little room to breath
+\setlength{\parskip}{3pt}
+\RequirePackage{setspace}
+\setstretch{1.05}
+
+% Issue and article metadata ---------------------------------------------------
+
+% Basic front matter information about the issue: volume, number, and
+% date.
+
+\newcommand{\volume}[1]{\def\RJ@volume{#1}}
+\newcommand{\volnumber}[1]{\def\RJ@number{#1}}
+\renewcommand{\month}[1]{\def\RJ@month{#1}}
+\renewcommand{\year}[1]{\def\RJ@year{#1}}
+
+
+% Individual articles correspond to
+% chapters, and are contained in |article| environments. This makes it
+% easy to have figures counted within articles and hence hyperlinked
+% correctly.
+
+% An article has an author, a title, and optionally a subtitle. We use
+% the obvious commands for specifying these. Articles will be put in certain
+% journal sections, named by \sectionhead.
+
+\newcommand {\sectionhead} [1]{\def\RJ@sectionhead{#1}}
+\renewcommand{\author} [1]{\def\RJ@author{#1}}
+\renewcommand{\title} [1]{\def\RJ@title{#1}}
+\newcommand {\subtitle} [1]{\def\RJ@subtitle{#1}}
+
+% Control appearance of titles: make slightly smaller than usual, and
+% suppress section numbering. See http://tex.stackexchange.com/questions/69749
+% for why we don't use \setcounter{secnumdepth}{-1}
+
+\usepackage[medium]{titlesec}
+\usepackage{titletoc}
+\titleformat{\section} {\normalfont\large\bfseries}{\arabic{section}}{1em}{}
+\titleformat{\subsection}{\normalfont\normalsize\bfseries}{\arabic{section}.\arabic{subsection}}{0.5em}{}
+\titlecontents{chapter} [0em]{}{}{}{\titlerule*[1em]{.}\contentspage}
+
+% Article layout ---------------------------------------------------------------
+
+% Environment |article| clears the article header information at its beginning.
+% We use |\FloatBarrier| from the placeins package to keep floats within
+% the article.
+\RequirePackage{placeins}
+\newenvironment{article}{\author{}\title{}\subtitle{}\FloatBarrier}{\FloatBarrier}
+
+% Refereed articles should have an abstract, so we redefine |\abstract| to
+% give the desired style
+
+\renewcommand{\abstract}[1]{\noindent\textbf{Abstract} #1}
+\renewenvironment{abstract}{\noindent\textbf{Abstract}~}{}
+
+% The real work is done by a redefined version of |\maketitle|. Note
+% that even though we do not want chapters (articles) numbered, we
+% need to increment the chapter counter, so that figures get correct
+% labelling.
+
+\renewcommand{\maketitle}{%
+\noindent
+ \chapter{\RJ@title}\refstepcounter{chapter}
+ \ifx\empty\RJ@subtitle
+ \else
+ \noindent\textbf{\RJ@subtitle}
+ \par\nobreak\addvspace{\baselineskip}
+ \fi
+ \ifx\empty\RJ@author
+ \else
+ \noindent\textit{\RJ@author}
+ \par\nobreak\addvspace{\baselineskip}
+ \fi
+ \@afterindentfalse\@nobreaktrue\@afterheading
+}
+
+% Now for some ugly redefinitions. We do not want articles to start a
+% new page. (Actually, we do, but this is handled via explicit
+% \newpage
+%
+% The name@of@eq is a hack to get hyperlinks to equations to work
+% within each article, even though there may be multiple eq.(1)
+% \begin{macrocode}
+\renewcommand\chapter{\secdef\RJ@chapter\@schapter}
+\providecommand{\nohyphens}{%
+ \hyphenpenalty=10000\exhyphenpenalty=10000\relax}
+\newcommand{\RJ@chapter}{%
+ \edef\name@of@eq{equation.\@arabic{\c@chapter}}%
+ \renewcommand{\@seccntformat}[1]{}%
+ \@startsection{chapter}{0}{0mm}{%
+ -2\baselineskip \@plus -\baselineskip \@minus -.2ex}{\p@}{%
+ \phantomsection\normalfont\huge\bfseries\raggedright}}
+
+% Book reviews should appear as sections in the text and in the pdf bookmarks,
+% however we wish them to appear as chapters in the TOC. Thus we define an
+% alternative to |\maketitle| for reviews.
+\newcommand{\review}[1]{
+ \pdfbookmark[1]{#1}{#1}
+ \section*{#1}
+ \addtocontents{toc}{\protect\contentsline{chapter}{#1}{\thepage}{#1.1}}
+}
+
+% We want bibliographies as starred sections within articles.
+%
+\RequirePackage[sectionbib,round]{natbib}
+\bibliographystyle{abbrvnat}
+\renewcommand{\bibsection}{\section*{References}}
+
+% Equations, figures and tables are counted within articles, but we do
+% not show the article number. For equations it becomes a bit messy to avoid
+% having hyperref getting it wrong.
+
+% \numberwithin{equation}{chapter}
+\renewcommand{\theequation}{\@arabic\c@equation}
+\renewcommand{\thefigure}{\@arabic\c@figure}
+\renewcommand{\thetable}{\@arabic\c@table}
+
+% Issue layout -----------------------------------------------------------------
+
+% Need to provide our own version of |\tableofcontents|. We use the
+% tikz package to get the rounded rectangle. Notice that |\section*|
+% is really the same as |\chapter*|.
+\renewcommand{\contentsname}{Contents}
+\renewcommand\tableofcontents{%
+ \vspace{1cm}
+ \section*{\contentsname}
+ { \@starttoc{toc} }
+}
+
+\renewcommand{\titlepage}{%
+ \thispagestyle{empty}
+ \hypersetup{
+ pdftitle={The R Journal Volume \RJ@volume/\RJ@number, \RJ@month \RJ@year},%
+ pdfauthor={R Foundation for Statistical Computing},%
+ }
+ \noindent
+ \begin{center}
+ \fontsize{50pt}{50pt}\selectfont
+ The \raisebox{-8pt}{\includegraphics[height=77pt]{Rlogo-5}}\hspace{10pt}
+ Journal
+
+ \end{center}
+ {\large \hfill Volume \RJ@volume/\RJ@number, \RJ@month{} \RJ@year \quad}
+
+ \rule{\textwidth}{1pt}
+ \begin{center}
+ {\Large A peer-reviewed, open-access publication of the \\
+ R Foundation for Statistical Computing}
+ \end{center}
+
+ % And finally, put in the TOC box. Note the way |tocdepth| is adjusted
+ % before and after producing the TOC: thus, we can ensure that only
+ % articles show up in the printed TOC, but that in the PDF version,
+ % bookmarks are created for sections and subsections as well (provided
+ % that the non-starred forms are used).
+ \setcounter{tocdepth}{0}
+ \tableofcontents
+ \setcounter{tocdepth}{2}
+ \clearpage
+}
+
+% Text formatting --------------------------------------------------------------
+
+\newcommand{\R}{R}
+\newcommand{\address}[1]{\addvspace{\baselineskip}\noindent\emph{#1}}
+\newcommand{\email}[1]{\href{mailto:#1}{\normalfont\texttt{#1}}}
+
+% Simple font selection is not good enough. For example, |\texttt{--}|
+% gives `\texttt{--}', i.e., an endash in typewriter font. Hence, we
+% need to turn off ligatures, which currently only happens for commands
+% |\code| and |\samp| and the ones derived from them. Hyphenation is
+% another issue; it should really be turned off inside |\samp|. And
+% most importantly, \LaTeX{} special characters are a nightmare. E.g.,
+% one needs |\~{}| to produce a tilde in a file name marked by |\file|.
+% Perhaps a few years ago, most users would have agreed that this may be
+% unfortunate but should not be changed to ensure consistency. But with
+% the advent of the WWW and the need for getting `|~|' and `|#|' into
+% URLs, commands which only treat the escape and grouping characters
+% specially have gained acceptance
+
+\DeclareRobustCommand\code{\bgroup\@noligs\@codex}
+\def\@codex#1{\texorpdfstring%
+{{\normalfont\ttfamily\hyphenchar\font=-1 #1}}%
+{#1}\egroup}
+\newcommand{\kbd}[1]{{\normalfont\texttt{#1}}}
+\newcommand{\key}[1]{{\normalfont\texttt{\uppercase{#1}}}}
+\DeclareRobustCommand\samp{`\bgroup\@noligs\@sampx}
+\def\@sampx#1{{\normalfont\texttt{#1}}\egroup'}
+\newcommand{\var}[1]{{\normalfont\textsl{#1}}}
+\let\env=\code
+\newcommand{\file}[1]{{`\normalfont\textsf{#1}'}}
+\let\command=\code
+\let\option=\samp
+\newcommand{\dfn}[1]{{\normalfont\textsl{#1}}}
+% \acronym is effectively disabled since not used consistently
+\newcommand{\acronym}[1]{#1}
+\newcommand{\strong}[1]{\texorpdfstring%
+{{\normalfont\fontseries{b}\selectfont #1}}%
+{#1}}
+\let\pkg=\strong
+\newcommand{\CRANpkg}[1]{\href{https://CRAN.R-project.org/package=#1}{\pkg{#1}}}%
+\let\cpkg=\CRANpkg
+\newcommand{\ctv}[1]{\href{https://CRAN.R-project.org/view=#1}{\emph{#1}}}
+\newcommand{\BIOpkg}[1]{\href{https://www.bioconductor.org/packages/release/bioc/html/#1.html}{\pkg{#1}}}
+
+% Example environments ---------------------------------------------------------
+\RequirePackage{fancyvrb}
+\RequirePackage{alltt}
+
+\DefineVerbatimEnvironment{example}{Verbatim}{}
+\renewenvironment{example*}{\begin{alltt}}{\end{alltt}}
+
+% Support for output from Sweave, and generic session style code
+% These used to have fontshape=sl for Sinput/Scode/Sin, but pslatex
+% won't use a condensed font in that case.
+
+% Update (2015-05-28 by DS): remove fontsize=\small to match example environment
+
+\DefineVerbatimEnvironment{Sinput}{Verbatim}{}
+\DefineVerbatimEnvironment{Soutput}{Verbatim}{}
+\DefineVerbatimEnvironment{Scode}{Verbatim}{}
+\DefineVerbatimEnvironment{Sin}{Verbatim}{}
+\DefineVerbatimEnvironment{Sout}{Verbatim}{}
+\newenvironment{Schunk}{}{}
+
+% Mathematics ------------------------------------------------------------------
+
+% The implementation of |\operatorname| is similar to the mechanism
+% \LaTeXe{} uses for functions like sin and cos, and simpler than the
+% one of \AmSLaTeX{}. We use |\providecommand| for the definition in
+% order to keep the one of the \pkg{amstex} if this package has
+% already been loaded.
+% \begin{macrocode}
+\providecommand{\operatorname}[1]{%
+ \mathop{\operator@font#1}\nolimits}
+\RequirePackage{amsfonts}
+
+\renewcommand{\P}{%
+ \mathop{\operator@font I\hspace{-1.5pt}P\hspace{.13pt}}}
+\newcommand{\E}{%
+ \mathop{\operator@font I\hspace{-1.5pt}E\hspace{.13pt}}}
+\newcommand{\VAR}{\operatorname{var}}
+\newcommand{\COV}{\operatorname{cov}}
+\newcommand{\COR}{\operatorname{cor}}
+
+% Figures ----------------------------------------------------------------------
+
+% For use with pandoc > 3.2.1
+\newsavebox\pandoc@box
+\newcommand*\pandocbounded[1]{% scales image to fit in text height/width
+ \sbox\pandoc@box{#1}%
+ \Gscale@div\@tempa{\textheight}{\dimexpr\ht\pandoc@box+\dp\pandoc@box\relax}%
+ \Gscale@div\@tempb{\linewidth}{\wd\pandoc@box}%
+ \ifdim\@tempb\p@<\@tempa\p@\let\@tempa\@tempb\fi% select the smaller of both
+ \ifdim\@tempa\p@<\p@\scalebox{\@tempa}{\usebox\pandoc@box}%
+ \else\usebox{\pandoc@box}%
+ \fi%
+}
+
+\RequirePackage[font=small,labelfont=bf]{caption}
+
+% Wide environments for figures and tables -------------------------------------
+\RequirePackage{environ}
+
+% An easy way to make a figure span the full width of the page
+\NewEnviron{widefigure}[1][]{
+\begin{figure}[#1]
+\advance\leftskip-2cm
+\begin{minipage}{\dimexpr\textwidth+4cm\relax}%
+ \captionsetup{margin=2cm}
+ \BODY
+\end{minipage}%
+\end{figure}
+}
+
+\NewEnviron{widetable}[1][]{
+\begin{table}[#1]
+\advance\leftskip-2cm
+\begin{minipage}{\dimexpr\textwidth+4cm\relax}%
+ \captionsetup{margin=2cm}
+ \BODY
+\end{minipage}%
+\end{table}
+}
diff --git a/_articles/RJ-2025-034/RJwrapper.tex b/_articles/RJ-2025-034/RJwrapper.tex
new file mode 100644
index 0000000000..831fed17b8
--- /dev/null
+++ b/_articles/RJ-2025-034/RJwrapper.tex
@@ -0,0 +1,72 @@
+\documentclass[a4paper]{report}
+\usepackage[utf8]{inputenc}
+\usepackage[T1]{fontenc}
+\usepackage{RJournal}
+\usepackage{amsmath,amssymb,array}
+\usepackage{booktabs}
+
+
+% tightlist command for lists without linebreak
+\providecommand{\tightlist}{%
+ \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}
+
+\usepackage{longtable}
+
+% Always define CSL refs as bib entries are contained in separate doc
+% Pandoc citation processing
+%From Pandoc 3.1.8
+% definitions for citeproc citations
+\NewDocumentCommand\citeproctext{}{}
+\NewDocumentCommand\citeproc{mm}{%
+ \begingroup\def\citeproctext{#2}\cite{#1}\endgroup}
+\makeatletter
+ % allow citations to break across lines
+ \let\@cite@ofmt\@firstofone
+ % avoid brackets around text for \cite:
+ \def\@biblabel#1{}
+ \def\@cite#1#2{{#1\if@tempswa , #2\fi}}
+\makeatother
+\newlength{\cslhangindent}
+\setlength{\cslhangindent}{1.5em}
+\newlength{\csllabelwidth}
+\setlength{\csllabelwidth}{3em}
+\newenvironment{CSLReferences}[2] % #1 hanging-indent, #2 entry-spacing
+ {\begin{list}{}{%
+ \setlength{\itemindent}{0pt}
+ \setlength{\leftmargin}{0pt}
+ \setlength{\parsep}{0pt}
+ % turn on hanging indent if param 1 is 1
+ \ifodd #1
+ \setlength{\leftmargin}{\cslhangindent}
+ \setlength{\itemindent}{-1\cslhangindent}
+ \fi
+ % set entry spacing
+ \setlength{\itemsep}{#2\baselineskip}}}
+ {\end{list}}
+\usepackage{calc}
+\newcommand{\CSLBlock}[1]{#1\hfill\break}
+\newcommand{\CSLLeftMargin}[1]{\parbox[t]{\csllabelwidth}{#1}}
+\newcommand{\CSLRightInline}[1]{\parbox[t]{\linewidth - \csllabelwidth}{#1}\break}
+\newcommand{\CSLIndent}[1]{\hspace{\cslhangindent}#1}
+
+
+\usepackage{amsfonts}
+
+
+\begin{document}
+
+
+%% do not edit, for illustration only
+\sectionhead{Contributed research article}
+\volume{17}
+\volnumber{4}
+\year{2025}
+\month{December}
+\setcounter{page}{37}
+
+\begin{article}
+ \input{RJ-2025-034}
+\end{article}
+
+
+\end{document}
diff --git a/_articles/RJ-2025-034/Rlogo-5.png b/_articles/RJ-2025-034/Rlogo-5.png
new file mode 100644
index 0000000000..1a04f27478
Binary files /dev/null and b/_articles/RJ-2025-034/Rlogo-5.png differ
diff --git a/_articles/RJ-2025-034/fig/Lexis_local_hazard.pdf b/_articles/RJ-2025-034/fig/Lexis_local_hazard.pdf
new file mode 100644
index 0000000000..88fb61cc9a
Binary files /dev/null and b/_articles/RJ-2025-034/fig/Lexis_local_hazard.pdf differ
diff --git a/_articles/RJ-2025-034/fig/Lexis_local_hazard.png b/_articles/RJ-2025-034/fig/Lexis_local_hazard.png
new file mode 100644
index 0000000000..daea63f675
Binary files /dev/null and b/_articles/RJ-2025-034/fig/Lexis_local_hazard.png differ
diff --git a/_articles/RJ-2025-034/fig/Lexis_local_hazard.tex b/_articles/RJ-2025-034/fig/Lexis_local_hazard.tex
new file mode 100644
index 0000000000..ec71ce4538
--- /dev/null
+++ b/_articles/RJ-2025-034/fig/Lexis_local_hazard.tex
@@ -0,0 +1,99 @@
+\documentclass{standalone}
+
+\usepackage{tikz}
+ \usetikzlibrary{shapes,shapes.misc,arrows}
+ \tikzset{cross/.style={cross out, draw=black, minimum size=2*(#1-\pgflinewidth), inner sep=0pt, outer sep=0pt},
+% %default radius will be 1pt.
+ cross/.default={2pt}}
+\usepackage{amsmath}
+\usepackage{fourier}
+
+\begin{document}
+
+\begin{tikzpicture}
+\fill [gray!10] (0,0) rectangle (3,5);
+\fill [gray!30] (0,0) rectangle (3,1);
+\fill [gray!30] (0,2) rectangle (3,3);
+\fill [gray!30] (0,4) rectangle (3,5);
+\draw[dashed](3,-0.1)--(3,5); %vertical right bar
+% \draw[dotted, very thick, color=white](0,3.7)--(3,3.7);
+% \draw[dotted, very thick, color=white](0,1.2)--(3,1.2);
+% \draw(3.8,1.2)--(3.9,1.2) node[right] at (3.9,1.2) {\small $t_B$};
+% \draw(3.8,0.8)--(3.9,0.8) node[right] at (3.9,0.8) {\small $c_2-x_D$};
+% \draw(3.8,1.6)--(3.9,1.6) node[right] at (3.9,1.6) {\small $c_1-x_C$};
+% \draw(3.8,3.2)--(3.9,3.2) node[right] at (3.9,3.2) {\small $c_1-x_A$};
+% \draw(3.8,4.6)--(3.9,4.6) node[right] at (3.9,4.6) {\small $c_2-x_C$};
+% \draw(3.8,3.7)--(3.9,3.7) node[right] at (3.9,3.7) {\small $t_A$};
+% \draw (3.7,0)--(3.9,0) node[right]{\small $0$};
+
+\node[anchor=base,inner sep=0pt, outer sep=0pt] at (0,5.1) {Excess lifetime};
+\draw (0,0)--(3.2,0);
+\draw (0,-0.1)--(0,5);
+\node[below] at(0,-0.1) {$c_1$};
+\node[below] at(3,-0.1) {$c_2$};
+\node[left] at (0, 0) {$0$};
+\draw (-0.1,0)--(0,0);
+\draw (-0.1,1)--(0,1);
+\draw (-0.1,2)--(0,2);
+\draw (-0.1,3)--(0,3);
+\draw (-0.1,4)--(0,4);
+\node[left] at (0, 1) {$a_1$};
+\node[left] at (0, 2) {$a_2$};
+\node[left] at (0, 3) {$a_3$};
+\node[left] at (0, 4) {$a_4$};
+% \draw (2.2,0)--(2.2,-0.1) ;
+% \draw (0.4,0)--(0.4,-0.1);
+% \draw (-3.2,0)--(-3.2,-0.1);
+% Draw C
+\draw (0,1.6)-- (3,4.6);
+\draw[dashed] (3,4.6) -- (3.3,4.9);
+\draw[fill=black] (3,4.6) circle (1pt) ;
+\node[right] at (3,4.6) {$C$};
+
+% Draw D
+\node[above left] at (-0.2,0.8) {};
+\draw (2.2,0)-- (3,0.8);
+\draw[fill=black] (3,0.8) circle (1pt);
+\draw[dashed] (3,0.8) -- (3.3,1.1);
+\draw (0,3.2)-- (0.5,3.7);
+\draw (0.5,3.7) node[cross,rotate=45] {};
+\node[right] at (0.5,3.7) {$A$};
+\draw (0.4,0)-- (1.6,1.2);
+\draw (1.6,1.2) node[cross,rotate=45] {};
+\node[right] at (1.6,1.2) {$B$};
+
+\draw (1.1,0) -- (1.8, 0.7);
+\draw (1.8,0.7) node[cross,rotate=45] {};
+
+
+
+\draw (0,3) -- (0.2, 3.2);
+\draw (0.2, 3.2) node[cross,rotate=45] {};
+
+\draw (2.8,0) -- (2.95, 0.15);
+\draw (2.95, 0.15) node[cross,rotate=45] {};
+
+
+\node [below] at (3,-0.8){calendar time};
+% \draw[->,>=stealth](-0.5,0)--(-0.5,5);
+\end{tikzpicture}
+
+\end{document}
+{\scalebox{0.5}{
+\begin{tikzpicture}[ every node/.style={scale=2}]
+\path[clip] (-6,-2) rectangle (5.5,10.5);
+\fill [gray!20] (-5,0) rectangle (0,10);
+\fill [gray!10] (0,0) rectangle (3,10);
+\draw[->](-5,0) -- (3.5,0);
+\draw[->](-5,-0) -- (-5, 10.5);
+\draw(-5,1) -- (2,8) node[cross,rotate=45] {};;
+\draw[dashed, color = white](2,8)--(4,10);
+\draw(-5,3) -- (-3,5) node[cross,rotate=45] {};;
+\node[left] at(-5,0){\footnotesize };
+\draw[dashed, color = white, thick](-3,5)--(2,10);
+\node[below] at(-5,-0.1) {$c_1$};
+\node[below] at(3,-0.1) {$c_2$};
+\node[below] at(0,-0.1) {$m$};
+\node[anchor=base,inner sep=0pt, outer sep=0pt] at (-2,10) {Excess lifetime};
+\end{tikzpicture}
+}}
diff --git a/_articles/RJ-2025-034/fig/nesting_graph.pdf b/_articles/RJ-2025-034/fig/nesting_graph.pdf
new file mode 100644
index 0000000000..d05790ba43
Binary files /dev/null and b/_articles/RJ-2025-034/fig/nesting_graph.pdf differ
diff --git a/_articles/RJ-2025-034/fig/nesting_graph.png b/_articles/RJ-2025-034/fig/nesting_graph.png
new file mode 100644
index 0000000000..e813e0d8b7
Binary files /dev/null and b/_articles/RJ-2025-034/fig/nesting_graph.png differ
diff --git a/_articles/RJ-2025-034/fig/nesting_graph.tex b/_articles/RJ-2025-034/fig/nesting_graph.tex
new file mode 100644
index 0000000000..e93cb76713
--- /dev/null
+++ b/_articles/RJ-2025-034/fig/nesting_graph.tex
@@ -0,0 +1,75 @@
+\documentclass{standalone}
+\usepackage{tikz}
+\RequirePackage{palatino,mathpazo}
+\RequirePackage[scaled=1.02]{inconsolata}
+\RequirePackage[T1]{fontenc}
+\begin{document}
+% This code uses the tikz package
+\begin{tikzpicture}[scale=2]
+\node (v0) at (0.00,0.00) {\texttt{exp}};
+\node (v1) at (-1.00,-1.00) {\texttt{gomp}};
+\node (v2) at (1.50,0.00) {\texttt{weibull}};
+\node (v3) at (1.00,-1.00) {\texttt{gp}};
+\node (v4) at (0.00,-2.00) {\texttt{extgp}};
+\node (v5) at (-2.00,-2.00) {\texttt{gompmake}};
+\node (v6) at (2,-2.00) {\texttt{gppiece}};
+\node(v7) at (-1.5, 0) {\texttt{perks}};
+\node(v8) at (-3.5, -0.5) {\texttt{perksmake}};
+\node(v9) at (-2.5, -1) {\texttt{beard}};
+\node[left](v10) at (-3.5, -2) {\texttt{beardmake}};
+\node [right](v11) at (2.0,-1) {\texttt{extweibull}};
+\draw [-stealth, dashed] (v1) edge (v0);
+\draw [-stealth] (v2) edge (v0);
+\draw [-stealth] (v3) edge (v0);
+\draw [-stealth, dashed] (v8) edge (v7);
+\draw [-stealth, dashed] (v10) edge (v9);
+\draw [-stealth, dashed] (v10) edge (v5);
+\draw [-stealth, dashed] (v9) edge (v1);
+\draw [-stealth, dashed] (v7) edge (v0);
+\draw [-stealth] (v10) edge (v8);
+\draw [-stealth] (v9) edge (v7);
+\draw [-stealth] (v4) edge (v1);
+\draw [-stealth] (v11) edge (v2);
+\draw [-stealth] (v11) edge (v3);
+\draw [-stealth, dashed] (v5) edge (v1);
+\draw [-stealth, dashed] (v4) edge (v3);
+\draw [-stealth] (v6) edge (v3);
+\node[scale=0.75,fill=white] at (-0.5,-0.5) {$\beta=0$}; %Gomp vs exp
+\node[scale=0.75,fill=white] at (-0.75,0) {$\nu=0$}; %Perk vs exp
+\node[scale=0.75,fill=white] at (-2,-0.5) {$\beta=1$}; %beard vs perks
+\node[scale=0.75,fill=white] at (-3.75,-1.25) {$\beta=1$};
+\node[scale=0.75,fill=white] at (-3.25,-1.5) {$\lambda=0$};
+\node[scale=0.75,fill=white] at (-1.75,-1) {$\beta=0$};
+\node[scale=0.75,fill=white] at (0.75,0) {$\alpha=1$};
+\node[scale=0.75,fill=white] at (1.6,-1) {$\alpha=1$};
+\node[scale=0.75,fill=white] at (0.5,-0.5) {$\xi=0$};
+\node[scale=0.75,fill=white] at (2,-0.5) {$\xi=0$};
+\node[scale=0.75,fill=white] at (-1.5,-1.5) {$\lambda=0$};
+\node[scale=0.75,fill=white] at (-3,-2) {$\beta=0$};
+\node[scale=0.75,fill=white] at (-2.5,-0.25) {$\lambda=0$};
+\node[scale=0.75,fill=white] at (1.5,-1.5) {$\xi_1=\cdots =\xi_K$};
+\node[scale=0.75,fill=white] at (-0.5,-1.5) {$\xi=0$};
+\node[scale=0.75,fill=white] at (0.5,-1.5) {$\beta=0$};
+\end{tikzpicture}
+\end{document}
+dag <- dagitty::dagitty('dag {
+A [pos="0.000,0.000",label="exp"]
+B [pos="-1.000,1.000",label="gomp"]
+C [pos="1.000,0.000",label="weibull"]
+D [pos="1.000,1.000",label="gp"]
+E [pos="0.000,2.000",label="extgp"]
+F [pos="-2.000,2.000",label="gompmake"]
+G [pos="2.000,2.000",label="gppiece"]
+H[pos="1.000,1.000",label="extweibull"]
+A -> B
+A -> C
+A -> D
+B -> E
+B -> F
+D -> E
+D -> G
+C -> H
+G -> H
+}'
+)
+plot(dag)
diff --git a/_articles/RJ-2025-034/longevity.bib b/_articles/RJ-2025-034/longevity.bib
new file mode 100644
index 0000000000..fd3a9203a0
--- /dev/null
+++ b/_articles/RJ-2025-034/longevity.bib
@@ -0,0 +1,780 @@
+@article{Perks:1932,
+ title = {On Some Experiments in the Graduation of Mortality Statistics},
+ author = {Perks, Wilfred},
+ year = 1932,
+ journal = {Journal of the Institute of Actuaries},
+ publisher = {Cambridge University Press},
+ volume = 63,
+ number = 1,
+ pages = {12--57},
+ doi = {10.1017/s0020268100046680}
+}
+@article{Chernoff:1954,
+ title = {On the Distribution of the Likelihood Ratio},
+ author = {Herman Chernoff},
+ year = 1954,
+ journal = {The Annals of Mathematical Statistics},
+ publisher = {Institute of Mathematical Statistics},
+ volume = 25,
+ number = 3,
+ pages = {573--578},
+ doi = {10.1214/aoms/1177728725},
+ url = {https://doi.org/10.1214/aoms/1177728725}
+}
+@article{Kaplan.Meier:1958,
+ title = {Nonparametric estimation from incomplete observations},
+ author = {Edward L. Kaplan and Paul Meier},
+ year = 1958,
+ journal = {Journal of the American Statistical Association},
+ volume = 53,
+ pages = {457--481},
+ doi = {10.1080/01621459.1958.10501452},
+ url = {https://doi.org/10.1080/01621459.1958.10501452}
+}
+@article{Lynden-Bell:1971,
+ title = {A Method of Allowing for Known Observational Selection in Small Samples Applied to {3CR} Quasars},
+ author = {Lynden-Bell, D.},
+ year = 1971,
+ journal = {Monthly Notices of the Royal Astronomical Society},
+ volume = 155,
+ number = 1,
+ pages = {95--118},
+ doi = {10.1093/mnras/155.1.95},
+ issn = {0035-8711},
+ url = {https://doi.org/10.1093/mnras/155.1.95}
+}
+@article{Turnbull:1976,
+ title = {The empirical distribution function with arbitrarily grouped, censored and truncated data},
+ author = {Bruce W. Turnbull},
+ year = 1976,
+ journal = {Journal of the Royal Statistical Society, Series B},
+ volume = 38,
+ pages = {290--295},
+ doi = {10.1111/j.2517-6161.1976.tb01597.x},
+ url = {https://doi.org/10.1111/j.2517-6161.1976.tb01597.x}
+}
+@article{Self.Liang:1987,
+ title = {Asymptotic Properties of Maximum Likelihood Estimators and Likelihood Ratio Tests under Nonstandard Conditions},
+ author = {Steven G. Self and Kung-Yee Liang},
+ year = 1987,
+ journal = {Journal of the American Statistical Association},
+ publisher = {Taylor \& Francis},
+ volume = 82,
+ number = 398,
+ pages = {605--610},
+ doi = {10.1080/01621459.1987.10478472},
+ url = {https://doi.org/10.1080/01621459.1987.10478472}
+}
+@article{Tsai.Jewell.Wang:1987,
+ title = {A Note on the Product-Limit Estimator Under Right Censoring and Left Truncation},
+ author = {Wei-Yann Tsai and Nicholas P. Jewell and Mei-Cheng Wang},
+ year = 1987,
+ journal = {Biometrika},
+ volume = 74,
+ number = 4,
+ pages = {883--886},
+ doi = {10.1093/biomet/74.4.883},
+ url = {https://doi.org/10.1093/biomet/74.4.883}
+}
+@article{Aragon.Eberly:1992,
+ title = {On Convergence of Convex Minorant Algorithms for Distribution Estimation with Interval-Censored Data},
+ author = {Jorge Arag\'{o}n and David Eberly},
+ year = 1992,
+ journal = {Journal of Computational and Graphical Statistics},
+ publisher = {Taylor \& Francis},
+ volume = 1,
+ number = 2,
+ pages = {129--140},
+ doi = {10.1080/10618600.1992.10477009},
+ url = {https://doi.org/10.1080/10618600.1992.10477009}
+}
+@article{Waller.Turnbull:1992,
+ title = {Probability plotting with censored data},
+ author = {Lance A. Waller and Bruce W. Turnbull},
+ year = 1992,
+ journal = {American Statistician},
+ volume = 46,
+ number = 1,
+ pages = {5--12},
+ doi = {10.1080/00031305.1992.10475837},
+ url = {https://doi.org/10.1080/00031305.1992.10475837}
+}
+@article{Gentleman.Geyer:1994,
+ title = {Maximum likelihood for interval censored data: Consistency and computation},
+ author = {Gentleman, Robert and Charles J. Geyer},
+ year = 1994,
+ journal = {Biometrika},
+ volume = 81,
+ number = 3,
+ pages = {618--623},
+ doi = {10.1093/biomet/81.3.618},
+ issn = {0006-3444},
+ url = {https://doi.org/10.1093/biomet/81.3.618}
+}
+@article{Frydman:1994,
+ title = {A Note on Nonparametric Estimation of the Distribution Function from Interval-Censored and Truncated Observations},
+ author = {Halina Frydman},
+ year = 1994,
+ journal = {Journal of the Royal Statistical Society. Series B (Methodological)},
+ publisher = {[Royal Statistical Society, Wiley]},
+ volume = 56,
+ number = 1,
+ pages = {71--74},
+ doi = {10.1111/j.2517-6161.1994.tb01960.x},
+ issn = {00359246},
+ url = {https://doi.org/10.1111/j.2517-6161.1994.tb01960.x}
+}
+@article{Ihaka.Gentleman:1996,
+ title = {{R}: A Language for Data Analysis and Graphics},
+ author = {Ihaka, Ross and Gentleman, Robert},
+ year = 1996,
+ journal = {Journal of Computational and Graphical Statistics},
+ volume = 5,
+ number = 3,
+ pages = {299--314},
+ doi = {10.1080/10618600.1996.10474713},
+ url = {https://doi.org/10.1080/10618600.1996.10474713}
+}
+@article{Yee.Wild:1996,
+ title = {Vector Generalized Additive Models},
+ author = {Yee, Thomas W. and Wild, C. J.},
+ year = 1996,
+ journal = {Journal of the Royal Statistical Society: Series B (Methodological)},
+ volume = 58,
+ number = 3,
+ pages = {481--493},
+ doi = {10.1111/j.2517-6161.1996.tb02095.x},
+ url = {https://doi.org/10.1111/j.2517-6161.1996.tb02095.x}
+}
+@article{Lindsey.Ryan:1998,
+ title = {Methods for interval-censored data},
+ author = {Lindsey, Jane C. and Ryan, Louise M.},
+ year = 1998,
+ journal = {Statistics in Medicine},
+ volume = 17,
+ number = 2,
+ pages = {219--238},
+ doi = {10.1002/(sici)1097-0258(19980130)17:2<219::aid-sim735>3.0.co;2-o},
+ url = {https://doi.org/10.1002/(SICI)1097-0258(19980130)17:2<219::AID-SIM735>3.0.CO;2-O}
+}
+@article{Efron.Petrosian:1999,
+ title = {Nonparametric Methods for Doubly Truncated Data},
+ author = {Bradley Efron and Vahe Petrosian},
+ year = 1999,
+ journal = {Journal of the American Statistical Association},
+ volume = 94,
+ number = 447,
+ pages = {824--834},
+ doi = {10.1080/01621459.1999.10474187},
+ url = {https://doi.org/10.1080/01621459.1999.10474187}
+}
+@article{Anderson:2000,
+ title = {A vitality-based model relating stressors and environmental properties to organism survival},
+ author = {Anderson, James J.},
+ year = 2000,
+ journal = {Ecological Monographs},
+ volume = 70,
+ number = 3,
+ pages = {445--470},
+ doi = {10.1890/0012-9615(2000)070[0445:avbmrs]2.0.co;2},
+ url = {https://doi.org/10.1890/0012-9615(2000)070[0445:AVBMRS]2.0.CO;2}
+}
+@article{Rotnitzky:2000,
+ title = {Likelihood-based inference with singular information matrix},
+ author = {Andrea Rotnitzky and David R. Cox and Matteo Bottai and James Robins},
+ year = 2000,
+ journal = {Bernoulli},
+ publisher = {Bernoulli Society for Mathematical Statistics and Probability},
+ volume = 6,
+ number = 2,
+ pages = {243 -- 284},
+ doi = {10.2307/3318576},
+ url = {https://doi.org/10.2307/3318576}
+}
+@article{evd,
+ title = {{evd}: Extreme Value Distributions},
+ author = {A. G. Stephenson},
+ year = 2002,
+ journal = {R News},
+ volume = 2,
+ number = 2,
+ url = {https://cran.r-project.org/doc/Rnews/Rnews_2002-2.pdf}
+}
+@article{Royston.Parmar:2002,
+ title = {Flexible parametric proportional-hazards and proportional-odds models for censored survival data, with application to prognostic modelling and estimation of treatment effects},
+ author = {Royston, Patrick and Parmar, Mahesh K. B.},
+ year = 2002,
+ journal = {Statistics in Medicine},
+ volume = 21,
+ number = 15,
+ pages = {2175--2197},
+ doi = {10.1002/sim.1203},
+ url = {https://doi.org/10.1002/sim.1203}
+}
+@article{Li.Anderson:2009,
+ title = {The vitality model: A way to understand population survival and demographic heterogeneity},
+ author = {Ting Li and James J. Anderson},
+ year = 2009,
+ journal = {Theoretical Population Biology},
+ volume = 76,
+ number = 2,
+ pages = {118--131},
+ doi = {10.1016/j.tpb.2009.05.004},
+ issn = {0040-5809},
+ url = {https://doi.org/10.1016/j.tpb.2009.05.004}
+}
+@article{interval,
+ title = {Exact and Asymptotic Weighted Logrank Tests for Interval Censored Data: The {interval} {R} Package},
+ author = {Fay, Michael P. and Shaw, Pamela A.},
+ year = 2010,
+ journal = {Journal of Statistical Software},
+ volume = 36,
+ number = 2,
+ pages = {1--34},
+ doi = {10.18637/jss.v036.i02},
+ url = {https://doi.org/10.18637/jss.v036.i02}
+}
+@article{Shen:2010,
+ title = {Nonparametric analysis of doubly truncated data},
+ author = {Shen, Pao-Sheng},
+ year = 2010,
+ journal = {Annals of the Institute of Statistical Mathematics},
+ volume = 62,
+ number = 5,
+ pages = {835--853},
+ doi = {10.1007/s10463-008-0192-2},
+ issn = {1572-9052},
+ url = {https://doi.org/10.1007/s10463-008-0192-2}
+}
+@article{Watanabe:2010,
+ title = {Asymptotic Equivalence of {B}ayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory},
+ author = {Watanabe, Sumio},
+ year = 2010,
+ journal = {Journal of Machine Learning Research},
+ publisher = {JMLR.org},
+ volume = 11,
+ pages = {3571--3594},
+ issn = {1532-4435},
+ url = {https://dl.acm.org/doi/10.5555/1756006.1953045}
+}
+@article{lubridate-package,
+ title = {Dates and Times Made Easy with {lubridate}},
+ author = {Garrett Grolemund and Hadley Wickham},
+ year = 2011,
+ journal = {Journal of Statistical Software},
+ volume = 40,
+ number = 3,
+ pages = {1--25},
+ doi = {10.18637/jss.v040.i03},
+ url = {https://www.jstatsoft.org/v40/i03/}
+}
+@article{Sun.Genton:2011,
+ title = {Functional Boxplots},
+ author = {Ying Sun and Marc G. Genton},
+ year = 2011,
+ journal = {Journal of Computational and Graphical Statistics},
+ publisher = {Taylor \& Francis},
+ volume = 20,
+ number = 2,
+ pages = {316--334},
+ doi = {10.1198/jcgs.2011.09224}
+}
+@article{Richards:2012,
+ title = {A handbook of parametric survival models for actuarial use},
+ author = {Stephens J. Richards},
+ year = 2012,
+ journal = {Scandinavian Actuarial Journal},
+ publisher = {Taylor \& Francis},
+ volume = 2012,
+ number = 4,
+ pages = {233--257},
+ doi = {10.1080/03461238.2010.506688},
+ url = {https://doi.org/10.1080/03461238.2010.506688}
+}
+@article{Austin.Simon.Betensky:2014,
+ title = {Computationally simple estimation and improved efficiency for special cases of double truncation},
+ author = {Austin, Matthew D. and Simon, David K. and Betensky, Rebecca A.},
+ year = 2014,
+ day = {01},
+ journal = {Lifetime Data Analysis},
+ volume = 20,
+ number = 3,
+ pages = {335--354},
+ doi = {10.1007/s10985-013-9287-z},
+ url = {https://doi.org/10.1007/s10985-013-9287-z}
+}
+@article{bshazard,
+ title = {{bshazard}: A Flexible Tool for Nonparametric Smoothing of the Hazard Function},
+ author = {Paola Rebora and Agus Salim and Marie Reilly},
+ year = 2014,
+ journal = {The R Journal},
+ volume = 6,
+ number = 2,
+ pages = {114--122},
+ doi = {10.32614/rj-2014-028},
+ url = {https://doi.org/10.32614/RJ-2014-028}
+}
+@article{Northrop.Coleman:2014,
+ title = {Improved diagnostic plots for extreme value analyses},
+ author = {Paul J. Northrop and Claire L. Coleman},
+ year = 2014,
+ journal = {Extremes},
+ volume = 17,
+ pages = {289--303},
+ doi = {10.1007/s10687-014-0183-z},
+ url = {https://doi.org/10.1007/s10687-014-0183-z}
+}
+@article{fitdistrplus-package,
+ title = {{fitdistrplus}: An {R} Package for Fitting Distributions},
+ author = {Marie Laure Delignette-Muller and Christophe Dutang},
+ year = 2015,
+ journal = {Journal of Statistical Software},
+ volume = 64,
+ number = 4,
+ pages = {1--34},
+ doi = {10.18637/jss.v064.i04}
+}
+@article{extRemes,
+ title = {{extRemes} 2.0: An Extreme Value Analysis Package in {R}},
+ author = {Eric Gilleland and Richard W. Katz},
+ year = 2016,
+ journal = {Journal of Statistical Software},
+ volume = 72,
+ number = 8,
+ pages = {1--39},
+ doi = {10.18637/jss.v072.i08}
+}
+@article{flexsurv,
+ title = {{flexsurv}: A Platform for Parametric Survival Modeling in {R}},
+ author = {Jackson, Christopher},
+ year = 2016,
+ journal = {Journal of Statistical Software},
+ volume = 70,
+ number = 8,
+ pages = {1--33},
+ doi = {10.18637/jss.v070.i08},
+ url = {https://www.jstatsoft.org/index.php/jss/article/view/v070i08}
+}
+@article{Anderson-Bergman:2017,
+ title = {An Efficient Implementation of the {EMICM} Algorithm for the Interval Censored {NPMLE}},
+ author = {Clifford Anderson-Bergman},
+ year = 2017,
+ journal = {Journal of Computational and Graphical Statistics},
+ publisher = {Taylor \& Francis},
+ volume = 26,
+ number = 2,
+ pages = {463--467},
+ doi = {10.1080/10618600.2016.1208616},
+ url = {https://doi.org/10.1080/10618600.2016.1208616}
+}
+@article{icensReg,
+ title = {{icenReg}: Regression Models for Interval Censored Data in {R}},
+ author = {Clifford Anderson-Bergman},
+ year = 2017,
+ journal = {Journal of Statistical Software},
+ volume = 81,
+ number = 12,
+ pages = {1--23},
+ doi = {10.18637/jss.v081.i12},
+ url = {https://doi.org/10.18637/jss.v081.i12}
+}
+@article{Barbi:2018,
+ title = {The plateau of human mortality: Demography of longevity pioneers},
+ author = {Barbi, Elisabetta and Lagona, Francesco and Marsili, Marco and Vaupel, James W. and Wachter, Kenneth W.},
+ year = 2018,
+ journal = {Science},
+ publisher = {American Association for the Advancement of Science},
+ volume = 360,
+ number = 6396,
+ pages = {1459--1461},
+ doi = {10.1126/science.aat3119},
+ issn = {0036-8075},
+ url = {https://doi.org/10.1126/science.aat3119}
+}
+@article{Peto:1973,
+ title = {Experimental Survival Curves for Interval-Censored Data},
+ author = {Peto, Richard},
+ year = 2018,
+ journal = {Journal of the Royal Statistical Society Series C: Applied Statistics},
+ volume = 22,
+ number = 1,
+ pages = {86--91},
+ doi = {10.2307/2346307},
+ issn = {0035-9254},
+ url = {https://doi.org/10.2307/2346307}
+}
+@article{rstpm2,
+ title = {Parametric and penalized generalized survival models},
+ author = {Xing-Rong Liu and Yudi Pawitan and Mark Clements},
+ year = 2018,
+ journal = {Statistical Methods in Medical Research},
+ volume = 27,
+ number = 5,
+ pages = {1531--1546},
+ doi = {10.1177/0962280216664760},
+ url = {https://doi.org/10.1177/0962280216664760}
+}
+@article{Einmahl:2019,
+ title = {Limits to Human Life Span Through Extreme Value Theory},
+ author = {Einmahl, Jesson J. and Einmahl, John H. J. and de Haan, Laurens},
+ year = 2019,
+ journal = {Journal of the American Statistical Association},
+ publisher = {Taylor \& Francis},
+ volume = 114,
+ number = 527,
+ pages = {1075--1080},
+ doi = {10.1080/01621459.2018.1537912},
+ url = {https://doi.org/10.1080/01621459.2018.1537912}
+}
+@article{Belzile:2021,
+ title = {Human mortality at extreme age},
+ author = {Belzile, L\'{e}o R. and Anthony C. Davison and Holger Rootz\'en and Dmitrii Zholud},
+ year = 2021,
+ journal = {Royal Society Open Science},
+ volume = 202097,
+ doi = {10.1098/rsos.202097},
+ url = {https://doi.org/10.1098/rsos.202097}
+}
+@article{ARSIA:2022,
+ title = {Is there a cap on longevity? {A} statistical review},
+ author = {Belzile, L\'{e}o R. and Anthony C. Davison and Jutta Gampe and Holger Rootz\'en and Dmitrii Zholud},
+ year = 2022,
+ journal = {Annual Review of Statistics and its Application},
+ volume = 9,
+ pages = {22--45},
+ doi = {10.1146/annurev-statistics-040120-025426},
+ url = {https://doi.org/10.1146/annurev-statistics-040120-025426}
+}
+@article{casebase,
+ title = {{casebase}: An Alternative Framework for Survival Analysis and Comparison of Event Rates},
+ author = {Bhatnagar, Sahir Rai and Turgeon, Maxime and Islam, Jesse and Hanley, James A. and Saarela, Olli},
+ year = 2022,
+ journal = {The R Journal},
+ volume = 14,
+ pages = {59--79},
+ doi = {10.32614/rj-2022-052},
+ issn = {2073-4859},
+ url = {https://doi.org/10.32614/RJ-2022-052},
+ issue = 3
+}
+@article{Camarda:2022,
+ title = {The curse of the plateau. Measuring confidence in human mortality estimates at extreme ages},
+ author = {Carlo Giovanni Camarda},
+ year = 2022,
+ journal = {Theoretical Population Biology},
+ volume = 144,
+ pages = {24--36},
+ doi = {10.1016/j.tpb.2022.01.002},
+ url = {https://doi.org/10.1016/j.tpb.2022.01.002}
+}
+@article{Sailynoja.Burkner.Vehtari:2021,
+ title = {Graphical Test for Discrete Uniformity and its Applications in Goodness of Fit Evaluation and Multiple Sample Comparison},
+ author = {S\"{a}ilynoja, Teemu and B\"{u}rkner, Paul-Christian and Vehtari, Aki},
+ year = 2022,
+ day = 24,
+ journal = {Statistics and Computing},
+ volume = 32,
+ number = 2,
+ pages = 32,
+ doi = {10.1007/s11222-022-10090-6},
+ url = {https://doi.org/10.1007/s11222-022-10090-6}
+}
+@article{evgam,
+ title = {{evgam}: An {R} Package for Generalized Additive Extreme Value Models},
+ author = {Youngman, Benjamin D.},
+ year = 2022,
+ journal = {Journal of Statistical Software},
+ volume = 103,
+ number = 3,
+ pages = {1--26},
+ doi = {10.18637/jss.v103.i03},
+ url = {https://www.jstatsoft.org/index.php/jss/article/view/v103i03}
+}
+@article{Belzile.Dutang.Northrop.Opitz:2022,
+ title = {A modeler's guide to extreme value software},
+ author = {Belzile, L\'{e}o R. and Dutang, Christophe and Northrop, Paul J. and Opitz, Thomas},
+ year = 2023,
+ journal = {Extremes},
+ publisher = {Springer},
+ volume = 26,
+ pages = {595--638},
+ doi = {10.1007/s10687-023-00475-9},
+ url = {https://doi.org/10.1007/s10687-023-00475-9}
+}
+@book{Devroye:1986,
+ title = {Non-Uniform Random Variate Generation},
+ author = {Devroye, L.},
+ year = 1986,
+ publisher = {Springer},
+ address = {New York},
+ url = {http://www.nrbook.com/devroye/},
+ page = 843
+}
+@book{Groeneboom.Wellner:1992,
+ title = {Information Bounds and Nonparametric Maximum Likelihood Estimation},
+ author = {Piet Groeneboom and Jon A. Wellner},
+ year = 1992,
+ address = {Basel, Switzerland},
+ pages = 128,
+ doi = {10.1007/978-3-0348-8621-5},
+ isbn = {978-3-7643-2794-1},
+ url = {https://doi.org/10.1007/978-3-0348-8621-5},
+ editor = {Birkh\"{a}user}
+}
+@book{Davison.Hinkley:1997,
+ title = {Bootstrap Methods and Their Application},
+ author = {Davison, A. C. and Hinkley, D. V.},
+ year = 1997,
+ publisher = {Cambridge University Press},
+ address = {New York},
+ pages = 582,
+ doi = {10.1017/cbo9780511802843},
+ url = {https://doi.org/10.1017/CBO9780511802843}
+}
+@book{survival-book,
+ title = {Modeling Survival Data: Extending the {C}ox Model},
+ author = {Terry M. Therneau and Patricia M. Grambsch},
+ year = 2000,
+ publisher = {Springer},
+ address = {New York},
+ doi = {10.1007/978-1-4757-3294-8},
+ isbn = {0-387-98784-3},
+ url = {https://doi.org/10.1007/978-1-4757-3294-8}
+}
+@book{VGAMbook,
+ title = {Vector Generalized Linear and Additive Models},
+ author = {Thomas W. Yee},
+ year = 2015,
+ location = {New York, NY},
+ publisher = {Springer},
+ doi = {10.1007/978-1-4939-2818-7},
+ url = {https://doi.org/10.1007/978-1-4939-2818-7},
+ subtitle = {With an Implementation in {R}}
+}
+@book{ggplot2,
+ title = {{ggplot2}: Elegant Graphics for Data Analysis},
+ author = {Hadley Wickham},
+ year = 2016,
+ publisher = {Springer-Verlag New York},
+ isbn = {978-3-319-24277-4},
+ url = {https://ggplot2.tidyverse.org}
+}
+@book{ExceptionalLifespans,
+ title = {Exceptional Lifespans},
+ year = 2021,
+ location = {Cham, Switzerland},
+ publisher = {Springer},
+ series = {Demographic Research Monographs},
+ doi = {10.1007/978-3-030-49970-9},
+ url = {https://doi.org/10.1007/978-3-030-49970-9},
+ editor = {Maier, Heiner and Jeune, Bernard and Vaupel, James W.}
+}
+@inbook{Sevcikova:2016,
+ title = {Age-Specific Mortality and Fertility Rates for Probabilistic Population Projections},
+ author = {{\v{S}}ev{\v{c}}{\'i}kov{\'a}, Hana and Li, Nan and Kantorov{\'a}, Vladim{\'i}ra and Gerland, Patrick and Raftery, Adrian E.},
+ year = 2016,
+ booktitle = {Dynamic Demographic Analysis},
+ publisher = {Springer},
+ address = {Cham, Switzerland},
+ pages = {285--310},
+ doi = {10.1007/978-3-319-26603-9_15},
+ isbn = {978-3-319-26603-9},
+ url = {https://doi.org/10.1007/978-3-319-26603-9_15},
+ editor = {Schoen, Robert}
+}
+@incollection{IDL:2021,
+ title = {The International Database on Longevity: Data Resource Profile},
+ author = {Dmitri A. Jdanov and Vladimir M. Shkolnikov and Sigrid Gellers-Barkmann},
+ year = 2021,
+ booktitle = {Exceptional Lifespans},
+ publisher = {Springer},
+ address = {Cham, Switzerland},
+ series = {Demographic Research Monographs},
+ pages = {22--24},
+ doi = {10.1007/978-3-030-49970-9_2},
+ url = {https://doi.org/10.1007/978-3-030-49970-9_2},
+ editor = {Maier, Heiner and Jeune, Bernard and Vaupel, James W.}
+}
+@inproceedings{Beard:1963,
+ title = {A theory of mortality based on actuarial, biological and medical considerations},
+ author = {Beard, Robert E.},
+ year = 1963,
+ booktitle = {Proceedings of the International Population Conference, New York},
+ publisher = {International Union for the Scientific Study of Population},
+ address = {London},
+ volume = 1
+}
+@manual{Rsolnp-pkg,
+ title = {{Rsolnp}: General Non-linear Optimization Using Augmented {L}agrange Multiplier Method},
+ author = {Alexios Ghalanos and Stefan Theussl},
+ year = 2015,
+ doi = {10.32614/CRAN.package.Rsolnp},
+ url = {https://CRAN.R-project.org/package=Rsolnp},
+ note = {R package version 1.16.}
+}
+@manual{vitality-package,
+ title = {{vitality}: Fitting Routines for the Vitality Family of Mortality Models},
+ author = {Gregor Passolt and James J. Anderson and Ting Li and David H. Salinger and David J. Sharrow},
+ year = 2018,
+ doi = {10.32614/CRAN.package.vitality},
+ url = {https://CRAN.R-project.org/package=vitality},
+ note = {R package version 1.3}
+}
+@manual{ReIns,
+ title = {{ReIns}: Functions from "Reinsurance: Actuarial and Statistical Aspects"},
+ author = {Tom Reynkens and Roel Verbelen},
+ year = 2020,
+ doi = {10.32614/CRAN.package.ReIns},
+ url = {https://CRAN.R-project.org/package=ReIns},
+ note = {R package version 1.0.10}
+}
+@manual{muhaz-package,
+ title = {{muhaz}: Hazard Function Estimation in Survival Analysis},
+ author = {Kenneth Hess and Robert Gentleman},
+ year = 2021,
+ doi = {10.32614/CRAN.package.muhaz},
+ url = {https://CRAN.R-project.org/package=muhaz},
+ note = {R package version 1.2.6.4}
+}
+@manual{tranSurv-package,
+ title = {{tranSurv}: Transformation Model Based Estimation of Survival and Regression Under Dependent Truncation and Independent Censoring},
+ author = {Sy Han (Steven) Chiou and Jing Qian},
+ year = 2021,
+ doi = {10.32614/CRAN.package.tranSurv},
+ url = {https://CRAN.R-project.org/package=tranSurv},
+ note = {R package version 1.2.2}
+}
+@manual{DTDA-package,
+ title = {{DTDA}: Doubly Truncated Data Analysis},
+ author = {Carla Moreira and Jacobo {de U\~{n}a-\'{A}lvarez} and Rosa Crujeiras},
+ year = 2022,
+ doi = {10.32614/CRAN.package.DTDA},
+ url = {https://bioconductor.org/packages/Icens/},
+ note = {R package version 3.0.1}
+}
+@manual{MortCast-package,
+ title = {{MortCast}: Estimation and Projection of Age-Specific Mortality Rates},
+ author = {Hana \v{S}ev\v{c}\'{\i}kov\'{a} and Nan Li and Patrick Gerland},
+ year = 2022,
+ doi = {10.32614/CRAN.package.MortCast},
+ url = {https://CRAN.R-project.org/package=MortCast},
+ note = {R package version 2.7-0}
+}
+@manual{survival-package,
+ title = {A Package for Survival Analysis in {R}},
+ author = {Terry M Therneau},
+ year = 2022,
+ doi = {10.32614/CRAN.package.survival},
+ url = {https://CRAN.R-project.org/package=survival},
+ note = {R package version 3.3-1}
+}
+@manual{mev-package,
+ title = {{mev}: Modelling Extreme Values},
+ author = {Belzile, L\'{e}o R. and others},
+ year = 2023,
+ doi = {10.32614/CRAN.package.mev},
+ url = {https://CRAN.R-project.org/package=mev},
+ note = {R package version 1.15}
+}
+@manual{dplyr-package,
+ title = {{dplyr}: A Grammar of Data Manipulation},
+ author = {Hadley Wickham and Romain Fran\c{c}ois and Lionel Henry and Kirill M\"{u}ller and Davis Vaughan},
+ year = 2023,
+ doi = {10.32614/CRAN.package.dplyr},
+ url = {https://CRAN.R-project.org/package=dplyr},
+ note = {R package version 1.1.4}
+}
+@manual{dblcens-package,
+ title = {{dblcens}: Compute the {NPMLE} of Distribution Function from Doubly Censored Data, Plus the Empirical Likelihood Ratio for $F(T)$},
+ author = {Mai Zhou and Li Lee and Kun Chen and Yifan Yang},
+ year = 2023,
+ doi = {10.32614/CRAN.package.dblcens},
+ url = {https://CRAN.R-project.org/package=dblcens},
+ note = {R package version 1.1.9}
+}
+@manual{rust-package,
+ title = {{rust}: Ratio-of-Uniforms Simulation with Transformation},
+ author = {Paul J. Northrop},
+ year = 2023,
+ doi = {10.32614/CRAN.package.rust},
+ url = {https://paulnorthrop.github.io/rust/},
+ note = {R package version 1.4.2, https://github.com/paulnorthrop/rust}
+}
+@manual{demography-package,
+ title = {{demography}: Forecasting Mortality, Fertility, Migration and Population Data},
+ author = {Rob Hyndman},
+ year = 2023,
+ doi = {10.32614/CRAN.package.demography},
+ url = {https://pkg.robjhyndman.com/demography/},
+ note = {R package version 2.0}
+}
+@manual{fda-package,
+ title = {{fda}: Functional Data Analysis},
+ author = {James Ramsay},
+ year = 2024,
+ doi = {10.32614/CRAN.package.fda},
+ url = {http://www.functionaldata.org},
+ note = {R package version 6.1.8}
+}
+@manual{MortalityLaws-package,
+ title = {{MortalityLaws}: Parametric Mortality Models, Life Tables and {HMD}},
+ author = {Marius D. Pascariu},
+ year = 2024,
+ doi = {10.32614/CRAN.package.MortalityLaws},
+ url = {https://CRAN.R-project.org/package=MortalityLaws},
+ note = {R package version 2.1.0}
+}
+@manual{Icens-package,
+ title = {{Icens}: {NPMLE} for Censored and Truncated Data},
+ author = {Robert Gentleman and Alain Vandal},
+ year = 2024,
+ doi = {10.18129/B9.bioc.Icens},
+ url = {https://CRAN.R-project.org/package=DTDA},
+ note = {R package version 1.76.0}
+}
+@manual{prodlim-package,
+ title = {{prodlim}: Product-Limit Estimation for Censored Event History Analysis},
+ author = {Thomas A. Gerds},
+ year = 2024,
+ doi = {10.32614/CRAN.package.prodlim},
+ url = {https://CRAN.R-project.org/package=prodlim},
+ note = {R package version 2024.06.25}
+}
+@incollection{Belzile.Neslehova:2025,
+ title = {Statistics of extremes for incomplete data, with application to lifetime and liability claim modeling},
+ author = {Belzile, L\'eo R. and Ne\v{s}lehov\'a, Johanna G.},
+ year = 2025,
+ booktitle = {Handbook on Statistics of Extremes},
+ publisher = {CRC Press},
+ pages = {in press},
+ chapter = 31,
+ editor = {Miguel de Carvalho and Rapha\"el Huser and Philippe Naveau and Reich, Brian J.}
+}
+@phdthesis{Ye:1987,
+ title = {Interior Algorithms for Linear, Quadratic, and Linearly Constrained Non-Linear Programming},
+ author = {Yinyu Ye},
+ year = 1987,
+ school = {Department of {ESS}, Stanford University}
+}
+@article{Miralles.Davison:2023,
+ title = {Timing and spatial selection bias in rapid extreme event attribution},
+ author = {Oph{\'e}lia Miralles and Anthony C. Davison},
+ year = 2023,
+ journal = {Weather and Climate Extremes},
+ volume = 41,
+ pages = 100584,
+ doi = {10.1016/j.wace.2023.100584},
+ url = {https://doi.org/10.1016/j.wace.2023.100584}
+}
+@article{tidyverse,
+ title = {Welcome to the {tidyverse}},
+ author = {Hadley Wickham and Mara Averick and Jennifer Bryan and Winston Chang and Lucy D'Agostino McGowan and Romain François and Garrett Grolemund and Alex Hayes and Lionel Henry and Jim Hester and Max Kuhn and Thomas Lin Pedersen and Evan Miller and Stephan Milton Bache and Kirill Müller and Jeroen Ooms and David Robinson and Dana Paige Seidel and Vitalie Spinu and Kohske Takahashi and Davis Vaughan and Claus Wilke and Kara Woo and Hiroaki Yutani},
+ year = 2019,
+ journal = {Journal of Open Source Software},
+ volume = 4,
+ number = 43,
+ pages = 1686,
+ doi = {10.21105/joss.01686},
+ url = {https://doi.org/10.21105/joss.01686}
+}
+
diff --git a/_articles/RJ-2025-035/PerFer-MarCam_movieROC.R b/_articles/RJ-2025-035/PerFer-MarCam_movieROC.R
new file mode 100644
index 0000000000..30b09494d8
--- /dev/null
+++ b/_articles/RJ-2025-035/PerFer-MarCam_movieROC.R
@@ -0,0 +1,128 @@
+# R script to reproduce the results presented in the paper ----
+# "movieROC: Visualizing the Decision Rules Underlying Binary Classification"
+
+library(movieROC)
+data(HCC)
+str(HCC)
+table(HCC$tumor)
+
+
+## Figure 2 ----
+
+par(mfrow = c(1,3))
+for(gene in c("20202438", "18384097", "03515901")){
+ roc <- gROC(X = HCC[,paste0("cg",gene)], D = HCC$tumor)
+ plot_densities(roc, histogram = TRUE, lwd = 3, main = paste("Gene", gene),
+ legend = (gene == "03515901"), pos.legend = "topleft",
+ xlim = c(0.4*(gene == "20202438"),1))
+ plot_densities(roc, lwd = 3, new = FALSE,
+ col = adjustcolor(c('#485C99','#8F3D52'), alpha.f = 0.8))}
+
+par(mfrow = c(1,1))
+
+
+## Figure 3 ----
+
+for(gene in c("20202438", "18384097", "03515901")){
+ roc <- gROC(X = HCC[,paste0("cg",gene)], D = HCC$tumor,
+ side = ifelse(gene == "03515901", "left", "right"))
+ plot(roc, col = "gray50", main = paste("Gene", gene), lwd = 2)
+ groc <- gROC(X = HCC[,paste0("cg",gene)], D = HCC$tumor, side = "both")
+ plot(groc, new = FALSE, lwd = 2)
+ legend("bottomright", paste(c("AUC =", "gAUC ="),
+ format(c(roc$auc, groc$auc), digits = 3)),
+ col = c("gray50", "black"), lwd = 2, bty = "n", inset = .01)
+}
+
+
+## Figure extra HTML ----
+
+roc_selg1 <- gROC(X = HCC$cg20202438, D = HCC$tumor, side = "right")
+roc_selg1
+predict(roc_selg1, FPR = .1)
+
+plot_densityROC(roc_selg1, C = .77, build.process = TRUE)
+plot_buildROC(roc_selg1, C = .77, build.process = TRUE, reduce = FALSE)
+movieROC(roc_selg1, reduce = FALSE, file = "StandardROC_gene20202438.gif")
+
+
+## Figure 4 ----
+
+X <- HCC[ ,c("cg20202438", "cg18384097")]; D <- HCC$tumor
+biroc_12_PT <- multiROC(X, D, method = "fixedLinear", methodLinear = "PepeThompson")
+biroc_12_Meis <- multiROC(X, D, method = "dynamicMeisner", verbose = TRUE)
+biroc_12_lrm <- multiROC(X, D)
+biroc_12_kernel <- multiROC(X, D, method = "kernelOptimal")
+list_biroc <- list(PepeTh = biroc_12_PT, Meisner = biroc_12_Meis,
+ LRM = biroc_12_lrm, KernelDens = biroc_12_kernel)
+lapply(names(list_biroc), function(x) movieROC(list_biroc[[x]], display.method = "OV",
+ xlab = "Gene 20202438", ylab = "Gene 18384097",
+ lwd.curve = 4, cex = 1.2, alpha.points = 1,
+ file = paste0(x, ".gif")))
+
+
+## Figure 5 ----
+
+multiroc_PT <- multiROC(X = HCC[,c("cg20202438", "cg18384097", "cg03515901")],
+ D = HCC$tumor, method = "fixedLinear", methodLinear = "PepeThompson")
+multiroc_PT
+plot_buildROC(multiroc_PT, cex = 1.2, lwd.curve = 4, alpha.points = 1)
+plot_buildROC(multiroc_PT, display.method = "OV", displayOV = c(1,3), cex = 1.2,
+ xlab = "Gene 20202438", ylab = "Gene 03515901", lwd.curve = 4, alpha.points = 1)
+
+
+## Figure 6 ----
+
+groc_selg1 <- gROC(X = HCC$cg20202438, D = HCC$tumor, side = "both")
+groc_selg1
+predict(groc_selg1, FPR = .1)
+
+groc_selg1_C <- gROC(X = HCC$cg20202438, D = HCC$tumor, side = "both",
+ restric = TRUE, optim = TRUE)
+
+plot_regions(roc_selg1, cex.legend = 1.5, plot.auc = TRUE,
+ main = "Standard right-sided assumption [Classification subsets]")
+plot_regions(groc_selg1, plot.auc = TRUE, legend = F, main.plotroc = "gROC curve",
+ main = "General approach [Classification subsets]")
+plot_regions(groc_selg1_C, plot.auc = TRUE, legend = F, main.plotroc = "gROC curve",
+ main = "General approach with restriction (C) [Classific. subsets]",
+ xlab = "Gene 20202438 expression intensity")
+
+
+## Figure 7 ----
+
+X <- HCC$cg18384097; D <- HCC$tumor
+hroc_cubic_selg2 <- hROC(X, D)
+hroc_cubic_selg2
+hroc_rcs_selg2 <- hROC(X, D, formula.lrm = "D ~ rcs(X,8)")
+hroc_lkr1_selg2 <- hROC(X, D, type = "kernel")
+hroc_lkr3_selg2 <- hROC(X, D, type = "kernel", kernel.h = 3)
+hroc_overfit_selg2 <- hROC(X, D, type = "overfitting")
+
+groc_selg2_C <- gROC(X, D, side = "both", restric = TRUE, optim = TRUE)
+
+list_hroc <- list(Cubic = hroc_cubic_selg2, Splines = hroc_rcs_selg2,
+ Overfit = hroc_overfit_selg2, LikRatioEst_h3 = hroc_lkr3_selg2,
+ LikRatioEst_h1 = hroc_lkr1_selg2, gAUC_restC = groc_selg2_C)
+AUCs <- sapply(list_hroc, function(x) x$auc)
+round(AUCs, 3)
+
+par(mfrow = c(2,3))
+lapply(list_hroc, function(x) plot_funregions(x, FPR = .15, FPR2 = .5))
+
+### To modify titles:
+for(i in seq_along(list_hroc)){
+ main <- NULL
+ if(i == 4) main <- "Likelihood ratio estimation \n(bandwidth = 3)"
+ if(i == 5) main <- "Likelihood ratio estimation \n(bandwidth = 1)"
+ if(i == 6) main <- "General approach \n under restriction (C)"
+ plot_funregions(list_hroc[[i]], FPR = .15, FPR2 = .5, main = main)
+}
+
+
+### Figure 8 ----
+
+plot_regions(hroc_rcs_selg2, FPR = .5, cex.legend = 1.5, plot.auc = TRUE)
+plot_regions(groc_selg2_C, FPR = .5, legend = FALSE, plot.auc = TRUE,
+ main = "Classification subsets: General approach with restriction (C)",
+ main.plotroc = "gROC curve", xlab = "Gene 18384097 expression intensity")
diff --git a/_articles/RJ-2025-035/PerFer-MarCam_movieROC.bib b/_articles/RJ-2025-035/PerFer-MarCam_movieROC.bib
new file mode 100644
index 0000000000..6b5575ca5b
--- /dev/null
+++ b/_articles/RJ-2025-035/PerFer-MarCam_movieROC.bib
@@ -0,0 +1,491 @@
+@manual{animationCRAN2021,
+ Author = {Yihui Xie and Christian Mueller and Lijia Yu and Weicheng Zhu},
+ Note = {R package version 2.7},
+ Title = {animation: A Gallery of Animations in Statistics and Utilities to Create Animations},
+ Url = {https://yihui.org/animation/},
+ Year = {2021}
+}
+@manual{ksCRAN2023,
+ Author = {Tarn Duong},
+ Note = {R package version 1.14.1},
+ Title = {ks: Kernel Smoothing},
+ Url = {https://CRAN.R-project.org/package=ks},
+ Year = {2023}
+}
+@manual{e1071CRAN2023,
+ Author = {David Meyer and Evgenia Dimitriadou and Kurt Hornik and Andreas Weingessel and Friedrich Leisch},
+ Note = {R package version 1.7-13},
+ Title = {e1071: Misc Functions of the Department of Statistics, Probability Theory Group (Formerly: E1071), TU Wien},
+ Url = {https://CRAN.R-project.org/package=e1071},
+ Year = {2023}
+}
+@article{Floyd1962,
+ Author = {Robert W. Floyd},
+ Journal = {Communications of the ACM},
+ Pages = {345--345},
+ Title = {Algorithm 97: Shortest path},
+ Url = {https://doi.org/10.1145/367766.368168},
+ Volume = {5},
+ Year = {1962}
+}
+@article{Hanley1988,
+ Author = {James A. Hanley},
+ Journal = {Medical Decision Making},
+ Note = {PMID: 3398748},
+ Number = {3},
+ Pages = {197-203},
+ Title = {The Robustness of the "Binormal" Assumptions Used in Fitting {ROC} Curves},
+ Url = {https://doi.org/10.1177/0272989X8800800308},
+ Volume = {8},
+ Year = {1988}
+}
+@article{DiazCoto2020,
+ Author = {Susana D{\'\i}az-Coto},
+ Journal = {Computational Statistics},
+ Pages = {1231--1251},
+ Title = {{smoothROCtime: an {R} package for time-dependent {ROC} curve estimation}},
+ Url = {https://doi.org/10.1007/s00180-020-00955-7},
+ Volume = {35},
+ Year = {2020}
+}
+@article{Hotelling1933,
+ Author = {Harold Hotelling},
+ Journal = {Journal of Educational Psychology},
+ Number = {6},
+ Pages = {417--441},
+ Publisher = {Warwick \& York},
+ Title = {Analysis of a complex of statistical variables into principal components},
+ Url = {https://doi.org/10.1037/h0071325},
+ Volume = {24},
+ Year = {1933}
+}
+@manual{DiazCoto2023,
+ Author = {Susana D{\'\i}az-Coto},
+ Note = {R package version 0.1.1},
+ Title = {sMSROC: Assessment of Diagnostic and Prognostic Markers},
+ Url = {https://CRAN.R-project.org/package=sMSROC},
+ Year = {2023}
+}
+@article{Dorfman1997,
+ Author = {D.D. Dorfman and K.S. Berbaum and C.E. Metz and R.V. Lenth and J.A. Hanley and H.A. Dagga},
+ Journal = {Academic Radiology},
+ Number = {2},
+ Pages = {138--149},
+ Title = {Proper receiver operating characteristic analysis: The bigamma model},
+ Url = {https://doi.org/10.1016/s1076-6332(97)80013-x},
+ Volume = {4},
+ Year = {1997}
+}
+@book{Nakas2023,
+ Author = {Nakas, C.T. and Bantis, L.E. and Gatsonis, C.},
+ Isbn = {9781032480220},
+ Lccn = {2022055093},
+ Publisher = {CRC Press},
+ Series = {Chapman \& Hall/CRC biostatistics series},
+ Title = {ROC Analysis for Classification and Prediction in Practice},
+ Url = {https://www.routledge.com/ROC-Analysis-for-Classification-and-Prediction-in-Practice/Nakas-Bantis-Gatsonis/p/book/9781482233704},
+ Year = {2023}
+}
+@article{Duong2007,
+ Author = {Duong, Tarn},
+ Journal = {Journal of Statistical Software},
+ Number = {7},
+ Pages = {1--16},
+ Title = {ks: Kernel Density Estimation and Kernel Discriminant Analysis for Multivariate Data in {R}},
+ Url = {https://doi.org/10.18637/jss.v021.i07},
+ Volume = {21},
+ Year = {2007}
+}
+@article{Camblor2021b,
+ Author = {Mart{\'\i}nez-Camblor, Pablo and P{\'e}rez-Fern{\'a}ndez, Sonia and D{\'\i}az-Coto, Susana},
+ Journal = {AStA Advances in Statistical Analysis},
+ Number = {4},
+ Pages = {581--599},
+ Title = {Optimal classification scores based on multivariate marker transformations},
+ Url = {https://doi.org/10.1007/s10182-020-00388-z},
+ Volume = {105},
+ Year = {2021}
+}
+@article{Camblor2021a,
+ Author = {Mart{\'\i}nez-Camblor, Pablo and P{\'e}rez-Fern{\'a}ndez, Sonia and D{\'\i}az-Coto, Susana},
+ Journal = {The International Journal of Biostatistics},
+ Number = {1},
+ Pages = {293--306},
+ Publisher = {De Gruyter},
+ Title = {The area under the generalized receiver-operating characteristic curve},
+ Url = {https://doi.org/10.1515/ijb-2020-0091},
+ Volume = {18},
+ Year = {2021}
+}
+@article{Bantis2021a,
+ Author = {Bantis, Leonidas E. and Tsimikas, John V. and Chambers, Gregory R. and Capello, Michela and Hanash, Samir and Feng, Ziding},
+ Journal = {Statistics in Medicine},
+ Number = {7},
+ Pages = {1767--1789},
+ Title = {The length of the receiver operating characteristic curve and the two cutoff {Youden} index within a robust framework for discovery, evaluation, and cutoff estimation in biomarker studies involving improper receiver operating characteristic curves},
+ Url = {https://doi.org/10.1002/sim.8869},
+ Volume = {40},
+ Year = {2021}
+}
+@article{Liu2011,
+ Author = {Liu, Chunling and Liu, Aiyi and Halabi, Susan},
+ Journal = {Statistics in Medicine},
+ Number = {16},
+ Pages = {2005--2014},
+ Title = {A min--max combination of biomarkers to improve diagnostic accuracy},
+ Url = {https://doi.org/10.1002/sim.4238},
+ Volume = {30},
+ Year = {2011}
+}
+@article{Zou1997,
+ Author = {Zou, Kelly H. and Hall, W. J. and Shapiro, David E.},
+ Journal = {Statistics in Medicine},
+ Number = {19},
+ Pages = {2143--2156},
+ Title = {Smooth non-parametric receiver operating characteristic ({ROC}) curves for continuous diagnostic tests},
+ Url = {https://doi.org/10.1002/(SICI)1097-0258(19971015)16:19<2143::AID-SIM655>3.0.CO;2-3},
+ Volume = {16},
+ Year = {1997}
+}
+@article{Hsieh1996,
+ Author = {Hsieh, Fushing and Turnbull, Bruce W.},
+ Journal = {The Annals of Statistics},
+ Number = {1},
+ Pages = {25--40},
+ Publisher = {The Institute of Mathematical Statistics},
+ Title = {Nonparametric and semiparametric estimation of the receiver operating characteristic curve},
+ Url = {https://doi.org/10.1214/aos/1033066197},
+ Volume = {24},
+ Year = {1996}
+}
+@article{Fluss2005,
+ Author = {Ronen Fluss and David Faraggi and Benjamin Reiser},
+ Journal = {Biometrical Journal},
+ Number = {4},
+ Pages = {458--472},
+ Title = {Estimation of the {Youden} Index and its Associated Cutoff Point},
+ Url = {https://doi.org/10.1002/bimj.200410135},
+ Volume = {47},
+ Year = {2005}
+}
+@article{Goncalves2014,
+ Author = {Luzia Gon{\c c}alves and Ana Subtil and Mar{\'\i}a Ros{\'a}rio Oliveira and Patricia De Zea Bermudez},
+ Journal = {REVSTAT Statistical Journal},
+ Number = {1},
+ Pages = {1--20},
+ Title = {{ROC} curve estimation: An overview},
+ Url = {https://doi.org/10.57805/revstat.v12i1.141},
+ Volume = {12},
+ Year = {2014}
+}
+@article{Camblor2018a,
+ Author = {Mart{\'\i}nez-Camblor, P.},
+ Journal = {Statistics in Medicine},
+ Number = {7},
+ Pages = {1222--1224},
+ Title = {{On the paper ``Notes on the overlap measure as an alternative to the {Youden} index''}},
+ Url = {https://doi.org/10.1002/sim.7517},
+ Volume = {37},
+ Year = {2018}
+}
+@article{Murtaugh1995,
+ Author = {Paul A. Murtaugh},
+ Journal = {Biometrics},
+ Number = {4},
+ Pages = {1514--1522},
+ Title = {{ROC} Curves with Multiple Marker Measurements},
+ Url = {https://doi.org/10.2307/2533281},
+ Volume = {51},
+ Year = {1995}
+}
+@article{Mayeux2004,
+ Author = {Mayeux, R.},
+ Journal = {NeuroRx},
+ Number = {2},
+ Pages = {182--188},
+ Title = {Biomarkers: Potential uses and limitations},
+ Url = {https://doi.org/10.1602/neurorx.1.2.182},
+ Volume = {1},
+ Year = {2004}
+}
+@misc{ROCnReg:CRAN2023,
+ Author = {Maria Xos{\'e} Rodr{\'\i}guez-{\'A}lvarez and Vanda Inacio},
+ Note = {R package version 1.0-8},
+ Title = {{ROCnReg}: {ROC} Curve Inference with and without Covariates},
+ Url = {https://CRAN.R-project.org/package=ROCnReg},
+ Year = {2023}
+}
+@article{ROCnReg2021,
+ Author = {Mar{\'\i}a Xos{\'e} Rodr{\'\i}guez-{\'A}lvarez and Vanda In{\'a}cio},
+ Journal = {The R Journal},
+ Number = {1},
+ Pages = {525--555},
+ Title = {{ROCnReg}: An {R} Package for Receiver Operating Characteristic Curve Inference With and Without Covariates},
+ Url = {https://doi.org/10.32614/RJ-2021-066},
+ Volume = {13},
+ Year = {2021}
+}
+@article{nsROC2018,
+ Author = {Sonia P{\'e}rez-Fern{\'a}ndez and Pablo Mart{\'\i}nez-Camblor and Peter Filzmoser and Norberto Corral},
+ Journal = {{The R Journal}},
+ Number = {2},
+ Pages = {55--74},
+ Title = {{nsROC}: An {R} package for Non-Standard {ROC} Curve Analysis},
+ Url = {https://doi.org/10.32614/RJ-2018-043},
+ Volume = {10},
+ Year = {2018}
+}
+@article{OptimalCutpoints2014,
+ Author = {M{\'o}nica L{\'o}pez-Rat{\'o}n and Mar{\'\i}a Xos{\'e} Rodr{\'\i}guez-{\'A}lvarez and Carmen Cadarso Su{\'a}rez and Francisco Gude Sampedro},
+ Journal = {Journal of Statistical Software},
+ Number = {8},
+ Pages = {1--36},
+ Title = {{OptimalCutpoints}: An {R} Package for Selecting Optimal Cutpoints in Diagnostic Tests},
+ Url = {https://doi.org/10.18637/jss.v061.i08},
+ Volume = {61},
+ Year = {2014}
+}
+@misc{OptimalCutpoints:CRAN2021,
+ Author = {M{\'o}nica L{\'o}pez-Rat{\'o}n and Mar{\'\i}a Xos{\'e} Rodr{\'\i}guez-{\'A}lvarez},
+ Note = {R package version 1.1-5},
+ Title = {{OptimalCutpoints: Computing Optimal Cutpoints in Diagnostic Tests}},
+ Url = {https://CRAN.R-project.org/package=OptimalCutpoints},
+ Year = {2021}
+}
+@article{pROC2011,
+ Author = {Xavier Robin and Natacha Turck and Alexandre Hainard and Natalia Tiberti and Fr{\'e}d{\'e}rique Lisacek and Jean-Charles Sanchez and Markus M{\"u}ller},
+ Journal = {BMC Bioinformatics},
+ Number = {77},
+ Pages = {1--8},
+ Title = {p{ROC}: an open-source package for {R} and {S+} to analyze and compare {ROC} curves},
+ Url = {https://doi.org/10.1186/1471-2105-12-77},
+ Volume = {12},
+ Year = {2011}
+}
+@misc{pROC:CRAN2023,
+ Author = {Xavier Robin and Natacha Turck and Alexandre Hainard and Natalia Tiberti and Fr{\'e}d{\'e}rique Lisacek and Jean-Charles Sanchez and Markus M{\"u}ller and Stefan Siegert and Matthias Doering and Zane Billings},
+ Note = {R package version 1.18.4},
+ Title = {p{ROC}: Display and Analyze {ROC} Curves},
+ Url = {https://CRAN.R-project.org/package=pROC},
+ Year = {2023}
+}
+@manual{nsROC:CRAN2018,
+ Author = {Sonia P\'erez-Fern\'andez},
+ Note = {R package version 1.1},
+ Title = {ns{ROC}: Non-Standard {ROC} Curve Analysis},
+ Url = {https://CRAN.R-project.org/package=nsROC},
+ Year = {2018}
+}
+@article{Shen2012,
+ Author = {Shen, Jing and Wang, Shuang and Zhang, Yu-Jing and Kappil, Maya and Wu, Hui-Chen and Kibriya, Muhammad G and Wang, Qiao and Jasmine, Farzana and Ahsan, Habib and Lee, Po-Huang and others},
+ Journal = {Hepatology},
+ Number = {6},
+ Pages = {1799--1808},
+ Publisher = {Wiley Online Library},
+ Title = {Genome-wide {DNA} methylation profiles in hepatocellular carcinoma},
+ Url = {https://doi.org/10.1002/hep.25569},
+ Volume = {55},
+ Year = {2012}
+}
+@techreport{Kauppi2016,
+ Author = {Heikki Kauppi},
+ Institution = {Aboa Centre for Economics},
+ Number = {114},
+ Title = {The Generalized Receiver Operating Characteristic Curve},
+ Type = {Discussion paper},
+ Url = {https://www.econstor.eu/bitstream/10419/233329/1/aboa-ce-dp114.pdf},
+ Year = {2016}
+}
+@article{Inacio2021,
+ Author = {In\'{a}cio, Vanda and Rodr\'{\i}guez-\'{A}lvarez, Mar\'{\i}a Xos\'{e} and Gayoso-Diz, Pilar},
+ Journal = {Annual Review of Statistics and Its Application},
+ Number = {1},
+ Pages = {41--67},
+ Title = {Statistical Evaluation of Medical Tests},
+ Url = {https://doi.org/10.1146/annurev-statistics-040720-022432},
+ Volume = {8},
+ Year = {2021}
+}
+@article{Meisner2021,
+ Author = {Allison Meisner and Marco Carone and Margaret Sullivan Pepe and Kathleen F. Kerr},
+ Journal = {Biometrical Journal},
+ Number = {6},
+ Pages = {1223--1240},
+ Title = {Combining Biomarkers by Maximizing the True Positive Rate for a Fixed False Positive Rate},
+ Url = {https://doi.org/10.1002/bimj.202000210},
+ Volume = {63},
+ Year = {2021}
+}
+@book{Pepe2003a,
+ Author = {Margaret Sullivan Pepe},
+ Isbn = {9780198509844},
+ Publisher = {Oxford University Press},
+ Series = {Oxford Statistical Science Series},
+ Title = {The Statistical Evaluation of Medical Tests for Classification and Prediction},
+ Url = {https://global.oup.com/academic/product/the-statistical-evaluation-of-medical-tests-for-classification-and-prediction-9780198565826?cc=es&lang=en&},
+ Year = {2003}
+}
+@book{Zhou2002,
+ Author = {Xiao-Hua Zhou and Donna K. McClish and Nancy A. Obuchowski},
+ Publisher = {Wiley},
+ Series = {Wiley Series in Probability and Statistics},
+ Title = {Statistical Methods in Diagnostic Medicine},
+ Url = {https://doi.org/10.1002/9780470317082},
+ Volume = {414},
+ Year = {2002}
+}
+@article{Su1993,
+ Author = {John Q. Su and Jun S. Liu},
+ Journal = {Journal of the American Statistical Association},
+ Number = {424},
+ Pages = {1350--1355},
+ Title = {Linear combinations of multiple diagnostic markers},
+ Url = {https://doi.org/10.1080/01621459.1993.10476417},
+ Volume = {88},
+ Year = {1993}
+}
+@article{Hanley1982,
+ Author = {James A. Hanley and Barbara J McNeil},
+ Journal = {Radiology},
+ Number = {1},
+ Pages = {29--36},
+ Title = {The Meaning and Use of the Area Under a Receiver Operating Characteristic ({ROC}) Curve},
+ Url = {https://doi.org/110.1148/radiology.143.1.7063747},
+ Volume = {143},
+ Year = {1982}
+}
+@article{Bamber1975,
+ Author = {Donald Bamber},
+ Journal = {Journal of Mathematical Psychology},
+ Number = {4},
+ Pages = {387--415},
+ Title = {The area above the ordinal dominance graph and the area below the receiver operating characteristic graph},
+ Url = {https://doi.org/10.1016/0022-2496(75)90001-2},
+ Volume = {12},
+ Year = {1975}
+}
+@article{Youden1950,
+ Author = {Youden, W.J.},
+ Journal = {Cancer},
+ Number = {1},
+ Pages = {32--35},
+ Title = {Index for rating diagnostic tests},
+ Url = {https://doi.org/10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3},
+ Volume = {3},
+ Year = {1950}
+}
+@article{PerezFernandez2020,
+ Author = {Sonia P{\'e}rez-Fern{\'a}ndez and Pablo Mart{\'\i}nez-Camblor and Peter Filzmoser and Norberto Corral},
+ Journal = {{AStA Advances in Statistical Analysis}},
+ Number = {1},
+ Pages = {135--161},
+ Title = {Visualizing the decision rules behind the {ROC} curves: understanding the classification process},
+ Url = {https://doi.org/10.1007/s10182-020-00385-2},
+ Volume = {105},
+ Year = {2021}
+}
+@article{Pepe2000,
+ Author = {Margaret Sullivan Pepe and Mary Lou Thompson},
+ Journal = {Biostatistics},
+ Number = {2},
+ Pages = {123--140},
+ Title = {Combining diagnostic test results to increase accuracy},
+ Url = {https://doi.org/10.1093/biostatistics/1.2.123},
+ Volume = {1},
+ Year = {2000}
+}
+@article{Camblor2019c,
+ Author = {Pablo Mart{\'\i}nez-Camblor and Juan Carlos Pardo-Fern{\'a}ndez},
+ Journal = {The International Journal of Biostatistics},
+ Number = {1},
+ Title = {The {Youden} Index in the Generalized Receiver Operating Characteristic Curve Context},
+ Url = {https://doi.org/10.1515/ijb-2018-0060},
+ Volume = {15},
+ Year = {2019}
+}
+@article{Camblor2019a,
+ Author = {Pablo Mart{\'\i}nez-Camblor and Sonia P{\'e}rez-Fern{\'a}ndez and Susana D{\'\i}az-Coto},
+ Journal = {Journal of Applied Statistics},
+ Number = {9},
+ Pages = {1550--1566},
+ Title = {Improving the biomarker diagnostic capacity via functional transformations},
+ Url = {https://doi.org/10.1080/02664763.2018.1554628},
+ Volume = {46},
+ Year = {2019}
+}
+@article{Camblor2019b,
+ Author = {Pablo Mart{\'\i}nez-Camblor and Juan Carlos Pardo-Fern{\'a}ndez},
+ Journal = {Statistical Methods in Medical Research},
+ Number = {7},
+ Pages = {2032--2048},
+ Title = {Parametric estimates for the receiver operating characteristic curve generalization for non-monotone relationships},
+ Url = {https://doi.org/10.1177/0962280217747009},
+ Volume = {28},
+ Year = {2019}
+}
+@article{Camblor2017a,
+ Author = {Pablo Mart{\'\i}nez-Camblor and Norberto Corral and Corsino Rey and Julio Pascual and Eva Cernuda-Moroll{\'o}n},
+ Journal = {Statistical Methods in Medical Research},
+ Number = {1},
+ Pages = {113--123},
+ Title = {Receiver operating characteristic curve generalization for non-monotone relationships},
+ Url = {https://doi.org/10.1177/0962280214541095},
+ Volume = {26},
+ Year = {2017}
+}
+@article{Pepe2003b,
+ Author = {Pepe, {M.S.} and G. Longton and Anderson, {G.L.} and M. Schummer},
+ Journal = {Biometrics},
+ Number = {1},
+ Pages = {133--142},
+ Title = {Selecting differentially expressed genes from microarray experiments},
+ Url = {https://doi.org/10.1111/1541-0420.00016},
+ Volume = {59},
+ Year = {2003}
+}
+@article{Kang2016,
+ Author = {Le Kang and Aiyi Liu and Lili Tian},
+ Journal = {Statistical Methods in Medical Research},
+ Number = {4},
+ Pages = {1359--1380},
+ Title = {Linear combination methods to improve diagnostic/prognostic accuracy on future observations},
+ Url = {https://doi.org/10.1177/0962280213481053},
+ Volume = {25},
+ Year = {2016}
+}
+@article{McIntosh2002,
+ Author = {Martin W. McIntosh and Margaret Sullivan Pepe},
+ Journal = {Biometrics},
+ Number = {3},
+ Pages = {657--664},
+ Title = {Combining Several Screening Tests: Optimality of the Risk Score},
+ Url = {https://doi.org/10.1111/j.0006-341X.2002.00657.x},
+ Volume = {58},
+ Year = {2002}
+}
+@manual{R,
+ Address = {Vienna, Austria},
+ Author = {{R Core Team}},
+ Note = {{ISBN} 3-900051-07-0},
+ Organization = {R Foundation for Statistical Computing},
+ Title = {{R}: A Language and Environment for Statistical Computing},
+ Url = {https://www.R-project.org/},
+ Year = {2016}
+}
+@article{ihaka:1996,
+ Author = {Ihaka, Ross and Gentleman, Robert},
+ Journal = {Journal of Computational and Graphical Statistics},
+ Number = {3},
+ Pages = {299--314},
+ Title = {{R}: A Language for Data Analysis and Graphics},
+ Url = {https://doi.org/10.1080/10618600.1996.10474713},
+ Volume = {5},
+ Year = {1996}
+}
+@manual{rmsCRAN2023,
+ Author = {Frank E {Harrell Jr}},
+ Note = {R package version 6.7-1},
+ Title = {rms: Regression Modeling Strategies},
+ Url = {https://CRAN.R-project.org/package=rms},
+ Year = {2023}
+}
diff --git a/_articles/RJ-2025-035/PerFer-MarCam_movieROC.tex b/_articles/RJ-2025-035/PerFer-MarCam_movieROC.tex
new file mode 100644
index 0000000000..b47c042f49
--- /dev/null
+++ b/_articles/RJ-2025-035/PerFer-MarCam_movieROC.tex
@@ -0,0 +1,708 @@
+% !TeX root = RJwrapper.tex
+% !TEX spellcheck = en_US
+
+\title{movieROC: Visualizing the Decision Rules Underlying Binary Classification}
+\author{by Sonia P\'erez-Fern\'andez, Pablo Mart\'inez-Camblor and Norberto Corral-Blanco}
+
+\maketitle
+
+\abstract{
+The receiver operating characteristic (ROC) curve is a graphical tool commonly used to depict the binary classification accuracy of a continuous marker in terms of its sensitivity and specificity. The standard ROC curve assumes a monotone relationship between the marker and the response, inducing classification subsets of the form $(c,\infty)$ with $c \in \mathbb R$.
+However, in non-standard cases, the involved classification regions are not so clear, highlighting the importance of tracking the decision rules.
+This paper introduces the R package movieROC, which provides visualization tools for understanding the ability of markers to identify a characteristic of interest, complementing the ROC curve representation. This tool accommodates multivariate scenarios and generalizations involving different decision rules.
+The main contribution of this package is the visualization of the underlying classification regions, with the associated gain in interpretability. Adding the time (videos) as a third dimension, this package facilitates the visualization of binary classification in multivariate problems. It constitutes a good tool to generate graphical material for presentations.
+}
+
+\section{Introduction} \label{introduction}
+
+The use of data to detect a characteristic of interest is a cornerstone of many disciplines such as medicine (to diagnose a pathology or to predict a patient outcome), finance (to detect fraud) or machine learning (to evaluate a classification algorithm), among others. Continuous markers are surrogate measures for the characteristic under study, or predictors of a potential subsequent event. They are measured in subjects, some of them with the characteristic (\dfn{positive}), and some without it (\dfn{negative}). In addition to reliability and feasibility, a good marker must have two relevant properties: interpretability and accuracy \citep{Mayeux2004}. High binary classification \dfn{accuracy} can be achieved if there exists a strong relationship between the marker and the \dfn{response}. The latter is assessed by a \dfn{gold standard} for the presence or absence of the characteristic of interest. \dfn{Interpretability} refers to the \dfn{decision rules} or \dfn{subsets} considered in the classification process. This piece of research seeks to elucidate both desirable properties for a marker by the implementation of a graphical tool in R language. We propose a novel approach involving the generation of videos as a solution to effectively capture the classification procedure for univariate and multivariate markers. Graphical analysis plays a pivotal role in data exploration, interpretation, and communication. Its burgeoning potential is underscored by the fast pace of technological advances, which empower the creation of insightful graphical representations.
+
+A usual practice when the binary classification accuracy of a marker is of interest involves the representation of the \dfn{Receiver Operating Characteristic (ROC) curve}, summarized by the \dfn{Area Under the Curve} (\dfn{AUC}) \citep{Hanley1982}. The resulting plot reflects the trade-off between the sensitivity and the complementary of the specificity. \dfn{Sensitivity} and \dfn{specificity} are probabilities of correctly classifying subjects, either positive or negative, respectively. Mathematically, let $\xi$ and $\chi$ be the random variables modeling the marker values in the positive and the negative population, respectively, with $F_\xi(\cdot)$ and $F_\chi(\cdot)$ their associated cumulative distribution functions. Assuming that the expected value of the marker is larger in the positive than in the negative population, the standard ROC curve is based on \dfn{classification subsets} of the form $s = (c, \infty)$, where $c$ is the so-called \dfn{cut-off value or threshold} in the support of the marker $X$, $\mathcal{S}(X)$. One subject is classified as a positive if its marker value is within this region, and as a negative otherwise. This type of subsets has two important advantages: first, their interpretability is clear; second, for each specificity $1-t \in [0,1]$, the corresponding $s_t = (c_t, \infty)$ is univocally defined by $c_t = F_\chi^{-1}(1-t)$ for absolutely continuous markers.
+
+
+When differences in marker distribution between the negative and the positive population are only in location but not in shape, then $F_\chi(\cdot) < F_\xi(\cdot)$, and the classification is direct by using these decision rules. However, when this is not the case, the standard ROC curve may cross the main diagonal, resulting in an \dfn{improper} curve \citep{Dorfman1997}. This may be due to three different scenarios:
+\begin{itemize}
+\item[i)] the behavior of the marker in the two studied populations is different but it is not possible to determine the decision rules. Notice that the binary classification problem goes further than distinction between the two populations: the classification subsets should be highly likely in one population and highly unlikely in the other one \citep{Camblor2018a};
+\item[ii)] there exists a relationship between the marker and the response with a potential classification use, but this is not monotone;
+\item[iii)] there is no relationship between the marker and the response at all (main diagonal ROC curve).
+\end{itemize}
+
+In the second case, we have to define classification subsets different from standard $s_t=(c_t,\infty)$.
+Therefore, the use of the marker becomes more complex. With the aim of accommodating scenarios where both higher and lower values of the marker are associated with a higher risk of having the characteristic, \citet{Camblor2017a} proposed the so-called \dfn{generalized ROC (gROC) curve}. This curve tracks the highest sensitivity for every specificity in the unit interval resulting from subsets of the form $s_t=(-\infty, x_t^L] \cup (x_t^U, \infty)$ with $x_t^L \leq x_t^U \in \mathcal{S}(X)$.
+
+Despite final decisions are based on the underlying classification subsets, they are typically not depicted.
+This omission is not a shortcoming in standard cases, as for each specificity $1-t \in [0,1]$, there is only one rule of the form $s_t = (c_t, \infty)$ which such specificity. Particularly, $s_t$ is univocally defined by $c_t = 1 - F_\chi^{-1}(1-t)$; and the same applies if we fix a sensitivity. Nevertheless, if the gROC curve is taken, there are infinite subsets of the form $s_t=(-\infty, x_t^L] \cup (x_t^U, \infty)$ resulting in $\P(\chi \in s_t) = t$.
+This loss of univocity underlines the importance of reporting (numerically and/or graphically) the decision rules actually proposed for classifying. This gap is covered in the presented package.
+
+An alternative approach to assess the classification performance of a marker involves considering a transformation of it. This transformation $h(\cdot)$ aims to capture differences in distribution between the two populations in the ROC sense. Once $h(\cdot)$ is identified, the standard ROC curve for $h(X)$ is represented, resulting in the \dfn{efficient ROC (eROC) curve} \citep{Kauppi2016}. Arguing as before, for a fixed specificity, the classification subsets $s_t=(c_t, \infty)$ in the transformed space are univocally defined, where a subject is classified as positive if $h(x) \in s_t$ and negative otherwise (with $x$ representing its marker value). However, they may have any shape in the original space, depending on the monotonicity of the functional transformation $h(\cdot)$ \citep{Camblor2019a}. Emphasizing the importance of tracking the decision rules underlying the eROC curve, this monitoring process enables an assessment of whether the improved accuracy of the marker justifies the potential loss in interpretability.
+
+The ROC curve is defined for classification accuracy evaluation of univariate markers. To deal with multivariate markers, the usual practice is to consider a transformation $\boldsymbol{h}(\cdot)$ to reduce it to a univariate one, and then to construct the standard ROC curve. Same considerations as before apply when a functional transformation is taken. In the proposed R library, we consider methods from the literature to define and estimate $\boldsymbol{h}(\cdot)$ in the multivariate case \citep{Kang2016, Meisner2021}.
+
+
+Focusing on the classification subsets underlying the decision rules, the \CRANpkg{movieROC} package incorporates methods to visualize the construction process of ROC curves by presenting the classification accuracy of these subsets.
+For univariate markers, the library includes both the classical (standard ROC curve) and the generalized (gROC curve) approach. Besides it enables the display of decision rules for various transformations of the marker, seeking to maximize performance allowing for flexibility in the final shape of the subsets (eROC curve). For multidimensional markers, the proposed tool visualizes the evolution of decision subsets when different objective functions are employed for optimization, even imposing restrictions on the underlying regions. In this case, displaying the decision rules associated with every specificity in a single static image is no longer feasible. Therefore, \dfn{dynamic representations} (videos) are implemented, drawing on time as an extra dimension to capture the variation in specificity.
+
+Much software available in R could be discussed here covering diverse topics related to ROC curves:
+the \CRANpkg{pROC} package is a main reference including tools to visualize, estimate and compare ROC curves \citep{pROC2011};
+\CRANpkg{ROCnReg} explicitly considers covariate information to estimate the covariate-specific and the covariate-adjusted ROC curves \citep{ROCnReg2021};
+\CRANpkg{smoothROCtime} implements smooth estimation of time-dependent ROC curves based on the bivariate kernel density estimator for $(X, \textit{time-to-event})$ \citep{DiazCoto2020};
+\CRANpkg{OptimalCutpoints} includes point and interval estimation methods for optimal thresholds \citep{OptimalCutpoints2014};
+and \CRANpkg{nsROC} performs non-standard analysis such as gROC estimation \citep{nsROC2018}; among others.
+
+This paper introduces and elucidates the diverse functionalities of the newly developed \CRANpkg{movieROC} package, aimed at facilitating the visualization and comprehension of the decision rules underlying the binary classification process, encompassing various generalizations. Despite the availability of numerous R packages implementing related analyses, we have identified the main gaps covered in this library: tracking the decision rules underlying the ROC curve, including multivariate markers and non-standard scenarios (i.e. non-monotonous). The rest of the paper is structured as follows. In Section~\ref{section:functionality}, we introduce the main R functions and objects implemented, and briefly explain the dataset employed throughout this manuscript to demonstrate the utility of the R library. Section~\ref{section:regularroc} is devoted to reconsidering the definition of the standard ROC curve from the perspective of classification subsets, including an extension to multivariate scenarios. Sections~\ref{section:groc} and \ref{section:efficientroc} revisit the gROC curve and the eROC curve, respectively, covering various methods to capture the potential classification accuracy of the marker under study.
+Each of these sections begins with a state-of-the-art overview, followed by the main syntax of the corresponding R functions. In addition, examples of implementation using the dataset presented in Section \ref{subsection:dataset} are provided. Finally, the paper concludes with a concise summary and computational details regarding the implemented tool.
+
+
+
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\section{Main functions of the movieROC package and illustrative dataset}\label{section:functionality}
+
+Sections~\ref{subsec:functionality} and \ref{subsec:functions} provide a detailed description of the main objectives of the implemented R functions. To reflect the practical usage of the developed R package, we employ a real dataset throughout this manuscript, which is introduced in Section~\ref{subsection:dataset}.
+
+
+
+\subsection{Functionality of the movieROC package} \label{subsec:functionality}
+
+A graphical tool was developed to showcase static and dynamic graphics displaying the classification subsets derived from maximizing diagnostic accuracy under certain assumptions, ensuring the preservation of the interpretability. The R package facilitates the construction of the ROC curve across various specificities, providing visualizations of the resulting classification regions. The proposed tool comprises multiple R functions that generate objects with distinct class attributes (see function names where red arrows depart from and red nodes in Figure~\ref{figure:functionality}, respectively). Once the object of interest is created, different methods may be used, in order to plot the underlying regions (\code{plot\_regions()}, \code{plot\_funregions()}), to track the resulting ROC curve (\code{plot\_buildROC()}, \code{plot()}), to \code{predict} decision rules for a particular specificity, and to \code{print} relevant information, among others. The main function of the package, \code{movieROC()}, produces videos to exhibit the classification procedure.
+
+\begin{figure}[htbp]
+ \centering
+ \includegraphics[width=\textwidth]{figures/movieROC_mainFunctions.pdf}
+ \includegraphics[width=.8\textwidth]{figures/movieROC_extraFunctions.pdf}
+ \caption{R functions of the \CRANpkg{movieROC} package. The blue nodes include the names of the R functions and the red nodes indicate the different R objects that can be created and worked with. The red arrows depart from those R functions engaged in creating R objects and the black arrows indicate which R functions can be applied to which R objects. The grey dashed arrows show internal dependencies.}
+ \label{figure:functionality}
+\end{figure}
+
+It includes algorithms to visualize the regions that underlie the binary classification problem, considering different approaches:
+\begin{itemize}
+\item make the classification subsets flexible in order to cover non-standard scenarios, by considering two cut-off values (\code{gROC()} function); explained in Section~\ref{section:groc};
+\item transform the marker by a proper function $h(\cdot)$ (\code{hROC()} function); introduced in Section~\ref{section:efficientroc};
+\item when dealing with multivariate markers, consider a functional transformation with some fixed or dynamic parameters resulting from different methods available in the literature (\code{multiROC()} function); covered in Section~\ref{section:multiroc}.
+\end{itemize}
+
+
+
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsection{Class methods for \pkg{movieROC} objects} \label{subsec:functions}
+
+By using the \code{gROC()}, the \code{multiROC()} or the \code{hROC()} function, the user obtains an R object of class `\code{groc}', `\code{multiroc}' or `\code{hroc}', respectively. These will be called \CRANpkg{movieROC} objects. Once the object of interest is created, the implemented package includes many functions (methods) to pass to it. Some of them are generic methods (\code{print()}, \code{plot()} and \code{predict()}), commonly used in R language over different objects according to their class attributes. The rest of the functions are specific for this library and therefore only applicable to \CRANpkg{movieROC} objects. Table~\ref{table:funcmovieROC} summarizes all these functions and provides their target and main syntax (with default input parameters).
+
+{\footnotesize
+\begin{longtable}{@{}p{2.5cm}p{11.1cm}@{}}
+\caption{Methods for a \CRANpkg{movieROC} object: `\code{groc}', `\code{multiroc}' or `\code{hroc}' object (output of the \code{gROC()}, \code{multiROC()} or \code{hROC()} function, respectively). The main input parameters are displayed.\label{table:funcmovieROC}}\\
+\toprule
+\multicolumn{2}{c}{Generic functions}\\
+\hline
+\centering \code{print()} & Print some relevant information.\\
+\centering \code{plot()} & Plot the ROC curve estimate.\\
+\centering \code{predict()} & Print the classification subsets corresponding to a particular false-positive rate (\code{FPR}) introduced by the user. For a `\code{groc}' object, the use may specify a cut-off value \code{C} (for the standard ROC curve) or two cut-off values \code{XL} and \code{XU} (for the gROC curve).\\
+\midrule
+\multicolumn{2}{c}{Specific functions}\\
+\hline
+\centering \code{plot\_regions()} & Applicable to a `\code{groc}' or a `\code{hroc}' object. \par \vspace{1mm}
+Plot two graphics in the same figure: left, classification subsets for each false-positive rate (grey color by default); right, $90^\circ$ rotated ROC curve. \par \vspace{1mm}
+\textsc{Main syntax:} \par \hspace{3mm}
+\code{plot\_regions(obj, plot.roc = TRUE, plot.auc = FALSE, FPR = 0.15, ...)} \par \vspace{1mm}
+If the input parameter \code{FPR} is specified, the corresponding classification region reporting such false-positive rate and the point in the ROC curve are highlighted in blue color.\\
+\midrule
+\centering \code{plot\_funregions()} & Applicable to a `\code{groc}' or a `\code{hroc}' object. \par \vspace{1mm}
+Plot the transforming function and the classification subsets reporting the false-positive rate(s) indicated in the input parameter(s) \code{FPR} and \code{FPR2}. \par \vspace{1mm}
+\textsc{Main syntax:} \par \hspace{3mm}
+\code{plot\_funregions(obj, FPR = 0.15, FPR2 = NULL, plot.subsets = TRUE, ...)}\\
+\midrule
+\centering \code{plot\_buildROC()} & Applicable to a `\code{groc}' or a `\code{multiroc}' object. \par \vspace{2mm}
+- For a `\code{groc}' object: \par \vspace{1mm}
+Plot four (if input \code{reduce} is FALSE) or two (if \code{reduce} is TRUE, only those on the top) graphics in the same figure: top-left, density function estimates for the marker in both populations with the areas corresponding to FPR and TPR colored (blue and red, respectively) for the optional input parameter \code{FPR}, \code{C} or \code{XL, XU}; top-right, the empirical ROC curve estimate; bottom-left, boxplots in both groups; bottom-right, classification subsets for every FPR (grey color). \par \vspace{1mm}
+\textsc{Main syntax:} \par \hspace{3mm}
+\code{plot\_buildROC(obj, FPR = 0.15, C, XL, XU, h = c(1,1), histogram = FALSE, breaks = 15,} \linebreak \hspace*{2.3cm} \code{reduce = TRUE, build.process = FALSE, completeROC = FALSE, ...)} \par \vspace{1mm}
+If \code{build.process} is FALSE, the whole ROC curve is displayed; otherwise, if \code{completeROC} is TRUE, the portion of the ROC curve until the fixed FPR is highlighted in black and the rest is shown in gray, while if \code{completeROC} is FALSE, only the first portion of the curve is displayed.\par \vspace{2mm}
+- For a `\code{multiroc}' object: \par \vspace{1mm}
+Plot two graphics in the same figure: right, the ROC curve highlighting the point and the threshold for the resulting univariate marker; left, scatterplot with the marker values in both positive (red color) and negative (blue color) subjects
+\begin{itemize}
+\item for $p=2$: over the original/feature bivariate space;
+\item for $p>2$: projected over two selected components of the marker (if \code{display.method = "OV"} with components selection in \code{displayOV}, \code{c(1,2)} by default) or the first two principal components from PCA (if \code{display.method = "PCA"}, default);
+\end{itemize}
+and the classification subset (gold color) reporting the \code{FPR} selected by the user (\code{FPR} $\neq$ \code{NULL}). \par \vspace{1mm}
+\textsc{Main syntax:}
+\begin{itemize}
+\item for $p=2$:
+\item[] \code{plot\_buildROC(obj, FPR = 0.15, build.process = FALSE, completeROC = TRUE, ...)}
+\item for $p>2$:
+\item[] \code{plot\_buildROC(obj, FPR = 0.15, display.method = c("PCA","OV"), displayOV = } \linebreak \hspace*{2cm} \code{c(1,2), build.process = FALSE, completeROC = TRUE, ...)}
+\end{itemize}
+If \code{build.process} is FALSE, the whole ROC curve is displayed; otherwise, if \code{completeROC} is TRUE, the portion of the ROC curve until the fixed FPR is highlighted in black and the rest is shown in gray, while if \code{completeROC} is FALSE, only the first portion of the curve is shown.\\
+\midrule\\\pagebreak\midrule
+\centering \code{movieROC()} & Applicable to a `\code{groc}' or a `\code{multiroc}' object. \par \vspace{2mm}
+Save a video as a GIF illustrating the construction of the ROC curve. \par \vspace{1mm}
+- For a `\code{groc}' object: \par \vspace{1mm}
+\textsc{Main syntax:} \par \hspace{3mm}
+\code{movieROC(obj, fpr=NULL, h=c(1,1), histogram = FALSE, breaks = 15, reduce = TRUE,} \linebreak \hspace*{1.6cm} \code{completeROC = FALSE, videobar = TRUE, file = "animation1.gif", ...)} \par \vspace{1mm}
+For each element in vector \code{fpr} (optional input parameter), the function executed is \code{plot\_buildROC(obj, FPR = fpr[i], build.process = TRUE, ...)}. The vector of false-positive rates illustrated in the video is \code{NULL} by default: if length of output parameter \code{t} for \code{gROC()} function is lower than 150, such vector is taken as \code{fpr}; otherwise, an equally-space vector of length 100 covering the range of the marker values is considered.\\
+\phantom{\centering \code{plot\_buildROC()}} &
+- For a `\code{multiroc}' object: \par \vspace{1mm}
+\textsc{Main syntax:}
+\begin{itemize}
+\item for $p=2$:
+\item[] \code{movieROC(obj, fpr = NULL, file = "animation1.gif", save = TRUE, border = TRUE,} \par \hspace*{1.2cm} \code{completeROC = FALSE, ...)};
+\item for $p>2$:
+\item[] \code{movieROC(obj, fpr = NULL, display.method = c("PCA","OV"), displayOV = c(1,2),} \par
+\hspace*{1.2cm} \code{file = "animation1.gif", save = TRUE, border = TRUE, } \par
+\hspace*{1.2cm} \code{completeROC = FALSE, ...)}.
+\end{itemize}
+The video is \code{save}d by default as a GIF with the name indicated in argument \code{file} (extension \code{.gif} should be added). A \code{border} for the classification subsets is drawn by default.\par \vspace*{1mm}
+For each element in vector \code{fpr} (optional input parameter), the function executed is
+\begin{itemize}
+\item for $p=2$:
+\item[] \code{plot\_buildROC(obj, FPR = fpr[i], build.process = TRUE, completeROC, ...)};
+\item for $p>2$:
+\item[] \code{plot\_buildROC(obj, FPR = fpr[i], build.process = TRUE, display.method,} \par \hspace*{1.9cm} \code{displayOV, completeROC, ...)}
+\end{itemize}
+
+Same considerations about the input \code{fpr} as those for \code{movieROC()} over a `\code{groc}' object.\\
+\bottomrule
+\end{longtable}}
+
+
+
+
+
+
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsection{Illustrative dataset} \label{subsection:dataset}
+
+In order to illustrate the functionality of our R package, we consider the \code{HCC} data. This dataset is derived from gene expression arrays of tumor and adjacent non-tumor tissues of 62 Taiwanese cases of hepatocellular carcinoma (HCC). The goal of the original study \citep{Shen2012} was to identify, with a genome-wide approach, additional genes hypermethylated in HCC that could be used for more accurate analysis of plasma DNA for early diagnosis, by using Illumina methylation arrays (Illumina, Inc., San Diego, CA) that screen 27,578 autosomal CpG sites. The complete dataset was deposited in NCBI’s Gene Expression Omnibus (GEO) and it is available through series accession number GSE37988 (\url{www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE37988}). It is included in the presented package (\code{HCC} dataset), selecting 948 genes with complete information.
+
+The following code loads the R package and the \code{HCC} dataset (see the \href{https://cran.r-project.org/web/packages/movieROC/vignettes/movieROC_vignette.pdf}{vignette} for main structure).
+\begin{example}
+R> library(movieROC)
+R> data(HCC)
+\end{example}
+
+We selected the genes 20202438, 18384097, and 03515901. On the one hand, we chose the gene 03515901 as an example of a monotone relationship between the marker and the response, reporting a good ROC curve. On the other hand, relative gene expression intensities of the genes 20202438 and 18384097 tend to be more extreme in tissues with tumor than in those without it. These are non-standard cases, so if we limit ourselves to detect ``appropriate'' genes on the basis of the standard ROC curve, they would not be chosen. However, extending the decision rules by means of the gROC curve, those genes may be considered as potential biomarkers (locations) to differ between the two groups. The R code estimating and displaying the density probability function for gene expression intensities of the selected genes in each group (Figure~\ref{fig:densities}) is included in the \href{https://cran.r-project.org/web/packages/movieROC/vignettes/movieROC_vignette.pdf}{vignette}.
+
+
+\begin{figure}[h!]
+\includegraphics[width=\textwidth,trim={0 4mm 0 2mm},clip]{figures/Histogram_3genes.pdf}
+ \caption{Density histograms and kernel density estimations (lighter) for gene expression intensities of the genes 20202438, 18384097 and 03515901 in negative (non-tumor) and positive (tumor) tissues.}
+ \label{fig:densities}
+\end{figure}
+
+
+
+\newpage
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\section{Regular ROC curve} \label{section:regularroc}
+
+Assuming that there exists a monotone relationship between the marker and the response, the \dfn{regular, right-sided} or \dfn{standard ROC curve} associated with the marker $X$ considers classification subsets of the form $s_t=(c_t,\infty)$.
+For each specificity $1-t=\P(\chi \notin s_t) \in [0,1]$, also called \dfn{true-negative rate}, there exists only one subset $s_t$ reporting such specificity and thus a particular sensitivity, also called \dfn{true-positive rate}, $\P(\xi \in s_t)$.
+This results in a simple correspondence between each point of the ROC curve $\mathcal{R}_r(t) = 1-F_\xi \big(F_\chi^{-1}(1-t)\big)$ and its associated classification region $s_t \in \mathcal{I}_r(t)$, where
+$$\mathcal{I}_r(t) = \Big\{ s_t = (c_t, \infty) \mbox{ : } c_t \in \mathcal{S}(X) \mbox{ , } \P(\chi \in s_t) = t \Big\}$$
+is the \dfn{right-sided family of eligible classification subsets}. The definition of this family captures the shape of the decision rules and the target specificity.
+
+If higher values of the marker are associated with a higher probability of not having the characteristic (see gene 03515901 in Figure~\ref{fig:densities}), the ROC curve would be defined by the \dfn{left-sided family of eligible classification subsets} \citep{Camblor2017a}, $\mathcal{I}_l(t)$, similarly to $\mathcal{I}_r(t)$ but with the form $s_t = (\infty, c_t]$.
+%$$\mathcal{I}_l(t) = \Big\{ s_t = (\infty, c_t] \mbox{ : } c_t \in \mathcal{S}(X) \mbox{ , } \P(\chi \in s_t) = t \Big\},$$
+It results in $\mathcal{R}_l(t) = F_\xi \big(F_\chi^{-1}(t) \big)$, $t\in[0,1]$, and the decision rules are also univocally defined in this case.
+
+The ROC curve and related problems were widely studied in the literature; interested reader is referred to the monographs of \citet{Zhou2002}, \citet{Pepe2003a}, and \citet{Nakas2023}, as well as the review by \citet{Inacio2021}.
+By definition, the ROC curve is confined within the unit square, with optimal performance achieved when it approaches the left-upper corner (AUC closer to 1). Conversely, proximity to the main diagonal (AUC closer to 0.5) means diminished discriminatory ability, resembling a random classifier.
+
+In practice, let $(\xi_1, \xi_2, \dots, \xi_n)$ and $(\chi_1, \chi_2, \dots, \chi_m)$ be two independent and identically distributed (i.i.d.) samples from the positive and the negative population, respectively. Different estimation procedures are implemented in the \CRANpkg{movieROC} package, such as the empirical estimator \citep{Hsieh1996} (by default in the \code{gROC()} function), accompanied by its summary indices: the AUC and the Youden index \citep{Youden1950}. Alternatively, semiparametric approaches based on kernel density estimation for the involved distributions may be considered \citep{Zou1997}. The \code{plot\_densityROC()} function provides plots for both right- and left-sided ROC curved estimated by this method. On the other hand, assuming that the marker follows a gaussian distribution in both populations, that is, $\xi \sim \mathcal{N}(\mu_\xi, \sigma_\xi)$ and $\chi \sim \mathcal{N}(\mu_\chi, \sigma_\chi)$, parametric approaches propose plug-in estimators by estimating the unknown parameters while using the known distributions \citep{Hanley1988}. This parametric estimation is included in the \code{gROC\_param()} function, which works similarly to \code{gROC()}.
+
+\textsc{Main syntax:} \par \hspace{3mm}
+\code{gROC(X, D, side = "right", ...)} \hspace{8mm} \code{gROC\_param(X, D, side = "right", ...)}
+
+\noindent Table 1 in the \href{https://cran.r-project.org/web/packages/movieROC/vignettes/movieROC_vignette.pdf}{vignette} provides the main input and output parameters of these R functions, which estimate the regular ROC curve (right-sided of left-sided with \code{side = "right"} or \code{"left"}, respectively) and associated decision rules. Its output is an R object of class `\code{groc}`, to which the functions listed in Table~\ref{table:funcmovieROC} above can be applied. Most of them are visualization tools, but the user may also \code{print()} summary information and \code{predict()} classification regions for a particular specificity.
+
+Figure~\ref{fig:roccurves} graphically represents the empirical estimation of the standard (gray line) and generalized (black line) ROC curves for each gene in Figure~\ref{fig:densities}. To construct the standard ROC curve for the first two genes (20202438 and 18384097), the right-sided ROC curve is considered; and the left-sided curve for the third one (03515901). As expected following the discussion about Figure~\ref{fig:densities}, the standard and gROC curves are similar for the third gene because there exists a monotone relationship between the marker and the response. However, these curves differ for the first two genes due to the lack of monotonicity in those scenarios. The empirical gROC curve estimator is explained in detail in Section~\ref{section:groc}.
+
+\noindent Next chunk of code generates the figure, providing as example of the use of \code{gROC()} function, \code{plot()} and how to get access to the AUC.
+
+\begin{example}
+R> for(gene in c("20202438", "18384097", "03515901")){
++ roc <- gROC(X = HCC[,paste0("cg",gene)], D = HCC$tumor,
++ side = ifelse(gene == "03515901", "left", "right"))
++ plot(roc, col = "gray50", main = paste("Gene", gene), lwd = 3)
++ groc <- gROC(X = HCC[,paste0("cg",gene)], D = HCC$tumor, side = "both")
++ plot(groc, new = FALSE, lwd = 3)
++ legend("bottomright", paste(c("AUC =", "gAUC ="), format(c(roc$auc, groc$auc),
++ digits = 3)), col = c("gray50", "black"), lwd = 3, bty = "n", inset = .01)}
+\end{example}
+
+\begin{figure}[h!]
+ \centering
+ \includegraphics[width=.95\textwidth,trim={0 3mm 0 5mm},clip]{figures/ROCcurves_3genes.pdf}
+ \caption{Standard ROC curve (in gray) and gROC curve (in black) empirical estimation for the capacity of genes 20202438, 18384097 and 03515901 to differ between tumor and non-tumor group.}
+ \label{fig:roccurves}
+\end{figure}
+
+
+The following code snippet estimates the standard ROC curve for gene 20202438, prints its basic information, and predicts the classification region and sensitivity resulting in a specificity of 0.9. It provides an illustrative example of utilizing the \code{print()} and \code{predict()} functions.
+
+
+\begin{example}
+R> roc_selg1 <- gROC(X = HCC$cg20202438, D = HCC$tumor, side = "right")
+R> roc_selg1
+\end{example}
+\vspace*{-3mm}
+
+\begin{example}
+Data was encoded with nontumor (controls) and tumor (cases).
+It is assumed that larger values of the marker indicate larger confidence that a
+ given subject is a case.
+There are 62 controls and 62 cases.
+The specificity and sensitivity reported by the Youden index are 0.855 and 0.403,
+ respectively, corresponding to the following classification subset: (0.799, Inf).
+The area under the right-sided ROC curve (AUC) is 0.547.
+\end{example}
+
+\begin{example}
+R> predict(roc_selg1, FPR = .1)
+\end{example}
+\vspace*{-3mm}
+\begin{example}
+$ClassSubsets $Specificity $Sensitivity
+[1] 0.8063487 Inf [1] 0.9032258 [1] 0.3064516
+\end{example}
+
+%Figure~\ref{fig:classproc_roc} was generated by \code{plot\_densityROC()} and \code{plot\_buildROC()} functions as follows. These R functions compute the kernel density estimation and the empirical ROC curve estimate, respectively. The second graphic can be seen as a snapshot from the videos generated by the \code{movieROC()} function.
+
+%\begin{figure}[h!]
+% \centering
+% \includegraphics[width=.45\textwidth]{figures/plot_densityROC_gene20202438.pdf} \hspace*{4mm}
+% \includegraphics[width=.46\textwidth,trim={0 4mm 0 0mm},clip]{figures/plotbuildROC_gene20202438.pdf}
+% \caption{Classification procedure until cut-off value 0.77 for the gene 20202438 by using the \code{plot\_densityROC()} (left) and the \code{plot\_buildROC()} (right) function.}
+% \label{fig:classproc_roc}
+%\end{figure}
+
+%\begin{example}
+%R> plot_densityROC(roc_selg1, C = .77, build.process = TRUE)
+%R> plot_buildROC(roc_selg1, C = .77, build.process = TRUE, reduce = FALSE)
+%\end{example}
+
+
+\noindent The following line of code displays the whole construction of the empirical standard ROC curve for gene 20202438. The video is saved by default as a GIF with the name provided.
+\begin{example}
+R> movieROC(roc_selg1, reduce = FALSE, file = "StandardROC_gene20202438.gif")
+\end{example}
+
+
+
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsection{Multivariate ROC curve} \label{section:multiroc}
+
+In practice, many cases may benefit from combining information from different markers to enhance classification accuracy. Rather than assessing univariate markers separately, taking the multivariate marker resulting from merging them can yield relevant gain. However, note that the ROC curve and related indices are defined only for univariate markers, as they require the existence of a total order. To address this limitation, a common approach involves transforming the $p$-dimensional multivariate marker $\boldsymbol{X}$ into a univariate one through a functional transformation $\boldsymbol{h}: \mathbb{R}^p \longrightarrow \mathbb{R}$. This transformation $\boldsymbol{h}(\cdot)$ seeks to optimize an objective function related to the classification accuracy, usually the AUC \citep{Su1993, McIntosh2002, Camblor2019a}.
+
+We enumerate the methods included in the proposed R tool by the \code{multiROC()} function (with the input parameter \code{method}), listed according to the objective function to optimize. Recall that the output of \code{multiROC()} is an object of class `\code{multiroc}`, containing information about the estimation of the ROC curve and subsets for multivariate scenarios. Table 2 in the \href{https://cran.r-project.org/web/packages/movieROC/vignettes/movieROC_vignette.pdf}{vignette} includes the usage of this function.
+
+\textsc{Main syntax:} \par \hspace{3mm}
+\code{multiROC(X, D, method = "lrm", } \par \hspace{1.8cm}
+\verb"formula = 'D ~ X.1 + I(X.1^2) + X.2 + I(X.2^2) + I(X.1*X.2)'" \code{, ...)}
+
+
+\begin{itemize}
+\item[1.-] AUC: Different procedures to estimate the $\boldsymbol{h}(\cdot)$ maximizing the AUC in the multidimensional case have been studied in the literature. Among all families of functions, linear combinations ($\mathcal{L}_{\boldsymbol{\beta}}(\boldsymbol{X}) = \beta_1 X_1 + \dots + \beta_p X_p$) are widely used due to their simplicity; an extensive review of the existing methods was conducted by \citet{Kang2016}.
+\item[] Computation: In the \code{multiROC()} function, fixing input parameters \code{method = "fixedLinear"} and \code{methodLinear} to one from \code{"SuLiu"} \citep{Su1993}, \code{"PepeThompson"} \citep{Pepe2000}, or \code{"minmax"} \citep{Liu2011}. The R function also admits quadratic combinations when $p=2$, i.e. $\mathcal{Q}_{\boldsymbol{\beta}}(\boldsymbol{X}) = \beta_1 X_1 + \beta_2 X_2 + \beta_3 X_1 X_2 + \beta_4 X_1^2 + \beta_5 X_2^2$, by fixing \code{method = "fixedQuadratic"} for a particular \code{coefQuadratic} $= \boldsymbol{\beta} = (\beta_1, \dots, \beta_5)$.
+\item[2.-] The risk score function $\mbox{logit} \left\{ \P(D = 1 \, | \, \boldsymbol{X}) \right\}$: Our package allows the user to fit a logistic regression model (\code{method = "lrm"}) considering any family of functions (linear, quadratic, whether considering interactions or not...) by means of the input parameter \code{formula.lrm}. A stepwise regression model is fitted if \code{stepModel = TRUE}. Details are explained in Section~\ref{section:efficientroc}.
+\item[3.-] The sensitivity for a particular specificity:
+\begin{itemize}
+\item[a)] Considering the theoretical discussion about the search of the optimal transformation $\boldsymbol{h}(\cdot)$ pointed out in Section~\ref{section:efficientroc}, \citet{Camblor2021b} proposed to estimate it by multivariate kernel density estimation for positive and negative groups separately.
+\item[] Computation: The \code{multiROC()} function integrates the estimation procedures for the bandwidth matrix developed by \citet{Duong2007}, by fixing \code{method = "kernelOptimal"} and choosing a proper method to estimate the bandwidth (\code{"kernelOptimal.H"}).
+\item[b)] Mainly linear combinations have been explored to date in the scientific literature \citep{Meisner2021,PerezFernandez2020}. For a fixed specificity $t\in[0,1]$, we seek the linear combination $\mathcal{L}_{\boldsymbol{\beta}(t)}(\boldsymbol{X}) = \beta_1(t) X_1 + \dots + \beta_p(t) X_p$ maximizing the true-positive rate by considering standard subsets for the transformed marker.
+The coefficients $\boldsymbol{\beta}(t)$ are called `dynamic parameters' because they may be different for each $t \in [0,1]$.
+\item[] Computation: Since our objective is to display the ROC curve, $\mathcal{L}_{\boldsymbol{\beta}(t)}(\boldsymbol{X})$ is estimated for every $t$ in a grid of the unit interval, resulting in one $\boldsymbol{\hat{\beta}}(t)$ for each $t$. This approach is time-consuming, especially when it is based on the plug-in empirical estimators involved (\code{method = "dynamicEmpirical"}, only implemented for $p=2$), and may result in overfitting. Instead, \citet{Meisner2021} method is recommended (\code{method = "dynamicMeisner"}).
+\end{itemize}
+\end{itemize}
+
+
+Once the classification subsets for a multivariate marker are constructed by the \code{multiROC()} function, several R methods may be used for the output object (see Table~\ref{table:funcmovieROC}). They include \code{print} relevant information or \code{plot} the resulting ROC curve. The main contribution of the package is to plot the construction of the ROC curve together with the classification subsets in static figure for a particular FPR (\code{plot\_buildROC()} function), or in a video for tracking the whole process (\code{movieROC()} function).
+
+Figure~\ref{fig:biroc} illustrates snapshots of videos resulting from \code{movieROC()} function for two FPR: $0.1$ and $0.55$. Particularly, classification accuracy of the bivariate marker \code{(cg20202438, cg18384097)} was studied by using four different approaches indicated on the captions, considering linear combinations (top) and nonlinear transformations (bottom). This figure was implemented by the code below, integrating \code{multiROC()} and \code{movieROC()} functions. Four videos are saved as GIF files with names \code{"PepeTh.gif"} (a), \code{"Meisner.gif"} (b), \code{"LRM.gif"} (c), and \code{"KernelDens.gif"} (d).
+
+\begin{figure}[h!]
+ \begin{subfigure}{0.49\textwidth}
+ \includegraphics[width=\linewidth,page=1]{figures/movieROC_g1g2_PT.pdf}
+ \includegraphics[width=\linewidth,page=2]{figures/movieROC_g1g2_PT.pdf}
+ \caption{Linear combinations with fixed parameters by \citet{Pepe2000}.}
+ \end{subfigure} \hspace{0.01\textwidth}
+ \begin{subfigure}{0.49\textwidth}
+ \includegraphics[width=\linewidth,page=1]{figures/movieROC_g1g2_Meis.pdf}
+ \includegraphics[width=\linewidth,page=2]{figures/movieROC_g1g2_Meis.pdf}
+ \caption{Linear combinations with dynamic parameters by \citet{Meisner2021}.}
+ \end{subfigure}
+
+\end{figure}
+
+\begin{figure}[h!]\ContinuedFloat
+ \begin{subfigure}{0.49\textwidth}
+ \includegraphics[width=\linewidth,page=1]{figures/movieROC_g1g2_lrm.pdf}
+ \includegraphics[width=\linewidth,page=2]{figures/movieROC_g1g2_lrm.pdf}
+ \caption{Logistic regression model with quadratic formula by default (see \code{formula.lrm} in Table 2 of the \href{https://cran.r-project.org/web/packages/movieROC/vignettes/movieROC_vignette.pdf}{vignette}). \hspace*{2cm}}
+ \end{subfigure} \hspace{0.01\textwidth}
+ \begin{subfigure}{0.49\textwidth}
+ \includegraphics[width=\linewidth,page=1]{figures/movieROC_g1g2_optimalT.pdf}
+ \includegraphics[width=\linewidth,page=2]{figures/movieROC_g1g2_optimalT.pdf}
+ \caption{Optimal transformation by multivariate kernel density estimation with \code{"Hbcv"} method by default \citep{Camblor2021b}.}
+ \end{subfigure}
+
+ \vspace*{-1mm}
+
+\caption{Snapshots (from \code{movieROC()} videos) of the classification procedure and ROC curve for the bivariate marker \code{(cg20202438, cg18384097)} when false-positive rate equals 0.1 (top of each subfigure) and 0.55 (bottom of each subfigure). Four different methods for classification are displayed.}
+\label{fig:biroc}
+\end{figure}
+
+\begin{example}
+R> X <- HCC[ ,c("cg20202438", "cg18384097")]; D <- HCC$tumor
+R> biroc_12_PT <- multiROC(X, D, method = "fixedLinear", methodLinear = "PepeThompson")
+R> biroc_12_Meis <- multiROC(X, D, method = "dynamicMeisner", verbose = TRUE)
+R> biroc_12_lrm <- multiROC(X, D)
+R> biroc_12_kernel <- multiROC(X, D, method = "kernelOptimal")
+R> list_biroc <- list(PepeTh = biroc_12_PT, Meisner = biroc_12_Meis,
++ LRM = biroc_12_lrm, KernelDens = biroc_12_kernel)
+R> lapply(names(list_biroc), function(x) movieROC(list_biroc[[x]],
++ display.method = "OV", xlab = "Gene 20202438", ylab = "Gene 18384097",
++ cex = 1.2, alpha.points = 1, lwd.curve = 4, file = paste0(x, ".gif")))
+\end{example}
+
+
+\bigskip
+
+When the marker has a dimension higher than two it is difficult to visualize the data and the classification regions. Therefore, the \code{movieROC()} function offers two options for showing the results, both on a bidimensional space. On the one hand, to choose two of the components of the multivariate marker and project the classification subsets on the plain defined by them (Figure~\ref{fig:multiroccurve}, middle). On the other, to project the classification regions on the plain defined by the two first principal components (Figure~\ref{fig:multiroccurve}, left).
+The R function \code{prcomp()} from \pkg{stats} is used to perform Principal Components Analysis (PCA) \citep{Hotelling1933}.
+
+Figure~\ref{fig:multiroccurve} shows the difficulty in displaying the decision rules when $p>2$ (the 3 genes used along this manuscript), even with the two options implemented in our package. It was generated using \code{multiROC()} and \code{plot\_buildROC()}:
+
+\begin{example}
+R> multiroc_PT <- multiROC(X = HCC[ ,c("cg20202438", "cg18384097", "cg03515901")],
++ D = HCC$tumor, method = "fixedLinear", methodLinear = "PepeThompson")
+R> multiroc_PT
+\end{example}
+
+\begin{example}
+Data was encoded with nontumor (controls) and tumor (cases).
+There are 62 controls and 62 cases.
+A total of 3 variables have been considered.
+A linear combination with fixed parameters estimated by PepeThompson approach has
+ been considered.
+The specificity and sensitivity reported by the Youden index are 0.855 and 0.742,
+ respectively, corresponding to the cut-off point -0.0755 for the transformation
+ h(X) = 0.81*cg20202438 - 0.1*cg18384097 - 1*cg03515901.
+The area under the ROC curve (AUC) is 0.811.
+\end{example}
+
+\begin{example}
+R> plot_buildROC(multiroc_PT, cex = 1.2, lwd.curve = 4)
+R> plot_buildROC(multiroc_PT, display.method = "OV", displayOV = c(1,3), cex = 1.2,
++ xlab = "Gene 20202438", ylab = "Gene 03515901", lwd.curve = 4)
+\end{example}
+\begin{figure}[h!]
+ \centering
+ \includegraphics[height=4.6cm,page=1,trim={0 4mm 12.5cm 0},clip]{figures/buildROC_multi3.pdf}
+ \includegraphics[height=4.6cm,page=2,trim={0 4mm 0 0},clip]{figures/buildROC_multi3.pdf}
+ \caption{Multivariate ROC curve estimation for the simultaneous diagnostic accuracy of genes 20202438, 18384097 and 03515901. \citet{Pepe2000} approach was used to estimate the linear coefficients and classification rules (yellow and gray border for positive and negative class, respectively) for a FPR 0.15 are displayed. Left, projected over the 2 principal components from PCA; middle, over the 1st and the 3rd selected genes.}
+ \label{fig:multiroccurve}
+\end{figure}
+
+
+
+
+
+
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\section{Generalized ROC curve} \label{section:groc}
+
+There are scenarios whose standard ROC curves are not concave (first two genes in Figure~\ref{fig:roccurves}, gray solid line), reflecting that the standard initial assumption of existence of a monotone relationship between the marker and the response is misleading. In Figure~\ref{fig:densities}, we may see that difference in gene 20202438 distribution between those tissues who have the characteristic and those who do not is mainly on dispersion. To accommodate this common type of scenarios, \citet{Camblor2017a} extended the ROC curve definition to the case where both extremes for marker values are associated with a higher risk of having the characteristic of interest, by considering the \dfn{both-sided family of eligible classification subsets}:
+$$\mathcal{I}_g(t) = \Big\{ s_t = (-\infty,x_t^L] \cup (x_t^U, \infty) \mbox{ : } x_t^L \leq x_t^U \in \mathcal{S}(X) \mbox{ , } \P(\chi \in s_t) = t \Big\}.$$
+It becomes crucial to consider the supremum in the definition of the generalized ROC curve because the decision rule for each $t \in [0,1]$ is not univocally defined: there exist infinite pairs $x_t^L \leq x_t^U$ reporting a specificity $1-t$ (i.e. $\mathcal{I}_g(t)$ is uncountably infinite).
+Computationally, this optimization process incurs a time-consuming estimation, depending on the number of different marker values in the sample.
+
+After the introduction of this extension, several studies followed up regarding estimation of the gROC curve \citep{Camblor2017a, Camblor2019b} and related measures such as its area (gAUC) \citep{Camblor2021a} and the Youden index \citep{Camblor2019c, Bantis2021a}.
+By considering this generalization, another property of the classification subsets may be lost: the regions may not be self-contained over the increase in false-positive rate. It may happen that a subject is classified as a positive for a particular FPR $t_1$, but as a negative for a higher FPR $t_2$. Therefore, it is natural to establish a restriction \textit{(C)} on the classification subsets, ensuring that any subject classified as a positive for a fixed specificity (or sensitivity) will be also classified as a positive for any classification subset with lower specificity (higher sensitivity). \citet{PerezFernandez2020} proposed an algorithm to estimate the gROC curve under restriction \textit{(C)}, included in the \code{gROC()} function of the presented R package. See final Section for computational details about this algorithm implementation.
+
+\textsc{Main syntax:} \par \hspace{3mm}
+\code{gROC(X, D, side = "both", ...)} \hspace{8mm} \code{gROC\_param(X, D, side = "both", ...)}
+
+\noindent Table 1 in the \href{https://cran.r-project.org/web/packages/movieROC/vignettes/movieROC_vignette.pdf}{vignette} collects the input and output parameters of the \code{gROC()} function, which estimates the gROC curve, both in the mentioned direction (\code{side = "both"}) and in the opposite, i.e. when classification subsets of the form $s_t=(x_t^L, x_t^U]$ are considered (\code{side = "both2"}). In addition, all the particular methods for a `\code{groc}' object collected in Table~\ref{table:funcmovieROC} above may be used in this general scenario.
+
+%\begin{table}[h!]
+% \renewcommand{\arraystretch}{1.5}
+%{\footnotesize
+%\begin{tabular}{@{}p{1.7cm}|p{11.9cm}@{}}
+%\toprule
+%\multicolumn{2}{c}{Input parameters}\\
+%\hline
+%\centering \code{X} & Vector of marker values. \\
+%\centering \code{D} & Vector of response values. Two levels; if more, the two
+%first ones are used.\\
+%\centering \code{side} & Type of ROC curve. One of \code{"right"} ($\mathcal{R}_r(\cdot)$),
+%\code{"left"} ($\mathcal{R}_l(\cdot)$), \code{"both"} ($\mathcal{R}_g(\cdot)$) or \code{"both2"} ($\mathcal{R}_{g'}(\cdot)$). Default: \code{"right"}.\\
+%\centering \code{N} & Length of the vector of FPR used to build the ROC curve: $t \in \{ 0, 1/N, 2/N, \dots, 1 \}$. Default: \code{1000}. Only used for the \code{gROC\_param()} function.\\
+%\hdashline
+%\centering \code{restric} & If TRUE, the gROC curve with restriction \textit{(C)} is computed. Default: FALSE.\\
+%\centering \code{optim} & If TRUE (and \code{restric = TRUE}), the computation of the optimal gROC curve under restriction \textit{(C)} is performed by using Floyd's algorithm \citep{Floyd1962}. It is computed by the \code{allShortestPaths()} function in the \CRANpkg{e1071} package. Default: TRUE.\\
+%\centering \code{t0} & An integer number between 1 and $m+1$. If \code{restric = TRUE}, the restricted gROC curve is computed departing from (\code{t0}$-1$)/$m$. Default: the one reporting the Youden index.\\
+%\centering \code{t0max} & If TRUE, the computation of the gROC curve under restriction \textit{(C)} is performed departing from every possible \code{t0} and the one reporting the maximum AUC is selected.\\
+%\bottomrule
+%\end{tabular}
+%\begin{tabular}{@{}p{2cm}|p{11.6cm}@{}}
+%\toprule
+%\multicolumn{2}{c}{Output parameters}\\
+%\hline
+%\centering \code{controls, cases} & Marker values of negative and positive subjects, respectively. \\
+%\centering \code{t} & Vector of false-positive rates.\\
+%\centering \code{roc} & Vector of values of the ROC curve for \code{t}.\\
+%\centering \code{c} & Vector of marker thresholds resulting in (\code{t}, \code{roc}) if \code{side = "right" | "left"}.\\
+%\centering \code{xl, xu} & Vectors of marker thresholds resulting in (\code{t}, \code{roc}) if \code{side = "both" | "both2"}.\\
+%\centering \code{auc} & Area under the curve estimate.\\
+%\centering \code{a, b} & Estimates for parameters $a$ and $b$ used in ROC curve estimation: $\hat{a} = \left[ \overline{\xi_n} - \overline{\chi_m} \right]/\hat{s}_\xi$, $\hat{b} = \hat{s}_\chi / \hat{s}_\xi$. Only used for the \code{gROC\_param()} function.\\
+%\centering \code{p0} & Estimate of the `central value', $\mu^*$, about to which the thresholds $x^L$ and $x^U$ are symmetrical. Only used for the \code{gROC\_param()} function.\\
+%\hdashline
+%\centering \code{aucfree} & Area under the curve estimate without restrictions.\\
+%\centering \code{aucsi0} & gAUC under restriction \textit{(C)} departing from every false-positive rate, $FPR \in \{ 0, 1/m, \dots, 1 \}$.\\
+%\bottomrule
+%\end{tabular}
+%}
+%\caption{\color{yellow}
+%Most relevant parameters of the \code{gROC()} function to estimate the standard and the gROC curve. The \code{gROC\_param()} function works similarly when the binormal scenario is assumed.}
+%\label{table:gROC}
+%\end{table}
+
+
+
+Following the gene 20202438 expression intensity diagnostic accuracy is evaluated by the gROC curve without restrictions (\code{groc\_selg1} object) and under the restriction \textit{(C)} (\code{groc\_selg1\_C} object). The classification subsets and sensitivity for a specificity of $0.9$ are displayed with the \code{predict()} function.
+
+\begin{example}
+R> groc_selg1 <- gROC(X = HCC$cg20202438, D = HCC$tumor, side = "both")
+R> predict(groc_selg1, FPR = .1)
+\end{example}
+
+\begin{example}
+$ClassSubsets $Specificity $Sensitivity
+ [,1] [,2] [1] 0.9032258 [1] 0.4032258
+[1,] -Inf 0.7180623
+[2,] 0.8296072 Inf
+\end{example}
+
+\begin{example}
+R> groc_selg1_C <- gROC(X = HCC$cg20202438, D = HCC$tumor, side = "both",
++ restric = TRUE, optim = TRUE)
+\end{example}
+
+All the classification regions underlying the standard and the generalized ROC curves without and with restrictions are represented in Figure~\ref{fig:regionsgroc}. The following code was used to generate the figure, illustrating the usage and output of the \code{plot\_regions()} function. Besides displaying all the classification regions underlying every specificity (in gray), the one chosen by the user (FPR = 0.15 by default) is highlighted in blue. Note that the ROC curves are rotated $90^\circ$ to the right, in order to use the vertical axis for FPR in both plots.
+
+\begin{example}
+R> plot_regions(roc_selg1, cex.legend = 1.5, plot.auc = TRUE,
++ main = "Standard right-sided assumption [Classification subsets]")
+R> plot_regions(groc_selg1, plot.auc = TRUE, legend = F, main.plotroc = "gROC curve",
++ main = "General approach [Classification subsets]")
+R> plot_regions(groc_selg1_C, plot.auc = TRUE, legend = F, main.plotroc = "gROC curve",
++ main = "General approach with restriction (C) [Classific. subsets]",
++ xlab = "Gene 20202438 expression intensity")
+\end{example}
+
+\begin{figure}[h!]
+\centering
+ \includegraphics[width=.91\linewidth,page=1,trim={5mm 1cm 1.2cm 3mm},clip]{figures/plotregions_gene20202438.pdf}
+ \includegraphics[width=.91\linewidth,page=2,trim={5mm 1cm 1.2cm 0},clip]{figures/plotregions_gene20202438.pdf}
+ \includegraphics[width=.91\linewidth,page=3,trim={5mm 2mm 1.2cm 0},clip]{figures/plotregions_gene20202438.pdf}
+ \caption{Classification regions and the ROC curve (90º rotated) for evaluation of gene 20202438 expression intensity assuming i) standard scenario (top), ii) generalized scenario without restrictions (middle), iii) generalized scenario under restriction \textit{(C)} over the subsets (bottom).}
+ \label{fig:regionsgroc}
+\end{figure}
+
+It is clear the gain achieved for considering the generalized scenario for this marker, which fits better its distribution in each group. Standard estimated AUC is 0.547, while the gAUC increases to 0.765. The gAUC is not especially affected by imposing the restriction \textit{(C)}, resulting in 0.762.
+
+
+
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\section{Efficient ROC curve: pursuing an optimal transformation} \label{section:efficientroc}
+
+By keeping classification subsets of the form $s_t = (c_t, \infty)$, an alternative approach can be explored: transforming the univariate marker through a suitable function $h: \mathbb{R} \longrightarrow \mathbb{R}$ to enhance its accuracy.
+Henceforth, the transformation $h^*(\cdot)$ reporting the dominant ROC curve compared to the one from any other function (i.e. $\mathcal{R}_{h^*}(\cdot) \geq \mathcal{R}_h(\cdot)$) will be referred to as \dfn{optimal transformation} (in the ROC sense), and the resulting ROC curve is called eROC \citep{Kauppi2016}. Following the well-known Neyman–Pearson lemma, \citet{McIntosh2002} proved that $h^*(\cdot)$ is the likelihood ratio.
+
+We enumerate the methods included in the proposed R tool by the \code{hROC()} function (with the input parameter \code{type}), listed according to the procedure considered to estimate $h^*(\cdot)$. The output of this function is an object of class `\code{hroc}`. See Table 3 in the \href{https://cran.r-project.org/web/packages/movieROC/vignettes/movieROC_vignette.pdf}{vignette} for function usage and output details.
+
+\textsc{Main syntax:} \hspace{3mm}
+\code{hROC(X, D, type = "lrm", formula.lrm = } \verb"'D ~ pol(X,3)'" \code{, ...)}
+
+\begin{itemize}
+\item[1.-] \citet{Camblor2019a} exploited the result proved by \citet{McIntosh2002}, suggesting to estimate the logit of the risk function by logistic regression, since it is a monotone increasing transformation of the likelihood ratio.
+\item[] Computation: By the proposed R tool, the user can define any transformation $h(\cdot)$ for the right-hand side of the logistic regression model to be fitted, $\mbox{logit} \big\{ \P(D = 1 \, | \, x) \big\} = h(x)$. Particularly, by fixing the input parameters: \code{type = "lrm"} and defining the function $h(\cdot)$ as \code{formula.lrm}.
+
+\item[2.-] Arguing as in \citet{Camblor2021b} for univariate markers instead of multivariate, the optimal transformation in the ROC sense is equivalent to $h^*(\cdot)=f_\xi(\cdot)/\big(f_\xi(\cdot) + f_\chi(\cdot)\big)$, where $f(\cdot)$ denotes the density function.
+In order to estimate $h^*(\cdot)$, different estimation procedures for the density functions separately may be used, such as the kernel density estimator.
+\item[] Computation: By the \code{hROC()} function, the user may fix \code{type = "kernel"} and choosing a proper bandwidth for the kernel estimation by \code{kernel.h} in order to compute this method.
+
+\item[3.-] \citet{Camblor2019a} also included the estimation of the \dfn{overfitting function}, $h_{of}(\cdot)$, defined as the optimal one when no restrictions on the shape of $h^*(\cdot)$ are imposed. It takes the value 1 for the positive marker values and 0 for the negative ones, reporting an estimated AUC of 1, but totally depending on the available sample (the resulting rules cannot be extended).
+\item[] Computation: $h_{of}(\cdot)$ may be estimated by fixing the input parameter \code{type = "overfitting"}.
+\end{itemize}
+
+
+
+Following code and figures study the capacity of improving the classification performance of the gene 18384097 expression intensity via the above functional transformations and its impact on the final decision rules. The first one considering an ordinary cubic polynomial formula (\code{hroc\_cubic\_selg2}), and a linear tail-restricted cubic splines (\code{hroc\_rcs\_selg2}) for the right-hand side of logistic regression model. The second one using two different bandwidths ($h=1$ and $h=3$) for density function estimation. For a comparative purpose, the last one estimates the gROC curve under restriction \textit{(C)}.
+
+\begin{example}
+R> X <- HCC$cg18384097; D <- HCC$tumor
+\end{example}
+\begin{example}
+R> hroc_cubic_selg2 <- hROC(X, D); hroc_cubic_selg2
+\end{example}
+\vspace*{-3mm}
+
+\begin{example}
+Data was encoded with nontumor (controls) and tumor (cases).
+There are 62 controls and 62 cases.
+A logistic regression model of the form D ~ pol(X,3) has been performed.
+The estimated parameters of the model are the following:
+ Intercept X X^2 X^3
+ "1.551" "32.054" "-120.713" "100.449"
+The specificity and sensitivity reported by the Youden index are 0.935 and 0.532,
+ respectively, corresponding to the following classification subset:
+ (-Inf, 0.442) U (0.78, Inf).
+The area under the ROC curve (AUC) is 0.759.
+\end{example}
+
+\begin{example}
+R> hroc_rcs_selg2 <- hROC(X, D, formula.lrm = "D ~ rcs(X,8)")
+R> hroc_lkr1_selg2 <- hROC(X, D, type = "kernel")
+R> hroc_lkr3_selg2 <- hROC(X, D, type = "kernel", kernel.h = 3)
+R> hroc_overfit_selg2 <- hROC(X, D, type = "overfitting")
+
+R> groc_selg2_C <- gROC(X, D, side = "both", restric = TRUE, optim = TRUE)
+\end{example}
+
+The following code snippet compares the AUC achieved from each approach considered above:
+\begin{example}
+R> list_hroc <- list(Cubic = hroc_cubic_selg2, Splines = hroc_rcs_selg2,
++ Overfit = hroc_overfit_selg2, LikRatioEst_h3 = hroc_lkr3_selg2,
++ LikRatioEst_h1 = hroc_lkr1_selg2, gAUC_restC = groc_selg2_C)
+\end{example}
+
+\begin{example}
+R> AUCs <- sapply(list_hroc, function(x) x$auc)
+R> round(AUCs, 3)
+\end{example}
+\vspace*{-3mm}
+
+\begin{example}
+Cubic Splines Overfit LikRatioEst_h3 LikRatioEst_h1 gAUC_restC
+0.759 0.807 1.000 0.781 0.799 0.836
+\end{example}
+
+
+The shape of the classification regions over the original space $\mathcal{S}(X)$ depends on the monotonicity of $h^*(\cdot)$, which may be graphically studied by the \code{plot\_funregions()} function (see Figure~\ref{fig:transformations}). These regions can be visualized by the R function \code{plot\_regions()} (see Figure~\ref{fig:regionshroc}). Both are explained in Table~\ref{table:funcmovieROC} and illustrated below. The next chunk of code produced Figure~\ref{fig:transformations}, representing the different functional transformations estimated previously:
+\begin{example}
+R> lapply(list_hroc, function(x) plot_funregions(x, FPR = .15, FPR2 = .5))
+\end{example}
+
+\vspace*{-3mm}
+
+\begin{figure}[H]
+\centering
+\includegraphics[width=.85\textwidth]{figures/plotfunregions_hroc_lrm_gene18384097.pdf}
+ \caption{Different functional transformations and resulting classification subsets for gene 18384097. Rules for FPR 0.15 (blue) and 0.50 (red) are remarked. Top, from left to right: cubic polynomial function, restricted cubic splines (with 8 knots), and overfitted transformation. Bottom: likelihood ratio estimation with bandwidths 3 (left) and 1 (middle), and transformation resulting in the gROC curve under restriction \textit{(C)}.}
+ \label{fig:transformations}
+\end{figure}
+
+%\noindent The next chunk of code produced Figure~\ref{fig:transformations}, which represents the different functional transformations estimated previously by using the \code{plot\_funregions()} function.
+
+
+
+Finally, using the \code{plot\_regions()} function, Figure~\ref{fig:regionshroc} shows the resulting classification subsets over the original space for the best two of the six methods above. First method (fitting a logistic regression model with restricted cubic splines with 8 knots) reports an AUC of 0.804 (compared to 0.684 by the standard ROC curve), but the shape of some classification rules is complex, such as $s_t=(-\infty,a_t] \cup (b_t,c_t] \cup (d_t,\infty)$. This area increases to 0.836 by considering subsets of the form $s_t=(-\infty,x_t^L] \cup (x_t^U,\infty)$, even imposing the restriction $\textit{(C)}$ to get a functional transformation $h(\cdot)$.
+
+
+\begin{figure}[H]
+\centering
+ \includegraphics[width=.91\linewidth,page=1,trim={5mm 1cm 1.2cm 4mm},clip]{figures/plotregions_gene18384097.pdf}
+ \includegraphics[width=.91\linewidth,page=2,trim={5mm 3mm 1.2cm 0},clip]{figures/plotregions_gene18384097.pdf}
+ \caption{Classification regions and the resulting ROC curve (90º rotated) for the gene 18384097. Top, ROC curve for restricted cubic splines transformation with 8 knots; bottom, gROC curve under restriction \textit{(C)} for the original marker.}
+ \label{fig:regionshroc}
+\end{figure}
+
+
+
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\section{Summary and conclusion} \label{section:conclusion}
+
+Conducting binary classification using continuous markers requires establishment of decision rules. In the standard case, each specificity $t \in [0,1]$ entails a classification subset of the form $s_t = (c_t,\infty)$ univocally defined. However, in more complex situations -- such as there is a non-monotone relationship between the marker and the response or in multivariate scenarios -- these become not clear. Visualization of the decision rules becomes crucial in these cases. To address this, the \CRANpkg{movieROC} package incorporates novel visualization tools complementing the ROC curve representation.
+
+\noindent This R package offers a user-friendly and easily comprehensible software solution tailored for practical researchers. It implements statistical techniques to estimate and compare, and finally to graphically represent different classification procedures. While several R packages address ROC curve estimation, the proposed one emphasizes the classification process, tracking the decision rules underlying the studied binary classification problem. This tool incorporates different considerations and transformations which may be useful to capture the potential of the marker to classify in non-standard scenarios. Nevertheless, this library is also useful in standard cases, as well as when the marker itself comes from a classification or regression method (such as support vector machines), because it provides nice visuals and additional information not usually reported with the ROC curve.
+
+\noindent The main function of the package, \code{movieROC()}, allows to monitor how the resulting classification subsets change along different specificities, thereby building the corresponding ROC curve. Notably, it introduces time as a third dimension to keep those specificities, generating informative videos. For interested readers or potential users of \CRANpkg{movieROC}, the \href{https://cran.r-project.org/web/packages/movieROC/movieROC.pdf}{manual} available in CRAN provides complete information about the implemented functions and their parameters. In addition, a \href{https://cran.r-project.org/web/packages/movieROC/vignettes/movieROC_vignette.pdf}{vignette} is accessible, including mathematical formalism and details about the algorithms implemented.
+
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\section*{Computational considerations} \label{sec:complim}
+
+\paragraph{Dependencies} Some functions of our package depend on other libraries available on CRAN:
+\begin{itemize}
+\item \code{gROC(X, D, side = "both", restric = TRUE, optim = TRUE, ...)} uses the \code{allShortestPaths()} function in the \CRANpkg{e1071} package \citep{e1071CRAN2023}.
+\item \code{hROC(X, D, type = "lrm", ...)} and \code{multiROC(X, D, method = "lrm", ...)} use the \code{lrm()} function in the \CRANpkg{rms} package \citep{rmsCRAN2023}.
+\item \code{multiROC(X, D, method = "kernelOptimal", ...)} uses the \code{kde()} function in the \CRANpkg{ks} package \citep{ksCRAN2023}.
+\item \code{multiROC(X, D, method = "dynamicMeisner", ...)} uses the \code{maxTPR()} function in the \pkg{maxTPR} package \citep{Meisner2021}. This package was removed from the CRAN repository, so we integrated the code of the \code{maxTPR()} function into our package. This function uses \code{Rsolnp::solnp()} and \code{robustbase::BYlogreg()}.
+\item \code{multiROC(X, D, method = "fixedLinear", methodLinear, ...)} uses the R functions included in \citet{Kang2016} (Appendix). We integrated this code into our package.
+\item \code{movieROC(obj, save = TRUE, ...)} uses the \code{saveGIF()} function in the \CRANpkg{animation} package \citep{animationCRAN2021}.
+\end{itemize}
+
+
+\paragraph{Limitations}
+
+Users should be aware of certain limitations while working with this package:
+\begin{itemize}
+\item Some methods are potentially time-consuming, especially with medium to large sample sizes:
+
+\item[] The estimation of the gROC curve under restriction \textit{(C)} can be computationally intensive, especially when considering different FPR to locally optimize the search using \code{gROC(X, D, side = "both", restric = TRUE, optim = TRUE, t0max = TRUE)}. Note that this method involves a quite exhaustive search of the self-contained classification subsets leading to the optimal gROC curve estimate. However, even selecting different false-positive rates $t_0$ to start from, it may not result in the optimal achievable estimate under restriction \textit{(C)}. Input parameters \code{restric}, \code{optim}, \code{t0} and \code{t0max} for \code{gROC()} function included in Table 1 of the \href{https://cran.r-project.org/web/packages/movieROC/vignettes/movieROC_vignette.pdf}{vignette} serve to control this search.
+
+\item[] Similarly, it also occurs for multivariate markers when considering linear frontiers with dynamic parameters (by using \code{multiROC(X, D, method = "dynamicMeisner" | "dynamicEmpirical")}).
+\item Most implemented R functions consider empirical estimation for the resulting ROC curve, even if the procedure to estimate the decision rules is semi-parametric. An exception is the \code{gROC\_param()} function, which accommodates the binormal scenario.
+\item When visualizing classification regions for multivariate markers with high dimension \linebreak (\code{plot\_buildROC()} and \code{movieROC()} functions for a `\code{multiroc}' object), our package provides some alternatives, but additional improvements could provide further aid in interpretation.
+\end{itemize}
+
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\section*{Acknowledgements}
+
+The authors acknowledge support by the Grants PID2019-104486GB-I00 and PID2020-118101GB-I00 from Ministerio de Ciencia e Innovación (Spanish Government), and by a financial Grant for Excellence Mobility for lecturers and researchers subsidized by the University of Oviedo in collaboration with Banco Santander.
+
+
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\bibliography{PerFer-MarCam_movieROC}
+
+\address{Sonia P\' erez Fern\'andez\\
+ Department of Statistics and Operations Research and Mathematics Didactics\\
+ University of Oviedo, Asturias, Spain\\
+ (ORCiD: \href{https://orcid.org/0000-0002-2767-6399}{0000-0002-2767-6399})\\
+ \email{perezsonia@uniovi.es}}
+
+\address{Pablo Mart\'inez Camblor\\
+ Department of Anesthesiology\\
+ Geisel School of Medicine at Dartmouth, New Hampshire, USA\\
+ and\\
+ Faculty of Health Sciences\\
+ Universidad Autónoma de Chile, Chile\\
+ (ORCiD: \href{https://orcid.org/0000-0001-7845-3905}{0000-0001-7845-3905})\\
+ \email{Pablo.Martinez-Camblor@hitchcock.org}}
+
+\address{Norberto Corral Blanco\\
+ Department of Statistics and Operations Research and Mathematics Didactics\\
+ University of Oviedo, Asturias, Spain\\
+ (ORCiD: \href{https://orcid.org/0000-0002-6962-6154}{0000-0002-6962-6154})\\
+ \email{norbert@uniovi.es}}
diff --git a/_articles/RJ-2025-035/RJ-2025-035.Rmd b/_articles/RJ-2025-035/RJ-2025-035.Rmd
new file mode 100644
index 0000000000..a5b5693e6e
--- /dev/null
+++ b/_articles/RJ-2025-035/RJ-2025-035.Rmd
@@ -0,0 +1,1227 @@
+---
+title: 'movieROC: Visualizing the Decision Rules Underlying Binary Classification'
+abstract: |
+ The receiver operating characteristic (ROC) curve is a graphical tool
+ commonly used to depict the binary classification accuracy of a
+ continuous marker in terms of its sensitivity and specificity. The
+ standard ROC curve assumes a monotone relationship between the marker
+ and the response, inducing classification subsets of the form
+ $(c,\infty)$ with $c \in \mathbb{R}$. However, in non-standard cases,
+ the involved classification regions are not so clear, highlighting the
+ importance of tracking the decision rules. This paper introduces the R
+ package movieROC,
+ which provides visualization tools for understanding the ability of
+ markers to identify a characteristic of interest, complementing the
+ ROC curve representation. This tool accommodates multivariate
+ scenarios and generalizations involving different decision rules. The
+ main contribution of this package is the visualization of the
+ underlying classification regions, with the associated gain in
+ interpretability. Adding the time (videos) as a third dimension, this
+ package facilitates the visualization of binary classification in
+ multivariate problems. It constitutes a good tool to generate
+ graphical material for presentations.
+author:
+- name: Sonia Pérez-Fernández
+ affiliation: Department of Statistics and Operations Research and Mathematics Didactics,
+ University of Oviedo (Asturias, Spain)
+ orcid: 0000-0002-2767-6399
+ email: perezsonia@uniovi.es
+- name: Pablo Martínez-Camblor
+ affiliation: Department of Anesthesiology, Geisel School of Medicine at Dartmouth
+ (NH, USA); Faculty of Health Sciences, Universidad Autónoma de Chile (Chile)
+ orcid: 0000-0001-7845-3905
+ email: Pablo.Martinez-Camblor@hitchcock.org
+- name: Norberto Corral-Blanco
+ affiliation: Department of Statistics and Operations Research and Mathematics Didactics,
+ University of Oviedo (Asturias, Spain)
+ orcid: 0000-0002-6962-6154
+ email: norbert@uniovi.es
+date: '2026-02-18'
+date_received: '2024-02-09'
+journal:
+ firstpage: 59
+ lastpage: 79
+volume: 17
+issue: 4
+slug: RJ-2025-035
+packages:
+ cran:
+ - movieROC
+ - pROC
+ - ROCnReg
+ - OptimalCutpoints
+ - nsROC
+ - rms
+ - ks
+ - e1071
+ - animation
+ bioc: []
+preview: preview.png
+bibliography: PerFer-MarCam_movieROC.bib
+CTV: ~
+legacy_pdf: yes
+legacy_converted: yes
+output:
+ rjtools::rjournal_web_article:
+ self_contained: yes
+ toc: no
+ mathjax: https://cdn.jsdelivr.net/npm/mathjax@4/tex-mml-chtml.js
+ md_extension: -tex_math_single_backslash
+draft: no
+
+---
+
+
+::: article
+# Introduction
+
+The use of data to detect a characteristic of interest is a cornerstone
+of many disciplines such as medicine (to diagnose a pathology or to
+predict a patient outcome), finance (to detect fraud) or machine
+learning (to evaluate a classification algorithm), among others.
+Continuous markers are surrogate measures for the characteristic under
+study, or predictors of a potential subsequent event. They are measured
+in subjects, some of whom have the characteristic (_positive_), and some
+without it (_negative_). In addition to reliability and feasibility, a
+good marker must have two relevant properties: interpretability and
+accuracy [@Mayeux2004]. High binary classification _accuracy_ can be
+achieved if there exists a strong relationship between the marker and
+the _response_. The latter is assessed by a _gold standard_ for the presence
+or absence of the characteristic of interest. _Interpretability_ refers to
+the _decision rules_ or _subsets_ considered in the classification process.
+This piece of research seeks to elucidate both desirable properties for
+a marker by the implementation of a graphical tool in R language. We
+propose a novel approach involving the generation of videos as a
+solution to effectively capture the classification procedure for
+univariate and multivariate markers. Graphical analysis plays a pivotal
+role in data exploration, interpretation, and communication. Its
+burgeoning potential is underscored by the fast pace of technological
+advances, which empower the creation of insightful graphical
+representations.
+
+A usual practice when the binary classification accuracy of a marker is
+of interest involves the representation of the _Receiver Operating
+Characteristic (ROC) curve_, summarized by the _Area Under the Curve_ (_AUC_)
+[@Hanley1982]. The resulting plot reflects the trade-off between the
+sensitivity and the complement of the specificity. _Sensitivity_ and
+_specificity_ are probabilities of correctly classifying subjects, either
+positive or negative, respectively. Mathematically, let $\xi$ and $\chi$
+be the random variables modeling the marker values in the positive and
+the negative population, respectively, with $F_\xi(\cdot)$ and
+$F_\chi(\cdot)$ their associated cumulative distribution functions.
+Assuming that the expected value of the marker is larger in the positive
+than in the negative population, the standard ROC curve is based on
+_classification subsets_ of the form $s = (c, \infty)$, where $c$ is the
+so-called _cut-off value or threshold_ in the support of the marker $X$,
+$\mathcal{S}(X)$. One subject is classified as a positive if its marker
+value is within this region, and as a negative otherwise. This type of
+subsets has two important advantages: first, their interpretability is
+clear; second, for each specificity $1-t \in [0,1]$, the corresponding
+$s_t = (c_t, \infty)$ is univocally defined by $c_t = F_\chi^{-1}(1-t)$
+for absolutely continuous markers.
+
+When differences in marker distribution between the negative and the
+positive population are only in location but not in shape, then
+$F_\chi(\cdot) < F_\xi(\cdot)$, and the classification is direct by
+using these decision rules. However, when this is not the case, the
+standard ROC curve may cross the main diagonal, resulting in an
+_improper_ curve [@Dorfman1997]. This may be due to three different
+scenarios:
+
+i. the behavior of the marker in the two studied populations is
+ different but it is not possible to determine the decision rules.
+ Notice that the binary classification problem goes further than
+ distinction between the two populations: the classification subsets
+ should be highly likely in one population and highly unlikely in the
+ other one [@Camblor2018a];
+
+ii. there exists a relationship between the marker and the response with
+ a potential classification use, but this is not monotone;
+
+iii. there is no relationship between the marker and the response at all
+ (main diagonal ROC curve).
+
+In the second case, we have to define classification subsets different
+from standard $s_t=(c_t,\infty)$. Therefore, the use of the marker
+becomes more complex. With the aim of accommodating scenarios where both
+higher and lower values of the marker are associated with a higher risk
+of having the characteristic, @Camblor2017a proposed the so-called
+_generalized ROC (gROC) curve_. This curve tracks the highest sensitivity
+for every specificity in the unit interval resulting from subsets of the
+form $s_t=(-\infty, x_t^L] \cup (x_t^U, \infty)$ with
+$x_t^L \leq x_t^U \in \mathcal{S}(X)$.
+
+Although final decisions are based on the underlying classification
+subsets, they are typically not depicted. This omission is not a
+shortcoming in standard cases, as for each specificity $1-t \in [0,1]$,
+there is only one rule of the form $s_t = (c_t, \infty)$ with such
+specificity. Particularly, $s_t$ is univocally defined by
+$c_t = 1 - F_\chi^{-1}(1-t)$; and the same applies if we fix a
+sensitivity. Nevertheless, if the gROC curve is taken, there are
+infinite subsets of the form $s_t=(-\infty, x_t^L] \cup (x_t^U, \infty)$
+resulting in $\mathbb{P}(\chi \in s_t) = t$. This loss of univocity underlines
+the importance of reporting (numerically and/or graphically) the
+decision rules actually proposed for classifying. This gap is covered in
+the presented package.
+
+An alternative approach to assess the classification performance of a
+marker involves considering a transformation of it. This transformation
+$h(\cdot)$ aims to capture differences in distribution between the two
+populations in the ROC sense. Once $h(\cdot)$ is identified, the
+standard ROC curve for $h(X)$ is represented, resulting in the _efficient
+ROC (eROC) curve_ [@Kauppi2016]. Arguing as before, for a fixed
+specificity, the classification subsets $s_t=(c_t, \infty)$ in the
+transformed space are univocally defined, where a subject is classified
+as positive if $h(x) \in s_t$ and negative otherwise (with $x$
+representing its marker value). However, they may have any shape in the
+original space, depending on the monotonicity of the functional
+transformation $h(\cdot)$ [@Camblor2019a]. Emphasizing the importance of
+tracking the decision rules underlying the eROC curve, this monitoring
+process enables an assessment of whether the improved accuracy of the
+marker justifies the potential loss in interpretability.
+
+The ROC curve is defined for classification accuracy evaluation of
+univariate markers. To deal with multivariate markers, the usual
+practice is to consider a transformation $\boldsymbol{h}(\cdot)$ to
+reduce it to a univariate one, and then to construct the standard ROC
+curve. Same considerations as before apply when a functional
+transformation is taken. In the proposed R library, we consider methods
+from the literature to define and estimate $\boldsymbol{h}(\cdot)$ in
+the multivariate case [@Kang2016; @Meisner2021].
+
+Focusing on the classification subsets underlying the decision rules,
+the [**movieROC**](https://CRAN.R-project.org/package=movieROC) package
+incorporates methods to visualize the construction process of ROC curves
+by presenting the classification accuracy of these subsets. For
+univariate markers, the library includes both the classical (standard
+ROC curve) and the generalized (gROC curve) approach. Besides, it enables
+the display of decision rules for various transformations of the marker,
+seeking to maximize performance and allowing for flexibility in the final
+shape of the subsets (eROC curve). For multidimensional markers, the
+proposed tool visualizes the evolution of decision subsets when
+different objective functions are employed for optimization, even
+imposing restrictions on the underlying regions. In this case,
+displaying the decision rules associated with every specificity in a
+single static image is no longer feasible. Therefore, _dynamic
+representations_ (videos) are implemented, drawing on time as an extra
+dimension to capture the variation in specificity.
+
+Much software available in R could be discussed here covering diverse
+topics related to ROC curves: the
+[**pROC**](https://CRAN.R-project.org/package=pROC) package is a main
+reference including tools to visualize, estimate and compare ROC curves
+[@pROC2011]; [**ROCnReg**](https://CRAN.R-project.org/package=ROCnReg)
+explicitly considers covariate information to estimate the
+covariate-specific and the covariate-adjusted ROC curves [@ROCnReg2021];
+[**smoothROCtime**](https://CRAN.R-project.org/package=smoothROCtime)
+implements smooth estimation of time-dependent ROC curves based on the
+bivariate kernel density estimator for $(X, \textit{time-to-event})$
+[@DiazCoto2020];
+[**OptimalCutpoints**](https://CRAN.R-project.org/package=OptimalCutpoints)
+includes point and interval estimation methods for optimal thresholds
+[@OptimalCutpoints2014]; and
+[**nsROC**](https://CRAN.R-project.org/package=nsROC) performs
+non-standard analysis such as gROC estimation [@nsROC2018]; among
+others.
+
+This paper introduces and elucidates the diverse functionalities of the
+newly developed
+[**movieROC**](https://CRAN.R-project.org/package=movieROC) package,
+aimed at facilitating the visualization and comprehension of the
+decision rules underlying the binary classification process,
+encompassing various generalizations. Despite the availability of
+numerous R packages implementing related analyses, we have identified
+the main gaps covered in this library: tracking the decision rules
+underlying the ROC curve, including multivariate markers and
+non-standard scenarios (i.e. non-monotonic). The rest of the paper is
+structured as follows. In
+Section [2](#section:functionality){reference-type="ref"
+reference="section:functionality"}, we introduce the main R functions
+and objects implemented, and briefly explain the dataset employed
+throughout this manuscript to demonstrate the utility of the R library.
+Section [3](#section:regularroc){reference-type="ref"
+reference="section:regularroc"} is devoted to reconsidering the
+definition of the standard ROC curve from the perspective of
+classification subsets, including an extension to multivariate
+scenarios. Sections [4](#section:groc){reference-type="ref"
+reference="section:groc"} and
+[5](#section:efficientroc){reference-type="ref"
+reference="section:efficientroc"} revisit the gROC curve and the eROC
+curve, respectively, covering various methods to capture the potential
+classification accuracy of the marker under study. Each of these
+sections begins with a state-of-the-art overview, followed by the main
+syntax of the corresponding R functions. In addition, examples of
+implementation using the dataset presented in Section
+[2.3](#subsection:dataset){reference-type="ref"
+reference="subsection:dataset"} are provided. Finally, the paper
+concludes with a concise summary and computational details regarding the
+implemented tool.
+
+# Main functions of the movieROC package and illustrative dataset {#section:functionality}
+
+Sections [2.1](#subsec:functionality){reference-type="ref"
+reference="subsec:functionality"} and
+[2.2](#subsec:functions){reference-type="ref"
+reference="subsec:functions"} provide a detailed description of the main
+objectives of the implemented R functions. To reflect the practical
+usage of the developed R package, we employ a real dataset throughout
+this manuscript, which is introduced in
+Section [2.3](#subsection:dataset){reference-type="ref"
+reference="subsection:dataset"}.
+
+## Functionality of the movieROC package {#subsec:functionality}
+
+A graphical tool was developed to showcase static and dynamic graphics
+displaying the classification subsets derived from maximizing diagnostic
+accuracy under certain assumptions, ensuring the preservation of the
+interpretability. The R package facilitates the construction of the ROC
+curve across various specificities, providing visualizations of the
+resulting classification regions. The proposed tool comprises multiple R
+functions that generate objects with distinct class attributes (see
+function names where red arrows depart from and red nodes in
+Figure [1](#figure:functionality){reference-type="ref"
+reference="figure:functionality"}, respectively). Once the object of
+interest is created, different methods may be used, in order to plot the
+underlying regions (`plot_regions()`, `plot_funregions()`), to track the
+resulting ROC curve (`plot_buildROC()`, `plot()`), to `predict` decision
+rules for a particular specificity, and to `print` relevant information,
+among others. The main function of the package, `movieROC()`, produces
+videos to exhibit the classification procedure.
+
+```{r functionality, fig.cap="R functions of the movieROC package. The blue nodes include the names of the R functions and the red nodes indicate the different R objects that can be created and worked with. The red arrows depart from those R functions engaged in creating R objects and the black arrows indicate which R functions can be applied to which R objects. The grey dashed arrows show internal dependencies.", out.width="100%", echo=FALSE, fig.show="hold"}
+knitr::include_graphics(c(
+ "figures/movieROC_mainFunctions.png",
+ "figures/movieROC_extraFunctions.png"
+))
+```
+
+It includes algorithms to visualize the regions that underlie the binary
+classification problem, considering different approaches:
+
+- make the classification subsets flexible in order to cover
+ non-standard scenarios, by considering two cut-off values (`gROC()`
+ function); explained in
+ Section [4](#section:groc){reference-type="ref"
+ reference="section:groc"};
+
+- transform the marker by a proper function $h(\cdot)$ (`hROC()`
+ function); introduced in
+ Section [5](#section:efficientroc){reference-type="ref"
+ reference="section:efficientroc"};
+
+- when dealing with multivariate markers, consider a functional
+ transformation with some fixed or dynamic parameters resulting from
+ different methods available in the literature (`multiROC()`
+ function); covered in
+ Section [3.1](#section:multiroc){reference-type="ref"
+ reference="section:multiroc"}.
+
+## Class methods for movieROC objects {#subsec:functions}
+
+By using the `gROC()`, the `multiROC()` or the `hROC()` function, the
+user obtains an R object of class '`groc`', '`multiroc`' or '`hroc`',
+respectively. These will be called
+[**movieROC**](https://CRAN.R-project.org/package=movieROC) objects.
+Once the object of interest is created, the implemented package includes
+many functions (methods) to pass to it. Some of them are generic methods
+(`print()`, `plot()` and `predict()`), commonly used in R language over
+different objects according to their class attributes. The rest of the
+functions are specific for this library and therefore only applicable to
+[**movieROC**](https://CRAN.R-project.org/package=movieROC) objects.
+The following outline summarizes all these functions and
+provides their target and main syntax (with default input parameters).
+
+### Generic functions
+
+ + **`print()`**: Print some relevant information.
+ + **`plot()`**: Plot the ROC curve estimate.
+ + **`predict()`**: Print the classification subsets corresponding to a particular false-positive rate (`FPR`) introduced by the user. For a '`groc`' object, the user may specify a cut-off value `C` (for the standard ROC curve) or two cut-off values `XL` and `XU` (for the gROC curve).
+
+### Specific functions
+
+ * **`plot_regions()`**
+
+Applicable to a '`groc`' or a '`hroc`' object. Plot two graphics in the same figure: left, classification subsets for each false-positive rate (grey color by default); right, $90^\circ$ rotated ROC curve.
+
+ _Main syntax:_
+```{r eval = FALSE}
+ plot_regions(obj, plot.roc = TRUE, plot.auc = FALSE, FPR = 0.15, ...)
+```
+
+ If the input parameter `FPR` is specified, the corresponding classification region reporting such false-positive rate and the point in the ROC curve are highlighted in blue color.
+
+ * **`plot_funregions()`**
+
+ Applicable to a '`groc`' or a '`hroc`' object.
+Plot the transforming function and the classification subsets reporting the false-positive rate(s) indicated in the input parameter(s) `FPR` and `FPR2`.
+
+_Main syntax:_
+```{r eval = FALSE}
+plot_funregions(obj, FPR = 0.15, FPR2 = NULL, plot.subsets = TRUE, ...)
+```
+
+ * **`plot_buildROC()`**
+
+Applicable to a '`groc`' or a '`multiroc`' object.
+
+ - For a '`groc`' object: Plot four (if input `reduce` is FALSE) or two (if `reduce` is TRUE, only those on the top) graphics in the same figure: top-left, density function estimates for the marker in both populations with the areas corresponding to FPR and TPR colored (blue and red, respectively) for the optional input parameter `FPR`, `C` or `XL, XU`; top-right, the empirical ROC curve estimate; bottom-left, boxplots in both groups; bottom-right, classification subsets for every FPR (grey color).
+
+_Main syntax:_
+```{r eval = FALSE}
+plot_buildROC(obj, FPR = 0.15, C, XL, XU, h = c(1,1),
+ histogram = FALSE, breaks = 15, reduce = TRUE,
+ build.process = FALSE, completeROC = FALSE, ...)
+```
+
+If `build.process` is FALSE, the whole ROC curve is displayed; otherwise, if `completeROC` is TRUE, the portion of the ROC curve until the fixed FPR is highlighted in black and the rest is shown in gray, while if `completeROC` is FALSE, only the first portion of the curve is displayed.
+
+ - For a '`multiroc`' object: Plot two graphics in the same figure: right, the ROC curve highlighting the point and the threshold for the resulting univariate marker; left, scatterplot with the marker values in both positive (red color) and negative (blue color) subjects. About the left graphic: for $p=2$, over the original/feature bivariate space; for $p>2$, projected over two selected components of the marker (if `display.method = "OV"` with components selection in `displayOV`, `c(1,2)` by default) or the first two principal components from PCA (if `display.method = "PCA"`, default). The classification subset reporting the `FPR` selected by the user (`FPR` $\neq$ `NULL`) is displayed in gold color.
+
+_Main syntax:_
+
+for $p=2$:
+```{r eval = FALSE}
+plot_buildROC(obj, FPR = 0.15,
+ build.process = FALSE, completeROC = TRUE, ...)
+```
+
+for $p>2$:
+```{r eval = FALSE}
+plot_buildROC(obj, FPR = 0.15,
+ display.method = c("PCA","OV"), displayOV = c(1,2),
+ build.process = FALSE, completeROC = TRUE, ...)
+```
+
+If `build.process` is FALSE, the whole ROC curve is displayed; otherwise, if `completeROC` is TRUE, the portion of the ROC curve until the fixed FPR is highlighted in black and the rest is shown in gray, while if `completeROC` is FALSE, only the first portion of the curve is shown.
+
+ * **`movieROC()`**
+
+Applicable to a '`groc`' or a '`multiroc`' object. Save a video as a GIF illustrating the construction of the ROC curve.
+
+ + For a '`groc`' object:
+
+_Main syntax:_
+```{r eval = FALSE}
+movieROC(obj, fpr = NULL,
+ h = c(1,1), histogram = FALSE, breaks = 15,
+ reduce = TRUE, completeROC = FALSE, videobar = TRUE,
+ file = "animation1.gif", ...)
+```
+
+For each element in vector `fpr` (optional input parameter), the function executed is `plot_buildROC(obj, FPR = fpr[i], build.process = TRUE, ...)`. The vector of false-positive rates illustrated in the video is `NULL` by default: if length of output parameter `t` for `gROC()` function is lower than 150, such vector is taken as `fpr`; otherwise, an equally-spaced vector of length 100 covering the range of the marker values is considered.
+
+ + For a '`multiroc`' object:
+
+_Main syntax:_
+
+for $p=2$:
+```{r eval = FALSE}
+movieROC(obj, fpr = NULL,
+ file = "animation1.gif", save = TRUE,
+ border = TRUE, completeROC = FALSE, ...)
+```
+
+for $p>2$:
+```{r eval = FALSE}
+movieROC(obj, fpr = NULL,
+ display.method = c("PCA","OV"), displayOV = c(1,2),
+ file = "animation1.gif", save = TRUE,
+ border = TRUE, completeROC = FALSE, ...)
+```
+
+The video is `save`d by default as a GIF with the name indicated in argument `file` (extension `.gif` should be added). A `border` for the classification subsets is drawn by default.
+
+For each element in vector `fpr` (optional input parameter), the function executed is
+
+for $p=2$:
+```{r eval = FALSE}
+plot_buildROC(obj, FPR = fpr[i], build.process = TRUE, completeROC, ...)
+```
+
+for $p>2$:
+```{r eval = FALSE}
+plot_buildROC(obj, FPR = fpr[i], build.process = TRUE, completeROC,
+ display.method, displayOV, ...)
+```
+
+Same considerations about the input `fpr` as those for `movieROC()` over a '`groc`' object.
+
+## Illustrative dataset {#subsection:dataset}
+
+In order to illustrate the functionality of our R package, we consider
+the `HCC` data. This dataset is derived from gene expression arrays of
+tumor and adjacent non-tumor tissues of 62 Taiwanese cases of
+hepatocellular carcinoma (HCC). The goal of the original study
+[@Shen2012] was to identify, with a genome-wide approach, additional
+genes hypermethylated in HCC that could be used for more accurate
+analysis of plasma DNA for early diagnosis, by using Illumina
+methylation arrays (Illumina, Inc., San Diego, CA) that screen 27,578
+autosomal CpG sites. The complete dataset was deposited in NCBI's Gene
+Expression Omnibus (GEO) and it is available through series accession
+number GSE37988
+([www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE37988](http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE37988){.uri}).
+It is included in the presented package (`HCC` dataset), selecting 948
+genes with complete information.
+
+The following code loads the R package and the `HCC` dataset (see the
+[vignette](https://cran.r-project.org/web/packages/movieROC/vignettes/movieROC_vignette.pdf)
+for main structure).
+
+``` r
+R> library(movieROC)
+R> data(HCC)
+```
+
+We selected the genes 20202438, 18384097, and 03515901. On the one hand,
+we chose the gene 03515901 as an example of a monotone relationship
+between the marker and the response, reporting a good ROC curve. On the
+other hand, relative gene expression intensities of the genes 20202438
+and 18384097 tend to be more extreme in tissues with tumor than in those
+without it. These are non-standard cases, so if we limit ourselves to
+detect "appropriate" genes on the basis of the standard ROC curve, they
+would not be chosen. However, extending the decision rules by means of
+the gROC curve, those genes may be considered as potential biomarkers
+(locations) to differ between the two groups. The R code estimating and
+displaying the density probability function for gene expression
+intensities of the selected genes in each group
+(Figure [2](#fig:densities){reference-type="ref"
+reference="fig:densities"}) is included in the
+[vignette](https://cran.r-project.org/web/packages/movieROC/vignettes/movieROC_vignette.pdf).
+
+```{r densities, fig.cap="Density histograms and kernel density estimations (lighter) for gene expression intensities of the genes 20202438, 18384097 and 03515901 in negative (non-tumor) and positive (tumor) tissues.", fig.width=10, fig.height=4, echo=FALSE}
+knitr::include_graphics("figures/Histogram_3genes.png")
+```
+
+# Regular ROC curve {#section:regularroc}
+
+Assuming that there exists a monotone relationship between the marker
+and the response, the _regular, right-sided or standard ROC curve_
+associated with the marker $X$ considers classification subsets of the
+form $s_t=(c_t,\infty)$. For each specificity
+$1-t=\mathbb{P}(\chi \notin s_t) \in [0,1]$, also called _true-negative rate_,
+there exists only one subset $s_t$ reporting such specificity and thus a
+particular sensitivity, also called _true-positive rate_,
+$\mathbb{P}(\xi \in s_t)$. This results in a simple correspondence between each
+point of the ROC curve
+$\mathcal{R}_r(t) = 1-F_\xi \big(F_\chi^{-1}(1-t)\big)$ and its
+associated classification region $s_t \in \mathcal{I}_r(t)$, where
+$$\mathcal{I}_r(t) = \Big\{ s_t = (c_t, \infty) : c_t \in \mathcal{S}(X) , \mathbb{P}(\chi \in s_t) = t \Big\}$$
+is the _right-sided family of eligible classification subsets_. The
+definition of this family captures the shape of the decision rules and
+the target specificity.
+
+If higher values of the marker are associated with a higher probability
+of not having the characteristic (see gene 03515901 in
+Figure [2](#fig:densities){reference-type="ref"
+reference="fig:densities"}), the ROC curve would be defined by the
+_left-sided family of eligible classification subsets_ [@Camblor2017a],
+$\mathcal{I}_l(t)$, similarly to $\mathcal{I}_r(t)$ but with the form
+$s_t = (\infty, c_t]$. It results in
+$\mathcal{R}_l(t) = F_\xi \big(F_\chi^{-1}(t) \big)$, $t\in[0,1]$, and
+the decision rules are also univocally defined in this case.
+
+The ROC curve and related problems were widely studied in the
+literature; interested readers are referred to the monographs of
+@Zhou2002, @Pepe2003a, and @Nakas2023, as well as the review by
+@Inacio2021. By definition, the ROC curve is confined within the unit
+square, with optimal performance achieved when it approaches the
+left-upper corner (AUC closer to 1). Conversely, proximity to the main
+diagonal (AUC closer to 0.5) means diminished discriminatory ability,
+resembling a random classifier.
+
+In practice, let $(\xi_1, \xi_2, \dots, \xi_n)$ and
+$(\chi_1, \chi_2, \dots, \chi_m)$ be two independent and identically
+distributed (i.i.d.) samples from the positive and the negative
+population, respectively. Different estimation procedures are
+implemented in the
+[**movieROC**](https://CRAN.R-project.org/package=movieROC) package,
+such as the empirical estimator [@Hsieh1996] (by default in the `gROC()`
+function), accompanied by its summary indices: the AUC and the Youden
+index [@Youden1950]. Alternatively, semiparametric approaches based on
+kernel density estimation for the involved distributions may be
+considered [@Zou1997]. The `plot_densityROC()` function provides plots
+for both right- and left-sided ROC curves estimated by this method. On
+the other hand, assuming that the marker follows a gaussian distribution
+in both populations, that is,
+$\xi \sim \mathcal{N}(\mu_\xi, \sigma_\xi)$ and
+$\chi \sim \mathcal{N}(\mu_\chi, \sigma_\chi)$, parametric approaches
+propose plug-in estimators by estimating the unknown parameters while
+using the known distributions [@Hanley1988]. This parametric estimation
+is included in the `gROC_param()` function, which works similarly to
+`gROC()`.
+
+_Main syntax:_
+
+```{r eval = FALSE}
+gROC(X, D, side = "right", ...)
+gROC_param(X, D, side = "right", ...)
+```
+
+Table 1 in the
+[vignette](https://cran.r-project.org/web/packages/movieROC/vignettes/movieROC_vignette.pdf)
+provides the main input and output parameters of these R functions,
+which estimate the regular ROC curve (right-sided or left-sided with
+`side = "right"` or `"left"`, respectively) and associated decision
+rules. Its output is an R object of class '`groc`', to which the
+functions listed in Section [2.2](#subsec:functions){reference-type="ref"
+reference="subsec:functions"} can be applied. Most of them are
+visualization tools, but the user may also `print()` summary information
+and `predict()` classification regions for a particular specificity.
+
+Figure [3](#fig:roccurves){reference-type="ref"
+reference="fig:roccurves"} graphically represents the empirical
+estimation of the standard (gray line) and generalized (black line) ROC
+curves for each gene in Figure [2](#fig:densities){reference-type="ref"
+reference="fig:densities"}. To construct the standard ROC curve for the
+first two genes (20202438 and 18384097), the right-sided ROC curve is
+considered; and the left-sided curve for the third one (03515901). As
+expected following the discussion about
+Figure [2](#fig:densities){reference-type="ref"
+reference="fig:densities"}, the standard and gROC curves are similar for
+the third gene because there exists a monotone relationship between the
+marker and the response. However, these curves differ for the first two
+genes due to the lack of monotonicity in those scenarios. The empirical
+gROC curve estimator is explained in detail in
+Section [4](#section:groc){reference-type="ref"
+reference="section:groc"}.
+
+Next chunk of code generates the figure, providing an example of the use
+of `gROC()` function, `plot()` and how to get access to the AUC.
+
+``` r
+R> for(gene in c("20202438", "18384097", "03515901")){
++ roc <- gROC(X = HCC[,paste0("cg",gene)], D = HCC$tumor,
++ side = ifelse(gene == "03515901", "left", "right"))
++ plot(roc, col = "gray50", main = paste("Gene", gene), lwd = 3)
++ groc <- gROC(X = HCC[,paste0("cg",gene)], D = HCC$tumor, side = "both")
++ plot(groc, new = FALSE, lwd = 3)
++ legend("bottomright", paste(c("AUC =", "gAUC ="), format(c(roc$auc, groc$auc),
++ digits = 3)), col = c("gray50", "black"), lwd = 3, bty = "n", inset = .01)}
+```
+```{r roccurves, fig.cap="Standard ROC curve (in gray) and gROC curve (in black) empirical estimation for the capacity of genes 20202438, 18384097 and 03515901 to differ between tumor and non-tumor group.", out.width="100%", echo=FALSE}
+knitr::include_graphics("figures/ROCcurves_3genes.png")
+```
+
+The following code snippet estimates the standard ROC curve for gene
+20202438, prints its basic information, and predicts the classification
+region and sensitivity resulting in a specificity of 0.9. It provides an
+illustrative example of utilizing the `print()` and `predict()`
+functions.
+
+``` r
+R> roc_selg1 <- gROC(X = HCC$cg20202438, D = HCC$tumor, side = "right")
+R> roc_selg1
+```
+
+``` r
+Data was encoded with nontumor (controls) and tumor (cases).
+It is assumed that larger values of the marker indicate larger confidence that a
+ given subject is a case.
+There are 62 controls and 62 cases.
+The specificity and sensitivity reported by the Youden index are 0.855 and 0.403,
+ respectively, corresponding to the following classification subset: (0.799, Inf).
+The area under the right-sided ROC curve (AUC) is 0.547.
+```
+
+``` r
+R> predict(roc_selg1, FPR = .1)
+```
+
+``` r
+$ClassSubsets $Specificity $Sensitivity
+[1] 0.8063487 Inf [1] 0.9032258 [1] 0.3064516
+```
+
+The following line of code displays the whole construction of the
+empirical standard ROC curve for gene 20202438. The video is saved by
+default as a GIF with the name provided.
+
+``` r
+R> movieROC(roc_selg1, reduce = FALSE, file = "StandardROC_gene20202438.gif")
+```
+{width=60%}
+
+## Multivariate ROC curve {#section:multiroc}
+
+In practice, many cases may benefit from combining information from
+different markers to enhance classification accuracy. Rather than
+assessing univariate markers separately, taking the multivariate marker
+resulting from merging them can yield relevant gain. However, note that
+the ROC curve and related indices are defined only for univariate
+markers, as they require the existence of a total order. To address this
+limitation, a common approach involves transforming the $p$-dimensional
+multivariate marker $\boldsymbol{X}$ into a univariate one through a
+functional transformation
+$\boldsymbol{h}: \mathbb{R}^p \longrightarrow \mathbb{R}$. This
+transformation $\boldsymbol{h}(\cdot)$ seeks to optimize an objective
+function related to the classification accuracy, usually the AUC
+[@Su1993; @McIntosh2002; @Camblor2019a].
+
+We enumerate the methods included in the proposed R tool by the
+`multiROC()` function (with the input parameter `method`), listed
+according to the objective function to optimize. Recall that the output
+of `multiROC()` is an object of class '`multiroc`', containing
+information about the estimation of the ROC curve and subsets for
+multivariate scenarios. Table 2 in the
+[vignette](https://cran.r-project.org/web/packages/movieROC/vignettes/movieROC_vignette.pdf)
+includes the usage of this function.
+
+_Main syntax:_
+
+```{r eval = FALSE}
+multiROC(X, D, method = "lrm",
+ formula = 'D ~ X.1 + I(X.1^2) + X.2 + I(X.2^2) + I(X.1*X.2)', ...)
+```
+
+1. AUC: Different procedures to estimate the $\boldsymbol{h}(\cdot)$
+ maximizing the AUC in the multidimensional case have been studied in
+ the literature. Among all families of functions, linear combinations
+ ($\mathcal{L}_{\boldsymbol{\beta}}(\boldsymbol{X}) = \beta_1 X_1 + \dots + \beta_p X_p$)
+ are widely used due to their simplicity; an extensive review of the
+ existing methods was conducted by @Kang2016.
+
+- Computation: In the `multiROC()` function, fixing input parameters
+ `method = "fixedLinear"` and `methodLinear` to one from `"SuLiu"`
+ [@Su1993], `"PepeThompson"` [@Pepe2000], or `"minmax"` [@Liu2011].
+ The R function also admits quadratic combinations when $p=2$, i.e.
+ $\mathcal{Q}_{\boldsymbol{\beta}}(\boldsymbol{X}) = \beta_1 X_1 + \beta_2 X_2 + \beta_3 X_1 X_2 + \beta_4 X_1^2 + \beta_5 X_2^2$,
+ by fixing `method = "fixedQuadratic"` for a particular
+ `coefQuadratic` $= \boldsymbol{\beta} = (\beta_1, \dots, \beta_5)$.
+
+2. The risk score function
+ logit$\left\{ \mathbb{P}(D = 1 \, | \, \boldsymbol{X}) \right\}$: Our
+ package allows the user to fit a logistic regression model
+ (`method = "lrm"`) considering any family of functions (linear,
+ quadratic, whether considering interactions or not\...) by means of
+ the input parameter `formula.lrm`. A stepwise regression model is
+ fitted if `stepModel = TRUE`. Details are explained in
+ Section [5](#section:efficientroc){reference-type="ref"
+ reference="section:efficientroc"}.
+
+3. The sensitivity for a particular specificity:
+
+a. Considering the theoretical discussion about the search of the
+ optimal transformation $\boldsymbol{h}(\cdot)$ pointed out in
+ Section [5](#section:efficientroc){reference-type="ref"
+ reference="section:efficientroc"}, @Camblor2021b proposed to
+ estimate it by multivariate kernel density estimation for
+ positive and negative groups separately.
+
+ - Computation: The `multiROC()` function integrates the estimation
+ procedures for the bandwidth matrix developed by @Duong2007, by
+ fixing `method = "kernelOptimal"` and choosing a proper method
+ to estimate the bandwidth (`"kernelOptimal.H"`).
+
+b. Mainly linear combinations have been explored to date in the
+ scientific literature [@Meisner2021; @PerezFernandez2020]. For a
+ fixed specificity $t\in[0,1]$, we seek the linear combination
+ $\mathcal{L}_{\boldsymbol{\beta}(t)}(\boldsymbol{X}) = \beta_1(t) X_1 + \dots + \beta_p(t) X_p$
+ maximizing the true-positive rate by considering standard
+ subsets for the transformed marker. The coefficients
+ $\boldsymbol{\beta}(t)$ are called 'dynamic parameters' because
+ they may be different for each $t \in [0,1]$.
+
+ - Computation: Since our objective is to display the ROC curve,
+ $\mathcal{L}_{\boldsymbol{\beta}(t)}(\boldsymbol{X})$ is
+ estimated for every $t$ in a grid of the unit interval,
+ resulting in one $\boldsymbol{\hat{\beta}}(t)$ for each $t$.
+ This approach is time-consuming, especially when it is based on
+ the plug-in empirical estimators involved
+ (`method = "dynamicEmpirical"`, only implemented for $p=2$), and
+ may result in overfitting. Instead, @Meisner2021 method is
+ recommended (`method = "dynamicMeisner"`).
+
+Once the classification subsets for a multivariate marker are
+constructed by the `multiROC()` function, several R methods may be used
+for the output object (see Section [2.2](#subsec:functions){reference-type="ref"
+reference="subsec:functions"}). They include `print` relevant
+information or `plot` the resulting ROC curve. The main contribution of
+the package is to plot the construction of the ROC curve together with
+the classification subsets in a static figure for a particular FPR
+(`plot_buildROC()` function), or in a video for tracking the whole
+process (`movieROC()` function).
+
+Figure \@ref(fig:biroc)
+illustrates the videos resulting from `movieROC()` function.
+Particularly, classification accuracy of the
+bivariate marker `(cg20202438, cg18384097)` was studied by using four
+different approaches indicated on the captions, considering linear
+combinations (top) and nonlinear transformations (bottom). This figure
+was implemented by the code below, integrating `multiROC()` and
+`movieROC()` functions. Four videos are saved as GIF files with names
+`"PepeTh.gif"` (a), `"Meisner.gif"` (b), `"LRM.gif"` (c), and
+`"KernelDens.gif"` (d).
+
+``` r
+R> X <- HCC[ ,c("cg20202438", "cg18384097")]; D <- HCC$tumor
+R> biroc_12_PT <- multiROC(X, D, method = "fixedLinear", methodLinear = "PepeThompson")
+R> biroc_12_Meis <- multiROC(X, D, method = "dynamicMeisner", verbose = TRUE)
+R> biroc_12_lrm <- multiROC(X, D)
+R> biroc_12_kernel <- multiROC(X, D, method = "kernelOptimal")
+R> list_biroc <- list(PepeTh = biroc_12_PT, Meisner = biroc_12_Meis,
++ LRM = biroc_12_lrm, KernelDens = biroc_12_kernel)
+R> lapply(names(list_biroc), function(x) movieROC(list_biroc[[x]],
++ display.method = "OV", xlab = "Gene 20202438", ylab = "Gene 18384097",
++ cex = 1.2, alpha.points = 1, lwd.curve = 4, file = paste0(x, ".gif")))
+```
+
+::: {.figure}
+
+| {width=100%} | {width=100%} |
+|:--------------------------------------------------------:|:-----------------------------------------------------:|
+| **(a)** Linear combinations with fixed parameters by @Pepe2000. | **(b)** Linear combinations with dynamic parameters by @Meisner2021. |
+
+| {width=100%} | {width=100%} |
+|:--------------------------------------------------------:|:---------------------------------------------------------:|
+| **(c)** Logistic regression model with quadratic formula by default (see `formula.lrm` in Table 2 of the [vignette](https://cran.r-project.org/web/packages/movieROC/vignettes/movieROC_vignette.pdf)). | **(d)** Optimal transformation by multivariate kernel density estimation with `"Hbcv"` method by default [@Camblor2021b]. |
+:::
+
+```{r biroc, echo=FALSE, out.width="1px", out.height="1px", fig.cap="Videos (from `movieROC()` function) of the classification procedure and ROC curve for the bivariate marker `cg20202438`, `cg18384097`). Four different methods for classification are displayed."}
+knitr::include_graphics("transparent.png")
+```
+
+When the marker has a dimension higher than two it is difficult to
+visualize the data and the classification regions. Therefore, the
+`movieROC()` function offers two options for showing the results, both
+on a bidimensional space. On the one hand, to choose two of the
+components of the multivariate marker and project the classification
+subsets on the plane defined by them
+(Figure [5](#fig:multiroccurve){reference-type="ref"
+reference="fig:multiroccurve"}, middle). On the other, to project the
+classification regions on the plane defined by the two first principal
+components (Figure [5](#fig:multiroccurve){reference-type="ref"
+reference="fig:multiroccurve"}, left). The R function `prcomp()` from `stats`
+is used to perform Principal Components Analysis (PCA) [@Hotelling1933].
+
+Figure [5](#fig:multiroccurve){reference-type="ref"
+reference="fig:multiroccurve"} shows the difficulty in displaying the
+decision rules when $p>2$ (the 3 genes used along this manuscript), even
+with the two options implemented in our package. It was generated using
+`multiROC()` and `plot_buildROC()`:
+
+``` r
+R> multiroc_PT <- multiROC(X = HCC[ ,c("cg20202438", "cg18384097", "cg03515901")],
++ D = HCC$tumor, method = "fixedLinear", methodLinear = "PepeThompson")
+R> multiroc_PT
+```
+
+``` r
+Data was encoded with nontumor (controls) and tumor (cases).
+There are 62 controls and 62 cases.
+A total of 3 variables have been considered.
+A linear combination with fixed parameters estimated by PepeThompson approach has
+ been considered.
+The specificity and sensitivity reported by the Youden index are 0.855 and 0.742,
+ respectively, corresponding to the cut-off point -0.0755 for the transformation
+ h(X) = 0.81*cg20202438 - 0.1*cg18384097 - 1*cg03515901.
+The area under the ROC curve (AUC) is 0.811.
+```
+
+``` r
+R> plot_buildROC(multiroc_PT, cex = 1.2, lwd.curve = 4)
+R> plot_buildROC(multiroc_PT, display.method = "OV", displayOV = c(1,3), cex = 1.2,
++ xlab = "Gene 20202438", ylab = "Gene 03515901", lwd.curve = 4)
+```
+
+```{r multiroccurve, fig.cap="Multivariate ROC curve estimation for the simultaneous diagnostic accuracy of genes 20202438, 18384097 and 03515901. Pepe and Thompson (2000) approach was used to estimate the linear coefficients and classification rules (yellow and gray border for positive and negative class, respectively) for a FPR 0.15 are displayed. Left, projected over the 2 principal components from PCA; middle, over the 1st and the 3rd selected genes.", out.width="100%", echo=FALSE}
+knitr::include_graphics("figures/buildROC_multi3_full.png")
+```
+
+# Generalized ROC curve {#section:groc}
+
+There are scenarios whose standard ROC curves are not concave (first two
+genes in Figure [3](#fig:roccurves){reference-type="ref"
+reference="fig:roccurves"}, gray solid line), reflecting that the
+standard initial assumption of existence of a monotone relationship
+between the marker and the response is misleading. In
+Figure [2](#fig:densities){reference-type="ref"
+reference="fig:densities"}, we may see that difference in gene 20202438
+distribution between those tissues which have the characteristic and those
+which do not is mainly in dispersion. To accommodate this common type of
+scenarios, @Camblor2017a extended the ROC curve definition to the case
+where both extremes for marker values are associated with a higher risk
+of having the characteristic of interest, by considering the _both-sided
+family of eligible classification subsets_:
+$$\mathcal{I}_g(t) = \Big\{ s_t = (-\infty,x_t^L] \cup (x_t^U, \infty) : x_t^L \leq x_t^U \in \mathcal{S}(X) , \mathbb{P}(\chi \in s_t) = t \Big\}.$$
+
+It becomes crucial to consider the supremum in the definition of the
+generalized ROC curve because the decision rule for each $t \in [0,1]$
+is not univocally defined: there exist infinite pairs $x_t^L \leq x_t^U$
+reporting a specificity $1-t$ (i.e. $\mathcal{I}_g(t)$ is uncountably
+infinite). Computationally, this optimization process incurs a
+time-consuming estimation, depending on the number of different marker
+values in the sample.
+
+After the introduction of this extension, several studies followed up
+regarding estimation of the gROC curve [@Camblor2017a; @Camblor2019b]
+and related measures such as its area (gAUC) [@Camblor2021a] and the
+Youden index [@Camblor2019c; @Bantis2021a]. By considering this
+generalization, another property of the classification subsets may be
+lost: the regions may not be self-contained over the increase in
+false-positive rate. It may happen that a subject is classified as a
+positive for a particular FPR $t_1$, but as a negative for a higher FPR
+$t_2$. Therefore, it is natural to establish a restriction *(C)* on the
+classification subsets, ensuring that any subject classified as a
+positive for a fixed specificity (or sensitivity) will also be
+classified as a positive for any classification subset with lower
+specificity (higher sensitivity). @PerezFernandez2020 proposed an
+algorithm to estimate the gROC curve under restriction *(C)*, included
+in the `gROC()` function of the presented R package. See final Section
+for computational details about this algorithm implementation.
+
+_Main syntax:_
+
+```{r eval = FALSE}
+gROC(X, D, side = "both", ...)
+gROC_param(X, D, side = "both", ...)
+```
+
+Table 1 in the
+[vignette](https://cran.r-project.org/web/packages/movieROC/vignettes/movieROC_vignette.pdf)
+collects the input and output parameters of the `gROC()` function, which
+estimates the gROC curve, both in the mentioned direction
+(`side = "both"`) and in the opposite, i.e. when classification subsets
+of the form $s_t=(x_t^L, x_t^U]$ are considered (`side = "both2"`). In
+addition, all the particular methods for a '`groc`' object collected in
+Section [2.2](#subsec:functions){reference-type="ref"
+reference="subsec:functions"} may be used in this general
+scenario.
+
+Following, the gene 20202438 expression intensity diagnostic accuracy is
+evaluated by the gROC curve without restrictions (`groc_selg1` object)
+and under the restriction *(C)* (`groc_selg1_C` object). The
+classification subsets and sensitivity for a specificity of $0.9$ are
+displayed with the `predict()` function.
+
+``` r
+R> groc_selg1 <- gROC(X = HCC$cg20202438, D = HCC$tumor, side = "both")
+R> predict(groc_selg1, FPR = .1)
+```
+
+``` r
+$ClassSubsets $Specificity $Sensitivity
+ [,1] [,2] [1] 0.9032258 [1] 0.4032258
+[1,] -Inf 0.7180623
+[2,] 0.8296072 Inf
+```
+
+``` r
+R> groc_selg1_C <- gROC(X = HCC$cg20202438, D = HCC$tumor, side = "both",
++ restric = TRUE, optim = TRUE)
+```
+
+All the classification regions underlying the standard and the
+generalized ROC curves without and with restrictions are represented in
+Figure [6](#fig:regionsgroc){reference-type="ref"
+reference="fig:regionsgroc"}. The following code was used to generate
+the figure, illustrating the usage and output of the `plot_regions()`
+function. Besides displaying all the classification regions underlying
+every specificity (in gray), the one chosen by the user (FPR = 0.15 by
+default) is highlighted in blue. Note that the ROC curves are rotated
+$90^\circ$ to the right, in order to use the vertical axis for FPR in
+both plots.
+
+``` r
+R> plot_regions(roc_selg1, cex.legend = 1.5, plot.auc = TRUE,
++ main = "Standard right-sided assumption [Classification subsets]")
+R> plot_regions(groc_selg1, plot.auc = TRUE, legend = F,
++ main.plotroc = "gROC curve",
++ main = "General approach [Classification subsets]")
+R> plot_regions(groc_selg1_C, plot.auc = TRUE, legend = F,
++ main.plotroc = "gROC curve",
++ main = "General approach with restriction (C) [Classific. subsets]",
++ xlab = "Gene 20202438 expression intensity")
+```
+
+```{r regionsgroc, fig.cap="Classification regions and the ROC curve (90º rotated) for evaluation of gene 20202438 expression intensity assuming i) standard scenario (top), ii) generalized scenario without restrictions (middle), iii) generalized scenario under restriction *(C)* over the subsets (bottom).", out.width="100%", echo=FALSE, fig.show="hold"}
+knitr::include_graphics(c("figures/plotregions_gene20202438_1.png", "figures/plotregions_gene20202438_2.png", "figures/plotregions_gene20202438_3.png"))
+```
+
+
+It is clear the gain achieved for considering the generalized scenario
+for this marker, which fits better its distribution in each group.
+Standard estimated AUC is 0.547, while the gAUC increases to 0.765. The
+gAUC is not especially affected by imposing the restriction *(C)*,
+resulting in 0.762.
+
+# Efficient ROC curve: pursuing an optimal transformation {#section:efficientroc}
+
+By keeping classification subsets of the form $s_t = (c_t, \infty)$, an
+alternative approach can be explored: transforming the univariate marker
+through a suitable function $h: \mathbb{R} \longrightarrow \mathbb{R}$
+to enhance its accuracy. Henceforth, the transformation $h^*(\cdot)$
+reporting the dominant ROC curve compared to the one from any other
+function (i.e. $\mathcal{R}_{h^*}(\cdot) \geq \mathcal{R}_h(\cdot)$)
+will be referred to as _optimal transformation_ (in the ROC sense), and
+the resulting ROC curve is called eROC [@Kauppi2016]. Following the
+well-known Neyman--Pearson lemma, @McIntosh2002 proved that $h^*(\cdot)$
+is the likelihood ratio.
+
+We enumerate the methods included in the proposed R tool by the `hROC()`
+function (with the input parameter `type`), listed according to the
+procedure considered to estimate $h^*(\cdot)$. The output of this
+function is an object of class '`hroc`'. See Table 3 in the
+[vignette](https://cran.r-project.org/web/packages/movieROC/vignettes/movieROC_vignette.pdf)
+for function usage and output details.
+
+_Main syntax:_
+
+```{r eval = FALSE}
+hROC(X, D, type = "lrm", formula.lrm = 'D ~ pol(X,3)', ...)
+```
+
+1. @Camblor2019a exploited the result proved by @McIntosh2002,
+ suggesting to estimate the logit of the risk function by logistic
+ regression, since it is a monotone increasing transformation of the
+ likelihood ratio.
+
+- Computation: By the proposed R tool, the user can define any
+ transformation $h(\cdot)$ for the right-hand side of the logistic
+ regression model to be fitted,
+ logit$\big\{ \mathbb{P}(D = 1 \, | \, x) \big\} = h(x)$. Particularly, by
+ fixing the input parameters: `type = "lrm"` and defining the
+ function $h(\cdot)$ as `formula.lrm`.
+
+2. Arguing as in @Camblor2021b for univariate markers instead of
+ multivariate, the optimal transformation in the ROC sense is
+ equivalent to
+ $h^*(\cdot)=f_\xi(\cdot)/\big(f_\xi(\cdot) + f_\chi(\cdot)\big)$,
+ where $f(\cdot)$ denotes the density function. In order to estimate
+ $h^*(\cdot)$, different estimation procedures for the density
+ functions separately may be used, such as the kernel density
+ estimator.
+
+- Computation: By the `hROC()` function, the user may fix
+ `type = "kernel"` and choose a proper bandwidth for the kernel
+ estimation by `kernel.h` in order to compute this method.
+
+3. @Camblor2019a also included the estimation of the _overfitting
+ function_, $h_{of}(\cdot)$, defined as the optimal one when no
+ restrictions on the shape of $h^*(\cdot)$ are imposed. It takes the
+ value 1 for the positive marker values and 0 for the negative ones,
+ reporting an estimated AUC of 1, but totally depending on the
+ available sample (the resulting rules cannot be extended).
+
+- Computation: $h_{of}(\cdot)$ may be estimated by fixing the input
+ parameter `type = "overfitting"`.
+
+The following code and figures study the capacity of improving the
+classification performance of the gene 18384097 expression intensity via
+the above functional transformations and its impact on the final
+decision rules. The first one considers an ordinary cubic polynomial
+formula (`hroc_cubic_selg2`), and a linear tail-restricted cubic spline
+(`hroc_rcs_selg2`) for the right-hand side of the logistic regression model.
+The second one uses two different bandwidths ($h=1$ and $h=3$) for
+density function estimation. For a comparative purpose, the last one
+estimates the gROC curve under restriction *(C)*.
+
+``` r
+R> X <- HCC$cg18384097; D <- HCC$tumor
+```
+
+``` r
+R> hroc_cubic_selg2 <- hROC(X, D); hroc_cubic_selg2
+```
+
+``` r
+Data was encoded with nontumor (controls) and tumor (cases).
+There are 62 controls and 62 cases.
+A logistic regression model of the form D ~ pol(X,3) has been performed.
+The estimated parameters of the model are the following:
+ Intercept X X^2 X^3
+ "1.551" "32.054" "-120.713" "100.449"
+The specificity and sensitivity reported by the Youden index are 0.935 and 0.532,
+ respectively, corresponding to the following classification subset:
+ (-Inf, 0.442) U (0.78, Inf).
+The area under the ROC curve (AUC) is 0.759.
+```
+
+``` r
+R> hroc_rcs_selg2 <- hROC(X, D, formula.lrm = "D ~ rcs(X,8)")
+R> hroc_lkr1_selg2 <- hROC(X, D, type = "kernel")
+R> hroc_lkr3_selg2 <- hROC(X, D, type = "kernel", kernel.h = 3)
+R> hroc_overfit_selg2 <- hROC(X, D, type = "overfitting")
+
+R> groc_selg2_C <- gROC(X, D, side = "both", restric = TRUE, optim = TRUE)
+```
+
+The following code snippet compares the AUC achieved from each approach
+considered above:
+
+``` r
+R> list_hroc <- list(Cubic = hroc_cubic_selg2, Splines = hroc_rcs_selg2,
++ Overfit = hroc_overfit_selg2, LikRatioEst_h3 = hroc_lkr3_selg2,
++ LikRatioEst_h1 = hroc_lkr1_selg2, gAUC_restC = groc_selg2_C)
+```
+
+``` r
+R> AUCs <- sapply(list_hroc, function(x) x$auc)
+R> round(AUCs, 3)
+```
+
+``` r
+Cubic Splines Overfit LikRatioEst_h3 LikRatioEst_h1 gAUC_restC
+0.759 0.807 1.000 0.781 0.799 0.836
+```
+
+The shape of the classification regions over the original space
+$\mathcal{S}(X)$ depends on the monotonicity of $h^*(\cdot)$, which may
+be graphically studied by the `plot_funregions()` function (see
+Figure [7](#fig:transformations){reference-type="ref"
+reference="fig:transformations"}). These regions can be visualized by
+the R function `plot_regions()` (see
+Figure [8](#fig:regionshroc){reference-type="ref"
+reference="fig:regionshroc"}). Both are explained in
+Section [2.2](#subsec:functions){reference-type="ref"
+reference="subsec:functions"} and illustrated below. The next chunk of
+code produced Figure [7](#fig:transformations){reference-type="ref"
+reference="fig:transformations"}, representing the different functional
+transformations estimated previously:
+
+``` r
+R> lapply(list_hroc, function(x) plot_funregions(x, FPR = .15, FPR2 = .5))
+```
+
+```{r transformations, fig.cap="Different functional transformations and resulting classification subsets for gene 18384097. Rules for FPR 0.15 (blue) and 0.50 (red) are remarked. Top, from left to right: cubic polynomial function, restricted cubic splines (with 8 knots), and overfitted transformation. Bottom: likelihood ratio estimation with bandwidths 3 (left) and 1 (middle), and transformation resulting in the gROC curve under restriction *(C)*.", out.width="100%", echo=FALSE}
+knitr::include_graphics("figures/plotfunregions_hroc_lrm_gene18384097.png")
+```
+
+
+Finally, using the `plot_regions()` function,
+Figure [8](#fig:regionshroc){reference-type="ref"
+reference="fig:regionshroc"} shows the resulting classification subsets
+over the original space for the best two of the six methods above. The first
+method (fitting a logistic regression model with restricted cubic
+splines with 8 knots) reports an AUC of 0.804 (compared to 0.684 by the
+standard ROC curve), but the shape of some classification rules is
+complex, such as $s_t=(-\infty,a_t] \cup (b_t,c_t] \cup (d_t,\infty)$.
+This area increases to 0.836 by considering subsets of the form
+$s_t=(-\infty,x_t^L] \cup (x_t^U,\infty)$, even imposing the restriction
+*(C)* to get a functional transformation $h(\cdot)$.
+
+```{r regionshroc, fig.cap="Classification regions and the resulting ROC curve (90º rotated) for the gene 18384097. Top, ROC curve for restricted cubic splines transformation with 8 knots; bottom, gROC curve under restriction *(C)* for the original marker.", out.width="100%", echo=FALSE, fig.show="hold"}
+knitr::include_graphics(c("figures/plotregions_gene18384097_1.png", "figures/plotregions_gene18384097_2.png"))
+```
+
+
+# Summary and conclusion {#section:conclusion}
+
+Conducting binary classification using continuous markers requires
+establishment of decision rules. In the standard case, each specificity
+$t \in [0,1]$ entails a classification subset of the form
+$s_t = (c_t,\infty)$ univocally defined. However, in more complex
+situations -- such as where there is a non-monotone relationship between the
+marker and the response or in multivariate scenarios -- these become not
+clear. Visualization of the decision rules becomes crucial in these
+cases. To address this, the
+[**movieROC**](https://CRAN.R-project.org/package=movieROC) package
+incorporates novel visualization tools complementing the ROC curve
+representation.
+
+This R package offers a user-friendly and easily comprehensible software
+solution tailored for practical researchers. It implements statistical
+techniques to estimate and compare, and finally to graphically represent
+different classification procedures. While several R packages address
+ROC curve estimation, the proposed one emphasizes the classification
+process, tracking the decision rules underlying the studied binary
+classification problem. This tool incorporates different considerations
+and transformations which may be useful to capture the potential of the
+marker to classify in non-standard scenarios. Nevertheless, this library
+is also useful in standard cases, as well as when the marker itself
+comes from a classification or regression method (such as support vector
+machines), because it provides nice visuals and additional information
+not usually reported with the ROC curve.
+
+The main function of the package, `movieROC()`, allows users to monitor how
+the resulting classification subsets change along different
+specificities, thereby building the corresponding ROC curve. Notably, it
+introduces time as a third dimension to keep those specificities,
+generating informative videos. For interested readers or potential users
+of [**movieROC**](https://CRAN.R-project.org/package=movieROC), the
+[manual](https://cran.r-project.org/web/packages/movieROC/movieROC.pdf)
+available in CRAN provides complete information about the implemented
+functions and their parameters. In addition, a
+[vignette](https://cran.r-project.org/web/packages/movieROC/vignettes/movieROC_vignette.pdf)
+is accessible, including mathematical formalism and details about the
+algorithms implemented.
+
+# Computational considerations {#sec:complim .unnumbered}
+
+### Dependencies
+
+Some functions of our package depend on other libraries available on
+CRAN:
+
+- `gROC(X, D, side = "both", restric = TRUE, optim = TRUE, ...)` uses
+ the `allShortestPaths()` function in the
+ [**e1071**](https://CRAN.R-project.org/package=e1071) package
+ [@e1071CRAN2023].
+
+- `hROC(X, D, type = "lrm", ...)` and
+ `multiROC(X, D, method = "lrm", ...)` use the `lrm()` function in
+ the [**rms**](https://CRAN.R-project.org/package=rms) package
+ [@rmsCRAN2023].
+
+- `multiROC(X, D, method = "kernelOptimal", ...)` uses the `kde()`
+ function in the [**ks**](https://CRAN.R-project.org/package=ks)
+ package [@ksCRAN2023].
+
+- `multiROC(X, D, method = "dynamicMeisner", ...)` uses the `maxTPR()`
+ function in the **maxTPR** package [@Meisner2021]. This package was
+ removed from the CRAN repository, so we integrated the code of the
+ `maxTPR()` function into our package. This function uses
+ `Rsolnp::solnp()` and `robustbase::BYlogreg()`.
+
+- `multiROC(X, D, method = "fixedLinear", methodLinear, ...)` uses the
+ R functions included in @Kang2016 (Appendix). We integrated this
+ code into our package.
+
+- `movieROC(obj, save = TRUE, ...)` uses the `saveGIF()` function in
+ the [**animation**](https://CRAN.R-project.org/package=animation)
+ package [@animationCRAN2021].
+
+### Limitations
+
+Users should be aware of certain limitations while working with this
+package:
+
+* Some methods are potentially time-consuming, especially with medium
+ to large sample sizes:
+
+ + The estimation of the gROC curve under restriction *(C)* can be
+ computationally intensive, especially when considering different FPR
+ to locally optimize the search using
+ `gROC(X, D, side = "both", restric = TRUE, optim = TRUE, t0max = TRUE)`.
+ Note that this method involves a quite exhaustive search of the
+ self-contained classification subsets leading to the optimal gROC
+ curve estimate. However, even selecting different false-positive
+ rates $t_0$ to start from, it may not result in the optimal
+ achievable estimate under restriction *(C)*. Input parameters
+ `restric`, `optim`, `t0` and `t0max` for `gROC()` function included
+ in Table 1 of the
+ [vignette](https://cran.r-project.org/web/packages/movieROC/vignettes/movieROC_vignette.pdf)
+ serve to control this search.
+
+ + Similarly, it also occurs for multivariate markers when considering
+ linear frontiers with dynamic parameters (by usingReferences
+ +Reuse
+Text and figures are licensed under Creative Commons Attribution CC BY 4.0. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from ...".
+Citation
+For attribution, please cite this work as
+Belzile, "longevity: An R Package for Modelling Excess Lifetimes", The R Journal, 2026+
BibTeX citation
+@article{RJ-2025-034,
+ author = {Belzile, Léo R.},
+ title = {longevity: An R Package for Modelling Excess Lifetimes},
+ journal = {The R Journal},
+ year = {2026},
+ note = {https://doi.org/10.32614/RJ-2025-034},
+ doi = {10.32614/RJ-2025-034},
+ volume = {17},
+ issue = {4},
+ issn = {2073-4859},
+ pages = {37-58}
+}
++ `multiROC(X, D, method = "dynamicMeisner" | "dynamicEmpirical")`). + +* Most implemented R functions consider empirical estimation for the + resulting ROC curve, even if the procedure to estimate the decision + rules is semi-parametric. An exception is the `gROC_param()` + function, which accommodates the binormal scenario. + +* When visualizing classification regions for multivariate markers + with high dimension (`plot_buildROC()` and `movieROC()` functions + for a '`multiroc`' object), our package provides some alternatives, + but additional improvements could provide further aid in + interpretation. + +# Acknowledgements {#acknowledgements .unnumbered} + +The authors acknowledge support by the Grants PID2019-104486GB-I00 and +PID2020-118101GB-I00 from Ministerio de Ciencia e Innovación (Spanish +Government), and by a financial Grant for Excellence Mobility for +lecturers and researchers subsidized by the University of Oviedo in +collaboration with Banco Santander. +::: diff --git a/_articles/RJ-2025-035/RJ-2025-035.html b/_articles/RJ-2025-035/RJ-2025-035.html new file mode 100644 index 0000000000..a75dfc757f --- /dev/null +++ b/_articles/RJ-2025-035/RJ-2025-035.html @@ -0,0 +1,3078 @@ + + + + + + + + + + + + + + + + + + + + + +
+
+
+
+
+movieROC: Visualizing the Decision Rules Underlying Binary Classification
+ + + +The receiver operating characteristic (ROC) curve is a graphical tool +commonly used to depict the binary classification accuracy of a +continuous marker in terms of its sensitivity and specificity. The +standard ROC curve assumes a monotone relationship between the marker +and the response, inducing classification subsets of the form +\((c,\infty)\) with \(c \in \mathbb{R}\). However, in non-standard cases, +the involved classification regions are not so clear, highlighting the +importance of tracking the decision rules. This paper introduces the R +package movieROC, +which provides visualization tools for understanding the ability of +markers to identify a characteristic of interest, complementing the +ROC curve representation. This tool accommodates multivariate +scenarios and generalizations involving different decision rules. The +main contribution of this package is the visualization of the +underlying classification regions, with the associated gain in +interpretability. Adding the time (videos) as a third dimension, this +package facilitates the visualization of binary classification in +multivariate problems. It constitutes a good tool to generate +graphical material for presentations.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+1 Introduction
+The use of data to detect a characteristic of interest is a cornerstone +of many disciplines such as medicine (to diagnose a pathology or to +predict a patient outcome), finance (to detect fraud) or machine +learning (to evaluate a classification algorithm), among others. +Continuous markers are surrogate measures for the characteristic under +study, or predictors of a potential subsequent event. They are measured +in subjects, some of whom have the characteristic (positive), and some +without it (negative). In addition to reliability and feasibility, a +good marker must have two relevant properties: interpretability and +accuracy (Mayeux 2004). High binary classification accuracy can be +achieved if there exists a strong relationship between the marker and +the response. The latter is assessed by a gold standard for the presence +or absence of the characteristic of interest. Interpretability refers to +the decision rules or subsets considered in the classification process. +This piece of research seeks to elucidate both desirable properties for +a marker by the implementation of a graphical tool in R language. We +propose a novel approach involving the generation of videos as a +solution to effectively capture the classification procedure for +univariate and multivariate markers. Graphical analysis plays a pivotal +role in data exploration, interpretation, and communication. Its +burgeoning potential is underscored by the fast pace of technological +advances, which empower the creation of insightful graphical +representations.
+A usual practice when the binary classification accuracy of a marker is +of interest involves the representation of the Receiver Operating +Characteristic (ROC) curve, summarized by the Area Under the Curve (AUC) +(Hanley and McNeil 1982). The resulting plot reflects the trade-off between the +sensitivity and the complement of the specificity. Sensitivity and +specificity are probabilities of correctly classifying subjects, either +positive or negative, respectively. Mathematically, let \(\xi\) and \(\chi\) +be the random variables modeling the marker values in the positive and +the negative population, respectively, with \(F_\xi(\cdot)\) and +\(F_\chi(\cdot)\) their associated cumulative distribution functions. +Assuming that the expected value of the marker is larger in the positive +than in the negative population, the standard ROC curve is based on +classification subsets of the form \(s = (c, \infty)\), where \(c\) is the +so-called cut-off value or threshold in the support of the marker \(X\), +\(\mathcal{S}(X)\). One subject is classified as a positive if its marker +value is within this region, and as a negative otherwise. This type of +subsets has two important advantages: first, their interpretability is +clear; second, for each specificity \(1-t \in [0,1]\), the corresponding +\(s_t = (c_t, \infty)\) is univocally defined by \(c_t = F_\chi^{-1}(1-t)\) +for absolutely continuous markers.
+When differences in marker distribution between the negative and the +positive population are only in location but not in shape, then +\(F_\chi(\cdot) < F_\xi(\cdot)\), and the classification is direct by +using these decision rules. However, when this is not the case, the +standard ROC curve may cross the main diagonal, resulting in an +improper curve (Dorfman et al. 1997). This may be due to three different +scenarios:
+-
+
the behavior of the marker in the two studied populations is +different but it is not possible to determine the decision rules. +Notice that the binary classification problem goes further than +distinction between the two populations: the classification subsets +should be highly likely in one population and highly unlikely in the +other one (Martı́nez-Camblor 2018);
+there exists a relationship between the marker and the response with +a potential classification use, but this is not monotone;
+there is no relationship between the marker and the response at all +(main diagonal ROC curve).
+
In the second case, we have to define classification subsets different +from standard \(s_t=(c_t,\infty)\). Therefore, the use of the marker +becomes more complex. With the aim of accommodating scenarios where both +higher and lower values of the marker are associated with a higher risk +of having the characteristic, Martı́nez-Camblor et al. (2017) proposed the so-called +generalized ROC (gROC) curve. This curve tracks the highest sensitivity +for every specificity in the unit interval resulting from subsets of the +form \(s_t=(-\infty, x_t^L] \cup (x_t^U, \infty)\) with +\(x_t^L \leq x_t^U \in \mathcal{S}(X)\).
+Although final decisions are based on the underlying classification +subsets, they are typically not depicted. This omission is not a +shortcoming in standard cases, as for each specificity \(1-t \in [0,1]\), +there is only one rule of the form \(s_t = (c_t, \infty)\) with such +specificity. Particularly, \(s_t\) is univocally defined by +\(c_t = 1 - F_\chi^{-1}(1-t)\); and the same applies if we fix a +sensitivity. Nevertheless, if the gROC curve is taken, there are +infinite subsets of the form \(s_t=(-\infty, x_t^L] \cup (x_t^U, \infty)\) +resulting in \(\mathbb{P}(\chi \in s_t) = t\). This loss of univocity underlines +the importance of reporting (numerically and/or graphically) the +decision rules actually proposed for classifying. This gap is covered in +the presented package.
+An alternative approach to assess the classification performance of a +marker involves considering a transformation of it. This transformation +\(h(\cdot)\) aims to capture differences in distribution between the two +populations in the ROC sense. Once \(h(\cdot)\) is identified, the +standard ROC curve for \(h(X)\) is represented, resulting in the efficient +ROC (eROC) curve (Kauppi 2016). Arguing as before, for a fixed +specificity, the classification subsets \(s_t=(c_t, \infty)\) in the +transformed space are univocally defined, where a subject is classified +as positive if \(h(x) \in s_t\) and negative otherwise (with \(x\) +representing its marker value). However, they may have any shape in the +original space, depending on the monotonicity of the functional +transformation \(h(\cdot)\) (Martı́nez-Camblor et al. 2019). Emphasizing the importance of +tracking the decision rules underlying the eROC curve, this monitoring +process enables an assessment of whether the improved accuracy of the +marker justifies the potential loss in interpretability.
+The ROC curve is defined for classification accuracy evaluation of +univariate markers. To deal with multivariate markers, the usual +practice is to consider a transformation \(\boldsymbol{h}(\cdot)\) to +reduce it to a univariate one, and then to construct the standard ROC +curve. Same considerations as before apply when a functional +transformation is taken. In the proposed R library, we consider methods +from the literature to define and estimate \(\boldsymbol{h}(\cdot)\) in +the multivariate case (Kang et al. 2016; Meisner et al. 2021).
+Focusing on the classification subsets underlying the decision rules, +the movieROC package +incorporates methods to visualize the construction process of ROC curves +by presenting the classification accuracy of these subsets. For +univariate markers, the library includes both the classical (standard +ROC curve) and the generalized (gROC curve) approach. Besides, it enables +the display of decision rules for various transformations of the marker, +seeking to maximize performance and allowing for flexibility in the final +shape of the subsets (eROC curve). For multidimensional markers, the +proposed tool visualizes the evolution of decision subsets when +different objective functions are employed for optimization, even +imposing restrictions on the underlying regions. In this case, +displaying the decision rules associated with every specificity in a +single static image is no longer feasible. Therefore, dynamic +representations (videos) are implemented, drawing on time as an extra +dimension to capture the variation in specificity.
+Much software available in R could be discussed here covering diverse +topics related to ROC curves: the +pROC package is a main +reference including tools to visualize, estimate and compare ROC curves +(Robin et al. 2011); ROCnReg +explicitly considers covariate information to estimate the +covariate-specific and the covariate-adjusted ROC curves (Rodrı́guez-Álvarez and Inácio 2021); +smoothROCtime +implements smooth estimation of time-dependent ROC curves based on the +bivariate kernel density estimator for \((X, \textit{time-to-event})\) +(Dı́az-Coto 2020); +OptimalCutpoints +includes point and interval estimation methods for optimal thresholds +(López-Ratón et al. 2014); and +nsROC performs +non-standard analysis such as gROC estimation (Pérez-Fernández et al. 2018); among +others.
+This paper introduces and elucidates the diverse functionalities of the +newly developed +movieROC package, +aimed at facilitating the visualization and comprehension of the +decision rules underlying the binary classification process, +encompassing various generalizations. Despite the availability of +numerous R packages implementing related analyses, we have identified +the main gaps covered in this library: tracking the decision rules +underlying the ROC curve, including multivariate markers and +non-standard scenarios (i.e. non-monotonic). The rest of the paper is +structured as follows. In +Section 2, we introduce the main R functions +and objects implemented, and briefly explain the dataset employed +throughout this manuscript to demonstrate the utility of the R library. +Section 3 is devoted to reconsidering the +definition of the standard ROC curve from the perspective of +classification subsets, including an extension to multivariate +scenarios. Sections 4 and +5 revisit the gROC curve and the eROC +curve, respectively, covering various methods to capture the potential +classification accuracy of the marker under study. Each of these +sections begins with a state-of-the-art overview, followed by the main +syntax of the corresponding R functions. In addition, examples of +implementation using the dataset presented in Section +2.3 are provided. Finally, the paper +concludes with a concise summary and computational details regarding the +implemented tool.
+2 Main functions of the movieROC package and illustrative dataset
+Sections 2.1 and +2.2 provide a detailed description of the main +objectives of the implemented R functions. To reflect the practical +usage of the developed R package, we employ a real dataset throughout +this manuscript, which is introduced in +Section 2.3.
+2.1 Functionality of the movieROC package
+A graphical tool was developed to showcase static and dynamic graphics
+displaying the classification subsets derived from maximizing diagnostic
+accuracy under certain assumptions, ensuring the preservation of the
+interpretability. The R package facilitates the construction of the ROC
+curve across various specificities, providing visualizations of the
+resulting classification regions. The proposed tool comprises multiple R
+functions that generate objects with distinct class attributes (see
+function names where red arrows depart from and red nodes in
+Figure 1, respectively). Once the object of
+interest is created, different methods may be used, in order to plot the
+underlying regions (plot_regions(), plot_funregions()), to track the
+resulting ROC curve (plot_buildROC(), plot()), to predict decision
+rules for a particular specificity, and to print relevant information,
+among others. The main function of the package, movieROC(), produces
+videos to exhibit the classification procedure.
+
+
+
+
+
++Figure 1: R functions of the movieROC package. The blue nodes include the names of the R functions and the red nodes indicate the different R objects that can be created and worked with. The red arrows depart from those R functions engaged in creating R objects and the black arrows indicate which R functions can be applied to which R objects. The grey dashed arrows show internal dependencies. +
+It includes algorithms to visualize the regions that underlie the binary +classification problem, considering different approaches:
+-
+
make the classification subsets flexible in order to cover +non-standard scenarios, by considering two cut-off values (
gROC()+function); explained in +Section 4;
+transform the marker by a proper function \(h(\cdot)\) (
hROC()+function); introduced in +Section 5;
+when dealing with multivariate markers, consider a functional +transformation with some fixed or dynamic parameters resulting from +different methods available in the literature (
multiROC()+function); covered in +Section 3.1.
+
2.2 Class methods for movieROC objects
+By using the gROC(), the multiROC() or the hROC() function, the
+user obtains an R object of class ‘groc’, ‘multiroc’ or ‘hroc’,
+respectively. These will be called
+movieROC objects.
+Once the object of interest is created, the implemented package includes
+many functions (methods) to pass to it. Some of them are generic methods
+(print(), plot() and predict()), commonly used in R language over
+different objects according to their class attributes. The rest of the
+functions are specific for this library and therefore only applicable to
+movieROC objects.
+The following outline summarizes all these functions and
+provides their target and main syntax (with default input parameters).
Generic functions
+-
+
print(): Print some relevant information.
+plot(): Plot the ROC curve estimate.
+predict(): Print the classification subsets corresponding to a particular false-positive rate (FPR) introduced by the user. For a ‘groc’ object, the user may specify a cut-off valueC(for the standard ROC curve) or two cut-off valuesXLandXU(for the gROC curve).
+
Specific functions
+-
+
plot_regions()
+
Applicable to a ‘groc’ or a ‘hroc’ object. Plot two graphics in the same figure: left, classification subsets for each false-positive rate (grey color by default); right, \(90^\circ\) rotated ROC curve.
+
+
+
+ plot_regions(obj, plot.roc = TRUE, plot.auc = FALSE, FPR = 0.15, ...)If the input parameter FPR is specified, the corresponding classification region reporting such false-positive rate and the point in the ROC curve are highlighted in blue color.
-
+
plot_funregions()
+
Applicable to a ‘groc’ or a ‘hroc’ object.
+Plot the transforming function and the classification subsets reporting the false-positive rate(s) indicated in the input parameter(s) FPR and FPR2.
+
+
+
+plot_funregions(obj, FPR = 0.15, FPR2 = NULL, plot.subsets = TRUE, ...)-
+
plot_buildROC()
+
Applicable to a ‘groc’ or a ‘multiroc’ object.
-
+
- For a ‘
groc’ object: Plot four (if inputreduceis FALSE) or two (ifreduceis TRUE, only those on the top) graphics in the same figure: top-left, density function estimates for the marker in both populations with the areas corresponding to FPR and TPR colored (blue and red, respectively) for the optional input parameterFPR,CorXL, XU; top-right, the empirical ROC curve estimate; bottom-left, boxplots in both groups; bottom-right, classification subsets for every FPR (grey color).
+
+
+
+
+plot_buildROC(obj, FPR = 0.15, C, XL, XU, h = c(1,1),
+ histogram = FALSE, breaks = 15, reduce = TRUE,
+ build.process = FALSE, completeROC = FALSE, ...)If build.process is FALSE, the whole ROC curve is displayed; otherwise, if completeROC is TRUE, the portion of the ROC curve until the fixed FPR is highlighted in black and the rest is shown in gray, while if completeROC is FALSE, only the first portion of the curve is displayed.
-
+
- For a ‘
multiroc’ object: Plot two graphics in the same figure: right, the ROC curve highlighting the point and the threshold for the resulting univariate marker; left, scatterplot with the marker values in both positive (red color) and negative (blue color) subjects. About the left graphic: for \(p=2\), over the original/feature bivariate space; for \(p>2\), projected over two selected components of the marker (ifdisplay.method = "OV"with components selection indisplayOV,c(1,2)by default) or the first two principal components from PCA (ifdisplay.method = "PCA", default). The classification subset reporting theFPRselected by the user (FPR\(\neq\)NULL) is displayed in gold color.
+
Main syntax:
+for \(p=2\): +
+
+for \(p>2\):
+
+
+plot_buildROC(obj, FPR = 0.15,
+ build.process = FALSE, completeROC = TRUE, ...)
+
+
+If build.process is FALSE, the whole ROC curve is displayed; otherwise, if completeROC is TRUE, the portion of the ROC curve until the fixed FPR is highlighted in black and the rest is shown in gray, while if completeROC is FALSE, only the first portion of the curve is shown.
-
+
movieROC()
+
Applicable to a ‘groc’ or a ‘multiroc’ object. Save a video as a GIF illustrating the construction of the ROC curve.
-
+
- For a ‘
groc’ object:
+
+
+
+
+movieROC(obj, fpr = NULL,
+ h = c(1,1), histogram = FALSE, breaks = 15,
+ reduce = TRUE, completeROC = FALSE, videobar = TRUE,
+ file = "animation1.gif", ...)For each element in vector fpr (optional input parameter), the function executed is plot_buildROC(obj, FPR = fpr[i], build.process = TRUE, ...). The vector of false-positive rates illustrated in the video is NULL by default: if length of output parameter t for gROC() function is lower than 150, such vector is taken as fpr; otherwise, an equally-spaced vector of length 100 covering the range of the marker values is considered.
-
+
- For a ‘
multiroc’ object:
+
Main syntax:
+for \(p=2\): +
+
+for \(p>2\):
+
+
+movieROC(obj, fpr = NULL,
+ file = "animation1.gif", save = TRUE,
+ border = TRUE, completeROC = FALSE, ...)
+
+
+The video is saved by default as a GIF with the name indicated in argument file (extension .gif should be added). A border for the classification subsets is drawn by default.
For each element in vector fpr (optional input parameter), the function executed is
+
+for \(p>2\):
+
+
+plot_buildROC(obj, FPR = fpr[i], build.process = TRUE, completeROC, ...)
+
+
+
+plot_buildROC(obj, FPR = fpr[i], build.process = TRUE, completeROC,
+ display.method, displayOV, ...)Same considerations about the input fpr as those for movieROC() over a ‘groc’ object.
2.3 Illustrative dataset
+In order to illustrate the functionality of our R package, we consider
+the HCC data. This dataset is derived from gene expression arrays of
+tumor and adjacent non-tumor tissues of 62 Taiwanese cases of
+hepatocellular carcinoma (HCC). The goal of the original study
+(Shen et al. 2012) was to identify, with a genome-wide approach, additional
+genes hypermethylated in HCC that could be used for more accurate
+analysis of plasma DNA for early diagnosis, by using Illumina
+methylation arrays (Illumina, Inc., San Diego, CA) that screen 27,578
+autosomal CpG sites. The complete dataset was deposited in NCBI’s Gene
+Expression Omnibus (GEO) and it is available through series accession
+number GSE37988
+(www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE37988).
+It is included in the presented package (HCC dataset), selecting 948
+genes with complete information.
The following code loads the R package and the HCC dataset (see the
+vignette
+for main structure).
R> library(movieROC)
+R> data(HCC)We selected the genes 20202438, 18384097, and 03515901. On the one hand, +we chose the gene 03515901 as an example of a monotone relationship +between the marker and the response, reporting a good ROC curve. On the +other hand, relative gene expression intensities of the genes 20202438 +and 18384097 tend to be more extreme in tissues with tumor than in those +without it. These are non-standard cases, so if we limit ourselves to +detect “appropriate” genes on the basis of the standard ROC curve, they +would not be chosen. However, extending the decision rules by means of +the gROC curve, those genes may be considered as potential biomarkers +(locations) to differ between the two groups. The R code estimating and +displaying the density probability function for gene expression +intensities of the selected genes in each group +(Figure 2) is included in the +vignette.
+
+
+
+
+
+
++Figure 2: Density histograms and kernel density estimations (lighter) for gene expression intensities of the genes 20202438, 18384097 and 03515901 in negative (non-tumor) and positive (tumor) tissues. +
+3 Regular ROC curve
+Assuming that there exists a monotone relationship between the marker +and the response, the regular, right-sided or standard ROC curve +associated with the marker \(X\) considers classification subsets of the +form \(s_t=(c_t,\infty)\). For each specificity +\(1-t=\mathbb{P}(\chi \notin s_t) \in [0,1]\), also called true-negative rate, +there exists only one subset \(s_t\) reporting such specificity and thus a +particular sensitivity, also called true-positive rate, +\(\mathbb{P}(\xi \in s_t)\). This results in a simple correspondence between each +point of the ROC curve +\(\mathcal{R}_r(t) = 1-F_\xi \big(F_\chi^{-1}(1-t)\big)\) and its +associated classification region \(s_t \in \mathcal{I}_r(t)\), where +\[\mathcal{I}_r(t) = \Big\{ s_t = (c_t, \infty) : c_t \in \mathcal{S}(X) , \mathbb{P}(\chi \in s_t) = t \Big\}\] +is the right-sided family of eligible classification subsets. The +definition of this family captures the shape of the decision rules and +the target specificity.
+If higher values of the marker are associated with a higher probability +of not having the characteristic (see gene 03515901 in +Figure 2), the ROC curve would be defined by the +left-sided family of eligible classification subsets (Martı́nez-Camblor et al. 2017), +\(\mathcal{I}_l(t)\), similarly to \(\mathcal{I}_r(t)\) but with the form +\(s_t = (\infty, c_t]\). It results in +\(\mathcal{R}_l(t) = F_\xi \big(F_\chi^{-1}(t) \big)\), \(t\in[0,1]\), and +the decision rules are also univocally defined in this case.
+The ROC curve and related problems were widely studied in the +literature; interested readers are referred to the monographs of +Zhou et al. (2002), Pepe (2003), and Nakas et al. (2023), as well as the review by +Inácio et al. (2021). By definition, the ROC curve is confined within the unit +square, with optimal performance achieved when it approaches the +left-upper corner (AUC closer to 1). Conversely, proximity to the main +diagonal (AUC closer to 0.5) means diminished discriminatory ability, +resembling a random classifier.
+In practice, let \((\xi_1, \xi_2, \dots, \xi_n)\) and
+\((\chi_1, \chi_2, \dots, \chi_m)\) be two independent and identically
+distributed (i.i.d.) samples from the positive and the negative
+population, respectively. Different estimation procedures are
+implemented in the
+movieROC package,
+such as the empirical estimator (Hsieh and Turnbull 1996) (by default in the gROC()
+function), accompanied by its summary indices: the AUC and the Youden
+index (Youden 1950). Alternatively, semiparametric approaches based on
+kernel density estimation for the involved distributions may be
+considered (Zou et al. 1997). The plot_densityROC() function provides plots
+for both right- and left-sided ROC curves estimated by this method. On
+the other hand, assuming that the marker follows a gaussian distribution
+in both populations, that is,
+\(\xi \sim \mathcal{N}(\mu_\xi, \sigma_\xi)\) and
+\(\chi \sim \mathcal{N}(\mu_\chi, \sigma_\chi)\), parametric approaches
+propose plug-in estimators by estimating the unknown parameters while
+using the known distributions (Hanley 1988). This parametric estimation
+is included in the gROC_param() function, which works similarly to
+gROC().
Main syntax:
+
+
+
+
+gROC(X, D, side = "right", ...)
+gROC_param(X, D, side = "right", ...)Table 1 in the
+vignette
+provides the main input and output parameters of these R functions,
+which estimate the regular ROC curve (right-sided or left-sided with
+side = "right" or "left", respectively) and associated decision
+rules. Its output is an R object of class ‘groc’, to which the
+functions listed in Section 2.2 can be applied. Most of them are
+visualization tools, but the user may also print() summary information
+and predict() classification regions for a particular specificity.
Figure 3 graphically represents the empirical +estimation of the standard (gray line) and generalized (black line) ROC +curves for each gene in Figure 2. To construct the standard ROC curve for the +first two genes (20202438 and 18384097), the right-sided ROC curve is +considered; and the left-sided curve for the third one (03515901). As +expected following the discussion about +Figure 2, the standard and gROC curves are similar for +the third gene because there exists a monotone relationship between the +marker and the response. However, these curves differ for the first two +genes due to the lack of monotonicity in those scenarios. The empirical +gROC curve estimator is explained in detail in +Section 4.
+Next chunk of code generates the figure, providing an example of the use
+of gROC() function, plot() and how to get access to the AUC.
R> for(gene in c("20202438", "18384097", "03515901")){
++ roc <- gROC(X = HCC[,paste0("cg",gene)], D = HCC$tumor,
++ side = ifelse(gene == "03515901", "left", "right"))
++ plot(roc, col = "gray50", main = paste("Gene", gene), lwd = 3)
++ groc <- gROC(X = HCC[,paste0("cg",gene)], D = HCC$tumor, side = "both")
++ plot(groc, new = FALSE, lwd = 3)
++ legend("bottomright", paste(c("AUC =", "gAUC ="), format(c(roc$auc, groc$auc),
++ digits = 3)), col = c("gray50", "black"), lwd = 3, bty = "n", inset = .01)}
+
+
+
+
+
++Figure 3: Standard ROC curve (in gray) and gROC curve (in black) empirical estimation for the capacity of genes 20202438, 18384097 and 03515901 to differ between tumor and non-tumor group. +
+The following code snippet estimates the standard ROC curve for gene
+20202438, prints its basic information, and predicts the classification
+region and sensitivity resulting in a specificity of 0.9. It provides an
+illustrative example of utilizing the print() and predict()
+functions.
R> roc_selg1 <- gROC(X = HCC$cg20202438, D = HCC$tumor, side = "right")
+R> roc_selg1Data was encoded with nontumor (controls) and tumor (cases).
+It is assumed that larger values of the marker indicate larger confidence that a
+ given subject is a case.
+There are 62 controls and 62 cases.
+The specificity and sensitivity reported by the Youden index are 0.855 and 0.403,
+ respectively, corresponding to the following classification subset: (0.799, Inf).
+The area under the right-sided ROC curve (AUC) is 0.547.R> predict(roc_selg1, FPR = .1)$ClassSubsets $Specificity $Sensitivity
+[1] 0.8063487 Inf [1] 0.9032258 [1] 0.3064516The following line of code displays the whole construction of the +empirical standard ROC curve for gene 20202438. The video is saved by +default as a GIF with the name provided.
+R> movieROC(roc_selg1, reduce = FALSE, file = "StandardROC_gene20202438.gif")
3.1 Multivariate ROC curve
+In practice, many cases may benefit from combining information from +different markers to enhance classification accuracy. Rather than +assessing univariate markers separately, taking the multivariate marker +resulting from merging them can yield relevant gain. However, note that +the ROC curve and related indices are defined only for univariate +markers, as they require the existence of a total order. To address this +limitation, a common approach involves transforming the \(p\)-dimensional +multivariate marker \(\boldsymbol{X}\) into a univariate one through a +functional transformation +\(\boldsymbol{h}: \mathbb{R}^p \longrightarrow \mathbb{R}\). This +transformation \(\boldsymbol{h}(\cdot)\) seeks to optimize an objective +function related to the classification accuracy, usually the AUC +(Su and Liu 1993; McIntosh and Pepe 2002; Martı́nez-Camblor et al. 2019).
+We enumerate the methods included in the proposed R tool by the
+multiROC() function (with the input parameter method), listed
+according to the objective function to optimize. Recall that the output
+of multiROC() is an object of class ‘multiroc’, containing
+information about the estimation of the ROC curve and subsets for
+multivariate scenarios. Table 2 in the
+vignette
+includes the usage of this function.
Main syntax:
+
+
+
+
+multiROC(X, D, method = "lrm",
+ formula = 'D ~ X.1 + I(X.1^2) + X.2 + I(X.2^2) + I(X.1*X.2)', ...)-
+
- AUC: Different procedures to estimate the \(\boldsymbol{h}(\cdot)\) +maximizing the AUC in the multidimensional case have been studied in +the literature. Among all families of functions, linear combinations +(\(\mathcal{L}_{\boldsymbol{\beta}}(\boldsymbol{X}) = \beta_1 X_1 + \dots + \beta_p X_p\)) +are widely used due to their simplicity; an extensive review of the +existing methods was conducted by Kang et al. (2016). +
-
+
- Computation: In the
multiROC()function, fixing input parameters +method = "fixedLinear"andmethodLinearto one from"SuLiu"+(Su and Liu 1993),"PepeThompson"(Pepe and Thompson 2000), or"minmax"(Liu et al. 2011). +The R function also admits quadratic combinations when \(p=2\), i.e. +\(\mathcal{Q}_{\boldsymbol{\beta}}(\boldsymbol{X}) = \beta_1 X_1 + \beta_2 X_2 + \beta_3 X_1 X_2 + \beta_4 X_1^2 + \beta_5 X_2^2\), +by fixingmethod = "fixedQuadratic"for a particular +coefQuadratic\(= \boldsymbol{\beta} = (\beta_1, \dots, \beta_5)\).
+
-
+
The risk score function +logit\(\left\{ \mathbb{P}(D = 1 \, | \, \boldsymbol{X}) \right\}\): Our +package allows the user to fit a logistic regression model +(
method = "lrm") considering any family of functions (linear, +quadratic, whether considering interactions or not...) by means of +the input parameterformula.lrm. A stepwise regression model is +fitted ifstepModel = TRUE. Details are explained in +Section 5.
+The sensitivity for a particular specificity:
+
-
+
- Considering the theoretical discussion about the search of the +optimal transformation \(\boldsymbol{h}(\cdot)\) pointed out in +Section 5, Martı́nez-Camblor et al. (2021a) proposed to +estimate it by multivariate kernel density estimation for +positive and negative groups separately. +
-
+
- Computation: The
multiROC()function integrates the estimation +procedures for the bandwidth matrix developed by Duong (2007), by +fixingmethod = "kernelOptimal"and choosing a proper method +to estimate the bandwidth ("kernelOptimal.H").
+
-
+
- Mainly linear combinations have been explored to date in the +scientific literature (Meisner et al. 2021; Pérez-Fernández et al. 2021). For a +fixed specificity \(t\in[0,1]\), we seek the linear combination +\(\mathcal{L}_{\boldsymbol{\beta}(t)}(\boldsymbol{X}) = \beta_1(t) X_1 + \dots + \beta_p(t) X_p\) +maximizing the true-positive rate by considering standard +subsets for the transformed marker. The coefficients +\(\boldsymbol{\beta}(t)\) are called ‘dynamic parameters’ because +they may be different for each \(t \in [0,1]\). +
-
+
- Computation: Since our objective is to display the ROC curve,
+\(\mathcal{L}_{\boldsymbol{\beta}(t)}(\boldsymbol{X})\) is
+estimated for every \(t\) in a grid of the unit interval,
+resulting in one \(\boldsymbol{\hat{\beta}}(t)\) for each \(t\).
+This approach is time-consuming, especially when it is based on
+the plug-in empirical estimators involved
+(
method = "dynamicEmpirical", only implemented for \(p=2\)), and +may result in overfitting. Instead, Meisner et al. (2021) method is +recommended (method = "dynamicMeisner").
+
Once the classification subsets for a multivariate marker are
+constructed by the multiROC() function, several R methods may be used
+for the output object (see Section 2.2). They include print relevant
+information or plot the resulting ROC curve. The main contribution of
+the package is to plot the construction of the ROC curve together with
+the classification subsets in a static figure for a particular FPR
+(plot_buildROC() function), or in a video for tracking the whole
+process (movieROC() function).
Figure 4
+illustrates the videos resulting from movieROC() function.
+Particularly, classification accuracy of the
+bivariate marker (cg20202438, cg18384097) was studied by using four
+different approaches indicated on the captions, considering linear
+combinations (top) and nonlinear transformations (bottom). This figure
+was implemented by the code below, integrating multiROC() and
+movieROC() functions. Four videos are saved as GIF files with names
+"PepeTh.gif" (a), "Meisner.gif" (b), "LRM.gif" (c), and
+"KernelDens.gif" (d).
R> X <- HCC[ ,c("cg20202438", "cg18384097")]; D <- HCC$tumor
+R> biroc_12_PT <- multiROC(X, D, method = "fixedLinear", methodLinear = "PepeThompson")
+R> biroc_12_Meis <- multiROC(X, D, method = "dynamicMeisner", verbose = TRUE)
+R> biroc_12_lrm <- multiROC(X, D)
+R> biroc_12_kernel <- multiROC(X, D, method = "kernelOptimal")
+R> list_biroc <- list(PepeTh = biroc_12_PT, Meisner = biroc_12_Meis,
++ LRM = biroc_12_lrm, KernelDens = biroc_12_kernel)
+R> lapply(names(list_biroc), function(x) movieROC(list_biroc[[x]],
++ display.method = "OV", xlab = "Gene 20202438", ylab = "Gene 18384097",
++ cex = 1.2, alpha.points = 1, lwd.curve = 4, file = paste0(x, ".gif")))
+
+
+
+
+
+ |
+ |
+
|---|---|
| (a) Linear combinations with fixed parameters by Pepe and Thompson (2000). | +(b) Linear combinations with dynamic parameters by Meisner et al. (2021). | +
 |
+ |
+
|---|---|
(c) Logistic regression model with quadratic formula by default (see formula.lrm in Table 2 of the vignette). |
+(d) Optimal transformation by multivariate kernel density estimation with "Hbcv" method by default (Martı́nez-Camblor et al. 2021a). |
+
+
+
+
+
+
+
+Figure 4: Videos (from movieROC() function) of the classification procedure and ROC curve for the bivariate marker cg20202438, cg18384097). Four different methods for classification are displayed.
+
When the marker has a dimension higher than two it is difficult to
+visualize the data and the classification regions. Therefore, the
+movieROC() function offers two options for showing the results, both
+on a bidimensional space. On the one hand, to choose two of the
+components of the multivariate marker and project the classification
+subsets on the plane defined by them
+(Figure 5, middle). On the other, to project the
+classification regions on the plane defined by the two first principal
+components (Figure 5, left). The R function prcomp() from stats
+is used to perform Principal Components Analysis (PCA) (Hotelling 1933).
Figure 5 shows the difficulty in displaying the
+decision rules when \(p>2\) (the 3 genes used along this manuscript), even
+with the two options implemented in our package. It was generated using
+multiROC() and plot_buildROC():
R> multiroc_PT <- multiROC(X = HCC[ ,c("cg20202438", "cg18384097", "cg03515901")],
++ D = HCC$tumor, method = "fixedLinear", methodLinear = "PepeThompson")
+R> multiroc_PTData was encoded with nontumor (controls) and tumor (cases).
+There are 62 controls and 62 cases.
+A total of 3 variables have been considered.
+A linear combination with fixed parameters estimated by PepeThompson approach has
+ been considered.
+The specificity and sensitivity reported by the Youden index are 0.855 and 0.742,
+ respectively, corresponding to the cut-off point -0.0755 for the transformation
+ h(X) = 0.81*cg20202438 - 0.1*cg18384097 - 1*cg03515901.
+The area under the ROC curve (AUC) is 0.811.R> plot_buildROC(multiroc_PT, cex = 1.2, lwd.curve = 4)
+R> plot_buildROC(multiroc_PT, display.method = "OV", displayOV = c(1,3), cex = 1.2,
++ xlab = "Gene 20202438", ylab = "Gene 03515901", lwd.curve = 4)
+
+
+
+
+
++Figure 5: Multivariate ROC curve estimation for the simultaneous diagnostic accuracy of genes 20202438, 18384097 and 03515901. Pepe and Thompson (2000) approach was used to estimate the linear coefficients and classification rules (yellow and gray border for positive and negative class, respectively) for a FPR 0.15 are displayed. Left, projected over the 2 principal components from PCA; middle, over the 1st and the 3rd selected genes. +
+4 Generalized ROC curve
+There are scenarios whose standard ROC curves are not concave (first two +genes in Figure 3, gray solid line), reflecting that the +standard initial assumption of existence of a monotone relationship +between the marker and the response is misleading. In +Figure 2, we may see that difference in gene 20202438 +distribution between those tissues which have the characteristic and those +which do not is mainly in dispersion. To accommodate this common type of +scenarios, Martı́nez-Camblor et al. (2017) extended the ROC curve definition to the case +where both extremes for marker values are associated with a higher risk +of having the characteristic of interest, by considering the both-sided +family of eligible classification subsets: +\[\mathcal{I}_g(t) = \Big\{ s_t = (-\infty,x_t^L] \cup (x_t^U, \infty) : x_t^L \leq x_t^U \in \mathcal{S}(X) , \mathbb{P}(\chi \in s_t) = t \Big\}.\]
+It becomes crucial to consider the supremum in the definition of the +generalized ROC curve because the decision rule for each \(t \in [0,1]\) +is not univocally defined: there exist infinite pairs \(x_t^L \leq x_t^U\) +reporting a specificity \(1-t\) (i.e. \(\mathcal{I}_g(t)\) is uncountably +infinite). Computationally, this optimization process incurs a +time-consuming estimation, depending on the number of different marker +values in the sample.
+After the introduction of this extension, several studies followed up
+regarding estimation of the gROC curve (Martı́nez-Camblor et al. 2017; Martı́nez-Camblor and Pardo-Fernández 2019a)
+and related measures such as its area (gAUC) (Martı́nez-Camblor et al. 2021b) and the
+Youden index (Martı́nez-Camblor and Pardo-Fernández 2019b; Bantis et al. 2021). By considering this
+generalization, another property of the classification subsets may be
+lost: the regions may not be self-contained over the increase in
+false-positive rate. It may happen that a subject is classified as a
+positive for a particular FPR \(t_1\), but as a negative for a higher FPR
+\(t_2\). Therefore, it is natural to establish a restriction (C) on the
+classification subsets, ensuring that any subject classified as a
+positive for a fixed specificity (or sensitivity) will also be
+classified as a positive for any classification subset with lower
+specificity (higher sensitivity). Pérez-Fernández et al. (2021) proposed an
+algorithm to estimate the gROC curve under restriction (C), included
+in the gROC() function of the presented R package. See final Section
+for computational details about this algorithm implementation.
Main syntax:
+
+
+
+
+gROC(X, D, side = "both", ...)
+gROC_param(X, D, side = "both", ...)Table 1 in the
+vignette
+collects the input and output parameters of the gROC() function, which
+estimates the gROC curve, both in the mentioned direction
+(side = "both") and in the opposite, i.e. when classification subsets
+of the form \(s_t=(x_t^L, x_t^U]\) are considered (side = "both2"). In
+addition, all the particular methods for a ‘groc’ object collected in
+Section 2.2 may be used in this general
+scenario.
Following, the gene 20202438 expression intensity diagnostic accuracy is
+evaluated by the gROC curve without restrictions (groc_selg1 object)
+and under the restriction (C) (groc_selg1_C object). The
+classification subsets and sensitivity for a specificity of \(0.9\) are
+displayed with the predict() function.
R> groc_selg1 <- gROC(X = HCC$cg20202438, D = HCC$tumor, side = "both")
+R> predict(groc_selg1, FPR = .1)$ClassSubsets $Specificity $Sensitivity
+ [,1] [,2] [1] 0.9032258 [1] 0.4032258
+[1,] -Inf 0.7180623
+[2,] 0.8296072 InfR> groc_selg1_C <- gROC(X = HCC$cg20202438, D = HCC$tumor, side = "both",
++ restric = TRUE, optim = TRUE)All the classification regions underlying the standard and the
+generalized ROC curves without and with restrictions are represented in
+Figure 6. The following code was used to generate
+the figure, illustrating the usage and output of the plot_regions()
+function. Besides displaying all the classification regions underlying
+every specificity (in gray), the one chosen by the user (FPR = 0.15 by
+default) is highlighted in blue. Note that the ROC curves are rotated
+\(90^\circ\) to the right, in order to use the vertical axis for FPR in
+both plots.
R> plot_regions(roc_selg1, cex.legend = 1.5, plot.auc = TRUE,
++ main = "Standard right-sided assumption [Classification subsets]")
+R> plot_regions(groc_selg1, plot.auc = TRUE, legend = F,
++ main.plotroc = "gROC curve",
++ main = "General approach [Classification subsets]")
+R> plot_regions(groc_selg1_C, plot.auc = TRUE, legend = F,
++ main.plotroc = "gROC curve",
++ main = "General approach with restriction (C) [Classific. subsets]",
++ xlab = "Gene 20202438 expression intensity")
+
+
+
+
+
++Figure 6: Classification regions and the ROC curve (90º rotated) for evaluation of gene 20202438 expression intensity assuming i) standard scenario (top), ii) generalized scenario without restrictions (middle), iii) generalized scenario under restriction (C) over the subsets (bottom). +
+It is clear the gain achieved for considering the generalized scenario +for this marker, which fits better its distribution in each group. +Standard estimated AUC is 0.547, while the gAUC increases to 0.765. The +gAUC is not especially affected by imposing the restriction (C), +resulting in 0.762.
+5 Efficient ROC curve: pursuing an optimal transformation
+By keeping classification subsets of the form \(s_t = (c_t, \infty)\), an +alternative approach can be explored: transforming the univariate marker +through a suitable function \(h: \mathbb{R} \longrightarrow \mathbb{R}\) +to enhance its accuracy. Henceforth, the transformation \(h^*(\cdot)\) +reporting the dominant ROC curve compared to the one from any other +function (i.e. \(\mathcal{R}_{h^*}(\cdot) \geq \mathcal{R}_h(\cdot)\)) +will be referred to as optimal transformation (in the ROC sense), and +the resulting ROC curve is called eROC (Kauppi 2016). Following the +well-known Neyman–Pearson lemma, McIntosh and Pepe (2002) proved that \(h^*(\cdot)\) +is the likelihood ratio.
+We enumerate the methods included in the proposed R tool by the hROC()
+function (with the input parameter type), listed according to the
+procedure considered to estimate \(h^*(\cdot)\). The output of this
+function is an object of class ‘hroc’. See Table 3 in the
+vignette
+for function usage and output details.
Main syntax:
+
+
+
+
+hROC(X, D, type = "lrm", formula.lrm = 'D ~ pol(X,3)', ...)-
+
- Martı́nez-Camblor et al. (2019) exploited the result proved by McIntosh and Pepe (2002), +suggesting to estimate the logit of the risk function by logistic +regression, since it is a monotone increasing transformation of the +likelihood ratio. +
-
+
- Computation: By the proposed R tool, the user can define any
+transformation \(h(\cdot)\) for the right-hand side of the logistic
+regression model to be fitted,
+logit\(\big\{ \mathbb{P}(D = 1 \, | \, x) \big\} = h(x)\). Particularly, by
+fixing the input parameters:
type = "lrm"and defining the +function \(h(\cdot)\) asformula.lrm.
+
-
+
- Arguing as in Martı́nez-Camblor et al. (2021a) for univariate markers instead of +multivariate, the optimal transformation in the ROC sense is +equivalent to +\(h^*(\cdot)=f_\xi(\cdot)/\big(f_\xi(\cdot) + f_\chi(\cdot)\big)\), +where \(f(\cdot)\) denotes the density function. In order to estimate +\(h^*(\cdot)\), different estimation procedures for the density +functions separately may be used, such as the kernel density +estimator. +
-
+
- Computation: By the
hROC()function, the user may fix +type = "kernel"and choose a proper bandwidth for the kernel +estimation bykernel.hin order to compute this method.
+
-
+
- Martı́nez-Camblor et al. (2019) also included the estimation of the overfitting +function, \(h_{of}(\cdot)\), defined as the optimal one when no +restrictions on the shape of \(h^*(\cdot)\) are imposed. It takes the +value 1 for the positive marker values and 0 for the negative ones, +reporting an estimated AUC of 1, but totally depending on the +available sample (the resulting rules cannot be extended). +
-
+
- Computation: \(h_{of}(\cdot)\) may be estimated by fixing the input
+parameter
type = "overfitting".
+
The following code and figures study the capacity of improving the
+classification performance of the gene 18384097 expression intensity via
+the above functional transformations and its impact on the final
+decision rules. The first one considers an ordinary cubic polynomial
+formula (hroc_cubic_selg2), and a linear tail-restricted cubic spline
+(hroc_rcs_selg2) for the right-hand side of the logistic regression model.
+The second one uses two different bandwidths (\(h=1\) and \(h=3\)) for
+density function estimation. For a comparative purpose, the last one
+estimates the gROC curve under restriction (C).
R> X <- HCC$cg18384097; D <- HCC$tumorR> hroc_cubic_selg2 <- hROC(X, D); hroc_cubic_selg2Data was encoded with nontumor (controls) and tumor (cases).
+There are 62 controls and 62 cases.
+A logistic regression model of the form D ~ pol(X,3) has been performed.
+The estimated parameters of the model are the following:
+ Intercept X X^2 X^3
+ "1.551" "32.054" "-120.713" "100.449"
+The specificity and sensitivity reported by the Youden index are 0.935 and 0.532,
+ respectively, corresponding to the following classification subset:
+ (-Inf, 0.442) U (0.78, Inf).
+The area under the ROC curve (AUC) is 0.759.R> hroc_rcs_selg2 <- hROC(X, D, formula.lrm = "D ~ rcs(X,8)")
+R> hroc_lkr1_selg2 <- hROC(X, D, type = "kernel")
+R> hroc_lkr3_selg2 <- hROC(X, D, type = "kernel", kernel.h = 3)
+R> hroc_overfit_selg2 <- hROC(X, D, type = "overfitting")
+
+R> groc_selg2_C <- gROC(X, D, side = "both", restric = TRUE, optim = TRUE)The following code snippet compares the AUC achieved from each approach +considered above:
+R> list_hroc <- list(Cubic = hroc_cubic_selg2, Splines = hroc_rcs_selg2,
++ Overfit = hroc_overfit_selg2, LikRatioEst_h3 = hroc_lkr3_selg2,
++ LikRatioEst_h1 = hroc_lkr1_selg2, gAUC_restC = groc_selg2_C)R> AUCs <- sapply(list_hroc, function(x) x$auc)
+R> round(AUCs, 3)Cubic Splines Overfit LikRatioEst_h3 LikRatioEst_h1 gAUC_restC
+0.759 0.807 1.000 0.781 0.799 0.836The shape of the classification regions over the original space
+\(\mathcal{S}(X)\) depends on the monotonicity of \(h^*(\cdot)\), which may
+be graphically studied by the plot_funregions() function (see
+Figure 7). These regions can be visualized by
+the R function plot_regions() (see
+Figure 8). Both are explained in
+Section 2.2 and illustrated below. The next chunk of
+code produced Figure 7, representing the different functional
+transformations estimated previously:
R> lapply(list_hroc, function(x) plot_funregions(x, FPR = .15, FPR2 = .5))
+
+
+
+
+
++Figure 7: Different functional transformations and resulting classification subsets for gene 18384097. Rules for FPR 0.15 (blue) and 0.50 (red) are remarked. Top, from left to right: cubic polynomial function, restricted cubic splines (with 8 knots), and overfitted transformation. Bottom: likelihood ratio estimation with bandwidths 3 (left) and 1 (middle), and transformation resulting in the gROC curve under restriction (C). +
+Finally, using the plot_regions() function,
+Figure 8 shows the resulting classification subsets
+over the original space for the best two of the six methods above. The first
+method (fitting a logistic regression model with restricted cubic
+splines with 8 knots) reports an AUC of 0.804 (compared to 0.684 by the
+standard ROC curve), but the shape of some classification rules is
+complex, such as \(s_t=(-\infty,a_t] \cup (b_t,c_t] \cup (d_t,\infty)\).
+This area increases to 0.836 by considering subsets of the form
+\(s_t=(-\infty,x_t^L] \cup (x_t^U,\infty)\), even imposing the restriction
+(C) to get a functional transformation \(h(\cdot)\).
+
+
+
+
+
++Figure 8: Classification regions and the resulting ROC curve (90º rotated) for the gene 18384097. Top, ROC curve for restricted cubic splines transformation with 8 knots; bottom, gROC curve under restriction (C) for the original marker. +
+6 Summary and conclusion
+Conducting binary classification using continuous markers requires +establishment of decision rules. In the standard case, each specificity +\(t \in [0,1]\) entails a classification subset of the form +\(s_t = (c_t,\infty)\) univocally defined. However, in more complex +situations – such as where there is a non-monotone relationship between the +marker and the response or in multivariate scenarios – these become not +clear. Visualization of the decision rules becomes crucial in these +cases. To address this, the +movieROC package +incorporates novel visualization tools complementing the ROC curve +representation.
+This R package offers a user-friendly and easily comprehensible software +solution tailored for practical researchers. It implements statistical +techniques to estimate and compare, and finally to graphically represent +different classification procedures. While several R packages address +ROC curve estimation, the proposed one emphasizes the classification +process, tracking the decision rules underlying the studied binary +classification problem. This tool incorporates different considerations +and transformations which may be useful to capture the potential of the +marker to classify in non-standard scenarios. Nevertheless, this library +is also useful in standard cases, as well as when the marker itself +comes from a classification or regression method (such as support vector +machines), because it provides nice visuals and additional information +not usually reported with the ROC curve.
+The main function of the package, movieROC(), allows users to monitor how
+the resulting classification subsets change along different
+specificities, thereby building the corresponding ROC curve. Notably, it
+introduces time as a third dimension to keep those specificities,
+generating informative videos. For interested readers or potential users
+of movieROC, the
+manual
+available in CRAN provides complete information about the implemented
+functions and their parameters. In addition, a
+vignette
+is accessible, including mathematical formalism and details about the
+algorithms implemented.
Computational considerations
+Dependencies
+Some functions of our package depend on other libraries available on +CRAN:
+-
+
gROC(X, D, side = "both", restric = TRUE, optim = TRUE, ...)uses +theallShortestPaths()function in the +e1071 package +(Meyer et al. 2023).
+hROC(X, D, type = "lrm", ...)and +multiROC(X, D, method = "lrm", ...)use thelrm()function in +the rms package +(Harrell Jr 2023).
+multiROC(X, D, method = "kernelOptimal", ...)uses thekde()+function in the ks +package (Duong 2023).
+multiROC(X, D, method = "dynamicMeisner", ...)uses themaxTPR()+function in the maxTPR package (Meisner et al. 2021). This package was +removed from the CRAN repository, so we integrated the code of the +maxTPR()function into our package. This function uses +Rsolnp::solnp()androbustbase::BYlogreg().
+multiROC(X, D, method = "fixedLinear", methodLinear, ...)uses the +R functions included in Kang et al. (2016) (Appendix). We integrated this +code into our package.
+movieROC(obj, save = TRUE, ...)uses thesaveGIF()function in +the animation +package (Xie et al. 2021).
+
Limitations
+Users should be aware of certain limitations while working with this +package:
+-
+
Some methods are potentially time-consuming, especially with medium +to large sample sizes:
+-
+
The estimation of the gROC curve under restriction (C) can be +computationally intensive, especially when considering different FPR +to locally optimize the search using +
gROC(X, D, side = "both", restric = TRUE, optim = TRUE, t0max = TRUE). +Note that this method involves a quite exhaustive search of the +self-contained classification subsets leading to the optimal gROC +curve estimate. However, even selecting different false-positive +rates \(t_0\) to start from, it may not result in the optimal +achievable estimate under restriction (C). Input parameters +restric,optim,t0andt0maxforgROC()function included +in Table 1 of the +vignette +serve to control this search.
+Similarly, it also occurs for multivariate markers when considering +linear frontiers with dynamic parameters (by using
+multiROC(X, D, method = "dynamicMeisner" | "dynamicEmpirical")).
+
+Most implemented R functions consider empirical estimation for the +resulting ROC curve, even if the procedure to estimate the decision +rules is semi-parametric. An exception is the
gROC_param()+function, which accommodates the binormal scenario.
+When visualizing classification regions for multivariate markers +with high dimension (
plot_buildROC()andmovieROC()functions +for a ‘multiroc’ object), our package provides some alternatives, +but additional improvements could provide further aid in +interpretation.
+
Acknowledgements
+The authors acknowledge support by the Grants PID2019-104486GB-I00 and +PID2020-118101GB-I00 from Ministerio de Ciencia e Innovación (Spanish +Government), and by a financial Grant for Excellence Mobility for +lecturers and researchers subsidized by the University of Oviedo in +collaboration with Banco Santander.
+6.1 CRAN packages used
+movieROC, pROC, ROCnReg, OptimalCutpoints, nsROC, rms, ks, e1071, animation
+6.2 CRAN Task Views implied by cited packages
+Cluster, Distributions, DynamicVisualizations, Econometrics, Environmetrics, MachineLearning, Psychometrics, ReproducibleResearch, Survival, TeachingStatistics
+6.3 Note
+This article is converted from a Legacy LaTeX article using the +texor package. +The pdf version is the official version. To report a problem with the html, +refer to CONTRIBUTE on the R Journal homepage.
+
+
+
+
+
+L. E. Bantis, J. V. Tsimikas, G. R. Chambers, M. Capello, S. Hanash and Z. Feng. The length of the receiver operating characteristic curve and the two cutoff Youden index within a robust framework for discovery, evaluation, and cutoff estimation in biomarker studies involving improper receiver operating characteristic curves. Statistics in Medicine, 40(7): 1767–1789, 2021. URL https://doi.org/10.1002/sim.8869.
+
+
+S. Dı́az-Coto. smoothROCtime: an R package for time-dependent ROC curve estimation. Computational Statistics, 35: 1231–1251, 2020. URL https://doi.org/10.1007/s00180-020-00955-7.
+
+
+D. D. Dorfman, K. S. Berbaum, C. E. Metz, R. V. Lenth, J. A. Hanley and H. A. Dagga. Proper receiver operating characteristic analysis: The bigamma model. Academic Radiology, 4(2): 138–149, 1997. URL https://doi.org/10.1016/s1076-6332(97)80013-x.
+
+
+T. Duong. Ks: Kernel density estimation and kernel discriminant analysis for multivariate data in R. Journal of Statistical Software, 21(7): 1–16, 2007. URL https://doi.org/10.18637/jss.v021.i07.
+
+
+T. Duong. Ks: Kernel smoothing. 2023. URL https://CRAN.R-project.org/package=ks. R package version 1.14.1.
+
+
+J. A. Hanley. The robustness of the "binormal" assumptions used in fitting ROC curves. Medical Decision Making, 8(3): 197–203, 1988. URL https://doi.org/10.1177/0272989X8800800308. PMID: 3398748.
+
+
+J. A. Hanley and B. J. McNeil. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143(1): 29–36, 1982. URL https://doi.org/110.1148/radiology.143.1.7063747.
+
+
+F. E. Harrell Jr. Rms: Regression modeling strategies. 2023. URL https://CRAN.R-project.org/package=rms. R package version 6.7-1.
+
+
+H. Hotelling. Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24(6): 417–441, 1933. URL https://doi.org/10.1037/h0071325.
+
+
+F. Hsieh and B. W. Turnbull. Nonparametric and semiparametric estimation of the receiver operating characteristic curve. The Annals of Statistics, 24(1): 25–40, 1996. URL https://doi.org/10.1214/aos/1033066197.
+
+
+V. Inácio, M. X. Rodrı́guez-Álvarez and P. Gayoso-Diz. Statistical evaluation of medical tests. Annual Review of Statistics and Its Application, 8(1): 41–67, 2021. URL https://doi.org/10.1146/annurev-statistics-040720-022432.
+
+
+L. Kang, A. Liu and L. Tian. Linear combination methods to improve diagnostic/prognostic accuracy on future observations. Statistical Methods in Medical Research, 25(4): 1359–1380, 2016. URL https://doi.org/10.1177/0962280213481053.
+
+
+H. Kauppi. The generalized receiver operating characteristic curve. 114. Aboa Centre for Economics. 2016. URL https://www.econstor.eu/bitstream/10419/233329/1/aboa-ce-dp114.pdf.
+
+
+C. Liu, A. Liu and S. Halabi. A min–max combination of biomarkers to improve diagnostic accuracy. Statistics in Medicine, 30(16): 2005–2014, 2011. URL https://doi.org/10.1002/sim.4238.
+
+
+M. López-Ratón, M. X. Rodrı́guez-Álvarez, C. C. Suárez and F. G. Sampedro. OptimalCutpoints: An R package for selecting optimal cutpoints in diagnostic tests. Journal of Statistical Software, 61(8): 1–36, 2014. URL https://doi.org/10.18637/jss.v061.i08.
+
+
+P. Martı́nez-Camblor. On the paper “Notes on the overlap measure as an alternative to the Youden index”. Statistics in Medicine, 37(7): 1222–1224, 2018. URL https://doi.org/10.1002/sim.7517.
+
+
+P. Martı́nez-Camblor, N. Corral, C. Rey, J. Pascual and E. Cernuda-Morollón. Receiver operating characteristic curve generalization for non-monotone relationships. Statistical Methods in Medical Research, 26(1): 113–123, 2017. URL https://doi.org/10.1177/0962280214541095.
+
+
+P. Martı́nez-Camblor and J. C. Pardo-Fernández. Parametric estimates for the receiver operating characteristic curve generalization for non-monotone relationships. Statistical Methods in Medical Research, 28(7): 2032–2048, 2019a. URL https://doi.org/10.1177/0962280217747009.
+
+
+P. Martı́nez-Camblor and J. C. Pardo-Fernández. The Youden index in the generalized receiver operating characteristic curve context. The International Journal of Biostatistics, 15(1): 2019b. URL https://doi.org/10.1515/ijb-2018-0060.
+
+
+P. Martı́nez-Camblor, S. Pérez-Fernández and S. Dı́az-Coto. Improving the biomarker diagnostic capacity via functional transformations. Journal of Applied Statistics, 46(9): 1550–1566, 2019. URL https://doi.org/10.1080/02664763.2018.1554628.
+
+
+P. Martı́nez-Camblor, S. Pérez-Fernández and S. Dı́az-Coto. Optimal classification scores based on multivariate marker transformations. AStA Advances in Statistical Analysis, 105(4): 581–599, 2021a. URL https://doi.org/10.1007/s10182-020-00388-z.
+
+
+P. Martı́nez-Camblor, S. Pérez-Fernández and S. Dı́az-Coto. The area under the generalized receiver-operating characteristic curve. The International Journal of Biostatistics, 18(1): 293–306, 2021b. URL https://doi.org/10.1515/ijb-2020-0091.
+
+
+R. Mayeux. Biomarkers: Potential uses and limitations. NeuroRx, 1(2): 182–188, 2004. URL https://doi.org/10.1602/neurorx.1.2.182.
+
+
+M. W. McIntosh and M. S. Pepe. Combining several screening tests: Optimality of the risk score. Biometrics, 58(3): 657–664, 2002. URL https://doi.org/10.1111/j.0006-341X.2002.00657.x.
+
+
+A. Meisner, M. Carone, M. S. Pepe and K. F. Kerr. Combining biomarkers by maximizing the true positive rate for a fixed false positive rate. Biometrical Journal, 63(6): 1223–1240, 2021. URL https://doi.org/10.1002/bimj.202000210.
+
+
+D. Meyer, E. Dimitriadou, K. Hornik, A. Weingessel and F. Leisch. e1071: Misc functions of the department of statistics, probability theory group (formerly: E1071), TU wien. 2023. URL https://CRAN.R-project.org/package=e1071. R package version 1.7-13.
+
+
+C. T. Nakas, L. E. Bantis and C. Gatsonis. ROC analysis for classification and prediction in practice. CRC Press, 2023. URL https://www.routledge.com/ROC-Analysis-for-Classification-and-Prediction-in-Practice/Nakas-Bantis-Gatsonis/p/book/9781482233704.
+
+
+M. S. Pepe. The statistical evaluation of medical tests for classification and prediction. Oxford University Press, 2003. URL https://global.oup.com/academic/product/the-statistical-evaluation-of-medical-tests-for-classification-and-prediction-9780198565826?cc=es&lang=en&.
+
+
+M. S. Pepe and M. L. Thompson. Combining diagnostic test results to increase accuracy. Biostatistics, 1(2): 123–140, 2000. URL https://doi.org/10.1093/biostatistics/1.2.123.
+
+
+S. Pérez-Fernández, P. Martı́nez-Camblor, P. Filzmoser and N. Corral. nsROC: An R package for non-standard ROC curve analysis. The R Journal, 10(2): 55–74, 2018. URL https://doi.org/10.32614/RJ-2018-043.
+
+
+S. Pérez-Fernández, P. Martı́nez-Camblor, P. Filzmoser and N. Corral. Visualizing the decision rules behind the ROC curves: Understanding the classification process. AStA Advances in Statistical Analysis, 105(1): 135–161, 2021. URL https://doi.org/10.1007/s10182-020-00385-2.
+
+
+X. Robin, N. Turck, A. Hainard, N. Tiberti, F. Lisacek, J.-C. Sanchez and M. Müller. PROC: An open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics, 12(77): 1–8, 2011. URL https://doi.org/10.1186/1471-2105-12-77.
+
+
+M. X. Rodrı́guez-Álvarez and V. Inácio. ROCnReg: An R package for receiver operating characteristic curve inference with and without covariates. The R Journal, 13(1): 525–555, 2021. URL https://doi.org/10.32614/RJ-2021-066.
+
+
+J. Shen, S. Wang, Y.-J. Zhang, M. Kappil, H.-C. Wu, M. G. Kibriya, Q. Wang, F. Jasmine, H. Ahsan, P.-H. Lee, et al. Genome-wide DNA methylation profiles in hepatocellular carcinoma. Hepatology, 55(6): 1799–1808, 2012. URL https://doi.org/10.1002/hep.25569.
+
+
+J. Q. Su and J. S. Liu. Linear combinations of multiple diagnostic markers. Journal of the American Statistical Association, 88(424): 1350–1355, 1993. URL https://doi.org/10.1080/01621459.1993.10476417.
+
+
+Y. Xie, C. Mueller, L. Yu and W. Zhu. Animation: A gallery of animations in statistics and utilities to create animations. 2021. URL https://yihui.org/animation/. R package version 2.7.
+
+
+W. J. Youden. Index for rating diagnostic tests. Cancer, 3(1): 32–35, 1950. URL https://doi.org/10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3.
+
+
+X.-H. Zhou, D. K. McClish and N. A. Obuchowski. Statistical methods in diagnostic medicine. Wiley, 2002. URL https://doi.org/10.1002/9780470317082.
+
+
+K. H. Zou, W. J. Hall and D. E. Shapiro. Smooth non-parametric receiver operating characteristic (ROC) curves for continuous diagnostic tests. Statistics in Medicine, 16(19): 2143–2156, 1997. URL https://doi.org/10.1002/(SICI)1097-0258(19971015)16:19<2143::AID-SIM655>3.0.CO;2-3.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/_articles/RJ-2025-035/RJ-2025-035.pdf b/_articles/RJ-2025-035/RJ-2025-035.pdf
new file mode 100644
index 0000000000..ca90a4f6f1
Binary files /dev/null and b/_articles/RJ-2025-035/RJ-2025-035.pdf differ
diff --git a/_articles/RJ-2025-035/RJ-2025-035_files/anchor-4.2.2/anchor.min.js b/_articles/RJ-2025-035/RJ-2025-035_files/anchor-4.2.2/anchor.min.js
new file mode 100644
index 0000000000..1342f5f6f9
--- /dev/null
+++ b/_articles/RJ-2025-035/RJ-2025-035_files/anchor-4.2.2/anchor.min.js
@@ -0,0 +1,9 @@
+// @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&dn=expat.txt Expat
+//
+// AnchorJS - v4.2.2 - 2019-11-14
+// https://www.bryanbraun.com/anchorjs/
+// Copyright (c) 2019 Bryan Braun; Licensed MIT
+//
+// @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&dn=expat.txt Expat
+!function(A,e){"use strict";"function"==typeof define&&define.amd?define([],e):"object"==typeof module&&module.exports?module.exports=e():(A.AnchorJS=e(),A.anchors=new A.AnchorJS)}(this,function(){"use strict";return function(A){function f(A){A.icon=A.hasOwnProperty("icon")?A.icon:"",A.visible=A.hasOwnProperty("visible")?A.visible:"hover",A.placement=A.hasOwnProperty("placement")?A.placement:"right",A.ariaLabel=A.hasOwnProperty("ariaLabel")?A.ariaLabel:"Anchor",A.class=A.hasOwnProperty("class")?A.class:"",A.base=A.hasOwnProperty("base")?A.base:"",A.truncate=A.hasOwnProperty("truncate")?Math.floor(A.truncate):64,A.titleText=A.hasOwnProperty("titleText")?A.titleText:""}function p(A){var e;if("string"==typeof A||A instanceof String)e=[].slice.call(document.querySelectorAll(A));else{if(!(Array.isArray(A)||A instanceof NodeList))throw new Error("The selector provided to AnchorJS was invalid.");e=[].slice.call(A)}return e}this.options=A||{},this.elements=[],f(this.options),this.isTouchDevice=function(){return!!("ontouchstart"in window||window.DocumentTouch&&document instanceof DocumentTouch)},this.add=function(A){var e,t,i,n,o,s,a,r,c,h,l,u,d=[];if(f(this.options),"touch"===(l=this.options.visible)&&(l=this.isTouchDevice()?"always":"hover"),0===(e=p(A=A||"h2, h3, h4, h5, h6")).length)return this;for(!function(){if(null!==document.head.querySelector("style.anchorjs"))return;var A,e=document.createElement("style");e.className="anchorjs",e.appendChild(document.createTextNode("")),void 0===(A=document.head.querySelector('[rel="stylesheet"], style'))?document.head.appendChild(e):document.head.insertBefore(e,A);e.sheet.insertRule(" .anchorjs-link { opacity: 0; text-decoration: none; -webkit-font-smoothing: antialiased; -moz-osx-font-smoothing: grayscale; }",e.sheet.cssRules.length),e.sheet.insertRule(" *:hover > .anchorjs-link, .anchorjs-link:focus { opacity: 1; }",e.sheet.cssRules.length),e.sheet.insertRule(" [data-anchorjs-icon]::after { content: attr(data-anchorjs-icon); }",e.sheet.cssRules.length),e.sheet.insertRule(' @font-face { font-family: "anchorjs-icons"; src: url(data:n/a;base64,AAEAAAALAIAAAwAwT1MvMg8yG2cAAAE4AAAAYGNtYXDp3gC3AAABpAAAAExnYXNwAAAAEAAAA9wAAAAIZ2x5ZlQCcfwAAAH4AAABCGhlYWQHFvHyAAAAvAAAADZoaGVhBnACFwAAAPQAAAAkaG10eASAADEAAAGYAAAADGxvY2EACACEAAAB8AAAAAhtYXhwAAYAVwAAARgAAAAgbmFtZQGOH9cAAAMAAAAAunBvc3QAAwAAAAADvAAAACAAAQAAAAEAAHzE2p9fDzz1AAkEAAAAAADRecUWAAAAANQA6R8AAAAAAoACwAAAAAgAAgAAAAAAAAABAAADwP/AAAACgAAA/9MCrQABAAAAAAAAAAAAAAAAAAAAAwABAAAAAwBVAAIAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAAMCQAGQAAUAAAKZAswAAACPApkCzAAAAesAMwEJAAAAAAAAAAAAAAAAAAAAARAAAAAAAAAAAAAAAAAAAAAAQAAg//0DwP/AAEADwABAAAAAAQAAAAAAAAAAAAAAIAAAAAAAAAIAAAACgAAxAAAAAwAAAAMAAAAcAAEAAwAAABwAAwABAAAAHAAEADAAAAAIAAgAAgAAACDpy//9//8AAAAg6cv//f///+EWNwADAAEAAAAAAAAAAAAAAAAACACEAAEAAAAAAAAAAAAAAAAxAAACAAQARAKAAsAAKwBUAAABIiYnJjQ3NzY2MzIWFxYUBwcGIicmNDc3NjQnJiYjIgYHBwYUFxYUBwYGIwciJicmNDc3NjIXFhQHBwYUFxYWMzI2Nzc2NCcmNDc2MhcWFAcHBgYjARQGDAUtLXoWOR8fORYtLTgKGwoKCjgaGg0gEhIgDXoaGgkJBQwHdR85Fi0tOAobCgoKOBoaDSASEiANehoaCQkKGwotLXoWOR8BMwUFLYEuehYXFxYugC44CQkKGwo4GkoaDQ0NDXoaShoKGwoFBe8XFi6ALjgJCQobCjgaShoNDQ0NehpKGgobCgoKLYEuehYXAAAADACWAAEAAAAAAAEACAAAAAEAAAAAAAIAAwAIAAEAAAAAAAMACAAAAAEAAAAAAAQACAAAAAEAAAAAAAUAAQALAAEAAAAAAAYACAAAAAMAAQQJAAEAEAAMAAMAAQQJAAIABgAcAAMAAQQJAAMAEAAMAAMAAQQJAAQAEAAMAAMAAQQJAAUAAgAiAAMAAQQJAAYAEAAMYW5jaG9yanM0MDBAAGEAbgBjAGgAbwByAGoAcwA0ADAAMABAAAAAAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAH//wAP) format("truetype"); }',e.sheet.cssRules.length)}(),t=document.querySelectorAll("[id]"),i=[].map.call(t,function(A){return A.id}),o=0;oReferences
+ +Reuse
+Text and figures are licensed under Creative Commons Attribution CC BY 4.0. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from ...".
+Citation
+For attribution, please cite this work as
+Pérez-Fernández, et al., "movieROC: Visualizing the Decision Rules Underlying Binary Classification", The R Journal, 2026+
BibTeX citation
+@article{RJ-2025-035,
+ author = {Pérez-Fernández, Sonia and Martínez-Camblor, Pablo and Corral-Blanco, Norberto},
+ title = {movieROC: Visualizing the Decision Rules Underlying Binary Classification},
+ journal = {The R Journal},
+ year = {2026},
+ note = {https://doi.org/10.32614/RJ-2025-035},
+ doi = {10.32614/RJ-2025-035},
+ volume = {17},
+ issue = {4},
+ issn = {2073-4859},
+ pages = {59-79}
+}
+':''} DOI: ${e.doi}`:''}function r(e){return''+e.title+' '}function o(e){if(e){var t=r(e);return t+=a(e)+'
',e.author&&(t+=n(e,'${L}, ${I}',', ',' and '),(e.year||e.date)&&(t+=', ')),t+=e.year||e.date?(e.year||e.date)+'. ':'. ',t+=i(e),t+=d(e),t}return'?'}function l(e){if(e){var t='';t+=''+e.title+'',t+=a(e),t+='
';var r=n(e,'${I} ${L}',', ')+'.',o=i(e).trim()+' '+e.year+'. '+d(e,!0);return t+=(r+o).length
Table of contents
+- `;for(const i of t){const e='D-TITLE'==i.parentElement.tagName,t=i.getAttribute('no-toc');if(e||t)continue;const a=i.textContent,d='#'+i.getAttribute('id');let r='
- '+a+' ';'H3'==i.tagName?r='
- '+r+'
',n+=r}n+='
`,e.githubCompareUpdatesUrl&&(t+=`View all changes to this article since it was first published.`),t+=` + If you see mistakes or want to suggest changes, please create an issue on GitHub.
+ `);const n=e.journal;return'undefined'!=typeof n&&'Distill'===n.title&&(t+=` +Reuse
+Diagrams and text are licensed under Creative Commons Attribution CC-BY 4.0 with the source available on GitHub, unless noted otherwise. The figures that have been reused from other sources don’t fall under this license and can be recognized by a note in their caption: “Figure from …”.
+ `),'undefined'!=typeof e.publishedDate&&(t+=` +Citation
+For attribution in academic contexts, please cite this work as
+${e.concatenatedAuthors}, "${e.title}", Distill, ${e.publishedYear}.
+ BibTeX citation
+${m(e)}
+ `),t}var An=Math.sqrt,En=Math.atan2,Dn=Math.sin,Mn=Math.cos,On=Math.PI,Un=Math.abs,In=Math.pow,Nn=Math.LN10,jn=Math.log,Rn=Math.max,qn=Math.ceil,Fn=Math.floor,Pn=Math.round,Hn=Math.min;const zn=['Sunday','Monday','Tuesday','Wednesday','Thursday','Friday','Saturday'],Bn=['Jan.','Feb.','March','April','May','June','July','Aug.','Sept.','Oct.','Nov.','Dec.'],Wn=(e)=>10>e?'0'+e:e,Vn=function(e){const t=zn[e.getDay()].substring(0,3),n=Wn(e.getDate()),i=Bn[e.getMonth()].substring(0,3),a=e.getFullYear().toString(),d=e.getUTCHours().toString(),r=e.getUTCMinutes().toString(),o=e.getUTCSeconds().toString();return`${t}, ${n} ${i} ${a} ${d}:${r}:${o} Z`},$n=function(e){const t=Array.from(e).reduce((e,[t,n])=>Object.assign(e,{[t]:n}),{});return t},Jn=function(e){const t=new Map;for(var n in e)e.hasOwnProperty(n)&&t.set(n,e[n]);return t};class Qn{constructor(e){this.name=e.author,this.personalURL=e.authorURL,this.affiliation=e.affiliation,this.affiliationURL=e.affiliationURL,this.affiliations=e.affiliations||[]}get firstName(){const e=this.name.split(' ');return e.slice(0,e.length-1).join(' ')}get lastName(){const e=this.name.split(' ');return e[e.length-1]}}class Gn{constructor(){this.title='unnamed article',this.description='',this.authors=[],this.bibliography=new Map,this.bibliographyParsed=!1,this.citations=[],this.citationsCollected=!1,this.journal={},this.katex={},this.publishedDate=void 0}set url(e){this._url=e}get url(){if(this._url)return this._url;return this.distillPath&&this.journal.url?this.journal.url+'/'+this.distillPath:this.journal.url?this.journal.url:void 0}get githubUrl(){return this.githubPath?'https://github.com/'+this.githubPath:void 0}set previewURL(e){this._previewURL=e}get previewURL(){return this._previewURL?this._previewURL:this.url+'/thumbnail.jpg'}get publishedDateRFC(){return Vn(this.publishedDate)}get updatedDateRFC(){return Vn(this.updatedDate)}get publishedYear(){return this.publishedDate.getFullYear()}get publishedMonth(){return Bn[this.publishedDate.getMonth()]}get publishedDay(){return this.publishedDate.getDate()}get publishedMonthPadded(){return Wn(this.publishedDate.getMonth()+1)}get publishedDayPadded(){return Wn(this.publishedDate.getDate())}get publishedISODateOnly(){return this.publishedDate.toISOString().split('T')[0]}get volume(){const e=this.publishedYear-2015;if(1>e)throw new Error('Invalid publish date detected during computing volume');return e}get issue(){return this.publishedDate.getMonth()+1}get concatenatedAuthors(){if(2 tag. We found the following text: '+t);const n=document.createElement('span');n.innerHTML=e.nodeValue,e.parentNode.insertBefore(n,e),e.parentNode.removeChild(e)}}}}).observe(this,{childList:!0})}}var Ti='undefined'==typeof window?'undefined'==typeof global?'undefined'==typeof self?{}:self:global:window,_i=f(function(e,t){(function(e){function t(){this.months=['jan','feb','mar','apr','may','jun','jul','aug','sep','oct','nov','dec'],this.notKey=[',','{','}',' ','='],this.pos=0,this.input='',this.entries=[],this.currentEntry='',this.setInput=function(e){this.input=e},this.getEntries=function(){return this.entries},this.isWhitespace=function(e){return' '==e||'\r'==e||'\t'==e||'\n'==e},this.match=function(e,t){if((void 0==t||null==t)&&(t=!0),this.skipWhitespace(t),this.input.substring(this.pos,this.pos+e.length)==e)this.pos+=e.length;else throw'Token mismatch, expected '+e+', found '+this.input.substring(this.pos);this.skipWhitespace(t)},this.tryMatch=function(e,t){return(void 0==t||null==t)&&(t=!0),this.skipWhitespace(t),this.input.substring(this.pos,this.pos+e.length)==e},this.matchAt=function(){for(;this.input.length>this.pos&&'@'!=this.input[this.pos];)this.pos++;return!('@'!=this.input[this.pos])},this.skipWhitespace=function(e){for(;this.isWhitespace(this.input[this.pos]);)this.pos++;if('%'==this.input[this.pos]&&!0==e){for(;'\n'!=this.input[this.pos];)this.pos++;this.skipWhitespace(e)}},this.value_braces=function(){var e=0;this.match('{',!1);for(var t=this.pos,n=!1;;){if(!n)if('}'==this.input[this.pos]){if(0
+
+`);class Di extends Ei(HTMLElement){connectedCallback(){this.outerSpan=this.root.querySelector('#citation-'),this.innerSpan=this.root.querySelector('.citation-number'),this.hoverBox=this.root.querySelector('d-hover-box'),window.customElements.whenDefined('d-hover-box').then(()=>{this.hoverBox.listen(this)})}static get observedAttributes(){return['key']}attributeChangedCallback(e,t,n){const i=t?'onCiteKeyChanged':'onCiteKeyCreated',a=n.split(','),d={detail:[this,a],bubbles:!0},r=new CustomEvent(i,d);document.dispatchEvent(r)}set key(e){this.setAttribute('key',e)}get key(){return this.getAttribute('key')}get keys(){return this.getAttribute('key').split(',')}set numbers(e){const t=e.map((e)=>{return-1==e?'?':e+1+''}),n='['+t.join(', ')+']';this.innerSpan&&(this.innerSpan.textContent=n)}set entries(e){this.hoverBox&&(this.hoverBox.innerHTML=`
+ ${e.map(l).map((e)=>`
`)}}const Mi=`
+d-citation-list {
+ contain: layout style;
+}
+
+d-citation-list .references {
+ grid-column: text;
+}
+
+d-citation-list .references .title {
+ font-weight: 500;
+}
+`;class Oi extends HTMLElement{static get is(){return'd-citation-list'}connectedCallback(){this.hasAttribute('distill-prerendered')||(this.style.display='none')}set citations(e){x(this,e)}}var Ui=f(function(e){var t='undefined'==typeof window?'undefined'!=typeof WorkerGlobalScope&&self instanceof WorkerGlobalScope?self:{}:window,n=function(){var e=/\blang(?:uage)?-(\w+)\b/i,n=0,a=t.Prism={util:{encode:function(e){return e instanceof i?new i(e.type,a.util.encode(e.content),e.alias):'Array'===a.util.type(e)?e.map(a.util.encode):e.replace(/&/g,'&').replace(/e.length)break tokenloop;if(!(y instanceof n)){c.lastIndex=0;var v=c.exec(y),w=1;if(!v&&f&&x!=d.length-1){if(c.lastIndex=i,v=c.exec(e),!v)break;for(var S=v.index+(g?v[1].length:0),C=v.index+v[0].length,T=x,k=i,p=d.length;T
+
+`);class Ni extends ei(Ii(HTMLElement)){renderContent(){if(this.languageName=this.getAttribute('language'),!this.languageName)return void console.warn('You need to provide a language attribute to your
+
+Footnotes
+
+`,!1);class Fi extends qi(HTMLElement){connectedCallback(){super.connectedCallback(),this.list=this.root.querySelector('ol'),this.root.style.display='none'}set footnotes(e){if(this.list.innerHTML='',e.length){this.root.style.display='';for(const t of e){const e=document.createElement('li');e.id=t.id+'-listing',e.innerHTML=t.innerHTML;const n=document.createElement('a');n.setAttribute('class','footnote-backlink'),n.textContent='[\u21A9]',n.href='#'+t.id,e.appendChild(n),this.list.appendChild(e)}}else this.root.style.display='none'}}const Pi=ti('d-hover-box',`
+
+
+
+
+`);class Hi extends Pi(HTMLElement){constructor(){super()}connectedCallback(){}listen(e){this.bindDivEvents(this),this.bindTriggerEvents(e)}bindDivEvents(e){e.addEventListener('mouseover',()=>{this.visible||this.showAtNode(e),this.stopTimeout()}),e.addEventListener('mouseout',()=>{this.extendTimeout(500)}),e.addEventListener('touchstart',(e)=>{e.stopPropagation()},{passive:!0}),document.body.addEventListener('touchstart',()=>{this.hide()},{passive:!0})}bindTriggerEvents(e){e.addEventListener('mouseover',()=>{this.visible||this.showAtNode(e),this.stopTimeout()}),e.addEventListener('mouseout',()=>{this.extendTimeout(300)}),e.addEventListener('touchstart',(t)=>{this.visible?this.hide():this.showAtNode(e),t.stopPropagation()},{passive:!0})}show(e){this.visible=!0,this.style.display='block',this.style.top=Pn(e[1]+10)+'px'}showAtNode(e){const t=e.getBoundingClientRect();this.show([e.offsetLeft+t.width,e.offsetTop+t.height])}hide(){this.visible=!1,this.style.display='none',this.stopTimeout()}stopTimeout(){this.timeout&&clearTimeout(this.timeout)}extendTimeout(e){this.stopTimeout(),this.timeout=setTimeout(()=>{this.hide()},e)}}class zi extends HTMLElement{static get is(){return'd-title'}}const Yi=ti('d-references',`
+
+`,!1);class Bi extends Yi(HTMLElement){}class Wi extends HTMLElement{static get is(){return'd-toc'}connectedCallback(){this.getAttribute('prerendered')||(window.onload=()=>{const e=document.querySelector('d-article'),t=e.querySelectorAll('h2, h3');k(this,t)})}}class Vi extends HTMLElement{static get is(){return'd-figure'}static get readyQueue(){return Vi._readyQueue||(Vi._readyQueue=[]),Vi._readyQueue}static addToReadyQueue(e){-1===Vi.readyQueue.indexOf(e)&&(Vi.readyQueue.push(e),Vi.runReadyQueue())}static runReadyQueue(){const e=Vi.readyQueue.sort((e,t)=>e._seenOnScreen-t._seenOnScreen).filter((e)=>!e._ready).pop();e&&(e.ready(),requestAnimationFrame(Vi.runReadyQueue))}constructor(){super(),this._ready=!1,this._onscreen=!1,this._offscreen=!0}connectedCallback(){this.loadsWhileScrolling=this.hasAttribute('loadsWhileScrolling'),Vi.marginObserver.observe(this),Vi.directObserver.observe(this)}disconnectedCallback(){Vi.marginObserver.unobserve(this),Vi.directObserver.unobserve(this)}static get marginObserver(){if(!Vi._marginObserver){const e=window.innerHeight,t=Fn(2*e),n=Vi.didObserveMarginIntersection,i=new IntersectionObserver(n,{rootMargin:t+'px 0px '+t+'px 0px',threshold:0.01});Vi._marginObserver=i}return Vi._marginObserver}static didObserveMarginIntersection(e){for(const t of e){const e=t.target;t.isIntersecting&&!e._ready&&Vi.addToReadyQueue(e)}}static get directObserver(){return Vi._directObserver||(Vi._directObserver=new IntersectionObserver(Vi.didObserveDirectIntersection,{rootMargin:'0px',threshold:[0,1]})),Vi._directObserver}static didObserveDirectIntersection(e){for(const t of e){const e=t.target;t.isIntersecting?(e._seenOnScreen=new Date,e._offscreen&&e.onscreen()):e._onscreen&&e.offscreen()}}addEventListener(e,t){super.addEventListener(e,t),'ready'===e&&-1!==Vi.readyQueue.indexOf(this)&&(this._ready=!1,Vi.runReadyQueue()),'onscreen'===e&&this.onscreen()}ready(){this._ready=!0,Vi.marginObserver.unobserve(this);const e=new CustomEvent('ready');this.dispatchEvent(e)}onscreen(){this._onscreen=!0,this._offscreen=!1;const e=new CustomEvent('onscreen');this.dispatchEvent(e)}offscreen(){this._onscreen=!1,this._offscreen=!0;const e=new CustomEvent('offscreen');this.dispatchEvent(e)}}if('undefined'!=typeof window){Vi.isScrolling=!1;let e;window.addEventListener('scroll',()=>{Vi.isScrolling=!0,clearTimeout(e),e=setTimeout(()=>{Vi.isScrolling=!1,Vi.runReadyQueue()},500)},!0)}const Ki=ti('d-interstitial',`
+
+
+
+`);class $i extends Ki(HTMLElement){connectedCallback(){if(this.shouldRemoveSelf())this.parentElement.removeChild(this);else{const e=this.root.querySelector('#interstitial-password-input');e.oninput=(e)=>this.passwordChanged(e)}}passwordChanged(e){const t=e.target.value;t===this.password&&(console.log('Correct password entered.'),this.parentElement.removeChild(this),'undefined'!=typeof Storage&&(console.log('Saved that correct password was entered.'),localStorage.setItem(this.localStorageIdentifier(),'true')))}shouldRemoveSelf(){return window&&window.location.hostname==='distill.pub'?(console.warn('Interstitial found on production, hiding it.'),!0):'undefined'!=typeof Storage&&'true'===localStorage.getItem(this.localStorageIdentifier())&&(console.log('Loaded that correct password was entered before; skipping interstitial.'),!0)}localStorageIdentifier(){return'distill-drafts'+(window?window.location.pathname:'-')+'interstitial-password-correct'}}var Xi=function(e,t){return e
+
+
+
+
+
+`),Dr={left:37,up:38,right:39,down:40,pageUp:33,pageDown:34,end:35,home:36};class Mr extends Er(HTMLElement){connectedCallback(){this.connected=!0,this.setAttribute('role','slider'),this.hasAttribute('tabindex')||this.setAttribute('tabindex',0),this.mouseEvent=!1,this.knob=this.root.querySelector('.knob-container'),this.background=this.root.querySelector('.background'),this.trackFill=this.root.querySelector('.track-fill'),this.track=this.root.querySelector('.track'),this.min=this.min?this.min:0,this.max=this.max?this.max:100,this.scale=me().domain([this.min,this.max]).range([0,1]).clamp(!0),this.origin=this.origin===void 0?this.min:this.origin,this.step=this.step?this.step:1,this.update(this.value?this.value:0),this.ticks=!!this.ticks&&this.ticks,this.renderTicks(),this.drag=Ar().container(this.background).on('start',()=>{this.mouseEvent=!0,this.background.classList.add('mousedown'),this.changeValue=this.value,this.dragUpdate()}).on('drag',()=>{this.dragUpdate()}).on('end',()=>{this.mouseEvent=!1,this.background.classList.remove('mousedown'),this.dragUpdate(),this.changeValue!==this.value&&this.dispatchChange(),this.changeValue=this.value}),this.drag(Sr(this.background)),this.addEventListener('focusin',()=>{this.mouseEvent||this.background.classList.add('focus')}),this.addEventListener('focusout',()=>{this.background.classList.remove('focus')}),this.addEventListener('keydown',this.onKeyDown)}static get observedAttributes(){return['min','max','value','step','ticks','origin','tickValues','tickLabels']}attributeChangedCallback(e,t,n){isNaN(n)||void 0===n||null===n||('min'==e&&(this.min=+n,this.setAttribute('aria-valuemin',this.min)),'max'==e&&(this.max=+n,this.setAttribute('aria-valuemax',this.max)),'value'==e&&this.update(+n),'origin'==e&&(this.origin=+n),'step'==e&&0
+
+
+
+
+
+
+ ${Or}
+ Distill
+
+
+
+`,!1);class Ir extends Ur(HTMLElement){}const Nr=`
+
+`;class jr extends HTMLElement{static get is(){return'distill-appendix'}set frontMatter(e){this.innerHTML=Ln(e)}}const Rr=ti('distill-footer',`
+
+
+
+
+
+ ${Or}
+ Distill
+ is dedicated to clear explanations of machine learning
+
+
+
+
+
+`);class qr extends Rr(HTMLElement){}const Fr=function(){if(1>window.distillRunlevel)throw new Error('Insufficient Runlevel for Distill Template!');if('distillTemplateIsLoading'in window&&window.distillTemplateIsLoading)throw new Error('Runlevel 1: Distill Template is getting loaded more than once, aborting!');else window.distillTemplateIsLoading=!0,console.info('Runlevel 1: Distill Template has started loading.');p(document),console.info('Runlevel 1: Static Distill styles have been added.'),console.info('Runlevel 1->2.'),window.distillRunlevel+=1;for(const[e,t]of Object.entries(hi.listeners))'function'==typeof t?document.addEventListener(e,t):console.error('Runlevel 2: Controller listeners need to be functions!');console.info('Runlevel 2: We can now listen to controller events.'),console.info('Runlevel 2->3.'),window.distillRunlevel+=1;if(2>window.distillRunlevel)throw new Error('Insufficient Runlevel for adding custom elements!');const e=[ki,wi,Ci,Li,Ai,Di,Oi,Ni,Ri,Fi,pi,Hi,zi,T,Bi,Wi,Vi,Mr,$i].concat([Ir,jr,qr]);for(const t of e)console.info('Runlevel 2: Registering custom element: '+t.is),customElements.define(t.is,t);console.info('Runlevel 3: Distill Template finished registering custom elements.'),console.info('Runlevel 3->4.'),window.distillRunlevel+=1,hi.listeners.DOMContentLoaded(),console.info('Runlevel 4: Distill Template initialisation complete.')};window.distillRunlevel=0,yi.browserSupportsAllFeatures()?(console.info('Runlevel 0: No need for polyfills.'),console.info('Runlevel 0->1.'),window.distillRunlevel+=1,Fr()):(console.info('Runlevel 0: Distill Template is loading polyfills.'),yi.load(Fr))});
+//# sourceMappingURL=template.v2.js.map
+}
diff --git a/_articles/RJ-2025-035/RJ-2025-035_files/header-attrs-2.30/header-attrs.js b/_articles/RJ-2025-035/RJ-2025-035_files/header-attrs-2.30/header-attrs.js
new file mode 100644
index 0000000000..dd57d92e02
--- /dev/null
+++ b/_articles/RJ-2025-035/RJ-2025-035_files/header-attrs-2.30/header-attrs.js
@@ -0,0 +1,12 @@
+// Pandoc 2.9 adds attributes on both header and div. We remove the former (to
+// be compatible with the behavior of Pandoc < 2.8).
+document.addEventListener('DOMContentLoaded', function(e) {
+ var hs = document.querySelectorAll("div.section[class*='level'] > :first-child");
+ var i, h, a;
+ for (i = 0; i < hs.length; i++) {
+ h = hs[i];
+ if (!/^h[1-6]$/i.test(h.tagName)) continue; // it should be a header h1-h6
+ a = h.attributes;
+ while (a.length > 0) h.removeAttribute(a[0].name);
+ }
+});
diff --git a/_articles/RJ-2025-035/RJ-2025-035_files/jquery-3.6.0/jquery-3.6.0.js b/_articles/RJ-2025-035/RJ-2025-035_files/jquery-3.6.0/jquery-3.6.0.js
new file mode 100644
index 0000000000..fc6c299b73
--- /dev/null
+++ b/_articles/RJ-2025-035/RJ-2025-035_files/jquery-3.6.0/jquery-3.6.0.js
@@ -0,0 +1,10881 @@
+/*!
+ * jQuery JavaScript Library v3.6.0
+ * https://jquery.com/
+ *
+ * Includes Sizzle.js
+ * https://sizzlejs.com/
+ *
+ * Copyright OpenJS Foundation and other contributors
+ * Released under the MIT license
+ * https://jquery.org/license
+ *
+ * Date: 2021-03-02T17:08Z
+ */
+( function( global, factory ) {
+
+ "use strict";
+
+ if ( typeof module === "object" && typeof module.exports === "object" ) {
+
+ // For CommonJS and CommonJS-like environments where a proper `window`
+ // is present, execute the factory and get jQuery.
+ // For environments that do not have a `window` with a `document`
+ // (such as Node.js), expose a factory as module.exports.
+ // This accentuates the need for the creation of a real `window`.
+ // e.g. var jQuery = require("jquery")(window);
+ // See ticket #14549 for more info.
+ module.exports = global.document ?
+ factory( global, true ) :
+ function( w ) {
+ if ( !w.document ) {
+ throw new Error( "jQuery requires a window with a document" );
+ }
+ return factory( w );
+ };
+ } else {
+ factory( global );
+ }
+
+// Pass this if window is not defined yet
+} )( typeof window !== "undefined" ? window : this, function( window, noGlobal ) {
+
+// Edge <= 12 - 13+, Firefox <=18 - 45+, IE 10 - 11, Safari 5.1 - 9+, iOS 6 - 9.1
+// throw exceptions when non-strict code (e.g., ASP.NET 4.5) accesses strict mode
+// arguments.callee.caller (trac-13335). But as of jQuery 3.0 (2016), strict mode should be common
+// enough that all such attempts are guarded in a try block.
+"use strict";
+
+var arr = [];
+
+var getProto = Object.getPrototypeOf;
+
+var slice = arr.slice;
+
+var flat = arr.flat ? function( array ) {
+ return arr.flat.call( array );
+} : function( array ) {
+ return arr.concat.apply( [], array );
+};
+
+
+var push = arr.push;
+
+var indexOf = arr.indexOf;
+
+var class2type = {};
+
+var toString = class2type.toString;
+
+var hasOwn = class2type.hasOwnProperty;
+
+var fnToString = hasOwn.toString;
+
+var ObjectFunctionString = fnToString.call( Object );
+
+var support = {};
+
+var isFunction = function isFunction( obj ) {
+
+ // Support: Chrome <=57, Firefox <=52
+ // In some browsers, typeof returns "function" for HTML