| Type: | Package |
| Title: | Structured Covariances Estimators for Pairwise and Spatial Covariates |
| Version: | 0.1.2 |
| Description: | Implements estimators for structured covariance matrices in the presence of pairwise and spatial covariates. Metodiev, Perrot-Dockès, Ouadah, Fosdick, Robin, Latouche & Raftery (2025) <doi:10.48550/arXiv.2411.04520>. |
| License: | GPL (≥ 3) |
| Encoding: | UTF-8 |
| RoxygenNote: | 7.3.2 |
| Imports: | Matrix, missMDA, mvtnorm, ohenery, pracma, quadprog, withr, purrr |
| VignetteBuilder: | knitr |
| Suggests: | knitr, markdown, corrplot |
| NeedsCompilation: | no |
| Packaged: | 2025-10-21 11:26:27 UTC; metodiev |
| Author: | Martin Metodiev 
[aut, cre, cph],
Marie Perrot-Dockès [aut],
Stéphane Robin [aut] |
| Maintainer: | Martin Metodiev <m.metodiev@tutanota.com> |
| Repository: | CRAN |
| Date/Publication: | 2025-10-25 12:20:02 UTC |
Computes the correlation matrix corresponding to the SCE model
Description
Computes the correlation matrix corresponding to the SCE model
Usage
CovMat_03(
parm,
matList,
adj_positions,
combined_effects = FALSE,
interaction_effects = list()
)
Arguments
parm |
the parameter |
matList |
a list of the matrices (pairwise+spatial) |
adj_positions |
position of the spatial effect (if embedded) |
combined_effects |
if yes, computes the Fosdick-Raftery model |
interaction_effects |
lists the interaction effects |
Value
a named list including the following elements: Phi: sum of scaled pairwise effect matrices Gamma: the null-matrix (no longer used) matList_combined list of all matrices (pairwise+spatial+interaction) Sigma the SCE-model correlation matrix alpha_delta linear effects (pairwise+spatial)
the correlation matrix corresponding to the SCE model
Computes the Fisher information matrix
Description
Computes the Fisher information matrix
Usage
Fisher_information(adj_positions, parm, matList, interaction_effects = list())
Arguments
adj_positions |
positions of the spatial effect (if embedded) |
parm |
the parameter |
matList |
list of the matrices (pairwise+spatial) |
interaction_effects |
list of pairwise effects (vectors of named pairs) |
Value
the Fisher information matrix
Calculates the gradient of the function of the transformed parameter
Description
Calculates the gradient of the function of the transformed parameter
Usage
GradLogLikLogParm_02(
adj_positions,
logParm,
matList,
dataset,
interaction_effects = list()
)
Arguments
adj_positions |
positions of the spatial effect (if embedded) |
logParm |
the transformed parameter |
matList |
the list of matrices (pairwise + spatial) |
dataset |
an n (observations) x d (dimensions) matrix |
interaction_effects |
list of interaction effects (vectors of names) |
Value
the gradient of the function of the transformed parameter
Calculates the gradient of the loglikelihood or the gradient of Sigma
Description
Calculates the gradient of the loglikelihood or the gradient of Sigma
Usage
GradLogLikParm_02(
adj_positions,
parm,
matList,
dataset,
interaction_effects = list(),
return_Sigma_der = FALSE
)
Arguments
adj_positions |
positions of the spatial effect (if embedded) |
parm |
the parameter |
matList |
the list of matrices (pairwise + spatial) |
dataset |
an n (observations) x d (dimensions) matrix |
interaction_effects |
list of interaction effects (vectors of names) |
return_Sigma_der |
returns the gradient of Sigma if TRUE |
Value
the gradient of the loglikelihood or the gradient of Sigma
Computes (a translation of) the loglikelihood for the transformed parameter
Description
Computes (a translation of) the loglikelihood for the transformed parameter
Usage
LogLikLogParm_02(
adj_positions,
logParm,
matList,
dataset,
interaction_effects = list()
)
Arguments
adj_positions |
positions of spatial effect (if embedded) |
logParm |
the transformed parameter |
matList |
the list of matrices (pairwise + spatial) |
dataset |
an n (observations) x d (dimension) matrix |
interaction_effects |
list of interaction effects (vectors of names) |
Value
(a translation of) the loglikelihood
Computes (a translation of) the loglikelihood
Description
Computes (a translation of) the loglikelihood
Usage
LogLikParm_02(
adj_positions,
parm,
matList,
dataset,
interaction_effects = list()
)
Arguments
adj_positions |
positions of spatial effect (if embedded) |
parm |
the parameter |
matList |
the list of matrices (pairwise + spatial) |
dataset |
an n (observations) x d (dimension) matrix |
interaction_effects |
list of interaction effects (vectors of names) |
Value
(a translation of) the loglikelihood
Calculate average effects (the mean effect over the matrix support)
Description
Calculate average effects (the mean effect over the matrix support)
Usage
avg_effect(parm, matList, adj_positions, interaction_effects = list())
Arguments
parm |
the parameter |
matList |
list of pairwise and spatial matrices |
adj_positions |
positions within the adjacency matrix |
interaction_effects |
list of interaction effects |
Value
the average effects (the mean effect over the matrix support)
Calculates the backward transformation of the parameter
Description
Calculates the backward transformation of the parameter
Usage
backward_transform_param(param)
Value
the parameter after applying the backward transformation
Calculates the Jacobian of the backwards transformation
Description
Calculates the Jacobian of the backwards transformation
Usage
backward_transform_param_jacobian(param)
Value
the Jacobian of the backwards transformation
Calculates the matrix D used in the quadratic minimization problem
Description
Calculates the matrix D used in the quadratic minimization problem
Usage
calc_Dmat(matList_full)
Arguments
matList_full |
combined list of pairwise and spatial matrices |
Value
the matrix D
Minimizes the Frobenius norm via quadratic optimization
Description
Minimizes the Frobenius norm via quadratic optimization
Usage
calc_Sigma_opt_frob(matList_full, corY, edge_constraints = c())
Arguments
matList_full |
combined list of pairwise and spatial matrices |
corY |
a correlation matrix estimate |
edge_constraints |
a vector indicating which effect is constrained |
Value
a list giving the optimal parameter and corresponding matrix
Calculates the vector d used in the quadratic minimization problem
Description
Calculates the vector d used in the quadratic minimization problem
Usage
calc_dvec(S, matList_full)
Arguments
S |
the Pearson correlation matrix |
matList_full |
combined list of pairwise and spatial matrices |
Value
the vector d
Calculates the average correlation for the spatial effect
Description
Calculates the average correlation for the spatial effect
Usage
calc_mean_neighbor_effect(matList, beta, delta, adj_positions)
Arguments
matList |
list of pairwise and spatial matrices |
beta |
autocorrelation parameter of the CAR model |
delta |
linear effect of the CAR model correlation matrix |
adj_positions |
positions within the adjacency matrix |
Value
the average correlation for the spatial effect
Calculates the derivative of the inverse of G (the CAR model matrix)
Description
Calculates the derivative of the inverse of G (the CAR model matrix)
Usage
calc_tilde_G_inv_partial_beta(M, A, beta)
Arguments
M |
matrix used in the definition of the CAR model |
A |
matrix used in the definition of the CAR model |
beta |
autocorrelation parameter of the CAR model |
Value
the derivative of the inverse of G (the CAR model matrix)
Adds combined effects to the matList via the Hadamard product
Description
Adds combined effects to the matList via the Hadamard product
Usage
combined_matList(matList, interaction_effects = NULL, check_redundancy = FALSE)
Arguments
matList |
a list of all matrices |
interaction_effects |
list of vector pairs indicating interactions |
Value
named list with all matList combinations and their positions
Computes the derivative of the correlation matrix w.r.t. beta
Description
Computes the derivative of the correlation matrix w.r.t. beta
Usage
combined_matList_partial_der(
matList,
link_matList,
tilde_G_inv_partial_beta,
interaction_effects = list()
)
Arguments
matList |
a list of all matrices |
link_matList |
not in use (could be removed) |
tilde_G_inv_partial_beta |
derivative of G w.r.t. beta |
interaction_effects |
list of vector pairs indicating interactions |
Value
the derivative of the correlation matrix w.r.t. beta
Computes non-positive-semidefinite approximation of correlation matrix
Description
Computes non-positive-semidefinite approximation of correlation matrix
Usage
compute_marginal_cor(dataset)
Arguments
dataset |
n x d matrix (n = number of observations, d = dimension) |
Value
non-positive-semidefinite approximation of correlation matrix
Computes correlation matrix from a normalized dataset (=standard errors)
Description
Computes correlation matrix from a normalized dataset (=standard errors)
Usage
cor_from_standard_errors(varepsilon)
Arguments
varepsilon |
n x d matrix (n = number of observations, d = dimension) |
Value
the correlation matrix
Estimates the correlation matrix of the dataset
Description
Estimates the correlation matrix of the dataset
Usage
correlation_matrix(dataset, mean_estim, sd_estim)
Arguments
dataset |
n x d matrix (n = number of observations, d = dimension) |
mean_estim |
estimate of the mean vector of the dataset |
sd_estim |
estimate of the standard deviation vector of the dataset |
Value
an estimate of the correlation matrix
Calculates the derivative of the spatial average effect
Description
Calculates the derivative of the spatial average effect
Usage
eta_D_der(
parm,
matList,
adj_positions,
interaction_effects = list(),
index = 4
)
Arguments
parm |
the parameter |
matList |
a list of the matrices (pairwise+spatial) |
adj_positions |
the positions of the spatial effect (if embedded) |
interaction_effects |
the interaction effect pairs (2-pair-name-vectors) |
index |
position of the spatial effect within the list |
Value
the derivative of the spatial average effect
Transforms the parameter using a logit and inverse softmax
Description
Transforms the parameter using a logit and inverse softmax
Usage
forward_transform_param(param)
Arguments
param |
the parameter |
Value
the transformed parameter
Calculates the Frobenius inner product between to square matrices
Description
Calculates the Frobenius inner product between to square matrices
Usage
frob_scalar_prod(A, B)
Arguments
A |
a square matrix |
B |
a square matrix |
Value
the Frobenius inner product
Computes the matrices needed for the spatial effect
Description
Computes the matrices needed for the spatial effect
Usage
get_M_A(adj_matrix)
Arguments
adj_matrix |
the adjacency matrix of the spatial effect |
Value
a list containing Ml and Al, which are used to compute the correlation matrix for the spatial effect
Computes the initial value estimator (IVE)
Description
This function computes the IVE estimator for large covariances in the presence of pairwise and spatial covariates from Metodiev et al. (2024).
Usage
ive(
pairwise_covariate_matrices = NULL,
adj_matrix = NULL,
dataset,
mean_estim = NULL,
sd_estim = NULL,
grid_size = 100,
parallelize = FALSE,
ncores = 8,
adj_positions = 1:nrow(adj_matrix),
interaction_effects = list()
)
Arguments
pairwise_covariate_matrices |
named list of square matrices |
adj_matrix |
adjacency matrix of the spatial covariate |
dataset |
the dataset given in matrix form |
mean_estim |
mean vector estimate |
sd_estim |
standard deviation vector estimate |
grid_size |
grid-size for spatial effect |
parallelize |
uses parallel-processing if TRUE |
ncores |
number of cores for the parallelization |
adj_positions |
positions within the adjacency matrix |
interaction_effects |
list of interaction effects |
Value
Returns a named list with the following elements:
parm, estimated parameters of pairwise, spatial effects average_effects, average effects of the covariates corrmat_estim, estimator of the correlation matrix covmat_estim, estimator of the covariance matrix
References
Metodiev, M., Perrot-Dockès, M., Ouadah, S., Fosdick, B. K., Robin, S., Latouche, P., & Raftery, A. E. (2024). A Structured Estimator for large Covariance Matrices in the Presence of Pairwise and Spatial Covariates. arXiv preprint arXiv:2411.04520.
Computes a measure of distance between the support of two matrices
Description
Computes a measure of distance between the support of two matrices
Usage
mat_support_distance(mat1, mat2)
Arguments
mat1 |
a square matrix |
mat2 |
a square matrix |
Value
a measure of distance between the support of two matrices
Computes the structured covariance matrix estimator (SCE)
Description
This function computes the SCE estimator for large covariances in the presence of pairwise and spatial covariates from Metodiev et al. (2024).
Usage
sce(
pairwise_covariate_matrices,
adj_matrix,
dataset,
mean_estim = NULL,
sd_estim = NULL,
grid_size = 100,
parallelize = TRUE,
ncores = 8,
adj_positions = 1:nrow(adj_matrix),
interaction_effects = list(),
init = NULL,
verbose = TRUE
)
Arguments
pairwise_covariate_matrices |
named list of square matrices |
adj_matrix |
adjacency matrix of the spatial covariate |
dataset |
the dataset given in matrix form |
mean_estim |
mean vector estimate |
sd_estim |
standard deviation vector estimate |
grid_size |
grid-size for spatial effect |
parallelize |
uses parallel-processing if TRUE |
ncores |
number of cores for the parallelization |
adj_positions |
positions within the adjacency matrix |
interaction_effects |
list of interaction effects |
init |
the initialization parameter vector |
verbose |
prints progress if TRUE |
Value
Returns a named list with the following elements:
parm, estimated parameters of pairwise, spatial effects, average_effects, average effects of the covariates, corrmat_estim, estimator of the correlation matrix, covmat_estim, estimator of the covariance matrix, bic, the Bayesian information criterion (BIC)
References
Metodiev, M., Perrot-Dockès, M., Ouadah, S., Fosdick, B. K., Robin, S., Latouche, P., & Raftery, A. E. (2024). A Structured Estimator for large Covariance Matrices in the Presence of Pairwise and Spatial Covariates. arXiv preprint arXiv:2411.04520.
Computes a structured estimator for covariance matrices
Description
This function computes the WSCE, SCE or IVE estimator for large covariances in the presence of pairwise and spatial covariates from Metodiev et al. (2024).
Usage
scov(
pairwise_covariate_matrices,
adj_matrix,
dataset,
mean_estim = NULL,
sd_estim = NULL,
grid_size = 100,
parallelize = FALSE,
ncores = 8,
adj_positions = 1:nrow(adj_matrix),
interaction_effects = list(),
init = NULL,
use_bootstrap = FALSE,
num_bootstrap_iters = 100,
semiparametric = FALSE,
misspecification = FALSE,
seed = 0,
verbose = TRUE
)
Arguments
pairwise_covariate_matrices |
named list of square matrices |
adj_matrix |
adjacency matrix of the spatial covariate |
dataset |
the dataset given in matrix form |
mean_estim |
mean vector estimate |
sd_estim |
standard deviation vector estimate |
grid_size |
grid-size for spatial effect |
parallelize |
uses parallel-processing if TRUE |
ncores |
number of cores for the parallelization |
adj_positions |
positions within the adjacency matrix |
interaction_effects |
list of interaction effects |
init |
the initialization parameter vector |
use_bootstrap |
uses bootstrapping if TRUE |
num_bootstrap_iters |
number of bootstrap simulations |
semiparametric |
computes the IVE if TRUE, the SCE else |
misspecification |
computes the WSCE if TRUE, the WSCE else |
seed |
a seed (can't be set to NULL) |
verbose |
prints progress if TRUE |
Value
Returns a named list with the following elements:
parm, estimated parameters of pairwise, spatial effects average_effects, average effects of the covariates corrmat_estim, estimator of the correlation matrix covmat_estim, estimator of the covariance matrix bic, the Bayesian information criterion (BIC) lambda, the asymptotically optimal weight of the WSCE
References
Metodiev, M., Perrot-Dockès, M., Ouadah, S., Fosdick, B. K., Robin, S., Latouche, P., & Raftery, A. E. (2024). A Structured Estimator for large Covariance Matrices in the Presence of Pairwise and Spatial Covariates. arXiv preprint arXiv:2411.04520.
Examples
intercept = matrix(1,ncol=4,nrow=4)
X1 = rbind(c(1,1,1,0),c(1,1,1,0),c(1,1,1,0),c(0,0,0,1))
X2 = rbind(c(1,0,0,0),c(0,1,1,1),c(0,1,1,1),c(0,1,1,1))
covar_mats = list(intercept=intercept,X1=X1,X2=X2)
adj_matrix = rbind(c(0,1,0,0),c(1,0,0,0),c(0,0,0,1),c(0,0,1,0))
mean = rep(0,4)
sigma = 0.05*intercept+0.2*X1+0.2*X2+0.1*X2*X1+0.4*(diag(4) + adj_matrix)
diag(sigma) = 1
dataset = mvtnorm::rmvnorm(1000,mean=mean,sigma=sigma)
scov(covar_mats, adj_matrix, dataset,
interaction_effects=list(c("X1","X2")),
parallelize=FALSE,ncores=1)
Computes the inverse of the correlation matrix of the CAR model
Description
Computes the inverse of the correlation matrix of the CAR model
Usage
tilde_G_inv(
M = NULL,
A = NULL,
beta,
U_full = NULL,
solve_U_full = NULL,
solve_M_no_islands = NULL,
eigen_real = NULL,
return_U_D_M = FALSE
)
Arguments
M |
used to define the CAR correlation matrix |
A |
used to define the CAR correlation matrix |
beta |
autocorrelation parameter of the CAR model |
U_full |
(optional) can be used to fasten computation |
solve_U_full |
(optional) can be used to fasten computation |
solve_M_no_islands |
(optional) can be used to fasten computation |
eigen_real |
(optional) can be used to fasten computation |
return_U_D_M |
(optoinal) can be used to fasten computation |
Value
the inverse of the correlation matrix of the CAR model
Maps the pairwise covariates to symmetric, positive definite matrices
Description
Maps the pairwise covariates to symmetric, positive definite matrices
Usage
to_positive_definite(pairwise_covariate_matrices, eig.tol = 1e-06)
Arguments
pairwise_covariate_matrices |
list of pairwise covariate matrices |
eig.tol |
treat EV a as negative if (a / largest eigenvalue) <= eig.tol |
Value
list of pairwise covariate matrices
Computes the "true" (i.e., not translated) log-likelihood (needed for BIC)
Description
Computes the "true" (i.e., not translated) log-likelihood (needed for BIC)
Usage
true_LogLikParm_02(
adj_positions,
parm,
matList,
dataset,
interaction_effects = list()
)
Arguments
adj_positions |
positions of spatial effect (if embedded) |
parm |
the parameter |
matList |
list of matrices (pairwise + spatial) |
dataset |
an n (observations) x d (dimension) matrix |
interaction_effects |
list of interaction effects (vector of names) |
Value
the "true" (i.e., not translated) log-likelihood
Computes the weighted structured covariance matrix estimator (WSCE)
Description
This function computes the WSCE estimator for large covariances in the presence of pairwise and spatial covariates from Metodiev et al. (2024).
Usage
wsce(
pairwise_covariate_matrices,
adj_matrix,
dataset,
mean_estim = NULL,
sd_estim = NULL,
grid_size = 100,
parallelize = FALSE,
ncores = 8,
adj_positions = 1:nrow(adj_matrix),
interaction_effects = list(),
init = NULL,
sce_init = NULL,
use_bootstrap = FALSE,
num_bootstrap_iters = 100,
seed = 0,
verbose = TRUE
)
Arguments
pairwise_covariate_matrices |
named list of square matrices |
adj_matrix |
adjacency matrix of the spatial covariate |
dataset |
the dataset given in matrix form |
mean_estim |
mean vector estimate |
sd_estim |
standard deviation vector estimate |
grid_size |
grid-size for spatial effect |
parallelize |
uses parallel-processing if TRUE |
ncores |
number of cores for the parallelization |
adj_positions |
positions within the adjacency matrix |
interaction_effects |
list of interaction effects |
init |
the initialization parameter vector |
sce_init |
the sce-initialization parameter vector |
use_bootstrap |
uses bootstrapping if TRUE |
num_bootstrap_iters |
number of bootstrap simulations |
seed |
a seed |
verbose |
prints progress if TRUE |
Value
Returns a named list with the following elements:
parm, estimated parameters of pairwise, spatial effects, average_effects, average effects of the covariates, corrmat_estim, estimator of the correlation matrix, covmat_estim, estimator of the covariance matrix, bic, the Bayesian information criterion (BIC), lambda, the asymptotically optimal weight of the WSCE
References
Metodiev, M., Perrot-Dockès, M., Ouadah, S., Fosdick, B. K., Robin, S., Latouche, P., & Raftery, A. E. (2024). A Structured Estimator for large Covariance Matrices in the Presence of Pairwise and Spatial Covariates. arXiv preprint arXiv:2411.04520.