Estimation of covariance and precision matrix, network structure, and a view toward systems biology
Wiley Interdisciplinary Reviews Computational StatisticsPublished 28 September 2017
Markku Kuismin, Mikko J. Sillanpää
Citations39
SJR quartileQ1
SJR score1.45
SNIP3.11
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
It is described how sparse precision matrix estimation methods can be used to infer network structure and methods that are suitable for gene coexpression network construction are discussed.
Abstract
Covariance matrix and its inverse, known as the precision matrix, have many applications in multivariate analysis because their elements can exhibit the variance, correlation, covariance, and conditional independence between variables. The practice of estimating the precision matrix directly without involving any matrix inversion has obtained significant attention in the literature. We review the methods that have been implemented in R and their R packages, particularly when there are more variables than data samples and discuss ideas behind them. We describe how sparse precision matrix estimation methods can be used to infer network structure. Finally, we discuss methods that are suitable for gene coexpression network construction. WIREs Comput Stat 2017, 9:e1415. doi: 10.1002/wics.1415 This article is categorized under: Statistical Models > Linear Models Applications of Computational Statistics > Computational and Molecular Biology Statistical and Graphical Methods of Data Analysis > Multivariate Analysis
Keywords
Biochemistry, Genetics and Molecular Biology
Journal of the Royal Statistical Society Series B (Statistical Methodology)Regression Shrinkage and Selection Via the Lasso
51,790 Citations1996Robert Tibshirani
A new method for estimation in linear models called the lasso, which minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant, is proposed.
BMC BioinformaticsWGCNA: an R package for weighted correlation network analysis
29,138 Citations2008Peter Langfelder, Steve Horvath
The WGCNA R software package is a comprehensive collection of R functions for performing various aspects of weighted correlation network analysis that includes functions for network construction, module detection, gene selection, calculations of topological properties, data simulation, visualization, and interfacing with external software.
Springer series in statisticsThe Elements of Statistical Learning
23,408 Citations2009Trevor Hastie, Robert Tibshirani +1 more
Proceedings of the National Academy of SciencesCluster analysis and display of genome-wide expression patterns
16,395 Citations1998Michael B. Eisen, Paul T. Spellman +2 more
A system of cluster analysis for genome-wide expression data from DNA microarray hybridization is described that uses standard statistical algorithms to arrange genes according to similarity in pattern of gene expression, finding in the budding yeast Saccharomyces cerevisiae that clustering gene expression data groups together efficiently genes of known similar function.
Journal of the American Statistical AssociationThe Adaptive Lasso and Its Oracle Properties
7,633 Citations2006Hui Zou
A new version of the lasso is proposed, called the adaptive lasso, where adaptive weights are used for penalizing different coefficients in the ℓ1 penalty, and the nonnegative garotte is shown to be consistent for variable selection.
BiostatisticsSparse inverse covariance estimation with the graphical lasso
6,574 Citations2007Jerome H. Friedman, Trevor Hastie +1 more
Using a coordinate descent procedure for the lasso, a simple algorithm is developed that solves a 1000-node problem in at most a minute and is 30-4000 times faster than competing methods.
Statistical Applications in Genetics and Molecular BiologyA General Framework for Weighted Gene Co-Expression Network Analysis
6,035 Citations2005Bin Zhang, Steve Horvath
A general framework for `soft' thresholding that assigns a connection weight to each gene pair is described and several node connectivity measures are introduced and provided empirical evidence that they can be important for predicting the biological significance of a gene.
SIAM Journal on Scientific ComputingA Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs
5,588 Citations1998George Karypis, Vipin Kumar
This work presents a new coarsening heuristic (called heavy-edge heuristic) for which the size of the partition of the coarse graph is within a small factor of theSize of the final partition obtained after multilevel refinement, and presents a much faster variation of the Kernighan--Lin (KL) algorithm for refining during uncoarsening.
Journal of Multivariate AnalysisA well-conditioned estimator for large-dimensional covariance matrices
2,931 Citations2003Olivier Ledoit, Michael Wolf
This paper introduces an estimator that is both well-conditioned and more accurate than the sample covariance matrix asymptotically, that is distribution-free and has a simple explicit formula that is easy to compute and interpret.
Statistical Learning with Sparsity
2,582 Citations2015Trevor Hastie, Robert Tibshirani +1 more
The Annals of StatisticsHigh-dimensional graphs and variable selection with the Lasso
2,406 Citations2006Nicolai Meinshausen, Peter Bühlmann
It is shown that neighborhood selection with the Lasso is a computationally attractive alternative to standard covariance selection for sparse high-dimensional graphs and is hence equivalent to variable selection for Gaussian linear models.
MPG.PuRe (Max Planck Society)A kernel two-sample test
2,231 Citations2012Arthur Gretton, Karsten Borgwardt +3 more
This work proposes a framework for analyzing and comparing distributions, which is used to construct statistical tests to determine if two samples are drawn from different distributions, and presents two distribution free tests based on large deviation bounds for the maximum mean discrepancy (MMD).
Journal of the Royal Statistical Society Series B (Statistical Methodology)Stability Selection
2,135 Citations2010Nicolai Meinshausen, Peter Bühlmann
It is proved for the randomized lasso that stability selection will be variable selection consistent even if the necessary conditions for consistency of the original lasso method are violated.
arXiv (Cornell University)The Dantzig selector: Statistical estimation when $p$ is much larger than $n$
1,725 Citations2005Candes, Emmanuel, Tao, Terence
Is it possible to estimate β reliably based on the noisy data y?
Journal of Machine Learning ResearchModel Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data
1,296 Citations2008Onureena Banerjee, Laurent El Ghaoui +1 more
Covariance regularization by thresholding
1,171 Citations2007Peter J. Bickel, Elizaveta Levina
Journal of the American Statistical AssociationA Constrained<i>ℓ</i><sub>1</sub>Minimization Approach to Sparse Precision Matrix Estimation
1,019 Citations2011Tommaso Cai, Weidong Liu +1 more
A constrained ℓ1 minimization method for estimating a sparse inverse covariance matrix based on a sample of n iid p-variate random variables and is applied to analyze a breast cancer dataset and is found to perform favorably compared with existing methods.
Journal of the American Statistical AssociationLarge-Scale Simultaneous Hypothesis Testing
1,018 Citations2004Bradley Efron
An empirical Bayes analysis plan is developed, using a local version of the false discovery rate to examine the inference issues and the importance of correctly choosing the null hypothesis.
The Annals of StatisticsCovariance regularization by thresholding
862 Citations2008Peter J. Bickel, Elizaveta Levina
It is shown that the thresholded estimate is consistent in the operator norm as long as the true covariance matrix is sparse in a suitable sense, the variables are Gaussian or sub-Gaussian, and $(\log p)/n\to0$, and obtain explicit rates.
arXiv (Cornell University)Extended Bayesian Information Criteria for Gaussian Graphical Models
713 Citations2010Rina Foygel, Mathias Drton
Proceedings of the National Academy of SciencesSpectral redemption in clustering sparse networks
657 Citations2013Florent Krząkała, Cristopher Moore +5 more
A way of encoding sparse data using a “nonbacktracking” matrix, and it is shown that the corresponding spectral algorithm performs optimally for some popular generative models, including the stochastic block model.
BiometrikaSquare-root lasso: pivotal recovery of sparse signals via conic programming
613 Citations2011Alexandre Belloni, Victor Chernozhukov +1 more
A pivotal method for estimating high-dimensional sparse linear regression models, where the overall number of regressors p is large, possibly much larger than n, but only s regressors are significant, which achieves near-oracle performance, attaining the convergence rate σl(s/n) log pr-super-1/2 in the prediction norm.
PLoS GeneticsIntegrating Genetic and Network Analysis to Characterize Genes Related to Mouse Weight
500 Citations2006Anatole Ghazalpour, Sudheer Doss +9 more
PubMedThe huge Package for High-dimensional Undirected Graph Estimation in R
485 Citations2020Tuo Zhao, Han Liu +3 more
An R package named huge which provides easy-to-use functions for estimating high dimensional undirected graphs from data and allows the user to apply both lossless and lossy screening rules to scale up large-scale problems, making a tradeoff between computational and statistical efficiency.
Genome biologySparse graphical Gaussian modeling of the isoprenoid gene network in Arabidopsis thaliana
390 Citations2004Anja Wille, Philip Zimmermann +11 more
A novel graphical Gaussian modeling approach for reverse engineering of genetic regulatory networks with many genes and few observations is presented and modules of closely connected genes and candidate genes for possible cross-talk between the isoprenoid pathways are detected.
Econometrics JournalAn overview of the estimation of large covariance and precision matrices
387 Citations2016Jianqing Fan, Yuan Liao +1 more
BiometrikaSparse estimation of a covariance matrix
364 Citations2011James P. Bien, Ryan J. Tibshirani
The proposed penalized maximum likelihood problem is not convex, so the method can be used to solve a previously studied special case in which a desired sparsity pattern is prespecified, and it uses a majorize-minimize approach in which it iteratively solve convex approximations to the original nonconvex problem.
Journal of Machine Learning ResearchHigh Dimensional Inverse Covariance Matrix Estimation via Linear Programming
350 Citations2010Ming Yuan
Journal of Computational and Graphical StatisticsNew Insights and Faster Computations for the Graphical Lasso
340 Citations2011Daniela Witten, Jerome H. Friedman +1 more
A very simple necessary and sufficient condition can be employed to determine whether the estimated inverse covariance matrix will be block diagonal, and if so, then to identify the blocks in the graphical lasso solution.
PLoS ONEWhen Is Hub Gene Selection Better than Standard Meta-Analysis?
336 Citations2013Peter Langfelder, Paul S. Mischel +1 more
It is demonstrated that intramodular hub gene status with respect to consensus modules is more useful than a meta- analysis p-value when identifying biologically meaningful gene lists, and standard meta-analysis methods perform as good as (if not better than) a consensus network approach in terms of validation success.
Annual Review of Statistics and Its ApplicationHigh-Dimensional Statistics with a View Toward Applications in Biology
278 Citations2014Peter Bühlmann, Markus Kalisch +1 more
This work reviews statistical methods for high-dimensional data analysis and pays particular attention to recent developments for assessing uncertainties in terms of controlling false positive statements (type I error) and p-values.
Bayesian AnalysisSimple Marginally Noninformative Prior Distributions for Covariance Matrices
255 Citations2013Alan Huang, M. P. Wand
BMC Systems BiologyA general co-expression network-based approach to gene expression analysis: comparison and applications
241 Citations2010Jianhua Ruan, Angela K Dean +1 more
It is demonstrated that the novel approach is very effective in discovering the modular structures in microarray data, both for genes and for samples, and may be applied to large data sets where the number of clusters is difficult to estimate.
BMC BioinformaticsRegularized estimation of large-scale gene association networks using graphical Gaussian models
239 Citations2009Nicole Krämer, Juliane Schäfer +1 more
A general framework for combining regularized regression methods with the estimation of Graphical Gaussian models is investigated, which includes various existing methods as well as two new approaches based on ridge regression and adaptive lasso, respectively.
Bayesian AnalysisBayesian Graphical Lasso Models and Efficient Posterior Computation
229 Citations2012Hao Wang
This paper introduces a fully Bayesian treatment of graphical lasso models, and indicates that the Bayesian adaptive graphical lasso appears to be the top overall performer among a range of frequentist and Bayesian methods.
Journal of Multivariate AnalysisSpectrum estimation: A unified framework for covariance matrix estimation and PCA in large dimensions
179 Citations2015Olivier Ledoit, Michael Wolf
Statistical ScienceHigh-Dimensional Inference: Confidence Intervals, $p$-Values and R-Software hdi
172 Citations2015Ruben Dezeure, Peter Bühlmann +2 more
A (selective) review of recent frequentist highdimensional inference methods for constructing p-values and confidence intervals in linear and generalized linear models and introduces the Rpackage hdi which easily allows the use of different methods and supports reproducibility.
PLoS ONEThe Molecular Genetic Architecture of Self-Employment
161 Citations2013Matthijs J. H. M. van der Loos, Cornelius A. Rietveld +66 more
The results suggest that common SNPs when considered jointly explain about half of the narrow-sense heritability of self-employment estimated in twin data, but a meta-analysis of genome-wide association studies across sixteen studies comprising 50,627 participants did not identify genome- wide significant SNPs.
QUIC: quadratic approximation for sparse inverse covariance estimation
158 Citations2014Cho‐Jui Hsieh, Mátyás A. Sustik +2 more
This work proposes a novel algorithm for solving the resulting optimization problem which is a regularized log-determinant program based on Newton's method and employs a quadratic approximation, but with some modifications that leverage the structure of the sparse Gaussian MLE problem.
Journal of the Royal Statistical Society Series B (Statistical Methodology)Condition-Number-Regularized Covariance Estimation
157 Citations2012Joong‐Ho Won, Johan Lim +2 more
This paper proposes a maximum likelihood approach, with the direct goal of obtaining a well-conditioned estimator, and investigates the theoretical properties of the regularized covariance estimator comprehensively, including its regularization path, and develops an approach that adaptively determines the level of regularization that is required.
Journal of the American Statistical AssociationPenalized Normal Likelihood and Ridge Regularization of Correlation and Covariance Matrices
153 Citations2008David I. Warton
Journal of Machine Learning ResearchHigh Dimensional Inverse Covariance Matrix Estimation via Linear Programming
140 Citations2010Yuan-ming
An estimating procedure is proposed that can effectively exploit "sparsity" of the inverse covariance matrix and can be computed using linear programming and therefore has the potential to be used in very high dimensional problems.
Bayesian AnalysisBayesian Structure Learning in Sparse Gaussian Graphical Models
137 Citations2015A. Mohammadi, E. C. Wit
A novel and efficient Bayesian framework for Gaussian graphical model determination which is a trans-dimensional Markov Chain Monte Carlo (MCMC) approach based on a continuous-time birth-death process and gives a principled and, in practice, sensible approach for structure learning.
Statistical ScienceMultiple Testing and Error Control in Gaussian Graphical Model Selection
134 Citations2007Mathias Drton, Michael D. Perlman
Gaussian graphical model selection can be performed by multiple testing of hypotheses about vanishing (partial) correlation coefficients associated with individual edges that are absent from the graph by demonstrating how this approach allows one to perform model selection while controlling error rates for incorrect edge inclusion.
arXiv (Cornell University)Exact covariance thresholding into connected components for large-scale Graphical Lasso
124 Citations2011Rahul Mazumder, Trevor Hastie
Journal of Statistical Planning and InferenceA SINful approach to Gaussian graphical model selection
116 Citations2007Mathias Drton, Michael D. Perlman
This work applies Fisher's z-transform, Sidak's correlation inequality, and Holm's step-down procedure to simultaneously test the multiple hypotheses specified by these zero values of corresponding partial correlation coefficients in multivariate Gaussian graphical models.
PubMedExact Covariance Thresholding into Connected Components for Large-Scale Graphical Lasso.
115 Citations2012Rahul Mazumder, Trevor Hastie
PLoS GeneticsAn Integrated Strategy for Analyzing the Unique Developmental Programs of Different Myoblast Subtypes
111 Citations2006Beatriz Estrada, Sung Choe +7 more
An integrated genetic, genomic, and computational strategy is used to comprehensively determine the molecular identities of distinct myoblast subpopulations within the Drosophila embryonic mesoderm at the time that cell fates are initially specified.
BiometricsJoint High‐Dimensional Bayesian Variable and Covariance Selection with an Application to eQTL Analysis
99 Citations2013Anindya Bhadra, Bani K. Mallick
A Bayesian technique to perform a sparse joint selection of significant predictor variables and significant inverse covariance matrix elements of the response variables in a high‐dimensional linear Gaussian sparse seemingly unrelated regression (SSUR) setting and perform an association analysis between the high-dimensional sets of predictors and responses in such a setting is described.
Electronic Journal of StatisticsTIGER: A tuning-insensitive approach for optimally estimating Gaussian graphical models
88 Citations2017Han Liu, Lie Wang
This work proposes a new procedure for estimating high dimensional Gaussian graphical models that is asymptotically tuning-free and non-asymptotic tuning-insensitive, and theoretically, the obtained estimator is simultaneously minimax optimal for precision matrix estimation under different norms.
Computational Statistics & Data AnalysisRidge estimation of inverse covariance matrices from high-dimensional data
86 Citations2016Wessel N. van Wieringen, Carel F.W. Peeters
The ridge estimation of the precision matrix is investigated in the setting where the number of variables is large relative to the sample size and the alternative estimators are compared to the archetypes with regard to eigenvalue shrinkage and risk.
Journal of Multivariate AnalysisFast and adaptive sparse precision matrix estimation in high dimensions
84 Citations2014Weidong Liu, Xi Luo
A novel approach to estimating sparse precision matrices in the high dimensional setting, called Sparse Column-wise Inverse Operator, which enjoys the advantage of fast computation for large-scale problems, via a coordinate descent algorithm.
Journal of Multivariate AnalysisEstimation of the precision matrix of a singular Wishart distribution and its application in high-dimensional data
68 Citations2008Tatsuya Kubokawa, Muni S. Srivastava
The Stein-Haff identity is established for a singular Wishart distribution with a positive definite mean matrix but with the dimension larger than the degrees of freedom and estimators of the precision matrix improving on the estimator based on the Moore-Penrose inverse of the Wishart matrix under the Efron-Morris loss function and its variants are obtained.
G3 Genes Genomes GeneticsGene Network Reconstruction by Integration of Prior Biological Knowledge
65 Citations2015Yupeng Li, Scott A. Jackson
A method for gene association network construction based on gene expression data that integrate a variety of biological resources, and the weighted graphical lasso (wglasso) outperforms the original glasso both on simulations and on data from Arabidopsis.
New PhytologistNorWood: a gene expression resource for evo‐devo studies of conifer wood development
60 Citations2017Soile Jokipii‐Lukkari, David Sundell +4 more
RNA‐sequencing data from replicated, high‐spatial‐resolution section series throughout the cambial and woody tissues of Picea abies were used to generate the NorWood.conGenIE.org web resource, which facilitates exploration of the associated gene expression profiles and co‐expression networks.
Computational Statistics & Data AnalysisNumerical implementation of the QuEST function
53 Citations2017Olivier Ledoit, Michael Wolf
Statistics and Its InterfaceThe Bayesian covariance lasso
47 Citations2013Haitao Chu, Joseph G. Ibrahim +3 more
A new method, called the Bayesian Covariance Lasso (BCLASSO), for the shrinkage estimation of a precision (covariance) matrix, which is permutation invariant and performs shrinkage and estimation simultaneously for non-full rank data.
Human HeredityBiophysically Motivated Regulatory Network Inference: Progress and Prospects
38 Citations2016Tarmo Äijö, Richard Bonneau
This perspective will focus on enumerating the elements of computational strategies that, when coupled to appropriate experimental designs, can lead to accurate large-scale models of chromatin-state and transcriptional regulatory structure and dynamics.
Journal of Statistical Planning and InferenceEstimation of covariance matrices based on hierarchical inverse-Wishart priors
35 Citations2012Mathilde Bouriga, Olivier Féron
Journal of Computational and Graphical StatisticsPenalized Covariance Matrix Estimation Using a Matrix-Logarithm Transformation
34 Citations2012Xinwei Deng, Kam‐Wah Tsui
The proposed method transforms the problem of estimating the covariance matrix into the problem of estimating a symmetric matrix, which can be solved efficiently by an iterative quadratic programming algorithm.
PLoS ONEUse of Wishart Prior and Simple Extensions for Sparse Precision Matrix Estimation
29 Citations2016Markku Kuismin, Mikko J. Sillanpää
An empirical Bayes procedure is used to select prior hyperparameters in high dimensional cases with extension to sparsity, and an interpretation of covariance estimates in terms of eigenvalues is presented to improve the performance of the estimation of sparse precision matrices and associated graphs.
Wiley Interdisciplinary Reviews Computational StatisticsEstimation of variances and covariances for high‐dimensional data: a selective review
28 Citations2014Tiejun Tong, Cheng Wang +1 more
Some recent developments in the estimation of variances, covariance matrix, and precision matrix are reviewed, with emphasis on the applications to microarray data analysis.
Journal of Computational and Graphical StatisticsPrecision Matrix Estimation With ROPE
24 Citations2017Markku Kuismin, Jukka Kemppainen +1 more
Journal of Statistical Computation and SimulationTuning-parameter selection in regularized estimations of large covariance matrices
22 Citations2015Yixin Fang, Binhuan Wang +1 more
It is demonstrated that if the estimation accuracy is measured in the Frobenius norm, the training set should consist of majority of the data; whereas if the estimator accuracy is measure in the operatornorm, the validation set should consists of majorityof the data.
BiometricsPartial correlation matrix estimation using ridge penalty followed by thresholding and re‐estimation
21 Citations2014Min Jin Ha, Wei Sun
Application on a yeast cell cycle gene expression data shows that the proposed statistical framework for estimating high‐dimensional partial correlation matrix delivers better predictions of the protein–protein interactions than the Graphic Lasso.
BMC Systems BiologyCluster and propensity based approximation of a network
6 Citations2013John Michael O. Rañola, Peter Langfelder +2 more
The CPBA of a network is theoretically appealing since a) it generalizes correlation and multigraph network methods, b) it improves likelihood based significance tests for edge counts, c) it directly models higher-order relationships between clusters, and d) it suggests novel clustering algorithms.
Journal of Computational and Graphical StatisticsSparse Steinian Covariance Estimation
4 Citations2016Brett Naul, Jonathan Taylor
This paper extends the UBEOR minimization problem by adding an ℓ1 penalty on the entries of the estimated covariance matrix, which encourages a sparse estimate and demonstrates via numerical simulations and an analysis of microarray data from breast cancer patients that the proposed method generally outperforms other methods for sparse covariance Matrix estimation and can be computed efficiently even in high dimensions.