AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
We investigate a curious relationship between the structure of a discrete graphical model and the support of the inverse of a generalized covariance matrix

Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses

ANNALS OF STATISTICS, no. 6 (2013): 3022-3049

Cited by: 179|Views235
EI

Abstract

We investigate the relationship between the structure of a discrete graphical model and the support of the inverse of a generalized covariance matrix. We show that for certain graph structures, the support of the inverse covariance matrix of indicator variables on the vertices of a graph reflects the conditional independence structure of ...More

Code:

Data:

0
Introduction
  • Graphical model inference is prevalent in many fields, running the gamut from computer vision and civil engineering to political science and epidemiology.
  • The authors' main result has a striking corollary in the context of tree-structured graphs: for any discrete graphical model, the inverse of a generalized covariance matrix is always graph-structured.
  • For binary variables, the inverse of the usual covariance matrix corresponds exactly to the edge structure of the tree.
Highlights
  • Graphical model inference is prevalent in many fields, running the gamut from computer vision and civil engineering to political science and epidemiology
  • It is well known that zeros in the inverse covariance matrix of a multivariate Gaussian distribution indicate the absence of an edge in the corresponding graphical model
  • For a multivariate Gaussian graphical model defined on G, standard theory predicts that the inverse covariance matrix Γ = Σ−1 of the distribution is graph-structured: Γst = 0 if and only if (s, t) ∈/ E
  • In settings where there exists a junction tree representation of the graph with only singleton separator sets, Corollary 2 has a number of useful implications for the consistency of methods that have traditionally only been applied for edge recovery in Gaussian graphical models
  • We have shown that it is worthwhile to consider the inverses of generalized covariance matrices, formed by introducing indicator functions for larger subsets of variables. When these subsets are chosen to reflect the structure of an underlying junction tree, the edge structure is reflected in the inverse covariance matrix
  • We have shown how our results may be used to establish consistency of the standard graphical Lasso applied to discrete graphs, even when observations are systematically corrupted by mechanisms such as additive noise and missing data
Results
  • In Section 3, the authors state the main results on the relationship between the support of generalized inverse covariance matrices and the edge structure of a discrete graphical model.
  • For a multivariate Gaussian graphical model defined on G, standard theory predicts that the inverse covariance matrix Γ = Σ−1 of the distribution is graph-structured: Γst = 0 if and only if (s, t) ∈/ E.
  • [Triangulation and block graph-structure.] Consider an arbitrary discrete graphical model of the form (4), and let T be the set of maximal cliques in any triangulation of G.
  • Theorem 1 implies that the inverse Γ of the augmented covariance matrix with sufficient statistics for all vertices and edges is graph-structured, and blocks of nonzeros in Γ correspond to edges in the graph.
  • In panel (c), the single separator set in the triangulation is {1, 3}, so augmenting the usual covariance matrix with the additional sufficient statistic X1X3 and taking the inverse should yield a graph-structured matrix.
  • Theorem 1 and Corollary 1 are clean results at the population level, forming the proper augmented covariance matrix requires some prior knowledge of the graph—namely, which edges are involved in a suitable triangulation.
  • In settings where there exists a junction tree representation of the graph with only singleton separator sets, Corollary 2 has a number of useful implications for the consistency of methods that have traditionally only been applied for edge recovery in Gaussian graphical models.
Conclusion
  • The authors describe how a combination of the population-level results and some concentration inequalities may be leveraged to analyze the statistical behavior of log-determinant methods for discrete tree-structured graphical models, and suggest extensions of these methods when observations are systematically corrupted by noise or missing data.
  • The correspondence between the inverse covariance matrix and graph structure of a Gauss-Markov random field is a classical fact, with many useful consequences for efficient estimation of Gaussian graphical models.
  • The authors have shown how the results may be used to establish consistency of the standard graphical Lasso applied to discrete graphs, even when observations are systematically corrupted by mechanisms such as additive noise and missing data.
Funding
  • PL acknowledges support from a Hertz Foundation Fellowship and an NDSEG Fellowship
  • MJW and PL were also partially supported by grants NSF-DMS-0907632 and AFOSR-09NL184
Reference
  • M.E.J. Newman and D.J. Watts. Scaling and percolation in the small-world network model. Phys. Rev. E, 60(6):7332–7342, December 1999.
    Google ScholarLocate open access versionFindings
  • T. Cai, W. Liu, and X. Luo. A constrained 1 minimization approach to sparse precision matrix estimation. Journal of the American Statistical Association, 106:594–607, 2011.
    Google ScholarLocate open access versionFindings
  • N. Meinshausen and P. Buhlmann. High-dimensional graphs and variable selection with the Lasso. Annals of Statistics, 34:1436–1462, 2006.
    Google ScholarLocate open access versionFindings
  • P. Ravikumar, M. J. Wainwright, G. Raskutti, and B. Yu. High-dimensional covariance estimation by minimizing 1-penalized log-determinant divergence. Electronic Journal of Statistics, 4:935–980, 2011.
    Google ScholarLocate open access versionFindings
  • M. Yuan. High-dimensional inverse covariance matrix estimation via linear programming. Journal of Machine Learning Research, 99:2261–2286, August 2010.
    Google ScholarLocate open access versionFindings
  • H. Liu, F. Han, M. Yuan, J.D. Lafferty, and L.A. Wasserman. High dimensional semiparametric Gaussian copula graphical models. arXiv e-prints, March 2012. Available at http://arxiv.org/abs/1202.2169.
    Findings
  • H. Liu, J.D. Lafferty, and L.A. Wasserman. The nonparanormal: Semiparametric estimation of high dimensional undirected graphs. Journal of Machine Learning Research, 10:2295–2328, 2009.
    Google ScholarLocate open access versionFindings
  • C.I. Chow and C.N. Liu. Approximating discrete probability distributions with dependence trees. IEEE Transactions on Information Theory, 14:462–467, 1968.
    Google ScholarLocate open access versionFindings
  • A. Jalali, P.D. Ravikumar, V. Vasuki, and S. Sanghavi. On learning discrete graphical models using group-sparse regularization. Journal of Machine Learning Research - Proceedings Track, 15:378–387, 2011.
    Google ScholarLocate open access versionFindings
  • P. Ravikumar, M.J. Wainwright, and J.D. Lafferty. High-dimensional Ising model selection using 1-regularized logistic regression. Annals of Statistics, 38:1287, 2010.
    Google ScholarLocate open access versionFindings
  • A. Anandkumar, V.Y.F. Tan, and A.S. Willsky. High-dimensional structure learning of Ising models: Local separation criterion. Annals of Statistics, 40(3):1346–1375, 2012.
    Google ScholarLocate open access versionFindings
  • G. Bresler, E. Mossel, and A. Sly. Reconstruction of markov random fields from samples: Some observations and algorithms. In APPROX-RANDOM, pages 343–356, 2008.
    Google ScholarLocate open access versionFindings
  • P. Loh and M.J. Wainwright. Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses. arXiv e-prints, November 2012.
    Google ScholarFindings
  • S.L. Lauritzen. Graphical Models. Oxford University Press, 1996.
    Google ScholarFindings
  • M. J. Wainwright and M. I. Jordan. Graphical models, exponential families, and variational inference. Found. Trends Mach. Learn., 1(1-2):1–305, January 2008.
    Google ScholarLocate open access versionFindings
  • R. T. Rockafellar. Convex Analysis. Princeton University Press, Princeton, 1970.
    Google ScholarFindings
  • R. A. Horn and C. R. Johnson. Matrix Analysis. Cambridge University Press, 1990.
    Google ScholarFindings
  • O. Banerjee, L. El Ghaoui, and A. d’Aspremont. Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. Journal of Machine Learning Research, 9:485–516, 2008.
    Google ScholarLocate open access versionFindings
  • J. Friedman, T. Hastie, and R. Tibshirani. Sparse inverse covariance estimation with the graphical Lasso. Biostatistics, 9(3):432–441, July 2008.
    Google ScholarLocate open access versionFindings
  • M. Yuan and Y. Lin. Model selection and estimation in the Gaussian graphical model. Biometrika, 94(1):19–35, 2007.
    Google ScholarLocate open access versionFindings
  • Narayana P. Santhanam and Martin J. Wainwright. Information-theoretic limits of selecting binary graphical models in high dimensions. IEEE Transactions on Information Theory, 58(7):4117–4134, 2012.
    Google ScholarLocate open access versionFindings
  • P. Loh and M.J. Wainwright. High-dimensional regression with noisy and missing data: Provable guarantees with non-convexity. Annals of Statistics, 40(3):1637–1664, 2012.
    Google ScholarLocate open access versionFindings
  • L. Jacob, G. Obozinski, and J. P. Vert. Group Lasso with Overlap and Graph Lasso. In International Conference on Machine Learning (ICML), pages 433–440, 2009.
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科