## AI helps you reading Science

## AI Insight

AI extracts a summary of this paper

Weibo:

# Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses

ANNALS OF STATISTICS, no. 6 (2013): 3022-3049

EI

Keywords

Abstract

We investigate the relationship between the structure of a discrete graphical model and the support of the inverse of a generalized covariance matrix. We show that for certain graph structures, the support of the inverse covariance matrix of indicator variables on the vertices of a graph reflects the conditional independence structure of ...More

Code:

Data:

Introduction

- Graphical model inference is prevalent in many fields, running the gamut from computer vision and civil engineering to political science and epidemiology.
- The authors' main result has a striking corollary in the context of tree-structured graphs: for any discrete graphical model, the inverse of a generalized covariance matrix is always graph-structured.
- For binary variables, the inverse of the usual covariance matrix corresponds exactly to the edge structure of the tree.

Highlights

- Graphical model inference is prevalent in many fields, running the gamut from computer vision and civil engineering to political science and epidemiology
- It is well known that zeros in the inverse covariance matrix of a multivariate Gaussian distribution indicate the absence of an edge in the corresponding graphical model
- For a multivariate Gaussian graphical model defined on G, standard theory predicts that the inverse covariance matrix Γ = Σ−1 of the distribution is graph-structured: Γst = 0 if and only if (s, t) ∈/ E
- In settings where there exists a junction tree representation of the graph with only singleton separator sets, Corollary 2 has a number of useful implications for the consistency of methods that have traditionally only been applied for edge recovery in Gaussian graphical models
- We have shown that it is worthwhile to consider the inverses of generalized covariance matrices, formed by introducing indicator functions for larger subsets of variables. When these subsets are chosen to reflect the structure of an underlying junction tree, the edge structure is reflected in the inverse covariance matrix
- We have shown how our results may be used to establish consistency of the standard graphical Lasso applied to discrete graphs, even when observations are systematically corrupted by mechanisms such as additive noise and missing data

Results

- In Section 3, the authors state the main results on the relationship between the support of generalized inverse covariance matrices and the edge structure of a discrete graphical model.
- For a multivariate Gaussian graphical model defined on G, standard theory predicts that the inverse covariance matrix Γ = Σ−1 of the distribution is graph-structured: Γst = 0 if and only if (s, t) ∈/ E.
- [Triangulation and block graph-structure.] Consider an arbitrary discrete graphical model of the form (4), and let T be the set of maximal cliques in any triangulation of G.
- Theorem 1 implies that the inverse Γ of the augmented covariance matrix with sufficient statistics for all vertices and edges is graph-structured, and blocks of nonzeros in Γ correspond to edges in the graph.
- In panel (c), the single separator set in the triangulation is {1, 3}, so augmenting the usual covariance matrix with the additional sufficient statistic X1X3 and taking the inverse should yield a graph-structured matrix.
- Theorem 1 and Corollary 1 are clean results at the population level, forming the proper augmented covariance matrix requires some prior knowledge of the graph—namely, which edges are involved in a suitable triangulation.
- In settings where there exists a junction tree representation of the graph with only singleton separator sets, Corollary 2 has a number of useful implications for the consistency of methods that have traditionally only been applied for edge recovery in Gaussian graphical models.

Conclusion

- The authors describe how a combination of the population-level results and some concentration inequalities may be leveraged to analyze the statistical behavior of log-determinant methods for discrete tree-structured graphical models, and suggest extensions of these methods when observations are systematically corrupted by noise or missing data.
- The correspondence between the inverse covariance matrix and graph structure of a Gauss-Markov random field is a classical fact, with many useful consequences for efficient estimation of Gaussian graphical models.
- The authors have shown how the results may be used to establish consistency of the standard graphical Lasso applied to discrete graphs, even when observations are systematically corrupted by mechanisms such as additive noise and missing data.

Funding

- PL acknowledges support from a Hertz Foundation Fellowship and an NDSEG Fellowship
- MJW and PL were also partially supported by grants NSF-DMS-0907632 and AFOSR-09NL184

Reference

- M.E.J. Newman and D.J. Watts. Scaling and percolation in the small-world network model. Phys. Rev. E, 60(6):7332–7342, December 1999.
- T. Cai, W. Liu, and X. Luo. A constrained 1 minimization approach to sparse precision matrix estimation. Journal of the American Statistical Association, 106:594–607, 2011.
- N. Meinshausen and P. Buhlmann. High-dimensional graphs and variable selection with the Lasso. Annals of Statistics, 34:1436–1462, 2006.
- P. Ravikumar, M. J. Wainwright, G. Raskutti, and B. Yu. High-dimensional covariance estimation by minimizing 1-penalized log-determinant divergence. Electronic Journal of Statistics, 4:935–980, 2011.
- M. Yuan. High-dimensional inverse covariance matrix estimation via linear programming. Journal of Machine Learning Research, 99:2261–2286, August 2010.
- H. Liu, F. Han, M. Yuan, J.D. Lafferty, and L.A. Wasserman. High dimensional semiparametric Gaussian copula graphical models. arXiv e-prints, March 2012. Available at http://arxiv.org/abs/1202.2169.
- H. Liu, J.D. Lafferty, and L.A. Wasserman. The nonparanormal: Semiparametric estimation of high dimensional undirected graphs. Journal of Machine Learning Research, 10:2295–2328, 2009.
- C.I. Chow and C.N. Liu. Approximating discrete probability distributions with dependence trees. IEEE Transactions on Information Theory, 14:462–467, 1968.
- A. Jalali, P.D. Ravikumar, V. Vasuki, and S. Sanghavi. On learning discrete graphical models using group-sparse regularization. Journal of Machine Learning Research - Proceedings Track, 15:378–387, 2011.
- P. Ravikumar, M.J. Wainwright, and J.D. Lafferty. High-dimensional Ising model selection using 1-regularized logistic regression. Annals of Statistics, 38:1287, 2010.
- A. Anandkumar, V.Y.F. Tan, and A.S. Willsky. High-dimensional structure learning of Ising models: Local separation criterion. Annals of Statistics, 40(3):1346–1375, 2012.
- G. Bresler, E. Mossel, and A. Sly. Reconstruction of markov random fields from samples: Some observations and algorithms. In APPROX-RANDOM, pages 343–356, 2008.
- P. Loh and M.J. Wainwright. Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses. arXiv e-prints, November 2012.
- S.L. Lauritzen. Graphical Models. Oxford University Press, 1996.
- M. J. Wainwright and M. I. Jordan. Graphical models, exponential families, and variational inference. Found. Trends Mach. Learn., 1(1-2):1–305, January 2008.
- R. T. Rockafellar. Convex Analysis. Princeton University Press, Princeton, 1970.
- R. A. Horn and C. R. Johnson. Matrix Analysis. Cambridge University Press, 1990.
- O. Banerjee, L. El Ghaoui, and A. d’Aspremont. Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. Journal of Machine Learning Research, 9:485–516, 2008.
- J. Friedman, T. Hastie, and R. Tibshirani. Sparse inverse covariance estimation with the graphical Lasso. Biostatistics, 9(3):432–441, July 2008.
- M. Yuan and Y. Lin. Model selection and estimation in the Gaussian graphical model. Biometrika, 94(1):19–35, 2007.
- Narayana P. Santhanam and Martin J. Wainwright. Information-theoretic limits of selecting binary graphical models in high dimensions. IEEE Transactions on Information Theory, 58(7):4117–4134, 2012.
- P. Loh and M.J. Wainwright. High-dimensional regression with noisy and missing data: Provable guarantees with non-convexity. Annals of Statistics, 40(3):1637–1664, 2012.
- L. Jacob, G. Obozinski, and J. P. Vert. Group Lasso with Overlap and Graph Lasso. In International Conference on Machine Learning (ICML), pages 433–440, 2009.

Tags

Comments

数据免责声明

页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果，我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问，可以通过电子邮件方式联系我们：report@aminer.cn