AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
We provide a simple characterization of it for tree structured graphs, and show how it can be used for approximations in non-tree graphs

An LP View of the M-best MAP problem.

NIPS, pp.567-575, (2009)

Cited by: 82|Views249
EI
Full Text
Bibtex
Weibo

Abstract

We consider the problem of finding the M assignments with maximum probability in a probabilistic graphical model. We show how this problem can be formulated as a linear program (LP) on a particular polytope. We prove that, for tree graphs (and junction trees in general), this polytope has a particularly simple form and differs from the ma...More

Code:

Data:

0
Introduction
  • A common task in probabilistic modeling is finding the assignment with maximum probability given a model.
  • Of particular interest is the case of MAP in graphical models, i.e., models where the probability factors into a product over small subsets of variables
  • For general models, this is an NP-hard problem [11], and approximation algorithms are required.
  • One possible approach is to use loopy max-product to obtain approximate max-marginals and use those to approximate the M best solutions [19]
  • This is largely a heuristic and does not provide any guarantees in terms of optimality certificates or bounds on the optimal values.
Highlights
  • A common task in probabilistic modeling is finding the assignment with maximum probability given a model
  • The goal of the current work is to leverage the power of linear program (LP) relaxations to the M best case
  • We begin by focusing on the problem of finding the second best solution. We show how it can be formulated as an LP over a polytope we call the “assignment-excluding marginal polytope”
  • We approximated each MAP with its LP relaxation, so that both STRIPES and Nilsson come with certificates of optimality when their LP solutions are integral
  • We provide a simple characterization of it for tree structured graphs, and show how it can be used for approximations in non-tree graphs
Methods
  • The authors compared the performance of STRIPES to the BMMF algorithm [19] and the Lawler/Nilsson algorithm [6, 8].
  • Nilsson’s algorithm is equivalent to PES where the 2nd best assignment is obtained from maximizations within O(n) partitions, so that its runtime is O(n) times the cost of finding a single MAP.
  • The authors approximated each MAP with its LP relaxation, so that both STRIPES and Nilsson come with certificates of optimality when their LP solutions are integral.
  • BMMF relies on loopy BP to approximate the M best solutions.7.
  • For run-time comparisons, the authors normalized the times by the longest-running algorithm for each example
Conclusion
  • The authors present a novel combinatorial object M (G, z) and show its utility in obtaining the M best MAP assignments.
  • As with the marginal polytope, many interesting questions arise about the properties of M (G, z).
  • In which non-tree cases can the authors provide a compact characterization.
  • Another compelling question is in which problems the spanning tree inequalities are provably optimal.
  • An interesting generalization of the method is to predict diverse solutions satisfying some local measure of “distance” from each other, e.g., as in [2]
Reference
  • F. Barahona. On cuts and matchings in planar graphs. Math. Program., 60(1):53–68, 1993.
    Google ScholarLocate open access versionFindings
  • M. Fromer and C. Yanover. Accurate prediction for atomic-level protein design and its application in diversifying the near-optimal sequence space. Proteins: Structure, Function, and Bioinformatics, 75:682–705, 2009.
    Google ScholarLocate open access versionFindings
  • A. Globerson and T. Jaakkola. Fixing max-product: Convergent message passing algorithms for MAP LP-relaxations. In J. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 21. MIT Press, Cambridge, MA, 2007.
    Google ScholarLocate open access versionFindings
  • E. Kloppmann, G. M. Ullmann, and T. Becker. An extended dead-end elimination algorithm to determine gap-free lists of low energy states. Journal of Comp. Chem., 28:2325–2335, 2007.
    Google ScholarLocate open access versionFindings
  • N. Komodakis and N. Paragios. Beyond loose LP-relaxations: Optimizing MRFs by repairing cycles. In D. Forsyth, P. Torr, and A. Zisserman, editors, ECCV, pages 806–820, Heidelberg, Germany, 2008. Springer.
    Google ScholarFindings
  • E. L. Lawler. A procedure for computing the K best solutions to discrete optimization problems and its application to the shortest path problem. Management Science, 18(7):401–405, 1972.
    Google ScholarLocate open access versionFindings
  • I. Ljubic, R. Weiskircher, U. Pferschy, G. W. Klau, P. Mutzel, and M. Fischetti. An algorithmic framework for the exact solution of the prize-collecting steiner tree problem. Mathematical Programming, 105:427–449, Feb 2006.
    Google ScholarLocate open access versionFindings
  • D. Nilsson. An efficient algorithm for finding the M most probable configurations in probabilistic expert systems. Statistics and Computing, 8:159–173, Jun 1998.
    Google ScholarLocate open access versionFindings
  • P. Ravikumar, A. Agarwal, and M. Wainwright. Message-passing for graph-structured linear programs: proximal projections, convergence and rounding schemes. In Proc. of the 25th international conference on Machine learning, pages 800–807, New York, NY, USA, 2008. ACM.
    Google ScholarLocate open access versionFindings
  • E. Santos. On the generation of alternative explanations with implications for belief revision. In Proc. of the 7th Annual Conference on Uncertainty in Artificial Intelligence, 1991.
    Google ScholarLocate open access versionFindings
  • Y. Shimony. Finding the MAPs for belief networks is NP-hard. Aritifical Intelligence, 68(2):399–410, 1994.
    Google ScholarLocate open access versionFindings
  • D. Sontag and T. Jaakkola. New outer bounds on the marginal polytope. In J. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 20, pages 1393–1400. MIT Press, Cambridge, MA, 2007.
    Google ScholarLocate open access versionFindings
  • D. Sontag, T. Meltzer, A. Globerson, T. Jaakkola, and Y. Weiss. Tightening LP relaxations for MAP using message passing. In Proc. of the 24th Annual Conference on Uncertainty in Artificial Intelligence, pages 503–510, 2008.
    Google ScholarLocate open access versionFindings
  • B. Taskar, S. Lacoste-Julien, and M. I. Jordan. Structured prediction, dual extragradient and bregman projections. J. Mach. Learn. Res., 7:1627–1653, 2006.
    Google ScholarLocate open access versionFindings
  • M. Wainwright and M. Jordan. Graphical models, exponential families, and variational inference. Found. Trends Mach. Learn., 1(1-2):1–305, 2008.
    Google ScholarLocate open access versionFindings
  • M. J. Wainwright, T. Jaakkola, and A. S. Willsky. A new class of upper bounds on the log partition function. IEEE Transactions on Information Theory, 51(7):2313–2335, 2005.
    Google ScholarLocate open access versionFindings
  • T. Werner. A linear programming approach to max-sum problem: A review. IEEE Trans. Pattern Anal. Mach. Intell., 29(7):1165–1179, 2007.
    Google ScholarLocate open access versionFindings
  • C. Yanover, T. Meltzer, and Y. Weiss. Linear programming relaxations and belief propagation – an empirical study. Journal of Machine Learning Research, 7:1887–1907, 2006.
    Google ScholarLocate open access versionFindings
  • C. Yanover and Y. Weiss. Finding the M most probable configurations using loopy belief propagation. In Advances in Neural Information Processing Systems 16. MIT Press, Cambridge, MA, 2004.
    Google ScholarLocate open access versionFindings
  • J. Yedidia, W. W.T. Freeman, and Y. Weiss. Constructing free-energy approximations and generalized belief propagation algorithms. IEEE Trans. on Information Theory, 51(7):2282– 2312, 2005.
    Google ScholarLocate open access versionFindings
Author
Amir Globerson
Amir Globerson
Your rating :
0

 

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科