## AI helps you reading Science

## AI Insight

AI extracts a summary of this paper

Weibo:

# An LP View of the M-best MAP problem.

NIPS, pp.567-575, (2009)

EI

Abstract

We consider the problem of finding the M assignments with maximum probability in a probabilistic graphical model. We show how this problem can be formulated as a linear program (LP) on a particular polytope. We prove that, for tree graphs (and junction trees in general), this polytope has a particularly simple form and differs from the ma...More

Code:

Data:

Introduction

- A common task in probabilistic modeling is finding the assignment with maximum probability given a model.
- Of particular interest is the case of MAP in graphical models, i.e., models where the probability factors into a product over small subsets of variables
- For general models, this is an NP-hard problem [11], and approximation algorithms are required.
- One possible approach is to use loopy max-product to obtain approximate max-marginals and use those to approximate the M best solutions [19]
- This is largely a heuristic and does not provide any guarantees in terms of optimality certificates or bounds on the optimal values.

Highlights

- A common task in probabilistic modeling is finding the assignment with maximum probability given a model
- The goal of the current work is to leverage the power of linear program (LP) relaxations to the M best case
- We begin by focusing on the problem of finding the second best solution. We show how it can be formulated as an LP over a polytope we call the “assignment-excluding marginal polytope”
- We approximated each MAP with its LP relaxation, so that both STRIPES and Nilsson come with certificates of optimality when their LP solutions are integral
- We provide a simple characterization of it for tree structured graphs, and show how it can be used for approximations in non-tree graphs

Methods

- The authors compared the performance of STRIPES to the BMMF algorithm [19] and the Lawler/Nilsson algorithm [6, 8].
- Nilsson’s algorithm is equivalent to PES where the 2nd best assignment is obtained from maximizations within O(n) partitions, so that its runtime is O(n) times the cost of finding a single MAP.
- The authors approximated each MAP with its LP relaxation, so that both STRIPES and Nilsson come with certificates of optimality when their LP solutions are integral.
- BMMF relies on loopy BP to approximate the M best solutions.7.
- For run-time comparisons, the authors normalized the times by the longest-running algorithm for each example

Conclusion

- The authors present a novel combinatorial object M (G, z) and show its utility in obtaining the M best MAP assignments.
- As with the marginal polytope, many interesting questions arise about the properties of M (G, z).
- In which non-tree cases can the authors provide a compact characterization.
- Another compelling question is in which problems the spanning tree inequalities are provably optimal.
- An interesting generalization of the method is to predict diverse solutions satisfying some local measure of “distance” from each other, e.g., as in [2]

Reference

- F. Barahona. On cuts and matchings in planar graphs. Math. Program., 60(1):53–68, 1993.
- M. Fromer and C. Yanover. Accurate prediction for atomic-level protein design and its application in diversifying the near-optimal sequence space. Proteins: Structure, Function, and Bioinformatics, 75:682–705, 2009.
- A. Globerson and T. Jaakkola. Fixing max-product: Convergent message passing algorithms for MAP LP-relaxations. In J. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 21. MIT Press, Cambridge, MA, 2007.
- E. Kloppmann, G. M. Ullmann, and T. Becker. An extended dead-end elimination algorithm to determine gap-free lists of low energy states. Journal of Comp. Chem., 28:2325–2335, 2007.
- N. Komodakis and N. Paragios. Beyond loose LP-relaxations: Optimizing MRFs by repairing cycles. In D. Forsyth, P. Torr, and A. Zisserman, editors, ECCV, pages 806–820, Heidelberg, Germany, 2008. Springer.
- E. L. Lawler. A procedure for computing the K best solutions to discrete optimization problems and its application to the shortest path problem. Management Science, 18(7):401–405, 1972.
- I. Ljubic, R. Weiskircher, U. Pferschy, G. W. Klau, P. Mutzel, and M. Fischetti. An algorithmic framework for the exact solution of the prize-collecting steiner tree problem. Mathematical Programming, 105:427–449, Feb 2006.
- D. Nilsson. An efficient algorithm for finding the M most probable configurations in probabilistic expert systems. Statistics and Computing, 8:159–173, Jun 1998.
- P. Ravikumar, A. Agarwal, and M. Wainwright. Message-passing for graph-structured linear programs: proximal projections, convergence and rounding schemes. In Proc. of the 25th international conference on Machine learning, pages 800–807, New York, NY, USA, 2008. ACM.
- E. Santos. On the generation of alternative explanations with implications for belief revision. In Proc. of the 7th Annual Conference on Uncertainty in Artificial Intelligence, 1991.
- Y. Shimony. Finding the MAPs for belief networks is NP-hard. Aritifical Intelligence, 68(2):399–410, 1994.
- D. Sontag and T. Jaakkola. New outer bounds on the marginal polytope. In J. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 20, pages 1393–1400. MIT Press, Cambridge, MA, 2007.
- D. Sontag, T. Meltzer, A. Globerson, T. Jaakkola, and Y. Weiss. Tightening LP relaxations for MAP using message passing. In Proc. of the 24th Annual Conference on Uncertainty in Artificial Intelligence, pages 503–510, 2008.
- B. Taskar, S. Lacoste-Julien, and M. I. Jordan. Structured prediction, dual extragradient and bregman projections. J. Mach. Learn. Res., 7:1627–1653, 2006.
- M. Wainwright and M. Jordan. Graphical models, exponential families, and variational inference. Found. Trends Mach. Learn., 1(1-2):1–305, 2008.
- M. J. Wainwright, T. Jaakkola, and A. S. Willsky. A new class of upper bounds on the log partition function. IEEE Transactions on Information Theory, 51(7):2313–2335, 2005.
- T. Werner. A linear programming approach to max-sum problem: A review. IEEE Trans. Pattern Anal. Mach. Intell., 29(7):1165–1179, 2007.
- C. Yanover, T. Meltzer, and Y. Weiss. Linear programming relaxations and belief propagation – an empirical study. Journal of Machine Learning Research, 7:1887–1907, 2006.
- C. Yanover and Y. Weiss. Finding the M most probable configurations using loopy belief propagation. In Advances in Neural Information Processing Systems 16. MIT Press, Cambridge, MA, 2004.
- J. Yedidia, W. W.T. Freeman, and Y. Weiss. Constructing free-energy approximations and generalized belief propagation algorithms. IEEE Trans. on Information Theory, 51(7):2282– 2312, 2005.

Tags

Comments

数据免责声明

页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果，我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问，可以通过电子邮件方式联系我们：report@aminer.cn