Estimating Conditional Probabilities by Mixtures of Low Order Conditional Distributions.

Estimating probabilities of a multinomial variable conditioned to a large set of variables is an important problem due to the fact that the number of parameters increases in an exponential way with the number of conditional variables. Some models, such as noisy-or gates make assumptions about the relationships between the variables that assume that the number of parameters is linear. However, there are cases in which these hypothesis do not make sense. In this paper, we present a procedure to estimate a large conditional probability distribution by means of an average of low order conditional probabilities. In this way the number of necessary parameters can be reduced to a quantity which can be estimated with available data. Different experiments show that the quality of the estimations can be improved with respect to a direct estimation.