Probabilistic Inference

The probabilistic inference algorithm used by py-bbn is an exact inference algorithm. Let’s go through an example on how to conduct exact inference.

Huang Graph

Below is the code to create the Huang Graph [HD99]. Note the typical procedure as follows.

• create a Bayesian Belief Network (BBN)

• create a junction tree from the graph

• assert evidence

• print out the marginal probabilities

digraph { node [fixedsize=true, width=0.3, shape=circle, fontname="Helvetica-Outline", color=crimson, style=filled] A -> B A -> C B -> D C -> E D -> F E -> F C -> G E -> H G -> H }

Huang Bayesian Belief Network structure.

from pybbn.graph.dag import Bbn
from pybbn.graph.edge import Edge, EdgeType
from pybbn.graph.jointree import EvidenceBuilder
from pybbn.graph.node import BbnNode
from pybbn.graph.variable import Variable
from pybbn.pptc.inferencecontroller import InferenceController

# create the nodes
a = BbnNode(Variable(0, 'a', ['on', 'off']), [0.5, 0.5])
b = BbnNode(Variable(1, 'b', ['on', 'off']), [0.5, 0.5, 0.4, 0.6])
c = BbnNode(Variable(2, 'c', ['on', 'off']), [0.7, 0.3, 0.2, 0.8])
d = BbnNode(Variable(3, 'd', ['on', 'off']), [0.9, 0.1, 0.5, 0.5])
e = BbnNode(Variable(4, 'e', ['on', 'off']), [0.3, 0.7, 0.6, 0.4])
f = BbnNode(Variable(5, 'f', ['on', 'off']), [0.01, 0.99, 0.01, 0.99, 0.01, 0.99, 0.99, 0.01])
g = BbnNode(Variable(6, 'g', ['on', 'off']), [0.8, 0.2, 0.1, 0.9])
h = BbnNode(Variable(7, 'h', ['on', 'off']), [0.05, 0.95, 0.95, 0.05, 0.95, 0.05, 0.95, 0.05])

# create the network structure
bbn = Bbn() \
    .add_node(a) \
    .add_node(b) \
    .add_node(c) \
    .add_node(d) \
    .add_node(e) \
    .add_node(f) \
    .add_node(g) \
    .add_node(h) \
    .add_edge(Edge(a, b, EdgeType.DIRECTED)) \
    .add_edge(Edge(a, c, EdgeType.DIRECTED)) \
    .add_edge(Edge(b, d, EdgeType.DIRECTED)) \
    .add_edge(Edge(c, e, EdgeType.DIRECTED)) \
    .add_edge(Edge(d, f, EdgeType.DIRECTED)) \
    .add_edge(Edge(e, f, EdgeType.DIRECTED)) \
    .add_edge(Edge(c, g, EdgeType.DIRECTED)) \
    .add_edge(Edge(e, h, EdgeType.DIRECTED)) \
    .add_edge(Edge(g, h, EdgeType.DIRECTED))

# convert the BBN to a join tree
join_tree = InferenceController.apply(bbn)

# insert an observation evidence
ev = EvidenceBuilder() \
    .with_node(join_tree.get_bbn_node_by_name('a')) \
    .with_evidence('on', 1.0) \
    .build()
join_tree.set_observation(ev)

# print the posterior probabilities
for node, posteriors in join_tree.get_posteriors().items():
    p = ', '.join([f'{val}={prob:.5f}' for val, prob in posteriors.items()])
    print(f'{node} : {p}')

A Bayesian Belief Network (BBN) is defined as a pair, G, P, where

• G is a directed acylic graph (DAG)

• P is a joint probability distribution

• and G satisfies the Markov Condition (nodes are conditionally independent of non-descendants given its parents)

Ideally, the API should force the user to define G and P separately. However, there will be a bit of cognitive friction with this API as we define nodes associated with their local probability models (conditional probability tables) and then the structure afterwards. But this approach seems a bit more concise, no?

Updating Conditional Probability Tables

Sometimes, you may want to preserve the join tree structure and just update the condtional probability tables (CPTs). Here’s how to do so.

from pybbn.graph.dag import Bbn
from pybbn.graph.edge import EdgeType, Edge
from pybbn.graph.node import BbnNode
from pybbn.graph.variable import Variable
from pybbn.pptc.inferencecontroller import InferenceController

# you have built a BBN
a = BbnNode(Variable(0, 'a', ['t', 'f']), [0.2, 0.8])
b = BbnNode(Variable(1, 'b', ['t', 'f']), [0.1, 0.9, 0.9, 0.1])
bbn = Bbn().add_node(a).add_node(b) \
    .add_edge(Edge(a, b, EdgeType.DIRECTED))

# you have built a junction tree from the BBN
# let's call this "original" junction tree the left-hand side (lhs) junction tree
lhs_jt = InferenceController.apply(bbn)

# you may just update the CPTs with the original junction tree structure
# the algorithm to find/build the junction tree is avoided
# the CPTs are updated
rhs_jt = InferenceController.reapply(lhs_jt, {0: [0.3, 0.7], 1: [0.2, 0.8, 0.8, 0.2]})

# let's print out the marginal probabilities and see how things changed
# print the marginal probabilities for the lhs junction tree
print('lhs probabilities')
# print the posterior probabilities
for node, posteriors in lhs_jt.get_posteriors().items():
    p = ', '.join([f'{val}={prob:.5f}' for val, prob in posteriors.items()])
    print(f'{node} : {p}')

# print the marginal probabilities for the rhs junction tree
print('rhs probabilities')
for node, posteriors in rhs_jt.get_posteriors().items():
    p = ', '.join([f'{val}={prob:.5f}' for val, prob in posteriors.items()])
    print(f'{node} : {p}')

Note that we use InferenceController.reapply(...) to apply the new CPTs to a previous one and that we get a new junction tree as an output.