Week 2 Homework -- Oct 17, 2011


1. Bayes Rule

	Bayes Net (note: all directed downward):
			A
			|
			B
    Probabilities:
	  P(A)    = 0.5
	  P(B|A)  = 0.2
	  P(B|~A) = 0.8

  A. P(A|B) = ??? = .2
  	  (note: probability of A given B)

    bayes:  P(A|B) = P(B|A) * P(A) / P(B)
	   		P(~A) = 0.5

	   		P'(A|B)  = P(B|A) * P(A)   = .2 * .5 = .1
	   		P'(~A|B) = P(B|~A) * P(~A) = .8 * .5 = .4

								alpha  = .1 + .4 = .5

	  --->>	P(A|B)  = P'(A|B)/alpha    = .1 / .5 = .2
	   		P(~A|B) = P'(~A|B)/alpha   = .4 / .5 = .8


       and just for fun ----- P(A|~B) -----
	   		P(~B|A) = .8
	   		P(~B|~A) = .2

	   		P'(A|~B)  = P(~B|A) * P(A)   = .8 * .5 = .4
	   		P'(~A|~B) = P(~B|~A) * P(~A) = .2 * .5 = .1

								alpha  = .1 + .4 = .5

	       	P(A|~B)  = P'(A|~B)/alpha    = .4 / .5 = .8
	   		P(~A|~B) = P'(~A|~B)/alpha   = .1 / .5 = .2


2. Simple Bayes Net

		all binary variables
		Xi's conditionally independent given A

	Bayes Net (note: all directed downward):
			  A
		   /  |  \
		  X1 X2  X3

	P(A)     = 0.5
	 for all i:
	P(Xi|A)  = 0.2
	P(Xi|~A) = 0.6

  A. P(A| X1,X2,~X3 ) = ??? = 0.1818
  	  (note: probability of A given X1, X2, and NOT X3)

    bayes:  P(A|Xi) = P(Xi|A) * P(A) / P(Xi)
	   		P(~A)     = 0.5
			P(~Xi|A)  = 0.8
			P(~Xi|~A) = 0.4

	   		P'(A|X1,X2,~X3)  = .5 * .2 * .2 * .8   = .016
	   		P'(~A|X1,X2,~X3) = .5 * .6 * .6 * .4   = .072

								alpha  = .016 + .072 = .088 = P(X1,X2,~X3)

	  --->>	P(A|B)  = P'(A|B)/alpha    = .016 / .088 = .1818
	   		P(~A|B) = P'(~A|B)/alpha   = .072 / .088 = .8181


3. Bayes Net 2

	with same network as #2

  A. P(X3|X1) = ??? = 0.5
  	  (note: probability of X3 given X1)

			P(A)     = 0.5

			 for all i:
			P(Xi|A)  = 0.2
			P(Xi|~A) = 0.6

       derived by subtraction:
	 		P(~A)     = 0.5
			P(~Xi|A)  = 0.8
			P(~Xi|~A) = 0.4

	   derived by computation:
		  total probability of any Xi
	        P(Xi)    = 0.4
			       P(Xi|A) * P(A) + P(Xi|~A) * P(~A) =
			          .2   *  .5  +  .6      *  .5
	        P(~Xi)   = 0.6
		  bayes:
		    P(A|Xi)  = .25
					P(Xi|A) * P(A) / P(Xi)
					   .2   *  .5  /  .4
			P(~A|Xi) = .75

  see: Unit 3.9 Videos 3.22-3.23...

          P(X3|X1) =
			 ....some demons...
			(seems that we can ignore X2?)
			total probability of X3, given X1
			 P(X3 | X1,A) * P(A|X1) + P(X3 | X1,~A) * P(~A|X1)

			 ....then more demons...
			 (and we can deduct the X1, CI and we have A instead?)
                   P(X3 | X1,A)  to   P(X3 | A)
				 so:
				P(X3 | A) * P(A|X1) + P(X3 | ~A) * P(~A|X1)
				   .2     *   .25   +    .6      *   .75
				                 .5

4. Conditional Independence 1

	Bayes Net (note: all directed downward):
			A
		   / \
		  B   C
		   \ /
		    D

	(note: "ci" is "conditionally independent of", the upside down T)

  A. is B ci C [ B T C ]            		 {yes/no}: N -- A-unknown-out
  B. is B ci C given D [ B T C | D ]		 {yes/no}: N -- D-known-in
  C. is B ci C given A [ B T C | A ]		 {yes/no}: Y -- A-known-out
  D. is B ci C given A and D [ B T C | A,D ] {yes/no}: N -- D-known-in


5. Conditional Indepedence 2

	Bayes Net (note: all directed downward):
			 A    C
		   / | \ /
		  B  |  D
		   \ | /
		     E

  A. is C ci E given A [ C T E | A ]		  {yes,no}: N -- D-unknown-out
  B. is B ci D given C and E [ B T D | C,E ]  {yes,no}: N -- E-known-in
  C. is A ci C given E [ A T C | E ]		  {yes,no}: N -- ???
  D. is A ci C given B [ A T C | B ]		  {yes,no}: Y -- ???


6. Parameter Count 

	Same network as #5

  A. Minimum number of parameters needed to specify
		  the joint distribution of all 5 variables:     16

		  		A - (A{t,f}    			  				= 1
				C - (C{t,f}    			  				= 1
				B - (B|A{t,f}) 			  				= 2
				D - (D|A{t,f} + D|C{t,f}) 				= 4
				E - (E|B{t,f} + E|D{t,f}) + E|A{t,f})	= 8