Week 3 Homework -- Oct 25, 2011 -- with Answers


1. Naive Bayes -- Laplacian smoothing with -- K = 1
     note: NOT maximum likelihood

  training data:
   Movie
	A Perfect World
	My Perfect Woman
	Pretty Woman

   Song
    A Perfect Day
	Electric Storm
	Another Rainy Day

 my work --
  total messages:  6
  total words:    16
  					M	S
  message count:	3	3
  word histogram:
	   a			1	1
	   perfect		2	1
	   world		1	0
	   my			1	0
	   woman		2	0
	   pretty		1	0
	   day			0	2
	   electric		0	1
	   storm		0	1
	   another		0	1
	   rainy		0	1
	      totals:   8	8 =	16


  query:
    Perfect Storm
	  --is Perfect Storm more likely a movie or a song?

  A. P(Movie) = 0.5

  B. P(Song)  = 0.5

  C. P("Perfect"|Movie) = .1579

  D. P("Perfect"|Song)  = .1053

  E. P("Storm"|Movie)   = .0526

  F. P("Storm"|Song)    = .1053


2. Naive Bayes 2 -- Laplacian smoothing with -- K = 1
     for same example

  A. P(Movie | "Perfect Storm") = .4286
      -- probability of movie title for my query
  

3. Maximum Likelihood -- Maximum Likelihood
	for same example use Maximum Likelihood estimator
	 instead of Laplacian Smoothing

  A. P(Movie | "Perfect Storm") = 0.0
      -- probability of movie title for my query


4. Linear Regression
  data:
  	x   y
	0   3
	1   6
	2   7
	3   8
	4  11

  A. Can this data be fit EXACTLY using
      a linear function that maps from X to Y {y,n} --  No


5. Linear Regression 2
    for same data as #4

	Perform Linear Regression on these data points
	 and calculate W0 and W1 as defined in ths class
	 (see Linear Regression lectures for definitions).

  A. W0 = 3.8

  B. W1 = 1.4


6. K Nearest Neighbors
	see image:  03week_HW_KNN1.jpg
     plus indicates a positive training example
	 and negative a minus training example in this 2-dimensional space
	 in the following places (red boxes) 
	 check whether they will be PLUS for K=5
	 (only check those boxes for which the label will be positive)

  numbering from left
  		1 -- +
		2 -- +
		3 -- -
		4 -- -
		5 -- -

7. K Nearest Neighbors 2
	see image:  03week_HW_KNN2.jpg
	 given all the black data points
	 I want to know if the red ones are classified as indicated
	 and I'm free to chose a different value for K (1,3,5,7,9,none).
	 Check any or all of the K's for which you believe these data
	 points are classified correctly relative to the black training set

  A. K=1   --
  B. K=3   --
  C. K=5   -- X
  D. K=7   --
  E. K=9   --  
  F. none  --


8. Perceptron 
	see image:  03week_HW_Perceptron.jpg
	  with the following two-dimensional data set where + indicates a
	  positive class label and - a negative class label

  A. Are these data Linearly Separable?               {yes,no} -- Y

  B. Then given initial peceptron separating plane (in red),
      will it actually converge?                      {yes,no} -- Y