{"id":461,"date":"2013-07-11T14:02:02","date_gmt":"2013-07-11T18:02:02","guid":{"rendered":"http:\/\/www.lichun.cc\/blog\/?p=461"},"modified":"2013-07-14T14:47:31","modified_gmt":"2013-07-14T18:47:31","slug":"understand-bayes-theorem-prior-likelihood-posterior-evidence","status":"publish","type":"post","link":"https:\/\/www.lichun.cc\/blog\/2013\/07\/understand-bayes-theorem-prior-likelihood-posterior-evidence\/","title":{"rendered":"Understand Bayes Theorem (prior\/likelihood\/posterior\/evidence)"},"content":{"rendered":"

Bayes Theorem<\/a> is a very common and fundamental theorem used in Data mining and Machine learning. Its formula is pretty simple:<\/p>\n

\r\nP(X|Y) = ( P(Y|X) * P(X) ) \/ P(Y), which is Posterior = ( Likelihood * Prior ) \/  Evidence\r\n<\/pre>\n
So I was wondering why they are called correspondingly like that.<\/p>\n
Let’s use an example to find out their meanings.<\/p>\n
<\/p>\n
Example<\/h2>\n
Suppose we have 100 movies and 50 books.
\nThere are 3 different movie types: Action, Sci-fi, Romance,
\n2 different book types: Sci-fi, Romance <\/p>\n
\r\n20 of those 100 movies are Action.\r\n30 are Sci-fi\r\n50 are Romance.\r\n\r\n15 of those 50 books are Sci-fi\r\n35 are Romance\r\n<\/pre>\n
So given a unclassified object,<\/p>\n
\r\nThe probability that it's a movie is 100\/150, 50\/150 for book.\r\nThe probability that it's a Sci-fi type is 45\/150, 20\/150 for Action and 85\/150 for Romance.\r\n<\/pre>\n
\r\nIf we already know it's a movie, then the probability that it's an action movie is 20\/100, 30\/100 for Sci-fi and 50\/100 for Romance.\r\nIf we already know it's a book, then that probability that it's an Sci-fi book is 15\/50, 35\/50 for Romance.\r\n<\/pre>\n
Right now, we want to know that given an object which has type Sci-fi, what the probability is if it’s a movie?<\/p>\n
Using Bayes theorem, we know that the formula is:<\/p>\n
\r\nP(movie|Sci-fi) = P(Sci-fi| Movie) * P(Movie) \/ P(Sci-fi)\r\n<\/pre>\n
Here, P(movie|Sci-fi) is called Posterior<\/b>,
\nP(Sci-fi|Movie) is Likelihood<\/b>,
\nP(movie) is Prior<\/b>,
\nP(Sci-fi) is Evidence<\/b>.<\/p>\n
Now let’s see why they are called like that.<\/p>\n
Prior<\/b>: Before<\/b> we observe<\/b> it’s a Sci-fi type, the object is completely unknown to us. Our goal is to find out the possibility that it’s a movie, we actually have the data prior(or before)<\/b> our observation<\/b>, which is the possibility that it’s a movie if it’s a completely unknown object: P(movie)<\/b>.<\/p>\n
Posterior<\/b>: After<\/b> we observed<\/b> it’s a Sci-fi type, we know something about the object. Because it’s post(or after)<\/b> the observation<\/b>, we call it posterior<\/b>: P(movie|Sci-fi).<\/p>\n
Evidence<\/b>: Because we’ve already known it’s a Sci-fi type, what has happened is happened. We witness<\/b> it’s appearance, so to us, it’s an evidence<\/b>, and the chance we get this evidence is P(Sci-fi)<\/b>.<\/p>\n
Likelihood<\/b>: The dictionary meaning of this word is chance or probability that one thing will happen. Here it means when it’s a movie, what the chance will be if it is also a Sci-fi type. This term is very important in Machine Learning.<\/p>\n
So why those probabilities are named like that, the observation time<\/b> is a very important reason.<\/p>\n","protected":false},"excerpt":{"rendered":"
Bayes Theorem is a very common and fundamental theorem used in Data mining and Machine learning. Its formula is pretty simple: P(X|Y) = ( P(Y|X) * P(X) ) \/ P(Y), which is Posterior = ( Likelihood * Prior ) \/ Evidence So I was wondering why they are called correspondingly like that. Let’s use an […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true},"categories":[67],"tags":[69,70],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p2s9sh-7r","jetpack_sharing_enabled":true,"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.lichun.cc\/blog\/wp-json\/wp\/v2\/posts\/461"}],"collection":[{"href":"https:\/\/www.lichun.cc\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.lichun.cc\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.lichun.cc\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.lichun.cc\/blog\/wp-json\/wp\/v2\/comments?post=461"}],"version-history":[{"count":26,"href":"https:\/\/www.lichun.cc\/blog\/wp-json\/wp\/v2\/posts\/461\/revisions"}],"predecessor-version":[{"id":489,"href":"https:\/\/www.lichun.cc\/blog\/wp-json\/wp\/v2\/posts\/461\/revisions\/489"}],"wp:attachment":[{"href":"https:\/\/www.lichun.cc\/blog\/wp-json\/wp\/v2\/media?parent=461"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.lichun.cc\/blog\/wp-json\/wp\/v2\/categories?post=461"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.lichun.cc\/blog\/wp-json\/wp\/v2\/tags?post=461"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}