Mathematics Bayes' Theorem, Theorem of total probability and Bayes' theorem

### Topic Covered

color{red} ♦ Bayes' Theorem
color{red} ♦ Partition of a sample space
color{red} ♦ Theorem of total probability
color{red} ♦ Prove the Bayes' theorem.

### Bayes' Theorem

=> Consider that there are two bags I and II. Bag I contains 2 white and 3 red balls and Bag II contains 4 white and 5 red balls. One ball is drawn at random from one of the bags. We can find the probability of selecting any of the bags (i.e.1/2 ) or probability of drawing a ball of a particular colour (say white) from a particular bag (say Bag I).

### Partition of a sample space

=> A set of events E_1, E_2, ..., E_n is said to represent a partition of the sample space S if

(a) E_ i ∩ E_j = φ, i ≠ j, i, j = 1, 2, 3, ..., n

(b) E_1 ∪ Ε_2 ∪ ... ∪ E_n= S and

(c) P(E_i) > 0 for all i = 1, 2, ..., n.

=> In other words, the events E_1, E_2, ..., E_n represent a partition of the sample space S if they are pairwise disjoint, exhaustive and have nonzero probabilities.

=> As an example, we see that any nonempty event E and its complement E′ form a partition of the sample space S since they satisfy E ∩ E′ = φ and E ∪ E′ = S.

=> From the Venn diagram in Fig 13.3, one can easily observe that if E and F are any two events associated with a sample space S, then the set {E ∩ F′, E ∩ F, E′ ∩ F, E′ ∩ F′} is a partition of the sample space S. It may be mentioned that the partition of a sample space is not unique. There can be several partitions of the same sample space.

### Theorem of total probability

=> Let {E_1, E_2,...,E_n} be a partition of the sample space S, and suppose that each of the events E_1, E_2,..., E_n has nonzero probability of occurrence. Let A be any event associated with S, then

color{blue}{P(A) = P(E_1) P(A|E_1) + P(E_2) P(A|E_2) + ... + P(E_n) P(A|E_n)}

color{blue}{= sum_(j=1)^n P(E_j) P(A|E_j)}

color{red} "Proof" Given that E_1, E_2,..., E_n is a partition of the sample space S (Fig). Therefore,

color{orange} {S = E_1 ∪ E_2 ∪ ... ∪ E_n} ... (1)

and E_i ∩ E_j = φ, i ≠ j, i, j = 1, 2, ..., n

Now, we know that for any event A,

A = A ∩ S

= A ∩ (E_1 ∪ E_2 ∪ ... ∪ E_n)

= (A ∩ E_1) ∪ (A ∩ E_2) ∪ ...∪ (A ∩ E_n)

Also A ∩ E_i and A ∩ E_j are respectively the subsets of E_i and E_j . We know that
E_i and E_j are disjoint, for i ≠ j , therefore, A ∩ E_i and A ∩ E_j are also disjoint for all

i ≠ j, i, j = 1, 2, ..., n.

Thus, P(A) = P [(A ∩ E_1) ∪ (A ∩ E_2)∪ .....∪ (A ∩ E_n)]

= P (A ∩ E_1) + P (A ∩ E_2) + ... + P (A ∩ E_n)

Now, by multiplication rule of probability, we have

P(A ∩ E_i) = P(E_i) P(A|E_i) as P (E_i) ≠ 0 ∀ i = 1,2,..., n

Therefore, P (A) = P (E_1) P (A|E_1) + P (E_2) P (A|E_2) + ... + P (E_n)P(A|E_n)

or color{green}{P(A) = sum_(j=1)^n P(E_j) P(A |E_j)}

### Bayes' theorem.

=> Bayes’ Theorem If E_1, E_2 ,..., E_n are n non empty events which constitute a partitionof sample space S, i.e. E_1, E_2 ,..., E_n are pairwise disjoint and E_1∪ E_2∪ ... ∪ E_n = S and A is any event of nonzero probability, then

"Proof : " color{blue}{P(E_i | A) = (P(E_i) P(A|E_i))/(Sigma_(j =1)^n P(E_j ) P (A|E_j))} for any i = 1, 2, 3, ..., n

color {red} "Proof" By formula of conditional probability, we know that

P(E_r|A) = (P(AnnE_i))/(P(A))

= (P(E_i)P(A|E_i))/(P(A) ) (by multiplication rule of probability)

= (P(E_i ) (PA | E_i))/(Sigma_(j=1)^n P(E_j) P(A| E_j)) (by the result of theorem of total probability)

color{blue} ul"Remark" :

The following terminology is generally used when Bayes' theorem is applied.
The events E_1, E_2, ..., E_n are called hypotheses.

=> The probability P(E_i) is called the priori probability of the hypothesis E_i

=> The conditional probability P(E_i |A) is called a posteriori probability of the hypothesis E_i.

=> Bayes' theorem is also called the formula for the probability of "causes". Since the E_i's are a partition of the sample space S, one and only one of the events E_i occurs (i.e. one of the events E_i must occur and only one can occur). Hence, the above formula gives us the probability of a particular E_i (i.e. a "Cause"), given that the event A has occurred.
The Bayes' theorem has its applications in variety of situations, few of which are illustrated in following examples.
Q 3187591487

Bag I contains 3 red and 4 black balls while another Bag II contains 5 red
and 6 black balls. One ball is drawn at random from one of the bags and it is found to
be red. Find the probability that it was drawn from Bag II.
Class 12 Chapter 13 Example 16
Solution:

Let E_1 be the event of choosing the bag I, E_2 the event of choosing the bag II
and A be the event of drawing a red ball.

Then P(E_1) = P(E_2) = 1/2

Also P(A|E_1) = P(drawing a red ball from Bag I) =3/7

and P(A|E_2) = P(drawing a red ball from Bag II) = 5/11

Now, the probability of drawing a ball from Bag II, being given that it is red,
is P(E_2|A)
By using Bayes' theorem, we have

P(E_2/A) = ( P(E_2) P(A| E_2) )/(P(E_1) P( A | E_1) + P ( E_2) p (A| E_2)) = (1/2 xx 5/11)/( 1/2 xx 3/7 + 1/2 xx 5/11 ) = 35/68
Q 3117591489

Given three identical boxes I, II and III, each containing two coins. In
box I, both coins are gold coins, in box II, both are silver coins and in the box III, there
is one gold and one silver coin. A person chooses a box at random and takes out a coin.
If the coin is of gold, what is the probability that the other coin in the box is also of gold?
Class 12 Chapter 13 Example 17
Solution:

Let E_1, E_2 and E_3 be the events that boxes I, II and III are chosen, respectively .

Then P(E_1) = P(E_2) = P(E_3) = 1/3

Also, let A be the event that ‘the coin drawn is of gold’

Then P(A|E_1) = P(a gold coin from bag I) = 2/2 = 1

P(A|E_2) = P(a gold coin from bag II) = 0

P(A|E_3) = P(a gold coin from bag III) = 1/2

Now, the probability that the other coin in the box is of gold
= the probability that gold coin is drawn from the box I.
= P(E_1|A)
By Bayes' theorem, we know that

P(E_1|A) = (P(E_1) P( A| E_1))/( P(E_1 )P(A|E_1 )+P(E_2 )P(A|E_2 )+P(E_3 )P(A|E_3 ) )

= (1/3 xx 1)/( 1/3 xx 1 + 1/3 xx 0 + 1/3 xx 1/2 ) = 2/3
Q 3147691583

Suppose that the reliability of a HIV test is specified as follows:
Of people having HIV, 90% of the test detect the disease but 10% go undetected. Of
people free of HIV, 99% of the test are judged HIV–ive but 1% are diagnosed as
showing HIV+ive. From a large population of which only 0.1% have HIV, one person
is selected at random, given the HIV test, and the pathologist reports him/her as
HIV+ive. What is the probability that the person actually has HIV?
Class 12 Chapter 13 Example 18
Solution:

Let E denote the event that the person selected is actually having HIV and A
the event that the person's HIV test is diagnosed as +ive. We need to find P(E|A).
Also E′ denotes the event that the person selected is actually not having HIV.

Clearly, {E, E′} is a partition of the sample space of all people in the population.
We are given that

P(E) = 0.1 % = (0.1)/(100) = 0.001

P(E′) = 1 – P(E) = 0.999

P(A|E) = P(Person tested as HIV+ive given that he/she
is actually having HIV)

= 90% = 90/100 = 0.9

and P(A|E′) = P(Person tested as HIV +ive given that he/she
is actually not having HIV)

 = 1%= 1/100 = 0.01

Now, by Bayes' theorem

P(E|A) = (P(E) P(A| E ))/(P(E) P(A|E) + P (E') P(A|E') )

= (0.001 xx 0.9)/(0.001 xx 0.9 + 0.999 xx 0.01 ) = 90/1089

= 0.083 approx.

Thus, the probability that a person selected at random is actually having HIV
given that he/she is tested HIV+ive is 0.083.
Q 3157691584

In a factory which manufactures bolts, machines A, B and C manufacture
respectively 25%, 35% and 40% of the bolts. Of their outputs, 5, 4 and 2 percent are
respectively defective bolts. A bolt is drawn at random from the product and is found
to be defective. What is the probability that it is manufactured by the machine B?
Class 12 Chapter 13 Example 19
Solution:

Let events B_1, B_2, B_3 be the following :

B_1 : the bolt is manufactured by machine A

B_2 : the bolt is manufactured by machine B

B_3 : the bolt is manufactured by machine C

Clearly, B_1, B_2, B_3 are mutually exclusive and exhaustive events and hence, they
represent a partition of the sample space.

Let the event E be ‘the bolt is defective’.

The event E occurs with B_1 or with B_2 or with B_3. Given that,

P(B_1) = 25% = 0.25, P (B_2) = 0.35 and P(B_3) = 0.40

Again P(E|B_1) = Probability that the bolt drawn is defective given that it is manufactured
by machine A = 5% = 0.05
Similarly, P(E|B_2) = 0.04, P(E|B_3) = 0.02.

Hence, by Bayes' Theorem, we have

P(B_2 |E) = ( P(B_2 )P (E | B_2 ) )/( P(B_1 ) P ( E | B_1 )+ P (B_2 ) P (E |B_2 )+P(B_3 )P(E|B_3 ) )

 = (0.35 xx 0.04)/( 0.25 xx 0.05 +0.35 xx 0.40 xx 0.02)

= (0.0140)/(0.0345) = 28/69
Q 3167691585

A doctor is to visit a patient. From the past experience, it is known that
the probabilities that he will come by train, bus, scooter or by other means of transport
are respectively 3/10 , 1/5 ,1/10 and 2/5 . The probabilities that he will be late are 1/4 ,1/3 , and 1/12 ,

if he comes by train, bus and scooter respectively, but if he comes by other means of
transport, then he will not be late. When he arrives, he is late. What is the probability
that he comes by train?
Class 12 Chapter 13 Example 20
Solution:

Let E be the event that the doctor visits the patient late and let T_1, T_2, T_3, T_4 be the events that the doctor comes by train, bus, scooter, and other means of transport
respectively.

Then P(T_1) = 3/10 , P (T_2) =1/5, P(T_3) = 1/10 and P(T_4) = 2/5 (given)

P(E|T_1) = Probability that the doctor arriving late comes by train = 1/4

Similarly, P(E|T_2) = 1//3 , P(E|T_3) = 1/12 and  P(E|T_4) = 0, since he is not late if he

comes by other means of transport.
Therefore, by Bayes' Theorem, we have

P(T_1|E) = Probability that the doctor arriving late comes by train

= (P(T_1 )P(E|T_1 ))/( P(T_1 )P( E | T_1 )+P(T_2 )P( E | T_2 )+P(T_3 )P(E|T_3 )+P(T_4 )P(E|T_4 ) )

= (3/10 xx 1/4 )/( 3/10 xx 1/4 + 1/5 xx 1/3 + 1/10 xx 1/12 +2/5 xx 0) = 3/40 xx 120/18 = 1/2

Hence, the required probability is 1/2
Q 3177691586

A man is known to speak truth 3 out of 4 times. He throws a die and
reports that it is a six. Find the probability that it is actually a six.
Class 12 Chapter 13 Example 21
Solution:

Let E be the event that the man reports that six occurs in the throwing of the
die and let S_1 be the event that six occurs and S_2 be the event that six does not occur.

Then P(S_1) = Probability that six occurs =1/6

P(S_2) = Probability that six does not occur = 5/6

P( E |S_1) = Probability that the man reports that six occurs when six has
actually occurred on the die

= Probability that the man speaks the truth = 3/4

P(E|S_2) = Probability that the man reports that six occurs when six has
not actually occurred on the die
= Probability that the man does not speak the truth = 1-3/4 = 1/4

Thus, by Bayes' theorem, we get
P(S_1|E) = Probability that the report of the man that six has occurred is
actually a six

= ( P(S_1 )P(E |S_1 ) )/( P(S_1 )P(E|S_1 )+P(S_2 )P(E|S_2 ) )

= (1/6 xx 3/4 )/( 1/6 xx 3/4 + 5/6 xx 1/4 ) = 1/8 xx 24/8 = 3/8

Hence, the required probability is 3/8