Theory Of Probability

Theory of Probability Sample Spaces and Events A set S of outcomes (events) of an experiment is called a sample space for the experiment. Each element in S is called simple outcome, or, simple event. An event E is defined to be any subset of S (including the empty set and the sample space set S). Event E is a simple event if it contains only one element and a compound event if it contains more than one element. We say that an event E occurs if any of the simple events in E occurs. Probability of an Event Given a simple space S = {e1 , e2 , · · · , en } with n simple events, to each simple event ei we assign a real number, denoted by P (ei ), called the probability of the event ei . These numbers can be assigned in a arbitrary manner as long as the following two conditions are satisfied: 1. The probability of a simple event is a number between 0 and 1, inclusive. That is, 0 ≤ P (ei ) ≤ 1 2. The sum of the probabilities of all simple events in the sample space is 1. That is, P (e1 ) + P (e2 ) + · · · + P (en ) = 1 Any probability assignment that meets conditions 1 and 2 is said to be an acceptable probability assignment. Probability of an Event E Given an acceptable probability assignment for the simple events in a sample space, S, we define the probability of an arbitrary event E, denoted by P (E), as follows: (A) If E (B) If E (C) If E in E. (D) If E

is the empty set, then P (E) = 0. is a simple event, then P (E) is already been assigned, is a compound event, then P (E) is the sum of the probabilities of all the simple events is the sample space, then P (E) = P (S) = 1.

Empirical Probability Approximation Empirical probability is based on the concept of relative frequency. If an experiment is conducted n times and event E occurs with frequency f (E), then the ratio f (E)/n is called relative frequency of the occurrence of event E in n trials. The empirical probability of E, denoted by P (E), by the number (if it exists) by the relative frequency. f (E) Frequency of occurrence of n = P (E) = Total number of trials n Equally Likely Assumption By assigning the same probability to each of simple event in S, we are actually making the assumption that each simple event is likely to occur as any other. This is referred to as an equally likely assumption. Probability of a Simple Event under an Equally Likely Assumption If, in a sample space S = {e1 , e2 , · · · , en } with n elements, we assume each simple event ei is as likely to occur as any other, then we assign the probability 1/n to each. That is, 1 P (ei ) = n Probability of a Arbitrary Event under an Equally Likely Assumption If we assume each simple event in sample space S is as likely to occur as any other, then the probability of an event E in S is given by P (E) =

Number of elements in E n(E) = Number of elements in S n(S)

Union, Intersection, and Complement of Events The union of two events A and B is the event that occurs if either A or B or both occur on a single performance of the experiment. The union of events A and B is denoted by the symbol A ∪ B. A ∪ B consists of all the sample points that belong to A or B or both (Figs. 1a and 1d). A ∪ B = {e ∈ S | e ∈ A or e ∈ B}

The intersection of two events A and B is the event that occurs if both A and B occur on a single performance of the experiment. The intersection of events A and B is denoted by the symbol A ∩ B. A ∩ B consists of all the sample points that belong to A and B (Figs. 1b and 1c). A ∩ B = {e ∈ S | e ∈ A and e ∈ B} Example Application - Survey and Marketing The results (in percentage) from electronic survey for marketing new product on ebay are given in Table I. Find (i). P (A) (ii). P (B) (iii). P (A ∪ B) and (iv). P (A ∩ B) given that Table 1: Response to electronic survey Age (years) Income < $25, 000 $25, 000 − $50, 000 > $50, 000 < 30 5% 12% 10% 30 − 50 14% 22% 16% > 50 8% 10% 3% A : {A respondent’s income is more than $50,000} B : {A respondent’s age is 30 or more} Solution The data is characterized in terms of age and income of the respondents to the ebay survey. Thus the experiment consists of selecting a respondent from collection of all respondents and observing which income and age group he/she occupies. The sample events are the six different age - income classification: E1 E2 E3 E7 E9

: : : : :

{< 30yrs, < $25, 000}, {30 − 50yrs, < $25, 000}, {> 50yrs, < $25, 000}, {< 30yrs, > $50, 000}, {> 50yrs, > $50, 000},

E4 : {< 30yrs, $25, 000 − $50, 000} E5 : {30 − 50yrs, $25, 000 − $50, 000} E6 : {> 50yrs, $25, 000 − $50, 000} E8 : {30 − 50yrs, > $50, 000}

Assign the probabilities to the sample points. Randomly selecting one of the respondents, the probability that he/she will occupy a particular age-income classification is just the relative frequency (proportion) of the respondent in the classification. Therefore the corresponding probabilities are P (E1 ) = 0.05,

P (E2 ) = 0.14

P (E3 ) P (E5 ) P (E7 ) P (E9 )

= = = =

0.08 0.22 0.10, 0.03

P (E4 ) = 0.12 P (E6 ) = 0.10 P (E8 ) = 0.16

Since the event A consists of income classification of more than ($50, 000) in all age classifications, the probability of A is the sum of the probabilities of the sample points in A: P (A) = P (E7 ) + P (E8 ) + P (E9 ) = 0.10 + 0.16 + 0.03 = 0.29 Similarly B = (≥ 30)yrs consists P (B) = P (E2 ) + P (E3 ) + P (E5 ) + P (E6 ) + P (E8 ) + P (E9 ) = 0.14 + 0.08 + 0.22 + .10 + 0.16 + 0.03 = 0.73 The union of A and B consists of all respondents whose income exceeds $50, 000 or whose age is 30 or more. P (A ∪ B) = 0.10 + 0.14 + 0.22 + 0.16 + 0.08 + 0.10 + 0.03 = 0.83 The intersection of events A and B consists of all sample points in both A and B. P (A ∩ B) = 0.16 + 0.03 = 0.19 Complement of an Event Consider a finite sample space S = {e1 , e2 , · · · , en } 0

that is divided into two subsets E and E such that 0

E∩E =Ø 0

that is, E and E are mutually exclusive, and 0

E∪E =S 0

0

Then E is called the complement of E relative to S. Thus E contains all the elements of S that are not in E. Also, 0

P (S) = P (E ∪ E ) 0 = P (E) + P (E ) = 1 Thus 0

P (E) = 1 − P (E ), Example - Birthday Problem

0

P (E ) = 1 − P (E)

In a group on n people, what is the probability that at least 2 people have the same birthday (the same month and day, excluding February 29)? Solution We will assume that the simple events in S are equally likely (i.e., that for any person in the group, any birthday is as likely as any other). Since any person could have any one of 365 birthdays (excluding February 29), the multiplication principle implies that the number of simple events in S is n(S) =

365 |{z}

·

365 |{z}

···

Ist person 2nd person = 365n

365 |{z} 2nd person

···

365 |{z} nth person

0

Now, let E be the event that at least 2 people in the group have the same birthday. Then E is the event that no 2 people have the same birthday. The multiplication principle also can be used 0 to determine the number of simple events in E : Ist person 2nd person 2nd person 0

n(E ) =

z}|{

z}|{

z}|{

nth person z

}|

{

365 · 364 · 363 · · · (366 − n) [365 · 364 · 363 · · · (366 − n)](365 − n)! = (365 − n)! 365! = 365 − n

Since we have assumed that S is an equally likely sample space, 0

n(E ) P (E ) = n(S) 0

= =

365! (365−n)! 365n

365! 365n (365

− n)!

Thus 0

P (E) = 1 − P (E ) 365! = 1− 365n (365 − n)! This equation is valid for any n satisfying 1 ≤ n ≤ 365. For example, in a group of 6 people, P (E) = 1 −

365! 3656 (359)!

= 0.04 It is interesting to note that as the size of the group increases, P (E) increases more rapidly than one might expect.

Rules of Probability In the study of probability it is often necessary to combine the probabilities of events. This is accomplished through both rules of addition and rules of multiplication. There are two rules for addition, ⇒ the special rule of addition and ⇒ the general rule of addition.

Special Rule of Addition The special rule of addition states that the probability of the event A or the event B occurring is equal to the probability of event A plus the probability of event B. The rule is expressed by P (A or B) = P (A) + P (B) • To apply the special rule of addition the events must be mutually exclusive. This means that when one event occurs none of the other events can occur at the same time (Fig. 2b). General Rule of Addition What if the events are not mutually exclusive? In that case the general rule of addition is used. The probability is computed using the formula P (A or B) = P (A) + P (B) − P (A and B) P (A ∪ B) = P (A) + P (B) − P (A ∩ B) For example, a study showed 15 percent of the work force to be unemployed, 20 percent of the work force to be minorities, and 5 percent to be both unemployed and minorities. What percent of the work force are either minorities or unemployed? Note that if P (unemployed) and p (minority) are totaled, the 5 percent who are both minorities and unemployed are counted in both group C that is, they are double-counted. They must be subtracted to avoid this double counting. Hence, P(unemployed or minority) = P(unemployed) + P (minority) −P(unemployed and minority) = 0.15 + 0.20 − 0.05 = 0.30 These two events are not mutually exclusive (Fig. 2a). Corollarry If A1 , A2 , · · · An are mutually exclusive, then P (A1 ∪ A2 ∪ · · · ∪ An ) = P (A1) + P (A2) + · · · + P (An ). If A1 , A2 , · · · An is a partition of a sample space S, then P (A1 ∪ A2 ∪ · · · ∪ An ) = P (A1 ) + P (A2 ) + · · · + P (An ) = P (S) = 1

Theorem For three events A, B and C, P (A ∪ B ∪ C) = P (A) + P (B) + P (C) − P (A ∩ B) − P (A ∩ C) −P (B ∩ C) + P (A ∩ b ∩ C) • When we want to find the probability that two events will both happen, we use the concept known as joint probability. Joint probability: A probability that measures the likelihood that two or more events will happen concurrently (at the same time).