P1

Statistics and Probabilities: the logic of quantifying uncertainty

Interesting extension reading about the history of Probabilities:

The most important root of probabilities:

Fermat and Pascal's correspondence in the 1650s

Some concepts:

A sample space: the set of all possible outcomes of an experiment.

An event: (?16:25?) a subset of the sample space

Naive definition of Prob.:

based on the assumptions that all outcomes have equally likely and finite sample space.

P(A) = #favorable outcomes/#possible outcomes

Then, how to compute the numerator and the denominator?

1 - Multiplication Rule:

P(Rth times experiment) = Nr

Each outcome of 1st experiments: P(1st) = n1, ...P(2st) = n2

Nr = n1*n2*n3*...*nr

2 - ?35:59? Binomial Coefficient

• 0 if K > n
• a subset of size k, order doesn't matter, of the group of n

Example: prob. of getting a full house (three cards of one rank and a pair of another, like three "K"s and two "A"s) in a standard deck(52 cards, 4 suits, 13 ranks).

for the denominator:

(52,5)

for the numerator:

• First time: choose 1 rank from 13 ranks - (13,1) = 13
• Second time: choose 3 suits from 4 cards that have the same rank - (4, 3)
• Third time: choose 1 rank from the rest 12 ranks - (12, 1) = 12
• Finally: choose 2 suits from 4 cards that have the same rank - (4, 2)

Sampling table:

Think and exercise by myself

1- Choosing 3 cards out of a standard deck, getting exactly 2 red cards

• If it's replaced: it's more like 3 independent events, then the pro. of = (pro.1st) times (prob. 2nd) times (pro. 3rd) = (26/52) times (26/52) times (26/52) times P(3,2) = 3 * (26/52)^3
• If it's not replaced: P(26,2)P(26,1)/P(52,3)

To prove that choose k out of n, replace and the order doesn't matter: P = (n+k-1,k)

• check the formula by an extreme and nontrivial example:
• n = 2 - ?17:08? P = (k+1, k) = (k+1, 1) = k+1

Gorgeous proof, brilliant:

• ?24:03? Convert "choose k out of n" to "put k dots in n box" and then to "combinations of k dots and n-1 lines", which means "choose k from n+k-1 objects" or " choose n-1 from n+k-1 objects"

P2

Tips:

(1) Do check answers by doing simple and extreme cases

(2) Label people or objects could be helpful to distinguish the question/sample

example:

• 10 people, split into a team of 6, a team of 4 - (10, 4) = (10, 6)
• 10 people, split into two teams of 5 - (10, 5)/ 2

Story proof

example:

(1) (n, k) = (n, n-k)

(2) n(n-1, k-1) = k(n, k)

Pick k people out of n, and pick one of k as the header

(3)

?37:03? Vandermonde's identity

Pick r people from a group of size m, pick k-r people from a group of size n

Non-naive definition of Prob.:

A probabilities sample consists of S and P, where S is a sample space, and P is a function that takes an event A is a subset of S (A ? S) as an input, returns P(A) ∈ [0, 1] as outputs.

Axiom

(1) P(?) = 0, P(S) = 1

(2)

?44:50? if A1, A2, ... are disjoint(non-overlap)

also called sum rules:

P(A+B) = P(A) + P(B), if A∩B = 0

P3 - Inclusion-exclusion

From axiom to properties

(1) P(Ac) = 1 - P(A)

(2) if A ? B, then P(A) <= P(B) ?27:10?

(3) P(AUB) = P(A) + P(B), if events A and B are disjoint, otherwise:

P(AUB) = P(A) + P(B) - P(A∩B) ?32:55?

?37:24?

(4) P(A∩B) or P(A,B) = P(A)*P(B), if the events A, B are independent.

NOTE: distinguish "independent" and "disjoint", A and B are disjoint, which means if A occurs then B won't occur.

Birthday problem

k people, find P that 2 have the same birthday

Assumption: exclude Feb. 29th, other 365 days equally likely, independence of birthday(no one's birthday affects anyone else's birthday)

If k > 365, P = 1 (pigeonhole principle)

If k <= 365, P’(no match) = (365*364...*(365-k+1))/(365^k)

(if it's two people who have the same birthday or an off-by-one-day birthday?

DeMontmort's Problem/ Matching Problem = 1/e

?01:18?

P4/5 - conditional prob.

P(A|B) = P(A∩B)/P(B), if P(B) > 0.

1 - P(A1,...,An) = P(A1)P(A2|A1)P(A3|A1,A2) ... P(An|A1,A2,...,An-1)

2 - P(A∩B) = P(B)P(A|B) = P(A)P(B|A)

3 - Bayes' Rule: P(A|B) = P(B|A)*P(A)/P(B)

P8

Binominal distribution Bin(n, p)

x~Bin(n, p):

1) x is #successes in n independent bern(p)

2) sum of indicator r.v.s: x = x1+x2+x3+...+xn,

, x1,...,xn i.i.d. Bern(p) - independent, identically distributed

3) PMF:

R. V. S.

In a sample space, each pebble has a number(prob.) that is a random variable(r.v.), and X=7 is an event which means two pebbles show up.

CDF - generally for all r.v.s

X <= x is an event

F(x) = P(X <= x), then F is the CDF of X(cumulative distribution function)

PMF - only for discrete r.v.s

Discrete: possible values can be listed, like a1, a2, a3, etc.

PMF: P(x = aj) = Pj for all j, Pj >=0, sum(Pj) = 1

Independent of r.v.s

X, Y are indep r.v.s if P(X<=x, Y<=y) = P(X<=x)P(Y<=y) for all x,y

If it's a discrete case: P(X=x, Y=y) = P(X=x)P(Y=y)

P9 - Expectation

Average (Means, Expected Value)

a simple example can be helpful to understand "average" with weigh

1, 1, 1, 1, 1, 3, 3, 5

E = 1*(5/8) + 3*(2/8) +5*(1/8)

Average of discrete r.v. X

X~Bern(p) - The Fundamental bridge:

For X~Bern(p),

E(X) = 1P(X=1) + 0P(X=0) = p

if A occurs, X = 1; otherwise, X = 0

then E(X) = P(A)

For X~Bin(n,p):

because the left thing is equal to 1 based on the binomial theorem,

so E(X) = np

Linearity

E(X+Y) = E(X) + E(Y), even if X and Y are dependent

E(cX) = cE(X) if c is a constant

Based on Linearity, redo E(X), X~Bin(n,p)

think it as the sum of n i.i.d Bern(p),

each Bernoulli trail E(x) = p

n of them, so E(X) = np

For hypergeometric:

E(X) = n* w / (w+b)

w - #successful trails

w+b - all trails

Geom(p): independent Bern(p) trials, count #failures before 1st success.

PMF: P(X = k) = (1-p)^k * p = q^k * p

For X ~ Geom(p):

E(X) = (1-p)/p = q/p