Introduction III – Probability

Introduction Probability

Probability is one of the most powerful things mathematicians came up with. It allows us to have a mathematical view on our decisions and risks and can even help us to win some games. Probability theory is also what some of the machine learning algorithms are based on and what allows banks and other financial institutions to make huge profits or to regulate their risks. Although we won’t start with huge risk calculations or stock-price predictions here, we will see some theories that will help us to make better decisions in the future.

Like always, we have to start with some definitions:

• Sample Space: $\Omega$ (“Omega”) is the sample space. It’s the set of all possible outcomes. In case of a six sided die, $\Omega$ would be {1, 2, 3, 4, 5, 6}
• Event: An event is a subset of the sample space. If the event is that the outcome of rolling a six sided die once is less-equal to 3 [P(X $\le$3)] then the set of the event is {1, 2, 3} and therefore a subset of the sample space.
• Probability Function: A function giving the probability of an outcome
• Random Variable: A random numerical outcome

Discrete Sample Space

The discrete sample space is a special version of the sample space. The outcomes are listable but they can be finite or infinite. Let’s try to formulate this carefully in other words: A discrete sample space is carefully formulated a sample space where one knows the outcomes. But that doesn’t mean the outcomes have to be finite. If my outcomes can take the values of all natural numbers ( $\mathbb{ N }$), I still know all possible outcomes even though I can’t write them down.

Examples: {1, 2, 3} and {1, 2, 3, 4, 5, 6, 7, 8, 9} and {1, 2, 3, 4, 5, 6, 7, 8, 9, …} are all discrete sample spaces even though the latter one is infinite.

Probability Function

A probability function is a function that assigns an outcome a probability. Or in other words:

The probability function P( $\omega$) assigns every $\omega$ a number which is the probability that $\omega$ occurs. The probability function has to fulfil two rules:

1. 0 $\le$ P( $\omega$) $\le$ 1
2. $\sum _{ j=1 }^{ n }{ P(\omega_{ j }) }$=  1

Probability of Event E

The Probability of event E is the sum of all $\omega$ that are in E. or in other words: the probability of event E is the sum of the probabilities of all elements in the set of event E. $P(E) = \sum_{ \omega \in E } { P(\omega) }$

Rules of Probability

There are some rules for probability that have to be fulfilled:

1. P( ${ A }^{ c }$) = 1-P(A); Since all the probabilities in a sample space add up to 1 and A is a subset of the sample space, the probability of the complement of the subset A is just 1 minus the probability of the subset A.
2. If A and B are disjoint and have therefore no element in common: P(A $\cup$ B) = P(A) + P(B)
3. If A and B on the opposite site are NOT disjoint and have therefore an intersection that is not 0: P(A $\cup$) = P(A) + P(B) – P(A $\cap$ B); We have to subtract the intersection, so we don’t count the intersection twice.